iOS Question [solved] FTP download crashes web-server (B4i only - not B4a!)

[yonson]

I'm using the same FTP routine to download a number of images from my webserver. I've displayed the relevant code below.

The very strange thing is that the code is idential to the B4A implementation which I've been using for years. Yet on B4i, the download usually stop around 40-50 images in, and after that my website crashes. I have contacted the hosting company too to see if there is a reason for this, but I was wondering if anyone could explain what the differences are between android and iOS and a suggested workaround (e.g. I was thinking perhaps putting a delay in between downloads, but this is obviously creating extra inefficiency)

    totalDownloads=0
  
    Dim thisFTP As FTP
    thisFTP.Initialize("thisFTP",FTP_images_address, FTP_images_port, FTP_images_username, FTP_images_password)

        For i=0 To missingFileList.Size-1
            Dim thisFile As Map
            thisFile=missingFileList.Get(i)


            ' check we haven't already downloaded it (another point might be using it)
            If (MiscUtils.isFilePresent(thisFile.Get("directory")&"/"&thisFile.Get("filename")) = False) Then
                File.MakeDir(File.DirDocuments,thisFile.Get("directory"))
        
              
                ' create directory if it doesn't exist
                Dim directoryPath As String
                directoryPath = File.DirDocuments & "/"&thisFile.Get("directory")
                If File.Exists(directoryPath, "") = False Then File.MakeDir(File.DirDocuments,thisFile.Get("directory"))
                thisFTP.DownloadFile("./"&thisFile.Get("directory")&"/"&thisFile.Get("filename"),False,File.DirDocuments&"/"&thisFile.Get("directory"),thisFile.Get("filename"))  
              
            Else
                Log("File ALREADY DOWNLOADED ON THIS DOWNLOAD CYCLE")
            End If
          
        Next

Sub thisFTP_DownloadCompleted (ServerPath As String, Success As Boolean)
' great - move onto next one
End Sub

 

[Erel]

Add this code after you call thisFTP.Initialize:

Dim no As NativeObject = thisFTP
no.GetField("server").SetField("persistConnection", true)

Does it change the behavior?

 

[yonson]

Hi Erel

unfortunately that actually makes the problem worse - it causes the download time to freeze up almost completely (so to download a file of say 20Kbs takes around 60 seconds).

 

[Erel]

Seems like your only solution is to add a delay of a few seconds between every several downloads.

 

[yonson]

Hi Erel
thanks - tried that but it didn't work. One thought, is the FTP actually downloading files in sequence? It might be my lack of knowledge, but it appears that in B4i the ftp is attempting to download all the files at once, and this would explain why the server crashes.

Could you have a look at the code and see if there is anything that jumps out which I could change to force sequential downloading (i.e. don't start next one until last one has finished)
Thanks
John

 

[Erel]

The requests are sent sequentially (unless you use multiple instances of FTP). However you can easily change your code to add a short delay between the downloads.

Add all the file paths to a List. Download the first one.
In the completed event remove the first item from the list and start a timer. When the timer ticks disable the timer and download the next file from the list.

 

[yonson]

Hi Erel

many thanks for your help. I've tried a number of solutions to try and get to the root of this issue, but no matter what delay time I use it always falls over and causes the webserver to crash. It is very odd indeed!

This is a slimmed down version of the code. I can't see anything which would cause the problem, and thoughts? As I say we have absolutely no issues in the equivalent B4A app.
Thanks
John

Dim missingFileList As List
missingFileList = getListofMissingFiles
thisFTP.Initialize("thisFTP",FTP_images_address, FTP_images_port, FTP_images_username, FTP_images_password)
  
' tried using this - actually makes things worse
'Dim no As NativeObject = thisFTP
  'no.GetField("server").SetField("persistConnection", True)

downloadNextFile


Sub downloadNextFile
    ' only continue if we have files left to download
    if (missingFileList.size<1) then return
  
    Dim now As Long
Dim ms as Long = 5000
   now=DateTime.now
   Do Until (DateTime.now>now+ms)
     ' dont' do anything
        'Log("waiting")
   Loop
    Log ("sleep ended at "&DateTime.now)

    Dim thisFile As Map
    thisFile=missingFileList.Get(0)
    If File.Exists(directoryPath, "") = False Then File.MakeDir(File.DirDocuments,thisFile.Get("directory"))
    Log("STARTING DOWNLOAD OF FILE "&thisFile.Get("filename"))
    thisFTP.downloadFile("./"&thisFile.Get("directory")&"/"&thisFile.Get("filename"),False,File.DirDocuments&"/"&thisFile.Get("directory"),thisFile.Get("filename"))  
End Sub

Sub thisFTP_DownloadCompleted (ServerPath As String, Success As Boolean)
    ' remove the first element from the missingFileList, as this is the file we've removed
    missingFileList.RemoveAt(0)  
    downloadNextFile
End Sub

 

[Erel]

I don't see any delay here. Start a timer that will call downloadNextFile in 5000ms and then disable it.

 

[yonson]

Hi Erel

thanks - I actually removed the delay because it didn't change the behaviour, i.e. it still crashed the app even when waiting 60 seconds between file downloads. I've put the delay back in to show you the code. Any thoughts?
Thanks

 

[Erel]

Does it crash the app or the ftp server?

 

[yonson]

the webserver crashes i.e. the website goes down completely. The app itself just 'hangs' i.e. the next download is never completed.

If it would help I could prepare you a sample project with the relevant code and send it over, the web server is usually back online in about an hour.

 

[Erel]

You need to access the server logs and see the error. If you are only interested in downloading, then try to use HttpUtils2 to download the files from the http server.