Android Question OkHttpUtils2 PutBytes with a large amount of data

[ThePlankton]

I have a server that requires me to use PUT when uploading a file. I'm using HttpJob PutBytes and that works fine for very small files. The issue is with larger files (16meg) it immediately fails. I'm able to use PUT for the same file using Postman. Its using Transfer-Encoding →chunked. I have not been able to find any examples of doing the same thing in code. Any help would be appreciated.

 

[Erel]

Please post the logs.

 

[ThePlankton]

Not much to post. Same code works fine as long as the payload is less than 120k I believe

*** Service (starter) Create ***
** Service (starter) Start **
** Activity (main) Create, isFirst = true **
** Activity (main) Resume **
*** Service (httputils2service) Create ***
** Service (httputils2service) Start **
ResponseError. Reason: java.net.SocketException: Broken pipe, Response:

Code:

    Dim j As HttpJob
    j.Initialize("", Me)

    j.Username = "admin"
    j.Password = "admin"
    
    Dim strFile As String = "3.4.0.33.bin"
    Dim b() As Byte = File.ReadBytes(File.DirAssets, strFile)
    
    j.PutBytes("http://192.168.100.1/ws/files/userfs/WEB/" & strFile, b)
    
    Wait For (j) JobDone(j As HttpJob)
    If j.Success Then
        
        LogColor("WORKED!!", Colors.Red)
        
    Else
        
        LogColor("Failed", Colors.Red)
                
    End If
    
    j.Release

 

[drgottjr]

the unfiltered log is usually what is meant. but i think things will boil down to a
limitation on the size of the payload.

as the server appears to be yours, presumably you could increase (or remove a
limitation to) the payload size.

putbytes() reveals the size of the byte array up front. this would trigger an immediate
reaction of some sort. broken pipe appears to have popped up first, but i'm pretty
sure you'll find that the file is too big to put as a block. (how the server chooses to
deal with 16mb of data would determine whether or not it sets a size limit.)

a way around this is chunked data. the protocol is very simple, but it's
not going to work with the current version of httputils. the principal difference
between chunked and not chunked is that chunked data does not indicate the size
of the payload prior to the transfer (it does have to indicate the size of each
chunk, however).

you will need to implement the http protocol yourself (simple) and figure out a
way either to allow network activity on the main thread or implement a thread to
perform the transfer. there is a handy inline snippet for that by erel somewhere
here.

even if you can enable network activity on the main thread, android might still pull
the plug on a 16mb file if it locks up the gui for too long. matters such as the http
protocol and background threads are hidden from us thanks to httputils.

i chose to address the issue with inline java (a network connection on a background
thread mimicking the http protocol for encoding chunked data). you might be able
to accomplish the same thing in b4a with sockets or async streams, but you'll have to
implement the protocol yourself. i'm just more comfortable inline (even though i'm
essentially re-inventing the wheel since many b4a libraries that do the same work
are already available to us if one takes the time to learn them).

what do you think you can handle?

 

[Erel]

i chose to address the issue with inline java (a network connection on a background
thread mimicking the http protocol for encoding chunked data).

Big mistake. AsyncStreams will do it for you.

ResponseError. Reason: java.net.SocketException: Broken pipe, Response:

It is your server that rejects the request.

 

[ThePlankton]

Thank you for the replies. I did start looking into using AsyncStreams and implementing the http protocol myself.

 

[Hanz]

I have the same issue with couchdb. I am able to upload jpg with small size. But for larger ones, only a portion of the file is upload. But the server works well with curl command when I tested. So, I don't know where is the problem. Please do share if you find the solution.

 

[ThePlankton]

I ended up using AsyncStreams and manually created the http headers but using HTTP/1.0 and set Content-Length to avoid chunking. Seems to be working fine, thank you for the replies.