Android Tutorial httputils2: Send a large array as a JSON string via Job.Download2 to a php script

KMatle · Feb 19, 2015

EDIT: Use the PostString method as it can carry more data (5 MB is ok - it's a server parameter). Download2 uses the "get-method" which is limited. To retrieve the data in php use:

B4X:

$json = file_get_contents("php://input");

instead of

B4X:

$json = $_GET["MyJSON"];

Job.Download2 is used to call a php script to send data like this:

B4X:

Dim Job1 As HttpJob
Job1.Initialize("MyJob", Me)
Job1.Download2("http://MyDomain.com/MyFolder/MyPhpScript.php",  Array As String(P1, P2, P3, P4))

Imagine you have a List with colums and rows (=array) and want to send the content to a php script to insert the values in a MySql database.

You could send "one row" with one Job which is not very nice. One job of x could crash on it's way and it is complicated to handle.

So why not pack all the rows (array) in a JSON string and encode it later with php with ONE job?

In B4A:

Dim a List we add maps to (a map is one row)

B4X:

Dim JSONList As List
JSONList.Initialize

In a loop get all the values you need and put them into a new map (=row). Add that map to the list.

B4X:

Dim ZeileMap As Map
        ZeileMap.Initialize
        ZeileMap.put("Tisch", tisch.SelectedItem)
        ZeileMap.put("AID", ArtikelID)
        ZeileMap.put("Besteller",LoggedInUser)
        ZeileMap.put("PosNr", P)
        ZeileMap.put("Zeit", zeit)
        ZeileMap.put("Posten", postKlar)
        ZeileMap.put("Preis", Preis)
        ZeileMap.put("Mwst", mwst)
        ZeileMap.put("Kommentar",BArtikelKommentar(i))
     
        JSONList.add(ZeileMap)

Dim a JSONGenerator and Initialise it with the list above.

B4X:

Dim JSONGenerator As JSONGenerator
   JSONGenerator.Initialize2(JSONList)

Call the JsonGenerator to create a JSON string which contains our data

B4X:

Dim JSONstring As String
   JSONstring = JSONGenerator.ToString

The result looks like:

B4X:

[{"Kommentar":"","Mwst":"19.00","Zeit":"20150219221021","AID":"7","PosNr":1,"Besteller":"Gabi","Posten":"Bier","Preis":3,"Tisch":"1"},{"Kommentar":"","Mwst":"19.00","Zeit":"20150219221021","AID":"1","PosNr":2,"Besteller":"Gabi","Posten":"Cola (normal)","Preis":3,"Tisch":"1"},{"Kommentar":"","Mwst":"19.00","Zeit":"20150219221021","AID":"2","PosNr":3,"Besteller":"Gabi","Posten":"Kaffee","Preis":5,"Tisch":"1"},{"Kommentar":"","Mwst":"19.00","Zeit":"20150219221021","AID":"6","PosNr":4,"Besteller":"Gabi","Posten":"Wasser (ohne)","Preis":3,"Tisch":"1"}]

B4X:

   Dim J1 As HttpJob
       J1.Initialize("J1", Me)
       J1.Download2("http://192.168.178.21/MyFolder/MyPhp.php", _
          Array As String("MyJSON", JSONstring  ))

In the php script:

1. The JSONString is encoded into $jsall. It's an big array (=list) which contains arrays (maps) (like our list containing maps)
2. In a loop we get each "map" from $jsall and put it to another array (single map) called $jsone
3. From here we have access to the single values by the key (f.e. "AID")

B4X:

        $json = $_GET["MyJSON"];
        $jsall = array();
        $jsone = array();
        $jsall=json_decode($json, true);
        $x = 0;
        while($x < count($jsall)) {
            $jsone=$jsall[$x];
         
            $tisch = $jsone["Tisch"];
            $besteller=$jsone["Besteller"];
            $posnr=$jsone["PosNr"];
            $posten=$jsone["Posten"];
            $preis=$jsone["Preis"];
            $mwst=$jsone["Mwst"];
            $aid=$jsone["AID"];
            $zeit=$jsone["Zeit"];
            $kommentar=$jsone["Kommentar"];

            *handle the data here*       

                $x++;
        }

Dependencies: HTTPUTILS, JSON
Tags: Json, MySql, php, httputils, Job.Download2

Ed Brown · Dec 16, 2015

This is exactly what I've been looking for!
Thanks @KMatle for a great job in explaining how to create a JSON string.

aeric · Apr 26, 2016

I am not sure I should bump this old thread. I am running into an error "Request-URI Too Long" when sending a long json data to the web service.

From the link below, it seems I need to modify the LimitRequestLine in Apache conf:
http://stackoverflow.com/questions/2891574/how-do-i-resolve-a-http-414-request-uri-too-long-error

Here is my code

B4X:

' Used to Add New / Update Existing Notes to Cloud
Sub UploadNotes
    Dim cur As Cursor
    Dim qry As String
    Dim jsn As String
    Dim gen As JSONGenerator   
    Dim L1 As List
    Dim M1 As Map
    L1.Initialize   
    Try
    qry = "SELECT * FROM MyNotes ORDER BY id"
    cur = DB.ExecQuery(qry)
    For i = 0 To cur.RowCount - 1
        cur.Position = i
        M1.Initialize
        M1.Put("id", cur.GetInt("id"))
        M1.Put("aa", cur.GetString("aa"))
        M1.Put("bb", cur.GetString("bb"))
        M1.Put("cc", cur.GetString("dd"))
        M1.Put("ee", cur.GetInt("ee"))
        L1.Add(M1)
    Next   
    gen.Initialize2(L1)
    jsn = gen.ToString
    Dim HJ As HttpJob
    Dim strURL As String   
    HJ.Initialize("UploadNotes", Me)
    strURL = ServerURL & "note-upload.php"
    HJ.Download2(strURL, Array As String("ui", gUser, "pw", gPassword, "json", jsn))
    Catch
        Log("Error: " & LastException.Message)
        ToastMessageShow("Error: UploadNotes", False)
    End Try
End Sub

Please advice.

Erel · Apr 27, 2016

aeric said:
I am not sure I should bump this old thread

You shouldn't. Please start a new thread in the questions forum.

aeric · Apr 27, 2016

Erel said:
You shouldn't. Please start a new thread in the questions forum.

Thanks Erel.
I created a new thread here

Neojoy · Dec 1, 2016

Is there some limit to Map list or JSONList?

KMatle · Dec 1, 2016

Neojoy said:
Is there some limit to Map list or JSONList?

Just the memory on the app side. On the server side there's a parameter (mine has 5 MB). You can compress it, too.

I use it a lot to upload f.e. pictures (300 KB) and corresponding data with encryption. If you need huge data streams better use FTP.

Neojoy · Dec 2, 2016

KMatle said:
Just the memory on the app side. On the server side there's a parameter (mine has 5 MB). You can compress it, too.

I use it a lot to upload f.e. pictures (300 KB) and corresponding data with encryption. If you need huge data streams better use FTP.

Thanks for your explanation, it was that I needed to know

kostefar · Jan 4, 2017

I´m struggling a bit with this. I´ve followed the instructions given, I believe, but end up with a weird result. This is where it gets interesting:

B4X:

Dim dastring As String
Dim postmessagejob As HttpJob
dastring = value
postmessagejob.Initialize("postmessagejob", Me)
    Log (dastring)
    Log ("jsonarray",dastring)
    postmessagejob.download2(MainURL &  "post_message.php?",Array As String("jsonarray",dastring))

The log says:

B4X:

[{"timestamp":"2017-01-03%2019:41:52","sender":"cz","message":"kkjrjgoijwoijgoiwjgiowgjowrjgiwogjworgjio","phone":"111111","passwordtmp":"vx","msgtype":"0","subject":"","usernametmp":"vx","threaid":"92337321312222016034304"}]
[Ljava.lang.String;@532bcd34

The reason why I added the logging lines was because nothing happened on the server side, so I decided to see what is being sent. As far as I can see, the string to be sent, which has been json encoded as instructed, looks formatted correctly, but Log ("jsonarray",streng) shows nothing like it.
On the server side, indeed I have

B4X:

$jsonarray = $_GET["jsonarray"];

followed by matches for my custom parameters.

Any clues?

Thanks in advance!

KMatle · Jan 6, 2017

kostefar said:
Any clues?

Use

B4X:

$json = file_get_contents("php://input");

If "nothing happens" just use a print command in the php to see what the variables contain. In JobDone see what it responds.

kostefar · Jan 6, 2017

KMatle said:
Use

B4X:

$json = file_get_contents("php://input");

If "nothing happens" just use a print command in the php to see what the variables contain. In JobDone see what it responds.

Thanks, but I decided to just post it like (parametervalues,"") so that it´s all one big URL. That works fine, and now I´m just wondering if there are any issues with using this method, such as a limitation to the length of the URL?

KMatle · Jan 6, 2017

It's a server parameter. Mine has 5 MB which should be more than enough for just JSON text data. Usually you will not upload that many of data from a "small" device. If you need to upload big files switch to multipart upload (browse the forum for that).

kostefar · Jan 6, 2017

KMatle said:
It's a server parameter. Mine has 5 MB which should be more than enough for just JSON text data. Usually you will not upload that many of data from a "small" device. If you need to upload big files switch to multipart upload (browse the forum for that).

Thanks KMatle, I´ll check with my host what their limit is.
So is 5 mb limit regardless of whether if you use the method I´m now using, or is it more if you use

B4X:

Array As String("MyJSON", JSONstring  ))

?

KMatle · Jan 6, 2017

It's a server parameter: http://stackoverflow.com/questions/2364840/what-is-the-size-limit-of-a-post-request

The main thing here is to send just one big string (instead of x single parameters). So you're only limited by the length.

kostefar · Jan 6, 2017

KMatle said:
It's a server parameter: http://stackoverflow.com/questions/2364840/what-is-the-size-limit-of-a-post-request

The main thing here is to send just one big string (instead of x single parameters). So you're only limited by the length.

Cheers, so then I´ll just continue doing what I´m doing then and not worry about what´s going wrong when trying to do things the other way.

kostefar · Jan 16, 2017

KMatle said:
It's a server parameter: http://stackoverflow.com/questions/2364840/what-is-the-size-limit-of-a-post-request

The main thing here is to send just one big string (instead of x single parameters). So you're only limited by the length.

So I just tried sending a json array with a base64 encoded image of 103 kb in it. Looks good in b4a, but from the server I get "
Request-URI Too Large
The requested URL's length exceeds the capacity limit for this server."

The php limit is set to 20M by default, so it seems to be my provider, ipage, that´s got a limit in their server nginx.conf. I´ve already been in touch with them, and it took me 45 mins to make the guy understand that it´s not an email I´m trying to send, and that he´s wrong when he calls the base64 encoded stuff "junk characters" - it´s exactly the opposite. Anyway, he´d contact some tech specialists from which I haven´t heard yet, but in the meantime I´d be curious to know if anybody else here had such a problem, and if they found any workarounds for it. I tested to see when the error occurs, and it´s when bytes being sent is above 8200 - strange number indeed.

DonManfred · Jan 16, 2017

dont send the jsonstring as get parameter.... Send it as poststring (if the server accept it) or using post-variables (multipartpost without sending files)

Something like:

B4X:

    Dim m As Map
    m.Initialize
    m.Put("jsonarray",dastring)
  
    Job.PostMultipart(MainURL &  "post_message.php",m,Null)

in post_message.php you get the value with

PHP:

$json = $_POST["jsonarray"];

I suggest to create a new thread for this issue

kostefar · Jan 16, 2017

DonManfred said:
dont send the jsonstring as get parameter.... Send it as poststring (if the server accept it) or using post-variables (multipartpost without sending files)

Something like:

B4X:

Dim m As Map m.Initialize m.Put("jsonarray",dastring) Job.PostMultipart(MainURL & "post_message.php",m,Null)

in post_message.php you get the value with

PHP:

$json = $_POST["jsonarray"];

I suggest to create a new thread for this issue

Thanks Master Manfred, I´ll try with $_POST in php instead and otherwise try with multipartpost. If this does not work, indeed I will create a new thread.

Android Tutorial httputils2: Send a large array as a JSON string via Job.Download2 to a php script

Similar threads

KMatle

Expert

Ed Brown

Active Member

aeric

Expert

Erel

Administrator

aeric

Expert

Neojoy

Member

KMatle

Expert

Neojoy

Member

kostefar

Active Member

KMatle

Expert

kostefar

Active Member

KMatle

Expert

kostefar

Active Member

KMatle

Expert

kostefar

Active Member

kostefar

Active Member

DonManfred

Expert

kostefar

Active Member