Android Question JSON PostString - 500 internal server error (continued) [SOLVED]

[mfstuart]

Back in this post I asked about posting JSON data to my web service: HERE

It is better to put the Dim line inside the loop however it is not related to this issue.

Maybe you need to set the request content type to application/json.

Hi Erel,
I was hoping to get a response on how to use a PostString with JSON and set the content.type to application/json?

To date, I've not been able to get the post to work with the data length over 1024 in size, using B4A.
Smaller JSON data from B4A, has been accepted to my web service successfully.
The trouble is, I've been able to use PostMan to post to my web service with large JSON data (longer than 1024 and copied from the B4A Log) successfully.

Any continued help on this would be appreciated.

Thanx,
Mark Stuart

 

[OliverA]

PostMan

This (https://www.getpostman.com/)?

 

[KMatle]

B4x uses standard methods like "PostString". Nothing special here. I'm sending huge JSON data with images, etc. to my servers. No problem.

What we do not know so far is how your webservice works (is it php?). How is the server configured?

 

[mfstuart]

This (https://www.getpostman.com/)?

That's it. A great tool for testing JSON.
I loads/launches from Chrome browser as an App.

 

[OliverA]

That's it. A great tool for testing JSON.

It looks like PostMan can generate source code for requests (not from personal experience, just what I gleaned from the site). Could you generate such source code and post it here? I don't think the language matters much, I just would like to see how the data is posted/submitted.

 

[mfstuart]

The in-network http web service is developed in Alpha Anywhere, with a published name like:
http://servername/sub/GetPostedData.a5w

Since it is an in-network web service, there's no need for https type service. Someone cannot access the web service externally.
Using B4A's PostString function, it sends the data to the web service no problem. As I mention already, with small data, but as more data is sent from B4A it returns the 500 internal server error.

Alpha XBasic web service code:

<%a5
dim cn         as sql::Connection
dim txt as c
txt = request.Body

if cn.Open("::Name::InvJSON") then
  'executes the stored procedure with the request JSON data
  cn.Execute("exec Add_ScanDet @json='"+ txt +"'")
  'returns the response success result:
  responseJson =  "{ \"items\" : [{ \"RecordsInserted\": \"ok\"  }] }"
  ? json_reformat( responseJson )

else
  'returns the error text:
  dim resp.error as c = cn.CallResult.Text
  ? json_generate(resp)
end if

%>

Erel mentioned that I should apply the application/json header. That's something I haven't done as yet, as I don't know how.
How do I do that?
I do notice that PostMan applies 2 header lines:
Authorization: Basic bWFyay5zdHVhcnQ6UGFsYWNlMjAxNw==
ContentType: text/plain

PostMan Body and B4A JSON request data:

[
  {
    "hid": 506,
    "plu": "000630135615",
    "description": "TOO TARTS LIQUID CNDY",
    "qty": 1,
    "price": 3.99,
    "userid": "mark.stuart",
    "scandatetime": "2017-08-02 11:22:07.089"
  },
  {
    "hid": 506,
    "plu": "000630135615",
    "description": "TOO TARTS LIQUID CNDY",
    "qty": 1,
    "price": 3.99,
    "userid": "mark.stuart",
    "scandatetime": "2017-08-02 11:24:44.443"
  }
]

Alpha Anywhere to PostMan response data:
{
    "items": [
        {
            "RecordsInserted": "ok"
        }
    ]
}

And again, remember that the Alpha Anywhere web service works now (without the error 500 internal server error).
It's now the amount of data I'm submitting to the web service from B4A that is failing.
And again, PostMan has no problem delivering the request and getting a response, no matter the size of data sent.

Any help is surely appreciated

Thanx,
Mark Stuart

 

[mfstuart]

Forgot to mention the server config:
Web Server = Windows Server 2012
Web Service = Apache

 

[nwhitfield]

Are you sure it's the size of the data, or not just something (a special character, for example) that only occurs a certain way into your dataset that is causing the problem?

An error 500 might occurr, for example, if there's an unescaped quote mark in the data, which causes the parsing of the JSON to fail horribly. It's also possible that there may be a per-script maximum memory limit or maximum post size specificed (eg, Apache's LimitBodyRequest directive)

 

[OliverA]

Authorization: Basic bWFyay5zdHVhcnQ6UGFsYWNlMjAxNw==

Looking at the original post (link in your initial post here), you never used the Username and Password properties of HTTPJob. These two properties will set up the Authorization header for you. Maybe that has been your problem all along.

 

[mfstuart]

It looks like PostMan can generate source code for requests (not from personal experience, just what I gleaned from the site). Could you generate such source code and post it here? I don't think the language matters much, I just would like to see how the data is posted/submitted.

Hi Oliver,
If you're referring to B4A code, then I placed a link in post #1 above as HERE, with my B4A code.
As far as PostMan goes, I just use it to test my B4A generated JSON data with my web service.

 

[OliverA]

If you're referring to B4A code

No, I'm aware of your original post and the B4A code. Postman can generate code for the request it generates (at least that is what their site says). If possible, let it generate the code (pick any language) and post it here.

 

[mfstuart]

Are you sure it's the size of the data, or not just something (a special character, for example) that only occurs a certain way into your dataset that is causing the problem?

An error 500 might occurr, for example, if there's an unescaped quote mark in the data, which causes the parsing of the JSON to fail horribly. It's also possible that there may be a per-script maximum memory limit or maximum post size specificed (eg, Apache's LimitBodyRequest directive)

I will scan some more data, this time with the same record (PLU item) that has been accepted by the web service. Then all data will be the same and should rule out erroneous characters.

 

[OliverA]

From your original post:

        Dim HJ As HttpJob
        HJ.Initialize("PostScanDet",Me)
        'we use Alpha Anywhere application for GETS and POSTS as the web service host app,
        'hence the .a5w in the URL
        HJ.PostString("http://ServerNameHere:PortNumberHere/PostJSONData.a5w",jsn)
     
        'Resumable Sub code:
        Wait For (HJ) JobDone(HJ As HttpJob)

Try

        Dim HJ As HttpJob
        HJ.Initialize("PostScanDet",Me)
        'we use Alpha Anywhere application for GETS and POSTS as the web service host app,
        'hence the .a5w in the URL
        '
        ' Add the authentication here
        HJ.Username = "someusername"
        HJ.Password = "averysecurepassword"
        HJ.PostString("http://ServerNameHere:PortNumberHere/PostJSONData.a5w",jsn)
        '
        ' Add content type here (if necessary. It may already be plain/text)
        HJ.GetRequest.SetContentType("text/plain")

        'Resumable Sub code:
        Wait For (HJ) JobDone(HJ As HttpJob)

Still would like a code sample from PostMan though.

Edit: Fixed content type setting. Had it wrong.

 

[mfstuart]

No, I'm aware of your original post and the B4A code. Postman can generate code for the request it generates (at least that is what their site says). If possible, let it generate the code (pick any language) and post it here.

Hi Oliver,
Sorry, I didn't see that as a feature. Maybe it's in the Pro or Enterprise versions.

Anyway, PostMan has no errors working with my Alpha Anywhere web service - small or large data, where B4A does.
Using the Username and Password are not needed in our in-network web service. I do have some time up my sleeve now that version 1.00 of the app was sent out a few weeks ago and was a hit with our theme and water parks across the country - USA. It's for scanning their inventory.
2 prior in-house developed scanning apps failed with other developers, and I (being new to the company) suggested B4A as I had known about it for years, and the management said yes let's buy it and you build it. Version 2.00 is on the way.
I've thoroughly enjoyed developing in B4A.

 

[mfstuart]

From your original post:

        Dim HJ As HttpJob
        HJ.Initialize("PostScanDet",Me)
        'we use Alpha Anywhere application for GETS and POSTS as the web service host app,
        'hence the .a5w in the URL
        HJ.PostString("http://ServerNameHere:PortNumberHere/PostJSONData.a5w",jsn)
    
        'Resumable Sub code:
        Wait For (HJ) JobDone(HJ As HttpJob)

Try

        Dim HJ As HttpJob
        HJ.Initialize("PostScanDet",Me)
        'we use Alpha Anywhere application for GETS and POSTS as the web service host app,
        'hence the .a5w in the URL
        '
        ' Add the authentication here
        HJ.Username = "someusername"
        HJ.Password = "averysecurepassword"
        HJ.PostString("http://ServerNameHere:PortNumberHere/PostJSONData.a5w",jsn)
        '
        ' Add content type here (if necessary. It may already be plain/text)
        HJ.GetRequest.SetContentType("plain/text")

        'Resumable Sub code:
        Wait For (HJ) JobDone(HJ As HttpJob)

Still would like a code sample from PostMan though.

Hi Oliver,
I typed in: HJ.GetRequest.
but I don't see any other "items" after the .
I'm using B4A version 7.01 on Windows 10 Pro 64 bit.
JSON library version 1.10
Is there something I'm missing here?

Thanx,
Mark Stuart

 

[OliverA]

1) I updated my post (I had the content type wrong)
2) Add the okHttp library (if you are using okHttpUtils2).
3) OT: I'm not going to be able to respond for several hours

 

[mfstuart]

Are you sure it's the size of the data, or not just something (a special character, for example) that only occurs a certain way into your dataset that is causing the problem?

An error 500 might occurr, for example, if there's an unescaped quote mark in the data, which causes the parsing of the JSON to fail horribly. It's also possible that there may be a per-script maximum memory limit or maximum post size specificed (eg, Apache's LimitBodyRequest directive)

Hi nwhitfield,
In my B4A app, I scanned 1 item and posted to the web service without any error.
Then scanned 7 items (all the same PLU item as prior scan, the same data) and the post failed with 500 internal server error.
Deleted 1 record (now 6 records all the same data), posted without any error.

Here's the B4A log: (notice my comments starting with -- in the log text)
Notice the JSON character count in my comments.
I seriously think this has something to do with the failure.

--Scan 1 item.
[
  {
    "hid": 1783,
    "plu": "012000151163",
    "qty": 1,
    "price": "3.74",
    "userid": "mark",
    "scandatetime": "2017-10-20 12:12:07.012"
  }
]
*** Service (httputils2service) Create ***
** Service (httputils2service) Start **
--Successfully posted from B4A app to web service.
--Web service response:
{
    "items": [
        {
            "RecordsInserted": "ok"
        }
    ]
}
** Activity (uploadscandetail) Pause, UserClosed = true **
** Activity (setup) Resume **
** Activity (setup) Pause, UserClosed = true **
** Activity (main) Resume **
** Activity (main) Pause, UserClosed = false **
** Activity (scan) Create, isFirst = false **
** Activity (scan) Resume **
** Activity (scan) Pause, UserClosed = false **
** Activity (scan) Create, isFirst = false **
** Activity (scan) Resume **
** Activity (scan) Pause, UserClosed = false **
** Activity (scan) Create, isFirst = false **
** Activity (scan) Resume **
** Activity (scan) Pause, UserClosed = true **
** Activity (main) Resume **
** Activity (main) Pause, UserClosed = false **
** Activity (setup) Create, isFirst = false **
** Activity (setup) Resume **
** Activity (setup) Pause, UserClosed = false **
** Activity (uploadscandetail) Create, isFirst = false **
** Activity (uploadscandetail) Resume **
--Trying 7 records, which are all the same PLU item.
--It will fail with 500 internal server error.
--Following JSON character count: 1151
[
  {
    "hid": 1783,
    "plu": "012000151163",
    "qty": 1,
    "price": "3.74",
    "userid": "mark",
    "scandatetime": "2017-10-20 12:14:17.821"
  },
  {
    "hid": 1783,
    "plu": "012000151163",
    "qty": 1,
    "price": "3.74",
    "userid": "mark",
    "scandatetime": "2017-10-20 12:14:18.445"
  },
  {
    "hid": 1783,
    "plu": "012000151163",
    "qty": 1,
    "price": "3.74",
    "userid": "mark",
    "scandatetime": "2017-10-20 12:14:19.076"
  },
  {
    "hid": 1783,
    "plu": "012000151163",
    "qty": 1,
    "price": "3.74",
    "userid": "mark",
    "scandatetime": "2017-10-20 12:14:19.696"
  },
  {
    "hid": 1783,
    "plu": "012000151163",
    "qty": 1,
    "price": "3.74",
    "userid": "mark",
    "scandatetime": "2017-10-20 12:14:20.359"
  },
  {
    "hid": 1783,
    "plu": "012000151163",
    "qty": 1,
    "price": "3.74",
    "userid": "mark",
    "scandatetime": "2017-10-20 12:14:21.260"
  },
  {
    "hid": 1783,
    "plu": "012000151163",
    "qty": 1,
    "price": "3.74",
    "userid": "mark",
    "scandatetime": "2017-10-20 12:14:23.341"
  }
]
ResponseError. Reason: Internal Server Error, Response: <!DOCTYPE html>

<html><head>

<title>500 Internal Server Error</title>

</head><body>

<h1>500 Internal Server Error</h1>

The server encountered a fatal error processing this request and could not continue.

<p>

<hr>

<address>Alpha Anywhere Application Server Application Server/12.0 Build/4119-4825 at SAV-APP1 Port 80</address>

</body></html>
** Activity (uploadscandetail) Pause, UserClosed = true **
** Activity (setup) Resume **
** Activity (setup) Pause, UserClosed = false **
** Activity (scanneditemsview) Create, isFirst = false **
** Activity (scanneditemsview) Resume **
** Activity (scanneditemsview) Pause, UserClosed = true **
** Activity (setup) Resume **
** Activity (setup) Pause, UserClosed = false **
** Activity (uploadscandetail) Create, isFirst = false **
** Activity (uploadscandetail) Resume **
--I then deleted one record, making 6 records.
--All the same PLU items.
--Post works fine.
--Following JSON character count: 987
[
  {
    "hid": 1783,
    "plu": "012000151163",
    "qty": 1,
    "price": "3.74",
    "userid": "mark",
    "scandatetime": "2017-10-20 12:14:17.821"
  },
  {
    "hid": 1783,
    "plu": "012000151163",
    "qty": 1,
    "price": "3.74",
    "userid": "mark",
    "scandatetime": "2017-10-20 12:14:18.445"
  },
  {
    "hid": 1783,
    "plu": "012000151163",
    "qty": 1,
    "price": "3.74",
    "userid": "mark",
    "scandatetime": "2017-10-20 12:14:19.696"
  },
  {
    "hid": 1783,
    "plu": "012000151163",
    "qty": 1,
    "price": "3.74",
    "userid": "mark",
    "scandatetime": "2017-10-20 12:14:20.359"
  },
  {
    "hid": 1783,
    "plu": "012000151163",
    "qty": 1,
    "price": "3.74",
    "userid": "mark",
    "scandatetime": "2017-10-20 12:14:21.260"
  },
  {
    "hid": 1783,
    "plu": "012000151163",
    "qty": 1,
    "price": "3.74",
    "userid": "mark",
    "scandatetime": "2017-10-20 12:14:23.341"
  }
]
{
    "items": [
        {
            "RecordsInserted": "ok"
        }
    ]
}
** Activity (uploadscandetail) Pause, UserClosed = true **
** Activity (setup) Resume **
** Activity (setup) Pause, UserClosed = true **
** Activity (main) Resume **
** Activity (main) Pause, UserClosed = true **

 

[mfstuart]

UPDATE:
SUCCESSFULLY POSTED 7 ITEMS, ALL THE SAME PLU
Caps aside now. Gentleman you are brilliant and so are all the people on this forum, including you Erel.

PROBLEM solved for now:
Oliver solved the issue where adding the OkHttp library and placing the following code AFTER the Job.PostString function:

 HJ.Username = "someusername"
 HJ.Password = "averysecurepassword"
 HJ.PostString("http://ServerNameHere:PortNumberHere/PostJSONData.a5w",jsn)
 'Add content type here (if necessary. It may already be plain/text)
 'The following line requires the OkHttp library to be selected, else you will not see the .SetContentType option.
 HJ.GetRequest.SetContentType("plain/text")      'THIS LINE IS IMPORTANT

Thank you to all that helped me in this endeavour, else the boss was ready to use a .NET web service solution.
Mark Stuart

 

[mfstuart]

Just scanned 50 items and all data posted successfully to my Alpha Anywhere web service.
So the key was setting the .SetContentType("plain/text").

Thanx,
Mark Stuart