Android Question Difference Between httpjob request and Chrome site navigation

[Marcos Alves]

Hello,

I'm trying to get some json data from a website but it looks that when it's used chrome the returned content is different than when httpjob is used, changing some of json fields. Is there any diference between using job.download(link) and to browse the same link using chrome?
Does header of download job is different ? If yes, is there any way to send a chrome similar request to the remote site in order to get the same return using httpjob?

Thanks!

 

[JohnC]

One thing that I can think of is that an HTTPJob will not run any scripts in the page.

 

[Marcos Alves]

One thing that I can think of is that an HTTPJob will not run any scripts in the page.

Yes I know. I'm not sure but it looks that it's not effect of any script because I have the complete data (more than 300 json fields) and only one single field comes incorrect. This field contains the information if the store is open or not at the moment. It looks that something is passed (using header???) by chrome that gives to the remote server the timezone or something like this (and isn't being passed by httpjob.)
Then: what are the differences between httpjob request and chrome request, or, is there any way to the remote server to identify the client and send a different initial data?

 

[drgottjr]

headers can cause different reactions. a server may
serve a different version of a "page" depending on the
user-agent that requests it.

there can be a big difference between what you get with
httpjob and what you see in a browser. a browser is a
totally different animal. if you download a page with httpjob
and look at the text, you may see references to .css and
.js files. where are they? when a browser downloads the
same page, it automatically goes and downloads the
.css and .js files to complete the full page. httpjob does
not do this. you have to go back and download each of
those ancillary files by hand.

you stand a better chance of seeing what the browser
sees with webview and webviewextras. but even then,
there can be differences. you'd have to try. if you
study the page's source in chrome and see exactly how
it populates a given json object, you may well be able to
duplicate that behavior with webviewextras.

 

[Marcos Alves]

headers can cause different reactions. a server may
serve a different version of a "page" depending on the
user-agent that requests it.

there can be a big difference between what you get with
httpjob and what you see in a browser. a browser is a
totally different animal. if you download a page with httpjob
and look at the text, you may see references to .css and
.js files. where are they? when a browser downloads the
same page, it automatically goes and downloads the
.css and .js files to complete the full page. httpjob does
not do this. you have to go back and download each of
those ancillary files by hand.

you stand a better chance of seeing what the browser
sees with webview and webviewextras. but even then,
there can be differences. you'd have to try. if you
study the page's source in chrome and see exactly how
it populates a given json object, you may well be able to
duplicate that behavior with webviewextras.

thanks! I'll try

 

[JohnC]

You can also setup a local server and have chrome and your app access a page on it, then compare the headers that are sent from each.

 

[MarcoRome]

If Header... try so:
Open your browser, go this link : https://www.whatismybrowser.com/detect/what-is-my-user-agent
You have your user agent.

in B4X

Dim j As HttpJob
j.Initialize("", Me)
j.Download(<link>) 'it can also be PostString or any of the other methods
j.GetRequest.SetHeader("User-Agent", "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/84.0.4147.89 Safari/537.36")
Wait for ...

 

[Marcos Alves]

If Header... try so:
Open your browser, go this link : https://www.whatismybrowser.com/detect/what-is-my-user-agent
You have your user agent.

in B4X

Dim j As HttpJob
j.Initialize("", Me)
j.Download(<link>) 'it can also be PostString or any of the other methods
j.GetRequest.SetHeader("User-Agent", "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/84.0.4147.89 Safari/537.36")
Wait for ...

Thanks @MarcoRome ! One thing more. It looks that sometimes the call will require to be changed due Chorme version upgrade no? Due this situation, maybe instead of "hard coding" could be interesting to store the header data in a server and recover as an app configuration parameter correct?

 

[MarcoRome]

Thanks @MarcoRome ! One thing more. It looks that sometimes the call will require to be changed due Chorme version upgrade no? Due this situation, maybe instead of "hard coding" could be interesting to store the header data in a server and recover as an app configuration parameter correct?

I don't think this is important