Android Question How do I download a web page and get the HTML/text

wwregistry

Member
Licensed User
Longtime User
I used HttpUtil2 to get the pages HTML text but they are blocked by the site - Error: Forbidden? But when using the phones browser and typing in the address from the keyboard the page comes up fine.

So in basic4android how do i use the phone browser to get the HTML text.

This is the code that generates the Error
'=========================================================HttpUtils2
Sub Activity_Create(FirstTime As Boolean)

Page.Initialize("Page", Me)

Page.Download("http://www.411.com/search/FindNearby?utf8=✓&street=11406+boulder+ln&where=78726")

End Sub

Sub JobDone (Job As HttpJob)
Log("JobName = " & Job.JobName & ", Success = " & Job.Success)
If Job.Success = True Then
Select Job.JobName
Case "Page"
Dim w as String
w = Job.getString
Log(w)
End Select
Else
Log("Error: " & Job.ErrorMessage) '<<<< this is where the error message is generated
End If

Job.Release

End Sub
'=========================================================HttpUtils2

HOWEVER

Typing http://www.411.com/search/FindNearby?utf8=✓&street=11406+boulder+ln&where=78726 into the phone browser works fine.
 

wwregistry

Member
Licensed User
Longtime User
I am not sure about user agents?

But it seems to me that
If android will pull down the web page and interpret the HTML code to display it on the screen?
I should be able to access the same code <<-- that is what i want to do
 
Upvote 0

DonManfred

Expert
Licensed User
Longtime User
See this post. And use this to find out the user-agent your browser is using...
Example.
 
Upvote 0

wwregistry

Member
Licensed User
Longtime User
pardon my ignorance
but this code seems to be excessively complicated to acquire the HTML text residing in the phone by virtue of StartActivity(ph.OpenBrowser(.... above
 
Upvote 0

DonManfred

Expert
Licensed User
Longtime User
i just wanted to show an example which is using headers...

the maincode is
B4X:
Dim job_GetInfo As HttpJob

  job_GetInfo.Initialize("job_GetInfo", Me)
  job_GetInfo.PostString(Post_URL, Post_Data)
  job_GetInfo.GetRequest.SetHeader("Sign", API_Signed)
  job_GetInfo.GetRequest.SetHeader("Key", API_Key)

  'Stuff that appears unnecessary but since others use it.....
  job_GetInfo.GetRequest.SetHeader("User-Agent","Mozilla/4.0 (compatible; Cryptsy API B4A client)")
  job_GetInfo.GetRequest.SetContentType("application/x-www-form-urlencoded")
Please note that ALL GetRequest-calls are AFTER PostString (Download or whatever)
 
Upvote 0

wwregistry

Member
Licensed User
Longtime User
i ran this code and got the usual error

Sub Activity_Create(FirstTime As Boolean)
Dim Post_URL, Post_Data, API_Signed, API_Key As String
MakeURL: Post_URL = s
job_GetInfo.Initialize("job_GetInfo", Me)
job_GetInfo.PostString(Post_URL, Post_Data)
job_GetInfo.GetRequest.SetHeader("Sign", API_Signed)
job_GetInfo.GetRequest.SetHeader("Key", API_Key)

'Stuff that appears unnecessary but since others use it.....
job_GetInfo.GetRequest.SetHeader("User-Agent","Mozilla/4.0 (compatible; Cryptsy API B4A client)")
job_GetInfo.GetRequest.SetContentType("application/x-www-form-urlencoded")


job_GetInfo.Download(s)
 
Upvote 0

DonManfred

Expert
Licensed User
Longtime User
I did not say that you should use this code unchanged!

You was searching for an example of how to use a useragent
B4X:
job_GetInfo.GetRequest.SetHeader("User-Agent","Mozilla/4.0 (compatible; Cryptsy API B4A client)")

THATS the line to set one.

I´m out here now...
 
Upvote 0

wwregistry

Member
Licensed User
Longtime User
That is amazing. It works great! You must be a very smart person. I had tried that HTTP code too but did not (still do not LOL) understand the complexities of grabbing a web page. Thank you again.

P.S. I sent you a little something.
 
Upvote 0

NJDude

Expert
Licensed User
Longtime User
There's nothing really complex about that, when you use the HTTP lib you send a request and you get a reply, which you can capture in a file, like in this example.

Thanks for the little something, I appreciate it.
 
Upvote 0
Top