Android Question How do I download a web page and get the HTML/text

wwregistry

Member
Licensed User
I used HttpUtil2 to get the pages HTML text but they are blocked by the site - Error: Forbidden? But when using the phones browser and typing in the address from the keyboard the page comes up fine.

So in basic4android how do i use the phone browser to get the HTML text.

This is the code that generates the Error
'=========================================================HttpUtils2
Sub Activity_Create(FirstTime As Boolean)

Page.Initialize("Page", Me)

Page.Download("http://www.411.com/search/FindNearby?utf8=✓&street=11406+boulder+ln&where=78726")

End Sub

Sub JobDone (Job As HttpJob)
Log("JobName = " & Job.JobName & ", Success = " & Job.Success)
If Job.Success = True Then
Select Job.JobName
Case "Page"
Dim w as String
w = Job.getString
Log(w)
End Select
Else
Log("Error: " & Job.ErrorMessage) '<<<< this is where the error message is generated
End If

Job.Release

End Sub
'=========================================================HttpUtils2

HOWEVER

Typing http://www.411.com/search/FindNearby?utf8=✓&street=11406+boulder+ln&where=78726 into the phone browser works fine.
 

NJDude

Expert
Licensed User
That site might be checking for headers (like what browser is being used for example), without the correct headers any request gets rejected.
 

wwregistry

Member
Licensed User
I am not sure about user agents?

But it seems to me that
If android will pull down the web page and interpret the HTML code to display it on the screen?
I should be able to access the same code <<-- that is what i want to do
 

wwregistry

Member
Licensed User
pardon my ignorance
but this code seems to be excessively complicated to acquire the HTML text residing in the phone by virtue of StartActivity(ph.OpenBrowser(.... above
 

DonManfred

Expert
Licensed User
i just wanted to show an example which is using headers...

the maincode is
B4X:
Dim job_GetInfo As HttpJob

  job_GetInfo.Initialize("job_GetInfo", Me)
  job_GetInfo.PostString(Post_URL, Post_Data)
  job_GetInfo.GetRequest.SetHeader("Sign", API_Signed)
  job_GetInfo.GetRequest.SetHeader("Key", API_Key)

  'Stuff that appears unnecessary but since others use it.....
  job_GetInfo.GetRequest.SetHeader("User-Agent","Mozilla/4.0 (compatible; Cryptsy API B4A client)")
  job_GetInfo.GetRequest.SetContentType("application/x-www-form-urlencoded")
Please note that ALL GetRequest-calls are AFTER PostString (Download or whatever)
 

wwregistry

Member
Licensed User
i ran this code and got the usual error

Sub Activity_Create(FirstTime As Boolean)
Dim Post_URL, Post_Data, API_Signed, API_Key As String
MakeURL: Post_URL = s
job_GetInfo.Initialize("job_GetInfo", Me)
job_GetInfo.PostString(Post_URL, Post_Data)
job_GetInfo.GetRequest.SetHeader("Sign", API_Signed)
job_GetInfo.GetRequest.SetHeader("Key", API_Key)

'Stuff that appears unnecessary but since others use it.....
job_GetInfo.GetRequest.SetHeader("User-Agent","Mozilla/4.0 (compatible; Cryptsy API B4A client)")
job_GetInfo.GetRequest.SetContentType("application/x-www-form-urlencoded")


job_GetInfo.Download(s)
 

wwregistry

Member
Licensed User
That is amazing. It works great! You must be a very smart person. I had tried that HTTP code too but did not (still do not LOL) understand the complexities of grabbing a web page. Thank you again.

P.S. I sent you a little something.
 

NJDude

Expert
Licensed User
There's nothing really complex about that, when you use the HTTP lib you send a request and you get a reply, which you can capture in a file, like in this example.

Thanks for the little something, I appreciate it.
 
Top