Android Question Obtaining web page data. WebView Vs OkHttpUtils2

Mikonios · Nov 12, 2018

After different checks I conclude that the same result is not obtained between WebView and OkHttpUtils2.
The objective is to obtain the data through Service Module, so I must use OkHttpUtils2, but I can not get them from Yahoo or AcuWeather because to the same url's ::

Code:

Dim UrlAemet As String = "http://www.aemet.es/es/eltiempo/prediccion/municipios/zaragoza-id50297"
Dim UrlYahoo As String = "https://es.noticias.yahoo.com/tiempo/españa/zaragoza/zaragoza-779063/"
Dim UrlAcuWh As String = "https://www.accuweather.com/es/es/zaragoza/306788/daily-weather-forecast/306788"
Dim UrlcDola As String = "http://cincodias.com/mercados/divisas/eurosxdolares_usa/41/"

Job.GetString generates a different response ::

Code:

Sub JobDone(Job As HttpJob)
    Dim xHtml As String
    If Job.Success = True Then
 
        xHtml = Job.GetString
        Select Job.JobName
            Case "JbAemet"
                File.WriteString(DirOut, FileHtml, xHtml)
            Case "JbYahoo"
                File.WriteString(DirOut, FileHtml, xHtml)
            Case "JbAcuWh"
                File.WriteString(DirOut, FileHtml, xHtml)
            Case "JbDolar"
                File.WriteString(DirOut, FileHtml, xHtml)
        End Select
        ProcessHTML(xHtml)
 
    Else
        xHtml = Job.ErrorMessage
        Log("Error: " & Job.ErrorMessage)
        ToastMessageShow("Error: " & Job.ErrorMessage, True)
        Job.Release
        ProcessHTML(xHtml)
    End If
    Job.Release
 
End Sub

from WebView::

Code:

Sub WebView1_PageFinished (Url As String)
    Dim Javascript As String = "B4A.CallSub('ProcessHTML', false, document.documentElement.outerHTML)"
    WebViewExtras1.ExecuteJavascript(WebView1, Javascript)    ' WebViewExtras 1.42
    'WebViewExtras1.ExecuteJavascript(Javascript)            ' WebViewExtras2 no fire ProcessHtml
End Sub

Code:

Sub ProcessHTML(Html As String)
    '    This is the Sub that we'll get the web page to send it's HTML content to
    '    Log may truncate a large page so you'll not see all of the HTML in the log but the 'html' String should still contain all of the web page HTML
    Html = Html.Replace(Chr(34), "'")
    Do While Html.IndexOf(CRLF & CRLF) >= 0
        Html = Html.Replace(CRLF & CRLF, CRLF)
    Loop
 
    If Html.IndexOf("<title>Google</title>") < 0 Then
        File.WriteString(DirOut, FileHtml, Html)
        GetDataAndPrintResult(Html)
    End If
 
End Sub

Someone can help me with obtaining this data.
Thank you very much in advance.
Attach source proyect!.
Need: Core V8.50, LabelExtras1.12, OkHttpUtils2.70, RuntimePervissions1.10, StringUtils1.12, WebViewExtras1.42

DonManfred · Nov 12, 2018

Mikonios said:
Someone can help me with obtaining this data.

Which data?
Which SDK you are using?
Where is the documentation of the SDK you are using?
How is a webview related to an HTTP-Call to a webservice?

Sorry, my glass ball is defect.

Mikonios · Nov 12, 2018

I forgot to attach the source proyect so that tests can be done.
Sorry for the lack but I did not admit .rar and I got involved.

Very easy DonManfred::
Which data?
The data that the WebView response returns with respect to the JobDone.

Which SDK you are using?
I guess the same SDK as you sdkmanager.bat (v 3.29)

Where is the documentation of the SDK you are using?
I understand that with the sources of the project you do not need the documentation of the sdkmanager.bat but if so I guess Erel can provide it.

How is a webview related to an HTTP-Call to a webservice?
Exactly what is it that you did not understand about the question because I did not understand you?

I'm afraid you need to drink 2, 4 or 8 beers, rest a bit and then if you feel like you still have this!

Erel · Nov 13, 2018

OkHttpUtils2 download the html page.

WebView downloads the html page and all referenced resources, runs the page scripts and possibly downloads more resources.

You need to start with understanding how the information is retrieved from the server. Use the PC browser developer tools and monitor the network requests.

Mikonios · Nov 13, 2018

Hello Erel,
Thank you very much for answering. And your answer is correct.
The only thing that strange me, that with the previous version of httputils if I could download by service module, Yahoo data.
Now for some reason that I do not know OkHttpUtils2 does not generate the same result, or I should make a "POST" request before trying to get the data.
On the other hand, if it is true that accuweather has modified its Web page, I do not know how to develop it with a service module.
Again, thanks for the response.

Erel · Nov 14, 2018

It doesn't matter whether you use HttpUtils2 or OkHttpUtils2.

Mikonios · Nov 14, 2018

Actually I have an android 5.0.1 that every hour launches the download process.
This process is developed with b4a 7.30, httputils and everything except AcuW works.
I do not want to go back since I upgraded to b4a 8.30 and I do not want to recompile this source, but the apk is still working. And this is the reason for the post.

I wanted to migrate the whole process but I think it's more complicated than I thought.
Thanks anyway for your time

Code:

20181114:090508 UpSndSrv.TimerAt.(20181114:100508.0H60M0S05I12Y)SM-P605-100%-S
20181114:090510 XxxxXxxUpSnd <------- SM-P605 Orange5G-0652|TRUE|TRUE
20181114:090510    PrepSnd00  111|C4:62:EA:86:56:4B|-51dB|90%|351Mb/s
20181114:090511    Main.Activity_Resume:      --------------------------
20181114:090512    X55_MainEntra IpExterna:   xx.xx.xx.xx(0)SINDATO(0)
20181114:090521    X55_MainEntra DwnWw:       DwnWwDolarBrentOk
20181114:090522    X55_MainEntra Cotiz:       CotizOk: 11293/6542(R0)
20181114:090526    X55_MainEntra Electricos:  FDH(20180124.10.20180203)
20181114:090526    X10_LoadRadaresNoUpdate:   19700101
20181114:090536    X55_MainEntra AcuAem:      Ok 52(0)
20181114:090601    X55_MainEntra AcuYah:      Ok 25/25(0)/DESC
20181114:090611    X55_MainEntra AcuW:        java.net.SocketTimeoutException: Read timed out
20181114:090611    X55_MainEntra AcuW:        Check:Router.Down/WiFiErr
20181114:090800    X55_MainEntra Trafic:      T/O/G/I = 529/367/528/17
20181114:090849    X01_BBDDCreaYCop :         4.73Mb / 6.61Mb
20181114:090849    ----------------------------------------------------------------
20181114:090849    10Cli   :0.015Mb:20181114090612 11Tra   :0.048Mb:20181114090800
20181114:090849    09Rdr   :0.041Mb:20171111165300 08Ele   :0.471Mb:20180208131759
20181114:090849    ----------------------------------------------------------------
20181114:090849    X61_MainFin.StActvty(Intent1) -> xxxxWebs.b4a Prb:0   [100%]
20181114:090849    X61_MainFin                EndOk.

Erel · Nov 14, 2018

B4A version is not relevant. You can still use HttpUtils2 with B4A v8.x. You will need to use an older version of Android SDK.

Mikonios · Nov 14, 2018

Then try to migrate the process with an earlier version of Android SDK (and B4a 8.30), leaving for later AccuWhea downloads.
Again, thank you very much for your time.
I understand perfectly, everyone applauds you.

DonManfred · Nov 14, 2018

I don´t understand where the problem with AccuWeather is. As far as i remember they do provide an Rest-Api which you should use.

https://developer.accuweather.com/apis

You can call the api with okhttputils2 and even with the older httputils.

Mikonios · Nov 14, 2018

It's no problem cuestion.

Base Monthly Fee CPM Rate Base Monthly Fee CPM
Are 50 calls/day. Limit: 1 key/developer $25.00 per month.
https://developer.accuweather.com/packages

And the process need 500 request/day.

Android Question Obtaining web page data. WebView Vs OkHttpUtils2

Mikonios

Active Member

Attachments

DonManfred

Expert

Mikonios

Active Member

Erel

B4X founder

Mikonios

Active Member

Erel

B4X founder

Mikonios

Active Member

Erel

B4X founder

Mikonios

Active Member

DonManfred

Expert

Mikonios

Active Member

Similar Threads