HttpUtils2 and XMLSax

pixelpop

Active Member
Licensed User
Longtime User
I am attempting to use HttpUtils to download an XML file and send the results to XMLSax, but I'm wound around an axle. I have read all the posts and it makes my head spin. I am a fairly proficient VB programmer, so I'm embarressed to ask--can someone post a complete code snippet that will downlaod an xml file and then send the contents to the parser?

On a side note, can XMLSax parse this into it's parts:

<photo type="jpg_large" width="512" height="341" filename="\photos\5ad800b7b8e26812140f6a7067004fcd.jpg"/>

Thanks and sorry for seeming so dim on this.
 

pixelpop

Active Member
Licensed User
Longtime User
I am so close on this. Here is my code:

B4X:
Sub Process_Globals
   Dim parser As SaxParser
End Sub

Sub Globals
   Dim myURL As String
   Dim ListView1 As ListView
   Dim pubDate, hl_short, hl_expanded, hl_body As String
   Dim indata As InputStream

End Sub

Sub Activity_Create (FirstTime As Boolean)
   
   ListView1.Initialize("Main")
   myURL = "http://xx.xxx.xx.xxx/test/topnews.xml"
   HttpUtils.CallbackActivity = "Main" 'Current activity name
   HttpUtils.CallbackJobDoneSub = "JobDone"
   HttpUtils.Download("Job1", myURL)
   
End Sub

Sub JobDone (Job As String)

   If HttpUtils.IsSuccess(myURL) Then
      indata = HttpUtils.GetInputStream(myURL) 
   End If
   parser.Parse(indata, "Parser")
   
End Sub
Here is my problem. If I use GetInputStream, I get a Null Pointer Exception error and if I use String, I get a this:

src\test\app\main.java:311: error: inconvertible types
_parser.Parse((java.io.InputStream)(mostCurrent._indata),"Parser");
^
required: InputStream
found: String

I'm stumped and yes, I'm not using HttpUtils2, just trying to get something that works.
 
Upvote 0

pixelpop

Active Member
Licensed User
Longtime User
Erel,

I edited the JobDone code by adding a second IsSuccess:

B4X:
Sub JobDone (Job As InputStream)

   If HttpUtils.IsSuccess(myURL) Then
      indata = HttpUtils.GetInputStream(myURL)
      If HttpUtils.IsSuccess(myURL) Then
         parser.Parse(indata, "Parser")
      End If
   End If
   
End Sub

It compiles withour errors, but I get this on the emulator:

(see attachment)

Any ideas? Thanks.

Also, I would be open to converting to HttpUtils2, but I would need some hand holding to get this to work. The code above is as close as I've gotten using any method.
 

Attachments

  • error.png
    error.png
    33.9 KB · Views: 337
Last edited:
Upvote 0

pixelpop

Active Member
Licensed User
Longtime User
OK, I changed the Job parameter to a string. But now I'm back to the Null Pointer Exception error on the emulator (see attached).

B4X:
Sub JobDone (Job As String)

   If HttpUtils.IsSuccess(myURL) Then
      indata = HttpUtils.GetInputStream(myURL)
      parser.Parse(indata, "Parser")
   End If
   
End Sub

Note that I tried both (Job as String) and (Job1 as String). Neither caused a compile error, but both generated the Null Pointer Exception error.
 

Attachments

  • error_npe.png
    error_npe.png
    24.2 KB · Views: 267
Upvote 0

pixelpop

Active Member
Licensed User
Longtime User
Since HttpUtils was making me nuts, I decided to try HttpUtils2 again. Here is the complete code:

B4X:
Sub Process_Globals
   'These global variables will be declared once when the application starts.
   'These variables can be accessed from all modules.
   Dim parser As SaxParser
End Sub

Sub Globals
   'These global variables will be redeclared each time the activity is created.
   'These variables can only be accessed from this module.
   Dim headline, hl_short, hl_expanded, pubDate As String
   Dim ListView1 As ListView
End Sub

Sub Activity_Create(FirstTime As Boolean)

   Dim job1 As HttpJob
   job1.Initialize("Job1", Me)
   
   'Send a GET request
   job1.Initialize("Job1", Me)
   job1.DownLoad("http://xxx.xxx.xxx.xxx/test/topnews.xml")
   
End Sub
Sub JobDone (Job As HttpJob)

   Dim inputstr As InputStream
   
   If Job.Success = True Then
      inputstr = Job.GetInputStream
      parser.Parse(inputstr, "Parser")
   End If
   Job.Release
   
End Sub
Sub Activity_Resume

End Sub

Sub Activity_Pause (UserClosed As Boolean)

End Sub
Sub Parser_EndElement (Uri As String, Name As String, Text As StringBuilder)
   If parser.Parents.IndexOf("headline") > -1 Then
      If Name = "pub_date" Then
         pubDate = Text.ToString
      Else If Name = "headline_short" Then
         hl_short = Text.ToString
      Else If Name = "headline_expanded" Then
         hl_expanded = Text.ToString
      End If
   End If
   If Name = ""headline_short" Then
      ListView1.AddSingleLine2(hl_short, pubDate) 'add the title as the text and the link as the value
   End If
End Sub

This should be pretty straightforward, but when I compile with Debug, I get this:

B4X:
Compiling code.                         0.02
Compiling layouts code.                 0.00
Generating R file.                      0.00
Compiling generated Java code.          Error
B4A line: 6
End Sub
javac 1.7.0_03
src\main\app\main.java:402: error: cannot find symbol
      return new Object[] {"Activity",_activity,"parser",_parser,"headline",_headline,"hl_short",_hl_short,"pubDate",_pubdate,"ListView1",_listview1,"inputstr",_inputstr,"HttpUtils2Service",Debug.moduleToString(main.app.httputils2service.class)};
                                                                                                                                                                                                                       ^
  symbol:   class app
  location: class main
1 error

Please point out the err of my ways. This function is critical to the app I am writing, so I'm at a standstill right now

Thanks so much for your time and patience.
 
Upvote 0

pixelpop

Active Member
Licensed User
Longtime User
OK, I have this running smoothly. I must have had a corrupted HttpUtils2 file because a new, blank project was generating the same compile error. So I redownloaded the HttpUtils2 zip file and started from scratch. It's all good now. Thanks!
 
Upvote 0

jeng

Member
Licensed User
Longtime User
I need to read this code from my webserver.

Mit dieser XML-Datei sind anscheinend keine Style-Informationen verknüpft. Nachfolgend wird die Baum-Ansicht des Dokuments angezeigt.
<oscam version="1.20-unstable_svn build r8599" revision="8599" starttime="2013-04-05T08:46:08+0200" uptime="247" readonly="0">
<status>
<client type="s" name="root" desc="" protocol="server" protocolext="" au="0" thid="0x960a3f0">
<request caid="0000" srvid="0000" ecmtime="" ecmhistory="" answered=""/>
<times login="2013-04-05T08:46:08+0200" online="247" idle="1"/>
<connection ip="127.0.0.1" port="0">OK</connection>
</client>
<client type="r" name="imagin" desc="" protocol="mouse" protocolext="" au="-1" thid="0x9612e40">
<request caid="0000" srvid="0000" ecmtime="" ecmhistory="" answered=""/>
<times login="2013-04-05T08:46:08+0200" online="247" idle="247"/>
<connection ip="127.0.0.1" port="0">CARDOK</connection>
</client>
<client type="r" name="sat-hd+" desc="" protocol="mouse" protocolext="" au="-1" thid="0x9614a80">
<request caid="0000" srvid="0000" ecmtime="" ecmhistory="" answered=""/>
<times login="2013-04-05T08:46:08+0200" online="247" idle="247"/>
<connection ip="127.0.0.1" port="0">CARDOK</connection>
</client>
</status>

I try to do it with the same code as pixelpop does but it does not work.
I get access denied, but defined job1.Username and job1.Password.
Here my code and log.
B4X:
Sub Process_Globals
    'These global variables will be declared once when the application starts.
    'These variables can be accessed from all modules.
    Dim parser As SaxParser
End Sub

Sub Globals
    'These global variables will be redeclared each time the activity is created.
    'These variables can only be accessed from this module.
    Dim headline, hl_short, hl_expanded, pubDate As String
    Dim ListView1 As ListView
End Sub

Sub Activity_Create(FirstTime As Boolean)

    Dim job1 As HttpJob
    job1.Initialize("Job1", Me)
    job1.Username = "test"
   job1.Password = "test"
    'Send a GET request
    job1.Initialize("Job1", Me)
    job1.DownLoad("http://192.168.178.220:8181/oscamapi.html?part=status")
    
End Sub
Sub JobDone (Job As HttpJob)

    Dim inputstr As InputStream
    
    If Job.Success = True Then
        inputstr = Job.GetInputStream
        parser.Parse(inputstr, "Parser")
    End If
    Job.Release
    
End Sub
Sub Activity_Resume

End Sub

Sub Activity_Pause (UserClosed As Boolean)

End Sub
Sub Parser_EndElement (Uri As String, Name As String, Text As StringBuilder)
    If parser.Parents.IndexOf("headline") > -1 Then
        If Name = "times" Then
            pubDate = Text.ToString
        Else If Name = "connection" Then
            hl_short = Text.ToString
        Else If Name = "request" Then
            hl_expanded = Text.ToString
        End If
    End If
    If Name = "connection" Then
        ListView1.AddSingleLine2(hl_short, pubDate) 'add the title as the text and the link as the value
    End If
End Sub
Log File

** Activity (main) Resume **
Installing file.
** Activity (main) Pause, UserClosed = false **
PackageAdded: package:b4a.example
** Activity (main) Create, isFirst = true **
** Activity (main) Resume **
startService: class b4a.example.httputils2service
** Service (httputils2service) Create **
** Service (httputils2service) Start **
Access denied.
** Service (httputils2service) Destroy **
** Activity (main) Resume **

Many Thanks for any Help.
 
Last edited:
Upvote 0

jeng

Member
Licensed User
Longtime User
This means that your server didn't accept the credentials (or is using an authentication mode other than Basic or Digest).

Hmm, strange,
I found this on developpers site:

B4X:
Q: Authentication does not work when trying to authenticate via my own program/library xy. 
A: Make sure that your program/library uses digest authentication and NOT basic authentication. Wget also seems to have it's bugs with digest authentication so use curl instead. An example usage of curl is

curl --verbose --digest -u <user>:<password> "http://user:[email protected]:12345/oscamapi.html?part=status"

And if i try to use it like the above example i got the same error!
B4X:
job1.DownLoad("http://test:test@localhost:8181/oscamapi.xml?part=status")

Please :sign0085:
 
Upvote 0

jeng

Member
Licensed User
Longtime User
access denied

You do not need to modify the URL. Are you sure that the username add password are correct?

Do you have access to the server logs?

Sorry, i was verry busy last week.

Ok
Username and Password are correct.

Log from app.
B4X:
** Activity (main) Create, isFirst = true **
** Activity (main) Resume **
** Activity (main) Pause, UserClosed = false **
** Activity (main) Resume **
----------------------------
Job1=test
Job1=test
Job1=Http://xxxxxxxxx.dyndns.tv:8181/oscamapi.xml?part=userstats

[errormessage=, httputils2service=null, jobname=Job1
, main=null, mlink=, password=test
, req=anywheresoftware.b4a.http.HttpClientWrapper$HttpUriRequestWrapper@41c5df30, username=test, target=class Oscam.Control.main
, taskid=, success=false]
----------------------------
startService: class Oscam.Control.httputils2service

** Service (httputils2service) Create **
** Service (httputils2service) Start **
Access denied.
JobName = Job1, Success = false

Here my code.

B4X:
        job1.Initialize("Job1", Me)
   job1.Username = EditText1.Text
   job1.Password = EditText2.Text
        job1.DownLoad("http://" & EditText3.Text & ":" & EditText4.Text & "/oscamapi.xml?part=userstats")

Here the log from server.

B4X:
2013/04/12 12:41:16 0 --- Skipped 1 duplicated log lines ---

2013/04/12 12:41:16 9CB28C0 h WebIf: Origin checked. Result: access from xxx.xxx.xxx.91 => allowed
2013/04/12 12:41:16 0 --- Skipped 1 duplicated log lines ---
2013/04/12 12:41:16 9CB28C0 h WebIf: Received no auth header from xxx.xxx.xxx.91.
2013/04/12 12:41:16 9CB28C0 h WebIf: Origin checked. Result: access from xxx.xxx.xxx.91 => allowed
2013/04/12 12:41:16 9CB28C0 h WebIf: Received no auth header from xxx.xxx.xxx.91.
2013/04/12 12:41:16 9CB28C0 h WebIf: Origin checked. Result: access from xxx.xxx.xxx.91 => allowed

Here code from server.

B4X:
   if (cfg.http_user && cfg.http_pwd){
if(!authok || strlen(opaque) != MD5_DIGEST_LENGTH*2) calculate_opaque(addr, opaque);
if(authok != 2){   
if(!authok){
if(authheader){
cs_debug_mask(D_CLIENT, "WebIf: Received wrong auth header from %s:", cs_inet_ntoa(addr));
cs_debug_mask(D_CLIENT, "%s", authheader);
} else
cs_debug_mask(D_CLIENT, "WebIf: Received no auth header from %s.", cs_inet_ntoa(addr));
}
calculate_nonce(NULL, expectednonce, opaque);
}
if(authok != 1){
snprintf(authheadertmp, sizeof(authheadertmp), "WWW-Authenticate: Digest algorithm=\"MD5\", realm=\"%s\", qop=\"auth\", opaque=\"%s\", nonce=\"%s\"", AUTHREALM, opaque, expectednonce);
if(authok == 2) strncat(authheadertmp, ", stale=true", sizeof(authheadertmp) - strlen(authheadertmp) - 1);   
} else
snprintf(authheadertmp, sizeof(authheadertmp), "Authentication-Info: nextnonce=\"%s\"", expectednonce);
extraheader = authheadertmp;
if(authok != 1){
char *msg = "Access denied.\n";
send_headers(f, 401, "Unauthorized", extraheader, "text/html", 0, strlen(msg), msg, 0);
webif_write(msg, f);
NULLFREE(authheader);
free(filebuf);
if(*keepalive) continue;
else return 0;
}
} else NULLFREE(authheader);
Here a screen shot of the login page.
 

Attachments

  • login.jpg
    login.jpg
    57.1 KB · Views: 343
Last edited:
Upvote 0

jeng

Member
Licensed User
Longtime User
You can try to manually add the header that it expects.

I'd love to, but I do not know how.
Maybe you have an idea for me?
B4X:
'setHeader("WWW-Authenticate", "Digest realm="OpenACS",qop=\"auth\",nonce=\"" + cvtHex(nonce) + "\"");
    
   job1.GetRequest.SetHeader("Authenticate:","Digest")

or maybe...

B4X:
job1.PostString("Authenticate:", "Digest")

:sign0013::BangHead:
 
Upvote 0

jeng

Member
Licensed User
Longtime User

Ok Thanks,
I had already read but how can I set the variables correctly?
B4X:
req.SetHeader("WWW-Authenticate", "Digest realm=\"[email protected]\",qop=\"auth,auth-int\",nonce=\"dcd98b7102dd2f0e8b11d0f600bfb0c093\",opaque=\"5ccc069c403ebaf9f0171e9517f40e41\"");

Ok got it this way:

B4X:
Dim vRealm, algo, qop, opaque As String
      vRealm = Chr(34) & "Forbidden" & Chr(34) & ", "
      algo = Chr(34) & "MD5" & Chr(34) &", "
      qop = Chr(34) & "auth" & Chr(34) &", "
      ...
job2.GetRequest.SetHeader("WWW-Authenticate: Digest", "algorithm=" & algo & "realm=" & vRealm & "qop=" & qop)

Can you just tell me how i do read the header from server into a string?
And thanx for your Help.
I'll pay you a beer:cool:
 
Last edited:
Upvote 0

jeng

Member
Licensed User
Longtime User
You will need to modify hc_ResponseSuccess. You can read the headers there.

:sign0161:
So I thought that the httputils2 lib can do digest auth.
Looks unfortunately not so, because I have to implement the whole structure for it.

See this,

Overview

Digest access authentication was originally specified by RFC 2069 (An Extension to HTTP: Digest Access Authentication). RFC 2069 specifies roughly a traditional digest authentication scheme with security maintained by a server-generated nonce value. The authentication response is formed as follows (where HA1, HA2, A1, A2 are names of string variables):

\mathrm{HA1} = \mathrm{MD5}\Big(\mathrm{A1}\Big) = \mathrm{MD5}\Big( \mathrm{username} : \mathrm{realm} : \mathrm{password} \Big)
\mathrm{HA2} = \mathrm{MD5}\Big(\mathrm{A2}\Big) = \mathrm{MD5}\Big( \mathrm{method} : \mathrm{digestURI} \Big)
\mathrm{response} = \mathrm{MD5}\Big( \mathrm{HA1} : \mathrm{nonce} : \mathrm{HA2} \Big)

RFC 2069 was later replaced by RFC 2617 (HTTP Authentication: Basic and Digest Access Authentication). RFC 2617 introduced a number of optional security enhancements to digest authentication; "quality of protection" (qop), nonce counter incremented by client, and a client-generated random nonce. These enhancements are designed to protect against, for example, chosen-plaintext attack cryptanalysis.

If the algorithm directive's value is "MD5" or unspecified, then HA1 is

\mathrm{HA1} = \mathrm{MD5}\Big(\mathrm{A1}\Big) = \mathrm{MD5}\Big( \mathrm{username} : \mathrm{realm} : \mathrm{password} \Big)

If the algorithm directive's value is "MD5-sess", then HA1 is

\mathrm{HA1} = \mathrm{MD5}\Big(\mathrm{A1}\Big) = \mathrm{MD5}\Big(\mathrm{MD5}\Big( \mathrm{username} : \mathrm{realm} : \mathrm{password} \Big) : \mathrm{nonce} : \mathrm{cnonce} \Big)

If the qop directive's value is "auth" or is unspecified, then HA2 is

\mathrm{HA2} = \mathrm{MD5}\Big(\mathrm{A2}\Big) = \mathrm{MD5}\Big( \mathrm{method} : \mathrm{digestURI} \Big)

If the qop directive's value is "auth-int", then HA2 is

\mathrm{HA2} = \mathrm{MD5}\Big(\mathrm{A2}\Big) = \mathrm{MD5}\Big( \mathrm{method} : \mathrm{digestURI} : \mathrm {MD5}(entityBody)\Big)

If the qop directive's value is "auth" or "auth-int", then compute the response as follows:

\mathrm{response} = \mathrm{MD5}\Big( \mathrm{HA1} : \mathrm{nonce} : \mathrm{nonceCount} : \mathrm{clientNonce} : \mathrm{qop} : \mathrm{HA2} \Big)

If the qop directive is unspecified, then compute the response as follows:

\mathrm{response} = \mathrm{MD5}\Big( \mathrm{HA1} : \mathrm{nonce} : \mathrm{HA2} \Big)

The above shows that when qop is not specified, the simpler RFC 2069 standard is followed.
Impact of MD5 security on digest authentication

The MD5 calculations used in HTTP digest authentication is intended to be "one way", meaning that it should be difficult to determine the original input when only the output is known. If the password itself is too simple, however, then it may be possible to test all possible inputs and find a matching output (a brute-force attack) – perhaps aided by a dictionary or suitable look-up list.

The HTTP scheme was designed by Phillip Hallam-Baker at CERN in 1993 and does not incorporate subsequent improvements in authentication systems, such as the development of keyed-hash message authentication code (HMAC). Although the cryptographic construction that is used is based on the MD5 hash function, collision attacks were in 2004 generally believed to not affect applications where the plaintext (i.e. password) is not known.[1][citation needed] However, claims in 2006[2] cause some doubt over other MD5 applications as well. So far, however, MD5 collision attacks have not been shown to pose a threat to digest authentication, and the RFC 2617 allows servers to implement mechanisms to detect some collision and replay attacks.
 
Upvote 0
Top