B4J Question How to bypass "Invalid cookie" error in webviews pagefinished

[kostefar]

Dear All,

On rare occasions I get an error like this:

ResponseError. Reason: java.lang.IllegalArgumentException: Unexpected char 0x01 at 545 in Cookie value: visid_incap_1244263=r+imxV2ZR0Klizpbd+cSA0RcfVsAAAAAQUIPAAAAAAAY1c2fTSk22G3/GLMNWtPM; nlbi_1244263=/52ZfrxGmG6P79RYK3zvtwAAAADgTKKjncgXmP3qWzgQhVPb; incap_ses_769_1244263=pLrhEMc2/EvufisC+wmsCgf6g1sAAAAAC3bywtBOgN1uVng7TVJ9LA==; incap_ses_768_1244263=zABpO48HMAPqHAG/YnyoCocmhFsAAAAApfIshuivRDuMyJolJgDe8A==; .ASPXANONYMOUS=pG0h3cwIxS3iJ44tCyqOiVOQJGpE-lkGj0NcegV7yF5n0ZxUh_IlKx07lg1jCXfj45SiDyG39qEKFsGBaeas4ExG-FFlJf0My0RV0mcGiOu9FRfR_vIDq_9W9kGQGGFFuJxOXTLxyT2DUR6jI4xsjg2; CryptopiaLang=en; ___utmvmBzuYcMIB=xGgKqqbcKsW; ___utmvaBzuYcMIB=jqAysOg; ___utmvbBzuYcMIB=xZL, Response:
main$ResumableSub_wv1_pagefinished.resume (java line: 1270)
java.lang.NullPointerException
    at b4j.example.main$ResumableSub_wv1_pagefinished.resume(main.java:1270)
    at anywheresoftware.b4a.BA.checkAndRunWaitForEvent(BA.java:135)
    at anywheresoftware.b4a.BA.raiseEvent2(BA.java:84)
    at anywheresoftware.b4a.BA.raiseEvent(BA.java:77)
    at anywheresoftware.b4a.keywords.Common$3.run(Common.java:1086)
    at com.sun.javafx.application.PlatformImpl.lambda$null$172(PlatformImpl.java:295)
    at java.security.AccessController.doPrivileged(Native Method)
    at com.sun.javafx.application.PlatformImpl.lambda$runLater$173(PlatformImpl.java:294)
    at com.sun.glass.ui.InvokeLaterDispatcher$Future.run(InvokeLaterDispatcher.java:95)
    at com.sun.glass.ui.win.WinApplication._runLoop(Native Method)
    at com.sun.glass.ui.win.WinApplication.lambda$null$147(WinApplication.java:177)
    at java.lang.Thread.run(Thread.java:748)

which seems to be triggered in:

Sub wv1_pagefinished (uri As String)
    If uri = "https://www.cryptopia.co.nz/" Then
        wait for (Get_Tradepairs)  complete (rawlist As String)
        If rawlist.Contains($""Success":true"$) Then
            Dim cryptopairslist As List  = Build_TPList (rawlist)
            Init_WvWshttp(cryptopairslist)

        End If
    End If
End Sub

And when I say rare, we´re talking like every two weeks. I just had it happening today and did not see it till half an hour after that the app had crashed. Restarting it resolved the issue.

What I would like to know is whether if there´s a way to simply bypass it, have a timer wait for 20 secs and then try again.

Thanks

 

[Erel]

The error happens in your code.
I guess that rawlist is somehow Null. What happens in Get_Tradepairs.

 

[kostefar]

The error happens in your code.
I guess that rawlist is somehow Null. What happens in Get_Tradepairs.

It happened again today. I´m now also logging the content of rawlist to see if it´s null as you said.

Here´s Get_Tradepairs:

Sub Get_Tradepairs As ResumableSub
    Dim j As HttpJob

    j.Initialize("GetTradePairs",Me)
    j.Download("https://www.cryptopia.co.nz/api/GetTradePairs")
    
    j.GetRequest.Timeout = 30000
    j.GetRequest.SetHeader("User-Agent", "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/64.0.3282.140 Safari/537.36")
    Wait For (j) jobdone(j As HttpJob)
    If j.Success Then
    Dim getstring As String = j.Getstring
    j.Release
    Return getstring
    End If
End Sub

 

[Daestrum]

Pretty sure you cannot use .contains on a string that has a null value. Maybe in the code in post #3, on failure return a space, then the check for $""Success":true"$ will still fail, but will not generate the NullPointerException.

 

[kostefar]

It happened again today. I´m now also logging the content of rawlist to see if it´s null as you said.

Here´s Get_Tradepairs:

Sub Get_Tradepairs As ResumableSub
    Dim j As HttpJob

    j.Initialize("GetTradePairs",Me)
    j.Download("https://www.cryptopia.co.nz/api/GetTradePairs")
   
    j.GetRequest.Timeout = 30000
    j.GetRequest.SetHeader("User-Agent", "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/64.0.3282.140 Safari/537.36")
    Wait For (j) jobdone(j As HttpJob)
    If j.Success Then
    Dim getstring As String = j.Getstring
    j.Release
    Return getstring
    End If
End Sub

Ok, I get what you´re saying but if it´s "just" that being the reason for the crash, why is it then saying

Unexpected char 0x01 at 545 in Cookie value: visid_incap_1244263=r+imxV2ZR0Klizpbd+cSA0RcfVsAAAAAQUIPAAAAAAAY1c2fTSk22G3/GLMNWtPM; nlbi_1244263=/52ZfrxGmG6P79RYK3zvtwAAAADgTKKjncgXmP3qWzgQhVPb; incap_ses_769_1244263=pLrhEMc2/EvufisC+wmsCgf6g1sAAAAAC3bywtBOgN1uVng7TVJ9LA==; incap_ses_768_1244263=zABpO48HMAPqHAG/YnyoCocmhFsAAAAApfIshuivRDuMyJolJgDe8A==; .ASPXANONYMOUS=pG0h3cwIxS3iJ44tCyqOiVOQJGpE-lkGj0NcegV7yF5n0ZxUh_IlKx07lg1jCXfj45SiDyG39qEKFsGBaeas4ExG-FFlJf0My0RV0mcGiOu9FRfR_vIDq_9W9kGQGGFFuJxOXTLxyT2DUR6jI4xsjg2; CryptopiaLang=en; ___utmvmBzuYcMIB=xGgKqqbcKsW; ___utmvaBzuYcMIB=jqAysOg; ___utmvbBzuYcMIB=xZL, Response:

 

[Erel]

Your code is wrong. The compiler should show a warning that you shouldn't ignore. You must return a result even if the request was not successful.

It is a different question why the request fails from time to time. The first step is to fix your code to properly handle this.

 

[kostefar]

Your code is wrong. The compiler should show a warning that you shouldn't ignore. You must return a result even if the request was not successful.

It is a different question why the request fails from time to time. The first step is to fix your code to properly handle this.

Do you mean that it´s wrong for the same reason as daestrum i saying? I do not get any warnings from the compiler.

I did the following to test what happens if string is null, and it does not generate any errors, just gives me "not contained" (as instructed to in the code):

Dim streng As String = Null

If streng.Contains("blah") Then
    Log ("contained")
    Else
        Log ("not contained")
End If

 

[Daestrum]

I think to set a String to a null value you have to use

Dim aString As String = ""

I seem to recall that using = Null, will result in the string containing the word 'null'
(Try aString = null then check its length)

 

[OliverA]

You actually should get the following warning in the top window of the logs screen

Not all code paths return a value. (warning #2)

Modified code:

Sub Get_Tradepairs As ResumableSub
    Dim j As HttpJob
    Dim getstring As String = ""  '<---- Declare your return variable here and give it a default return value

    j.Initialize("GetTradePairs",Me)
    j.Download("https://www.cryptopia.co.nz/api/GetTradePairs")
    
    j.GetRequest.Timeout = 30000
    j.GetRequest.SetHeader("User-Agent", "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/64.0.3282.140 Safari/537.36")
    Wait For (j) jobdone(j As HttpJob)
    If j.Success Then
       getstring = j.Getstring '<---- Set your return value
    End If
    j.Release '<---- Job needs to be released independent of j.Success value
    Return getstring '<---- Return your result
End Sub

Note: j.Release was also in the wrong place. It needs to be called, no matter what j.Success ends up being.

 

[kostefar]

I think to set a String to a null value you have to use

Dim aString As String = ""

I seem to recall that using = Null, will result in the string containing the word 'null'
(Try aString = null then check its length)

Ok, this does the same as the other test.

 

[kostefar]

You actually should get the following warning in the top window of the logs screen

Modified code:

Sub Get_Tradepairs As ResumableSub
    Dim j As HttpJob
    Dim getstring As String = ""  '<---- Declare your return variable here and give it a default return value

    j.Initialize("GetTradePairs",Me)
    j.Download("https://www.cryptopia.co.nz/api/GetTradePairs")
   
    j.GetRequest.Timeout = 30000
    j.GetRequest.SetHeader("User-Agent", "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/64.0.3282.140 Safari/537.36")
    Wait For (j) jobdone(j As HttpJob)
    If j.Success Then
       getstring = j.Getstring '<---- Set your return value
    End If
    j.Release '<---- Job needs to be released independent of j.Success value
    Return getstring '<---- Return your result
End Sub

Note: j.Release was also in the wrong place. It needs to be called, no matter what j.Success ends up being.

Thanks, I changed it to the way you´re suggesting, and also ammended pagefinished a bit:

Sub wv1_pagefinished (uri As String)
    If uri = "https://www.cryptopia.co.nz/" Then
        wait for (Get_Tradepairs)  complete (rawlist As String)
If rawlist.Length > 0 Then
        If rawlist.Contains($""Success":true"$) Then
            Dim cryptopairslist As List  = Build_TPList (rawlist)
            Init_WvWshttp(cryptopairslist)
End if
        End If
    End If
End Sub

Better now?

Still I hope for a solution to the cookie value problem...

 

[OliverA]

You now need to run your program in Debug mode and get the actual application line that's causing the issue (to me it looks like you're running in Release mode, but I could be mistaken).

 

[kostefar]

You now need to run your program in Debug mode and get the actual application line that's causing the issue (to me it looks like you're running in Release mode, but I could be mistaken).

Thanks OliverA,

Yeah, that´s what I´m doing now (it was in release mode before indeed). Not the nicest thing to have to do as I need debug mode for other things, where this app is part of a logging setup that is running 24/7 - interruptions should be as few as possible. It may take weeks till I get the error again, and I´m not so sure about if b4j has a way to bypass a problem at that level but hopefully I´m wrong.

 

[OliverA]

Looking at the source of jHttpUtils2 (https://www.b4x.com/android/forum/threads/b4x-okhttputils2-ihttputils2-httputils2-source-code.82632/), I'm pretty sure you still will get the

ResponseError. Reason: java.lang.IllegalArgumentException: Unexpected char 0x01 at 545 in Cookie value: visid_incap_1244263=r+imxV2ZR0Klizpbd+cSA0RcfVsAAAAAQUIPAAAAAAAY1c2fTSk22G3/GLMNWtPM; nlbi_1244263=/52ZfrxGmG6P79RYK3zvtwAAAADgTKKjncgXmP3qWzgQhVPb; incap_ses_769_1244263=pLrhEMc2/EvufisC+wmsCgf6g1sAAAAAC3bywtBOgN1uVng7TVJ9LA==; incap_ses_768_1244263=zABpO48HMAPqHAG/YnyoCocmhFsAAAAApfIshuivRDuMyJolJgDe8A==; .ASPXANONYMOUS=pG0h3cwIxS3iJ44tCyqOiVOQJGpE-lkGj0NcegV7yF5n0ZxUh_IlKx07lg1jCXfj45SiDyG39qEKFsGBaeas4ExG-FFlJf0My0RV0mcGiOu9FRfR_vIDq_9W9kGQGGFFuJxOXTLxyT2DUR6jI4xsjg2; CryptopiaLang=en; ___utmvmBzuYcMIB=xGgKqqbcKsW; ___utmvaBzuYcMIB=jqAysOg; ___utmvbBzuYcMIB=xZL, Response:

message. I think what was solved here is the

java.lang.NullPointerException

issue which was caused by not having a return for your resumable sub when j.Success was False (as it would have been with the ResponseError log entry. When this happens, j.Success is set to False by the jHttpUtils2 source).

 

[OliverA]

As to the unexpected char issue, that could be an implementation issue with the underlying library that okHttpUtils is based upon/uses or it could be an issue with the site that generates the cookie. It does not help that cookie values seem to be messy, and therefore one may run into communication issues with different sites implementation of cookies. See https://stackoverflow.com/a/1969339

 

[kostefar]

As to the unexpected char issue, that could be an implementation issue with the underlying library that okHttpUtils is based upon/uses or it could be an issue with the site that generates the cookie. It does not help that cookie values seem to be messy, and therefore one may run into communication issues with different sites implementation of cookies. See https://stackoverflow.com/a/1969339

Thanks for all your time spent here OliverA, so are you saying that there´s probably not a way to bypass the cookie problem?

 

[OliverA]

There is a hint (https://github.com/square/okhttp/issues/2939) that OkHttp (http://square.github.io/okhttp/) may have fixed this in a newer version of their library. Please note that the fix is to just drop cookies with invalid characters. If you read the post, this is not an uncommon thing to do for a browser (either that, or truncate the cookie). If there is a newer library, then @Erel would have to bundle it with a future release of B4J. Even then, it would just get rid of the "annoying" error message (without really causing a program fault).

 

[kostefar]

There is a hint (https://github.com/square/okhttp/issues/2939) that OkHttp (http://square.github.io/okhttp/) may have fixed this in a newer version of their library. Please note that the fix is to just drop cookies with invalid characters. If you read the post, this is not an uncommon thing to do for a browser (either that, or truncate the cookie). If there is a newer library, then @Erel would have to bundle it with a future release of B4J. Even then, it would just get rid of the "annoying" error message (without really causing a program fault).

That´s what I thought. But is the conclusion that the actual crash is due to the null and not the cookie thingie? In that case it´s all good!

 

[OliverA]

Yes. The invalid cookie just exposed a flaw in the code. With the code fixed, an invalid cookie will just be ignored.

 

[kostefar]

Yes. The invalid cookie just exposed a flaw in the code. With the code fixed, an invalid cookie will just be ignored.

Cheers OliverA, let´s see how it goes!