Android Question Http digest authorization fails in B4A

fs007 · Jan 24, 2024

I need to write an app which sends a HTTP GET to devices named "Shelly" (these are so called "SmartSwitches, shelly.com).
They need digest authorization as described here: https://shelly-api-docs.shelly.cloud/gen2/General/Authentication/
My code (which does NOT work) is:

B4X:

Dim hj As HttpJob
hj.Initialize("",Me)
hj.Username="admin"
hj.Password="password"
hj.Download("http://192.168.5.188/relay/0?turn=on")
Wait for (hj) JobDone(hj As HttpJob)
If hj.Success Then
    label1.Text = hj.GetString
Else
    label1.Text ="Error"
End If

I get an error "Unauthorized", but the credentials are correct. What's wrong here ???

OliverA · Feb 2, 2024

fs007 said:
what would be the proper format for sha-256 digest auth ? code snippet would be nice

Try the code below. I've tested it against https://httpbin.org/digest-auth/auth/test/test3/SHA-256 and it seems to work. The code is not all-encompassing regarding the RFC spec and only handles GET requests, but hopefully, it implements enough to work for your needs.

Usage example:

B4X:

    Wait For(AuthenticateUrl("https://httpbin.org/digest-auth/auth/test/test3/SHA-256", "test", "test3")) Complete (j As HttpJob)
    If j.Success = True
        'successfully authenticated
        Log(j.GetString)
    Else
        Log(j.ErrorMessage)
    End If

The implementation code:

B4X:

'See...

JohnC · Jan 24, 2024

Please post the exact text of the error because we don't know exactly who is reporting that error (httpjob, or the shelly device, etc)

There is a chance that it can be fixed by adding a line in your app's manifest to allow non-secure "http" connections:

CLEARTEXT communication to 192.168.0.101 not permitted by network security policy

Hi All, I tried to implement the JRDC2 implementation of Erel in one program. The implementation of Erel is in the below URL: https://www.b4x.com/android/forum/threads/b4x-jrdc2-b4j-implementation-of-rdc-remote-database-connector.61801/ I get the following error: ResponseError. Reason...

www.b4x.com

drgottjr · Jan 24, 2024

I get an error "Unauthorized", but the credentials are correct. What's wrong here ???

pretty much everything.

in addition to @JohnC's comment, where's the digest? how are you constructing it? where's the realm and the nonce returned by the server? there's a series of handshaking steps to be taken; it's not a 1-step GET. server and browser communicate with each other through http headers. then you have to use the appropriate digest builder. i don't see any of this in the code posted. the documentation you link to spells things out. did you mean to post other code?

aeric · Jan 25, 2024

It seems you need to construct the authentication by concatenating "username":"device-model":"password" and hash it with SHA256. Then pass the value in Get header.

drgottjr · Jan 25, 2024

aeric said:
It seems you need to construct the authentication by concatenating "username":"device-model":"password" and hash it with SHA256. Then pass the value in Get header.

more involved than that.

fs007 · Jan 25, 2024

JohnC said:
Please post the exact text of the error

It's an http 401 error

drgottjr said:
where's the digest? how are you constructing it? where's the realm and the nonce returned by the server? there's a series of handshaking steps to be taken;

i thought that Android or the okhttputils2 lib would handle all this stuff ?!?!
The authorization needed in this case is described in detail here: https://shelly-api-docs.shelly.cloud/gen2/General/Authentication/
Doesn't B4A handle this ?

aeric · Jan 25, 2024

fs007 said:
i thought that Android or the okhttputils2 lib would handle all this stuff ?!?!

There is no one standard in implementing security in API. Some API developers put the security in header, some put in body, some build a digest of both. The user name and password way you use is for Basic Authentication. There are also JWT, HMAC, Public-Private key, Digital Signature, API Keys, etc.
Note: Putting ?!?! to express your anger doesn't help.

fs007 said:
Doesn't B4A handle this ?

You mean can I use B4A to make such API calls?
Sure, but you need to study the API documetation.

Can you specify which device or model you are working on?

fs007 · Jan 25, 2024

aeric said:
Can you specify which device or model you are working on?

It's a Shelly plus 2 PM. A detailed docu about the authorization can be read here: https://shelly-api-docs.shelly.cloud/gen2/General/Authentication/

aeric said:
Note: Putting ?!?! to express your anger doesn't help.

I have no anger. just wanted to express that i was wondering why the okhttputils2 lib doesn't work with that kind of authorization.

aeric said:
The user name and password way you use is for Basic Authentication.

That's not correct. The docu to httpjob says that digest authorization would work that way too ! See this: https://www.b4x.com/android/forum/threads/set-http-request-header-with-httputils2.78408/
Erel writes in post #2: "If the server implements the standard basic authentication or digest authentication methods then you should set Job.Username and Password. No need to set any header."

aeric · Jan 25, 2024

fs007 said:
It's a Shelly plus 2 PM. A detailed docu about the authorization can be read here: https://shelly-api-docs.shelly.cloud/gen2/General/Authentication/

I have no anger. just wanted to express that i was wondering why the okhttputils2 lib doesn't work with that kind of authorization.

That's not correct. The docu to httpjob says that digest authorization would work that way too ! See this: https://www.b4x.com/android/forum/threads/set-http-request-header-with-httputils2.78408/
Erel writes in post #2: "If the server implements the standard basic authentication or digest authentication methods then you should set Job.Username and Password. No need to set any header."

Usually for Basic Authentication, we use Base64Encode username : password like example by @EnriqueGonzalez

But for this Shelly API, I think it is some kind of HMAC.

It is difficult for us to help because we don't have the device to test.

I attach a code. Not sure if it is working.

B4X:

Sub Process_Globals
    Private Const SHELLY As String = "192.168.5.188:80"
    Private username As String = "admin"
    Private password As String = "password"
    Private realm As String = "shellyplus2pm-f008d1d8b8b8"
End Sub

Sub AppStart (Args() As String)
    Dim data As Map = CreateMap("id": 1, "method": "Shelly.GetStatus")
    
    ShellyHttpCall("Post", $"http://${SHELLY}/rpc"$, data.As(JSON).ToCompactString)
    
    StartMessageLoop
End Sub

Sub ShellyHttpCall (HttpMethod As String, Link As String, JsonData As String)
    Private AuthParams As Map
    AuthParams.Initialize
    Dim job As HttpJob
    job.Initialize("", Me)
    If HttpMethod.ToUpperCase = "POST" Then
        job.PostString(Link, JsonData)
        job.GetRequest.SetContentType("application/json")
    Else If HttpMethod.ToUpperCase = "GET" Then
        job.Download(Link)
    Else
        Log($"[${HttpMethod.ToUpperCase}] Is this supported?"$)
        Return
    End If
    
    Wait For (job) JobDone(job As HttpJob)
    If job.Success Then
        Log(job.GetString)
    Else
        Log(job.ErrorMessage)
        If job.Response.StatusCode = 401 Then
            Dim headers As List
            headers = job.response.GetHeaders.Get("WWW-Authenticate")
            For Each header In headers
                Log(header)
                Dim keyvalue() As String
                keyvalue = Regex.Split("=", header)
                If keyvalue.Length = 2 Then
                    AuthParams.Put(keyvalue(0), keyvalue(1))
                End If
            Next
            ShellyHttpCall(HttpMethod, Link, GetAuthData(AuthParams, JsonData))
        End If
    End If
    job.Release
End Sub

Sub GetAuthData (AuthParams As Map, JsonString As String) As String
    Dim algorithm As String = "SHA-256"
    Dim ha1 As String = SHA256($"${username}:${realm}:${password}"$)
    Dim ha2 As String = SHA256("dummy_method:dummy_uri")
    Dim nonce As Int = AuthParams.Get("nonce")
    Dim cnonce As Int = Rnd(100000000, 999999999)
    Dim nc As Int = 1
    Dim response As String = SHA256($"${ha1}:${nonce}:${nc}:${cnonce}:auth:${ha2}"$)

    Dim authMap As Map
    authMap.Initialize
    authMap.Put("realm", realm)
    authMap.Put("username", username)
    authMap.Put("nonce", nonce)
    authMap.Put("cnonce", cnonce)
    authMap.Put("response", response)
    authMap.Put("algorithm", algorithm)

    Dim JsonMap As Map = JsonString.As(JSON).ToMap
    JsonMap.Put("auth", authMap.As(JSON).ToCompactString)
    Return JsonMap.As(JSON).ToCompactString
End Sub

' Return SHA-256 in hexadecimal
Public Sub SHA256 (str As String) As String
    Dim data() As Byte
    Dim MD As MessageDigest
    Dim BC As ByteConverter
    data = BC.StringToBytes(str, "UTF8")
    data = MD.GetMessageDigest(data, "SHA-256")
    Return BC.HexFromBytes(data).ToLowerCase
End Sub

fs007 · Jan 25, 2024

aeric said:
I attach a code ...

Thank you very much for this code, i'll give it a try. A few things will have to be added, i.e. realm is not known prior first http request ...

Still i don't understand one thing:
The authentification used by these Shelly devices is nothing special: Every Browser i tried, CAN login, so i have to assume, that this kind of authentificaton is kind of a standard.

Is there really no "ready to use" lib in B4A that handles this authentification ?

aeric · Jan 25, 2024

The project is B4J. It is easier to test on desktop first. If it works then use the code on B4A.
Take note on the returned response headers. Maybe the nonce is a string value instead of a number. Put a Log to see the item in the list.
Port 80 in the link may not needed.
Change the id in realm variable.
It may work for POST request. GET or others may not be supported.

OliverA · Jan 25, 2024

Looking at the docs it says:

Steps in the process:

Client requests a protected resource without providing credentials.

Server response containing error 401 (unauthorized) is received.

Client requests the same protected resource but this time providing credentials.

The request is successful and access to the resource is granted.

And later it mentions that one of the requests for step #3 above can be:

Or:

Request2
curl --digest -u admin:mypass http://${SHELLY}/rpc/Shelly.DetectLocation

So I would just try the following first:

B4X:

Dim url As String = "http://192.168.5.188/relay/0?turn=on"
Dim hj As HttpJob
hj.Initialize("",Me)
hj.Download(url)
Wait for (hj) JobDone(hj As HttpJob)
If hj.Success Then
    label1.Text = $"Unexpected success: ${hj.GetString}"$
Else
    'Add some code here to ensure you are getting a 401, not something else as a response code.
    '401 is expected, anything else is an error
    Dim hj2 As HttpJob
    hj2.Initialize("",Me)
    hj2.Username="admin"
    hj2.Password="password"
    hj2.Download(url)
    Wait for (hj2) JobDone(hj2 As HttpJob)
    If hj2.Success Then
        label1.Text = hj2.GetString
    Else
       label1.Text ="Error"
    End If
    hj2.release
End If
hj.release

Note: un-tested code that may have typos/logic errors

So the 401 (if using HTTP calls) is always expected as a proper step#2.

fs007 · Jan 25, 2024

OliverA said:
So I would just try the following first:

Just tried it, but I'm sorry: doesn't work: server answers again with 401 error to h2j

OliverA said:
So the 401 (if using HTTP calls) is always expected as a proper step#2.

as a proper step#1 !

drgottjr · Jan 25, 2024

this is not how it works. the crucial part (from the documentation): "Error 401 with authentication challenge" needs to be implemented.
client asks for resource that he (already) knows will not be returned. server sends 401 with challenge. it's just a dance between server and client. client uses various parts of the challenge to generate the digest. digest is returned to server. if digest has been correctly implemented, server will return the requested resource. b4a can, of course, handle all of this. the documentation spells everything out. it is a simple implementaton of a digest authentication. there are more involved versions.

OliverA · Jan 25, 2024

drgottjr said:
this is not how it works.

Who are you addressing? If you take the info here https://superuser.com/a/1575028 into account, I'm pretty sure I'm on the right track. And yes, I'm following the docs provided by one of the links above

aeric · Jan 25, 2024

@drgottjr is right. This is not an "one time success" request.

I will try to explain in simple words.

To make a successful call, the client needs to provide and fullfil the security requirements to access the resources.

Providing username and password alone is not sufficient in this case for the first time making the call.

In such a case, it is always expected the first time call to be failed (error status code 401).
Meaning, even the client has provided the right credentials, there is no way the client could successfully get a success code 200 (without calling the second time).

With the failed call on first time, the client is now receiving a response from the server together with some information in the header. The interesting info here is the nonce. Another one is the algorithm, which is SHA-256 but it is less important as this may not change as long as the API not changing to a new version.

With the nonce, now the client can make a second attempt. A complete data can be send this time. To generate a complete secured data to be sent, it needs to follow the steps as documented. Concatenating the variables such as username, realm, password, nonce and other mandatory values and hash them with SHA-256 (a few times).

A retry call need to be make. This time the server will receive a request with the required information. So a successful response will be expected.

OliverA · Jan 25, 2024

aeric said:
@drgottjr is right. This is not an "one time success" request.

fs007 said:
What's wrong here ???

Technically the OkHttpClientWrapper should take care of digest authorization (including the 401 return) internally (and therefore my solution is wrong). Meaning, that you do one HTTP call with a username and password and OkHttpClientWrapper should take care of it. So why is it not working? Because looking at the source (here: https://github.com/AnywhereSoftware...ware/b4h/okhttp/OkHttpClientWrapper.java#L257), it looks like OkHttpClientWrapper only uses MD5 hashing for digest, whereas a more modern digest version may use (upon other things) SHA-256 (see https://en.wikipedia.org/wiki/Digest_access_authentication).

So:

1) The author's first post should have just worked
2) My previous posts are wrong
3) The fix is either
a) roll your own, via the JSON method in the docs
b) make a wish for @Erel to add other hashing algorithms to the digest handling portion of OkHttpClientWrapper

drgottjr · Jan 26, 2024

have op unzip the attached to his additional libraries folders.
then, in his project in the additional libraries tab, have him
uncheck "okhttp" and check "okhttp2". if the project is already
open when he unzips the archive, he will need to refresh the
additional libraries tab (right click).
according to square's documentation, SHA-256 is supported.
as a stop-gap measure (and an experiment), i recompiled the
okhttp library and substituted SHA-256 for MD5. okhttp performs
the full www-authentication process, but - as pointed out by member
@OliverA - only with MD5 (the original digest algorithm).
pending a possible update of this internal library, let's see if this
temporary fix gets op past the front door. ok's implementation
of the authentication is fuller than that required by op's server,
so that could be an issue. but let's see.
i ran a quick test, and the modified library seemed to perform
normally. op needs to remember to unselect the internal okhttp
library and select the modified version.

aeric · Jan 26, 2024

I guess no one understood what I was talking about.

It is not about okHttpUtils2 library cannot use in this case. I am using this library.

But no one could make a real test on the device without having it. We just need to wait for the OP to response, if he have tested any posted solution.

If I didn't say clearly, let me say again. The API is not a common API as we usually seen to use with sample code on this forum out of the box. It is a specific customized format. So a little extra work is needed. I have deal with API which required HMAC and nonce before with different format ie using timestamp to ensure information is not tampered by man in the middle.

By the way, one thing I am not sure. Is the device come by default with authentication set? To enable authentication, it may also need to undergo an API call. So the user or owner of the device might successfully made the API call to enable the authentication setting.

Unless the OP is trying to communicate with a device which has been set up by another person.

Otherwise I guess the device by factory setting is set authentication to disabled. Then things will be much simpler. He doesn't need to authenticate by passing the credentials. Just consume the API as normal or pass null when an authentication parameter is required.

Final note: I did read the documentation and study the Nodejs example. Even I not fully understand the code but I got the idea and flow.

aeric · Jan 26, 2024

aeric said:
To enable authentication, it may also need to undergo an API call.

From doc: Authentication can be enabled by setting authentication details through the RPC method Shelly.SetAuth.

aeric said:
Meaning, even the client has provided the right credentials, there is no way the client could successfully get a success code 200 (without calling the second time).

From doc: When communicating over HTTP, this process must be repeated for each request you send to the device.

Alternative :

If the OP doesn't like to make 2 HTTP API calls, then he can use websocket. I have no experience with websocket in B4A. So I didn't provide a solution using websocket.

From doc: However, for communication over websocket there is no need to pass through steps 1 and 2 more than once: you need to construct the auth object only once

Android Question Http digest authorization fails in B4A

Member

Expert

Expert

Expert

Expert

Member

Expert

Member

Expert

Attachments

Member

Expert

Expert

Member

Expert

Expert

Expert

Expert

Expert

Attachments

Expert

Expert

Similar Threads

Privacy & Transparency

Privacy & Transparency