B4J Question [SOLVED] Connect RPi to WiFi via code

[bdunkleysmith]

With it clear that an Arduino would not support my project and the recommendation that I should upgrade to a RPI covered in the thread rMQTT HostName with query parameters and despite my previous less than satisfactory experience with RPi projects, I'm pleasantly surprised with progress, however I'm going to need the support of the B4X community with a couple of roadhumps.

The script from the thread Raspberry PI - setup as WiFi Access Point and Client has enabled me to set up my RPi as an access point and run a server on it to allow configuration of the device through a web app. First I want to configure and connect to a preferred/user selectable WiFi network.

I have successfully collected the SSIDs available to the RPI by running a script containing the command sudo iwlist wlan0 scan:

Sub ScanForWiFiNetworks
    Log("Scanning networks . . ")
    Try
        shl.Initialize("shl", "./scanwifi.sh", Null)
        shl.WorkingDirectory = "/home/pi"
        shl.Run(-1)
    Catch
        Log(LastException.Message)
    End Try
End Sub

Sub shl_ProcessCompleted (Success As Boolean, ExitCode As Int, StdOut As String, StdErr As String)
    If Success And ExitCode = 0 Then
        Dim index As Int = 10
        Dim start As Int = 0
        Dim finish As Int
        WiFiList.Clear
        Do While index < StdOut.LastIndexOf("ESSID:")
            start = StdOut.IndexOf2("ESSID:", index) + 7
            finish = StdOut.IndexOf2("""", start + 2)
            Log(StdOut.SubString2(start, finish))
            WiFiList.Add(StdOut.SubString2(start, finish))
            index = finish + 2
        Loop
    Else
        Log("Shell Command Error: " & StdErr)
    End If
End Sub

so the user can select the preferred WiFi network they want to log into from a dropdown list and enter the password via a simple browser interface:

Sub Handle(req As ServletRequest, resp As ServletResponse)
    resp.Write("<!DOCTYPE html>")
    resp.Write("<html>")
    resp.Write("<body>")

    resp.Write("<h2>Select the preferred WiFI Network</h2>")

    resp.Write("<form action=""/FormHandler"">")
    
    resp.Write("<label For=""ssid"">Choose a network: </label>")
    resp.Write("<Select name=""ssid"" id=""ssid"">")   
    For Each network As String In Main.WiFiList
        resp.Write("<option value=" & network & ">" & network & "</option>")
    Next
    resp.Write("</Select>")
    
    resp.Write("<br><br><label For=""pwd"">Password:</label><br>")
    resp.Write("<input Type=""password"" id=""pwd"" name=""pwd""><br><br>")
    resp.Write("<input Type=""submit"" value=""Submit""><br><br>")
    resp.Write("</form>")
                
    resp.Write("</body>")
    resp.Write("</html>")
End Sub

However while I have the SSID and password in FormHandler, I am unsure how I can "input" those to the RPi so it logs into the preferred SSID.

There are several issues I'm grappling with:

• Despite my research I'm uncertain what command I need to execute to cause the RPi to log into the selected SSID or if indeed there is one.
  
  
• It seems the only way to execute the command would be to put it in a Bash script file and run it, similar to the method used in the ScanForWiFiNetworks sub above
  
  
• However the script file would need to be updated each time the user selects the SSID and inputs their password, and I wonder if it would retain its executable status each time it is updated.

It may be that the above approach is wrong and won't work, and so I'd appreciate comments as to whether the approach would work and if so what is the correct command, or what a more appropriate approach would be, for instance would effectively editing wpa_supplicant.conf and then restarting wlan0 work.

 

[Chris2]

It's a long time since I was playing with a RaspPi, but I've dug out what I had back then and tried to decipher it.
I didn't really know what I was doing. It was a project that never got finished, and I know there were bits that didn't work satisfactorily, so take everything that follows with a pinch of salt.

As in your last comment, it seems that in order to change the Wifi settings I was writing to the 'wpa_supplicant.conf' file directly. I'm pretty sure this was working OK:

Private strPhrase As String

'build new wpa_supplicant.conf file
Private Sub BuildSupplicant (strCountry As String, newSSID As String, strPW As String)
    Getpsk(newSSID, strPW)
    Wait For shpsk_Complete 'strPhrase now has encrypted passphrase
    
    Dim str As String = $"ctrl_interface=DIR=/var/run/wpa_supplicant GROUP=netdev"$ & CRLF & _
                        $"update_config=1"$ & CRLF & _
                        $"country="$ & strCountry & CRLF & _
                        CRLF & _
                        $"network={"$ & CRLF & _
                        $"        ssid=""$ & newSSID & $"""$ & CRLF & _
                        $"        "$ & strPhrase
    File.WriteString("/etc/wpa_supplicant", "wpa_supplicant.conf", str)
    CallSubDelayed(Me, "BuildSupplicant_Complete")
End Sub

'get encrypted psk for use in wpa_supplicant.conf file
Private Sub Getpsk (strSSID As String, strPW As String)
    Dim str1, str2 As String
    str1 = $"""$ & strSSID & $"""$
    str2 = $"""$ & strPW & $"""$
    sh.Initialize("shpsk", "wpa_passphrase", Array As String(str1, str2))
    sh.WorkingDirectory = "/home/pi"
    sh.Run(-1)
    ws.Flush
End Sub

Private Sub shpsk_ProcessCompleted (Success As Boolean, ExitCode As Int, StdOut As String, StdErr As String)
    If Success And ExitCode = 0 Then
        Log("shpsk success" & CRLF & StdOut)
        Dim idxPSK As Int = StdOut.LastIndexOf($"psk="$)
        strPhrase = StdOut.SubString(idxPSK)
    Else
        Log("shpsk shell Error: " & CRLF & StdErr)
        Dim error As String = "Site WIFI page shpsk_ProcessCompleted error:" & CRLF &  "ExitCode: " & CRLF & ExitCode & CRLF &  "StdErr: " & CRLF & StdErr
        LogFile.AppendToLogFile(error)
    End If
    ws.Flush
    CallSubDelayed(Me, "shpsk_Complete")
End Sub

To execute the changes I found (at the time) I had to reboot the Pi:

Private Sub RebootPi
    sh.Initialize("shReboot", "reboot", Array As String("now"))
    sh.WorkingDirectory = "/home/pi"
    sh.Run(-1)
    ws.Flush
End Sub

I did try the following to instate the new SSID or PW, but I have notes saying it didn't work:

'RestartWlan                'THIS NOT WORKING!!
DisEnableWifi("down")        'THIS NOT WORKING!!
DisEnableWifi("up")            'MAY JUST HAVE TO REBOOT THE UNIT AND HANDLE IT IN THE WEB INTERFACE AND APP!

Private Sub DisEnableWifi (UpDown As String)
    sh.Initialize("shDisable", "ip", Array As String("link", "set", "wlan0", UpDown))
    sh.WorkingDirectory = "/home/pi"
    sh.Run(-1)
    ws.Flush
End Sub

Private Sub shDisable_ProcessCompleted (Success As Boolean, ExitCode As Int, StdOut As String, StdErr As String)
    If Success And ExitCode = 0 Then
        Log("shDisable success")
    Else
        Log("shDisable shell Error: " & CRLF & StdErr)
        Dim error As String = "Site WIFI page shDisable_ProcessCompleted error:" & CRLF &  "ExitCode: " & CRLF & ExitCode & CRLF &  "StdErr: " & CRLF & StdErr
        LogFile.AppendToLogFile(error)
    End If
    ws.Flush
    CallSubDelayed(Me, "shpsk_Complete")
End Sub

I think a lot of this came from these two websites:

Raspberry pi 3 B+ as Access Point and Wifi client - Raspberry Pi Forums

forums.raspberrypi.com

Raspberry Pi Documentation - Configuration

The official documentation for Raspberry Pi computers and microcontrollers

www.raspberrypi.com

 

[bdunkleysmith]

Thanks for your feedback @Chris2 and the links to further references. I'll digest all of that and see how to proceed.

 

[Chris2]

No worries. I'll be interested to hear if you find a way to get the RPi to switch to the new WiFi network without having to reboot it.

 

[Chris2]

Despite my research I'm uncertain what command I need to execute to cause the RPi to log into the selected SSID or if indeed there is one.

I didn't spot this before but the forum's Similar Threads function below may have proved its worth here...

[Solved] RPi connect to a different wifi network

I have probably overseen something simple, if so forgive my question. I am using a Rpi which is connected via bridge in a wifi network to develop my program. In my b4j program I need to change the wifi network and connect to a different wifi containing a mqtt broker. My development network is...

www.b4x.com

 

[bdunkleysmith]

Thanks @Chris2 I don't know how I missed that thread in all of my Forum searching given the title is so relevant to my problem, but thank you for being so observant and pointing me to it.

I have been distracted with other matters over the last couple of days and so I will try to implement that concept in coming days.

Incidentally, I have a "love/hate" relationship with RPi. I am very comfortable with developing apps for PC with B4J, Arduino with B4R and even Android with B4A because they require minimal knowledge of the underlying operating system. But for RPI it seems a good knowledge of Linux is required because B4J can't command many functions directly, but relies on shell scripts, something I still struggle with. However I will persist with this project because the concept of being able to configure a non-ui application via a web interface rather than a clumsy 5 button / 24x2 character LCD I have used in a similar Arduino project is very attractive. Therefore assistance (= encouragement) from other Forum users such as yourself is very important to me.

 

[bdunkleysmith]

Well more frustrating hours on this project . . . RPi obviously doesn't love me!

Excited by the solution posted by @Mark Read in this post, I tried switching between WiFi networks using the following on the command line:

sudo wpa_cli select_network 0

and

sudo wpa_cli select_network 1

however I see no sign of the RPi swapping WiFi connections. Hovering over the WiFi icon on the bottom task bar on the RPi GUI shows it remaining connected to the original network and that is confirmed by looking at the devices connected to each network.

I have tried:

sudo wpa_cli save_config

which does appear to update wpa_suppliant.conf file to mark all but the selected network as "disabled=1"

and

sudo wpa_cli reconfigure

which I understand forces wpa_supplicant to reread its configuration file and so thought it might force a swap of WiFi network, but the RPi is firmly attached to the original network.

I can swap WiFi networks with the RPi GUI and so why is it so hard to do it from the command line?

I'm obviously missing something, but what . . . .

 

[bdunkleysmith]

Eureka!

I presume because I have the RPI configured as both an AP and WiFi client, I need to be explicit in the selection of the network interface. Therefore the command becomes:

sudo wpa_cli -i wlan0 select_network 0

Of course I need to finalise the code to update wpa_suppliant.conf and identify the desired network index to use in that command, but I've now passed over the roadhump that was preventing the actual network selection.

Thanks @Chris2 and @Mark Read.

 

[Chris2]

Incidentally, I have a "love/hate" relationship with RPi. I am very comfortable with developing apps for PC with B4J, Arduino with B4R and even Android with B4A because they require minimal knowledge of the underlying operating system. But for RPI it seems a good knowledge of Linux is required because B4J can't command many functions directly, but relies on shell scripts, something I still struggle with. However I will persist with this project because the concept of being able to configure a non-ui application via a web interface rather than a clumsy 5 button / 24x2 character LCD I have used in a similar Arduino project is very attractive. Therefore assistance (= encouragement) from other Forum users such as yourself is very important to me.

Eureka!

I'll feel the same. Without the help/encouragement from these forums, I would really struggle.
I do try to remember though that the more frustrating a problem is, the greater the relief & satisfaction when you find a solution!

sudo wpa_cli -i wlan0 select_network 0

Are you able to make this work directly with jShell, or did you have to create a script file as @Mark Read seemed to in the other thread?

 

[bdunkleysmith]

Are you able to make this work directly with jShell, or did you have to create a script file as @Mark Read seemed to in the other thread?

Yes @Chris2 and in fact I seem to be having more success invoking commands directly than via a script file. Using the following works for me:

shl.Initialize("shl", "wpa_cli", Array As String("-i", "wlan0", "select_network", 0))

However I am now working on a problem whereby I lose my web browser connection to the RPi AP when I switch WiFi networks back and forth. For my application, I need to maintain that connection because it is the means by which I want to configure my app, which includes WiFi network selection and I'd prefer not to have to reconnect the client to the AP each time I change WiFi networks. I'm out of my depth, but I wonder if it's a dhcp issue which affects the AP even though I'm changing the WiFi network.

So while I have solved the problem of how to detect available WiFi networks and select a network, I've not solved the problem of maintaining web browser access via the AP when changing WiFi networks. I'll keep chipping away at it.

 

[Chris2]

That's good news that you're able to work the command with jShell.

I'm not sure why switching wlan0 would affect the AP but thinking out loud...

What action are you having to take to reconnect to it? Is the action needed at the client or the RPi?
Is the AP still available after you make the wlan0 change (i.e. the client just disconnects, but can reconnect freely when you tell it to)?
Could the IP address ranges being used by the AP & the WiFi be conflicting in some way?

 

[bdunkleysmith]

Thanks for sharing your thoughts out loud @Chris2.

Yes I'm pleased to be able to use jShell directly rather than via scripts. I have also been able to replace:

shl.Initialize("shl", "./scanwifi.sh", Null)

used to scan for available WiFi networks in my original code in post #1 with

shl.Initialize("shl", "iwlist", Array As String("wlan0", "scan"))

I don't think it's related to a conflict of IP address ranges because the AP is 10.0.0.XX and the other two WiFi networks I'm swapping between are 192.168.43.XX and 192.168.00.XX.

As far as I can see from having a couple of devices connected to the AP, the AP remains visible, however when the swap of WiFi networks occurs, the AP drops the connection. Now on one device it reconnects to the AP because it has an Auto reconnect option on, but on another which doesn't have such an option, it seems to prefer to swap to my home AP, although I can manually re-connect it.

This is probably related more to the RPi than B4J and so perhaps I should pose the question on an RPi forum.

 

[bdunkleysmith]

Isn't it often the way that when solving a problem you suspect a complex cause and it turns out to be something quite basic!

As an RF Engineer by profession, it should have been obvious that with the RPI having just a single RF transceiver to support both WiFi and AP connections, the channel number of both connections would have to remain the same at all times to achieve my objective.

While at boot-up I set the RPi AP to channel 2, when the RPi connects via WiFi, the transceiver will switch to the channel assigned by the AP to which the RPi connects. This may be any channel number and the RPi will accordingly switch to the same channel, say 9 in my test case.

My external device will connect to the RPi using that channel, and I can successfully identify WiFi networks available to the RPi and switch to the desired network via the code in post #10 by specifying the index of the appropriate network entry in wpa_supplicant.conf.
Note that @Chris2's code in post #2 shows how to write a wpa_supplicant.conf file.

However, when the RPi connects to the new WiFi network, its transceiver will change to the channel number of that network, which is unlikely to be the original channel number. For instance in my test case it is channel 1. And of course the AP channel now becomes that new channel number, with the now obvious consequential effect of dropping the connection with the device previously connected to perform the configuration setup. Of course it can be reconnected, but I don't like that inconvenience for the user.

So given with the B4X community's help I've been able to connect to a WiFi network via code as per the thread title, I'll mark this thread as SOLVED, but unfortunately my proposed arrangement of configuring WiFi networks via the AP connection won't work as desired with the RPi. I will either have to accept the need to reconnect to the AP after a WiFi network switch, add a WiFi dongle to the RPi to be able to provide two separate channels or I am thinking of exploring use of a Bluetooth connection over which to do the configuration.

 

[Chris2]

I'm glad you managed to find the issue, and come to a (albeit unsatisfactory) conclusion!

Note that @Chris2's code in post #2 shows how to write a wpa_supplicant.conf file.

... although not necessarily with the tidiest code. I clearly didn't understand the capabilities of the Smart String Literal when I wrote that. I think all those '& CRLF &' are completely unnecessary!