B4J Question Why doesn't Wait For work in a loop?

[rgarnett1955]

Hi.

The purpose of the following code is to test how a Wait For (trigger) will pause the loop until some condition is met.

Except that it doesn't work; the loop just ignores the Wait For.

So I guess the question is how do I implement a semaphore in B4x.

#Region Project Attributes
    #MainFormWidth: 600
    #MainFormHeight: 600
#End Region

Sub Process_Globals
    Private fx As JFX
    Private MainForm As Form
    Private xui As XUI
    Private Button1 As B4XView
    
    Private bc As ByteConverter
    Private b(20) As Byte
    
    Private strToReceive As String
    Private noStringsReceived As Int
    Private bigString As String
    Private RxDataComplete As Boolean
End Sub

Sub AppStart (Form1 As Form, Args() As String)
    MainForm = Form1
    MainForm.RootPane.LoadLayout("Layout1")
    MainForm.Show
End Sub

Sub Button1_Click
    bigString = ""
    RxDataComplete = True
    strToReceive = "My new String"
    
    trigger
    
    For k = 0 To 4
        Log(DateTime.Time(DateTime.Now))
        wait for (trigger) Complete(ret As Boolean)
        Log(DateTime.Time(DateTime.Now))
        
        b = bc.StringToBytes((strToReceive & " " & k), "UTF8")
        strm_NewData(b)
    Next
    
    Log("All done")
End Sub

Private Sub strm_NewData(buffer() As Byte)
    strToReceive = bc.StringFromBytes(buffer, "UTF8")
    
    Log(strToReceive)
    
    ProcessData(strToReceive)
End Sub

Private Sub ProcessData(s As String)
    Sleep(2000)
    noStringsReceived = noStringsReceived + 1
    bigString = bigString & s & Chr(10)
    Log(bigString)
    
    If noStringsReceived < 5 Then
        Log("No Received  " & noStringsReceived & "  " & DateTime.Time(DateTime.Now))
    End If
    
    Sleep(10)
End Sub

Private Sub trigger As ResumableSub
    Sleep(1)
    Return True
End Sub

 

[Sagenut]

The Wait For it's not ignored.
But what do you expect with a 1ms Sleep?
I don't know what you are trying to obtain, but if you want all the requested Subs to be executed in a row before going on with the For cycle you need to call all of them with a Wait For.
Check if this is what you want.

 

[rgarnett1955]

Hi,

Thanks for that.

I understand how it works now. I didn't get that the Wait For sub actually called the sub, I thought it just waited until the sub had been run by another part of the code.

The problem is that when using the "real" asyncStreams asyncStreams_NewData with ethernet, or serial ports this sub is called by the serial port, it is not executed as

Wait For "(asyncStream_NewData(b)) complete (useless As Object)."

It is automatically executed when the I/O port has a certain amount of data after some operating system time interval.

So the question is how can you create a semaphore in B4x to coordinate asynchronous read/writes to a port such that you can wait until all bytes are received before issuing a new command (write) to a serial port.? I can't use prefix mode as I only program one end of the serial link. I do know how much data I will get and there is a frame terminator. Sadly asyncStreams doesn't have a frame terminator feature so data turns up in chunks with repeated calls to asyncStreams_NewData.

I struggle a bit with these high level languages, because I have done heaps of code in C for embedded systems using FreeRTOS where you have semaphores, notifications and mutexes to coordinate I/O, it's very flexible and easy to do.

Kind regards
Rob

 

[Chris2]

OK, I think I know what you're getting at now. A few points....

..I didn't get that the Wait For sub actually called the sub, ...

That's it, so:

Sub Button1_Click
:
''    trigger        'As you say, this shouldn't be here because.....'
   
    For k = 0 To 4
:
        wait for (trigger) Complete(ret As Boolean)     '....this calls trigger
:
    Next
End Sub

But, there's more than one way to use Wait For, which may be where your confusion lies....

Sadly asyncStreams doesn't have a frame terminator feature so data turns up in chunks with repeated calls to asyncStreams_NewData.

I think that's where the AsyncStreamsText class comes in. In your other thread you said you have a sub that is a version of AsyncStreamsText. but you should use the whole AsyncStreamsText class. Use its astreams_NewData sub to buffer the incoming data until you hit the termination character, then send it back to the calling sub.
So, in the AsyncStreamsText.astreams_NewData sub, you'll have something like:

Private Sub astreams_NewData (Buffer() As Byte)
   
    buffer.Append(BytesToString(Buffer, 0, Buffer.Length, charset))  'buffer is StringBuilder'
    Dim s As String = buffer.ToString

    Dim endFound As Boolean = s.EndsWith("EndCharacter") 'Or better use regex to make sure you've got a suitable, complete reply.

    If endFound Then
        CallSubDelayed2(mTarget, mEventName & "_NewText", s)  'callback to AsyncStreamsText_NewText in the calling module'
        buffer.Initialize                                     'clear the buffer
    End If
End Sub

Then in the module that calls AsyncStreamsText something like:

Private ast As AsyncStreamsText

Sub Button1_Click
ast.Initialize(Me, "ast", NewSocket.InputStream, NewSocket.OutputStream) 'initialize AsyncStreamsText with the socket streams.
Wait For Reply_Complete    'which is called when AsyncStreamsText passes a complete reply back to ast_NewText in this module
End Sub


'Called from class AsyncStreamsText.astreams_NewData'
Private Sub ast_NewText (Text As String)

'Do whatever validation, processing you want........'  
    CallSubDelayed(Me, "Reply_Complete")  'Or CallSubDelayed2(...) with a 'success' parameter if you want for example

End Sub

Edit: Corrected typo 'buffers.EndsWith' to 's.EndsWith'

 

[emexes]

that's where the AsyncStreamsText class comes in

AsyncStreamsText would work great except for receiving the ready for next command prompt that likely has no end-of-line termination character.

 

[Chris2]

What is the ready for next command prompt?
Is it a string of ascii characters or something else?

I may be missing somthing here but if it's "cr> ", then why can't @rgarnett1955 just pick it out of the incoming stream in the AsyncStreamsText.astreams_NewData sub using regex or buffer.contains or buffer.EndsWith, then act on it accordingly?

 

[emexes]

What is the ready for next command prompt?
Is it a string of ascii characters or something else?

It is a string of 5 ASCII characters: linefeed 'c' 'r' '>' space

It then returns a reply dependent on the command issued. At the end of the reply it sends a <LF>cr>space.

 

[Jeffrey Cameron]

OK, I think I know what you're getting at now. A few points....

That's it, so:

Sub Button1_Click
:
''    trigger        'As you say, this shouldn't be here because.....'
   
    For k = 0 To 4
:
        wait for (trigger) Complete(ret As Boolean)     '....this calls trigger
:
    Next
End Sub

But, there's more than one way to use Wait For, which may be where your confusion lies....

I think that's where the AsyncStreamsText class comes in. In your other thread you said you have a sub that is a version of AsyncStreamsText. but you should use the whole AsyncStreamsText class. Use its astreams_NewData sub to buffer the incoming data until you hit the termination character, then send it back to the calling sub.
So, in the AsyncStreamsText.astreams_NewData sub, you'll have something like:

Private Sub astreams_NewData (Buffer() As Byte)
   
    buffer.Append(BytesToString(Buffer, 0, Buffer.Length, charset))  'buffer is StringBuilder'
    Dim s As String = buffer.ToString

    Dim endFound As Boolean = buffers.EndsWith("EndCharacter") 'Or better use regex to make sure you've got a suitable, complete reply.

    If endFound Then
        CallSubDelayed2(mTarget, mEventName & "_NewText", s)  'callback to AsyncStreamsText_NewText in the calling module'
        buffer.Initialize                                     'clear the buffer
    End If
End Sub

Then in the module that calls AsyncStreamsText something like:

Private ast As AsyncStreamsText

Sub Button1_Click
ast.Initialize(Me, "ast", NewSocket.InputStream, NewSocket.OutputStream) 'initialize AsyncStreamsText with the socket streams.
Wait For Reply_Complete    'which is called when AsyncStreamsText passes a complete reply back to ast_NewText in this module
End Sub


'Called from class AsyncStreamsText.astreams_NewData'
Private Sub ast_NewText (Text As String)

'Do whatever validation, processing you want........'  
    CallSubDelayed(Me, "Reply_Complete")  'Or CallSubDelayed2(...) with a 'success' parameter if you want for example

End Sub

FWIW: I would point out that in serial communications there is a possibility of multiple responses being returned concurrently, you may get more than one full or full-and-partial response in the receive event. Your buffering logic should account for that and maintain the buffer accordingly.

 

[Chris2]

It is a string of 5 ASCII characters: linefeed 'c' 'r' '>' space

OK, then in the AsyncStreamsText.astreams_NewData sub couldn't @rgarnett1955 do something like:

Private Sub astreams_NewData (Buffer() As Byte)
    
    buffer.Append(BytesToString(Buffer, 0, Buffer.Length, charset))  'buffer is StringBuilder'
    Dim s As String = buffer.ToString

    Dim endFound As Boolean = s.EndsWith("EndCharacter")   'Or better, use regex to make sure you've got a suitable, complete reply.

    If endFound Then
      
       CallSubDelayed2(mTarget, mEventName & "_NewText", s)  'callback to AsyncStreamsText_NewText in the calling module'
          
    Else If s.EndsWith($"${Chr(10)}cr> "$) Then                         'again regex or s.Contains might be better
        
        CallSubDelayed(mTarget, mEventName & "_ReadyForNextCommand")  'callback to AsyncStreamsText_ReadyForNextCommand in the calling module'
        buffer.Initialize                                     'clear the buffer
    
    End If
End Sub

you may get more than one full or full-and-partial response in the receive event

All the partial bits should arrive in the correct order though, right?
So if we're buffering until we see the 'EndCharacter' &/or the 'ready for next command prompt' then we can be sure that what we have in the buffer contains everything we need, in the right order? (even if we have to apply some regex matching to remove surplus characters)

 

[Jeffrey Cameron]

All the partial bits should arrive in the correct order though, right?
So if we're buffering until we see the 'EndCharacter' &/or the 'ready for next command prompt' then we can be sure that what we have in the buffer contains everything we need, in the right order? (even if we have to apply some regex matching to remove surplus characters)

One time at band camp... No, seriously, I ran into a problem with a specific series of Dell computers that had an odd UART implementation. Other PC's were fine, just that one series had an issue communicating with a serial control device.

Let's say the device is using "X" as the EOT character. _Sometimes_, and only sometimes, we'd get the first bit of the next message in line tacked on to the end of the first message. Instead of

Message 01X
Message 02X

We would receive

Message 01XMe
ssage 02X

Took forever to track that edge case down. Once we reimplemented the receive method to only send full messages, and preserve partial starts for the next cycle, it worked fine.

 

[rgarnett1955]

What is the ready for next command prompt?
Is it a string of ascii characters or something else?

I may be missing somthing here but if it's "cr> ", then why can't @rgarnett1955 just pick it out of the incoming stream in the AsyncStreamsText.astreams_NewData sub using regex or bH

Hi,

I do pick it out the end of frame, I do use regex, but how do I stop the next command being sent out until I receive the End Of Frame (eof) characters. In b4x you cannot do this with the asynch streams.

If you put a sequence of commands in a loop you cannot make the loop wait for the detection of the eof characters in the receive string.

As I explained the TSA is designed to be used as a terminal with something like Terra term so it outputs a prompt when it is ready for new data.

?
ch> scan 320M 330M 21 3                    <== Text input         Tx to TSA
320000000 -1.178125e+02 0.000000000        <== Data returned    Rx From TSA
320500000 -1.163125e+02 0.000000000            .
321000000 -1.168125e+02 0.000000000            .
321500000 -1.153125e+02 0.000000000            .
322000000 -1.163125e+02 0.000000000            .
322500000 -1.168125e+02 0.000000000            .
323000000 -1.138125e+02 0.000000000            .
323500000 -1.158125e+02 0.000000000            .
324000000 -1.178125e+02 0.000000000            .
324500000 -1.188125e+02 0.000000000            .
325000000 -1.158125e+02 0.000000000            .
325500000 -1.168125e+02 0.000000000            .
326000000 -1.203125e+02 0.000000000            .
326500000 -1.178125e+02 0.000000000            .
327000000 -1.188125e+02 0.000000000            .
327500000 -1.148124e+02 0.000000000            .
328000000 -1.153125e+02 0.000000000            .
328500000 -1.143125e+02 0.000000000            .
329000000 -1.153125e+02 0.000000000            .
329500000 -1.158125e+02 0.000000000            .
330000000 -1.163125e+02 0.000000000            < Up to a maximum of 260 lines
ch>                                            <== TSA Prompt

No one seems to have looked at my flowchart or my code which I published in detail.

Again, the question isn't how to detect the end of a receive stream it is how do I stop a loop that sends commands to the TSA from incrementing on until I the eof from the NewData sub has been received.

In any event I have given up on trying to create some sort of semaphore in B4x and have restructured the program so that a semaphore isn't required To do this I therefore have to depend on an arbitrary time delay to wait until I have "hopefully" all the data. The time delay is cannot be fixed and has to be determined by a trial run because the time between the TSA getting a scan command and transmitting back the data is highly variable. It can range from less than 100 ms to minutes depending on the number of points scanned and the real-time bandwidth setting of the TSA. The TSA can transmit back a maximum of 260 lines of data of three numbers per line seperated by a space.

So the time delay has to be estimated, tested and set before continuous scanning using a timer set for longer than the delay is commenced.

If jSerial and Asynchstreams had the facility to detect an eof with a timeout such that the new data sub didn't fire until the eof or timeout then it would be simple, but it does not have this facility.

It is not a problem with the TSA. I sent 100,000 scan requests to the TSA and for each request I correctly got back the 21 point records within 100 ms using Docklight scripting. I analysed this data using regex to look for exceptions.


Regards
Rob

 

[Chris2]

the question isn't how to detect the end of a receive stream it is how do I stop a loop that sends commands to the TSA from incrementing on until I the eof from the NewData sub has been received.

If jSerial and Asynchstreams had the facility to detect an eof with a timeout such that the new data sub didn't fire until the eof or timeout then it would be simple, but it does not have this facility.

I'm perfectly willing to believe that I can't give you what you're after, but not that B4X can't
So let me have one more go:

Sub Process_Globals
    Private ast As AsyncStreamsText
    Private ListOfCommands as List
End Sub


Sub Button1_Click
 
       ast.Initialize(Me, "ast", socket.InputStream, socket.OutputStream)

    For k = 0 To ListOfCommands.Size-1
 
        Dim command as string = ListOfCommands.Get(k)
        Dim cancel(1) As Boolean
        Timeout(cancel, 2, "Reply_Complete")
        ast.Write(command)
        Wait For Reply_Complete
     
        If cancel(0) Then    'command timed out.
            Log("Time out: No valid reply received.")
            Exit 'Exit the loop unless you want the next command send regardless'
        Else
            cancel(0) = True    'cancels the timeout
            Log("Valid Reply Received. Continue to next command")
        End If
     
    Next
 
End Sub


'Text received here has been validated in AsyncStreamsText as a complete valid reply with the eof characters'
Private Sub ast_NewText (Text As String)
    Log(Text)
    CallSubDelayed(Me, "Reply_Complete")
End Sub


'https://www.b4x.com/android/forum/threads/b4x-b4xlib-waitforwithtimeout-timeouthandler.110703/post-690966'
Sub Timeout(Cancel() As Boolean, Duration As Int, EventName As String)
   Do While Duration > 0
       'lblCountdown.Text = Duration
       Sleep(1000)
       If Cancel(0) Then Return
       Duration = Duration - 1
   Loop
   Cancel(0) = True
   CallSubDelayed(Me, EventName)
End Sub

#Event: NewText (Text As String)
#Event: Terminated

'version: 1.00
'Class module
Sub Class_Globals
    Private mTarget As Object
    Private mEventName As String
    Private astreams As AsyncStreams
    Private sb As StringBuilder
End Sub

Public Sub Initialize (TargetModule As Object, EventName As String, In As InputStream, out As OutputStream)
    mTarget = TargetModule
    mEventName = EventName
    astreams.Initialize(In, out, "astreams")
    sb.Initialize
End Sub

'Sends the text. Note that this method does not add end of line characters.
Public Sub Write(Text As String)
    astreams.Write(Text.GetBytes("ASCII"))
End Sub

Private Sub astreams_NewData (Buffer() As Byte)
 
    sb.Append(BytesToString(Buffer, 0, Buffer.Length, "ASCII"))
    Dim s As String = sb.ToString
 
    Dim endFound As Boolean = s.EndsWith("ch> ")   'Or use regex to make sure you've got a suitable, complete reply as far as possible.

    If endFound Then
   
       CallSubDelayed2(mTarget, mEventName & "_NewText", s)  'callback to ast_NewText in the calling module'
        sb.Initialize   'clear the buffer ready for the next command's reply

    End If


End Sub


Private Sub astreams_Terminated
    CallSubDelayed(mTarget, mEventName & "_Terminated")
End Sub

Private Sub astreams_Error
    astreams.Close
    CallSubDelayed(mTarget, mEventName & "_Terminated")
End Sub

Public Sub Close
    astreams.Close
End Sub

 

[emexes]

the question isn't how to detect the end of a receive stream it is how do I stop a loop that sends commands to the TSA from incrementing on until I the eof from the NewData sub has been received.

Did you get this working to do what you need done?

I feel that if you only send the next command when you "detect the end of a receive stream" of the response to the previous command, with a watchdog timer to restart the communication "conversation" if it has died for some reason (serial data interruption, TSA rebooted, app restarted, etc) then it should work fine.

This is assuming that, at the end of each response, the TSA sends a "command prompt" to indicate it is ready for the next command.

The command prompt being 4 bytes with values: 10 (linefeed) 93 (lowercase c) 114 (lowercase r) and 62 (greater than).

I suspect that the overlap of the last byte (linefeed) of the previous response being the first byte (linefeed) of the command prompt is where most usual parsing styles might trip up, especially at TSA startup (eg no leading linefeed issued, just the 93, 114 and 62 bytes) or if there is noise on the serial line after the end of the previous response (eg receive previous response data bytes, 10, random noise bytes, 92, 114, 62).

But I also think that those situations would be rare, and would be quickly resolved by the watchdog timer, assuming that there are no large gaps or delays in the TSA's responses (because the watchdog period would need to be set to the longest of those gaps or delays).

If you're suffering burnout on this topic and your eyes are glazing over as you read this, then have a break from it (if you can) and then I'd be happy to go through implementing it with you afresh. Admittedly it was 15 years ago now, but I spent more than a decade interfacing to automotive test equipment, gas analysers, data acquisition systems, online parts ordering portals, etc, all via RS232. Lol prior to that, I was scraping stock prices off MoneyWatch (CommBank's pre-internet Viatel service at 1200/75 baud) and before that, using VK100 terminal (that had inbuilt BASIC) to emulate a (very fast) operator to a VAX minicomputer. Lol, those were the days.

 

[OliverA]

This may be a solution (complete project attached):

Sub Class_Globals
    Private Root As B4XView
    Private xui As XUI
    
    Private TimeoutTimer As Timer        'Required for this solution
    Private buffer As StringBuilder        'Required for this solution
    Private testData As List            'For test data
    Private listPtr As Int                'For test data
End Sub

Public Sub Initialize
'    B4XPages.GetManager.LogEvents = True
    TimeoutTimer.Initialize("Timeout", 2000) 'Required - a large value for testing purposes
    buffer.Initialize                         'Required
    'For setting up test data
    testData.Initialize
    testData.Add($"320000000 -1.178125e+02 0.000000000${CRLF}"$)
    testData.Add($"320500000 -1.163125e+02 0.000000000${CRLF}"$)
    testData.Add($"ch> "$)
End Sub

'This event will be called once, before the page becomes visible.
Private Sub B4XPage_Created (Root1 As B4XView)
    Root = Root1
    Root.LoadLayout("MainPage")
End Sub

'You can see the list of page related events in the B4XPagesManager object. The event name is B4XPage.

'For emulating receivign test data
Private Sub Button1_Click
    Log($"Button1_Click: Sending ${testData.Get(listPtr)}"$)
    strm_NewData(testData.Get(listPtr).As(String).GetBytes("UTF8"))
    listPtr = listPtr + 1
    If listPtr >= testData.Size Then listPtr = 0
End Sub


'Required
Private Sub Timeout_Tick
    TimeoutTimer.Enabled = False
    Log($"Capturing data timed out. Received ${buffer.Length} bytes"$)
    If buffer.Length > 0 Then Log($"Data received: ${buffer.ToString}"$)
    CallSubDelayed2(Me, "TSA_NewData", "")
    buffer.Initialize ' Reset Buffer
End Sub

'Required
Private Sub strm_NewData(data() As Byte)
    If TimeoutTimer.Enabled Then
        Dim chunk As String = BytesToString(data, 0, data.Length, "UTF8")
        buffer.Append(chunk)
        If chunk.ToLowerCase.EndsWith("ch> ") Then
            TimeoutTimer.Enabled = False
            CallSubDelayed2(Me, "TSA_NewData", buffer.ToString)
            buffer.Initialize ' Reset Buffer
        End If
    Else
        Log($"Not expecting any new data, discarding: ${BytesToString(data,0,data.Length, "UTF8")}"$) '
    End If
End Sub

'For emulating the Wait For of TSA_NewData
Private Sub Button2_Click
    If TimeoutTimer.Enabled = False Then
        Log("Waiting for data...")
        TimeoutTimer.Enabled = True                'Enabling the timer goes hand in hand with
        Wait For TSA_NewData(data As String)    'Waiting for the event
        Log("TSA_NewData:")
        If data <> "" Then
            Log($"${TAB}Raw/unparsed data received: ${CRLF}${data}"$)
        Else
            Log($"${TAB}Timout occurred"$)
        End If
    Else
        Log("Already waiting...")
    End If
End Sub

Code is from B4XMainPage class module of a B4XPages project

 

[OliverA]

Except that it doesn't work; the loop just ignores the Wait For.

So I guess the question is how do I implement a semaphore in B4x.

The above also works in a loop. Change Button2_Click to

'For emulating the Wait For of TSA_NewData
Private Sub Button2_Click
    If TimeoutTimer.Enabled = False Then
        Dim loopCount As Int = 4
        Log($"Waiting for data ${loopCount} times"$)
        For x = 0 To loopCount - 1
            Log($"Wait #${x+1}"$)
            TimeoutTimer.Enabled = True                'Enabling the timer goes hand in hand with
            Wait For TSA_NewData(data As String)    'Waiting for the event
            Log("TSA_NewData:")
            If data <> "" Then
                Log($"${TAB}Raw/unparsed data received: ${CRLF}${data}"$)
            Else
                Log($"${TAB}Timout occurred"$)
            End If
        Next
    Else
        Log("Already waiting...")
    End If
End Sub

Note: I think the semaphore you were looking for/were trying to implement something like is the Timer object?

 

[emexes]

Could you please confirm whether the TSA prompt is "cr> " or "ch> " ?

the TSA has received some data and has replied with its prompt "cr> "
...
At the end of the reply it sends a <LF>cr>space.

?
ch> scan 320M 330M 21 3                    <== Text input         Tx to TSA
320000000 -1.178125e+02 0.000000000        <== Data returned    Rx From TSA
320500000 -1.163125e+02 0.000000000            .
321000000 -1.168125e+02 0.000000000            .
...
329000000 -1.153125e+02 0.000000000            .
329500000 -1.158125e+02 0.000000000            .
330000000 -1.163125e+02 0.000000000            < Up to a maximum of 260 lines
ch>                                            <== TSA Prompt