B4J Question USB Serial Port

[ElliotHC]

Hi

I've just come to do a project where I need to read bytes coming in from the serial port, is there a simple example I can dissect?

Thanks

 

[Erel]

jSerial library

 

[ElliotHC]

Thanks Erel,

Do you know why when I've got some hardware sending out "TEST" every second, if I try to connect it says:

Waiting for debugger to connect...
Program started.
java.lang.RuntimeException: Message size too large. Prefix mode can only work if both sides of the connection follow the 'prefix' protocol.
at anywheresoftware.b4a.randomaccessfile.AsyncStreams$AIN.run(AsyncStreams.java:220)
at java.lang.Thread.run(Thread.java:748)
Error: (RuntimeException) java.lang.RuntimeException: Message size too large. Prefix mode can only work if both sides of the connection follow the 'prefix' protocol.
Connection is broken.

 

[emexes]

Prefix mode adds a 32-bit length count at the start of each transmitted data block, and expects the same in return. Turn prefix mode off.

Note that prefix mode can only work if both sides of the connection follow the protocol.
Change AStream.InitializePrefix to AStream.Initialize to disable prefix mode.

 

[ElliotHC]

 

[emexes]

Brief trivia sidetrack: the four characters TEST translates to packet size of 1414743380 bytes, which would explain Message size too large.

 

[emexes]

View attachment 82466

You've probably already moved on by now, but astream.Initialize has three parameters, not four:

 

[emexes]

This code:

Sub Process_Globals
  
    Private sp As Serial
    Private astream As AsyncStreams
  
End Sub

Sub AppStart (Form1 As Form, Args() As String)
  
    sp.Initialize("sp")
  
    Log(sp.ListPorts)
  
    sp.Open("COM11")
    sp.SetParams(115200, 8, 1, 0)
  
    astream.Initialize(sp.GetInputStream, sp.GetOutputStream, "astream")

End Sub

Sub AStream_NewData (Buffer() As Byte)
  
    Log(BytesToString(Buffer, 0, Buffer.Length, "UTF8"))

End Sub

is receiving data from an ESP32 module via serial over USB on COM11:

Waiting for debugger to connect...
Program started.
(ArrayList) [COM11]
ets Jun  8 2016 00:22:57
rst:0x1 (POWERON_RESET),boot:0x13 (SPI_FAST_FLASH_BOOT)
configsip: 0, SPIWP:0xee
clk_drv:0x00,q_drv:0x00,d_drv:0x00,cs0_drv:0x00,hd_drv:0x00,wp_drv:0x00
mode:DIO, clock div:1
load:0x3fff0018,len:4
load:0x3fff001c,len:1100
load:0x40078000,len:9232
load:0x40080400,len:6400
entry 0x400806a8
Starting BLE work!
Characteristic defined! Now you can read it in your phone!

 

[ElliotHC]

Go it all sorted, many thanks.

 

[emexes]

With serial-over-USB, watch out for data split over several AStream_NewData events.

Eg, if the sending program sends a 100-byte packet, then a few ms pause, then another 100-byte packet, then the receiving program might have 3 NewData events: eg the first NewData has 80 bytes of the first packet; the second NewData has 20 bytes of the first packet and 60 bytes of the second packet; and the third NewData has 40 bytes of the second packet.

Admittedly the problem might not occur very often if there are significant gaps between the packets, but being intermittent just makes the problem more insidious.

 

[ElliotHC]

Thanks for the heads up. My hardware will send out a start packet, then there will be a large gap while a timer runs, it then sends out the run time... It's working pretty well already..

 

[ElliotHC]

Does anyone know how to set the timeout? I think it might be fractionally longer than my output pace.

 

[emexes]

I think polling period was property of asyncstream. Or maybe the serial port.

 

[emexes]

https://www.b4x.com/b4j/help/jserial.html

ReadingIntervalTimer?

or maybe PurgePort via a timer?

 

[ElliotHC]

With serial-over-USB, watch out for data split over several AStream_NewData events.

Eg, if the sending program sends a 100-byte packet, then a few ms pause, then another 100-byte packet, then the receiving program might have 3 NewData events: eg the first NewData has 80 bytes of the first packet; the second NewData has 20 bytes of the first packet and 60 bytes of the second packet; and the third NewData has 40 bytes of the second packet.

Admittedly the problem might not occur very often if there are significant gaps between the packets, but being intermittent just makes the problem more insidious.

This is exactly what is happening.

I am currently sending "ST" when my timer starts, then "0.000" < the time in seconds.. I'm running the timer in the app but the kit that is timing is very accurate so I am replacing the time with one from the kit.

The issue is that sometimes the data gets split up I only get 'S', 'T' '.000' or sometimes '000'.

How do I prevent this from happening? I don't have time for handshaking or packet checksum etc..


Surely, somewhere there is a timeout setting that I can extend? Madness not to have.

 

[ElliotHC]

www,b4x.com/help/jserial.html

ReadingIntervalTimer

or maybe PurgePort via a timer

ReadingThreadInterval

It was ReadingThreadInterval

I've set sp.ReadingThreadInterval=100 which has totally sorted it!

Thanks

 

[emexes]

Surely, somewhere there is a timeout setting that I can extend?

Yes, but no matter how long you make it, there is always a chance that you might start sending just as the timeout expires, and the packet will still be split.

Madness not to have.

I think it's more that the serial port driver author realised they couldn't cover every possible use case, and so rather than complicate their code by adding special cases each time somebody rolls their own unique protocol, they've just made it as simple as it can be.

 

[emexes]

This is exactly what is happening.

Thank you for the confirmation that my crystal ball is not totally kaput. It's had a few misses recently.

I've set sp.ReadingThreadInterval=100 which has totally sorted it!

That may well do the trick, as long as:
1/ you can live with the delay that it probably introduces
2/ you don't send packets closer together than the period (100 ms?)

Depending on how the timeout works, there is still a chance that packets might be split.
eg, the timeout could be:
1/ every 100 ms, check the serial port
2/ 100 ms after receiving the first character into an empty buffer
3/ 100 ms since the last character received

Option 1, which simply checks the serial port every 100 ms, might happen to check it midway through receiving a packet, and give you what it's got at that point, ie, the front part of a packet; 100 ms later, you'll get the back part of the packet.

 

[emexes]

I don't have time for handshaking or packet checksum etc..

If you're just sending text lines, then it's not hard to accumulate the incoming characters into a buffer and then dole them out when you have a complete line, eg:

Dim TextBuffer As String    'global serial port input buffer

Sub AStream_NewData (Buffer() As Byte)
  
    'add to buffer
    TextBuffer = TextBuffer & BytesToString(Buffer, 0, Buffer.Length, "UTF8")

    Dim EOLChar As String = Char(13)    'or 10 or 0 or whatever

    'dole out lines (if any)
    Do While True
        Dim EOLCharAt As Int = TextBuffer.IndexOf(EOLChar)
        If EOLCharAt = -1 Then
            Exit
        End If

        Dim TextLine As String = TextBuffer.SubString2(0, EOLCharAt)    'get next line
        TextBuffer = TextBuffer.SubString(EOLCharAt + 1)    'remove from buffer

        HandleTextLine(TextLine)
    Loop

End Sub

or if you prefer clarity over efficiency, you could change this bit:

'''Do While True
'''    Dim EOLCharAt As Int = TextBuffer.IndexOf(EOLChar)
'''    If EOLCharAt = -1 Then
'''        Exit
'''    End If

Do While TextBuffer.Contains(EOLChar)
    Dim EOLCharAt As Int = TextBuffer.IndexOf(EOLChar)    'should always find EOLChar

 

[ElliotHC]

Thank you for the confirmation that my crystal ball is not totally kaput. It's had a few misses recently.

That may well do the trick, as long as:
1/ you can live with the delay that it probably introduces
2/ you don't send packets closer together than the period (100 ms?)

Depending on how the timeout works, there is still a chance that packets might be split.
eg, the timeout could be:
1/ every 100 ms, check the serial port
2/ 100 ms after receiving the first character into an empty buffer
3/ 100 ms since the last character received

Option 1, which simply checks the serial port every 100 ms, might happen to check it midway through receiving a packet, and give you what it's got at that point, ie, the front part of a packet; 100 ms later, you'll get the back part of the packet.

I might just change it to something like 1000ms. The minimum time between the two packets is about 1.7s as it's impossible to cover the distance in the the time it's measuring.. 5 bytes at 19,200 baud is about 2.1ms.. I'd have a 1 in 500 chance of it happening which I can live with until I have time for a two way checksum routine.. Event is tomorrow and I have a lot to do.. It'll be on my list for next week!

Thanks