Android Question How do I keep adding to a serial buffer until the correct length expected is met?

[mscientist33]

I have a serial device that does not send an end of line character or anything but the length of the data being sent is always suppose to be 22. How do I receive the buffer and then continue to receive from the buffer until the proper length is met? This is a buffer of bytes and the usable data is in bytes not string characters.

 

[emexes]

My way would be to have a buffer of 22 bytes, and in the received-some-bytes routine, add the received bytes to the buffer, and whenever the number of received bytes in the buffer reaches 22, call the handle-received-complete-packet routine, passing it the buffer of 22 bytes.

Does the packet have a leading synch value to help recover from serial line errors?

 

[mscientist33]

Actually, the first byte is a 7E. I am actually getting packets from an xbee radio but I don't get the whole packet through serial at the same time. Would you put the buffer in an array then redim the array as they come in until it reaches 22?

 

[emexes]

I'm not set up to test, but the code would look like this:

sub globals()

    dim PacketReceiveBuffer(22) as byte
    dim PacketReceiveCount as byte = 0

end sub

sub ReceivedSerialData(Incoming() as byte)
   
    for I = 0 to Incoming.Length - 1    'process new bytes, one by one
        if PacketReceiveCount = 0 and Incoming(i) <> 0x7E then
            'ignore line noise between packets - packet should start with 0x7E
        else
            PacketReceiveBuffer(PacketReceiveCount) = Incoming(I)
            PacketReceiveCount = PacketReceiveCount + 1
       
            if PacketReceiveCount = 22 then
                call HandleCompletePacket(PacketReceiveBuffer())
                PacketReceiveCount = 0    'ready for next packet
            end if
        end if
    next I
   
end sub

sub HandleCompletePacket(Packet() as byte)
   
    dim Msg as string = "Received packet (bytes in decimal) ="
   
    for I = 0 to Packet.Length - 1
        Msg = Msg & " " & Packet(I)
    next I
   
    log(Msg)
   
end sub

 

[Erel]

No. No. No.

Never treat raw bytes as string characters.

Use B4XBytesBuilder from B4XCollections to collect the data.

 

[emexes]

Would you put the buffer in an array then redim the array as they come in until it reaches 22?

If you know the length (or maximum length) of the packet / buffer then it is way easier and more efficient to just dim / allocate the array once, rather than for each and every byte or glob of bytes or new packet received. ?

 

[emexes]

Never treat raw bytes as string characters.

Also: never say never.

Like, if the incoming bytes are text eg ASCII, then to assemble them using StringBuilder would seem a reasonable approach.

 

[Erel]

Like, if the incoming bytes are text eg ASCII,

ASCII text is indeed the only case where it is a valid solution. Note that with any of the various unicode encodings a single character (code point) can be encoded with two or more bytes and can theoretically be split into two messages.

B4XBytesBuilder is very similar to StringBuilder. It is quite simple to work with.

 

[mscientist33]

That worked like a charm @emexes . I did take the received data and use B4XBytesBuilder. Now to figure out how to debounce the received data.

 

[emexes]

Now to figure out how to debounce the received data.

If you're talking key/switch contact debouncing, then that should be being done at the other end of the radio link.

You'll possibly save much time by checking first to see that there is indeed a problem to be solved.

Debouncing at the receive end of the radio link also adds new problems eg what about if packets go missing, or if the transmit end only sends when switch state changes?

If you do get involved in debouncing, here is a useful and practical discussion of the topic:

http://www.ganssle.com/debouncing.htm

(which I've just noticed was edited over a ten year period, which should be a clue that the topic is not as simple as it first appears ? )

 

[emexes]

I did take the received data and use B4XBytesBuilder.

Lol, fair enough. A bit of stab to my efficiency-loving heart, but in today's GHz world, you'll probably get away with it. ?

On a related conversational note, the current newsletter by the debouncing author above coincidentally leads with this:

How much abstraction is too much? Phil Matthews ponders this:

Recently I had to work on some embedded software for a PIC24 that would reset another chip by pulsing an I/O pin low for 160usec.

The code I found had a function "reset_chip()", which created a message that was posted on a message bus to a task in a "reset object" that subscribed to the message. The message was dynamically allocated on the heap and linked into a message queue. The message contained a pointer to the message data which was also dynamically allocated on the heap. That "object" had 4-layers of hardware abstraction that ultimately pulled a GPIO pin low. Another timer object also subscribed to the same message and started a timeout with minimum resolution 10msec. The timer object was dynamically created in a linked list of timers. When the timer expired it generated a timer event object that posted a timeout message to a another task that subscribed to the timeout message. This caused the reset object, through 4 layers of hardware abstraction and logical pin mapping, to pull the pin high. The reset pin was low for anywhere between 10-20msec. Also it didn't always work, and sometimes went low and failed to go high (This is why I was working on it) .

I deleted 1500 bytes of code and replaced it with 3 lines of C-code:

Set_gpio_pin(6, low);
Delay_ms(1);
Set_gpio_pin(6, hi);

When I pointed out how ridiculous the original code was, and asked why it was done that way I was told point blank, "It's object oriented". When I asked what problem is being solved by doing it this was the reply was "Everything is object oriented. It's the way software is done".

Admittedly this is an extreme example. However, over the years I have noticed embedded software becoming more complex for no apparent benefit. It looks structured, and follows some academic pattern. But is way too complicated, slow, memory hungry and full of bugs. I don't mean complicated systems. I'm meaning simple systems with unnecessarily complicated software. Why has this come about? Is it software guys becoming more and more distant from hardware?

When I started writing embedded software it had to be crammed into limited hardware. Simplicity was elegance.

I am from the "had to be crammed into limited hardware" school, and (i) old habits die hard, and (ii) still sometimes save butts today. ?

 

[emexes]

Actually, the first byte is a 7E. I am actually getting packets from an xbee radio

If your incoming packets are like the format described here:

https://www.digi.com/resources/docu...1942-13/reference/r_zigbee_frame_examples.htm

then you might want to consider the length bytes at the beginning and the checksum at the end too.

Eg, there might be packets longer or shorter than 22 bytes, and what might be the effect of corrupted data bytes that make it seem as though the (eg) launch or detonate button has been pressed? ?

If your packets are indeed guaranteed to be only 22 bytes long, then the length bytes will always be 0x00 0x12 and thus the first three bytes of every packet will be 0x7E 0x00 0x12 which you can use to add more robustness to your receiving code and reduce the likelihood of synchronising with a random 0x7E.

 

[emexes]

I have a serial device

How's this project going? Are the Zigbee packets getting through ok, or still as elusive as strands of mist? ?

Are you reading electricity meters, by any chance? Most meters here just have a flashing LED eg 1000 flashes per kWh, but the government now has a scheme going to provide homes with free bluetooth electricity monitors, and consequently we're getting random telephone sales calls every day and evening from installers desperate to provide the monitors and collect the government bounty. ?

 

[mscientist33]

How's this project going? Are the Zigbee packets getting through ok, or still as elusive as strands of mist? ?

Are you reading electricity meters, by any chance? Most meters here just have a flashing LED eg 1000 flashes per kWh, but the government now has a scheme going to provide homes with free bluetooth electricity monitors, and consequently we're getting random telephone sales calls every day and evening from installers desperate to provide the monitors and collect the government bounty. ?

It is going fairly well, no issues so far. No, it's not meters. I am using it on a target system that reads hits on a target (up to three targets) and sends them back to a controller.