Android Question asyncstreams byte received order

[mterveen]

i have a service that sends eight bytes to a microcontroller one byte at a time through a usb ftdi cable. each byte sent is returned with a value. so one byte in, one byte out. the bytes are sent using asyncstreams with a timer set to send one byte every 5 mSecs. i sporadically have the bytes come back out of order.

i attached my logic analyzer to the tx/rx lines going into the microcontroller and can verify that the bytes going in and coming back out are in the correct order. i currently have the bytes returned captured as follows for testing purposes:

Sub astreams_NewData (Buffer() As Byte)
    ecutrace.Add("rec time:" & DateTime.Now)
    ecutrace.AddAll(Buffer)
end sub

ecutrace is a list. when i print out the list it shows the bytes being captured out of order. 95% of the time they are correct but it obviously needs to be 100% correct. sometimes the buffer array contains 1 byte and other times multiple bytes, which makes sense due to the threading and ability of the thread to respond to the bytes coming in.

1) any ideas on what is happening?
2) does the newdata sub ALWAYS fully complete before being fired again?
3) How big is the "internal" buffer used by the stream?


i am able to get it to work if i only process one byte at a time and wait for it to come back before sending the next byte. unfortunately it is dreadfully slow.

 

[Beja]

Hi,
Must confess I didn't get it completely.. but do you think it is a good idea to use acknowledge and confirmation
techniques, instead of relying on a timer for the sync?

 

[mterveen]

the timer isn't the sync. all the timer does is feed the microcontroller every 5mSecs with new data. i am attempting to negate the usb latency built into the ftdi / android operating system. otherwise the delay is around 20 to 30 mSecs to process one byte. so since asyncstreams uses a multithreading concept and the microcontroller can respond with the data in less than 1.5 mSecs, there should be no reason to not continue to feed the microcontroller with data. i accumulate the data returned by asyncstreams in a list and when the correct number of bytes have been returned i process the list.

i did notice that when the bytes get out of order it appears that they are just shifted down by one and the last byte received actually is first in the list. e.g. if i am expecting 50, 60, 70, 80 to be returned they will come back as 80, 50, 60, 70. weird.

 

[KitCarlson]

I think you may have found the solution to time travel.

 

[Beja]

I provide for noise, bad cable and propagation delays in both sides, not just rely on the processing time of
the mCrocontroller, so I would first, for troubleshooting, increase the delay time from 5 mSecs to 20, then come down until
the xmission is good, then add 10% tolerance.
Better way.. check for any noise and other delays source and filter it out.
As I said I am not totally in the picture, but can ask if you are using CRC16.

 

[Tom Christman]

Have a look at Library Class AsyncStreamsText. The data is transferred as text, but hasn't the problems that asyncstreams byte has with byte order.

 

[mterveen]

i don't have control over the microcontroller program so it's basically byte in, value out. since each byte in can have a byte out from 0 to 255 its hard to filter bad data out other than the number of bytes returned. i'll increase the delay to see what happens and also look at the asyncstreamstext class. its just strange that the bytes can get out of order after the microcontroller. i thought perhaps it might be the ftdi cable but i have a parallel program that runs on windows that runs flawlessly at a much higher rate, i.e. a byte every 1 to 2 mSecs. the only thing i can think of on the ftdi cable side would be that it is powered by the usb port. maybe the devices i use for testing (Galaxy S4 and Nexus 10) don't have enough power to drive it reliably vs a PC usb port at that rate. although it doesn't seem logical since the correct bytes come back, just out of sequence! Wonder if the list.addall(buffer()) command is flawed some how. i'll look at processing the returned bytes using list.add(buffer(x)) instead.

 

[Beja]

another troubleshooting try to read the first byte only and then exit. see what happens.

 

[Erel]

Not sure whether it was mentioned above. Are you showing a Msgbox (or any other modal dialog)? Modal dialogs and DoEvents will cause the data to be out of order.

See AsyncStreams tutorial.

 

[mterveen]

-no modal dialogs during the logging process. just a start/stop button. as the service logs the data it stores the retrieved "sample" in a global variable that is accessed by separate routines outside the service.
-no doevents in the service that runs the logging routines. HOWEVER, i do have 2 other timers that are started after the service is started that run in the calling activity that 1) captures the data sample to a "log" file (e.g every 100mSecs) and 2) updates the screen display (e.g every 250 mSecs). the screen update routine does have 1 doevents command in it. would this interfere with the service thread that does the logging?

 

[Erel]

The DoEvents call in the activity can change the messages order.

 

[mterveen]

i'll comment out the doevents in the screen draw routine and see if that helps. DoEvents appears to be an allpowerful command!!! i'm surprised it would affect services and background threads outside the main activity thread.

Thanks for your input. i'll report back.

 

[Erel]

Services do not run in any other thread. AsyncStreams uses the internal message queue to pass messages read with the help of a background thread to the main thread. Calling DoEvents (or showing a modal dialog) causes the internal message queue to be processed slightly differently.

 

[mterveen]

IT WORKED. thankyou thankyou thankyou. the single doevent i had in the program messed it up.

in summary ---

logging activity
   global loggedsample
   global logtimer
   global displaytimer

   starts service that does logging and stores data to loggedsample
   logtimer captures sample into a list that is eventually written to a file
   displaytimer draws gauges with most recent loggedsample data.

i had the doevents in the displaytimer sub routine after all the graphs had been drawn. by deleting that doevent the bytes captured in the service no longer reported back out of order. since i was using panel.invalidate after i drew the gauges anyhow the doevents wasn't really needed to begin with.

WHEW.