Android Question [felUSBSerial] Missing data while receiving bytes

emexes

Well-Known Member
Licensed User
I'd hook that 'scope up. At worst, it'll give you something to look at while you mull over what could be causing bytes to go AWOL. At best, it might trigger a random thought that leads to a solution.

RS485 brought back memories about timing issues with the transmit enable - many UARTs generate a transmit (buffer empty) interrupt after the last data bit has been sent, but before/whilst the stop bit is sent, which *shouldn't* be a problem but... if the bus isn't terminated, then perhaps there is a glitch that gets misinterpreted as a start bit, something like that.

It is good that you have software that is apparently not dropping bytes, because this means that what you want to do is indeed actually possible; all we need to do is find out the reason for the difference. How hard can that be?!?! Perhaps the serial terminal programs are more relaxed about framing errors, whereas the felUSBSerial is rejecting such marginal cases and dropping the bytes.
 

FredBerlin

Member
Licensed User
Hi guys,

further testing gave more insights:
1. the missing bytes always come as two bytes in series, in the middle of the Byte() array,
that the message is split into junks has nothing to do with it
2. the RS485 is properly terminated, the signal on the scope is all good
3. if I switch to an older interface (ftdi-usb-chip): no missing bytes at all !!!

To sum up:
1. problems are gone if I switch to an older interface (not an option, since the old one is no longer produced)
2. problems are gone if I switch to Android Studio (which I hate to do, rather would like to stay in B4A)
3. problems may be gone switching from felUSBSerial to USBSerial, but the newer chip isn't supported in USBSerial

At the time being I wonder if I either look into "writing a library for B4A", to move the working code from Android Studio
to B4A, or if that may take longer than coding the whole app in Java (which doesn't attract me that much).

Any suggestions? Further analysis requested? Your opinion on how to proceed further?

Thank you all so far,
best,
Fred
 

agraham

Expert
Licensed User
So your rather thorough analysis seems to point to a bug in the lower level driver part of felUSBSerial when used with a newer FTDI chip :(

If it were me I would look to encapsulate the known working USB side of your Android Studio code as a B4A library. As you already have working code in Android Studio it really should be quite a simple task, much easier than moving the whole app to Java, and as a bonus you would know how to write B4A libraries should you need to in future :)

I don't know much about Android Studio as I used to write my libraries using Eclipse before I gave up using Android a few years ago. So I don't know if Android Studio can produce jar files for use as B4A libraries. Having recently resumed using B4A I have updated and recompiled a couple of my existing libraries and created quite a complex new one using only Notepad++ to edit the code and Erel's Simple Library Compiler to compile it.
 

FredBerlin

Member
Licensed User
That sounds like a plan. Always hesitating to install another huge package (Eclipse), I'm -like you- in favour of small and neat solutions.
Notepad++, JAVA, and Erel's "Simple Library Compiler" may be appealing. I'll find the first two on my own. And Erel's compiler by searching
the forum I guess.
 

FredBerlin

Member
Licensed User
Meanwhile I used a PC and RealTerm to send a text file to felUSBSerial (57600 baud, 8N1).
Here my findings: If I include 500µs idle time between the chars (one can easily do this with RealTerm)
there are no errors at all with felUSBSerial and the new FTDI chip, but removing the idle time and
sending it as a continuous stream, the drop outs (always as pair of two bytes) are back in.
So really appears to be a low-level driver problem. This information may help others in the (near) future.
Those new FTDI chips may definitely be used by a growing number of product suppliers/users.
 

agraham

Expert
Licensed User
As an example of how easy it it to make a library here is a zip of a folder that will compile my Jpeg library. Just unzip it anywhere, navigate to the Jpeg folder in File Explorer and double click CompileJpeg.bat. You will end up with a shiny new library in your Additional Libraries folder.
 

Attachments

FredBerlin

Member
Licensed User
Cool! Thank you so much. That will definitely shorten my development time.
In the end the serial interface will be just a tiny bit of a larger app that will process the data.
The sooner I get to the main project, the better.
 

FredBerlin

Member
Licensed User
There is highly likely something going wrong in felUSBSerial with the new FTDI chip.
While doing further diagnoses to find out when missing bytes start to occur (I inserted less/more idle time between bytes sent) I once a while got this log msg:
New_Try________________________
length: 68
--> Total: 68
length: 75
--> Total: 143
length: 74
--> Total: 217
New_Try________________________
length: 36
--> Total: 36
length: 56
--> Total: 92
length: 56
--> Total: 148
length: 62
--> Total: 210
length: 13
--> Total: 223
New_Try________________________
length: 15
--> Total: 15
java.lang.ArrayIndexOutOfBoundsException: length=61; index=61
at com.felhr.usbserial.FTDISerialDevice$FTDIUtilities.copyData(FTDISerialDevice.java:594)
at com.felhr.usbserial.FTDISerialDevice$FTDIUtilities.adaptArray(FTDISerialDevice.java:502)
at com.felhr.usbserial.UsbSerialDevice$WorkerThread.run(UsbSerialDevice.java:248)
length: 54
--> Total: 69
If I read this correctly, it stems from deep within felUSBSerial. Index problems could also explain missing bytes! So it seems that there's a strange interplay between the new FTDI chip and the current version of felUSBSerial - definitely this chip has to be on the "is not yet supported" list of SerialUSB AND felUSBSerial at the time being. :-(
 

FredBerlin

Member
Licensed User
Now here is the idea: comparing the old FTDI chip and the new one, it pops into the eye that the max. pack. size is 512 instead of 64.
Let's say the chip throws an interrupt a few bytes before the buffer is full (default threshold may be 4), then the old chip delivers chunks
of around 60 bytes max, whereas the new one could in principle go up to 508 bytes.
The crash occurs always when Array-index is 61... go figure.

_________ WORKING ______NOT WORKING
Manufact.:_FTD____________FTDI
Product:___FT2____________USB <-> Serial Converter
Serial:_____AI0____________FT3FSMK2
DeviceID:__0x3EA__________0x3EA
Prod.ID:___0x6001_________0x6014
VendorID:__0x403__________0x403
max. Pack.:_64_____________512
Since this information has been read by the app "USB Device Info" (to be found on F-Droid), the chip directly reports
the needed buffer size (and did this already in the past). So my guess is, that felUSBSerial could be even more versatile
and future-save, if this property is actually read and used for buffer allocation.

Erel, just in case you are in the mood to set the buffer size to 512 within felUSBSerial,
I'd be happy to re-run all tests and report on it.

Meanwhile I look into library writing which may be helpful anyway ;-)

Best,
Fred
 

FredBerlin

Member
Licensed User
It's not my project, but done by Kai Morich. He provided the source code on github.
Opening his project I see:

B4X:
import com.hoho.android.usbserial.driver.CdcAcmSerialDriver;
import com.hoho.android.usbserial.driver.ProbeTable;
import com.hoho.android.usbserial.driver.UsbSerialProber;
See here: https://github.com/kai-morich/SimpleUsbTerminal
 
Top