Android Question BLE2 DataAvailable buffer truncated after 20 bytes

[divinglog]

Hello

I have the same problem posted here:

https://www.b4x.com/android/forum/threads/ble2-library-20-byte-notification-limit.100074/

I have a hardware device that sends more than 20 bytes via notifications, but the data is truncated after 20 bytes and no other event is raised containing the remaing bytes. Is there any way to retrieve the remaing bytes?

Thank you!

 

[divinglog]

Every time I post a question, I find the answer shortly after . I have not tested it yet, but looks like this is the solution:

https://www.b4x.com/android/forum/t...mtu-in-ble-application-solved.101722/#content

 

[Frank.G]

Every time I post a question, I find the answer shortly after . I have not tested it yet, but looks like this is the solution:

https://www.b4x.com/android/forum/t...mtu-in-ble-application-solved.101722/#content

Happened to me the last two tiimes as well. I am glad that th[erel] is so much patience with us noobs.

Greetz
Frank

 

[divinglog]

Yes, I've really searched a lot, but if you don't know the exact search term, it is difficult. In the end I used google to find the term "MTU" to increase the packet size and then I quickly found the thread here in the forum.

 

[emexes]

Whenever I hear "20" and "BLE" in same sentence, my first thought is that the HM-10 et al BLE-to-serial modules chunk the serial data into 20-byte blocks.

The second thought, if BLE data is going missing, is: try using indications rather than notifications. Indications try harder to make sure the data gets to the recipient.

The link you refer to mentions increasing the MTU to 247. My first thought there was: good luck with that. I have a vague recollection of maximum BLE packet payload being 37 bytes or thereabouts. Erel's solution used an MTU of 40 and presumably works - there is no confirmation in later posts of success with higher MTU values, so if you are using MTU > 40 and getting errors, try winding back the MTU.

 

[divinglog]

Thank you for the explanation! I've captured the protocol on the PC and it looks like the hardware device is sending packets of 80 bytes maximum (actually 77) and I don't think there is a way to change that behavior. It's probably implemented that way in the device firmware. I don't think I can switch to indications either, because the Gatt descriptor supports only notifications.

 

[emexes]

Thank you for the explanation!

Lol. I feel like I'm stumbling about in the dark a bit too. Many times I find that reality differs from documentation, so (too) much of my programming involves staying well within defined or suspected limits, and... once I find a solution that works, I stick with it. Slow and steady wins the race ;-)

it looks like the hardware device is sending packets of 80 bytes maximum (actually 77)

This rings alarm bells because I vaguely remember that each BLE interchange can have some small number of BLE packets (I thought was 3, but maybe it was 3 additional) and with a 20 byte payload per packet, that would correspond to things getting cut off at some multiple of 20 bytes, if the source serial data is at a high bitrate.

Can you see the BLE chip or module that is being used by your hardware device?

What are the service and characteristic UUIDs being used for this serial-over-BLE interface? (thinking being: are they the same as HM-10 "standard"? in which case the OEM datasheet has a decent blurb about actual data rate limits vs serial bitrates)

 

[divinglog]

What are the service and characteristic UUIDs being used for this serial-over-BLE interface?

Service UUID: fe25c237-0ece-443c-b0aa-e02033e7029d
Characteristic UUID: 27b7570b-359e-45a3-91bb-cf7e70049bd2 (WriteWithoutResponse/Write/Notify)

Can you see the BLE chip or module that is being used by your hardware device?

The problem is that I don't even have the device for testing. I work with customers and debug logs, but I have already implemented this device on Windows, so I see the data protocols and it looks like the packet size is 77 bytes payload. I did not even noticed when implementing this on Windows, because it just worked. But on Android I was struggling the last few days because the data seemed to be truncated and I didn't know why.

The first read is just 18 bytes, so that worked, but the 2nd read is 21 bytes and just 20 bytes arrived.

The test version should be out later today to beta testers, so I will see if it works or not.

 

[emexes]

The problem is that I don't even have the device for testing. I work with customers and debug logs

My respect for your programming skill just skipped up to near to Apollo Guidance Computer level.

The best I've ever achieved like this is an interrupt-driven serial port driver. When it worked first go, I couldn't believe it. Just sat there stunned, watching data come up on screen. But I had hardware. You don't. Yikes!

 

[emexes]

Was the Windows interface also via BLE? Not Bluetooth SPP?

And the 77 bytes arrived in one packet? Or is there perhaps something in the Windows BlLE driver or similar that is reassembling subpackets into a larger packet?

 

[emexes]

This page: https://www.bluetooth.com/blog/exploring-bluetooth-5-how-fast-can-it-be/

indicates that Bluetooth 4.0/4.1 has maximum payload size of 27 bytes (2^5-1-4), and Bluetooth 4.2 increased that to 251 bytes (2^8-1-4).

But even the lower limit of 27 bytes doesn't explain the 20 byte cutoff you're experiencing. Then I found this:

in this document: https://www.bluetooth.com/wp-conten...Range-Higher-speed-and-Increased-Capacity.pdf

so perhaps the limit on notifications is smaller than the limit on data generally. For version 4.2, the corresponding limit is 244 bytes. Both of these limits are 7 bytes smaller than the corresponding payload limits mentioned earlier.

So it might be that Windows supported version 4.2 with the larger limits, and Android (or perhaps the B4X BLE2 library) does not, and thus notifications are maximum of 20 bytes. I'd be poking about to find out if perhaps there are multiple notifications being sent for the one large dive-data packet, but... that's gonna be fun to do without having hardware to hand.

 

[divinglog]

My respect for your programming skill just skipped up to near to Apollo Guidance Computer level.

The best I've ever achieved like this is an interrupt-driven serial port driver. When it worked first go, I couldn't believe it. Just sat there stunned, watching data come up on screen. But I had hardware. You don't. Yikes!

It's even worse, the data is processed in a native C-code library. So I had to write a custom Java library and a JNI C-wrapper for the native library first. And I've never written a single line of Java or C before So there is a lot of code where something could go wrong. But the native library takes care of the communication protocol and data parsing, so all I have to do is collecting and forwarding the data to C-code and sending the commands from C-Code to the device. And I have other hardware devices for testing, but they use just 20 byte packets.

Was the Windows interface also via BLE? Not Bluetooth SPP?

And the 77 bytes arrived in one packet? Or is there perhaps something in the Windows BlLE driver or similar that is reassembling subpackets into a larger packet?

Yes, I think so. I don't know if some magic is happening in the Windows API, but it looks like one packet. I think this is only possible in Bluetooth 4.2 and newer. And good news is, I got positive feedback today from a customer and BluetoothLE.RequestMtu(80) is indeed working

Data Protocol before (not working):

000:01:823   INFO: Write: size=10, data=0100FF010400228010C0
000:01:940   INFO: Read: size=18, data=010001FF0C006280103337304343433033C0
000:01:973   INFO: Write: size=10, data=0100FF010400228011C0
000:02:023   INFO: Read: size=20, data=010001FF0F0062801156373120436C6173736963
000:12:018   INFO: Read: size=0, data=
000:12:020   ERROR: Failed to receive the packet.

And after calling BluetoothLE.RequestMtu(80) (working):

000:01:705   INFO: Write: size=10, data=0100FF010400228010C0
000:01:821   INFO: Read: size=18, data=010001FF0C006280103338303846363339C0
000:01:828   INFO: Write: size=10, data=0100FF010400228011C0
000:01:956   INFO: Read: size=21, data=010001FF0F0062801156373120436C6173736963C0
000:01:960   INFO: Write: size=10, data=0100FF010400228050C0
000:02:042   INFO: Read: size=12, data=010001FF06006280500C0DC0
000:02:057   INFO: Write: size=10, data=0100FF010400228021C0
000:02:139   INFO: Read: size=19, data=010001FF0D00628021018000000000020080C0
000:02:172   INFO: Write: size=17, data=0100FF010B00350034E0000000000600C0
000:02:222   INFO: Read: size=10, data=010001FF0400751082C0
000:02:255   INFO: Write: size=9, data=0100FF0103003601C0
000:02:307   INFO: Read: size=77, data=020001FF83007601A5C4016A5D7CA4DBDD5D7CB3A700000ED1016E0008002C7740002D132000020102A5C401695D72B8765D72C3E200000B7201470007002BE080002C5AA000020102A5C40168
...