Android Code Snippet [B4X] Audio Streaming UDP

Star-Dust

Expert
Licensed User
Based on this @Erel code that simulated a Walkie Talkie, I developed a system for streaming audio between Android and Desktop devices with UDP.
You can connect with multiple devices.

I found that some routers lose a lot of UDP packets (a high percentage) and the sound is not uniform. If you notice a discontinuous and disturbed sound, it could be the router system.

You can find the version that uses the TCP protocol in this link

Note: The B4J version requires the Audiolorecord library. The B4A requires the Audio library. The B4a version works better if you have headphones (avoid the return of audio) or turn off listening when you speak and vice versa ...
 

Attachments

Last edited:

max123

Active Member
Licensed User
Hi @Star-Dust , I've developed a similar project some time ago, but I use ESP8266 and ESP32 that read audio wave from microsd card and stream it over UDP to Android (B4A) or desktop devices, PC and Raspberry Pi (B4J) or another ESP8266 with external i2S audio DAC. Tried it not only in local network but even over internet from one city to other city and works pretty well. I will release the full (Arduino IDE, B4A, B4J) projects next days.

I want suggest that you can receive the UDP stream even with VLC or Netcat, with last on Linux you can pass directly the audio stream to the linux ALSA driver and then use AudioJack to route it to other applications and even use audio plugins to process it in realtime, like equalizer, reverb, delay and so... or just to record it eg. with Audacity, Ardour etc...

On linux using VLC to receive the audio stream I use the follow command on terminal that start an UDP server that listen to:
vlc --demux=rawaud --rawaud-channels=2 --rawaud-samplerate=44100 udp://@:8266
I've used port 8266, you can chànge it and other parameters as you need based on the trasmitter

Using Netcat the follow command pass the received audio stream to the linux ALSA driver to be routed to other applications using AudioJack or even to PulseAudio:
nc -u -p 8266 --l | play -t raw -r 44100 -b 16 -c 2 -e signed-integer -

The ESP8266 receiver as I wrote, receives the audio stream and send to external audio DAC, then the audio will go to headphones or RCA LR connected to the amplifer, as DAC I've used the Raspberry Pimoroni PHAT DAC, the ESP8266 i2S API permits max 16 bits audio (so no 24 bits), I will try next days on ESP32 that have a better i2S API and support 2 I2s ports with configurable pins and a 24 bits audio up 192Khz, I think this not a problem because I wrote an audio library for ESP8266 and ESP32, for the first tested and works well, for the last I just need to test by connecting the DAC.

My code can stream up 192Khz 16 bit Stereo audio, but the max quality I've tried successfully is 96Khz 16 bits Stereo that is high quality audio too, I think this is a limitation of ESP8266 that read the audio from microsd and not anyway referred to the network bandwith.

In my implementation one device work as trasmitter and other as receiver, not fullduplex, an interesting solution is to use both on both side, like a walkie-talkie or network audio call, on this direction 'I wrote a fast hardware optimized library for SSD1331 (96x64) RGB color small 0.95 inch oled for ESP8266 and ESP32 capable to play videos with very high framerate, up 140 FPS without audio or up 24 FPS with 44100 16 bits audio (with DAC attached) and using ESP32-CAM, i2S DAC, i2S mic etc... I want to implement an UDP video call (Audio + Video) over network between two ESPs or ESP-Desktop devices.

I noticed you warn up users for some losed packets using UDP protocol, from my tests the problem was using a broadcast address to stream to all devices, using an unicastast address I noticed that (at least on my network using FritzBox router) there is a very small amount of missed packets, like 1 missed packet every 30.000 packets, not bad.

I do not use B4R, it is installed but I use very old PCs to develop, with Win XP 😉😉 and I've problems with it, any time I compile it recompile all the code without use a temporary file, so require very long time like the first time I compile on Arduino IDE, after this my systems are 32 bits so I cannot install anyway latest releases of the IDE. Maybe when I release my C++ code for ESP8266 and ESP32 someone interested to port it to B4R.

Please try my suggestion with VLC and Netcat if you use Linux and have some time, even on Raspberry will work, and inform me if you have success with it.

Regards
Massimo from Italy
 
Last edited:

embedded

Member
Licensed User
Hi @Star-Dust , I've developed a similar project some time ago, but I use ESP8266 and ESP32 that read audio wave from microsd card and stream it over UDP to Android (B4A) or desktop devices, PC and Raspberry Pi (B4J) or another ESP8266 with external i2S audio DAC. Tried it not only in local network but even over internet from one city to other city and works pretty well. I will release the full (Arduino IDE, B4A, B4J) projects next days.

I wànt suggest that you can receive the UDP stream even with VLC or Netcat, with last on Linux you can pass directly the audio stream to the linux ALSA driver and then use AudioJack to route it to other applications and even use audio plugins to process it in realtime, like equalizer, reverb, delay and so... or just to record it eg. with Audacity, Ardour etc...

On linux using VLC to receive the audio stream I use the follow command on terminal that start an UDP server that listen to:
vlc --demux=rawaud --rawaud-channels=2 --rawaud-samplerate=44100 udp://@:8266
I've used port 8266, you can chànge it and other parameters as you need based on the trasmitter

Using Netcat the follow command pass the received audio stream to the linux ALSA driver to be routed to other applications using AudioJack or even to PulseAudio:
nc -u -p 8266 --l | play -t raw -r 44100 -b 16 -c 2 -e signed-integer -

The ESP8266 receiver as I wrote, receives the audio stream and send to external audio DAC, then the audio will go to headphones or RCA LR connected to the amplifer, as DAC I've used the Raspberry Pimoroni PHAT DAC, the ESP8266 i2S API permits max 16 bits audio (so no 24 bits), I will try next days on ESP32 that have a better i2S API and support 2 I2s ports with configurable pins and a 24 bits audio up 192Khz, I think this not a problem because I've wrote an audio library for ESP8266 and ESP32, for the first tested and works well, for the last I just need to test by connecting the DAC.

My code can stream up 192Khz 16 bit Stereo audio, but the max quality I've tried successfully is 96Khz 16 bits Stereo that is high quality audio too, I think this is a limitation of ESP8266 that read the audio from microsd and not anyway referred to the network bandwith.

In my implementation one device work as trasmitter and other as receiver, not fullduplex, an interesting solution is to use both on both side, like a walkie-talkie or network audio call, on this direction I've wrote a fast hardware optimized library for SSD1331 (96x64) RGB color small 0.95 inch oled for ESP8266 and ESP32 capable to play videos with very high framerate, up 140 FPS without audio or up 24 FPS with 44100 16 bits audio (with DAC attached) and using ESP32-CAM, i2S DAC, i2S mic etc... I want to implement an UDP video call (Audio + Video) over network between two ESPs or ESP-Desktop devices.

I noticed you warn up users for some missed packets using UDP protocol, from my tests the problem was using a broadcast address to stream to all devices, using an unicastast address I noticed that (at least on my network using FritzBox router) there is a very small amount of missed packets, like 1 missed packet every 30.000 packets, not bad.

I do not use B4R, it is installed but I use very old PCs to develop, with Win XP 😉😉 and I've problems with it, any time I compile it recompile all the code without use a temporary file, so require very long time like the first time I compile on Arduino IDE, after this my systems are 32 bits so I cannot install anyway latest releases of the IDE. Maybe when I release my C++ code for ESP8266 and ESP32 someone interested to port it to B4R.

Please try my suggestion with VLC and Netcat if you use Linux and have some time, even on Raspberry will work, and inform me if you have success with it.

Regards
Massimo from Italy
Please post your project in forum...it will help us... waiting for your release...With love from India
 

Herbert32

Member
Licensed User
I noticed you warn up users for some missed packets using UDP protocol, from my tests the problem was using a broadcast address to stream to all devices, using an unicastast address I noticed that (at least on my network using FritzBox router) there is a very small amount of missed packets, like 1 missed packet every 30.000 packets, not bad.
I made similar experiences:

UDP-Broadcast Wifi/Wifi (both parties are connected over Wifi) : up to 30% Packet-loss
UDP-Broadcast Wifi/LAN (sender connected by Wifi, receiver connected by LAN) : max 1% Packet-loss
direct UDP-Transmission (Wifi/Wifi or Wifi/LAN) : max 0.01% Packet-loss

@max123 - waiting for your Project :cool: - sounds cool
 

max123

Active Member
Licensed User
For user interested to my ESP32 UDP Network Video Call project here you can find my oled library in action, it is a very small 65.536 color oled, good to make an ESP based smartwatch. These video show a very old release of my library, the first one show a small video up 30 fps that is limited because ESP read consecutive JPEG images (one by one) from flash memory. Now I've developed with B4J two encoders (RawVideoBitmap and RawVideoJpeg) capable to read a series of images and pack inside a single file in a way that microcontrollers can read frames as fast possible and (as I wrote) can reach a very high framerate (up 140 fps depending on video size and format) and can play from microsd very long videos, many hours without interruption depending on microsd card size.

I will release for free files for this project when I finish to develop it, even the video encoders compatible with my oled library and my libraries on github.

Sorry, now I cannot write other infos because I don't want going off-topics of this post, I've posted it just because I managed with UDP audio streaming as original post and some users ask for a complete audio-video solution over network.

 
Last edited:
Top