Android Tutorial [B4X] Simple audio call over internet with own TURN/relay server

Biswajit

Active Member
Licensed User
Hi everyone,

I posted a thread long ago about the audio/video calling over the internet. Here is a simple example of how you can call almost (will discuss below) realtime.

Why only a simple audio calling example while I said that I will post both audio and video calling examples?
While working on this project I realized that a good quality audio/video calling application with all required functionalities needs huge time and effort. Because like whatsapp, google duo if you want faster calling app then you have to compress the audio/video buffer before sending and decompress the audio/video buffer after receiving. This may sound like a simple byte compression job but it isn't. Compressing/decompressing audio/video buffer in realtime and playing requires lots of research about audio/video processing, audio/video codes, how conversion works, and many other things. Thus I decided to post a simple example of how it works for free and will post a full example that will be chargeable. That will have lots of functionalities, check below to know more.

This will work,
  1. From Android to Android
  2. From Android to iOS
  3. From iOS to Android
  4. From iOS to iOS
Functionalities:
  1. Apps:
    1. List all the connected users and update in realtime
    2. Call any connected users
    3. Check if the user is busy or not
    4. Show custom in-app incoming call screen with accept/reject button
    5. Play ringtone for incoming call
    6. Automatic screen dimming while talking
    7. Call accepted/rejected/ended/busy/offline status
    8. Call timer
    9. Incoming call notification while the app is in the background (Android)
    10. Noise cancellation, Acoustic echo cancellation, Automatic gain control (Android)
  2. Server:
    1. Can handle multiple connections simultaneously.
    2. Deliver call requests and status
    3. Broadcast connected user list
    4. Check if the callee is busy or not
    5. Work as a relay server for delivering audio buffers to respective clients
Limitations:
  1. Few milliseconds delay in delivering audio buffer (sometimes). Because,
    1. The example apps are processing uncompressed audio buffer (24Kb-30Kb buffer per seconds)
    2. The test server (you can find the IP inside the project) has been configured to limit bandwidth usage for testing purposes.
  2. No notifications when the app is closed. I haven't included that because it's a simple example.
How to run:
  1. App:
    1. Download latest socket.io client library wrapper (B4A / B4I)
    2. Download, extract and run.
    3. For ios, you need a working developer certificate because it will not work on the simulator.
  2. Server: The test server is always up. If you want to test with the test server then just run the app. If you want to run it on your local server, then
    1. Install Node.js
    2. Extract the server code
    3. Open terminal/cmd and navigate to the folder where you have extracted the code
    4. Run npm install (it will take few minutes)
    5. Run node app.js
    6. That's it.
Functionalities that the full example will have,
  1. Audio/Video call
  2. End to end encryption
  3. Switching audio output between speakers/headphone
  4. Faster calling even with low internet speed (5-10kbps)
  5. Incoming calling screen that will show even when the app is not running.
  6. Register a name for your device. So the other users can see who is calling
  7. Example of STUN server as default server and fallback to TURN server
  8. Example of Socket.IO server along with WebSocket Server
  9. *Maybe group calling.
The full example will be chargeable. I haven't decided the amount yet. It will be between $50 - $100.

Though the test server has bandwidth limitations, it can handle more than 100-200 users.

Download the test apk for testing.
 

Attachments

Last edited:

MarcoRome

Expert
Licensed User
Super job. Waiting for an Audio / Video demo. If everything is ok, I will certainly make the purchase.
At the moment if you use a wi-fi it gives problems (the voice comes with a flickering after several seconds), on 4G sometimes it is very fluid, other times there is a delay of up to 3-4 seconds
 

Biswajit

Active Member
Licensed User
Super job. Waiting for an Audio / Video demo. If everything is ok, I will certainly make the purchase.
At the moment if you use a wi-fi it gives problems (the voice comes with a flickering after several seconds), on 4G sometimes it is very fluid, other times there is a delay of up to 3-4 seconds
Yes, a delay is there. Its because of the test server bandwidth limitation and the unprocessed raw PCM buffer. It will be smoother with compressed buffer.
 

mehdipass

Member
hi,
I will test your example on my server but it gives an error.
B4X:
io.socket.engineio.client.EngineIOException: xhr poll error
why?
 

Ertan

Active Member
Licensed User
Hi,

i testing your project, i run the application and the application crashes while connecting to the server.


B4X:
Günlükçü şuna bağlandı: 11160b24ac4a3502
--------- beginning of crash
--------- beginning of main
*** Service (starter) Create ***
** Service (starter) Start **
Connecting...
** Activity (main) Create, isFirst = true **
** Activity (main) Resume **
** Activity (main) Pause, UserClosed = false **
Connection established
sending message to waiting queue (CallSubDelayed - update_user_list)
sending message to waiting queue (activity_permissionresult)
running waiting messages (2)
** Activity (main) Resume **
** Activity (main) Pause, UserClosed = false **
--------- beginning of system
*** Service (starter) Create ***
** Service (starter) Start **
Connecting...
** Activity (main) Create, isFirst = true **
** Activity (main) Resume **
io.socket.engineio.client.EngineIOException: xhr poll error
** Activity (main) Pause, UserClosed = false **
Tring to Reconnect
Reconnecting
io.socket.engineio.client.EngineIOException: xhr poll error
Reconnection Error
sending message to waiting queue (activity_permissionresult)
running waiting messages (1)
** Activity (main) Resume **
Tring to Reconnect
main_update_status (java line: 526)
android.view.ViewRootImpl$CalledFromWrongThreadException: Only the original thread that created a view hierarchy can touch its views.
    at android.view.ViewRootImpl.checkThread(ViewRootImpl.java:7816)
    at android.view.ViewRootImpl.invalidateChildInParent(ViewRootImpl.java:1345)
    at android.view.ViewGroup.invalidateChild(ViewGroup.java:5446)
    at android.view.View.invalidateInternal(View.java:14755)
    at android.view.View.invalidate(View.java:14719)
    at android.view.View.invalidate(View.java:14703)
    at android.widget.TextView.checkForRelayout(TextView.java:8536)
    at android.widget.TextView.setText(TextView.java:5083)
    at android.widget.TextView.setText(TextView.java:4908)
    at android.widget.TextView.setText(TextView.java:4883)
    at anywheresoftware.b4a.objects.TextViewWrapper.setText(TextViewWrapper.java:39)
    at b4a.example.main._update_status(main.java:526)
    at java.lang.reflect.Method.invoke(Native Method)
    at anywheresoftware.b4a.BA.raiseEvent2(BA.java:213)
    at anywheresoftware.b4a.keywords.Common.CallSub4(Common.java:1082)
    at anywheresoftware.b4a.keywords.Common.CallSubNew2(Common.java:1037)
    at b4a.example.starter._socket_onreconnecting(starter.java:308)
    at java.lang.reflect.Method.invoke(Native Method)
    at anywheresoftware.b4a.BA.raiseEvent2(BA.java:213)
    at anywheresoftware.b4a.BA.raiseEvent(BA.java:193)
    at com.biswajit.socketio.SocketIO$8.call(SocketIO.java:151)
    at io.socket.emitter.Emitter.emit(Emitter.java:117)
    at io.socket.client.Socket.access$601(Socket.java:19)
    at io.socket.client.Socket$5.run(Socket.java:177)
    at io.socket.thread.EventThread.exec(EventThread.java:55)
    at io.socket.client.Socket.emit(Socket.java:173)
    at io.socket.client.Manager.emitAll(Manager.java:156)
    at io.socket.client.Manager.access$600(Manager.java:20)
    at io.socket.client.Manager$11$1.run(Manager.java:555)
    at io.socket.thread.EventThread$2.run(EventThread.java:80)
    at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1133)
    at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:607)
    at java.lang.Thread.run(Thread.java:762)
** Activity (main) Pause, UserClosed = true **
Screenshot_3.png
 
Last edited:

Biswajit

Active Member
Licensed User
hi,
I will test your example on my server but it gives an error.
B4X:
io.socket.engineio.client.EngineIOException: xhr poll error
why?
This means your app wasn't able to connect to the server. Are you testing this with your own local/remote server? or the test server?
 

Biswajit

Active Member
Licensed User
Hi,

i testing your project, i run the application and the application crashes while connecting to the server.


B4X:
Günlükçü şuna bağlandı: 11160b24ac4a3502
--------- beginning of crash
--------- beginning of main
*** Service (starter) Create ***
** Service (starter) Start **
Connecting...
** Activity (main) Create, isFirst = true **
** Activity (main) Resume **
** Activity (main) Pause, UserClosed = false **
Connection established
sending message to waiting queue (CallSubDelayed - update_user_list)
sending message to waiting queue (activity_permissionresult)
running waiting messages (2)
** Activity (main) Resume **
** Activity (main) Pause, UserClosed = false **
--------- beginning of system
*** Service (starter) Create ***
** Service (starter) Start **
Connecting...
** Activity (main) Create, isFirst = true **
** Activity (main) Resume **
io.socket.engineio.client.EngineIOException: xhr poll error
** Activity (main) Pause, UserClosed = false **
Tring to Reconnect
Reconnecting
io.socket.engineio.client.EngineIOException: xhr poll error
Reconnection Error
sending message to waiting queue (activity_permissionresult)
running waiting messages (1)
** Activity (main) Resume **
Tring to Reconnect
main_update_status (java line: 526)
android.view.ViewRootImpl$CalledFromWrongThreadException: Only the original thread that created a view hierarchy can touch its views.
    at android.view.ViewRootImpl.checkThread(ViewRootImpl.java:7816)
    at android.view.ViewRootImpl.invalidateChildInParent(ViewRootImpl.java:1345)
    at android.view.ViewGroup.invalidateChild(ViewGroup.java:5446)
    at android.view.View.invalidateInternal(View.java:14755)
    at android.view.View.invalidate(View.java:14719)
    at android.view.View.invalidate(View.java:14703)
    at android.widget.TextView.checkForRelayout(TextView.java:8536)
    at android.widget.TextView.setText(TextView.java:5083)
    at android.widget.TextView.setText(TextView.java:4908)
    at android.widget.TextView.setText(TextView.java:4883)
    at anywheresoftware.b4a.objects.TextViewWrapper.setText(TextViewWrapper.java:39)
    at b4a.example.main._update_status(main.java:526)
    at java.lang.reflect.Method.invoke(Native Method)
    at anywheresoftware.b4a.BA.raiseEvent2(BA.java:213)
    at anywheresoftware.b4a.keywords.Common.CallSub4(Common.java:1082)
    at anywheresoftware.b4a.keywords.Common.CallSubNew2(Common.java:1037)
    at b4a.example.starter._socket_onreconnecting(starter.java:308)
    at java.lang.reflect.Method.invoke(Native Method)
    at anywheresoftware.b4a.BA.raiseEvent2(BA.java:213)
    at anywheresoftware.b4a.BA.raiseEvent(BA.java:193)
    at com.biswajit.socketio.SocketIO$8.call(SocketIO.java:151)
    at io.socket.emitter.Emitter.emit(Emitter.java:117)
    at io.socket.client.Socket.access$601(Socket.java:19)
    at io.socket.client.Socket$5.run(Socket.java:177)
    at io.socket.thread.EventThread.exec(EventThread.java:55)
    at io.socket.client.Socket.emit(Socket.java:173)
    at io.socket.client.Manager.emitAll(Manager.java:156)
    at io.socket.client.Manager.access$600(Manager.java:20)
    at io.socket.client.Manager$11$1.run(Manager.java:555)
    at io.socket.thread.EventThread$2.run(EventThread.java:80)
    at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1133)
    at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:607)
    at java.lang.Thread.run(Thread.java:762)
** Activity (main) Pause, UserClosed = true **
View attachment 112197
This means the socket.io client process is running on different thread. Only the main thread of the application can control any view modification. I tested the code its working fine. Did you changed any code or running the code as it is?
 

Ertan

Active Member
Licensed User
This means the socket.io client process is running on different thread. Only the main thread of the application can control any view modification. I tested the code its working fine. Did you changed any code or running the code as it is?
I haven't touched any code. I just wrote the local ip address of my server in the mobile application.
  1. Install Node.js
  2. Extract the server code
  3. Open terminal/cmd and navigate to the folder where you have extracted the code
  4. Run npm install (it will take few minutes)
  5. Run node app.js
  6. That's it.
I performed these steps.
I updated the IP address in the application and uploaded it to my phone. I got the same error.
 

Biswajit

Active Member
Licensed User
I haven't touched any code. I just wrote the local ip address of my server in the mobile application.
  1. Install Node.js
  2. Extract the server code
  3. Open terminal/cmd and navigate to the folder where you have extracted the code
  4. Run npm install (it will take few minutes)
  5. Run node app.js
  6. That's it.
I performed these steps.
I updated the IP address in the application and uploaded it to my phone. I got the same error.
Ok. I will check the code again. What is your b4a version?
 

PassionDEV

Well-Known Member
Licensed User
I have tested this project. The echo canceler that you have set within the source code is not effective on most Android devices tested on Android 7,8,11
The echo is a bottleneck for any duplex chat platform
 

Ertan

Active Member
Licensed User
Ok. I will check the code again. What is your b4a version?
I found the solution to the problem.

After saying "npm install" and installing it will give you a warning.

Screenshot_4.png


If you say "npm audit fix --force" as it said, it will make a few changes in your project. These changes will cause your project to not work properly. So I ignored this warning and started the project by typing "node app.js" directly. I was able to perform my test without any trouble.

But..

1) Two-sided audio departure is very delayed (3-4 seconds) sometimes it doesn't even go away. What is the reason of this? How can we avoid this situation?
2) On my Android 7.0 phone, the project is running in Debug mode, but in Release mode my project crashes and gives an error.

This is not a problem on my Android 10 phone.

Crash:

B4X:
--------- beginning of system
*** Service (starter) Create ***
** Service (starter) Start **
Connecting...
** Activity (main) Create, isFirst = true **
** Activity (main) Resume **
io.socket.engineio.client.EngineIOException: xhr poll error
Tring to Reconnect
main_update_status (java line: 484)
android.view.ViewRootImpl$CalledFromWrongThreadException: Only the original thread that created a view hierarchy can touch its views.
    at android.view.ViewRootImpl.checkThread(ViewRootImpl.java:7816)
    at android.view.ViewRootImpl.invalidateChildInParent(ViewRootImpl.java:1345)
    at android.view.ViewGroup.invalidateChild(ViewGroup.java:5446)
    at android.view.View.invalidateInternal(View.java:14755)
    at android.view.View.invalidate(View.java:14719)
    at android.view.View.invalidate(View.java:14703)
    at android.widget.TextView.checkForRelayout(TextView.java:8536)
    at android.widget.TextView.setText(TextView.java:5083)
    at android.widget.TextView.setText(TextView.java:4908)
    at android.widget.TextView.setText(TextView.java:4883)
    at anywheresoftware.b4a.objects.TextViewWrapper.setText(TextViewWrapper.java:39)
    at b4a.example.main._update_status(main.java:484)
    at java.lang.reflect.Method.invoke(Native Method)
    at anywheresoftware.b4a.BA.raiseEvent2(BA.java:213)
    at anywheresoftware.b4a.keywords.Common.CallSub4(Common.java:1082)
    at anywheresoftware.b4a.keywords.Common.CallSubNew2(Common.java:1037)
    at b4a.example.starter._socket_onreconnecting(starter.java:308)
    at java.lang.reflect.Method.invoke(Native Method)
    at anywheresoftware.b4a.BA.raiseEvent2(BA.java:213)
    at anywheresoftware.b4a.BA.raiseEvent(BA.java:193)
    at com.biswajit.socketio.SocketIO$8.call(SocketIO.java:151)
    at io.socket.emitter.Emitter.emit(Emitter.java:117)
    at io.socket.client.Socket.access$601(Socket.java:19)
    at io.socket.client.Socket$5.run(Socket.java:177)
    at io.socket.thread.EventThread.exec(EventThread.java:55)
    at io.socket.client.Socket.emit(Socket.java:173)
    at io.socket.client.Manager.emitAll(Manager.java:156)
    at io.socket.client.Manager.access$600(Manager.java:20)
    at io.socket.client.Manager$11$1.run(Manager.java:555)
    at io.socket.thread.EventThread$2.run(EventThread.java:80)
    at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1133)
    at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:607)
    at java.lang.Thread.run(Thread.java:762)
** Activity (main) Pause, UserClosed = true **
 

PassionDEV

Well-Known Member
Licensed User
Does this matter on a local server? Unfortunately, I do not have much information on these matters. I'm asking to find out.
If this a localhost no it doesn't matter to encode or lower the size of the audio packet.
 

Ertan

Active Member
Licensed User
Does anyone have an idea to solve this problem?
I found the solution to the problem.

After saying "npm install" and installing it will give you a warning.

View attachment 112310

If you say "npm audit fix --force" as it said, it will make a few changes in your project. These changes will cause your project to not work properly. So I ignored this warning and started the project by typing "node app.js" directly. I was able to perform my test without any trouble.
 

Biswajit

Active Member
Licensed User
I have tested this project. The echo canceler that you have set within the source code is not effective on most Android devices tested on Android 7,8,11
The echo is a bottleneck for any duplex chat platform
You can lower your bitrate or even encode your audio for bandwidth saving before sending.
All the possible fixes will be posted as a paid version. I am down due to covid19. Once I feel good I will continue that paid version.
 
Top