Android Tutorial [B4X] Simple audio call over internet with own TURN/relay server

MarcoRome · Mar 15, 2021

Super job. Waiting for an Audio / Video demo. If everything is ok, I will certainly make the purchase.
At the moment if you use a wi-fi it gives problems (the voice comes with a flickering after several seconds), on 4G sometimes it is very fluid, other times there is a delay of up to 3-4 seconds

Biswajit · Mar 15, 2021

MarcoRome said:
Super job. Waiting for an Audio / Video demo. If everything is ok, I will certainly make the purchase.
At the moment if you use a wi-fi it gives problems (the voice comes with a flickering after several seconds), on 4G sometimes it is very fluid, other times there is a delay of up to 3-4 seconds

Yes, a delay is there. Its because of the test server bandwidth limitation and the unprocessed raw PCM buffer. It will be smoother with compressed buffer.

Ertan · Apr 24, 2021

Hi,

i testing your project, i run the application and the application crashes while connecting to the server.

B4X:

Günlükçü şuna bağlandı: 11160b24ac4a3502
--------- beginning of crash
--------- beginning of main
*** Service (starter) Create ***
** Service (starter) Start **
Connecting...
** Activity (main) Create, isFirst = true **
** Activity (main) Resume **
** Activity (main) Pause, UserClosed = false **
Connection established
sending message to waiting queue (CallSubDelayed - update_user_list)
sending message to waiting queue (activity_permissionresult)
running waiting messages (2)
** Activity (main) Resume **
** Activity (main) Pause, UserClosed = false **
--------- beginning of system
*** Service (starter) Create ***
** Service (starter) Start **
Connecting...
** Activity (main) Create, isFirst = true **
** Activity (main) Resume **
io.socket.engineio.client.EngineIOException: xhr poll error
** Activity (main) Pause, UserClosed = false **
Tring to Reconnect
Reconnecting
io.socket.engineio.client.EngineIOException: xhr poll error
Reconnection Error
sending message to waiting queue (activity_permissionresult)
running waiting messages (1)
** Activity (main) Resume **
Tring to Reconnect
main_update_status (java line: 526)
android.view.ViewRootImpl$CalledFromWrongThreadException: Only the original thread that created a view hierarchy can touch its views.
    at android.view.ViewRootImpl.checkThread(ViewRootImpl.java:7816)
    at android.view.ViewRootImpl.invalidateChildInParent(ViewRootImpl.java:1345)
    at android.view.ViewGroup.invalidateChild(ViewGroup.java:5446)
    at android.view.View.invalidateInternal(View.java:14755)
    at android.view.View.invalidate(View.java:14719)
    at android.view.View.invalidate(View.java:14703)
    at android.widget.TextView.checkForRelayout(TextView.java:8536)
    at android.widget.TextView.setText(TextView.java:5083)
    at android.widget.TextView.setText(TextView.java:4908)
    at android.widget.TextView.setText(TextView.java:4883)
    at anywheresoftware.b4a.objects.TextViewWrapper.setText(TextViewWrapper.java:39)
    at b4a.example.main._update_status(main.java:526)
    at java.lang.reflect.Method.invoke(Native Method)
    at anywheresoftware.b4a.BA.raiseEvent2(BA.java:213)
    at anywheresoftware.b4a.keywords.Common.CallSub4(Common.java:1082)
    at anywheresoftware.b4a.keywords.Common.CallSubNew2(Common.java:1037)
    at b4a.example.starter._socket_onreconnecting(starter.java:308)
    at java.lang.reflect.Method.invoke(Native Method)
    at anywheresoftware.b4a.BA.raiseEvent2(BA.java:213)
    at anywheresoftware.b4a.BA.raiseEvent(BA.java:193)
    at com.biswajit.socketio.SocketIO$8.call(SocketIO.java:151)
    at io.socket.emitter.Emitter.emit(Emitter.java:117)
    at io.socket.client.Socket.access$601(Socket.java:19)
    at io.socket.client.Socket$5.run(Socket.java:177)
    at io.socket.thread.EventThread.exec(EventThread.java:55)
    at io.socket.client.Socket.emit(Socket.java:173)
    at io.socket.client.Manager.emitAll(Manager.java:156)
    at io.socket.client.Manager.access$600(Manager.java:20)
    at io.socket.client.Manager$11$1.run(Manager.java:555)
    at io.socket.thread.EventThread$2.run(EventThread.java:80)
    at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1133)
    at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:607)
    at java.lang.Thread.run(Thread.java:762)
** Activity (main) Pause, UserClosed = true **

Biswajit · Apr 24, 2021

mehdipass said:
hi,
I will test your example on my server but it gives an error.

B4X:

io.socket.engineio.client.EngineIOException: xhr poll error

why?

This means your app wasn't able to connect to the server. Are you testing this with your own local/remote server? or the test server?

Biswajit · Apr 24, 2021

Ertan said:

Hi,

i testing your project, i run the application and the application crashes while connecting to the server.

B4X:

Günlükçü şuna bağlandı: 11160b24ac4a3502
--------- beginning of crash
--------- beginning of main
*** Service (starter) Create ***
** Service (starter) Start **
Connecting...
** Activity (main) Create, isFirst = true **
** Activity (main) Resume **
** Activity (main) Pause, UserClosed = false **
Connection established
sending message to waiting queue (CallSubDelayed - update_user_list)
sending message to waiting queue (activity_permissionresult)
running waiting messages (2)
** Activity (main) Resume **
** Activity (main) Pause, UserClosed = false **
--------- beginning of system
*** Service (starter) Create ***
** Service (starter) Start **
Connecting...
** Activity (main) Create, isFirst = true **
** Activity (main) Resume **
io.socket.engineio.client.EngineIOException: xhr poll error
** Activity (main) Pause, UserClosed = false **
Tring to Reconnect
Reconnecting
io.socket.engineio.client.EngineIOException: xhr poll error
Reconnection Error
sending message to waiting queue (activity_permissionresult)
running waiting messages (1)
** Activity (main) Resume **
Tring to Reconnect
main_update_status (java line: 526)
android.view.ViewRootImpl$CalledFromWrongThreadException: Only the original thread that created a view hierarchy can touch its views.
    at android.view.ViewRootImpl.checkThread(ViewRootImpl.java:7816)
    at android.view.ViewRootImpl.invalidateChildInParent(ViewRootImpl.java:1345)
    at android.view.ViewGroup.invalidateChild(ViewGroup.java:5446)
    at android.view.View.invalidateInternal(View.java:14755)
    at android.view.View.invalidate(View.java:14719)
    at android.view.View.invalidate(View.java:14703)
    at android.widget.TextView.checkForRelayout(TextView.java:8536)
    at android.widget.TextView.setText(TextView.java:5083)
    at android.widget.TextView.setText(TextView.java:4908)
    at android.widget.TextView.setText(TextView.java:4883)
    at anywheresoftware.b4a.objects.TextViewWrapper.setText(TextViewWrapper.java:39)
    at b4a.example.main._update_status(main.java:526)
    at java.lang.reflect.Method.invoke(Native Method)
    at anywheresoftware.b4a.BA.raiseEvent2(BA.java:213)
    at anywheresoftware.b4a.keywords.Common.CallSub4(Common.java:1082)
    at anywheresoftware.b4a.keywords.Common.CallSubNew2(Common.java:1037)
    at b4a.example.starter._socket_onreconnecting(starter.java:308)
    at java.lang.reflect.Method.invoke(Native Method)
    at anywheresoftware.b4a.BA.raiseEvent2(BA.java:213)
    at anywheresoftware.b4a.BA.raiseEvent(BA.java:193)
    at com.biswajit.socketio.SocketIO$8.call(SocketIO.java:151)
    at io.socket.emitter.Emitter.emit(Emitter.java:117)
    at io.socket.client.Socket.access$601(Socket.java:19)
    at io.socket.client.Socket$5.run(Socket.java:177)
    at io.socket.thread.EventThread.exec(EventThread.java:55)
    at io.socket.client.Socket.emit(Socket.java:173)
    at io.socket.client.Manager.emitAll(Manager.java:156)
    at io.socket.client.Manager.access$600(Manager.java:20)
    at io.socket.client.Manager$11$1.run(Manager.java:555)
    at io.socket.thread.EventThread$2.run(EventThread.java:80)
    at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1133)
    at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:607)
    at java.lang.Thread.run(Thread.java:762)
** Activity (main) Pause, UserClosed = true **

View attachment 112197

This means the socket.io client process is running on different thread. Only the main thread of the application can control any view modification. I tested the code its working fine. Did you changed any code or running the code as it is?

Ertan · Apr 24, 2021

Biswajit said:
This means the socket.io client process is running on different thread. Only the main thread of the application can control any view modification. I tested the code its working fine. Did you changed any code or running the code as it is?

I haven't touched any code. I just wrote the local ip address of my server in the mobile application.

Install Node.js
Extract the server code
Open terminal/cmd and navigate to the folder where you have extracted the code
Run npm install (it will take few minutes)
Run node app.js
That's it.

I performed these steps.
I updated the IP address in the application and uploaded it to my phone. I got the same error.

Biswajit · Apr 24, 2021

Ertan said:
I haven't touched any code. I just wrote the local ip address of my server in the mobile application.

Install Node.js

Extract the server code

Open terminal/cmd and navigate to the folder where you have extracted the code

Run npm install (it will take few minutes)

Run node app.js

That's it.

I performed these steps.
I updated the IP address in the application and uploaded it to my phone. I got the same error.

Ok. I will check the code again. What is your b4a version?

Ertan · Apr 24, 2021

Biswajit said:
Ok. I will check the code again. What is your b4a version?

v10.70

Addo · Apr 26, 2021

I have tested this project. The echo canceler that you have set within the source code is not effective on most Android devices tested on Android 7,8,11
The echo is a bottleneck for any duplex chat platform

Ertan · Apr 26, 2021

Biswajit said:
Ok. I will check the code again. What is your b4a version?

I found the solution to the problem.

After saying "npm install" and installing it will give you a warning.

If you say "npm audit fix --force" as it said, it will make a few changes in your project. These changes will cause your project to not work properly. So I ignored this warning and started the project by typing "node app.js" directly. I was able to perform my test without any trouble.

But..

1) Two-sided audio departure is very delayed (3-4 seconds) sometimes it doesn't even go away. What is the reason of this? How can we avoid this situation?
2) On my Android 7.0 phone, the project is running in Debug mode, but in Release mode my project crashes and gives an error.

This is not a problem on my Android 10 phone.

Crash:

B4X:

--------- beginning of system
*** Service (starter) Create ***
** Service (starter) Start **
Connecting...
** Activity (main) Create, isFirst = true **
** Activity (main) Resume **
io.socket.engineio.client.EngineIOException: xhr poll error
Tring to Reconnect
main_update_status (java line: 484)
android.view.ViewRootImpl$CalledFromWrongThreadException: Only the original thread that created a view hierarchy can touch its views.
    at android.view.ViewRootImpl.checkThread(ViewRootImpl.java:7816)
    at android.view.ViewRootImpl.invalidateChildInParent(ViewRootImpl.java:1345)
    at android.view.ViewGroup.invalidateChild(ViewGroup.java:5446)
    at android.view.View.invalidateInternal(View.java:14755)
    at android.view.View.invalidate(View.java:14719)
    at android.view.View.invalidate(View.java:14703)
    at android.widget.TextView.checkForRelayout(TextView.java:8536)
    at android.widget.TextView.setText(TextView.java:5083)
    at android.widget.TextView.setText(TextView.java:4908)
    at android.widget.TextView.setText(TextView.java:4883)
    at anywheresoftware.b4a.objects.TextViewWrapper.setText(TextViewWrapper.java:39)
    at b4a.example.main._update_status(main.java:484)
    at java.lang.reflect.Method.invoke(Native Method)
    at anywheresoftware.b4a.BA.raiseEvent2(BA.java:213)
    at anywheresoftware.b4a.keywords.Common.CallSub4(Common.java:1082)
    at anywheresoftware.b4a.keywords.Common.CallSubNew2(Common.java:1037)
    at b4a.example.starter._socket_onreconnecting(starter.java:308)
    at java.lang.reflect.Method.invoke(Native Method)
    at anywheresoftware.b4a.BA.raiseEvent2(BA.java:213)
    at anywheresoftware.b4a.BA.raiseEvent(BA.java:193)
    at com.biswajit.socketio.SocketIO$8.call(SocketIO.java:151)
    at io.socket.emitter.Emitter.emit(Emitter.java:117)
    at io.socket.client.Socket.access$601(Socket.java:19)
    at io.socket.client.Socket$5.run(Socket.java:177)
    at io.socket.thread.EventThread.exec(EventThread.java:55)
    at io.socket.client.Socket.emit(Socket.java:173)
    at io.socket.client.Manager.emitAll(Manager.java:156)
    at io.socket.client.Manager.access$600(Manager.java:20)
    at io.socket.client.Manager$11$1.run(Manager.java:555)
    at io.socket.thread.EventThread$2.run(EventThread.java:80)
    at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1133)
    at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:607)
    at java.lang.Thread.run(Thread.java:762)
** Activity (main) Pause, UserClosed = true **

Addo · Apr 26, 2021

You can lower your bitrate or even encode your audio for bandwidth saving before sending.

Ertan · Apr 26, 2021

PassionDEV said:
You can lower your bitrate or even encode your audio for bandwidth saving before sending.

Does this matter on a local server? Unfortunately, I do not have much information on these matters. I'm asking to find out.

Addo · Apr 26, 2021

Ertan said:
Does this matter on a local server? Unfortunately, I do not have much information on these matters. I'm asking to find out.

If this a localhost no it doesn't matter to encode or lower the size of the audio packet.

Ertan · May 11, 2021

mehdipass said:
Does anyone have an idea to solve this problem?

Ertan said:
I found the solution to the problem.

After saying "npm install" and installing it will give you a warning.

View attachment 112310

If you say "npm audit fix --force" as it said, it will make a few changes in your project. These changes will cause your project to not work properly. So I ignored this warning and started the project by typing "node app.js" directly. I was able to perform my test without any trouble.

Biswajit · May 11, 2021

PassionDEV said:
I have tested this project. The echo canceler that you have set within the source code is not effective on most Android devices tested on Android 7,8,11
The echo is a bottleneck for any duplex chat platform

PassionDEV said:
You can lower your bitrate or even encode your audio for bandwidth saving before sending.

All the possible fixes will be posted as a paid version. I am down due to covid19. Once I feel good I will continue that paid version.

Addo · May 11, 2021

@Biswajit get well soon you will beat that virus stay strong.

José A. · Mar 5, 2022

MarcoRome said:
Super job. Waiting for an Audio / Video demo. If everything is ok, I will certainly make the purchase.
At the moment if you use a wi-fi it gives problems (the voice comes with a flickering after several seconds), on 4G sometimes it is very fluid, other times there is a delay of up to 3-4 seconds

Hi! Nice to meet you.
Do you know anything about the project Simple audio call over the Internet with your own TURN/relay server?
Was it just done? I'm interested, I've written to Thread Starter Biswajit but they don't reply.
Thank you so much for everything.

José A. · Mar 5, 2022

And where do I put it, do you know anything about this project?
Thank you very much.

omarruben · Jun 8, 2022

Biswajit said:
Hi everyone,

I posted a thread long ago about the audio/video calling over the internet. Here is a simple example of how you can call almost (will discuss below) realtime.

Why only a simple audio calling example while I said that I will post both audio and video calling examples?
While working on this project I realized that a good quality audio/video calling application with all required functionalities needs huge time and effort. Because like whatsapp, google duo if you want faster calling app then you have to compress the audio/video buffer before sending and decompress the audio/video buffer after receiving. This may sound like a simple byte compression job but it isn't. Compressing/decompressing audio/video buffer in realtime and playing requires lots of research about audio/video processing, audio/video codes, how conversion works, and many other things. Thus I decided to post a simple example of how it works for free and will post a full example that will be chargeable. That will have lots of functionalities, check below to know more.

This will work,

From Android to Android

From Android to iOS

From iOS to Android

From iOS to iOS

Functionalities:

Apps:

List all the connected users and update in realtime

Call any connected users

Check if the user is busy or not

Show custom in-app incoming call screen with accept/reject button

Play ringtone for incoming call

Automatic screen dimming while talking

Call accepted/rejected/ended/busy/offline status

Call timer

Incoming call notification while the app is in the background (Android)

Noise cancellation, Acoustic echo cancellation, Automatic gain control (Android)

Server:

Can handle multiple connections simultaneously.

Deliver call requests and status

Broadcast connected user list

Check if the callee is busy or not

Work as a relay server for delivering audio buffers to respective clients

Limitations:

Few milliseconds delay in delivering audio buffer (sometimes). Because,

The example apps are processing uncompressed audio buffer (24Kb-30Kb buffer per seconds)

The test server (you can find the IP inside the project) has been configured to limit bandwidth usage for testing purposes.

No notifications when the app is closed. I haven't included that because it's a simple example.

How to run:

App:

Download latest socket.io client library wrapper (B4A / B4I)

Download, extract and run.

For ios, you need a working developer certificate because it will not work on the simulator.

Server: The test server is always up. If you want to test with the test server then just run the app. If you want to run it on your local server, then

Install Node.js

Extract the server code

Open terminal/cmd and navigate to the folder where you have extracted the code

Run npm install (it will take few minutes)

Run node app.js

That's it.

Functionalities that the full example will have,

Audio/Video call

End to end encryption

Switching audio output between speakers/headphone

Faster calling even with low internet speed (5-10kbps)

Incoming calling screen that will show even when the app is not running.

Register a name for your device. So the other users can see who is calling

Example of STUN server as default server and fallback to TURN server

Example of Socket.IO server along with WebSocket Server

*Maybe group calling.

The full example will be chargeable. I haven't decided the amount yet. It will be between $50 - $100.

Though the test server has bandwidth limitations, it can handle more than 100-200 users.

Download the test apk for testing.

do you have an example for B4j? thank you

Android Tutorial [B4X] Simple audio call over internet with own TURN/relay server

Attachments

MarcoRome

Expert

Biswajit

Active Member

Ertan

Active Member

Biswajit

Active Member

Biswajit

Active Member

Ertan

Active Member

Biswajit

Active Member

Ertan

Active Member

Addo

Well-Known Member

Ertan

Active Member

Addo

Well-Known Member

Ertan

Active Member

Addo

Well-Known Member

Ertan

Active Member

Biswajit

Active Member

Addo

Well-Known Member

José A.

Member

José A.

Member

omarruben

Active Member