B4J Tutorial [example] [B4X] Control the desktop with your phone

Erel

Administrator
Staff member
Licensed User
This example shows how simple it is to create network solutions with B4J, B4A and B4i.

In this case the network communication is based on UDP.

It implements a remote desktop mouse, where the Android device or iOS device control the desktop mouse.

The B4J app uses jAWTRobot library to control the mouse.
The commands are serialized with B4XSerializator.

Note that the B4J app doesn't show any UI except of a system tray icon.
It uses ControlsFX library to show notifications. Note that this requires adding a dummy form (with a UTILITY style to hide it from the task bar).



upload_2015-11-30_16-26-42.png
upload_2015-11-30_17-1-11.png


The nice thing about this example is that the code is very short and simple.

The desktop app listens on port 54323. See this post for information related to the firewall: http://www.b4x.com/android/forum/threads/ireleaselogger-read-the-logs-in-release-mode.51164/#content

An extended version that works over UDP, TCP or MQTT is available in post #6.
 

Attachments

Last edited:

Erel

Administrator
Staff member
Licensed User
UDP is a bit faster. Though you won't be able to see any difference in a solution such as this one.

UDP is stateless. It makes it easier to work with for a task such as this one. The desktop app just listens for incoming data and the clients send data to the correct port. There is no connection or session involved.

The downside of UDP compared to TCP is that packets can be lost.
 

Erel

Administrator
Staff member
Licensed User
upload_2015-12-2_13-18-37.png


An extended example that works with UDP ,TCP and MQTT is attached.

Unlike what I previously wrote, UDP behaves significantly better in this case.

This is a good example of using classes to easily switch between different network solutions.
 

Attachments

micro

Well-Known Member
Licensed User
Hi Erel
a question, you think it's difficult send mouse movements together a rapid screenshot of the desktop?
Of course with appropriate scaling on the device.
It would be great to have a real remote desktop!
 

Roycefer

Well-Known Member
Licensed User
The jAWTRobot library can take screenshots. The problem is that taking and sending screenshots at a rate that simulates video can use up a bit of processing power and a lot of bandwidth. There are some schemes I've used whereby you only send small rectangles that have changed their appearance. But it's still not a trivial system to implement.
 

derez

Expert
Licensed User
I have a non-ui server that handles commands and sends screenshots, using the jAWTRobot
B4X:
Public Sub Handle(req As ServletRequest, resp As ServletResponse)
    robot.ScreenCurrentRectangleSetAsArbitrary(0,0,1280,720)
    img  = robot.ScreenCaptureAsByteArray
    Dim su As StringUtils
    resp.Write("<img width=1280 height=720 src='data:image/png;base64,")
    resp.Write(su.EncodeBase64(img))
    resp.Write("'/img>").write("</br>")
End Sub
But it is working with a messageloop. How can I combine the UI application above with the Non-ui ?
The receiving app (b4a) code is this:
B4X:
Sub JobDone (Job As HttpJob)
     If Job.Success = True Then
       Select Job.jobname
          ...           
        Case "screen"
            Dim st As String = Job.GetString
            Dim su As StringUtils
            Dim barr() As Byte
            barr = su.DecodeBase64(st)
            Dim InputStream1 As InputStream
            InputStream1.InitializeFromBytesArray(barr, 0, barr.Length)
            Dim bmp As Bitmap
            bmp.Initialize2(InputStream1)
            iv.SetBackgroundImage(bmp)
            iv.Visible = True
           
    End Select
Sending the screenshot every few seconds is enough for my needs.
In another way, how to send the picture (lets say by a timer) and how to get it in the receiver app.
 
Last edited:

derez

Expert
Licensed User
I managed to do it with TCP because it has a two way connection (while UDP requires the ip of the receiver which I don't have as a fix ip).
The server sends this with a timer :
B4X:
robot.ScreenCurrentRectangleSetAsArbitrary(0,0,1280,720)
img  = robot.ScreenCaptureAsByteArray
astream.Write(img)
The b4a use this in the TCPconnector:
B4X:
Sub astream_NewData (Buffer() As Byte)
    Dim InputStream1 As InputStream
    InputStream1.InitializeFromBytesArray(Buffer, 0, Buffer.Length)
    bmp.Initialize2(InputStream1)
End Sub
What is left to do is to send somehow the position of the mouse and display it on the image (or draw it and then send ?) because it is not included in the screenshot.
 

derez

Expert
Licensed User
The attached files work as a remote control of a PC by a b4a device, sending mouse movements and receiving the screenshot with possibility of zoom in and out.
I left only the TCP for communication.
It works with my nexus5, I have not scaled it yet to work with anything else.
The port and host ip are different from the original because this is what I use at home, so change them for your setup.
It is just a demo !
 

Attachments

tufanv

Expert
Licensed User
This example shows how simple it is to create network solutions with B4J, B4A and B4i.

In this case the network communication is based on UDP.

It implements a remote desktop mouse, where the Android device or iOS device control the desktop mouse.

The B4J app uses jAWTRobot library to control the mouse.
The commands are serialized with B4XSerializator.

Note that the B4J app doesn't show any UI except of a system tray icon.
It uses ControlsFX library to show notifications. Note that this requires adding a dummy form (with a UTILITY style to hide it from the task bar).



View attachment 39318 View attachment 39323

The nice thing about this example is that the code is very short and simple.

The desktop app listens on port 54323. See this post for information related to the firewall: http://www.b4x.com/android/forum/threads/ireleaselogger-read-the-logs-in-release-mode.51164/#content

An extended version that works over UDP, TCP or MQTT is available in post #6.
if we want to use this for mac , how does the firewall port allowing works ? Because in windows , it automaticly requests for authorization with a popup , if we want to use it with mac , how can we allow the firewall inside the app , or does it have to be manually added ?
 

Hypnos

Active Member
Licensed User
Anyone know what is this error?

B4X:
Picked up _JAVA_OPTIONS: -Djavax.net.ssl.trustStore=c:\windows\sun\java\deployment\trusted.cacerts
main._shownotification (java line: 163)
java.lang.NoSuchMethodError: javafx.stage.Window.impl_getWindows()Ljava/util/Iterator;
    at org.controlsfx.tools.Utils.getWindow(Utils.java:50)
    at org.controlsfx.control.Notifications$NotificationPopupHandler.show(Notifications.java:310)
    at org.controlsfx.control.Notifications.show(Notifications.java:269)
    at org.controlsfx.control.Notifications.showInformation(Notifications.java:244)
    at anywheresoftware.b4j.objects.ControlsUtils.ShowNotification3(ControlsUtils.java:138)
    at b4j.example.main._shownotification(main.java:163)
    at b4j.example.tcpconnector._showip(tcpconnector.java:87)
    at b4j.example.tcpconnector._initialize(tcpconnector.java:75)
    at b4j.example.main._appstart(main.java:95)
    at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
    at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
    at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
    at java.base/java.lang.reflect.Method.invoke(Method.java:566)
    at anywheresoftware.b4a.BA.raiseEvent2(BA.java:91)
    at anywheresoftware.b4a.BA.raiseEvent(BA.java:78)
    at b4j.example.main.start(main.java:37)
    at javafx.graphics/com.sun.javafx.application.LauncherImpl.lambda$launchApplication1$9(LauncherImpl.java:846)
    at javafx.graphics/com.sun.javafx.application.PlatformImpl.lambda$runAndWait$12(PlatformImpl.java:455)
    at javafx.graphics/com.sun.javafx.application.PlatformImpl.lambda$runLater$10(PlatformImpl.java:428)
    at java.base/java.security.AccessController.doPrivileged(Native Method)
    at javafx.graphics/com.sun.javafx.application.PlatformImpl.lambda$runLater$11(PlatformImpl.java:427)
    at javafx.graphics/com.sun.glass.ui.InvokeLaterDispatcher$Future.run(InvokeLaterDispatcher.java:96)
    at javafx.graphics/com.sun.glass.ui.win.WinApplication._runLoop(Native Method)
    at javafx.graphics/com.sun.glass.ui.win.WinApplication.lambda$runLoop$3(WinApplication.java:174)
    at java.base/java.lang.Thread.run(Thread.java:834)
 
Top