B4J Tutorial [example] [B4X] Control the desktop with your phone

Discussion in 'B4J Tutorials' started by Erel, Nov 30, 2015.

  1. Erel

    Erel Administrator Staff Member Licensed User

    This example shows how simple it is to create network solutions with B4J, B4A and B4i.

    In this case the network communication is based on UDP.

    It implements a remote desktop mouse, where the Android device or iOS device control the desktop mouse.

    The B4J app uses jAWTRobot library to control the mouse.
    The commands are serialized with B4XSerializator.

    Note that the B4J app doesn't show any UI except of a system tray icon.
    It uses ControlsFX library to show notifications. Note that this requires adding a dummy form (with a UTILITY style to hide it from the task bar).

    [​IMG]

    upload_2015-11-30_16-26-42.png upload_2015-11-30_17-1-11.png

    The nice thing about this example is that the code is very short and simple.

    The desktop app listens on port 54323. See this post for information related to the firewall: http://www.b4x.com/android/forum/threads/ireleaselogger-read-the-logs-in-release-mode.51164/#content

    An extended version that works over UDP, TCP or MQTT is available in post #6.
     

    Attached Files:

    Last edited: Dec 2, 2015
    Kiffi, Pooya1, Toky Olivier and 21 others like this.
  2. wonder

    wonder Expert Licensed User

    Hi Erel! Thanks for this example!!
    To my knowledge, UDP is faster than TCP/IP, right? :)
     
  3. Erel

    Erel Administrator Staff Member Licensed User

    UDP is a bit faster. Though you won't be able to see any difference in a solution such as this one.

    UDP is stateless. It makes it easier to work with for a task such as this one. The desktop app just listens for incoming data and the clients send data to the correct port. There is no connection or session involved.

    The downside of UDP compared to TCP is that packets can be lost.
     
  4. wonder

    wonder Expert Licensed User

    Got it, thanks! :)
    Could this solution be implemented over the internet as well?
     
    Rockefeller Goldman likes this.
  5. Erel

    Erel Administrator Staff Member Licensed User

    Yes. In that case it would be simpler to run an external MQTT broker (such as cloudmqtt).
     
  6. Erel

    Erel Administrator Staff Member Licensed User

    upload_2015-12-2_13-18-37.png

    An extended example that works with UDP ,TCP and MQTT is attached.

    Unlike what I previously wrote, UDP behaves significantly better in this case.

    This is a good example of using classes to easily switch between different network solutions.
     

    Attached Files:

  7. micro

    micro Well-Known Member Licensed User

    Hi Erel
    a question, you think it's difficult send mouse movements together a rapid screenshot of the desktop?
    Of course with appropriate scaling on the device.
    It would be great to have a real remote desktop!
     
    Rockefeller Goldman likes this.
  8. Roycefer

    Roycefer Well-Known Member Licensed User

    The jAWTRobot library can take screenshots. The problem is that taking and sending screenshots at a rate that simulates video can use up a bit of processing power and a lot of bandwidth. There are some schemes I've used whereby you only send small rectangles that have changed their appearance. But it's still not a trivial system to implement.
     
    Rockefeller Goldman likes this.
  9. micro

    micro Well-Known Member Licensed User

    I know it's hard but I think that with a low resolution (but appropriate) you may be able.
     
  10. derez

    derez Expert Licensed User

    I have a non-ui server that handles commands and sends screenshots, using the jAWTRobot
    Code:
    Public Sub Handle(req As ServletRequest, resp As ServletResponse)
        robot.ScreenCurrentRectangleSetAsArbitrary(
    0,0,1280,720)
        img  = robot.ScreenCaptureAsByteArray
        
    Dim su As StringUtils
        resp.Write(
    "<img width=1280 height=720 src='data:image/png;base64,")
        resp.Write(su.EncodeBase64(img))
        resp.Write(
    "'/img>").write("</br>")
    End Sub
    But it is working with a messageloop. How can I combine the UI application above with the Non-ui ?
    The receiving app (b4a) code is this:
    Code:
    Sub JobDone (Job As HttpJob)
         
    If Job.Success = True Then
           
    Select Job.jobname
              ...           
            
    Case "screen"
                
    Dim st As String = Job.GetString
                
    Dim su As StringUtils
                
    Dim barr() As Byte
                barr = su.DecodeBase64(st)
                
    Dim InputStream1 As InputStream
                InputStream1.InitializeFromBytesArray(barr, 
    0, barr.Length)
                
    Dim bmp As Bitmap
                bmp.Initialize2(InputStream1)
                iv.SetBackgroundImage(bmp)
                iv.Visible = 
    True
               
        
    End Select
    Sending the screenshot every few seconds is enough for my needs.
    In another way, how to send the picture (lets say by a timer) and how to get it in the receiver app.
     
    Last edited: Dec 4, 2015
  11. derez

    derez Expert Licensed User

    I managed to do it with TCP because it has a two way connection (while UDP requires the ip of the receiver which I don't have as a fix ip).
    The server sends this with a timer :
    Code:
    robot.ScreenCurrentRectangleSetAsArbitrary(0,0,1280,720)
    img  = robot.ScreenCaptureAsByteArray
    astream.Write(img)
    The b4a use this in the TCPconnector:
    Code:
    Sub astream_NewData (Buffer() As Byte)
        
    Dim InputStream1 As InputStream
        InputStream1.InitializeFromBytesArray(Buffer, 
    0, Buffer.Length)
        bmp.Initialize2(InputStream1)
    End Sub
    What is left to do is to send somehow the position of the mouse and display it on the image (or draw it and then send ?) because it is not included in the screenshot.
     
  12. derez

    derez Expert Licensed User

    I would like to know the position of the mouse in the b4j app.
    I believe it should be retrieved somehow by the remotecommand object.
     
    Rockefeller Goldman likes this.
  13. Roycefer

    Roycefer Well-Known Member Licensed User

    In the B4J app, call robot.RobotMouseCurrentLocation() to get the mouse pointer's current location.
     
    Rockefeller Goldman and derez like this.
  14. derez

    derez Expert Licensed User

    The attached files work as a remote control of a PC by a b4a device, sending mouse movements and receiving the screenshot with possibility of zoom in and out.
    I left only the TCP for communication.
    It works with my nexus5, I have not scaled it yet to work with anything else.
    The port and host ip are different from the original because this is what I use at home, so change them for your setup.
    It is just a demo !
     

    Attached Files:

    Toky Olivier, hibrid0, spairo and 3 others like this.
  15. tufanv

    tufanv Expert Licensed User

    if we want to use this for mac , how does the firewall port allowing works ? Because in windows , it automaticly requests for authorization with a popup , if we want to use it with mac , how can we allow the firewall inside the app , or does it have to be manually added ?
     
  16. Erel

    Erel Administrator Staff Member Licensed User

    Please start a new thread for this question.
     
Loading...
  1. This site uses cookies to help personalise content, tailor your experience and to keep you logged in if you register.
    By continuing to use this site, you are consenting to our use of cookies.
    Dismiss Notice