B4J Library jAWTRobot - invoke keyboard and mouse events, etc...

This library started out as a simple wrapper for Oracle's java.awt.Robot package but it evolved out of hand into a library of general utilities. Some of these functions appear to duplicate functionality already present in B4J however this library is designed to be used in both UI apps and non-UI apps. Consequently, B4J functionality that requires the use of a JFX object (fx.showExternalDocument(), for example) is duplicated here but without the use of any objects only found in UI apps.

Things you can do with this library:
  • programmatically move the mouse and invoke system MouseButtonPressed events or KeyPressed events and much more
  • access the system clipboard
  • get a variety of general operating system, hardware and JVM info
  • get a variety of JVM, process and system performance info (memory usage, CPU load, etc...)
  • take screen shots of arbitrary rectangles on the system's screens
  • run arbitrary commands on the command line
  • redirect Standard Error and Standard Out to file
  • open external files or URLs
  • improved Exception-related stuff
  • improved Thread-related stuff
  • restart your app
The library is very heavily commented so you need only read them to see what is possible.

EDIT (8JUL2015): Added hostNameAndIPAddress() method. Library version remains the same, but it is a new file.
EDIT (30JUL2015): Updated to 1.1. See post #12 for details.
EDIT (25OCT2015): Updated to 1.3. See post #20 for details.
EDIT (26OCT2015): Updated to 1.35. See post #27 for details.
EDIT (22NOV2015): Updated to 1.37. See post #30 for details.
EDIT (30NOV2015): Updated to 1.4. See post #31 for details.
EDIT (30JAN2016): Updated to 1.42. See post #34 for details.
EDIT (31JAN2016): Updated to 1.45. See post #35 for details.
EDIT (31JAN2017): Updated to 1.50. See post #79 for details.
EDIT (02FEB2017): Updated to 1.51. See post #85 for details.
EDIT (06FEB2017): Updated to 1.52. Minor bug fix.
EDIT (04APR2017): Updated to 1.54. Bug fix for relaunchSelfX methods.
EDIT (28DEC2017): Updated to 1.55. Added ScreenCapture2. Read comments for more info.
EDIT (09JAN2019): Updated to 1.59. RobotPaste debugging build
EDIT (14JAN2019): Updated to 1.61. Another RobotPaste2 debugging build
 

Attachments

  • jAWTRobot.zip
    11.1 KB · Views: 1,082
  • jAWTRobot1.1.zip
    13.6 KB · Views: 717
  • jAWTRobot1.3.zip
    15.1 KB · Views: 675
  • jAWTRobot1.35.zip
    15.5 KB · Views: 700
  • jAWTRobot1.37.zip
    15.7 KB · Views: 673
  • jAWTRobot1.4.zip
    16.1 KB · Views: 724
  • jAWTRobot1.42.zip
    16.6 KB · Views: 647
  • jAWTRobot1.45.zip
    19.7 KB · Views: 865
  • jAWTRobot1.50.zip
    21.6 KB · Views: 677
  • jAWTRobot1.51.zip
    22.1 KB · Views: 617
  • jAWTRobot1.52.zip
    22.1 KB · Views: 731
  • jAWTRobot1.54.zip
    22.1 KB · Views: 850
  • jAWTRobot1.55.zip
    22.8 KB · Views: 908
  • jAWTRobot1.59.zip
    22.6 KB · Views: 559
  • jAWTRobot1.61.zip
    23.2 KB · Views: 1,586
Last edited:

lemonisdead

Well-Known Member
Licensed User
Longtime User
where can I find the temp batch file
For my part it is created in the DirApp folder. For example when I compile and execute from B4J, it is created in the Objects folder. But you will have to kill the restarting windows (with the timeout) or it will be deleted
 

Cableguy

Expert
Licensed User
Longtime User
Thanks for the suggestion about the killing, I was able to check the batch file...
Even though the path was correct, apparently it didn't like it much, so after I removed the File.DirApp and the leading slash, it works... so now I can start working on the creation of my own Batch file
 

Roycefer

Well-Known Member
Licensed User
Longtime User
You can also use Robot.relaunchSelfBatch() to relaunch your program via a custom batch file that you write. This is useful for when you have to launch your program with VM args or other special features and tasks beyond just "javaw -jar myjar.jar".
 

Cableguy

Expert
Licensed User
Longtime User
Thanks Roycefer...

I managed to get it working, and created my own simple batch
 

ThRuST

Well-Known Member
Licensed User
Longtime User
I have AWTRobot v1.45 (the latest I found) but can not use RobotPaste2, RobotPaste3 etc
Is there a newer version or how else did you get RobotPaste2? RobotPaste is not working on Mac so I ask for help. Thanks
 

rboeck

Well-Known Member
Licensed User
Longtime User
Hi, as i remember, RobotPaste2 and RobotPaste3 are older versions, before in version 1.45 the robotPaste routine was able to work for windows and mac; i made test an both systems in this time. Currently i have no mac to try, but i remember, that ver. 1.45 worked with mac without problems. (for my type of usecase)
 

ThRuST

Well-Known Member
Licensed User
Longtime User
This is how I solved it (jAWTRobot v1.45)

B4X:
Dim os As String = GetSystemProperty("os.name", "").ToLowerCase

If os.Contains("win") Then

        ' Windows
           fx.Clipboard.SetString ("hello")

        AWTRobot.RobotSpecialKeyCombo("ctrl_v")
   
   
    Else If os.Contains("mac") Then
   
        ' Macintosh
        fx.Clipboard.SetString ("world")

        AWTRobot.RobotSpecialKeyCombo("meta_v")

    Else
   
           ' Linux
   
    End If

Dunno about Linux, Solaris other O/S etc every comment is useful thanks
 
Last edited:

rboeck

Well-Known Member
Licensed User
Longtime User
I think, you dont need to check for os; try it this way:
B4X:
If fx.Clipboard.HasString Then
      AWTRobot.RobotPaste(Text)
End If
 

ThRuST

Well-Known Member
Licensed User
Longtime User
As far as I remember that did work well on PC, but not on my MacBook. In the intelisense it also says something like under the RobotPaste
If it does not work use the meta_v on Mac
I tested my solution an hour ago and it worked on both platforms. How about Linux etc?
 

rboeck

Well-Known Member
Licensed User
Longtime User
I have a running Ubuntu, but i am not able to make javafx working...
 

ThRuST

Well-Known Member
Licensed User
Longtime User
found this

The JavaFX SDK runs on Windows and Mac OS X. A beta release of the JavaFX SDK is provided for Ubuntu Linux and the OpenSolaris operating systems.
Go Here
 

Roycefer

Well-Known Member
Licensed User
Longtime User
The jAWTRobot does not rely on JavaFX. You can use it in non-UI B4J apps. For example, the fx.Clipboard.HasString() call is duplicated in the AWTRobot class with robot.ClipboardStringHas().

Users have reported success on Mac OS with the latest version of the jAWTRobot library (1.45) in using the robot.RobotPaste function. If it's not working for you, post the Exceptions you're getting and I'll see what I can do. I don't have a Mac so I can't directly test it.
 

ThRuST

Well-Known Member
Licensed User
Longtime User
Sorry I don't remember if it worked or not since I was solving a problem with the contextmenu staying under applications. However I solved it the way I described so that is an alternative solution I will use in my project, it works great because of your amazing library. In time you will see the result once I publish it.
I just wonder how to detect which mouse button that was clicked. How to do that? inside the mouseclick event?
 

ThRuST

Well-Known Member
Licensed User
Longtime User
Since my b4j project will have mouse button support I am working on a solution to add support for the Logitech G300S gaming mouse which I've bought, so I can hardcode support for 5 buttons (out of 9), the rest will have to use keyboard assignments which is done in the tray application provided by Logitech. Which means I will have to include support for some special keys that can be assigned to the mouse in Logitechs application. Buy the mouse and check my post Here
 

Roycefer

Well-Known Member
Licensed User
Longtime User
Yes, the mouse button info is provided in the NativeHook's _NativeMouseClicked() event. 1 is the left button, 2 is the right button and 3 is the center button. I don't know if there's support for numbered mouse buttons beyond that; my mouse only has 3 buttons. That's probably an OS-level question; I suspect the library will support whatever the OS supports.

If your Logitech mouse supports mapping some buttons to key combos (like ctrl_alt_f, though you can also listen for one-key "combos"), you should check out my KeyComboListener class. It greatly simplifies using NativeHook to listen for key combo events. It also allows for easy run-time creation of new key combos.
 

ThRuST

Well-Known Member
Licensed User
Longtime User
I have allready tested your hook library with my mouse and it's just to use numbers 1-5 for default buttons, which are 1=left, 2=right, 3=middle, 4 and 5=Buttonx1, buttonx2 (Microsoft also provide support for this in visual studio). I think that's why Logitech provides the binding application to use shortcuts to bind the rest of the buttons. Recently I got the idea that programmers can use a gaming mouse and benefit from using the many mouse buttons, and that goes well along with my new project which is a knowledge management system, so contextmenus will play a vital keyrole in how users will interact with development environments.
It's a bold chance by me to challenge traditional methods, but after watching numerous of interviews with Steve Jobs I understand that innovative ideas combined with the latest technology helps to push the industry forward. He also said that the hard working individuals are aware of this and intentionaly seek others who wants to do the same thing, which is just what is happening here :) The difficult part is to convince people who are used to 'the old ways' that technology is of great help and again there will be the people who strive for the better solution who is first to use it. Programming is one of the most powerful knowledges one can use in todays modern civilazations so it's a great thing that we evaluate the mouse, which will probablt be exchanged to eye tracking once the prices goes down, but then again currently there's sceptical people to that technology as well, which only makes us have to wait until they know they won't get eye cancer from it!! However with my mouse idea, programmers will have to get used to the many buttons, but the most you get used to it the more efficient the system will assist you. Since I am evaluating it by hard testing it with CSS driven contextmenus, I have discovered that the more complex designs I use in CSS, the slower the refresh is on the screen. I am not saying your code is to blame but since you probably had no idea about that I was going to do this, I recommend you to play around with my second design example and add more complexity to it, and then you click on it. Perhaps we have to tell Steve05 this since he provided the FloatingContextmenuv06' example and I made some CSS templates it's only good if it can be evaluated and perhaps updated. I wanted to tell you this since you created the hook, which is excellent as it is but maybe it can be tweaked a bit, so please take a look at it and let me know what you think.

EDIT: As the menus might get larger than in the example code either Steves code can be optimized or/and the hook library as well. Thanks for looking into this.
 
Last edited:

Roycefer

Well-Known Member
Licensed User
Longtime User
It's almost certainly not the NativeHook code that is slowing down. If you're calling code that modifies the UI based on NativeHook mouse clicked events, then the UI-related code isn't even running in the same thread as the NativeHook code.

I recommend you do some fine-grained benchmarking in your code to see where the slowdown is occurring. Example:
B4X:
Dim d as Long = DateTime.Now
'Execute some code that we'll call "Step 1"
Log($"Step 1 took ${DateTime.Now-d} milliseconds"$)
d = DateTime.Now
'Execute some other code that we'll call "Step 2"
Log($"Step 2 took ${DateTime.Now-d} milliseconds"$)
'so forth and so on....
You can see how the duration of each Step changes as you vary some other variable (CSS complexity, perhaps).

If millisecond precision is insufficient, use robot.SystemNanoTime() for nanosecond precision:
B4X:
Dim robot as AWTRobot
'....
Dim d as Long = robot.SystemNanoTime
'Execute some code that we'll call "Step 1"
Log($"Step 1 took ${robot.SystemNanoTime-d} nanoseconds"$)
'so forth and so on....
 

ThRuST

Well-Known Member
Licensed User
Longtime User
Wow, so you have allready baked that in as well, amazing :)
Now I want a bench marking application to check my system. When I posted my answer I was thinking about that I use a rather slow computer!!! it's a Packard-Bell EasyNote TE 1.7 celeron!!!!! :eek::D:confused: so it could be that, or what do you think haha
Since you have it in the library do you mind putting together a little form with a benchmark value and post here. Then everybody can use it. Great idea!!
Not strange my CSS styled contextmenu took resources since there were 3 contextmenu clones at the same time when clicking fast!!! :)
Most users don't use a 1.7Ghz computer anymore, and it's just a matter of time until they end up on the garbage mountain (like my code) just kidding a bit.
Your code does not need to be changed now we know celeron computers is not good for complex CSS themes.

Btw another question I still want to know, how can I catch which mouse was clicked in the MouseClick event? I tried if button = 2 then... but that didn't work.
I guess I made a simple misstake.
 

Roycefer

Well-Known Member
Licensed User
Longtime User
Which button was clicked is a field of the NativeMouseEvent Object passed as an argument in the _NativeMouseClicked() event:
B4X:
Sub nh_NativeMouseClicked(nme as NativeMouseEvent) As Boolean
   If nme.MouseButton==2 Then
         Log("Right mouse button was pressed.")
   End If
   Return False
End Sub

All the relevant info about the event will be stored in fields of nme. Use the code completion in the IDE to see what's available.

As for benchmarking, the examples I posted aren't for benchmarking a computer but for benchmarking specific sections of code.
 
Top