I've tried it with Java 8 and Java 11, on windows 10..
Without knowing what hardware you have installed, the only thing I can suggest is that you check the X value returned in FXSystemtrayHandler Module TIMouse_Event sub and see if it looks reasonable. If not, there may be alternative methods in the javaFX mouse event that will return a better value.
If it does look reasonable, then I would suggest that you make sure you have the latest drivers installed for your graphics card and take a look on Google to see if anyone has had a similar experience.
As I say, it is working as expected here with two monitors. Are your two monitors the same resolution?