Task is to properly scale the screenshots of various Android devices that got from remote devices and get the tap coordinate over them (for remote control).
I have
1) Screenshot that made by MediaProjection
2) And GetDeviceLayoutValues.Width and GetDeviceLayoutValues.Height of this device
1) Screenshots are different kind, if from different devices: if "old" device with mechanical\external system keys HOME, BACK, RECENT APPS - the screenshot has no these keys on the bottom.
If new device with the virtual system keys on the screen bottom - they are on the screenshot.
2) GetDeviceLayoutValues.Height - does not correspond the real screen vertical size in pixels, seem, the notification area (optionally) and\or Activity.Title height (also optionally) make influence...
But if to see the screenshot and making taps and swipes over them - the dimension and proportions must be exacly correct.
Super sub from Erel
B4X:
Sub FillImageToView(bmp As Bitmap, ImageView As View)
Dim bmpRatio As Float = bmp.Width / bmp.Height
Dim viewRatio As Float = ImageView.Width / ImageView.Height
If viewRatio > bmpRatio Then
Dim NewHeight As Int = bmp.Width / viewRatio
bmp = bmp.Crop(0, bmp.Height / 2 - NewHeight / 2, bmp.Width, NewHeight)
Else if viewRatio < bmpRatio Then
Dim NewWidth As Int = bmp.Height * viewRatio
bmp = bmp.Crop(bmp.Width / 2 - NewWidth / 2, 0, NewWidth, bmp.Height)
End If
ImageView.SetBackgroundImage(bmp.Resize(ImageView.Width, ImageView.Height, True))
End Sub
helps not much.
Is there way to correct setup screenshots showing (on the fullscreen Activity, IncludeTitle=false) and tapping ?
Here those 2 screenshots shown on the same Android9 device with the virtual system keys.
1) Manually done screenshots are 1080 x 2340 px - fully correct size of the screen
2) But the screenshot done via MediaProjection was 1088 x 2131 px from the same Android9 device.
3) The fullscreen Activity that shows the fullscreen screenshot - anyway reserved the notif area at the top, extra space...
Seems, this bottom area of the virtual system keys is the trouble, and notif area at the top...
Thanks Erel, but is it possible to get the screenshot (on the source device) and show it (on another device) so that tap coordinates (on another device) are to be fully corresponding (of the source device) ?
As i can understand - the notif area, Activity.Title and bottom system buttons area (of latest Android) - are all optional on the screenshot ?
yes, this is the question.
Remote control is at least:
1) Seeing the screen of the remote device (the screenshot)
2) And possibility to tap, long-tap, swipe over this screenshot (BTW, it's already works OK), but on another screen, with another size and scale.
3) And getting the tap coordinates that correspond to the remote display.
Not fully. In your example, as i can understand, just one panel of drawing, all parameters are absolute.
But common remote control (as RDP in Windows, or Teamviewer in Android) - the full screen is visible, and should be tappable... by the exact screen coordinates: tap on my screen1 to simulate tap on remote screen2.
Size, scale, density of Screen1 and Screen2 are not same, in whole.
Not fully. In your example, as i can understand, just one panel of drawing, all parameters are absolute.
But common remote control (as RDP in Windows, or Teamviewer in Android) - the full screen is visible, and should be tappable... by the exact screen coordinates: tap on my screen1 to simalate tap on remote screen2.
Size, scale, density of Screen1 and Screen2 are not same, in whole.
Sorry, it is not my example, Erel did it and, as you all know, he is always very approximate in his examples
(sooner or later Erel will beat me )
I meant to say that the ultimate goal I guess is something of that kind.
In a certain sense, you would like to "normalize", to manage the remote screen as if it were on your display (eventually displaying on your device a panel bigger than your display and swipe it).
Proportions, density, scales... Also Klaus could come into play here
When you have the screenshot on your device, get the coordinates of the click and transform them in Percent.
Passing back this information to the remote device should permit to replicate the click in the relative position, transforming the Percent into coordinates with the remote device width and height.
When you have the screenshot on your device, get the coordinates of the click and transform them in Percent.
Passing back this information to the remote device should permit to replicate the click in the relative position, transforming the Percent into coordinates with the remote device width and height.
Not so easy. Percent of? The displays may have different "ratio" w/h (thank you, display manufacturers; I look forward to television displays other than the current 16:9, also 13:10, 8:500, etc )
Percent of the Imageview that contain the screenshot of the remote device.
Of course I don't know if @peacemaker is using an Imageview or something else.
If the Imageview is 400 x 800 (just an example), clicking the center at 200, 400 will produce 50%x and 50%y.
Sending this percents to a device with screen of 1080 x 1920 will results in a click at 540, 960...... so the screen center of the remote device.
Yes ! if the screenshot is fully scaled to an ImageView with some proportional sides - It needs to send the tap coordinates in %, not in pixels. But before simulation - they must be recalculated into pixels of this local screen.
Percent of the Imageview that contain the screenshot of the remote device.
Of course I don't know if @peacemaker is using an Imageview or something else.
If the Imageview is 400 x 800 (just an example), clicking the center at 200, 400 will produce 50%x and 50%y.
Sending this percents to a device with screen of 1080 x 1920 will results in a click at 540, 960...... so the screen center of the remote device.