B4J Question find Area in Image

[Blueforcer]

Hello,

im searching for a solution to find a given Area in an Image.
I want to make screenshots and simulate Mousclicks on specific areas when something is found.
the Area bitmap is fix in my assets, and i need the coordinates where this little Area image is shown in the screenshot.
this area has the exact same pixels an in the Mainpicture.

is there something available for B4J that i can use for such a proccess?

thanks

 

[inakigarm]

You could use Opencv (https://opencv.org/opencv-3-4-1.html ) to find the image inside the other (https://stackoverflow.com/questions...matching-example-in-android/17516753#17516753, but you'll have to use inline java https://www.b4x.com/android/forum/threads/inline-javacv-opencv.53963/

Another option is a 'brute force' aproach https://stackoverflow.com/questions/12598818/finding-a-picture-in-a-picture-with-java

Sure the forum expert's will tune my answer ..

 

[Erel]

Can you post an example of such picture and the area you want to calculate?

 

[Blueforcer]

Sure,

for example i want to find this

in this

 

[Erel]

Are you looking for 100% exact match? It is easier.

100% means exact same size and colors.

 

[Blueforcer]

Yes the searching image will be available in the Main Image for 100%.
Similar would be great too, but for now 100% match is enough

 

[Erel]

Code:

Sub Process_Globals
   Private MainForm As Form
   Private xui As XUI
   Private ImageView1 As ImageView
End Sub

Sub AppStart (Form1 As Form, Args() As String)
   MainForm = Form1
   MainForm.RootPane.LoadLayout("1") 'Load the layout file.
   MainForm.Show
   Dim large As B4XBitmap = xui.LoadBitmap(File.DirAssets, "main.png")
   Dim small As B4XBitmap = xui.LoadBitmap(File.DirAssets, "small.png")
   Dim r As B4XRect = IndexOfImage(large, small)
   'just for the example:
   Dim lbc As BitmapCreator
   lbc.Initialize(large.Width, large.Height)
   lbc.CopyPixelsFromBitmap(large)
   If r.Width > 0 Then
       lbc.DrawRect(r, xui.Color_Red, True, 0)
   End If
   lbc.SetBitmapToImageView(lbc.Bitmap, ImageView1)
End Sub

Sub IndexOfImage(Large As B4XBitmap, Small As B4XBitmap) As B4XRect
   Dim lbc, sbc As BitmapCreator
   lbc.Initialize(Large.Width, Large.Height)
   lbc.CopyPixelsFromBitmap(Large)
   sbc.Initialize(Small.Width, Small.Height)
   sbc.CopyPixelsFromBitmap(Small)
   Dim pm1, pm2 As PremultipliedColor
   Dim r As B4XRect
   For ly = 0 To lbc.mHeight - sbc.mHeight
       For lx = 0 To lbc.mWidth - sbc.mWidth
           If IsMatch(lbc, sbc, lx, ly, pm1, pm2) Then
               r.Initialize(lx, ly, lx + sbc.mWidth, ly + sbc.mHeight)
               Return r
           End If
       Next
   Next
   r.Initialize(0, 0, 0, 0)
   Return r
End Sub

Sub IsMatch(lbc As BitmapCreator, sbc As BitmapCreator, lx As Int, ly As Int, pm1 As PremultipliedColor, pm2 As PremultipliedColor) As Boolean
   For sy = 0 To sbc.mHeight - 1
       For sx = 0 To sbc.mWidth - 1
           sbc.GetPremultipliedColor(sx, sy, pm2)
           lbc.GetPremultipliedColor(sx + lx, sy + ly, pm1)
           If pm1.a <> pm2.a Or pm1.r <> pm2.r Or pm1.g <> pm2.g Or pm1.b <> pm2.b Then Return False
       Next
   Next
   Return True
End Sub

It looks for identical matches.

Depends on the latest version of BitmapCreator: https://www.b4x.com/android/forum/threads/95208/#content

 

[Blueforcer]

Wholy...that was fast!
tried it with BitmapCreator the last hours with no success.
Now i need to study your code.. huge Thanks!

 

[Star-Dust]

Wholy...that was fast!
tried it with BitmapCreator the last hours with no success.
Now i need to study your code.. huge Thanks!

The Erel code uses the "brute force" method, ie the comparison of every single pixel.

Unlike the one suggested in the second link of @inakigarm, it does not calculate the possibility of a small difference in color. BUT you said it was not necessary.

 

[Blueforcer]

Since im not fimilar with Java, i have to go this way

 

[Blueforcer]

Maybe someone can help me to go a step further with the OpenCV variant?
erles function works great and it will also work in my application and will stay for some simple items, but sometimes some things look different.
I can also pay something for the help. maybe i can learn from that help too, since i didnt understand how to use this example in inline java

 

[JordiCP]

Hi,
If the template can be 'a bit different' but has the same size, it can be detected with OpenCV using Template matching and MinmaxLoc function. There are several examples over there. A bit of theory is explained HERE

If it isn't solved before, I can give it a try during the weekend if you need it