B4J Question Recognize playing cards using opencv

[jroriz]

Could someone help me to write a program to recognize playing cards using opencv?
The cards always appear in the same place and always with the same pattern.
What can change is the window size, which the user can freely change, but always in the same proportion.
Where the cards are is not my problem. I can with basic math know the size of the window and where
the cards would be.

Then I would crop the part of the screen where the cards are and apply opencv to the cropped image.

 

[jroriz]

The point is: OCR has become a problem due to the constant updates of the apps and also because the solution I developed (a HUD to help players make decisions) is for many different poker platforms. In two of them the cards are arranged at an angle of up to 45 degrees!

That's why I would like to use opencv, which I believe could recognize the cards regardless of platform or resolution (as long as a minimum resolution is observed).

 

[emexes]

In two of them the cards are arranged at an angle of up to 45 degrees!

it's almost like they're actively trying to throw a spanner in your works

 

[jroriz]

it's almost like they're actively trying to throw a spanner in your works

No, it's just a matter of design.
See how different apps can be:
https://www.pokerlistings.com/blog/the-most-addictive-useful-free-poker-apps-for-android-iphone
I use emulators like ldplayer to run apps on PC.

 

[emexes]

See how different apps can be:

Ah, I thought you meant the one app kept changing their design subtly to thwart automatic screen readers.

 

[jroriz]

Actually this happens too, but most of the time it's just a layout upgrade. The problem with using OCR is that if the layout changes even a few pixels to the left or right, it already interferes with the recognition of the letters.

 

[emexes]

if the layout changes even a few pixels to the left or right, it already interferes with the recognition of the letters.

I suspect that moving by a fraction of a pixel is causing most of that problem. And then with subpixel rendering on top of that... I am impressed that OCR works as well as it mostly does.

 

[jroriz]

I suspect that moving by a fraction of a pixel is causing most of that problem. And then with subpixel rendering on top of that... I am impressed that OCR works as well as it mostly does.

When the user reduces the size of the app's screen, the recognition of the letters becomes very difficult, even "isolating" the letters of the playing cards.
One feature that greatly improved recognition was repeating the image several times next to each other. For example: if the card is an A, I duplicate the image several times, say 5 times, which will generate AAAAA.

 

[Magma]

Sorry, can I ask again the scope-target of recognizing?

You have a counter of cards already played.. to not memorize them? Is it for personal use or something else to know?

 

[kimstudio]

I suggest to just use template matching function from opencv to realize this: cv.matchTemplate()

Template matching on target image blurred 3 times:

 

[emexes]

The point is: OCR has become a problem due to the constant updates of the apps and also because the solution I developed (a HUD to help players make decisions) is for many different poker platforms. In two of them the cards are arranged at an angle of up to 45 degrees!

Any angle is a bad angle.

 

[Magma]

I think opencv with matchTemplate and "planar" objects detection... may be... help... to rotate template and then check...

OpenCV: Detection of planar objects

docs.opencv.org

also why not also scaling width, height... 2-5 pixels... to make your life more difficult

 

[jroriz]

Sorry, can I ask again the scope-target of recognizing?

You have a counter of cards already played.. to not memorize them? Is it for personal use or something else to know?

A 'HUD' is a tool to assist in decision making.
The HUD shows opponents stats (how many times he voluntarily entered a hand, if he is aggressive, passive, etc).
In the case of playing cards detection, it is to know my probability of winning based on the cards I have and the cards that have already come out.
See below an example of a HUD "in action":

 

[jroriz]

Any angle is a bad angle.

View attachment 135552

I thought opencv would be able to recognize playing cards at any angle and size...

 

[jroriz]

Template matching on target image blurred 3 times:

View attachment 135537

Could you please share the code?
How did you "train" opencv?

 

[Magma]

Could you please share the code?
How did you "train" opencv?

as i know is not actually training you must have all possible images (rotated images) or create them on the fly to check into the "original" image-screenshot with opencv.... see planar objects... ofcourse you can have some techniques (playing with brightness, grayscable, rgb, %-similarity)...

tip: sometimes blurring, or scaling a picture to minimum size has better and faster results, but not all the time... you will need a lot time to find the right settings for your tool.