Android Question OpenCV Help with Warp Transformations please

[B G]

Hi, this is my first post to this forum, so apologies if I have posted this in the incorrect area.
I have been playing around with JordiCP's wrapper for the OpenCV library and so far so good. The examples previously posted were very helpful. However I was wondering if someone had a small sample project that uses Warp Transformations, more specifically that uses the 'GetPerspectiveTransform' and 'WarpPerspective' functions. I can't seem to find any examples anywhere.

In brief, I am trying to identify playing cards by pointing the camera of my tablet on them.. This is how far I have got;
1) I have used OCVJavaCameraView to get video frames as OCVMats
2) I have successfully grayscaled, blurred and created a threshold image
3) I have found all the contours of the card and successfully isolated the two shapes on the top left of the card. That is, the card face value and the suit.
4) I now have the contour of the card value and contour of the card suit.
5) But now I need to flatten those shapes to a size of say 200 x 300, so I can then compare it to a previously saved image of the card.

Any help or example code would be appreciated. Thank you

 

[hatzisn]

I am not exactly sure if I have understood correctly but (5) can be done by matrix multiplication. See this video (at 9:40):

 

[B G]

Hatzisn, thanks for your reply, however this is not what I need. All I need is a some example code of how to use the the 'GetPerspectiveTransform' and 'WarpPerspective' functions on a contour. From the examples I have seen (all written in Python), they seem to take the contour, get it's arclength (with the 'arclength' function) and then create a new OCVMat using the 'approxPolyDP' function. I have figured out how to do that ok but it's the next step, which is finding the corners of the contour and using the 2 perspective functions where I am stuck. I am attaching the Python code if that helps. I do not necessarily need a line by line translation of this code (you can if you like ! ) but if I can see an example of something similar in b4A, I will hopefully be able to work it out from there.

# Contour is previously obtained card value shape
# np is Python standard NumPy module which provides various mathematical functions
peri = cv2.arcLength(contour,True)
approx = cv2.approxPolyDP(contour,0.01*peri,True)
pts = np.float32(approx)

# Warp card into 200x300 flattened image using perspective transform
Temp_rect = np.zeros((4,2), dtype = "float32")
  
s = np.sum(pts, axis = 2)

tl = pts[np.argmin(s)]
br = pts[np.argmax(s)]

diff = np.diff(pts, axis = -1)
tr = pts[np.argmin(diff)]
bl = pts[np.argmax(diff)]

# Need to create an array listing points in order of
# [top left, top right, bottom right, bottom left]
# before doing the perspective transform
temp_rect[0] = tl
temp_rect[1] = tr
temp_rect[2] = br
temp_rect[3] = bl

maxWidth = 200
maxHeight = 300

# Create destination array, calculate perspective transform matrix,
# and warp card image
dst = np.array([[0,0],[maxWidth-1,0],[maxWidth-1,maxHeight-1],[0, maxHeight-1]], np.float32)
M = cv2.getPerspectiveTransform(temp_rect,dst)
warp = cv2.warpPerspective(image, M, (maxWidth, maxHeight))
warp = cv2.cvtColor(warp,cv2.COLOR_BGR2GRAY)

Many thanks.

 

[hatzisn]

I said "I am not exactly sure if I have understood...". Now I am exactly sure I had not understood.
I have no knowledge both for OpenCV usage as well as Python... Sorry.

 

[gezueb]

I have the same problem. The b4a wrapper of cpJordi asks for OCVMat types in the mImgProc.getperspective method. I am not sure what these python numpy arrays are, but they do not exist in b4a. So the big question is how to convert the corner points into an OCVMat. If you have found out in the mean time, please let me now!
Good luck, Georg

 

[JordiCP]

Not in front of my PC now, but I think that there is a class named OCVMatOfPoints ( or similar)
It extends OCVMat, so even if the Java bindings expect an OCVMat, passing an OCVMatOfPoints should work.
Please report back and if it does not work I'll look into it this afternoon.

 

[gezueb]

Thank you, JordiCP, I will try this. I must confess that I am rather "lost in translation" to and fro Python, Java and b4a types. They are all rather cryptic for me.

 

[gezueb]

I tried OCVMatofPoint, it compiles ok but the getperspective throw an error at runtime: CvException [org.opencv.core.CvException: cv::Exception: OpenCV(3.4.1) /build/master_pack-android/opencv/modules/imgproc/src/imgwarp.cpp:3232: error: (-215) src.checkVector(2, 5) == 4 && dst.checkVector(2, 5) == 4 in function cv::Mat cv::getPerspectiveTransform(cv::InputArray, cv::InputArray). Her is the code

    Dim M As OCVMat
    Dim pts_src,pts_dst As OCVMatOfPoint 'clockwise from top left.
    
    Dim src_points(4), dst_points(4) As OCVPoint

    src_points(0) = LTP 'these are of type OCVpoint
    src_points(1) = RTP
    src_points(2) = RLP
    src_points(3) = LLP
    pts_src.Initialize3(src_points)

    LH = CreatePoint(0,0) 'convert to type OCVPoint
    RH = CreatePoint(599,0)
    RL = CreatePoint(599,599)
    LL = CreatePoint(0,599)
    
    dst_points(0) = LH 'now, they are also of type OCVPoint
    dst_points(1) = RH
    dst_points(2) = LL
    dst_points(3) = RL
    pts_dst.Initialize3(dst_points)
    
    'the functions for conversion of the image are getPerspectiveTransform and then mImgProc.warpPerspective

    M = mImgProc.getPerspectiveTransform(pts_src,pts_dst)
    mImgProc.warpPerspective2(tmpMat,CorrMat,M,dSize)

 

[gezueb]

I have found the problem. The pts_src and pts_dst objects must be declared as OCVMatOfPoint2f, not only OCVMatofPoint! The
M = mImgProc.getPerspectiveTransform(pts_src,pts_dst) accepts only points in 32 bit float format (CV_32F...), while my previous code created int32(CV_32S..)
While b4a is easy in converting different number formats, Java seems to be more picky.

 

[gezueb]

And for anyone else who dares to venture into the OpenCV world: forget about "dip" endings of pixel points (unless in views of course): the scaling confuses the calculations and tends to make the bitmaps rather larger than necessary which in turn slows calculations.

 

[JordiCP]

That's right. The JavaCameraView provides a OCVMat which represents a bitmap where its element coordinates are pixels.
Another thing is how we visualize it on the screen (full size, scaled, ....) and that's where dip units appear, but regarding the processing pipeline, only the bitmap and its pixels should appear.
As you may already have figured, small resolutions (640x480 or even 320x240) are enough for most of the purposes.