Regarding the Sudoku itself, the processes involved are: image acquisition (in this case a recorded video), image pre-conditioning, grid detection, digit extraction and recognition, solver and drawing the results back.
For the grid recognition part, I made a mix of several existing posts. In this case, it uses Hough lines detection and some logic filters to remove duplicates and validate the grid only if there are enough lines for each group (vertical and horizontal), they keep similar distances between them, and other restrictions so that it can decide with enough confidence if it is a Sudoku grid or not.
Once detected, it calculates the line intersections, from where we get the outer corners, calculate the perspective and unwarp it. Then for each cell, we take the inner part, detect if there is something inside, and give it to a SVM which has been previously trained with the digits HOG features (a good compromise between speed and reliability in case of printed chars)
Once we have the 'predicted' digits, they are put in an array and given to a Sudoku solver module which uses a slightly modified backtracking algorithm that I found somewhere (there's also plenty of them in internet).
Since the solver can take a while, it uses a callback to notify the results. If the grid and the digits that are being detected at a given moment is still the same that has been passed to the solver, and that sudoku has been solved, it draws the solution in the cells of the unwarped image. Then we undo the unwarping and the final result is drawn.
Yes, the example uses a B4XCanvas to draw the corresponding Image to a Pane, scaling and all these things.
In order to get the B4J Image from the OCVMat, a 2-step conversion is necessary. First, the OCVMat is converted to a BufferedImage via an in-built-method of the OpenCV HighGui module, and then it is converted to a javafx.scene.image.Image via SwingFXUtils methods.
A similar process is made to get an OCVMat from an Image.
!!! 