OCR, FaceRecognition, BarcodeScanner using Google Vision

DonManfred

Expert
Licensed User
Longtime User

wes58

Active Member
Licensed User
Longtime User
I dont know how to add this as this is part of the library behind (i´m just importing the original from google)
Hi Don,

You could add this function to your library to scan bitmap.
You may need to add the check that detector is operational
B4X:
    public void scanBitmap(Bitmap b) {
        if(b != null){
            Frame frame = (new com.google.android.gms.vision.Frame.Builder()).setBitmap(b).build();
            SparseArray<TextBlock> textBlocks = detector.detect(frame);
            String blocks = "";
            String lines = "";
            String words = "";
           
            for (int index = 0; index < textBlocks.size(); index++) {
                //extract scanned text blocks here
                TextBlock tBlock = textBlocks.valueAt(index);
                blocks = blocks + tBlock.getValue() + "\n" + "\n";
                for (Text line : tBlock.getComponents()) {
                    //extract scanned text lines here
                    lines = lines + line.getValue() + "\n";
                    for (Text element : line.getComponents()) {
                        //extract scanned text words here
                        words = words + element.getValue() + ", ";
                    }
                }
            }       
   
            if(textBlocks.size() == 0) {
                if (ba.subExists(eventName + "_error_result")) {
                    ba.raiseEventFromDifferentThread(this, null, 0, eventName + "_error_result", true, new Object[] {"Scan Failed: Nothing to scan"});
                }                                   
            }
            else {
                if (ba.subExists(eventName + "_blocks_result")) {
                    ba.raiseEventFromDifferentThread(this, null, 0, eventName + "_blocks_result", true, new Object[] {blocks});
                }                           
                if (ba.subExists(eventName + "_lines_result")) {
                    ba.raiseEventFromDifferentThread(this, null, 0, eventName + "_lines_result", true, new Object[] {lines});
                }                                                   
                if (ba.subExists(eventName + "_words_result")) {
                    ba.raiseEventFromDifferentThread(this, null, 0, eventName + "_words_result", true, new Object[] {words});
                }                           
            }
        }
       
    }

What I do, is after taking a picture and scanning it, I sometimes need to crop a taken picture to select part of it, and then scan the cropped bitmap.
 

wes58

Active Member
Licensed User
Longtime User
This line does not return a SparseArray<Textblock>; it returns a SparseArray<Barcode> so the rest of the code will not work.
Of course it works. I have been using in for a couple of weeks now.

And it does return SparseArray<Textblock>, It is for detecting text (OCR) not barcode.

It is modified part from this github project http://www.truiton.com/2016/11/optical-character-recognition-android-ocr/ which is used John Schoeman, and probably your library is based on the same/similar java code since in the first post you link to this github project https://github.com/Truiton/MobileVisionAPI.
 

DonManfred

Expert
Licensed User
Longtime User
All i can say is that i get an error (red line) in eclipse saying that i should change the type to SparseArray<Barcode> instead of TextBlock...

I tried to cast it to a Textblock but it also shows an error at
B4X:
case Barcode.TEXT:
          m.Put("BarcodeType", "TEXT");
            TextBlock block = (TextBlock)code;
saying that it cannot cast Barcode to TextBlock.
 

wes58

Active Member
Licensed User
Longtime User
All i can say is that i get an error (red line) in eclipse saying that i should change the type to SparseArray<Barcode> instead of TextBlock...

I tried to cast it to a Textblock but it also shows an error at
B4X:
case Barcode.TEXT:
          m.Put("BarcodeType", "TEXT");
            TextBlock block = (TextBlock)code;
saying that it cannot cast Barcode to TextBlock.
This is probably, because you are using BarcodeDetector not TextRecognizer!
From your post #1, in your library have three components: BarcodeDetector, FaceDetector and TextRecogniser. You should use the code in TextRecogniser part of your library.
 

DonManfred

Expert
Licensed User
Longtime User
The problem was that i did added it to the Barcodedetector, not to the Textdetector-code....

i now added this to the Textdetector-code which raises ONE result-Event...

B4X:
    public void scanBitmap(Bitmap b) {
    if(b != null){
        Frame frame = (new com.google.android.gms.vision.Frame.Builder()).setBitmap(b).build();
        SparseArray<TextBlock> textBlocks = detector.detect(frame);
        String blocks = "";
        String lines = "";
        String words = "";
   
        for (int index = 0; index < textBlocks.size(); index++) {
            //extract scanned text blocks here
            TextBlock tBlock = textBlocks.valueAt(index);
            blocks = blocks + tBlock.getValue() + "\n" + "\n";
            for (Text line : tBlock.getComponents()) {
                //extract scanned text lines here
                lines = lines + line.getValue() + "\n";
                for (Text element : line.getComponents()) {
                    //extract scanned text words here
                    words = words + element.getValue() + ", ";
                }
            }
        }   

        if(textBlocks.size() == 0) {
            ba.raiseEventFromDifferentThread(this, null, 0, eventName + "_result", true, new Object[] {"Scan Failed: Nothing to scan", "","",""});
        } else {
            ba.raiseEventFromDifferentThread(this, null, 0, eventName + "_result", true, new Object[] {"BLOCK",blocks, lines, words});
        }
    }
    }

You can try V1.1

PD: Thank you for your comments! Appreciated
 

Attachments

  • MobileVisionV1.1.zip
    15.5 KB · Views: 271
  • MobileVisionV1.11.zip
    15.5 KB · Views: 297
  • MobileVisionEx2.zip
    8.3 KB · Views: 337
  • MobileVisionEx.zip
    8.3 KB · Views: 276
Last edited:

DonManfred

Expert
Licensed User
Longtime User
B4X:
Sub btntxtrecognizer_Click
    Log("Start Recognize... Recognizer ready = "&txtrecognizer.isOperational)
    If txtrecognizer.isOperational Then
        'txtrecognizer.takePicture
        Dim bmp As Bitmap
        bmp.Initialize(File.DirAssets,"cdcover.png")
        txtrecognizer.scanBitmap(bmp)
    End If
End Sub
Sub TxtRecognizer_Result(info As String, blocks As String, lines As String, words As String)
    Log($"TxtRecognizer_Result(${info},${blocks},${lines},${words})"$)   
End Sub

*** Service (starter) Create ***
** Service (starter) Start **
** Activity (main) Create, isFirst = true **
** Activity (main) Resume **
Start Recognize... Recognizer ready = true
TxtRecognizer_Result(BLOCK,Produced by
Carl Stubner,
I
9) Don't Want To Know
I Gold Dust Woman -
3 Ncver Going Back Again . - matchbox20
4 Don't Stop - Elton John
S Go Your Own Way The Cranberris
6 Songbird - Duncan Sheik
7 The Chain - Shawn Colvin
You Makc Loving Fun - - Jcwel
Goo Goo Dolls
Oh Daddy Tallulah
Sister Hazel
,Produced by
Carl Stubner,
I
9) Don't Want To Know
I Gold Dust Woman -
3 Ncver Going Back Again . - matchbox20
4 Don't Stop - Elton John
S Go Your Own Way The Cranberris
6 Songbird - Duncan Sheik
7 The Chain - Shawn Colvin
You Makc Loving Fun - - Jcwel
Goo Goo Dolls
Oh Daddy Tallulah
Sister Hazel
,Produced, by, Carl, Stubner,, I, 9), Don't, Want, To, Know, I, Gold, Dust, Woman, -, 3, Ncver, Going, Back, Again, ., -, matchbox20, 4, Don't, Stop, -, Elton, John, S, Go, Your, Own, Way, The, Cranberris, 6, Songbird, -, Duncan, Sheik, 7, The, Chain, -, Shawn, Colvin, You, Makc, Loving, Fun, -, -, Jcwel, Goo, Goo, Dolls, Oh, Daddy, Tallulah, Sister, Hazel, )
** Activity (main) Pause, UserClosed = false **
** Activity (main) Resume **

Needs V1.11 as i found a little mistake in the wrapper...
 

Attachments

  • MobileVisionEx2.zip
    8.3 KB · Views: 303
  • MobileVisionV1.11.zip
    15.5 KB · Views: 276

DonManfred

Expert
Licensed User
Longtime User
Logger verbunden mit: 9885e6514556383552
--------- beginning of crash
--------- beginning of main
*** Service (starter) Create ***
** Service (starter) Start **
** Activity (main) Create, isFirst = true **
** Activity (main) Resume **
Start Recognize... Recognizer ready = true
TxtRecognizer_Result(BLOCK,Produced by
Carl Stubner,
I
9) Don't Want To Know
I Gold Dust Woman -
3 Ncver Going Back Again . - matchbox20
4 Don't Stop - Elton John
S Go Your Own Way The Cranberris
6 Songbird - Duncan Sheik
7 The Chain - Shawn Colvin
You Makc Loving Fun - - Jcwel
Goo Goo Dolls
Oh Daddy Tallulah
Sister Hazel
,Produced by
Carl Stubner,
I
9) Don't Want To Know
I Gold Dust Woman -
3 Ncver Going Back Again . - matchbox20
4 Don't Stop - Elton John
S Go Your Own Way The Cranberris
6 Songbird - Duncan Sheik
7 The Chain - Shawn Colvin
You Makc Loving Fun - - Jcwel
Goo Goo Dolls
Oh Daddy Tallulah
Sister Hazel
,Produced, by, Carl, Stubner,, I, 9), Don't, Want, To, Know, I, Gold, Dust, Woman, -, 3, Ncver, Going, Back, Again, ., -, matchbox20, 4, Don't, Stop, -, Elton, John, S, Go, Your, Own, Way, The, Cranberris, 6, Songbird, -, Duncan, Sheik, 7, The, Chain, -, Shawn, Colvin, You, Makc, Loving, Fun, -, -, Jcwel, Goo, Goo, Dolls, Oh, Daddy, Tallulah, Sister, Hazel, )
(MyMap) {Language=und, BBbottom=302, BBleft=121, BBright=196, BBtop=287, BoundingBox=Rect(121, 287 - 196, 302), CornerPoints=[Landroid.graphics.Point;@d4ef299}
(MyMap) {Language=und, BBbottom=349, BBleft=107, BBright=184, BBtop=337, BoundingBox=Rect(107, 337 - 184, 349), CornerPoints=[Landroid.graphics.Point;@3de195e}
(MyMap) {Language=und, BBbottom=65, BBleft=402, BBright=492, BBtop=7, BoundingBox=Rect(402, 7 - 492, 65), CornerPoints=[Landroid.graphics.Point;@dffa13f}
(MyMap) {Language=und, BBbottom=163, BBleft=40, BBright=203, BBtop=150, BoundingBox=Rect(40, 150 - 203, 163), CornerPoints=[Landroid.graphics.Point;@eb30a0c}
(MyMap) {Language=und, BBbottom=228, BBleft=45, BBright=184, BBtop=215, BoundingBox=Rect(45, 215 - 184, 228), CornerPoints=[Landroid.graphics.Point;@931f55}
(MyMap) {Language=und, BBbottom=145, BBleft=43, BBright=321, BBtop=18, BoundingBox=Rect(43, 18 - 321, 145), CornerPoints=[Landroid.graphics.Point;@2ee6d6a}
(MyMap) {Language=und, BBbottom=184, BBleft=52, BBright=173, BBtop=169, BoundingBox=Rect(52, 169 - 173, 184), CornerPoints=[Landroid.graphics.Point;@ed7125b}
(MyMap) {Language=und, BBbottom=209, BBleft=57, BBright=203, BBtop=188, BoundingBox=Rect(57, 188 - 203, 209), CornerPoints=[Landroid.graphics.Point;@68cbaf8}
(MyMap) {Language=und, BBbottom=249, BBleft=58, BBright=156, BBtop=234, BoundingBox=Rect(58, 234 - 156, 249), CornerPoints=[Landroid.graphics.Point;@eb35bd1}
** Activity (main) Pause, UserClosed = false **
** Activity (main) Resume **

Needs V1.2 (Updated Lib in Post #1)
- The Scanbitmap result event have one parameter more... A List with a Map for each scanresult.

B4X:
Sub btntxtrecognizer_Click
    Log("Start Recognize... Recognizer ready = "&txtrecognizer.isOperational)
    If txtrecognizer.isOperational Then
        'txtrecognizer.takePicture
        Dim bmp As Bitmap
        bmp.Initialize(File.DirAssets,"cdcover.png")
        txtrecognizer.scanBitmap(bmp)
    End If
End Sub
Sub TxtRecognizer_Result(info As String, blocks As String, lines As String, words As String, scanresults As List)
    Log($"TxtRecognizer_Result(${info},${blocks},${lines},${words})"$)  
    If scanresults.IsInitialized And scanresults.Size>0 Then
        For i = 0 To scanresults.Size-1
            Log(scanresults.Get(i))
        Next
    End If
End Sub
 
Top