Android Question vision.TextRecognizer.Builder

[SMOOTSARA]

Hi all friends ?

I need a library or sampel for [com.google.android.gms.vision.text.TextRecognizer.Builder]

com.google.android.gms.vision.text | Google Play services | Google for Developers

developers.google.com

In the following topic, there is a very good and practical example by @Erel

[B4X] [B4XPages] Barcode Reader

B4A + B4i barcode reader. Don't miss: - #PlistExtra in B4i. - Manifest code + #AdditionalJar in B4A Depends on BCToast: https://www.b4x.com/android/forum/threads/b4x-bctoast-cross-platform-custom-toast-message.111046/#content (BCToast will be an internal library in the near future) While the...

www.b4x.com

Can you help me make a text recognition instead of barcode recognition?

com.google.android.gms.vision.barcode | Google Play services | Google for Developers

developers.google.com

 

[Erel]

I made some tests and didn't get valid results.

You can try it. Changes based on that example:

Add to CameraEx class:

Public Sub SetPreviewFormat(Format As Int)
    r.target = parameters
    r.RunMethod2("setPreviewFormat", Format, "java.lang.int")
End Sub

Public Sub GetSupportedPreviewFormats As List
    r.target = parameters
    Dim list1 As List = r.RunMethod("getSupportedPreviewFormats")
    Return list1
End Sub

B4XMainPage:

Private Sub StartCamera
    rp.CheckAndRequest(rp.PERMISSION_CAMERA)
    Wait For B4XPage_PermissionResult (Permission As String, Result As Boolean)
    If Result = False Then
        toast.Show("No permission!")
        Return
    End If
    StartCameraShared
    camEx.Initialize(pnlPreview, False, Me, "Camera1")
    Wait For Camera1_Ready (Success As Boolean)
    If Success Then
        camEx.SetContinuousAutoFocus
        Log(camEx.GetSupportedPreviewFormats) 'ignore
        camEx.SetPreviewFormat(17) 'NV21
        camEx.CommitParameters
        camEx.StartPreview
    Else
        toast.Show("Error opening camera")
        StopCamera
    End If
End Sub

Private Sub CreateTextDetector
    Dim ctxt As JavaObject
    ctxt.InitializeContext
    Dim builder As JavaObject
    builder.InitializeNewInstance("com/google/android/gms/vision/text/TextRecognizer.Builder".Replace("/", "."), Array(ctxt))
    detector = builder.RunMethod("build", Null)
    Dim operational As Boolean = detector.RunMethod("isOperational", Null)
    If operational = False Then
        toast.Show("Failed to create detector")
    End If
End Sub

Private Sub Camera1_Preview (data() As Byte)
    If DateTime.Now > LastPreview + IntervalBetweenPreviewsMs Then
        'Dim n As Long = DateTime.Now
        Dim frameBuilder As JavaObject
        Dim bb As JavaObject
        bb = bb.InitializeStatic("java.nio.ByteBuffer").RunMethod("wrap", Array(data))
        frameBuilder.InitializeNewInstance("com/google/android/gms/vision/Frame.Builder".Replace("/", "."), Null)
        Dim cs As CameraSize = camEx.GetPreviewSize
        frameBuilder.RunMethod("setImageData", Array(bb, cs.Width, cs.Height,  17))
        Dim frame As JavaObject = frameBuilder.RunMethod("build", Null)
        Dim SparseArray As JavaObject = detector.RunMethod("detect", Array(frame))
        LastPreview = DateTime.Now
        Dim Matches As Int = SparseArray.RunMethod("size", Null)
        For i = 0 To Matches - 1
            Dim text As JavaObject = SparseArray.RunMethod("valueAt", Array(i))
            Log(text.RunMethod("getValue", Null))
        Next
    End If
End Sub

And call CreateTextDetector in B4XPage_Created.

 

[drgottjr]

i use the text recognizer (based on the above code) all the time.
i don't understand why erel says he didn't get valid results. i don't use pages (yet. a slow learning curve for me.), so my use is non-page and with his camex class).
in comparing the code above with the code in my app, the only difference i noticed has to do with frame orientation. as you probably know, ocr can't read anything tending towards upside down.
my case is, perhaps, special, because it involved a nexus 5 and its particular camera orientation. i also use the code on non-nexus 5 devices. in any case, below find my code relating to frame orientation:

            Dim rotation As Int = camex.PreviewOrientation
            Log("rotation: " & rotation)
            Dim newrotation As Int
            '      .setRotation(metadata.getRotation())
            Select rotation
                Case 0
                    newrotation = 0
                Case 180
                    newrotation = 2
                Case 90
                    newrotation = 1
                Case 270
                    newrotation = 3
            End Select
           
            If newrotation <> 0 Then
                frameBuilder.RunMethod("setRotation",Array(newrotation))          ' 3 = 270
            End If

it goes just before passing the frame off to the detector

Dim SparseArray As JavaObject = ocrdetector.RunMethod("detect", Array(frame))

to be clear, it doesn't matter what you think you see on the screen; the detector may see the same thing at 90 degrees (or more) offset and cannot detect the text.

if erel happens to visit this thread again, he might confirm whether or not he thinks frame orientation is relevant to his code. you, of course, are welcome to try the code yourself.

 

[SMOOTSARA]

Hi dear friend
Could you please share your code here?
Thank you

i use the text recognizer (based on the above code) all the time.
i don't understand why erel says he didn't get valid results. i don't use pages (yet. a slow learning curve for me.), so my use is non-page and with his camex class).
in comparing the code above with the code in my app, the only difference i noticed has to do with frame orientation. as you probably know, ocr can't read anything tending towards upside down.
my case is, perhaps, special, because it involved a nexus 5 and its particular camera orientation. i also use the code on non-nexus 5 devices. in any case, below find my code relating to frame orientation:

            Dim rotation As Int = camex.PreviewOrientation
            Log("rotation: " & rotation)
            Dim newrotation As Int
            '      .setRotation(metadata.getRotation())
            Select rotation
                Case 0
                    newrotation = 0
                Case 180
                    newrotation = 2
                Case 90
                    newrotation = 1
                Case 270
                    newrotation = 3
            End Select
          
            If newrotation <> 0 Then
                frameBuilder.RunMethod("setRotation",Array(newrotation))          ' 3 = 270
            End If

it goes just before passing the frame off to the detector

Dim SparseArray As JavaObject = ocrdetector.RunMethod("detect", Array(frame))

to be clear, it doesn't matter what you think you see on the screen; the detector may see the same thing at 90 degrees (or more) offset and cannot detect the text.

if erel happens to visit this thread again, he might confirm whether or not he thinks frame orientation is relevant to his code. you, of course, are welcome to try the code yourself.

 

[drgottjr]

it's the same as what is posted in reply #2. i posted my orientation code above. just insert it right before the

Dim SparseArray As JavaObject = ocrdetector.RunMethod("detect", Array(frame))

statement at line 45 in post #2. did you use the code in reply #2? do that and see if your results are the same as those of erel.
then you can insert my orientation snippet while waiting for erel to come back.

 

[Erel]

Might be indeed related to the orientation. I haven't checked it too much.

 

[SMOOTSARA]

hi

I turn on the device flashlight to make the TEXT look better

But when the start => " sub StartCamera"
The flashlight turns off automatically

I made some tests and didn't get valid results.

You can try it. Changes based on that example:

Add to CameraEx class:

Public Sub SetPreviewFormat(Format As Int)
    r.target = parameters
    r.RunMethod2("setPreviewFormat", Format, "java.lang.int")
End Sub

Public Sub GetSupportedPreviewFormats As List
    r.target = parameters
    Dim list1 As List = r.RunMethod("getSupportedPreviewFormats")
    Return list1
End Sub

B4XMainPage:

Private Sub StartCamera
    rp.CheckAndRequest(rp.PERMISSION_CAMERA)
    Wait For B4XPage_PermissionResult (Permission As String, Result As Boolean)
    If Result = False Then
        toast.Show("No permission!")
        Return
    End If
    StartCameraShared
    camEx.Initialize(pnlPreview, False, Me, "Camera1")
    Wait For Camera1_Ready (Success As Boolean)
    If Success Then
        camEx.SetContinuousAutoFocus
        Log(camEx.GetSupportedPreviewFormats) 'ignore
        camEx.SetPreviewFormat(17) 'NV21
        camEx.CommitParameters
        camEx.StartPreview
    Else
        toast.Show("Error opening camera")
        StopCamera
    End If
End Sub

Private Sub CreateTextDetector
    Dim ctxt As JavaObject
    ctxt.InitializeContext
    Dim builder As JavaObject
    builder.InitializeNewInstance("com/google/android/gms/vision/text/TextRecognizer.Builder".Replace("/", "."), Array(ctxt))
    detector = builder.RunMethod("build", Null)
    Dim operational As Boolean = detector.RunMethod("isOperational", Null)
    If operational = False Then
        toast.Show("Failed to create detector")
    End If
End Sub

Private Sub Camera1_Preview (data() As Byte)
    If DateTime.Now > LastPreview + IntervalBetweenPreviewsMs Then
        'Dim n As Long = DateTime.Now
        Dim frameBuilder As JavaObject
        Dim bb As JavaObject
        bb = bb.InitializeStatic("java.nio.ByteBuffer").RunMethod("wrap", Array(data))
        frameBuilder.InitializeNewInstance("com/google/android/gms/vision/Frame.Builder".Replace("/", "."), Null)
        Dim cs As CameraSize = camEx.GetPreviewSize
        frameBuilder.RunMethod("setImageData", Array(bb, cs.Width, cs.Height,  17))
        Dim frame As JavaObject = frameBuilder.RunMethod("build", Null)
        Dim SparseArray As JavaObject = detector.RunMethod("detect", Array(frame))
        LastPreview = DateTime.Now
        Dim Matches As Int = SparseArray.RunMethod("size", Null)
        For i = 0 To Matches - 1
            Dim text As JavaObject = SparseArray.RunMethod("valueAt", Array(i))
            Log(text.RunMethod("getValue", Null))
        Next
    End If
End Sub

And call CreateTextDetector in B4XPage_Created.

 

[drgottjr]

you can turn on the flashlight (not flash mode) programatically in startcamera sub. unless you are programming in the dark, you shouldn't have to use the camera's flashlight. the effect (for ocr) is almost as bad as no light. if you have a recent device, its low-light performance is very good (if you hold the camera steady).

and, as you have seen, erel has suggested that orientation might be the problem. add my code to the previewready sub.

i don't remember which app of mine included turning on the flashlight, sorry. i'm pretty sure it's one of the settings you adjust in the camex class, just like any other setting.

 

[drgottjr]

it's called "torch". i found my code. see if you can figure out how to use it. basically, you have to make sure you have a "torch". then you can make a button to turn it on/off

       flashmodes = camex.GetSupportedFlashModes
        If flashmodes.IsInitialized Then    ' they may not have any flash...
            Dim i As Int
            For Each m As String In flashmodes
                If m.CompareTo("torch") = 0 Then
                    hasTorch = True
                    torchbutton.Visible = True
                    Exit
                End If
            Next




Sub torch_click
    If hasTorch = False Then
        Return
    End If

    ' get current flash mode.  if off, turn on torch else if on, turn torch off
    Dim mode As String = camex.GetFlashMode
    If mode.CompareTo("off") = 0 Then
        camex.SetFlashMode("torch")
    Else
        camex.SetFlashMode("off")
    End If
    
    camex.CommitParameters     ' which should include picturesize and previewsize
    
End Sub

it worked for me, but i don't think you should use it for ready things close up. the "torch" washes everything out. it's a very strong light at a close distance. that's why i stopped using it.

 

[SMOOTSARA]

i use the text recognizer (based on the above code) all the time.
i don't understand why erel says he didn't get valid results. i don't use pages (yet. a slow learning curve for me.), so my use is non-page and with his camex class).
in comparing the code above with the code in my app, the only difference i noticed has to do with frame orientation. as you probably know, ocr can't read anything tending towards upside down.
my case is, perhaps, special, because it involved a nexus 5 and its particular camera orientation. i also use the code on non-nexus 5 devices. in any case, below find my code relating to frame orientation:

            Dim rotation As Int = camex.PreviewOrientation
            Log("rotation: " & rotation)
            Dim newrotation As Int
            '      .setRotation(metadata.getRotation())
            Select rotation
                Case 0
                    newrotation = 0
                Case 180
                    newrotation = 2
                Case 90
                    newrotation = 1
                Case 270
                    newrotation = 3
            End Select
          
            If newrotation <> 0 Then
                frameBuilder.RunMethod("setRotation",Array(newrotation))          ' 3 = 270
            End If

it goes just before passing the frame off to the detector

Dim SparseArray As JavaObject = ocrdetector.RunMethod("detect", Array(frame))

to be clear, it doesn't matter what you think you see on the screen; the detector may see the same thing at 90 degrees (or more) offset and cannot detect the text.

if erel happens to visit this thread again, he might confirm whether or not he thinks frame orientation is relevant to his code. you, of course, are welcome to try the code yourself.

Hi dear friend
I found the source of the Earl code
Good luck

https://developer.android.com/reference/android/hardware/Camera?hl=fr#setDisplayOrientation(int)

 

[SMOOTSARA]

Hello friends, can you help me solve the following problem?

I need model number 1 --- That is, both the device and the text are in portrait mode

But I do not know which codes I should change ? ?

 

[SMOOTSARA]

Hey guys

After the release of the program, I noticed user feedback

Numerous users have said they do not recognize texts


For example devices :

SAMSUNG Galaxy g6 android 10
SAMSUNG note 10 plus android 10
google oneplus 6 android 10
asua 4zen android 8.1


Huawei honor 8X android 10
SAMSUNG Galaxy A30 android 10
samsung galaxy note 8 android 9
Huawei Nova 3e android 9



After multiple controls on the real device

I noticed there is an error in this section [[ Failed to create detector ]]
How do I know why this is a problem?

Thank you for your guidance??

Private Sub CreateDetector
    LogColor("Sub CreateDetector ",Colors.Magenta)

    Dim ctxt As JavaObject
    ctxt.InitializeContext
    Dim builder As JavaObject
    
    builder.InitializeNewInstance("com/google/android/gms/vision/text/TextRecognizer.Builder".Replace("/", "."), Array(ctxt))
    detector_java = builder.RunMethod("build", Null)
    
    Dim operational As Boolean = detector_java.RunMethod("isOperational", Null)
    
    
    If operational = False Then
        LogColor("Failed to create detector",Colors.Red)
        ToastMessageShow("Failed to create detector",True)
    Else
        LogColor("create detector",Colors.Green)
        ToastMessageShow("create detector",True)
    End If
    
    
End Sub

 

[SMOOTSARA]

Hey guys

After the release of the program, I noticed user feedback

Numerous users have said they do not recognize texts


For example devices :

SAMSUNG Galaxy g6 android 10
SAMSUNG note 10 plus android 10
google oneplus 6 android 10
asua 4zen android 8.1


Huawei honor 8X android 10
SAMSUNG Galaxy A30 android 10
samsung galaxy note 8 android 9
Huawei Nova 3e android 9



After multiple controls on the real device

I noticed there is an error in this section [[ Failed to create detector ]]
How do I know why this is a problem?

Thank you for your guidance??

Private Sub CreateDetector
    LogColor("Sub CreateDetector ",Colors.Magenta)

    Dim ctxt As JavaObject
    ctxt.InitializeContext
    Dim builder As JavaObject
   
    builder.InitializeNewInstance("com/google/android/gms/vision/text/TextRecognizer.Builder".Replace("/", "."), Array(ctxt))
    detector_java = builder.RunMethod("build", Null)
   
    Dim operational As Boolean = detector_java.RunMethod("isOperational", Null)
   
   
    If operational = False Then
        LogColor("Failed to create detector",Colors.Red)
        ToastMessageShow("Failed to create detector",True)
    Else
        LogColor("create detector",Colors.Green)
        ToastMessageShow("create detector",True)
    End If
   
   
End Sub

Hello friends

I researched and came to the conclusion that we need to check that all the necessary files are installed

According to this guide

Detector | Google Play services | Google for Developers

developers.google.com

If these files are not downloaded, how can we try again?