Android Question [B4X] [B4A] Add OCR features

[GMan]

Based on the 2013 project from @Erel (found here: https://www.b4x.com/android/forum/t...r-features-to-your-android-application.27080/ we tried to make this sample word, but without success.

Mentioned that the code is rather old and many things changed since that time we tried several solutions to make it work.
At last removed the httpJob.bas and the HttpUtils2.bas (replaced the last with OKHTTPUtils2) as mentioned also by Erel brings no other result.
Depending on the qualtity more or less charcters are recognized.
@DonManfred also tried to get it work with some success: it works, but maybe the Online-Part has issues
<a href="https://www.b4x.com/android/forum/t...-android-application.27080/page-2#post-656725">[Example] Add OCR features to your Android application</a>

Maybe the one or other has tried it in the past or has another solution

 

[Erel]

@DonManfred also tried to get it work with some success: it works, but maybe the Online-Part has issues

What are the problems?

 

[GMan]

Recognition of the numbers and the text isnt good enough...for the text it doesnt really matter, but if a 5 is recongnized as a 3 its not so good.
I am playing with different resolutions when taking a picture, about 760 x 1280 the most letters and numbers are scanned correctly.

Also its depending of the ligth...scanning the bill in an "ambient" restaurant should not work very serious ;-)

 

[GMan]

i am using this "bill-like-printed" paper (VERY old skool thermo printer ;-) ) from a german post office (an international company)
Here is one result (from the Log-entry):

{"ErrorMessage":"","OutputInformation":null,"AvailablePages":14,"ProcessedPages":1,"OCRText":[["in1iefc:rrngslreleg 1 Quittung itte Beleg 0ut aufbewahren eutsche Post AG 7918 1'örtisvorst 120261.49 7301 09.04.2019 15:34 3endungsnumrner : Empfangsland: E Irrt Gesamtumsatz (Brutto) Zahlbetrag: Servicenummer International 0228 4333112 Mo-Fr•8 00 - 18'00 Uhr RC326592082DE FR Internet www deut:ychepoit. delbraefsiatus Vielen Dank für Ihren Besuch. Ihre Deutsche Post AG *0,00 EUR *0,00 EUR "]],"OutputFileUrl":"","OutputFileUrl2":"","OutputFileUrl3":"","Reserved":[],"OCRWords":[],"TaskDescription":null}

After some teste i see, that the number of PAGES changes partly significant....as i understand it, the less the PAGES counter is the better is the result in recognition.

Another result with sunligth on the sheet:

{"ErrorMessage":"","OutputInformation":null,"AvailablePages":14,"ProcessedPages":1,"OCRText":[["in1iefc:rrngslreleg 1 Quittung itte Beleg 0ut aufbewahren eutsche Post AG 7918 1'örtisvorst 120261.49 7301 09.04.2019 15:34 3endungsnumrner : Empfangsland: E Irrt Gesamtumsatz (Brutto) Zahlbetrag: Servicenummer International 0228 4333112 Mo-Fr•8 00 - 18'00 Uhr RC326592082DE FR Internet www deut:ychepoit. delbraefsiatus Vielen Dank für Ihren Besuch. Ihre Deutsche Post AG *0,00 EUR *0,00 EUR "]],"OutputFileUrl":"","OutputFileUrl2":"","OutputFileUrl3":"","Reserved":[],"OCRWords":[],"TaskDescription":null}

 

[DonManfred]

Did you tried to add/change the given Parameters to the Rest-Api?

Check the Documentation here: http://www.ocrwebservice.com/api/restguide
Maybe you need to define a Zone or use a pagerange?

 

[GMan]

Tried 1st the parameter getWords=false (no recognition for "words" (as i understand).

job.PostBytes("https://www.ocrwebservice.com/restservices/processDocument?gettext=true&getWord=false@language=german,english", ReadFile(VideoFileDir, "1.jpg"))

But the result returns text also:

{"ErrorMessage":"","OutputInformation":null,"AvailablePages":19,"ProcessedPages":1,"OCRText":[["Ei41 eferongsbe leg I Guittung Bite Bel eg gut aufbewahren Deutsche Posk AG 47918 TOntsvorst 82026149 7301 08.04.2019 15:34 Sendungsnumer : Empfangsland : E lot RC326592082DE FR Gesasi tuutsatz (Brutto) Zahlbetrag: Servioenummer International 0228 4333112 Mo-Fr 800 - 18:00 Uhr lie net veee doutwhemt de/briet$totu,' Vi &ten Dank fur Ihren Besuch. Ihre Deutsche Post AG *0,00 FUR *0,00 FUR "]],"OutputFileUrl":"","OutputFileUrl2":"","OutputFileUrl3":"","Reserved":[],"OCRWords":[],"TaskDescription":null}

 

[GMan]

I,ll define zones, but makes not really sense: every coffeeshop prints hins own "design"...date on top, date on bottom, total on top, total on bottom etc etc
May be a solution to aks the user (just when scanning the bill) WHICH fields/areas (per tapping) should be scanned

 

[DonManfred]

&getWord=false@language=german,english

1. The parametername is "getwords"!
2. You should not use a @ here at all.

job.PostBytes("https://www.ocrwebservice.com/restservices/processDocument?gettext=true&getwords=false&language=german", ReadFile(VideoFileDir, "1.jpg"))

 

[DonManfred]

every coffeeshop prints hins own "design"...date on top, date on bottom, total on top, total on bottom etc etc

Sure, you are right. I guess you need to play around with it.
Knowing the Positions of each "Block" you could define base parameters and do a few OCRs on each bill.
I don´t know if you can get the touch position using the cameraex2 example and whether you can build a Rect which then could be scanned by the restapi. you need to get the right Zone(s) for the touch Rect.

I guess defining one or more Zones can help you getting better results. But you need to define the Zones first for sure.

Specifies the region on the image for zonal OCR. The coordinates in pixels relative to the left top corner in the following format: top:left:height:width. This parameter can contain several zones separated with commas, for example "zone=0:0:100:100,50:50:50:50"

I can not really help here as i yesterday just created a trial account and tried to get the old example working. I totally rewrote it to use the Rest Api and working with okhttputils2. Finally i got it working (not only partially; It is working. But i´m not resposible for the results you get from this webapi ;-))

PD: If you know a better alternative which offers a Restapi and a trialaccount i can help you get it running.

 

[GMan]

Yeah, seems the rigth way (at this time).

Could be a dialog step-by-step system with a fixed zone:

1. scan company label
2. scan bill date
3. scan bill number
4. etc

I played aroud with the zones...this gives a reliable result after all:
in this case the tracking number MUST be scanned 100% correct - and it was.

Using this:

job.PostBytes("https://www.ocrwebservice.com/restservices/processDocument?gettext=true&getWord=false@language=german,english@outputFormat=txt@zone=150:230:160:50", ReadFile(VideoFileDir, "1.jpg"))

creates a small rectangle which was scanned correct.

btw: in the returned content the text is always shown, even it getWord=false is set ;-)

The last result:

{"ErrorMessage":"","OutputInformation":null,"AvailablePages":9,"ProcessedPages":1,"OCRText":[["EiRlieferungsbeleg I Quitting Bitte Beleg gut aufbewahren Deutsche Post AG 47918 Tenisvorst 82026149 7301 08.04.2019 15:34 Gendungsnummer: RC3265920820E Empfangsland: FR 1/g1 E Int Gesamtumsatz (Brutto) Zahlbetrag: Servicenummer International 0228 4333112 Mo-Fr 8 00 - 18 00 Uhr *0,00 EUR *0,00 EUR hitomet eUtU deobuhupuit do/brietstolus Vielen Dank fiir Ihren Besuch. Ihre Deutsche Post AG I "]],"OutputFileUrl":"","OutputFileUrl2":"","OutputFileUrl3":"","Reserved":[],"OCRWords":[],"TaskDescription":null}

 

[DonManfred]

Here is another OCR Rest Api: https://ocr.space/ocrapi
They offer a free account too.

Here a example using this Api

    Dim bytes() As Byte = ReadFile(File.DirAssets, "IMG_20190414_102203.jpg")
    Dim base64 As String = su.EncodeBase64(bytes)
 
    Dim job As HttpJob
    job.Initialize("ocr", Me)
    Dim m As Map = CreateMap("language": "ger", "isOverlayRequired": True,"filetype":"jpg","isTable":True,"base64Image": $"data:image/jpeg;base64,${base64}"$)
    job.PostMultipart($"https://api.ocr.space/parse/image"$,m,Null)
    job.GetRequest.SetHeader("apikey", "yourapikey") ' get you own apikey and use it here
    Wait For (j) JobDone(j As HttpJob)
    If j.Success Then
        'File.WriteString(File.DirRootExternal, "JobResult.txt", j.GetString)
        Log(j.GetString)
    Else
        Log(j.ErrorMessage)
    End If
    j.Release

Especialy the parameter isTable = true is something you could need. Also isOverlayRequired to request the bounds of each result.

In this case i did uploaded the image you posted here. "Die Quittung von der Post"

This is the resulting JSON

{"ParsedResults":[{"TextOverlay":{"Lines":[{"Words":[{"WordText":"Bitte","Left":70.0,"Top":61.0,"Height":17.0,"Width":46.0},{"WordText":"Beleg","Left":128.0,"Top":59.0,"Height":18.0,"Width":49.0}],"MaxHeight":18.0,"MinTop":59.0},{"Words":[{"WordText":"Deutsche","Left":66.0,"Top":127.0,"Height":18.0,"Width":76.0},{"WordText":"Post","Left":155.0,"Top":126.0,"Height":15.0,"Width":37.0},{"WordText":"AG","Left":204.0,"Top":123.0,"Height":18.0,"Width":20.0}],"MaxHeight":18.0,"MinTop":123.0},{"Words":[{"WordText":"47918","Left":64.0,"Top":148.0,"Height":18.0,"Width":47.0},{"WordText":"Tönisvorst","Left":123.0,"Top":148.0,"Height":17.0,"Width":101.0}],"MaxHeight":18.0,"MinTop":148.0},{"Words":[{"WordText":"82026149","Left":62.0,"Top":170.0,"Height":19.0,"Width":78.0},{"WordText":"1301","Left":165.0,"Top":170.0,"Height":19.0,"Width":37.0},{"WordText":"OH","Left":225.0,"Top":173.0,"Height":16.0,"Width":19.0},{"WordText":"04","Left":256.0,"Top":174.0,"Height":16.0,"Width":19.0},{"WordText":"?019","Left":288.0,"Top":174.0,"Height":17.0,"Width":39.0},{"WordText":"34","Left":381.0,"Top":176.0,"Height":17.0,"Width":18.0}],"MaxHeight":19.0,"MinTop":170.0},{"Words":[{"WordText":"tange","Left":88.0,"Top":293.0,"Height":21.0,"Width":50.0},{"WordText":"land.","Left":142.0,"Top":294.0,"Height":18.0,"Width":46.0}],"MaxHeight":21.0,"MinTop":293.0},{"Words":[{"WordText":"Int","Left":75.0,"Top":342.0,"Height":18.0,"Width":30.0}],"MaxHeight":18.0,"MinTop":342.0},{"Words":[{"WordText":"Gesatunsatz","Left":53.0,"Top":448.0,"Height":19.0,"Width":129.0},{"WordText":"(Brutto)","Left":197.0,"Top":450.0,"Height":22.0,"Width":82.0}],"MaxHeight":22.0,"MinTop":448.0},{"Words":[{"WordText":"Zilbetrag:","Left":51.0,"Top":475.0,"Height":23.0,"Width":115.0}],"MaxHeight":23.0,"MinTop":475.0},{"Words":[{"WordText":"Serncenu•ner","Left":47.0,"Top":538.0,"Height":17.0,"Width":110.0},{"WordText":"International","Left":167.0,"Top":539.0,"Height":18.0,"Width":104.0}],"MaxHeight":18.0,"MinTop":538.0},{"Words":[{"WordText":"0228","Left":46.0,"Top":560.0,"Height":16.0,"Width":33.0},{"WordText":"4333112","Left":88.0,"Top":560.0,"Height":16.0,"Width":60.0}],"MaxHeight":16.0,"MinTop":560.0},{"Words":[{"WordText":"800","Left":104.0,"Top":582.0,"Height":17.0,"Width":33.0},{"WordText":"-","Left":148.0,"Top":591.0,"Height":2.0,"Width":6.0},{"WordText":"Uhr","Left":213.0,"Top":583.0,"Height":15.0,"Width":23.0}],"MaxHeight":17.0,"MinTop":582.0},{"Words":[{"WordText":"Vielen","Left":39.0,"Top":705.0,"Height":19.0,"Width":65.0},{"WordText":"Dank","Left":118.0,"Top":702.0,"Height":21.0,"Width":42.0},{"WordText":"für","Left":172.0,"Top":701.0,"Height":18.0,"Width":31.0},{"WordText":"Ihren","Left":216.0,"Top":701.0,"Height":19.0,"Width":51.0},{"WordText":"Besuch","Left":279.0,"Top":702.0,"Height":20.0,"Width":64.0}],"MaxHeight":21.0,"MinTop":701.0},{"Words":[{"WordText":"Ihre","Left":38.0,"Top":731.0,"Height":21.0,"Width":42.0},{"WordText":"Deutsche","Left":94.0,"Top":727.0,"Height":24.0,"Width":86.0},{"WordText":"Post","Left":193.0,"Top":727.0,"Height":20.0,"Width":42.0},{"WordText":"AG","Left":247.0,"Top":728.0,"Height":19.0,"Width":21.0}],"MaxHeight":24.0,"MinTop":727.0},{"Words":[{"WordText":"RC326592082DE","Left":368.0,"Top":271.0,"Height":19.0,"Width":133.0}],"MaxHeight":19.0,"MinTop":271.0},{"Words":[{"WordText":"EUR","Left":468.0,"Top":454.0,"Height":18.0,"Width":32.0}],"MaxHeight":18.0,"MinTop":454.0},{"Words":[{"WordText":"EUR","Left":468.0,"Top":482.0,"Height":19.0,"Width":33.0}],"MaxHeight":19.0,"MinTop":482.0}],"HasOverlay":true,"Message":"Total lines: 16"},"TextOrientation":"0","FileParseExitCode":1,"ParsedText":"Bitte Beleg \t\r\nDeutsche Post AG \t\r\n47918 Tönisvorst \t\r\n82026149 1301 OH 04 ?019 34 \t\r\nRC326592082DE \t\r\ntange land. \t\r\nInt \t\r\nGesatunsatz (Brutto) \tEUR \t\r\nZilbetrag: \tEUR \t\r\nSerncenu•ner International \t\r\n0228 4333112 \t\r\n800 - Uhr \t\r\nVielen Dank für Ihren Besuch \t\r\nIhre Deutsche Post AG \t\r\n","ErrorMessage":"","ErrorDetails":""}],"OCRExitCode":1,"IsErroredOnProcessing":false,"ProcessingTimeInMilliseconds":"692","SearchablePDFURL":"Searchable PDF not generated as it was not requested."}

This is the IDno

All in all i think this Api is more approbiate.
Give it a try with your Bills...

 

[GMan]

Thx @DonManfred - i will give (but tomorrow)
nice sunday evening

 

[GMan]

QuickTip: saw this just on Twitter