B4A Library OpenCV 3.x

OpenCV (Open Source Computer Vision Library) is a really huge project/framework actively developed, mainly written in C++ . It is released under a BSD license.

Read more here: http://opencv.org/​

OpenCV versions: https://opencv.org/releases/​

OpenCV library for B4A: wraps the official OpenCV 3.x release for Android (in fact not all, about 95% of it)

• Feel free to test and use it. You can even
• License: You can use it for your projects, but you are not allowed to distribute nor sell this library. Of course you can distribute apps that use it (remember that OpenCV itself has BSD license as stated before)
• Supported hardware is: armeabi-v7a and arm64-v8
• Versions
  • 1.04 (2020/05/17)
    • This version wraps OpenCV 3.4.1 Android release and fixes some bugs (mostly some instance methods and some not exposed classes) of the previous version.
    • One of the major additions is the DNNmodule and related classes.
      • LINK to example (too large to be uploaded here)
      • Result video
    • Link to the B4A library files: HERE
  • 1.00 (2017/09/27)
    • First B4A library wrapper (OpenCv320forB4A V1.00) which replicates (95%) the official OpenCV 3.20 Java API for Android.
    • (removed link. Use 1.04)

• Please note that my support will be limited to issues with the wrapper itself, not to help translating OpenCV code from other languages to B4A

========================================================================

(There may be some inaccuracies in what I expose in this post, related to the OpenCV project, since I am relatively new to it and don't know all about its internals. If you find any, please let me know and I will correct it)

A bit of explanation

Spoiler

There exist 'official' OpenCV wrappers for different languages and platforms. Android is one of them.
The official OpenCV 3.20 for Android API includes a lot of classes, organized in modules. But it does not include "all" the original OpenCV modules (since there are other 'experimental', non-free, or platform specific modules which may be present in other platforms but not for Android). Also, there are build options to "tune" it...

I have played quite a lot with it this last year, with a huge project which I started with inline Java, and also translating examples or testing features. But what I have used is just a small percent of the classes and methods exposed. So, there may be some (let's hope not too many) things to fix.

How to learn OpenCVforB4A

Spoiler

If you have worked before with OpenCV, the learning curve will be easy.
If it was using Java with OpenCV for Android, then it will be immediate, since all the methods have exactly the same syntax (except for initializers, polimorphism, and some special cases where simply I did my best).
Anyhow, the ways that I can think of, are (will add links later, also suggestions as online tutorials,.. are welcome)

• Attached examples.
• B4A OpenCV Tutorials. I will write a couple of them with what I consider the most important building pieces (for instance the Mat class, which in B4A is OCVMat) and modules
• Internet examples: there are A LOT of examples over there, written in C++, Java, Python, JavaCV. I would look for examples in the language that is easier to understand for you and then try to translate. Some tips about it (based on my experience)

• OpenCV syntax has changed as new versions. So there is an 'old' syntax in which nearly everything started with "cv...". Since version 3.X, there was 'cleaner' organization (project was written in C++ instead of C), and there were major syntax changes.
• JavaCV: Translating from JavaCV to OpenCV should be quite easy but not always direct. JavaCV uses a mix of the old OpenCV syntax with some of its own, and at the beginning it can be a bit confusing, but then it is also easy.
• Python: there is a lot of material...

First steps. Prepare for some crashes...

Spoiler

• In OpenCV nearly everything takes part in the native code.
• When we call a Sub/method/algorithm, it performs some internal checks to see if all the input data is correct. This check is perfomed in the native side. If something is not correct (wrong OCVMat dimensions, some incoherent parameters,...) it throws an exception and crashes. If we are lucky, perhaps we see in the log some clues about the check that made it crash.
• On the good side, it is very easy to achieve results with OpenCV (check the examples). The real difficult part, as with many other things, is to fine-tune it: OpenCV has a collection of really powerful 'primitive' objects and operations, and really complex algorithms that can do many things. But it is the user who has to glue all of them to achieve the desired results.

(from the previous Beta announcement)

Spoiler

• IMPORTANT: you must take this into account:

• OpenCV (the included binary modules) is a free(*) project, but subject to license terms as described here: http://opencv.org/
  • (*): There are some modules in the OpenCV project which are on-free, but here I am refering to the ones included in the library
• My work: (the B4A library) is free to test and use, but you can for it . I'll keep donators updated with "advanced" material and examples
• If you are interested, please PM me with your mail address and I will send you a link with the library and some basic examples. (be patient if you don't receive it immediately, I'll do it as soon as possible).

• There is no documentation. In short, the syntax is nearly-exactly the same as the OpenCV3.20 Java API, adding "OCV" prefix and only the minimum modifications to adapt to B4A, For reference (taking into account described syntax changes) you can look at http://docs.opencv.org/java/3.1.0/ (which is not the latest one, but the API is nearly the same).
• It would be preferable if you have worked before with OpenCV and/or can translate examples from Java/C++ and/or simply are interested in it.
• I recommend starting with the examples and try to understand what is done. Just experimenting can lead to crash after crash of the native libraries with nearly no useful information, and can be very discouraging.
• I forgot, the included binaries are for ameabi-v7a and arm64-v8 devices

---------------------------------------------------------------------
Some screenshots taken from the examples

Spoiler

Canny operator - Features2D - Color space conversion

Spoiler

2D-FFT

Spoiler

Color Blob detection

Spoiler

 

[biometrics]

Dim mImgProc as OCVImgProc
mImgProc.matchTemplate(.....)

That must be it, thanks!

Will certainly send you a donation when I get this done.

 

[biometrics]

Hi Jordi,

Please help port this line of C++ code. I need to read the pixel value at a location of an OCVMat:

float data = image.at<float>(y, x);

I've tried:

Dim data As Double
data = image.get5(y, x)

I get this error:

Compiling generated Java code.  Error
B4A line: 413
data = image.get5(y, x)
javac 1.8.0_131
src\isenzo\audiencemeasurement\main.java:1018: error: incompatible types: double[] cannot be converted to double
_data = (double)(_image.get5(_y,_x));

But if I change it to an array:

Dim data(1) As Double
data(0) = image.get5(y, x)

I then get this error:

Compiling generated Java code.  Error
B4A line: 413
data(0) = image.get5(y, x)
javac 1.8.0_131
src\isenzo\audiencemeasurement\main.java:1019: error: incompatible types: double[] cannot be converted to double
_data[(int) (0)] = (double)(_image.get5(_y,_x));
  ^

 

[JordiCP]

Try with

Dim data(1) as Double
image.get4(y,x,data)

I must look exactly what get5 does, and why it had a different syntax (if it is a typo of mine or there was a reason for it)

Anyway, what kind of OCVMat is Image? What does it hold? Can you log the value of image.type, image.depth and image.channels?

 

[biometrics]

Try with

Dim data(1) as Double
image.get4(y,x,data)

I must look exactly what get5 does, and why it had a different syntax (if it is a typo of mine or there was a reason for it)

Anyway, what kind of OCVMat is Image? What does it hold? Can you log the value of image.type, image.depth and image.channels?

Hi Jordi,

It compiles now. But admittedly this is the first pass of the porting so any number of things could be wrong. That said I logged as you requested:

Dim image As OCVMat 'actually passed in as a parameter on the sub call
Dim data(1) As Double
Dim iDummy As Int

Log(image.type)
Log(image.depth)
Log(image.channels)
iDummy = image.get4(y, x, data)

Log result is:
5
5
1

And now the error on the image.get line is:

Error occurred on line: 462 (Main)
java.lang.UnsupportedOperationException: Mat data type is not compatible: 5
   at com.appiotic.ocv4b4a.core.Mat.get(Mat.java:1150)
   at com.appiotic.ocv4b4a.core.Mat.get4(Mat.java:1152)
   at java.lang.reflect.Method.invoke(Native Method)
   at anywheresoftware.b4a.shell.Shell.runMethod(Shell.java:710)
   at anywheresoftware.b4a.shell.Shell.raiseEventImpl(Shell.java:339)
   at anywheresoftware.b4a.shell.Shell.raiseEvent(Shell.java:249)
   at java.lang.reflect.Method.invoke(Native Method)
   at anywheresoftware.b4a.ShellBA.raiseEvent2(ShellBA.java:139)
   at anywheresoftware.b4a.BA.raiseEvent2(BA.java:170)
   at anywheresoftware.b4a.debug.Debug.delegate(Debug.java:259)
   at isenzo.audiencemeasurement.main._multiplemaxloc(main.java:971)
   at isenzo.audiencemeasurement.main._fastmatchtemplate(main.java:810)
   at isenzo.audiencemeasurement.main._libopencvcameraview_newframe(main.java:1268)
   at java.lang.reflect.Method.invoke(Native Method)
   at anywheresoftware.b4a.shell.Shell.runMethod(Shell.java:710)
   at anywheresoftware.b4a.shell.Shell.raiseEventImpl(Shell.java:342)
   at anywheresoftware.b4a.shell.Shell.raiseEvent(Shell.java:249)
   at java.lang.reflect.Method.invoke(Native Method)
   at anywheresoftware.b4a.ShellBA.raiseEvent2(ShellBA.java:139)
   at anywheresoftware.b4a.BA.raiseEvent2(BA.java:170)
   at anywheresoftware.b4a.BA.raiseEvent(BA.java:166)
   at com.appiotic.ocv4b4a.android.JavaCameraView.rapidDebugModeCameraWorker(JavaCameraView.java:495)
   at com.appiotic.ocv4b4a.android.JavaCameraView.onPreviewFrame(JavaCameraView.java:387)
   at android.hardware.Camera$EventHandler.handleMessage(Camera.java:1136)
   at android.os.Handler.dispatchMessage(Handler.java:102)
   at android.os.Looper.loop(Looper.java:234)
   at android.app.ActivityThread.main(ActivityThread.java:5526)
   at java.lang.reflect.Method.invoke(Native Method)
   at com.android.internal.os.ZygoteInit$MethodAndArgsCaller.run(ZygoteInit.java:726)
   at com.android.internal.os.ZygoteInit.main(ZygoteInit.java:616)

On Monday I'll need to go over everything again to make sure the porting is correct, so unless you see an obvious problem, it's probably best to wait for me to review my changes.

 

[biometrics]

Btw, is this version of minMaxLoc available? http://docs.opencv.org/3.2.0/d2/de8/group__core__array.html#ga8873b86a29c5af51cafdcee82f8150a7

 

[JordiCP]

Seems that your OCVMat (type=5, see HERE) holds Floats. So, I think the correct one should be

   Dim image As OCVMat 'actually passed in as a parameter on the sub call
   Dim data(1) As Float
   Dim iDummy As Int
   iDummy = image.get3(y, x, data)    'get3 is the variant for floats.

About the minmaxLoc function, there are 2 variants: minmaxLoc and minmaxLoc1, which do the same as the C++ version, but the syntax is different because in Java you can not pass pointers to primitive types, so the results are in a OCVMinMaxLocResult class
So, the syntax would be

   Dim mMinMaxLocResult AS OCVMinMaxLocResult
   mMinMaxLocResult = mCore.MinMaxLoc(myMat,myMask)
   'or, if you don't use any mask
  mMinMaxLocResult = mCore.MinMaxLoc1(myMat)

The problem is that I forgot to expose the OCVMinMaxLocResult class , so it can't be used with release 1.0. I'll build a test release of the lib an send it so that you can test.

--EDIT--
Much easier. Just edit your OpenCV320forB4A.xml file and search for line

  <name>com.appiotic.ocv4b4a.core.Core.MinMaxLocResult</name>

and add this one below

  <shortname>OCVMinMaxLocResult</shortname>

 

[biometrics]

Yippee, it's working! Thanks so much Jordi. I'll arrange a donation this week. I really appreciate your effort.

It's a bit slow but I need need to go over the code again to make sure I ported it correctly. It's certainly faster in release mode.

Does it support multi cores? I know on Windows you can recompile OpenCV with the TBB or OpenMP options to support it. I haven't tried it myself yet though.

 

[JordiCP]

Glad to hear it! Once you've checked that it is correctly ported, it can be a good idea to measure the times for each OpenCV function, and detect where most of the time is spent.

Some tips:

• In OpenCV there are some 'surprises' as some functions (luckily, not many) are not very optimized. I have found some in which my 'C' implementation was faster. As I said, luckily there aren't many of these.
• Algorithms. There is a huge difference of time depending on parameters used. Fine tuning them, knowing which are critical for your needs and which not, can help to make them work faster.
• Resuse OCVMats: each JNI call has a cost. Declare all what can be declared out of the main processing loop, and reuse it.
  
• Resolution; many OpenCV detection features don't need really a high resolution and will output similar (but faster) results with previously scaled-down images.
• Also, release mode will run faster than debug mode, specially if you are using real time processing from the camera
• Performance in arm-v8 devices is far better than in armeabi-v7a, even if they support NEON. Which device are you using?

I am pretty sure that I compiled with TBB. But it will only be noticed in those functions that have an implementation that takes advantage of it.

 

[biometrics]

Glad to hear it! Once you've checked that it is correctly ported, it can be a good idea to measure the times for each OpenCV function, and detect where most of the time is spent.

Some tips:

• In OpenCV there are some 'surprises' as some functions (luckily, not many) are not very optimized. I have found some in which my 'C' implementation was faster. As I said, luckily there aren't many of these.
• Algorithms. There is a huge difference of time depending on parameters used. Fine tuning them, knowing which are critical for your needs and which not, can help to make them work faster.
• Resuse OCVMats: each JNI call has a cost. Declare all what can be declared out of the main processing loop, and reuse it.
  
• Resolution; many OpenCV detection features don't need really a high resolution and will output similar (but faster) results with previously scaled-down images.
• Also, release mode will run faster than debug mode, specially if you are using real time processing from the camera
• Performance in arm-v8 devices is far better than in armeabi-v7a, even if they support NEON. Which device are you using?

I am pretty sure that I compiled with TBB. But it will only be noticed in those functions that have an implementation that takes advantage of it.

Thanks for the tips. I just wanted to get it to run first, but after reviewing my changes I'll check out optimizing it. I've been using 640x480 resolution in Windows and Android for this precise reason. I know I have some Dim OCVMats in my main loop, so will move those to global. I've been testing on a nearly three year old Sony Z3 which has an armeabi-v7a CPU (according to the Droid Info app). I'll look for a arm-v8 device to compare the speed with. Any idea where can I find a list of devices with the arm-v8 CPU? Alternatively how do I know a CPU is arm-v8 from specifications?

This device (http://www.giadatech.com/pshow-124.html) for example has a CPU described as "RK3188 ARM Cortex-A9 Quad-Core 1.6GHz". How do I figure out what type it is?

Thanks for all the help!

 

[JordiCP]

arm64-v8 are 64-bits ARM processors (arm-v7, armeabi-v7a are 32-bit). The device in the link seems to have Cortex A9 MP architecture --> 4 processors that are 32-bit.
In order to look for 64-bit devices, just search for arm64 or 64-bit, there are many.

But I wouldn't yet spend any money on it until the rest has been optimized, for two reasons: a) perhaps at the end it won't be necessary, or b) if the processing algorithm is still 'slow' the 'jump' from 32 to 64bits won't help enough. Just my opinion.

 

[ingo.tw]

I'd be interested to see the lib too

Many thanks!

 

[JordiCP]

PM me with your mail address and I will add you to the distribution list

 

[Gorazdspajzer]

I would like to use it in my project!

 

[JordiCP]

I would like to use it in my project!

PM me with your mail address and I will add you to the distribution list

 

[biometrics]

• It is free to test and use. You can and this will encourage me to maintain it, and also build additional material for donators.

Hi Jordi,

The "Donate" button loads a Spanish language Paypal page. Can you give us an English version? I didn't see an obvious switch language option.

 

[JordiCP]

The "Donate" button loads a Spanish language Paypal page

Changed (I think)! Just need some beta-testers to check that it works properly (joking)

BTW, did you succeed with your project? I would be interested to know (if possible, of course) what it is related to.

 

[biometrics]

Changed (I think)! Just need some beta-testers to check that it works properly (joking)

BTW, did you succeed with your project? I would be interested to know (if possible, of course) what it is related to.

This is what I see. I'd like it to be English and in US$.

 

[JordiCP]

Thanks!!!

 

[DonManfred]

Just need some beta-testers to check that it works properly (joking)

ha ha ha....

I tried the paypal button and i got shown a English page.

 

[Swissmade]

Hi Jordi,
I'm very interested in this Library.
Can I use this in B4J i have seen some on youtube that this will work.
Thanks for repond