B4A Library OpenCV 3.x

JordiCP · May 22, 2017

OpenCV (Open Source Computer Vision Library) is a really huge project/framework actively developed, mainly written in C++ . It is released under a BSD license.

Read more here: http://opencv.org/

OpenCV versions: https://opencv.org/releases/

OpenCV library for B4A: wraps the official OpenCV 3.x release for Android (in fact not all, about 95% of it)

Feel free to test and use it. You can even
License: You can use it for your projects, but you are not allowed to distribute nor sell this library. Of course you can distribute apps that use it (remember that OpenCV itself has BSD license as stated before)
Supported hardware is: armeabi-v7a and arm64-v8
Versions
- 1.04 (2020/05/17)
  - This version wraps OpenCV 3.4.1 Android release and fixes some bugs (mostly some instance methods and some not exposed classes) of the previous version.
  - One of the major additions is the DNNmodule and related classes.
    - LINK to example (too large to be uploaded here)
    - Result video
  - Link to the B4A library files: HERE
- 1.00 (2017/09/27)
  - First B4A library wrapper (OpenCv320forB4A V1.00) which replicates (95%) the official OpenCV 3.20 Java API for Android.
  - (removed link. Use 1.04)

Please note that my support will be limited to issues with the wrapper itself, not to help translating OpenCV code from other languages to B4A

========================================================================

(There may be some inaccuracies in what I expose in this post, related to the OpenCV project, since I am relatively new to it and don't know all about its internals. If you find any, please let me know and I will correct it)

A bit of explanation

There exist 'official' OpenCV wrappers for different languages and platforms. Android is one of them.
The official OpenCV 3.20 for Android API includes a lot of classes, organized in modules. But it does not include "all" the original OpenCV modules (since there are other 'experimental', non-free, or platform specific modules which may be present in other platforms but not for Android). Also, there are build options to "tune" it...

I have played quite a lot with it this last year, with a huge project which I started with inline Java, and also translating examples or testing features. But what I have used is just a small percent of the classes and methods exposed. So, there may be some (let's hope not too many) things to fix.

How to learn OpenCVforB4A

If you have worked before with OpenCV, the learning curve will be easy.
If it was using Java with OpenCV for Android, then it will be immediate, since all the methods have exactly the same syntax (except for initializers, polimorphism, and some special cases where simply I did my best).
Anyhow, the ways that I can think of, are (will add links later, also suggestions as online tutorials,.. are welcome)

Attached examples.
B4A OpenCV Tutorials. I will write a couple of them with what I consider the most important building pieces (for instance the Mat class, which in B4A is OCVMat) and modules
Internet examples: there are A LOT of examples over there, written in C++, Java, Python, JavaCV. I would look for examples in the language that is easier to understand for you and then try to translate. Some tips about it (based on my experience)

OpenCV syntax has changed as new versions. So there is an 'old' syntax in which nearly everything started with "cv...". Since version 3.X, there was 'cleaner' organization (project was written in C++ instead of C), and there were major syntax changes.
JavaCV: Translating from JavaCV to OpenCV should be quite easy but not always direct. JavaCV uses a mix of the old OpenCV syntax with some of its own, and at the beginning it can be a bit confusing, but then it is also easy.
Python: there is a lot of material...

First steps. Prepare for some crashes...

In OpenCV nearly everything takes part in the native code.
When we call a Sub/method/algorithm, it performs some internal checks to see if all the input data is correct. This check is perfomed in the native side. If something is not correct (wrong OCVMat dimensions, some incoherent parameters,...) it throws an exception and crashes. If we are lucky, perhaps we see in the log some clues about the check that made it crash.
On the good side, it is very easy to achieve results with OpenCV (check the examples). The real difficult part, as with many other things, is to fine-tune it: OpenCV has a collection of really powerful 'primitive' objects and operations, and really complex algorithms that can do many things. But it is the user who has to glue all of them to achieve the desired results.

(from the previous Beta announcement)

IMPORTANT: you must take this into account:

OpenCV (the included binary modules) is a free(*) project, but subject to license terms as described here: http://opencv.org/
- (*): There are some modules in the OpenCV project which are on-free, but here I am refering to the ones included in the library
My work: (the B4A library) is free to test and use, but you can

for it . I'll keep donators updated with "advanced" material and examples
If you are interested, please PM me with your mail address and I will send you a link with the library and some basic examples. (be patient if you don't receive it immediately, I'll do it as soon as possible).

There is no documentation. In short, the syntax is nearly-exactly the same as the OpenCV3.20 Java API, adding "OCV" prefix and only the minimum modifications to adapt to B4A, For reference (taking into account described syntax changes) you can look at http://docs.opencv.org/java/3.1.0/ (which is not the latest one, but the API is nearly the same).
It would be preferable if you have worked before with OpenCV and/or can translate examples from Java/C++ and/or simply are interested in it.
I recommend starting with the examples and try to understand what is done. Just experimenting can lead to crash after crash of the native libraries with nearly no useful information, and can be very discouraging.
I forgot, the included binaries are for ameabi-v7a and arm64-v8 devices

---------------------------------------------------------------------
Some screenshots taken from the examples

Canny operator - Features2D - Color space conversion

2D-FFT

Color Blob detection

moster67 · Jun 4, 2017

Here is a video which is the result after some hours of coding to learn better OpenCV.

This video wants to demonstrate car detection in realtime using OpenCV for B4A.
The video was processed so each frame could be extracted and then a detection process was run..

The result can surely be much improved by experimenting with thresholds and other values to avoid false detection etc. This example could probably be improved to also distinguish cars from vans and lorries. You could count how many cars are passing by and their colors etc.

You can find examples of similar videos on the internet where c++ or python have been used. Now it is possible also with B4A!
Many thanks to @JordiCP for wrapping OpenCV for B4A and for letting me test it in this beta-phase!

JordiCP · Jun 4, 2017

Really great job Mike!!!

With your permission, will add your code to the examples. Thanks!

stu14t · Jun 4, 2017

moster67 said:
Here is a video which is the result after some hours of coding to learn better OpenCV.

This video wants to demonstrate car detection in realtime using OpenCV for B4A.
The video was processed so each frame could be extracted and then a detection process was run..

The result can surely be much improved by experimenting with thresholds and other values to avoid false detection etc. This example could probably be improved to also distinguish cars from vans and lorries. You could count how many cars are passing by and their colors etc.

You can find examples of similar videos on the internet where c++ or python have been used. Now it is possible also with B4A!
Many thanks to @JordiCP for wrapping OpenCV for B4A and for letting me test it in this beta-phase!

Again I can't thank @JordiCP enough for this lib.

I'm working on something similar. What do you think the limit is for the number of simultaneous classifiers running at the same time? Some of the HAAR xml files get quite big.

Some of the OpenCV examples show two or three but that is not in Android.

moster67 · Jun 4, 2017

stu14t said:
What do you think the limit is for the number of simultaneous classifiers running at the same time?

I really don't know - I just started out learning OpenCV. However, we must remember that there might be some restrictions on Android (see Jordi's post) what regards optimization of code and of course that the hardware has less performance than a PC. The performance of the sample I posted varied quite a lot when I ran it on my old Samsung Galaxy S3 and my wife's much newer (and faster) LG phone so hardware is definitively an important factor, especially for real time tasks.

JordiCP · Jun 4, 2017

Totally agree. As with desktop PCs, if we are talking of real-time, maximum performance will need programming with C++ and taking advantage of hardware acceleration.

However, making use of some simple tips (this applies to whatever platform), will make your app faster.

Minimize number of operations: declare all your vars out of the main loop. Reuse all what can be reused.Not only because of memory: JNI calls also take time.
Input resolution. Depending on which features you need, most of the times 640X480 is more than enough, even in some cases 320x240
Color or gray: If you are looking for shapes, consider starting with a gray image instead of rgb.
Measure: talking about performance in 32-bit devices with @moster67, we realized that blurring with a kernel size above 16 caused a terrible drop in performance. It can be due to cache issues, or internal branches in how the algorithm is handled internally (up to 16, it fits in a 256 structure, so can be optimized by a certain algorithm branch. And if it's bigger, then another algo is used)
Compare: the same algorithms have parameters which we tend to use as found in the examples. However, they can have a determinant effect in performance.
Frame skipping: if you are not doing some kind of face morphing or tracking for AR, but for instance object detection and recognition, consider if you need all the frames to be processed, or it is enough once a second.

JordiCP · Jun 4, 2017

Now Beta phase is over. A lot of syntax bugs to be fixed, and a remaining issue with Samsung Galaxy S7 camera.

Thanks a lot to all beta-testers!! Library version 1.0 will be released next weekend

roberto64 · Jun 5, 2017

hi jordiCP, I found this error.
regards

B4X:

** Activity (main) Create, isFirst = true **
** Activity (main) Resume **
JavaCameraViewInitialize java camera
JavaCameraViewTrying to open back camera
JavaCameraViewTrying to open camera with new open(0)
mCamera=android.hardware.Camera@41c48098
JavaCameraViewgetSupportedPreviewSizes()
JavaCameraViewSet preview size to 352x288
JavaCameraViewstartPreview
HelloThread!!!
Error occurred on line: 114 (Main)
CvException [com.appiotic.ocv4b4a.core.CvException: cv::Exception: C:/Develop/git/opencv_320/modules/core/src/matrix.cpp:491: error: (-215) 0 <= _colRange.start && _colRange.start <= _colRange.end && _colRange.end <= m.cols in function cv::Mat::Mat(const cv::Mat&, const cv::Range&, const cv::Range&)
]
    at com.appiotic.ocv4b4a.core.Mat.n_submat_rr(Native Method)
    at com.appiotic.ocv4b4a.core.Mat.submat(Mat.java:875)
    at com.appiotic.ocv4b4a.samples.cameratest.main._frameprocessor_newframe(main.java:508)
    at java.lang.reflect.Method.invokeNative(Native Method)
    at java.lang.reflect.Method.invoke(Method.java:515)
    at anywheresoftware.b4a.shell.Shell.runMethod(Shell.java:710)
    at anywheresoftware.b4a.shell.Shell.raiseEventImpl(Shell.java:339)
    at anywheresoftware.b4a.shell.Shell.raiseEvent(Shell.java:249)
    at java.lang.reflect.Method.invokeNative(Native Method)
    at java.lang.reflect.Method.invoke(Method.java:515)
    at anywheresoftware.b4a.ShellBA.raiseEvent2(ShellBA.java:139)
    at anywheresoftware.b4a.BA.raiseEvent2(BA.java:170)
    at anywheresoftware.b4a.BA.raiseEvent(BA.java:166)
    at com.appiotic.ocv4b4a.android.JavaCameraView.rapidDebugModeCameraWorker(JavaCameraView.java:483)
    at com.appiotic.ocv4b4a.android.JavaCameraView.onPreviewFrame(JavaCameraView.java:387)
    at android.hardware.Camera$EventHandler.handleMessage(Camera.java:1005)
    at android.os.Handler.dispatchMessage(Handler.java:102)
    at android.os.Looper.loop(Looper.java:146)
    at android.app.ActivityThread.main(ActivityThread.java:5511)
    at java.lang.reflect.Method.invokeNative(Native Method)
    at java.lang.reflect.Method.invoke(Method.java:515)
    at com.android.internal.os.ZygoteInit$MethodAndArgsCaller.run(ZygoteInit.java:1283)
    at com.android.internal.os.ZygoteInit.main(ZygoteInit.java:1099)
    at dalvik.system.NativeStart.main(Native Method)
Error occurred on line: 114 (Main)
java.lang.NullPointerException
    at anywheresoftware.b4a.shell.Shell.runGoodChain(Shell.java:463)
    at anywheresoftware.b4a.shell.Shell.raiseEventImpl(Shell.java:285)
    at anywheresoftware.b4a.shell.Shell.raiseEvent(Shell.java:249)
    at java.lang.reflect.Method.invokeNative(Native Method)
    at java.lang.reflect.Method.invoke(Method.java:515)
    at anywheresoftware.b4a.ShellBA.raiseEvent2(ShellBA.java:139)
    at anywheresoftware.b4a.BA.raiseEvent2(BA.java:170)
    at anywheresoftware.b4a.BA.raiseEvent(BA.java:166)
    at com.appiotic.ocv4b4a.android.JavaCameraView.rapidDebugModeCameraWorker(JavaCameraView.java:483)
    at com.appiotic.ocv4b4a.android.JavaCameraView.onPreviewFrame(JavaCameraView.java:387)
    at android.hardware.Camera$EventHandler.handleMessage(Camera.java:1005)
    at android.os.Handler.dispatchMessage(Handler.java:102)
    at android.os.Looper.loop(Looper.java:146)
    at android.app.ActivityThread.main(ActivityThread.java:5511)
    at java.lang.reflect.Method.invokeNative(Native Method)
    at java.lang.reflect.Method.invoke(Method.java:515)
    at com.android.internal.os.ZygoteInit$MethodAndArgsCaller.run(ZygoteInit.java:1283)
    at com.android.internal.os.ZygoteInit.main(ZygoteInit.java:1099)
    at dalvik.system.NativeStart.main(Native Method)

JordiCP · Jun 5, 2017

B4X:

CvException [com.appiotic.ocv4b4a.core.CvException: cv::Exception: C:/Develop/git/opencv_320/modules/core/src/matrix.cpp:491: error: (-215) 0 <= _colRange.start && _colRange.start <= _colRange.end && _colRange.end <= m.cols in function cv::Mat::Mat(const cv::Mat&, const cv::Range&, const cv::Range&)

The error is thrown when trying to define a subMat with a row/column range that is not within the original Mat boundaries (or defined in reverse order).

Is it the original example or you changed something?

roberto64 · Jun 5, 2017

Example and the original one

JordiCP · Jun 5, 2017

It is strange because the example tries to initialize the camera at 640x480 which I supposed that all cameras have (and some part of the code relied on it without checking)

B4X:

JavaCameraViewSet preview size to 352x288  '<-- the camera has been initted to this preview size, so something went wrong

This modified example should work for you, I adapted code so that it depends on real preview size and not the intended one.

Which device are you using?

roberto64 · Jun 5, 2017

samsung galaxy core 2

JordiCP · Jun 5, 2017

Really strange, in theory the device supports 640x480
The camera sets its preview size to the maximum (w,h) among the supported preview sizes, which is not superior to the minimum of (640,480) and (P.Width,P.height)

Did the new example work for you? If not I'll PM you and try to discover what happens

roberto64 · Jun 6, 2017

Hi JordiCP, the second example you have posted and ok, but for the example you first published by the same error, or even restarted the smatfhone but nothing, my fencing results support it.
Greetings

JordiCP · Jun 6, 2017

Thank you for testing and reporting
Yes, it seems that in some devices the setup routine fails to get the desired preview size, even if it exists. Will try to fix it.

roberto64 · Jun 6, 2017

Ok, you are doing a good job though.
regards

jchal · Jun 8, 2017

hi there i canot find the OCVOpenCVLoader can you help?

JordiCP · Jun 8, 2017

You need to copy the library files ( .jar and .xml) to the additional libraries folder (and then add the lib to the project by checking it in the 'Libraries manager' Tab)
For any other issue please PM me

JordiCP · Jun 12, 2017

Version 1.0 released (see first post)

did2kan · Jun 12, 2017

JordiCP said:
This example is a bit more complex and must be still optimized (so not released yet). I was "trying" to translate a camera calibration example, when I realised that I haven't much idea about cameras, focal length, ...
So, starting from the same example I jumped onto another and got something more funny

Must say that for "real time", if processing becomes a bit complex, Android will suffer. OpenCV algorithms are not so much optimized for mobile devices as for other plattforms (desktop). Also, some of these optimisations are only accessible from C++ but not from the standard OpenCV Android API. Hope this will be better as OpenCV is continuously evolving...

Great job, thanks you to work on calibration ;-)

roberto64 · Jun 12, 2017

Hi Jordi
CP, I'm trying to open an avi file as by code in the list, with VideoCapture I do not understand why the avi file finds it but it does not open.
Greetings

B4X:

Dim video_src As String  =  "video1.avi"

Sub MainCamera()
    Dim inframe As OCVMat
    Dim grab As Boolean
    Dim apre As Boolean
    Dim Videscr As String
    Videscr = File.DirRootExternal&"/"&video_src
    If File.Exists(File.DirRootExternal,video_src) Then
        Log("Yes")
        Else
        Log("No")
       
    End If
   
    cvvideo.Initialize2(0)
    cvvideo.set(mvideo.CAP_PROP_FRAME_HEIGHT,480)
    cvvideo.set(mvideo.CAP_PROP_FRAME_WIDTH,640)
   
If cvvideo.open1(Videscr) Then
        Log( "Could not open the video file1")
        Else
        Log( "Video1 loaded")
End If
    apre = cvvideo.isOpened
    If cvvideo.isOpened Then
        Log( "failed")
        Else
        Log( "opened")
        cvvideo.read(inframe)
        'cvvideo.retrieve(inframe,1)
    End If
   
    Do While apre
        cvvideo.read(inframe) ' inframe Mat
    Loop

End Sub

B4A Library OpenCV 3.x

Attachments

Expert

Expert

Active Member

Expert

Expert

Expert

Active Member

Expert

Active Member

Expert

Attachments

Active Member

Expert

Active Member

Expert

Active Member

Active Member

Expert

Expert

New Member

Active Member

Similar Threads

Privacy & Transparency

Privacy & Transparency