Android Question Is it possible to process images in real time?

irda

irda

Member
Licensed User
Longtime User
Hello. I am new and I need to know which of the following are possible which make with B4A and not because they are supported by a mobile. It is an application that analyzes images in real time to show some information at the end of the process.
- Start the camera. It is possible using the library CameraEx.
- Access the camera preview: if the function Sub Camera1_preview (Data () As Byte) of the same library.
- Extract RGB values of each pixel vector Data ()?. I think this format Data Ybr.
-Loop through all the pixels for values. Do you think that will be too slow? Can you capture less resolution to go faster?

All this should make about 20 times per second. Is it too much effort for a phone?

I appreciate any help.
 
Sort by date Sort by votes
thedesolatesoul

thedesolatesoul

Expert
Licensed User
Longtime User
irda said:
- Extract RGB values of each pixel vector Data ()?. I think this format Data Ybr.
-Loop through all the pixels for values. Do you think that will be too slow? Can you capture less resolution to go faster?

All this should make about 20 times per second. Is it too much effort for a phone?
Click to expand...
IMHO, this will be too slow. Firstly just getting the bitmap will be slow, then looping through all the pixels will be a slow operation.
20 fps is quite high with this approach.
You can decrease the resolution to make it faster. You can also try to do some bulk operations on the pixels to make it fast.
 
Upvote 0
irda

irda

Member
Licensed User
Longtime User
I need to detect and follow an item.
What do you mean when you are referring to bulk operations and how could I do them?
Thanks
 
Upvote 0
mindcom

mindcom

Member
Licensed User
Longtime User
Dear All,
I would like to process directly data() that comes from a _PictureTaken event (CameraEx library).
What is the format?
Can I make something like GetPixel directly on the data() array?
Many thx for your help.
Andrea
 
Upvote 0
mindcom

mindcom

Member
Licensed User
Longtime User
What I need is to use a function to extract color of some pixel in the image, just like GetPixel(x, y).
What I have to do now is to save the data() array to a file with cameraEx.SavePictureToFile(data, dir, filename). Than I have to open it again with LoadBitmap or LoadBitmapSample. And now I have a Bitmap object where I can use GetPixel.
No way to that process directly on data() array?
 
Upvote 0
Informatix

Informatix

Expert
Licensed User
Longtime User
mindcom said:
What I need is to use a function to extract color of some pixel in the image, just like GetPixel(x, y).
What I have to do now is to save the data() array to a file with cameraEx.SavePictureToFile(data, dir, filename). Than I have to open it again with LoadBitmap or LoadBitmapSample. And now I have a Bitmap object where I can use GetPixel.
No way to that process directly on data() array?
Click to expand...
Look at the GetPixels function of the ImageUtils class of my Accelerated Surface library.
 
Upvote 0
mindcom

mindcom

Member
Licensed User
Longtime User
Informatix said:
Look at the GetPixels function of the ImageUtils class of my Accelerated Surface library.
Click to expand...

Informatix, I did not found a solution with your AS library.

I have found a simple way to analize preview data...

Sub Camera1_Preview (data() As Byte)
Dim dataJpg() As Byte, iSt As InputStream, img As Bitmap
dataJpg = camEx.PreviewImageToJpeg(data, 80)
iSt.InitializeFromBytesArray(dataJpg, 0, dataJpg.Length)
img.Initialize2(iSt)
...

I would like to do the same whan I take a picture at full size...
Any idea?
 
Upvote 0
mindcom

mindcom

Member
Licensed User
Longtime User
Informatix said:
What do you mean? That you do not find the function in the library? Or it is not suited for your needs (so what are your needs because your initial request was to get the color of each pixel of an image)?
Click to expand...

I use without any problem the getpixel function. My problem is to get and android.graphics.Bitmap from the byte array data() in YUV format that you get from the _PictureTaken (Data() As Byte) event from Camera.
At the moment the only way I have found is to save the Data() byte array with the camera.SavePictureToFile(Data, folder_name, file_name) fuction, that generate a jpeg file. Than I have to open it again with LoadBitmap or LoadBitmapSample. So I obtain an android.graphics.Bitmap where I can use a getpixel function. But it is a very slow solution.

My question is:
1) is there a way to use a getpixel function directly on the Data() byte array?
or 2) is it possible to add to the Camera object a function like .PreviewImageToJpeg, able to process the full size image that comes from the _PictureTaken event?
or 3) any other solution to process directly data stream from the camera as fast as possible?
 
Upvote 0
Informatix

Informatix

Expert
Licensed User
Longtime User
mindcom said:
I use without any problem the getpixel function. My problem is to get and android.graphics.Bitmap from the byte array data() in YUV format that you get from the _PictureTaken (Data() As Byte) event from Camera.
At the moment the only way I have found is to save the Data() byte array with the camera.SavePictureToFile(Data, folder_name, file_name) fuction, that generate a jpeg file. Than I have to open it again with LoadBitmap or LoadBitmapSample. So I obtain an android.graphics.Bitmap where I can use a getpixel function. But it is a very slow solution.

My question is:
1) is there a way to use a getpixel function directly on the Data() byte array?
or 2) is it possible to add to the Camera object a function like .PreviewImageToJpeg, able to process the full size image that comes from the _PictureTaken event?
or 3) any other solution to process directly data stream from the camera as fast as possible?
Click to expand...

Ok I see. What result did you get when you pass your byte array to your Camera1_Preview sub (and pass it directly to InitializeFromBytesArray since the data are already in JPG format)?

Note that GetPixel and GetPixels are two different functions and I wouldn't use GetPixel to get all the pixels of an image because it's horribly slow.
 
Upvote 0
mindcom

mindcom

Member
Licensed User
Longtime User
Informatix said:
Ok I see. What result did you get when you pass your byte array to your Camera1_Preview sub (and pass it directly to InitializeFromBytesArray since the data are already in JPG format)?

Note that GetPixel and GetPixels are two different functions and I wouldn't use GetPixel to get all the pixels of an image because it's horribly slow.
Click to expand...

The _Preview event is generated by the Camera, and what I have done in preview is to use the .PreviewImageToJpeg function to convert the Data() byte array from YUV format to JPG stream. Then I have initilized an InputStream with the .InitializeFromBytesArray function. Then I have initialized a normal bitmap with the .Initialize2 function. In my opinion too passages to access memory, but it works fine and make it possible to do some simple analisys on preview frames.

Sub Camera1_Preview (data() As Byte)
Dim dataJpg() As Byte, iSt As InputStream, img As Bitmap
dataJpg = camEx.PreviewImageToJpeg(data, 80)
iSt.InitializeFromBytesArray(dataJpg, 0, dataJpg.Length)
img.Initialize2(iSt)
img.GetPixel(img.Width/2, img.Height/2) ...
...

But as you know the preview frame is not high resolution. On an S4 is 1440x1080, that is not bad but very different from the 12Mpixel of the full frame picture. To do deeper analisys on the image I need the full frame picture, and to do that now I need to save and open it again (TOO SLOW).
 
Upvote 0
Informatix

Informatix

Expert
Licensed User
Longtime User
mindcom said:
The _Preview event is generated by the Camera, and what I have done in preview is to use the .PreviewImageToJpeg function to convert the Data() byte array
Click to expand...

Yes, I saw that. My question is : why don't you reuse the same code for the data coming from the PictureTaken event? What result do you get?
To be more clear, replace these lines:
Dim dataJpg() As Byte, iSt As InputStream, img As Bitmap
dataJpg = camEx.PreviewImageToJpeg(data, 80)
by:
Dim iSt As InputStream, img As Bitmap
And replace dataJpg by data.
 
Upvote 0
mindcom

mindcom

Member
Licensed User
Longtime User
Informatix said:
why don't you reuse the same code for the data coming from the PictureTaken event? What result do you get?
Click to expand...

Yes, already tried..
I receive an error, may be because of the image dimensions. PreviewImageToJpeg function seems to be specialized for preview frames.
Do you know who has written the Camera library? Is it possible to go deep on that code?
 
Upvote 0
JordiCP

JordiCP

Expert
Licensed User
Longtime User
Hi, I have just read this.

I will share a small library which can convert from NV21 to RGB888 in short (provided that you don't plan to do the same app, and rate mine with many stars :D). Up to now I have been a "silent listener" here in the forum, learning from posts here and there from people as thedesolatesoul, Informatix and others, now it is time to contribute also.

But first a small disclaimer

I think it will be suitable for "pseudo-real-time", but take into account that overall performance will depend on resolution, device type and many other factors. There are also graphics libraries which will give you far better performance, but have never used them. The only reason is that I wanted to try doing it by myself as a part of my learning curve.

Hopefully will release it along this week, but can't promise anything.
 
Upvote 0
You must log in or register to reply here.
Top