Android Question Getting Pixel Co-ordinates from Camera Image

[ChocoScope]

I would like to generate a set of pixel co-ordinates from a camera image which are lit up RED and save it in a binary file. I want to generate it from the image in memory rather than grabbing a picture and storing it as .jpg as I want to do it multiple times. The final file will be used for data (pictures) from a 3D laser scanner. Can anyone please point me to or supply suitable programs to do this.

 

[JordiCP]

You can use THIS if you know where these coordinates "more or less" are, and there is not much processing involved.

But if you need to find the zones and do more than "a bit" of process (since I guess they will not be exactly seen as pure Red by the camera), you could use OpenCV. If interested please PM me with your mail address and I'll send you a link

 

[ChocoScope]

I would like to generate a set of pixel co-ordinates from a camera image which are lit up RED and save it in a binary file. I want to generate it from the image in memory rather than grabbing a picture and storing it as .jpg as I want to do it multiple times. The final file will be used for data (pictures) from a 3D laser scanner. Can anyone please point me to or supply suitable programs to do this.

Thank You JordiCP That's AMAZING!!! and it puts me well on the road to my goal. I have compiled it and have uploaded it to my tablet and it is very versatile and Fast!!! I have not yet got to grips as to where to find the image. I assume its "MyIV" but not sussed the extensions I need yet. Could they be MyIV.Red and then Read Bitmab(h, v)? Only had your reply for a short while and need to do further reading.
ANYWAYs ... Many Thanks

 

[JordiCP]

Thanks.
The lib simply converts the preview from NV21 to ARGB888 bitmap, but really fast. It does not detection, but takes you a bit nearer to it, since then you can look for pixel values and compare them (with some margin) with the colors you are looking for. Look at posts #20 and #21 in that thread for more clues (doing some kind of subsampling by getting only some pixels at a given distance)
Also, your process will be easier if for instance you know that the background is of a certain color, that will help you to decide comparation thresholds for the pixel values.

For more complex things, I really recommend you OpenCV.

 

[ChocoScope]

I'm sure my requirements are not complex, as I only want to interrogate the picture elements.

I have come to the conclusion that NV21 is simply the YUV format (approximately) consisting of width x height number of luminance values
as 8 bit bytes, followed by U/V pairs for the color information. I think I can use just the luminance value instead of red, so it will be easy to
extract the bytes from the first section of the bitmap. I still cant work out which is the name of the bitmap and so if you can't tell me its name,
can you tell me where I can view your NV21toRGB routines please.

 

[JordiCP]

I have come to the conclusion that NV21 is simply the YUV format (approximately) consisting of width x height number of luminance values as 8 bit bytes, followed by U/V pairs for the color information.

Yes, that's exactly it. If points can be distinguished by luminance it is much simpler and you don't need any library, since the array in camera_preview is the only thing you need!

I suppose you want to do something like that.

  'we are assuming that camera preview is 640x480. Also, the array is in camera orientation order, usually landscape.

  for row = 0 to 480-1 step 10   ' adapt the step value to your needs
    for col = 0 to 640-1 step 10
       Dim luminance as int = bit.And( data( 640*row + col), 0xFF)     ' needed since we will compare values that we know are "absolute" but in android bytes are signed...
       if luminance>myThreshold Then
           ' you have a rough estimation of a "bright" point at coordinates x=col, y=row
       End If
    next
next

I still cant work out which is the name of the bitmap and so if you can't tell me its name,
can you tell me where I can view your NV21toRGB routines please.

Sorry but I don't understand what you mean by the bitmap name. My routine gets the NV21 array as input and fills a bitmap (which has been defined in the same program) with the converted data. The bitmap name is "myBitmap", and it is declared in the same program.

  NV21toRGB.proceed( PreviewPic, myBitmap.Width*myScale, myBitmap.Height*myScale, myScale, myBitmap, camVertical , camEx.Front, camEx.DispRotation, myIndexEffect )

The conversion (NV21 to RGB) is explained in the same thread with a link to StackOverflow. The rest of the buttons and user interface are from the original CameraEx example, as my example was built on top of that
Anyway, if I understood correctly now you don't need it anymore.

 

[ChocoScope]

Sorry JordiCP it is my lack of understanding of the camera process and where it puts its image. I assumed it was in a bitmap in memory.
Anyway the routine you kindly wrote (above) will serve my purpose and you have even included a "threshold" which I would have had to do.
I think the word "data" in the above routine defines what I thought was a bitmap. You have totally answered my question and you are right,
I will not have to use your NV21 to RGB conversion routine.

Thank You for your Help.

 

[ChocoScope]

Do I have to stop preview with camEx.StopPreview and restart it again when I've finished with camEx.StartPreview?

 

[ChocoScope]

I tried entering the code above and "data" is not defined. How do I define it?

 

[JordiCP]

The preview just "starts" it.
If what you need is to acquire data during a while and then save results, I would add partial results (the detected points) to an array or structure, and when you have finished, close the cam and save the structure (always keeping in mind that it can not grow forever).

Also, even if your processing loop takes for instance 100msec, I would't stop the preview, as it does not help. You will just skip some frames and get the ones of the next interrupt when your routine has finished.

I tried entering the code above and "data" is not defined. How do I define it?

Sorry, it should be 'PreviewPic', I made it by memory

Sub Camera1_Preview (PreviewPic() As Byte)

   'we are assuming that camera preview is 640x480. Also, the array is in camera orientation order, usually landscape.
   For row = 0 to 480-1 step 10 ' adapt the step value to your needs
      For col = 0to640-1step10
         Dim luminance as int = bit.And( PreviewPic( 640*row + col), 0xFF) ' needed since we will compare values that we know are "absolute" but in android bytes are signed...
         If luminance>myThreshold Then  
            ' you have a rough estimation of a "bright" point at coordinates x=col, y=row
         End If
      Next
   Next
  
End Sub

 

[ChocoScope]

Thank You. I think that I will leave the preview running. I have put the code into Camera1_Preview with an empty loop and it compiles and runs ok.
How much of the original Camera1_Preview can I remove and what do I have to keep? I assume I can remove the NV1toRGB.proceed but I suspect I have to keep everything else.

 

[JordiCP]

if you know the preview size then you could use the code in my previous post as base

 

[ChocoScope]

Hi JordiCP, I have been looking through your program and before I can use your code to generate a threshold file, I would like to see the image the dots would produce. So I tried to make a new panel with a canvas on it. Managed to draw some lines on it so I knew it was working and then tried to put dots on the panel from the active pixels. It didn't work, so I made a routine outside of Preview_Pic and added a button to activate it.

Sub Dotty  (PreviewPic() As Byte)
 
myThreshold = 128
  
 'we are assuming that camera preview is 640x480. Also, the array is in camera orientation order, usually landscape.
  For row = 0 To 480-1 Step 10 ' adapt the step value to your needs 
  For col = 0 To 640-1 Step 10
  Dim luminance As Int = Bit.AND( PreviewPic( 640*row + col), 0xFF) ' needed since we will compare values that we know are "absolute" but in android bytes are signed...
  If luminance>myThreshold Then  
  ' you have a rough estimation of a "bright" point at coordinates x=col, y=row
 cvsGraph.DrawPoint(row, col, Colors.Red)
  End If 
  Next
  Next

End Sub

I could not call the subroutine as PreviewPic had not been initialised

Can you please guide me towards producing a preview panel of generated dots. Many thanks

 

[JordiCP]

Where are you calling 'Dotty()' from? You need to call it from your Camera1_Preview(...) subroutine and pass the PreviewPic() as parameter.

If you send me a small project showing how you are using it, I will try to "tune" it a bit. It will be the fastest option

 

[ChocoScope]

I started off with a copy of your program "Camera_NV21utils" and added the following. Originally, "Dotty" was a part of your
"Camera_Preview" routine as shown as I hoped it would refresh at the same rate, but it didn't work. I then set it up as a separate
subroutine, added a button to the layout and tried to call it from there as a "one shot". I would prefer the former and be able to disable
"Dotty " when it wasn't needed.

Sub Process_Globals
 'These global variables will be declared once when the application starts.
 'These variables can be accessed from all modules.
End Sub
Sub Globals
 'These global variables will be redeclared each time the activity is created.
 'These variables can only be accessed from this module.
 Dim pnlGraph  As Panel ' Graph panel (Grid)
 Dim rectGraph As Rect ' Graph rectangle
 Dim GraphX0 As Int ' left position of the Graph
 Dim GraphY0 As Int ' top position of the Graph
 Dim GraphW  As Int ' width of the Graph
 Dim GraphH  As Int ' height of the Graph
 Dim GraphColor As Int ' Graph background color
 Dim row As Int
 Dim col As Int
 Dim luminance As Int
 Dim myThreshold As Int
 Dim cvsGraph As Canvas 
 Dim Button1 As Button
End Sub

Sub Activity_Create(FirstTime As Boolean)
 'Do not forget to load the layout file created with the visual designer. For example:
 'Activity.LoadLayout("Layout1")
 
 Create_Graph

End Sub
Sub Activity_Resume
End Sub
Sub Activity_Pause (UserClosed As Boolean)
End Sub
Sub Create_Graph
 GraphX0 = 4%y 'Xstart of graphics frame
 GraphW = 70%x - 2 * GraphX0 'X(width) of graphics frame
 GraphY0 = 3%y 'Ystart of graphics frame
 GraphH = 100%y - 2 * GraphY0 'Y(height) of graphics frame
 rectGraph.Initialize(0, 0, GraphW, GraphH) 'corners of graphics frame
 GraphColor = Colors.White 'colour of graphics frame
 pnlGraph.Initialize("pnlGraph") 'initialize graphics panel
 Activity.AddView(pnlGraph, GraphX0, GraphY0, GraphW, GraphH) 'and add it to the screen
 cvsGraph.Initialize(pnlGraph) 'initialize canvas onto panel
 cvsGraph.DrawRect(rectGraph, GraphColor, True, 1) 'draw a white rectangle
 cvsGraph.DrawLine(100, 50, 200, 50, Colors.Black, 1) 'draw a black line
End Sub
Sub Button1_Click
 CallSub2 Dotty(PreviewPic()) 'DOESNT WORK
End Sub

Sub Dotty  (PreviewPic() As Byte)
 myThreshold = 128
 'we are assuming that camera preview is 640x480. Also, the array is in camera orientation order, usually landscape.
  For row = 0 To 480-1 Step 10 ' adapt the step value to your needs 
  For col = 0 To 640-1 Step 10
  Dim luminance As Int = Bit.AND( PreviewPic( 640*row + col), 0xFF) ' needed since we will compare values that we know are "absolute" but in android bytes are signed...
  If luminance>myThreshold Then  
  ' you have a rough estimation of a "bright" point at coordinates x=col, y=row
 cvsGraph.DrawPoint(row, col, Colors.Red)
  End If 
  Next
  Next

End Sub

Sub Camera1_Preview (PreviewPic() As Byte)

 'we are assuming that camera preview is 640x480. Also, the array is in camera orientation order, usually landscape.

 'prevent queued events from previous camera settings, if any. Just in case
 If PreviewPic.Length<>(3*myBitmap.Width*myScale*myBitmap.Height*myScale/2) Then
 Log("Not processing")
 Return
 End If
 NV21toRGB.proceed( PreviewPic, myBitmap.Width*myScale, myBitmap.Height*myScale, myScale, myBitmap, camVertical , camEx.Front, camEx.DispRotation, myIndexEffect )
 'DOTTY WAS ORIGINALLY HERE AS PART OF YOUR ROUTING..........
 
 myIV.invalidate 'Refresh the view
 
End Sub

 

[JordiCP]

Sub Button1_Click
CallSub2 Dotty(PreviewPic()) 'DOESNT WORK
End Sub

There are some mistakes here.
First, the CallSub syntax is not correct. Also, the parameter 'PreviewPic' is a local variable from Camera_Preview, so it won't be accessible out of that loop.
What you want should be similar to (notice the changes and their reason)

Sub Process_Globals
'These global variables will be declared once when the application starts.
'These variables can be accessed from all modules.
Dim flgProcessOnce as Boolean = False
End Sub
Sub Globals
'These global variables will be redeclared each time the activity is created.
'These variables can only be accessed from this module.
Dim pnlGraph  As Panel ' Graph panel (Grid)
Dim rectGraph As Rect ' Graph rectangle
Dim GraphX0 As Int ' left position of the Graph
Dim GraphY0 As Int ' top position of the Graph
Dim GraphW  As Int ' width of the Graph
Dim GraphH  As Int ' height of the Graph
Dim GraphColor As Int ' Graph background color
Dim row As Int
Dim col As Int
Dim luminance As Int
Dim myThreshold As Int
Dim cvsGraph As Canvas
Dim Button1 As Button
End Sub

Sub Activity_Create(FirstTime As Boolean)
'Do not forget to load the layout file created with the visual designer. For example:
'Activity.LoadLayout("Layout1")
Create_Graph

End Sub
Sub Activity_Resume
End Sub
Sub Activity_Pause (UserClosed As Boolean)
End Sub
Sub Create_Graph
GraphX0 = 4%y 'Xstart of graphics frame
GraphW = 70%x - 2 * GraphX0 'X(width) of graphics frame
GraphY0 = 3%y 'Ystart of graphics frame
GraphH = 100%y - 2 * GraphY0 'Y(height) of graphics frame
rectGraph.Initialize(0, 0, GraphW, GraphH) 'corners of graphics frame
GraphColor = Colors.White 'colour of graphics frame
pnlGraph.Initialize("pnlGraph") 'initialize graphics panel
Activity.AddView(pnlGraph, GraphX0, GraphY0, GraphW, GraphH) 'and add it to the screen
cvsGraph.Initialize(pnlGraph) 'initialize canvas onto panel
cvsGraph.DrawRect(rectGraph, GraphColor, True, 1) 'draw a white rectangle
cvsGraph.DrawLine(100, 50, 200, 50, Colors.Black, 1) 'draw a black line
End Sub

Sub Button1_Click
   flgProcessOnce=True     'Just tell the routine that next time you want it to do something
End Sub

Sub  Dotty( PreviewPic() as Byte )
'we are assuming that camera preview is 640x480. Also, the array is in camera orientation order, usually landscape.
  For row = 0 To 480-1 Step 10 ' adapt the step value to your needs
  For col = 0 To 640-1 Step 10
  Dim luminance As Int = Bit.AND( PreviewPic( 640*row + col), 0xFF) ' needed since we will compare values that we know are "absolute" but in android bytes are signed...
  If luminance>myThreshold Then 
  ' you have a rough estimation of a "bright" point at coordinates x=col, y=row
cvsGraph.DrawPoint(row, col, Colors.Red)
  End If
  Next
  Next

End Sub

Sub Camera1_Preview (PreviewPic() As Byte)

  'we are assuming that camera preview is 640x480. Also, the array is in camera orientation order, usually landscape.

  'prevent queued events from previous camera settings, if any. Just in case
  If PreviewPic.Length<>(3*myBitmap.Width*myScale*myBitmap.Height*myScale/2) Then
    Log("Not processing")
    Return
  End If

  if flgProcessOnce=True Then    'if flag has been raised because you clicked the button, process once
    Dotty( PreviewPic )       ' Direct call to Dotty(). Here you can pass the parameter 'PreviewPic' because it is a local var in this Sub
    flgProcessOnce=False   ' reset the flag
  End sub

  ' You don't need my library as you are processing directly on luminance. So this line can be removed.
  'NV21toRGB.proceed( PreviewPic, myBitmap.Width*myScale, myBitmap.Height*myScale, myScale, myBitmap, camVertical , camEx.Front, camEx.DispRotation, myIndexEffect )
'DOTTY WAS ORIGINALLY HERE AS PART OF YOUR ROUTING..........
myIV.invalidate 'Refresh the view
End Sub

I would take a look at some examples in the beginner's guide in order to get more familiarized with the CallSub and CallSub2 syntax, and also variables scope (local, globals)

 

[ChocoScope]

Thank you for your code, but when I compiled the corrected version it said ... inside of Camera1_preview

Parsing code. 0.05
Compiling code. Error
Error compiling program.
Error description: Cannot cast type: {Type=Double,Rank=0, RemoteObject=False} to: {Type=Byte,Rank=1, RemoteObject=True}
Occurred on line: 302
Dotty( PreviewPic )
Word: previewpic

 

[JordiCP]

just edited the post. Had forgotten a line

 

[ChocoScope]

where's the edit?

 

[JordiCP]

i edited the same post #16