Android Question PC openCv client and Android server image stream

blooddiamond

Member
Licensed User
Hello, at the beginning please forgive me my language mistakes.

Im running a client app (C++) on my raspbbery Pi using openCv and boost asio libs. Client sends frames from webcamera to an android server app. Everything works fine when it works on server and client running on raspberry (both C++ programs). I have also made phone to phone connection program something like CCTV using AsyncStreams in prefix mode and also worked fine. Now i want to have Raspberry - Phone connection and it makes me lots of problems.

This is my C++ client code:
PHP:
int main()
{
    std::string text1 = "this is some\r text, to change,\r if parsing data works ";
   VideoCapture cap(0);                        /* open webcam */
   if(!cap.isOpened())
   {
       return -1;
   }

   Mat frame;
   cap.set(CV_CAP_PROP_FRAME_WIDTH, 320);      /* set width */
   cap.set(CV_CAP_PROP_FRAME_HEIGHT, 240);     /* set height */

   while (true)
   {
       cap>>frame;
       imshow("client",frame);
      // waitKey(100);

       vector<int> p;
       p.push_back(CV_IMWRITE_JPEG_QUALITY);
       p.push_back(80);
       vector<uchar> buf;

      imencode(".bmp", frame, buf);    //encode to bitmap or JPEG ...
      buf.push_back(char(13));         // add Carriage Return character

      ////////////// when in string sending//////////////////////////////////////////////////
   frame = (frame.reshape(0,1)); // to make it continuous

   int num=frame.total();                      /* num = 320*240 */
   int num2=frame.elemSize();                  /* mun2 =3 channel */
   int  imgSize = frame.total()*frame.elemSize();

////////////////////////////////////////////////////////////////////////////////////////////////


   try
   {
       boost::asio::io_service io_service;
       tcp::endpoint end_point(boost::asio::ip::address::from_string("192.168.0.18"), 3200);

       tcp::socket socket(io_service);
       socket.connect(end_point);
       boost::system::error_code ignored_error;
       io_service.run();

   //    std::string message((char *)frame.data,230400); /* the size of mat data is 320*240*3 */
    //   message.insert(230400,"\r\n");

       cout<<"sending...."<<buf.size()<<endl;
       socket.write_some(boost::asio::buffer(buf), ignored_error);

       cout<<"send image finished"<<endl;
   }
   catch (std::exception& e)
   {
       std::cerr << e.what() << std::endl;
   }

   }
   return 0;

I can send a frame as string but then it is a 3-channel image in one continous vector so i create a bitmap or jpg (imencode) from the frame and then send it to a server.

This is my simple test server app running on phone and based on AsyncStreams in regular mode

B4X:
Sub Process_Globals
    Dim AStreams As AsyncStreams
    Dim Server As ServerSocket
    Dim Socket1 As Socket
End Sub
Sub Globals
    Dim Conv As ByteConverter
    Dim EditText1 As EditText
    Dim Label1 As Label
    Dim Label2 As Label
    Dim msg As String
    Dim msg2 As String
    Dim i As Int
    Dim j As Int
    Dim k As Int
    Dim Panel1 As Panel
    Private cvs As Canvas
End Sub

Sub Activity_Create(FirstTime As Boolean)
Activity.LoadLayout("dupa1")
    Activity.Title="eeeee"
    If FirstTime Then
        Server.Initialize(3200, "Server")
        Server.Listen
        Log("MyIp = " & Server.GetMyIP)
    End If
  ' EditText1.Initialize("EditText1")
     cvs.Initialize(Panel1)
    EditText1.ForceDoneButton = True
  '  Activity.AddView(EditText1, 10dip, 10dip, 300dip, 60dip)
  
  
    Label1.Width=Activity.Width
     Label1.Height=Activity.Height/7
     Label1.Top=(Activity.Height/10)*6
     Label1.Left=0
     Label1.Gravity = Gravity.CENTER_HORIZONTAL + Gravity.CENTER_VERTICAL
     Label1.TextSize=Activity.Height/10/5
     Label1.Text = "size/message"
  
    Label2.Width=Activity.Width
     Label2.Height=Activity.Height/10*3
     Label2.Top=Activity.Height/10*8
     Label2.Left=0
     Label2.Gravity = Gravity.CENTER_HORIZONTAL + Gravity.CENTER_VERTICAL
     Label2.TextSize=Activity.Height/10/5
     Label2.Text="waiting" & CRLF & "for data"
   
    Panel1.Width=Activity.Width
     Panel1.Height=Activity.Height/10*4
     Panel1.Top=(Activity.Height)/10*2
     Panel1.Left=0
End Sub

Sub Activity_Resume

End Sub

Sub Server_NewConnection (Successful As Boolean, NewSocket As Socket)
    If Successful Then
      '  ToastMessageShow("Connected", False)
        Socket1 = NewSocket
       AStreams.Initialize(Socket1.InputStream, Socket1.OutputStream, "AStreams")
    Else
        ToastMessageShow(LastException.Message, True)
    End If
    Server.Listen
End Sub

Sub AStreams_NewData (Buffer() As Byte)

     msg2 = Conv.StringFromBytes(Buffer, "UTF8")
    ' Label2.Text = "buf.length:" &Buffer.Length
   
     If msg2.IndexOf(Chr(13)) >0 Then
        i = msg2.IndexOf(Chr(13))
        j = msg2.LastIndexOf(Chr(13))
        k = msg2.IndexOf2(Chr(13),i+1)

              Dim arry() As Byte
              Dim In As InputStream
        'Label2.Text = "index nr:"& i&" " & k&" " & j

                msg =msg & msg2.SubString2(0,i)
                Label1.Text ="parsed data:" & msg.Length
                    arry = Conv.StringToBytes(msg,"UTF8")
          
            Try
            In.InitializeFromBytesArray(arry, 0, arry.Length)
            Dim bmp As Bitmap
            bmp.Initialize2(In)
          
            Dim r As Rect
            r.Initialize(0, 0, Panel1.Width, Panel1.Height)
            cvs.DrawBitmap(bmp, Null, r)
            Panel1.Invalidate
          
            Catch
            End Try
          
                msg = ""

                 Do While i<>msg2.LastIndexOf(Chr(13))
                 If  msg2.IndexOf2(Chr(13),i) >=0 Then
                 j = msg2.IndexOf2(Chr(13),i+1)
                    msg = msg2.SubString2(i+1,j)
                  
            'Try
            'In.InitializeFromBytesArray(arry, 0, arry.Length)
            'Dim bmp As Bitmap
            'bmp.Initialize2(In)
          
            'Dim r As Rect
            'r.Initialize(0, 0, Panel1.Width, Panel1.Height)
            'cvs.DrawBitmap(bmp, Null, r)
            'Panel1.Invalidate
          
        '    Catch
    '        End Try
                  
                    Label2.Text = msg
                  
                    msg=""
                    i=j
                End If
                Loop
        msg = msg2.SubString2(i+1,msg2.Length)
      
     Else
         msg = msg & msg2
        Label2.Text = "index nr:0   0   0"
    End If
    Log(msg)
End Sub

Sub AStreams_Error
    ToastMessageShow(LastException.Message, True)
End Sub

'press on the Done button to send text
Sub EditText1_EnterPressed
    If AStreams.IsInitialized = False Then Return
    If EditText1.Text.Length > 0 Then
        Dim buffer() As Byte
        buffer = EditText1.Text.GetBytes("UTF8")
        AStreams.Write(buffer)
        EditText1.SelectAll
        Log("Sending: " & EditText1.Text)
    End If
End Sub

Sub Activity_Pause(UserClosed As Boolean)
    If UserClosed Then
        Log("closing")
        AStreams.Close
        Socket1.Close
    End If
End Sub
First in AStreams_New_Data i receive a byte array and make a string from it to parse a data. I check where are Carriage Returns in received buffer and then build complete messages from it. After that i make back a byte array from string and try to create bitmap. It works very well when i test it whith string messages. It doesn't work with bitmaps as well. I've got log: "java.lang.RuntimeException: Error loading bitmap."

Can you give me any advice? What am i doing wrong?
 
Last edited:

blooddiamond

Member
Licensed User
Why are you converting the bytes to a string? Raw bytes are not valid strings.
i want to easy split messages received in buffer(). Should I search for my separators directly in byte array and then divide it into separate arrays ?

Can anybody give me advice how to use AsyncStreams in regular mode to do this? Do i have to use for eg. any checksums or data check algorithms to check where one frame ends and starts another one?
I mean, how to deal with packages of data...
 
Last edited:

blooddiamond

Member
Licensed User
so i have changed my C++ client code to make it 'prefix mode' possible.
i've added a prefix with frame size:
PHP:
 vector<int> prefix;
    prefix.push_back(buf.size());

socket.write_some(boost::asio::buffer(prefix), ignored_error);
it works and i can receive all of data - size matches.
but again i have an error in log : " java.lang.RuntimeException: Error loading bitmap." It doesnt matter if i send bmp or JPEG frames.

so it seems that my code for drawing bitmaps is wrong???

B4X:
Sub AStreams_NewData (Buffer() As Byte)
Dim In As InputStream
    Try
    In.InitializeFromBytesArray(Buffer, 0, Buffer.Length)
           Dim bmp As Bitmap
            bmp.Initialize2(In)
            Dim r As Rect
            r.Initialize(0, 0, Panel1.Width, Panel1.Height)
            cvs.DrawBitmap(bmp, Null, r)
            Panel1.Invalidate
Catch
End Try
 

blooddiamond

Member
Licensed User
I've noticed that i had an error in my C++ code. It works pretty well now. It's another problem and i dont know whats wrong again. I've checked the code several times..
As a canvas on the panel it can be seen a bitmap but it is incomplete - its only a part of it. See the uploaded files...

PS: i checked the length of buffer and it is correct ...

test.jpg Screenshot_2015-05-31-15-53-47.png
 
Last edited:

blooddiamond

Member
Licensed User
I use designer only to add views, not to positioning elements. This is to have a good-looking app on several phones, not only on mine.
I,ve uploaded the project. It is based on AsyncStreams tutorial.
I don't think that panel is hidden by other views... it seems to be strange..

EDIT:
I changed Panel1 parameter ColorDrawable to BitmapDrawable and I can see full bitmap but have problems with width and height. I've got 720x1280 screen but when i make:
B4X:
Panel1.Width=720
  Panel1.Height=540
it is very small bitmap on the screen. Also when i made for eg.:
B4X:
Panel1.Width=Activity.Width
  Panel1.Height=Activity.Height/2
i got very small bitmap

it works and looks good for:
B4X:
Panel1.Width=1020
  Panel1.Height=765
 

Attachments

Last edited:
Top