Android Question PC openCv client and Android server image stream

[blooddiamond]

Hello, at the beginning please forgive me my language mistakes.

Im running a client app (C++) on my raspbbery Pi using openCv and boost asio libs. Client sends frames from webcamera to an android server app. Everything works fine when it works on server and client running on raspberry (both C++ programs). I have also made phone to phone connection program something like CCTV using AsyncStreams in prefix mode and also worked fine. Now i want to have Raspberry - Phone connection and it makes me lots of problems.

This is my C++ client code:

int main()
{
    std::string text1 = "this is some\r text, to change,\r if parsing data works ";
   VideoCapture cap(0);                        /* open webcam */
   if(!cap.isOpened())
   {
       return -1;
   }

   Mat frame;
   cap.set(CV_CAP_PROP_FRAME_WIDTH, 320);      /* set width */
   cap.set(CV_CAP_PROP_FRAME_HEIGHT, 240);     /* set height */

   while (true)
   {
       cap>>frame;
       imshow("client",frame);
      // waitKey(100);

       vector<int> p;
       p.push_back(CV_IMWRITE_JPEG_QUALITY);
       p.push_back(80);
       vector<uchar> buf;

      imencode(".bmp", frame, buf);    //encode to bitmap or JPEG ...
      buf.push_back(char(13));         // add Carriage Return character

      ////////////// when in string sending//////////////////////////////////////////////////
   frame = (frame.reshape(0,1)); // to make it continuous

   int num=frame.total();                      /* num = 320*240 */
   int num2=frame.elemSize();                  /* mun2 =3 channel */
   int  imgSize = frame.total()*frame.elemSize();

////////////////////////////////////////////////////////////////////////////////////////////////


   try
   {
       boost::asio::io_service io_service;
       tcp::endpoint end_point(boost::asio::ip::address::from_string("192.168.0.18"), 3200);

       tcp::socket socket(io_service);
       socket.connect(end_point);
       boost::system::error_code ignored_error;
       io_service.run();

   //    std::string message((char *)frame.data,230400); /* the size of mat data is 320*240*3 */
    //   message.insert(230400,"\r\n");

       cout<<"sending...."<<buf.size()<<endl;
       socket.write_some(boost::asio::buffer(buf), ignored_error);

       cout<<"send image finished"<<endl;
   }
   catch (std::exception& e)
   {
       std::cerr << e.what() << std::endl;
   }

   }
   return 0;

I can send a frame as string but then it is a 3-channel image in one continous vector so i create a bitmap or jpg (imencode) from the frame and then send it to a server.

This is my simple test server app running on phone and based on AsyncStreams in regular mode

Sub Process_Globals
    Dim AStreams As AsyncStreams
    Dim Server As ServerSocket
    Dim Socket1 As Socket
End Sub
Sub Globals
    Dim Conv As ByteConverter
    Dim EditText1 As EditText
    Dim Label1 As Label
    Dim Label2 As Label
    Dim msg As String
    Dim msg2 As String
    Dim i As Int
    Dim j As Int
    Dim k As Int
    Dim Panel1 As Panel
    Private cvs As Canvas
End Sub

Sub Activity_Create(FirstTime As Boolean)
Activity.LoadLayout("dupa1")
    Activity.Title="eeeee"
    If FirstTime Then
        Server.Initialize(3200, "Server")
        Server.Listen
        Log("MyIp = " & Server.GetMyIP)
    End If
  ' EditText1.Initialize("EditText1")
     cvs.Initialize(Panel1)
    EditText1.ForceDoneButton = True
  '  Activity.AddView(EditText1, 10dip, 10dip, 300dip, 60dip)
  
  
    Label1.Width=Activity.Width
     Label1.Height=Activity.Height/7
     Label1.Top=(Activity.Height/10)*6
     Label1.Left=0
     Label1.Gravity = Gravity.CENTER_HORIZONTAL + Gravity.CENTER_VERTICAL
     Label1.TextSize=Activity.Height/10/5
     Label1.Text = "size/message"
  
    Label2.Width=Activity.Width
     Label2.Height=Activity.Height/10*3
     Label2.Top=Activity.Height/10*8
     Label2.Left=0
     Label2.Gravity = Gravity.CENTER_HORIZONTAL + Gravity.CENTER_VERTICAL
     Label2.TextSize=Activity.Height/10/5
     Label2.Text="waiting" & CRLF & "for data"
   
    Panel1.Width=Activity.Width
     Panel1.Height=Activity.Height/10*4
     Panel1.Top=(Activity.Height)/10*2
     Panel1.Left=0
End Sub

Sub Activity_Resume

End Sub

Sub Server_NewConnection (Successful As Boolean, NewSocket As Socket)
    If Successful Then
      '  ToastMessageShow("Connected", False)
        Socket1 = NewSocket
       AStreams.Initialize(Socket1.InputStream, Socket1.OutputStream, "AStreams")
    Else
        ToastMessageShow(LastException.Message, True)
    End If
    Server.Listen
End Sub

Sub AStreams_NewData (Buffer() As Byte)

     msg2 = Conv.StringFromBytes(Buffer, "UTF8")
    ' Label2.Text = "buf.length:" &Buffer.Length
   
     If msg2.IndexOf(Chr(13)) >0 Then
        i = msg2.IndexOf(Chr(13))
        j = msg2.LastIndexOf(Chr(13))
        k = msg2.IndexOf2(Chr(13),i+1)

              Dim arry() As Byte
              Dim In As InputStream
        'Label2.Text = "index nr:"& i&" " & k&" " & j

                msg =msg & msg2.SubString2(0,i)
                Label1.Text ="parsed data:" & msg.Length
                    arry = Conv.StringToBytes(msg,"UTF8")
          
            Try
            In.InitializeFromBytesArray(arry, 0, arry.Length)
            Dim bmp As Bitmap
            bmp.Initialize2(In)
          
            Dim r As Rect
            r.Initialize(0, 0, Panel1.Width, Panel1.Height)
            cvs.DrawBitmap(bmp, Null, r)
            Panel1.Invalidate
          
            Catch
            End Try
          
                msg = ""

                 Do While i<>msg2.LastIndexOf(Chr(13))
                 If  msg2.IndexOf2(Chr(13),i) >=0 Then
                 j = msg2.IndexOf2(Chr(13),i+1)
                    msg = msg2.SubString2(i+1,j)
                  
            'Try
            'In.InitializeFromBytesArray(arry, 0, arry.Length)
            'Dim bmp As Bitmap
            'bmp.Initialize2(In)
          
            'Dim r As Rect
            'r.Initialize(0, 0, Panel1.Width, Panel1.Height)
            'cvs.DrawBitmap(bmp, Null, r)
            'Panel1.Invalidate
          
        '    Catch
    '        End Try
                  
                    Label2.Text = msg
                  
                    msg=""
                    i=j
                End If
                Loop
        msg = msg2.SubString2(i+1,msg2.Length)
      
     Else
         msg = msg & msg2
        Label2.Text = "index nr:0   0   0"
    End If
    Log(msg)
End Sub

Sub AStreams_Error
    ToastMessageShow(LastException.Message, True)
End Sub

'press on the Done button to send text
Sub EditText1_EnterPressed
    If AStreams.IsInitialized = False Then Return
    If EditText1.Text.Length > 0 Then
        Dim buffer() As Byte
        buffer = EditText1.Text.GetBytes("UTF8")
        AStreams.Write(buffer)
        EditText1.SelectAll
        Log("Sending: " & EditText1.Text)
    End If
End Sub

Sub Activity_Pause(UserClosed As Boolean)
    If UserClosed Then
        Log("closing")
        AStreams.Close
        Socket1.Close
    End If
End Sub

First in AStreams_New_Data i receive a byte array and make a string from it to parse a data. I check where are Carriage Returns in received buffer and then build complete messages from it. After that i make back a byte array from string and try to create bitmap. It works very well when i test it whith string messages. It doesn't work with bitmaps as well. I've got log: "java.lang.RuntimeException: Error loading bitmap."

Can you give me any advice? What am i doing wrong?

 

[Erel]

Why are you converting the bytes to a string? Raw bytes are not valid strings.

 

[blooddiamond]

Why are you converting the bytes to a string? Raw bytes are not valid strings.

i want to easy split messages received in buffer(). Should I search for my separators directly in byte array and then divide it into separate arrays ?

Can anybody give me advice how to use AsyncStreams in regular mode to do this? Do i have to use for eg. any checksums or data check algorithms to check where one frame ends and starts another one?
I mean, how to deal with packages of data...

 

[blooddiamond]

so i have changed my C++ client code to make it 'prefix mode' possible.
i've added a prefix with frame size:

 vector<int> prefix;
    prefix.push_back(buf.size());

socket.write_some(boost::asio::buffer(prefix), ignored_error);

it works and i can receive all of data - size matches.
but again i have an error in log : " java.lang.RuntimeException: Error loading bitmap." It doesnt matter if i send bmp or JPEG frames.

so it seems that my code for drawing bitmaps is wrong???

Sub AStreams_NewData (Buffer() As Byte)
Dim In As InputStream
    Try
    In.InitializeFromBytesArray(Buffer, 0, Buffer.Length)
           Dim bmp As Bitmap
            bmp.Initialize2(In)
            Dim r As Rect
            r.Initialize(0, 0, Panel1.Width, Panel1.Height)
            cvs.DrawBitmap(bmp, Null, r)
            Panel1.Invalidate
Catch
End Try

 

[Erel]

Don't use empty catch blocks. You will lose important information.

1. Check the length of the buffer? Is it correct?
2. Save the buffer to a file and check its content (copy it to the pc).

 

[blooddiamond]

I've noticed that i had an error in my C++ code. It works pretty well now. It's another problem and i dont know whats wrong again. I've checked the code several times..
As a canvas on the panel it can be seen a bitmap but it is incomplete - its only a part of it. See the uploaded files...

PS: i checked the length of buffer and it is correct ...

 

[Erel]

Why aren't you using the designer to create the layout? It will be simpler. Maybe the panel is hidden by the other views? If you like upload your project (File - Export as zip) and it will be easier to help you.

 

[blooddiamond]

I use designer only to add views, not to positioning elements. This is to have a good-looking app on several phones, not only on mine.
I,ve uploaded the project. It is based on AsyncStreams tutorial.
I don't think that panel is hidden by other views... it seems to be strange..

EDIT:
I changed Panel1 parameter ColorDrawable to BitmapDrawable and I can see full bitmap but have problems with width and height. I've got 720x1280 screen but when i make:

Panel1.Width=720
  Panel1.Height=540

it is very small bitmap on the screen. Also when i made for eg.:

Panel1.Width=Activity.Width
  Panel1.Height=Activity.Height/2

i got very small bitmap

it works and looks good for:

Panel1.Width=1020
  Panel1.Height=765

 

[Erel]

See the warnings. You need to use dip units. Anyway this is not the correct way to build the layout.

Use the designer with anchors and script instead.

Note that (Activity.Height)/10 * 2 is the same as writing: 20%y