Android Question PC openCv client and Android server image stream

Discussion in 'Android Questions' started by blooddiamond, May 23, 2015.

  1. blooddiamond

    blooddiamond Member Licensed User

    Hello, at the beginning please forgive me my language mistakes.

    Im running a client app (C++) on my raspbbery Pi using openCv and boost asio libs. Client sends frames from webcamera to an android server app. Everything works fine when it works on server and client running on raspberry (both C++ programs). I have also made phone to phone connection program something like CCTV using AsyncStreams in prefix mode and also worked fine. Now i want to have Raspberry - Phone connection and it makes me lots of problems.

    This is my C++ client code:
    PHP:
    int main()
    {
        
    std::string text1 "this is some\r text, to change,\r if parsing data works ";
       
    VideoCapture cap(0);                        /* open webcam */
       
    if(!cap.isOpened())
       {
           return -
    1;
       }

       
    Mat frame;
       
    cap.set(CV_CAP_PROP_FRAME_WIDTH320);      /* set width */
       
    cap.set(CV_CAP_PROP_FRAME_HEIGHT240);     /* set height */

       
    while (true)
       {
           
    cap>>frame;
           
    imshow("client",frame);
          
    // waitKey(100);

           
    vector<intp;
           
    p.push_back(CV_IMWRITE_JPEG_QUALITY);
           
    p.push_back(80);
           
    vector<ucharbuf;

          
    imencode(".bmp"framebuf);    //encode to bitmap or JPEG ...
          
    buf.push_back(char(13));         // add Carriage Return character

          ////////////// when in string sending//////////////////////////////////////////////////
       
    frame = (frame.reshape(0,1)); // to make it continuous

       
    int num=frame.total();                      /* num = 320*240 */
       
    int num2=frame.elemSize();                  /* mun2 =3 channel */
       
    int  imgSize frame.total()*frame.elemSize();

    ////////////////////////////////////////////////////////////////////////////////////////////////


       
    try
       {
           
    boost::asio::io_service io_service;
           
    tcp::endpoint end_point(boost::asio::ip::address::from_string("192.168.0.18"), 3200);

           
    tcp::socket socket(io_service);
           
    socket.connect(end_point);
           
    boost::system::error_code ignored_error;
           
    io_service.run();

       
    //    std::string message((char *)frame.data,230400); /* the size of mat data is 320*240*3 */
        //   message.insert(230400,"\r\n");

           
    cout<<"sending...."<<buf.size()<<endl;
           
    socket.write_some(boost::asio::buffer(buf), ignored_error);

           
    cout<<"send image finished"<<endl;
       }
       catch (
    std::exceptione)
       {
           
    std::cerr << e.what() << std::endl;
       }

       }
       return 
    0;

    I can send a frame as string but then it is a 3-channel image in one continous vector so i create a bitmap or jpg (imencode) from the frame and then send it to a server.

    This is my simple test server app running on phone and based on AsyncStreams in regular mode

    Code:
    Sub Process_Globals
        
    Dim AStreams As AsyncStreams
        
    Dim Server As ServerSocket
        
    Dim Socket1 As Socket
    End Sub
    Sub Globals
        
    Dim Conv As ByteConverter
        
    Dim EditText1 As EditText
        
    Dim Label1 As Label
        
    Dim Label2 As Label
        
    Dim msg As String
        
    Dim msg2 As String
        
    Dim i As Int
        
    Dim j As Int
        
    Dim k As Int
        
    Dim Panel1 As Panel
        
    Private cvs As Canvas
    End Sub

    Sub Activity_Create(FirstTime As Boolean)
    Activity.LoadLayout("dupa1")
        
    Activity.Title="eeeee"
        
    If FirstTime Then
            
    Server.Initialize(3200"Server")
            
    Server.Listen
            
    Log("MyIp = " & Server.GetMyIP)
        
    End If
      
    ' EditText1.Initialize("EditText1")
         cvs.Initialize(Panel1)
        EditText1.ForceDoneButton = 
    True
      
    '  Activity.AddView(EditText1, 10dip, 10dip, 300dip, 60dip)
      
      
        Label1.Width=
    Activity.Width
         Label1.Height=
    Activity.Height/7
         Label1.Top=(
    Activity.Height/10)*6
         Label1.Left=
    0
         Label1.Gravity = 
    Gravity.CENTER_HORIZONTAL + Gravity.CENTER_VERTICAL
         Label1.TextSize=
    Activity.Height/10/5
         Label1.Text = 
    "size/message"
      
        Label2.Width=
    Activity.Width
         Label2.Height=
    Activity.Height/10*3
         Label2.Top=
    Activity.Height/10*8
         Label2.Left=
    0
         Label2.Gravity = 
    Gravity.CENTER_HORIZONTAL + Gravity.CENTER_VERTICAL
         Label2.TextSize=
    Activity.Height/10/5
         Label2.Text=
    "waiting" & CRLF & "for data"
       
        Panel1.Width=
    Activity.Width
         Panel1.Height=
    Activity.Height/10*4
         Panel1.Top=(
    Activity.Height)/10*2
         Panel1.Left=
    0
    End Sub

    Sub Activity_Resume

    End Sub

    Sub Server_NewConnection (Successful As Boolean, NewSocket As Socket)
        
    If Successful Then
          
    '  ToastMessageShow("Connected", False)
            Socket1 = NewSocket
           AStreams.Initialize(Socket1.InputStream, Socket1.OutputStream, 
    "AStreams")
        
    Else
            
    ToastMessageShow(LastException.Message, True)
        
    End If
        
    Server.Listen
    End Sub

    Sub AStreams_NewData (Buffer() As Byte)

         msg2 = Conv.StringFromBytes(Buffer, 
    "UTF8")
        
    ' Label2.Text = "buf.length:" &Buffer.Length
       
         
    If msg2.IndexOf(Chr(13)) >0 Then
            i = msg2.IndexOf(
    Chr(13))
            j = msg2.LastIndexOf(
    Chr(13))
            k = msg2.IndexOf2(
    Chr(13),i+1)

                  
    Dim arry() As Byte
                  
    Dim In As InputStream
            
    'Label2.Text = "index nr:"& i&" " & k&" " & j

                    msg =msg & msg2.SubString2(
    0,i)
                    Label1.Text =
    "parsed data:" & msg.Length
                        arry = Conv.StringToBytes(msg,
    "UTF8")
              
                
    Try
                
    In.InitializeFromBytesArray(arry, 0, arry.Length)
                
    Dim bmp As Bitmap
                bmp.Initialize2(
    In)
              
                
    Dim r As Rect
                r.Initialize(
    00, Panel1.Width, Panel1.Height)
                cvs.DrawBitmap(bmp, 
    Null, r)
                Panel1.Invalidate
              
                
    Catch
                
    End Try
              
                    msg = 
    ""

                     
    Do While i<>msg2.LastIndexOf(Chr(13))
                     
    If  msg2.IndexOf2(Chr(13),i) >=0 Then
                     j = msg2.IndexOf2(
    Chr(13),i+1)
                        msg = msg2.SubString2(i+
    1,j)
                      
                
    'Try
                'In.InitializeFromBytesArray(arry, 0, arry.Length)
                'Dim bmp As Bitmap
                'bmp.Initialize2(In)
              
                
    'Dim r As Rect
                'r.Initialize(0, 0, Panel1.Width, Panel1.Height)
                'cvs.DrawBitmap(bmp, Null, r)
                'Panel1.Invalidate
              
            
    '    Catch
        '        End Try
                      
                        Label2.Text = msg
                      
                        msg=
    ""
                        i=j
                    
    End If
                    
    Loop
            msg = msg2.SubString2(i+
    1,msg2.Length)
          
         
    Else
             msg = msg & msg2
            Label2.Text = 
    "index nr:0   0   0"
        
    End If
        
    Log(msg)
    End Sub

    Sub AStreams_Error
        
    ToastMessageShow(LastException.Message, True)
    End Sub

    'press on the Done button to send text
    Sub EditText1_EnterPressed
        
    If AStreams.IsInitialized = False Then Return
        
    If EditText1.Text.Length > 0 Then
            
    Dim buffer() As Byte
            buffer = EditText1.Text.GetBytes(
    "UTF8")
            AStreams.Write(buffer)
            EditText1.SelectAll
            
    Log("Sending: " & EditText1.Text)
        
    End If
    End Sub

    Sub Activity_Pause(UserClosed As Boolean)
        
    If UserClosed Then
            
    Log("closing")
            AStreams.Close
            Socket1.Close
        
    End If
    End Sub
    First in AStreams_New_Data i receive a byte array and make a string from it to parse a data. I check where are Carriage Returns in received buffer and then build complete messages from it. After that i make back a byte array from string and try to create bitmap. It works very well when i test it whith string messages. It doesn't work with bitmaps as well. I've got log: "java.lang.RuntimeException: Error loading bitmap."

    Can you give me any advice? What am i doing wrong?
     
    Last edited: May 23, 2015
  2. Erel

    Erel Administrator Staff Member Licensed User

    Why are you converting the bytes to a string? Raw bytes are not valid strings.
     
  3. blooddiamond

    blooddiamond Member Licensed User

    i want to easy split messages received in buffer(). Should I search for my separators directly in byte array and then divide it into separate arrays ?

    Can anybody give me advice how to use AsyncStreams in regular mode to do this? Do i have to use for eg. any checksums or data check algorithms to check where one frame ends and starts another one?
    I mean, how to deal with packages of data...
     
    Last edited: May 24, 2015
  4. blooddiamond

    blooddiamond Member Licensed User

    so i have changed my C++ client code to make it 'prefix mode' possible.
    i've added a prefix with frame size:
    PHP:
     vector<intprefix;
        
    prefix.push_back(buf.size());

    socket.write_some(boost::asio::buffer(prefix), ignored_error);
    it works and i can receive all of data - size matches.
    but again i have an error in log : " java.lang.RuntimeException: Error loading bitmap." It doesnt matter if i send bmp or JPEG frames.

    so it seems that my code for drawing bitmaps is wrong???

    Code:
    Sub AStreams_NewData (Buffer() As Byte)
    Dim In As InputStream
        
    Try
        
    In.InitializeFromBytesArray(Buffer, 0, Buffer.Length)
               
    Dim bmp As Bitmap
                bmp.Initialize2(
    In)
                
    Dim r As Rect
                r.Initialize(
    00, Panel1.Width, Panel1.Height)
                cvs.DrawBitmap(bmp, 
    Null, r)
                Panel1.Invalidate
    Catch
    End Try
     
  5. Erel

    Erel Administrator Staff Member Licensed User

    Don't use empty catch blocks. You will lose important information.

    1. Check the length of the buffer? Is it correct?
    2. Save the buffer to a file and check its content (copy it to the pc).
     
  6. blooddiamond

    blooddiamond Member Licensed User

    I've noticed that i had an error in my C++ code. It works pretty well now. It's another problem and i dont know whats wrong again. I've checked the code several times..
    As a canvas on the panel it can be seen a bitmap but it is incomplete - its only a part of it. See the uploaded files...

    PS: i checked the length of buffer and it is correct ...

    test.jpg Screenshot_2015-05-31-15-53-47.png
     
    Last edited: May 31, 2015
  7. Erel

    Erel Administrator Staff Member Licensed User

    Why aren't you using the designer to create the layout? It will be simpler. Maybe the panel is hidden by the other views? If you like upload your project (File - Export as zip) and it will be easier to help you.
     
  8. blooddiamond

    blooddiamond Member Licensed User

    I use designer only to add views, not to positioning elements. This is to have a good-looking app on several phones, not only on mine.
    I,ve uploaded the project. It is based on AsyncStreams tutorial.
    I don't think that panel is hidden by other views... it seems to be strange..

    EDIT:
    I changed Panel1 parameter ColorDrawable to BitmapDrawable and I can see full bitmap but have problems with width and height. I've got 720x1280 screen but when i make:
    Code:
    Panel1.Width=720
      Panel1.Height=
    540
    it is very small bitmap on the screen. Also when i made for eg.:
    Code:
    Panel1.Width=Activity.Width
      Panel1.Height=
    Activity.Height/2
    i got very small bitmap

    it works and looks good for:
    Code:
    Panel1.Width=1020
      Panel1.Height=
    765
     

    Attached Files:

    Last edited: Jun 1, 2015
  9. Erel

    Erel Administrator Staff Member Licensed User

    See the warnings. You need to use dip units. Anyway this is not the correct way to build the layout.

    Use the designer with anchors and script instead.

    Note that (Activity.Height)/10 * 2 is the same as writing: 20%y
     
Loading...
  1. This site uses cookies to help personalise content, tailor your experience and to keep you logged in if you register.
    By continuing to use this site, you are consenting to our use of cookies.
    Dismiss Notice