B4A Library NewHandwave - detecting hand motion (left, right, up, down) with front facing camera

I came across this Github project and thought it might be interesting to see if I could "shortcut" wrap it and make it work "from B4A".

The B4A library files are too large to post here. You can download them from here:
https://www.dropbox.com/s/83w5g8tolgd4n31/NewHandWaveLibFiles.zip?dl=0

...or you can download the B4A generated APK files from here and install the trial app:
https://www.dropbox.com/s/1s6d15owfkqzldd/b4aNewHandWave.apk?dl=0

You need to enable AppCompat V3.20 and DesignSupport V2.00 to make use of this app.

Start the app and then move your have from left to right, right to left, bottom to top, and top to bottom over the front facing camera (about 15cm or so above the camera) - it should report the detected motion via a toast message.

Extract resource.zip, DemoRes.Zip, and LibRes.zip and copy the folders with their contents to be on the same folder level as that of the /Files and /Objects folders of the B4A project

Have tested this on S4 mini with KitKat and also on a device with Lollipop.

I will spend some time on it and try to convert this to a library that can be used 100% from within B4A - but this posting just to demonstrate the motion detection via the front facing camera

1.png


2.png


3.png


4.png


5.png
 

Attachments

  • resource.zip
    424 bytes · Views: 264
  • LibRes.zip
    446 bytes · Views: 256
  • b4aNewHandWave.zip
    8.3 KB · Views: 265
  • DemoRes.zip
    32.9 KB · Views: 249
Last edited:

Johan Schoeman

Expert
Licensed User
Longtime User
OK - now 100% usable from within B4A. Probably one of the most interesting wrappers that I have done......Attached the B4A sample project whereby you can move the label around by means of left/right/up/down hand movements over the front facing camera (label added in B4A Designer)

You should be able to for eg scroll a wheel picker with hand gestures....

You need to download the new B4A library files from here (V1.01):
https://www.dropbox.com/s/jsbnkpljwmbxupw/NewHandWaveLibFiles_V1_01.zip?dl=0

No need for the /resource, /LibRes, and /DemoRes folders.

Take note of the designer script text for positioning of the CustomView (its Top is at 100%y)
Take note of the events that are raised in the B4A project
Added a method to the wrapper so that you can show/hide the toast message.

6.png



Sample Code:
B4X:
#Region  Project Attributes
    #ApplicationLabel: b4aNewHandWave
    #VersionCode: 1
    #VersionName:
    'SupportedOrientations possible values: unspecified, landscape or portrait.
    #SupportedOrientations: landscape
    #CanInstallToExternalStorage: False
#End Region

'#AdditionalRes: ..\DemoRes
'#AdditionalRes: ..\LibRes
'#AdditionalRes: ..\resource

'#AdditionalRes: C:\Android\extras\android\support\v7\appcompat\res, android.support.v7.appcompat
'#AdditionalRes: C:\Android\extras\google\google-play-services\libproject\google-play-services_lib\res, com.google.android.gms
'#AdditionalRes: C:\Android\extras\android\support\design\res, android.support.design

'#Extends: android.support.v7.app.AppCompatActivity

#Region  Activity Attributes
    #FullScreen: False
    #IncludeTitle: True
#End Region

Sub Process_Globals
    'These global variables will be declared once when the application starts.
    'These variables can be accessed from all modules.

End Sub

Sub Globals
    'These global variables will be redeclared each time the activity is created.
    'These variables can only be accessed from this module.
 
    Private hv As NewHandWave
    Private Label2 As Label
 
End Sub

Sub Activity_Create(FirstTime As Boolean)
    'Do not forget to load the layout file created with the visual designer. For example:
    Activity.LoadLayout("main")
    'hv.Initialize("hv")
    hv.ShowToast = False
 

End Sub

Sub Activity_Resume
 
hv.Start

End Sub

Sub Activity_Pause (UserClosed As Boolean)
 
hv.Stop
 

End Sub



Sub hv_gesture_left
 
    If Label2.Left <> 0%x Then
        Label2.SetLayoutAnimated(1000, Label2.Left - 5%x, Label2.Top, Label2.Width, Label2.Height)
    End If
 
 
End Sub

Sub hv_gesture_right
 
 
    If Label2.Left <> 100%x Then
        Label2.SetLayoutAnimated(1000, Label2.Left + 5%x, Label2.Top, Label2.Width, Label2.Height)
    End If
 
 
End Sub

Sub hv_gesture_up
 
    If Label2.top <> 0%y Then
        Label2.SetLayoutAnimated(1000, Label2.Left, Label2.Top - 5%y, Label2.Width, Label2.Height)
    End If
 
 
End Sub

Sub hv_gesture_down
 
    If Label2.top <> 100%y - Label2.height Then
        Label2.SetLayoutAnimated(1000, Label2.Left, Label2.Top + 5%y, Label2.Width, Label2.Height)
    End If 
 
 
End Sub

Library:
NewHandWave
Author:
Github: KraftyKoala, Wrapped by: Johan Schoeman
Version: 1.01
  • NewHandWave
    Events:
    • gesture_down
    • gesture_left
    • gesture_right
    • gesture_up
    Fields:
    • ba As BA
    Methods:
    • BringToFront
    • DesignerCreateView (base As PanelWrapper, lw As LabelWrapper, props As Map)
    • Initialize (EventName As String)
    • Invalidate
    • Invalidate2 (arg0 As Rect)
    • Invalidate3 (arg0 As Int, arg1 As Int, arg2 As Int, arg3 As Int)
    • IsInitialized As Boolean
    • RemoveView
    • RequestFocus As Boolean
    • SendToBack
    • SetBackgroundImage (arg0 As Bitmap)
    • SetColorAnimated (arg0 As Int, arg1 As Int, arg2 As Int)
    • SetLayout (arg0 As Int, arg1 As Int, arg2 As Int, arg3 As Int)
    • SetLayoutAnimated (arg0 As Int, arg1 As Int, arg2 As Int, arg3 As Int, arg4 As Int)
    • SetVisibleAnimated (arg0 As Int, arg1 As Boolean)
    • Start
    • Stop
    • loadOpenCV
    • onGestureDown (caller As CameraGestureSensor, gestureLength As Long)
    • onGestureLeft (caller As CameraGestureSensor, gestureLength As Long)
    • onGestureRight (caller As CameraGestureSensor, gestureLength As Long)
    • onGestureUp (caller As CameraGestureSensor, gestureLength As Long)
    • retrieveControls
    Permissions:
    • android.permission.CAMERA
    • android.permission.RECORD_AUDIO
    • android.permission.WRITE_EXTERNAL_STORAGE
    Properties:
    • Background As Drawable
    • Color As Int [write only]
    • Enabled As Boolean
    • Height As Int
    • Left As Int
    • Parent As Object [read only]
    • ShowToast As Boolean [write only]
    • Tag As Object
    • Top As Int
    • Visible As Boolean
    • Width As Int

No "funnies" required in the B4A Manifest file.
 

Attachments

  • b4aNewHandWave.zip
    8.4 KB · Views: 255
  • TheJavaCode.zip
    180.6 KB · Views: 227
Last edited:

MarcoRome

Expert
Licensed User
Longtime User
Hi great work john.
Is possible start this also in a SERVICE ?
Thank you
Marco
 

Johan Schoeman

Expert
Licensed User
Longtime User
Hi great work john.
Is possible start this also in a SERVICE ?
Thank you
Marco
Hi Marco, have not tried from a service. I am a dummy with B4A so best you try it and see what is does.
 

Johan Schoeman

Expert
Licensed User
Longtime User
Hi Johan, Do you know why appear this error?

I have a Nexus 4 with Android 5.1.1 and lastest version of B4A.

Thanks
Hi Alberto

@Informatix have the same problem with his Moto G. Seems like there is something in the parameters that are being set that his Moto G and your Nexus 4 does not like. Will have browse the web and see if I can find which one of the parameters is causing the crash. I have it working perfectly on 2 Samsumg devices and on a Vodaphone device (KitKat and Lollipop).

If I find something I will update the lib to accommodate the Nexus 4 and the Moto G
 

Johan Schoeman

Expert
Licensed User
Longtime User
Hey Johan,

Any news about "setParameter"??


thanks
Hi Alberto
Download V1.02 from here - https://www.dropbox.com/s/vl6wbbypt79ucgw/NewHandWave_V1_02.zip?dl=0. This is a lead that @Informatix gave me. Seems to me that the Nexus 5 don't support the ISO setting. So, I have added the following three methods to the library - add them to Sub Activity_Create:
B4X:
hv.ExposureCompensation = 2
hv.Enable_ISO_setting = False
hv.EnableWhiteBalance = True

I would suggest suggest you start with the code above (i.e disabling the ISO setting) and see if it works. If not, read up on the other 2 settings as far as your Nexus 5 is concerned and see if you can find anything unusual with regards to ExposureCompensation and WhiteBalance.

If the above does not solve your problem then see what you can find wrt these 3 methods for your Nexus 5 and I will then try to make changes to accommodate them in the library. I am shooting in the dark here as I don't have a Nexus 5 device to test it on.

JS
 

Johan Schoeman

Expert
Licensed User
Longtime User
My smartphone for tests is a NEXUS 4, not NEXUS 5
Android 5.1.1

Look the errors now put this new settings....

i tried to put False for whitebalance too

Thanks
Alberto, I don't understand why you have messages in the log stating that "hv" and "Label2" are not initialized. If you open the designer can you see the "hv" customview that has been added in the designer and the general designer script:
B4X:
'All variants script
AutoScaleAll


hv.Left = 0%x
hv.Top = 100%y
hv.width = 100%x
hv.Height = 100%y

Label2.Left = 35%x
Label2.Top = 45%y
Label2.Width = 30%x
Label2.Height = 15%y


Can you upload your B4A project?
 

Johan Schoeman

Expert
Licensed User
Longtime User
If I put hv.initialize before everything the error occurs, if I put those parameters before initialize, not get the error, but not work!

Look if this sample is ok.... please!
Are you using the B4A project in post #3? Your B4A manifest file seems to be from the project in post #1
 

Johan Schoeman

Expert
Licensed User
Longtime User
If I put hv.initialize before everything the error occurs, if I put those parameters before initialize, not get the error, but not work!

Look if this sample is ok.... please!
You need to use the B4A project in post #3 with the lib file (link) in post #9
 

Johan Schoeman

Expert
Licensed User
Longtime User
If I put hv.initialize before everything the error occurs, if I put those parameters before initialize, not get the error, but not work!

Look if this sample is ok.... please!
Here is the B4A project again...
 

Attachments

  • b4aNewHandWave.zip
    8.4 KB · Views: 172

Johan Schoeman

Expert
Licensed User
Longtime User
with this last example, look the error attach!

I think because is in layout and you set parameters in initialize, is correct? and not in the start method
Now use the same B4A project and copy the following code to it to replace the existing code:
B4X:
#Region  Project Attributes
    #ApplicationLabel: b4aNewHandWave
    #VersionCode: 1
    #VersionName:
    'SupportedOrientations possible values: unspecified, landscape or portrait.
    #SupportedOrientations: landscape
    #CanInstallToExternalStorage: False
#End Region

'#AdditionalRes: ..\DemoRes
'#AdditionalRes: ..\LibRes
'#AdditionalRes: ..\resource

'#AdditionalRes: C:\Android\extras\android\support\v7\appcompat\res, android.support.v7.appcompat
'#AdditionalRes: C:\Android\extras\google\google-play-services\libproject\google-play-services_lib\res, com.google.android.gms
'#AdditionalRes: C:\Android\extras\android\support\design\res, android.support.design

'#Extends: android.support.v7.app.AppCompatActivity

#Region  Activity Attributes
    #FullScreen: False
    #IncludeTitle: True
#End Region

Sub Process_Globals
    'These global variables will be declared once when the application starts.
    'These variables can be accessed from all modules.

End Sub

Sub Globals
    'These global variables will be redeclared each time the activity is created.
    'These variables can only be accessed from this module.
  
    Private hv As NewHandWave
    Private Label2 As Label
  
End Sub

Sub Activity_Create(FirstTime As Boolean)
    'Do not forget to load the layout file created with the visual designer. For example:
    Activity.LoadLayout("main")
    'hv.Initialize("hv")
    hv.ShowToast = False
  
hv.ExposureCompensation = 2
hv.Enable_ISO_setting = False
hv.EnableWhiteBalance = True
  
  

End Sub

Sub Activity_Resume
  
hv.Start

End Sub

Sub Activity_Pause (UserClosed As Boolean)
  
hv.Stop
  

End Sub



Sub hv_gesture_left
  
    If Label2.Left <> 0%x Then
        Label2.SetLayoutAnimated(200, Label2.Left - 5%x, Label2.Top, Label2.Width, Label2.Height)
    End If
  
  
End Sub

Sub hv_gesture_right
  
  
    If Label2.Left <> 100%x Then
        Label2.SetLayoutAnimated(200, Label2.Left + 5%x, Label2.Top, Label2.Width, Label2.Height)
    End If
  
  
End Sub

Sub hv_gesture_up
  
    If Label2.top <> 0%y Then
        Label2.SetLayoutAnimated(200, Label2.Left, Label2.Top - 5%y, Label2.Width, Label2.Height)
    End If
  
  
End Sub

Sub hv_gesture_down
  
    If Label2.top <> 100%y - Label2.height Then
        Label2.SetLayoutAnimated(200, Label2.Left, Label2.Top + 5%y, Label2.Width, Label2.Height)
    End If  
  
  
End Sub

And the play around with the 3 parameters as stated in post #9
 
Last edited:

Johan Schoeman

Expert
Licensed User
Longtime User
I tried too in a Samsung 5 and works, but not detect the movements very well :-(
I am using a Samsung S4 mini (with autofocus) and Vodaphone bla bla without autofocus and they both work well. Beware of the shadow that you might create (due to light) and also the distance from the front camera.
 
Top