B4i Library [Class] FaceDetector

[CaptKronos]

FaceDetector is a class that uses Apple's Vision framework which was introduced with iOS11.

The class basically returns a list of detected faces, and each face consists of a list of floats which represent the x and y coordinates of various face landmarks. The class explicitly exposes the left and right eye pupils, because those were the landmarks I'm interested in. To interpret the rest of the points you can refer here: https://stackoverflow.com/questions/45298639/ios11-vision-framework-mapping-all-face-landmarks.

When calling the scanFace function you have the option to request that the returned points are mapped to the bitmap's or an imageView's coordinates, or to just return the raw 0..1 values. The mapping is more convoluted than just multiplying by the image dimensions so I would advise going with one of the first two options.

afacedetect.scanFace(bmp, True, null) 'map to bmp's coordinates
afacedetect.scanFace(bmp, True, ImageView1) 'map to ImageView1's coordinates
afacedetect.scanFace(bmp, False, null) 'the raw values

Example of using the class

Sub Process_Globals
    'These global variables will be declared once when the application starts.
    'Public variables can be accessed from all modules.
    Public App As Application
    Public NavControl As NavigationController
    Private Page1 As Page
    Private afacedetect As FaceDetector
    Private ImageView1 As ImageView
End Sub

Private Sub Application_Start (Nav As NavigationController)
    'SetDebugAutoFlushLogs(True) 'Uncomment if program crashes before all logs are printed.
    NavControl = Nav
    Page1.Initialize("Page1")
    Page1.Title = "Page 1"
    Page1.RootPanel.Color = Colors.White
    NavControl.ShowPage(Page1)
End Sub

Private Sub Page1_Resize(Width As Int, Height As Int)
    ImageView1.Initialize("")
    Page1.RootPanel.AddView(ImageView1,0,0,Width,Height)
    Dim filename As String="XXXXXX"
    If File.Exists(File.DirAssets, filename)=False Then
        Msgbox("Please replace XXXXXX with a valid filename before running","")
        Return
    End If
    ImageView1.Bitmap=LoadBitmap(File.DirAssets , filename)
    afacedetect.Initialize(Me, "face_scanned")
    afacedetect.scanFace(LoadBitmap(File.DirAssets , filename), True, ImageView1)
End Sub

Sub mark(c As Canvas, pt As pointf, col As Int)
    c.DrawLine(pt.x-5,pt.y+5,pt.x+5,pt.y-5,col,2)
    c.DrawLine(pt.x-5,pt.y-5,pt.x+5,pt.y+5,col,2)
End Sub

Sub face_scanned
    Dim aFace As face
    Dim r As Rect
    If afacedetect.faces.Size=0 Then
        Msgbox("No faces","")
        Return
    End If
    Dim c As Canvas, col As Int
    c.Initialize(ImageView1)
    For i=0 To afacedetect.faces.Size-1
        aFace=afacedetect.faces.Get(i)
        r.Initialize(aFace.boundingRect.Left, aFace.boundingRect.Top, _
        aFace.boundingRect.Right, aFace.boundingRect.Bottom)
        If i=afacedetect.faceWithHighestConfidence Then
            col=Colors.white
        Else
            col=Colors.red
        End If
        c.DrawRect(r, col, False, 2)
        mark(c, aFace.leftPupil, col)
        mark(c, aFace.rightPupil, col)
        'uncomment if you want all points plotted
'        Dim pt As pointf
'        For j=0 To aFace.points.Size-2 Step 2
'            pt.x=aFace.points.Get(j)
'            pt.y=aFace.points.Get(j+1)
'            mark(c,pt, col)
'        Next
    Next
    c.Refresh
    c.Release
End Sub

 

[CaptKronos]

I have just updated FaceDetector.bas. I had the left and right pupils the wrong way around!