B4A Library [class] ML Kit Subject Segmentation

1714294035666.png



Input: image
Output: image with main subject

See attached project.
Don't miss the #AdditionalJars in the main module and the manifest editor snippets.

Usage:
B4X:
Private Sub Button1_Click
    Dim cc As ContentChooser
    cc.Initialize("cc")
    cc.Show("image/*", "choose image")
    Wait For cc_Result (Success As Boolean, Dir As String, FileName As String)
    If Success Then
        'I'm resizing the image here. It might be better to leave the image with the original resolution.
        Dim bmp As B4XBitmap = xui.LoadBitmapResize(Dir, FileName, B4XImageView1.mBase.Width, B4XImageView1.mBase.Height, True)
        B4XImageView1.Bitmap = bmp
        B4XImageView2.Clear
        Wait For (segmenter.Process(bmp)) Complete (Result As SegmentationResult)
        If Result.Success Then
            B4XImageView2.Bitmap = Result.ForegroundBitmap
        End If
    End If
End Sub

B4A Sdk Manager: search for mlkit and install all items. Search for odml and install as well. Don't install anything else.
 

Attachments

  • SubjectSegmentation.zip
    15.7 KB · Views: 64
Last edited:

Input: image
Output: image with main subject

See attached project.
Don't miss the #AdditionalJars in the main module and the manifest editor snippets.

Usage:
B4X:
Private Sub Button1_Click
    Dim cc As ContentChooser
    cc.Initialize("cc")
    cc.Show("image/*", "choose image")
    Wait For cc_Result (Success As Boolean, Dir As String, FileName As String)
    If Success Then
        'I'm resizing the image here. It might be better to leave the image with the original resolution.
        Dim bmp As B4XBitmap = xui.LoadBitmapResize(Dir, FileName, B4XImageView1.mBase.Width, B4XImageView1.mBase.Height, True)
        B4XImageView1.Bitmap = bmp
        B4XImageView2.Clear
        Wait For (segmenter.Process(bmp)) Complete (Result As SegmentationResult)
        If Result.Success Then
            B4XImageView2.Bitmap = Result.ForegroundBitmap
        End If
    End If
End Sub
While compiling I am Getting
Organizing libraries. Error
Maven artifact not found: com.google.firebase/firebase-encoders
Source: com.google.android.gms:play-services-mlkit-subject-segmentation
what should I download from b4a sdk manager to resolve the above error? b4a version is 12.80 and B4A Sdk Manager version: 4.10
 
Update to error message :
After installing:
com.google.firebase:firebase-encoders version 17.0.0
I get this error:
Dex merge Error
Error in c:\android\tools\..\extras\b4a_remote\com\google\firebase\firebase-encoders-json\16.1.0\unpacked-firebase-encoders-json-16.1.0\jars\classes.zip:classes.dex:
Type com.google.firebase.encoders.DataEncoder is defined multiple times: c:\android\tools\..\extras\b4a_remote\com\google\firebase\firebase-encoders-json\16.1.0\unpacked-firebase-encoders-json-16.1.0\jars\classes.zip:classes.dex, c:\android\tools\..\extras\b4a_remote\com\google\firebase\firebase-encoders\17.0.0\firebase-encoders-17.0.0.zip:classes.dex
Compilation failed
Do I have to remove anything already installed?
 

walterf25

Expert
Licensed User
Longtime User
View attachment 153169


Input: image
Output: image with main subject

See attached project.
Don't miss the #AdditionalJars in the main module and the manifest editor snippets.

Usage:
B4X:
Private Sub Button1_Click
    Dim cc As ContentChooser
    cc.Initialize("cc")
    cc.Show("image/*", "choose image")
    Wait For cc_Result (Success As Boolean, Dir As String, FileName As String)
    If Success Then
        'I'm resizing the image here. It might be better to leave the image with the original resolution.
        Dim bmp As B4XBitmap = xui.LoadBitmapResize(Dir, FileName, B4XImageView1.mBase.Width, B4XImageView1.mBase.Height, True)
        B4XImageView1.Bitmap = bmp
        B4XImageView2.Clear
        Wait For (segmenter.Process(bmp)) Complete (Result As SegmentationResult)
        If Result.Success Then
            B4XImageView2.Bitmap = Result.ForegroundBitmap
        End If
    End If
End Sub

B4A Sdk Manager: search for mlkit and install all items. Search for odml and install as well. Don't install anything else.
Modified this class a little to serve my purpose, thanks @Erel for this useful class, I had the need to retrieve the ForegroundMask so I modified the class a little bit, here are the changes made.
Segmentation:
'v1.01
Sub Class_Globals
    Type SegmentationResult (Success As Boolean, ForegroundBitmap As B4XBitmap, ForegroundMask As B4XBitmap)
    Private segmenter As JavaObject
    Private xui As XUI
End Sub

Public Sub Initialize
    Dim options As JavaObject
    options.InitializeNewInstance("com.google.mlkit.vision.segmentation.subject.SubjectSegmenterOptions.Builder", Null)
    options = options.RunMethodJO("enableForegroundBitmap", Null).RunMethodJO("enableForegroundConfidenceMask", Null).RunMethod("build", Null)
    Dim SubjectSegmentation As JavaObject
    SubjectSegmentation.InitializeStatic("com.google.mlkit.vision.segmentation.subject.SubjectSegmentation")
    segmenter = SubjectSegmentation.RunMethod("getClient", Array(options))
End Sub

Private Sub CreateInputImage(bmp As B4XBitmap) As Object
    Dim InputImage As JavaObject
    InputImage.InitializeStatic("com.google.mlkit.vision.common.InputImage")
    Return InputImage.RunMethod("fromBitmap", Array(bmp, 0))
End Sub

Public Sub Process (bmp As B4XBitmap, getForeGroundMask As Boolean, maskColor As Int) As ResumableSub
    Dim image As JavaObject = CreateInputImage(bmp)
    Dim width, height As Int
    width = image.RunMethod("getWidth", Null)
    height = image.RunMethod("getHeight", Null)
    Dim colores(width & height) As Int
    Dim task As JavaObject = segmenter.RunMethod("process", Array(image))
    Do While task.RunMethod("isComplete", Null).As(Boolean) = False
        Sleep(50)
    Loop
    Dim res As SegmentationResult
    res.Initialize
    If task.RunMethod("isSuccessful", Null) Then
        res.Success = True
        Dim SubjectSegmentationResult As JavaObject = task.RunMethod("getResult", Null)
        res.ForegroundBitmap = SubjectSegmentationResult.RunMethod("getForegroundBitmap", Null)
       
        If getForeGroundMask Then
        Dim ForegroundMask As JavaObject
        ForegroundMask = SubjectSegmentationResult.RunMethod("getForegroundConfidenceMask", Null)
        Dim bitmapMask As BitmapCreator
        bitmapMask.Initialize(width, height)
        For i = 0 To (width * height) - 1
            If (ForegroundMask.RunMethod("get", Null)) > 0.5 Then
                colores(i) = maskColor
            Else
                colores(i) = xui.Color_White
            End If
        Next
        Dim bmpx As JavaObject
        Dim foregroundBitmap As B4XBitmap
        bmpx = bmpx.InitializeStatic("android.graphics.Bitmap")
        Dim config As JavaObject
        config = config.InitializeStatic("android.graphics.Bitmap.Config")
        bmpx = bmpx.RunMethodJO("createBitmap", Array(colores, width, height, config.GetField("ARGB_8888")))
        foregroundBitmap = bmpx
        res.ForegroundMask = foregroundBitmap
        End If
    End If
    Return res
End Sub

The Type SegmentationResult will return the foreGroundBitmap and the ForegroundMask as well.

The added code is based on example code found on this link.

Cheers,
Walter
 
Top