B4A Library [class] ML Kit Subject Segmentation

LucaMs · Apr 28, 2024

Great, thank you, Erel.

[Just a couple of days ago I found this site. It has various tools. It removes the background in an instant and understands what you want to keep. Incredible!]

personalapps · Apr 29, 2024

Erel said:

Subject Segmentation | ML Kit | Google for Developers

developers.google.com

Input: image
Output: image with main subject

See attached project.
Don't miss the #AdditionalJars in the main module and the manifest editor snippets.

Usage:

B4X:

Private Sub Button1_Click
    Dim cc As ContentChooser
    cc.Initialize("cc")
    cc.Show("image/*", "choose image")
    Wait For cc_Result (Success As Boolean, Dir As String, FileName As String)
    If Success Then
        'I'm resizing the image here. It might be better to leave the image with the original resolution.
        Dim bmp As B4XBitmap = xui.LoadBitmapResize(Dir, FileName, B4XImageView1.mBase.Width, B4XImageView1.mBase.Height, True)
        B4XImageView1.Bitmap = bmp
        B4XImageView2.Clear
        Wait For (segmenter.Process(bmp)) Complete (Result As SegmentationResult)
        If Result.Success Then
            B4XImageView2.Bitmap = Result.ForegroundBitmap
        End If
    End If
End Sub

While compiling I am Getting

Organizing libraries. Error
Maven artifact not found: com.google.firebase/firebase-encoders
Source: com.google.android.gmslay-services-mlkit-subject-segmentation

what should I download from b4a sdk manager to resolve the above error? b4a version is 12.80 and B4A Sdk Manager version: 4.10

personalapps · Apr 29, 2024

Update to error message :
After installing:

com.google.firebase:firebase-encoders version 17.0.0

I get this error:

Dex merge Error
Error in c:\android\tools\..\extras\b4a_remote\com\google\firebase\firebase-encoders-json\16.1.0\unpacked-firebase-encoders-json-16.1.0\jars\classes.zip:classes.dex:
Type com.google.firebase.encoders.DataEncoder is defined multiple times: c:\android\tools\..\extras\b4a_remote\com\google\firebase\firebase-encoders-json\16.1.0\unpacked-firebase-encoders-json-16.1.0\jars\classes.zip:classes.dex, c:\android\tools\..\extras\b4a_remote\com\google\firebase\firebase-encoders\17.0.0\firebase-encoders-17.0.0.zip:classes.dex
Compilation failed

Do I have to remove anything already installed?

Erel · Apr 29, 2024

I think that you are using an old version of Android SDK. Delete it and download a new one: https://www.b4x.com/b4a.html

You should then open B4A Sdk Manager, search for mlkit and install the items. Don't install anything else.

personalapps · Apr 29, 2024

Erel said:
I think that you are using an old version of Android SDK. Delete it and download a new one: https://www.b4x.com/b4a.html

You should then open B4A Sdk Manager, search for mlkit and install the items. Don't install anything else.

Tried it and it worked

Thanks. By the way, was not able to upvote and mark your reply as solution.

Johan Hormaza · Apr 29, 2024

I have problem:

personalapps · Apr 30, 2024

Johan Hormaza said:
I have problem:

View attachment 153239

Search for 'odml' and install it.

Johan Hormaza · Apr 30, 2024

personalapps said:
Search for 'odml' and install it.

Thank!

Magma · May 1, 2024

A silly question all these "goodies" - is it possible to work at b4j ?

Erel · May 1, 2024

Magma said:
silly question all these "goodies" - is it possible to work at b4j ?

No. MLKit is an Android + iOS solution.

Some of the services will be supported by B4i.

Johan Hormaza · May 1, 2024

Erel said:
Some of the services will be supported by B4i.

Magnificent!

personalapps · May 24, 2024

lingqdy1 said:
View attachment 153984

I don't know what's going on, but it's not working. Huawei's Hung Meng

Probably because it depends on Google Play Services and Huawei doesn't have access to it.

walterf25 · Jul 31, 2024

Erel said:
View attachment 153169

Subject Segmentation | ML Kit | Google for Developers

developers.google.com

Input: image
Output: image with main subject

See attached project.
Don't miss the #AdditionalJars in the main module and the manifest editor snippets.

Usage:

B4X:

Private Sub Button1_Click Dim cc As ContentChooser cc.Initialize("cc") cc.Show("image/*", "choose image") Wait For cc_Result (Success As Boolean, Dir As String, FileName As String) If Success Then 'I'm resizing the image here. It might be better to leave the image with the original resolution. Dim bmp As B4XBitmap = xui.LoadBitmapResize(Dir, FileName, B4XImageView1.mBase.Width, B4XImageView1.mBase.Height, True) B4XImageView1.Bitmap = bmp B4XImageView2.Clear Wait For (segmenter.Process(bmp)) Complete (Result As SegmentationResult) If Result.Success Then B4XImageView2.Bitmap = Result.ForegroundBitmap End If End If End Sub

B4A Sdk Manager: search for mlkit and install all items. Search for odml and install as well. Don't install anything else.

Modified this class a little to serve my purpose, thanks @Erel for this useful class, I had the need to retrieve the ForegroundMask so I modified the class a little bit, here are the changes made.

Segmentation:

'v1.01
Sub Class_Globals
    Type SegmentationResult (Success As Boolean, ForegroundBitmap As B4XBitmap, ForegroundMask As B4XBitmap)
    Private segmenter As JavaObject
    Private xui As XUI
End Sub

Public Sub Initialize
    Dim options As JavaObject
    options.InitializeNewInstance("com.google.mlkit.vision.segmentation.subject.SubjectSegmenterOptions.Builder", Null)
    options = options.RunMethodJO("enableForegroundBitmap", Null).RunMethodJO("enableForegroundConfidenceMask", Null).RunMethod("build", Null)
    Dim SubjectSegmentation As JavaObject
    SubjectSegmentation.InitializeStatic("com.google.mlkit.vision.segmentation.subject.SubjectSegmentation")
    segmenter = SubjectSegmentation.RunMethod("getClient", Array(options))
End Sub

Private Sub CreateInputImage(bmp As B4XBitmap) As Object
    Dim InputImage As JavaObject
    InputImage.InitializeStatic("com.google.mlkit.vision.common.InputImage")
    Return InputImage.RunMethod("fromBitmap", Array(bmp, 0))
End Sub

Public Sub Process (bmp As B4XBitmap, getForeGroundMask As Boolean, maskColor As Int) As ResumableSub
    Dim image As JavaObject = CreateInputImage(bmp)
    Dim width, height As Int
    width = image.RunMethod("getWidth", Null)
    height = image.RunMethod("getHeight", Null)
    Dim colores(width & height) As Int
    Dim task As JavaObject = segmenter.RunMethod("process", Array(image))
    Do While task.RunMethod("isComplete", Null).As(Boolean) = False
        Sleep(50)
    Loop
    Dim res As SegmentationResult
    res.Initialize
    If task.RunMethod("isSuccessful", Null) Then
        res.Success = True
        Dim SubjectSegmentationResult As JavaObject = task.RunMethod("getResult", Null)
        res.ForegroundBitmap = SubjectSegmentationResult.RunMethod("getForegroundBitmap", Null)
       
        If getForeGroundMask Then
        Dim ForegroundMask As JavaObject
        ForegroundMask = SubjectSegmentationResult.RunMethod("getForegroundConfidenceMask", Null)
        Dim bitmapMask As BitmapCreator
        bitmapMask.Initialize(width, height)
        For i = 0 To (width * height) - 1
            If (ForegroundMask.RunMethod("get", Null)) > 0.5 Then
                colores(i) = maskColor
            Else
                colores(i) = xui.Color_White
            End If
        Next
        Dim bmpx As JavaObject
        Dim foregroundBitmap As B4XBitmap
        bmpx = bmpx.InitializeStatic("android.graphics.Bitmap")
        Dim config As JavaObject
        config = config.InitializeStatic("android.graphics.Bitmap.Config")
        bmpx = bmpx.RunMethodJO("createBitmap", Array(colores, width, height, config.GetField("ARGB_8888")))
        foregroundBitmap = bmpx
        res.ForegroundMask = foregroundBitmap
        End If
    End If
    Return res
End Sub

The Type SegmentationResult will return the foreGroundBitmap and the ForegroundMask as well.

The added code is based on example code found on this link.

Cheers,
Walter

jkhazraji · Jan 1, 2025

walterf25 said:

Modified this class a little to serve my purpose, thanks @Erel for this useful class, I had the need to retrieve the ForegroundMask so I modified the class a little bit, here are the changes made.

Segmentation:

'v1.01
Sub Class_Globals
    Type SegmentationResult (Success As Boolean, ForegroundBitmap As B4XBitmap, ForegroundMask As B4XBitmap)
    Private segmenter As JavaObject
    Private xui As XUI
End Sub

Public Sub Initialize
    Dim options As JavaObject
    options.InitializeNewInstance("com.google.mlkit.vision.segmentation.subject.SubjectSegmenterOptions.Builder", Null)
    options = options.RunMethodJO("enableForegroundBitmap", Null).RunMethodJO("enableForegroundConfidenceMask", Null).RunMethod("build", Null)
    Dim SubjectSegmentation As JavaObject
    SubjectSegmentation.InitializeStatic("com.google.mlkit.vision.segmentation.subject.SubjectSegmentation")
    segmenter = SubjectSegmentation.RunMethod("getClient", Array(options))
End Sub

Private Sub CreateInputImage(bmp As B4XBitmap) As Object
    Dim InputImage As JavaObject
    InputImage.InitializeStatic("com.google.mlkit.vision.common.InputImage")
    Return InputImage.RunMethod("fromBitmap", Array(bmp, 0))
End Sub

Public Sub Process (bmp As B4XBitmap, getForeGroundMask As Boolean, maskColor As Int) As ResumableSub
    Dim image As JavaObject = CreateInputImage(bmp)
    Dim width, height As Int
    width = image.RunMethod("getWidth", Null)
    height = image.RunMethod("getHeight", Null)
    Dim colores(width & height) As Int
    Dim task As JavaObject = segmenter.RunMethod("process", Array(image))
    Do While task.RunMethod("isComplete", Null).As(Boolean) = False
        Sleep(50)
    Loop
    Dim res As SegmentationResult
    res.Initialize
    If task.RunMethod("isSuccessful", Null) Then
        res.Success = True
        Dim SubjectSegmentationResult As JavaObject = task.RunMethod("getResult", Null)
        res.ForegroundBitmap = SubjectSegmentationResult.RunMethod("getForegroundBitmap", Null)
      
        If getForeGroundMask Then
        Dim ForegroundMask As JavaObject
        ForegroundMask = SubjectSegmentationResult.RunMethod("getForegroundConfidenceMask", Null)
        Dim bitmapMask As BitmapCreator
        bitmapMask.Initialize(width, height)
        For i = 0 To (width * height) - 1
            If (ForegroundMask.RunMethod("get", Null)) > 0.5 Then
                colores(i) = maskColor
            Else
                colores(i) = xui.Color_White
            End If
        Next
        Dim bmpx As JavaObject
        Dim foregroundBitmap As B4XBitmap
        bmpx = bmpx.InitializeStatic("android.graphics.Bitmap")
        Dim config As JavaObject
        config = config.InitializeStatic("android.graphics.Bitmap.Config")
        bmpx = bmpx.RunMethodJO("createBitmap", Array(colores, width, height, config.GetField("ARGB_8888")))
        foregroundBitmap = bmpx
        res.ForegroundMask = foregroundBitmap
        End If
    End If
    Return res
End Sub

The Type SegmentationResult will return the foreGroundBitmap and the ForegroundMask as well.

The added code is based on example code found on this link.

Cheers,
Walter

After replacing the code you recommended and using getForeGroundMask as True and maskColor as colors.White, I did not get it working. Can you supply an example please.

B4A Library [class] ML Kit Subject Segmentation

Subject Segmentation | ML Kit | Google for Developers

Attachments

LucaMs

Expert

personalapps

Member

Subject Segmentation | ML Kit | Google for Developers

personalapps

Member

Erel

B4X founder

personalapps

Member

Johan Hormaza

Well-Known Member

personalapps

Member

Johan Hormaza

Well-Known Member

Magma

Expert

Erel

B4X founder

Johan Hormaza

Well-Known Member

personalapps

Member

walterf25

Expert

Subject Segmentation | ML Kit | Google for Developers

jkhazraji

Active Member

Similar Threads