B4A Library [class] ML Kit Subject Segmentation

Subject Segmentation | ML Kit | Google for Developers

developers.google.com

Input: image
Output: image with main subject

See attached project.
Don't miss the #AdditionalJars in the main module and the manifest editor snippets.

Usage:

Private Sub Button1_Click
    Dim cc As ContentChooser
    cc.Initialize("cc")
    cc.Show("image/*", "choose image")
    Wait For cc_Result (Success As Boolean, Dir As String, FileName As String)
    If Success Then
        'I'm resizing the image here. It might be better to leave the image with the original resolution.
        Dim bmp As B4XBitmap = xui.LoadBitmapResize(Dir, FileName, B4XImageView1.mBase.Width, B4XImageView1.mBase.Height, True)
        B4XImageView1.Bitmap = bmp
        B4XImageView2.Clear
        Wait For (segmenter.Process(bmp)) Complete (Result As SegmentationResult)
        If Result.Success Then
            B4XImageView2.Bitmap = Result.ForegroundBitmap
        End If
    End If
End Sub

B4A Sdk Manager: search for mlkit and install all items. Search for odml and install as well. Don't install anything else.

 

[LucaMs]

Great, thank you, Erel.


[Just a couple of days ago I found this site. It has various tools. It removes the background in an instant and understands what you want to keep. Incredible!]

 

[personalapps]

Subject Segmentation | ML Kit | Google for Developers

developers.google.com

Input: image
Output: image with main subject

See attached project.
Don't miss the #AdditionalJars in the main module and the manifest editor snippets.

Usage:

Private Sub Button1_Click
    Dim cc As ContentChooser
    cc.Initialize("cc")
    cc.Show("image/*", "choose image")
    Wait For cc_Result (Success As Boolean, Dir As String, FileName As String)
    If Success Then
        'I'm resizing the image here. It might be better to leave the image with the original resolution.
        Dim bmp As B4XBitmap = xui.LoadBitmapResize(Dir, FileName, B4XImageView1.mBase.Width, B4XImageView1.mBase.Height, True)
        B4XImageView1.Bitmap = bmp
        B4XImageView2.Clear
        Wait For (segmenter.Process(bmp)) Complete (Result As SegmentationResult)
        If Result.Success Then
            B4XImageView2.Bitmap = Result.ForegroundBitmap
        End If
    End If
End Sub

While compiling I am Getting

Organizing libraries. Error
Maven artifact not found: com.google.firebase/firebase-encoders
Source: com.google.android.gmslay-services-mlkit-subject-segmentation

what should I download from b4a sdk manager to resolve the above error? b4a version is 12.80 and B4A Sdk Manager version: 4.10

 

[personalapps]

Update to error message :
After installing:

com.google.firebase:firebase-encoders version 17.0.0

I get this error:

Dex merge Error
Error in c:\android\tools\..\extras\b4a_remote\com\google\firebase\firebase-encoders-json\16.1.0\unpacked-firebase-encoders-json-16.1.0\jars\classes.zip:classes.dex:
Type com.google.firebase.encoders.DataEncoder is defined multiple times: c:\android\tools\..\extras\b4a_remote\com\google\firebase\firebase-encoders-json\16.1.0\unpacked-firebase-encoders-json-16.1.0\jars\classes.zip:classes.dex, c:\android\tools\..\extras\b4a_remote\com\google\firebase\firebase-encoders\17.0.0\firebase-encoders-17.0.0.zip:classes.dex
Compilation failed

Do I have to remove anything already installed?

 

[Erel]

I think that you are using an old version of Android SDK. Delete it and download a new one: https://www.b4x.com/b4a.html

You should then open B4A Sdk Manager, search for mlkit and install the items. Don't install anything else.

 

[personalapps]

I think that you are using an old version of Android SDK. Delete it and download a new one: https://www.b4x.com/b4a.html

You should then open B4A Sdk Manager, search for mlkit and install the items. Don't install anything else.

Tried it and it worked Thanks. By the way, was not able to upvote and mark your reply as solution.

 

[Johan Hormaza]

I have problem:

 

[personalapps]

I have problem:

View attachment 153239

Search for 'odml' and install it.

 

[Johan Hormaza]

Search for 'odml' and install it.

Thank!

 

[Magma]

A silly question all these "goodies" - is it possible to work at b4j ?

 

[Erel]

silly question all these "goodies" - is it possible to work at b4j ?

No. MLKit is an Android + iOS solution.

Some of the services will be supported by B4i.

 

[Johan Hormaza]

Some of the services will be supported by B4i.

Magnificent!

 

[personalapps]

View attachment 153984

I don't know what's going on, but it's not working. Huawei's Hung Meng

Probably because it depends on Google Play Services and Huawei doesn't have access to it.

 

[walterf25]

View attachment 153169

Subject Segmentation | ML Kit | Google for Developers

developers.google.com

Input: image
Output: image with main subject

See attached project.
Don't miss the #AdditionalJars in the main module and the manifest editor snippets.

Usage:

Private Sub Button1_Click
    Dim cc As ContentChooser
    cc.Initialize("cc")
    cc.Show("image/*", "choose image")
    Wait For cc_Result (Success As Boolean, Dir As String, FileName As String)
    If Success Then
        'I'm resizing the image here. It might be better to leave the image with the original resolution.
        Dim bmp As B4XBitmap = xui.LoadBitmapResize(Dir, FileName, B4XImageView1.mBase.Width, B4XImageView1.mBase.Height, True)
        B4XImageView1.Bitmap = bmp
        B4XImageView2.Clear
        Wait For (segmenter.Process(bmp)) Complete (Result As SegmentationResult)
        If Result.Success Then
            B4XImageView2.Bitmap = Result.ForegroundBitmap
        End If
    End If
End Sub

B4A Sdk Manager: search for mlkit and install all items. Search for odml and install as well. Don't install anything else.

Modified this class a little to serve my purpose, thanks @Erel for this useful class, I had the need to retrieve the ForegroundMask so I modified the class a little bit, here are the changes made.
[CODE lang="b4x" title="Segmentation" highlight="3, 11, 24, 25, 26, 27, 28, 40, 41, 42, 43, 44, 45, 46, 47, 48, 49, 50, 51, 52, 53, 54, 55, 56, 57, 58, 59, 60"]'v1.01
Sub Class_Globals
Type SegmentationResult (Success As Boolean, ForegroundBitmap As B4XBitmap, ForegroundMask As B4XBitmap)
Private segmenter As JavaObject
Private xui As XUI
End Sub

Public Sub Initialize
Dim options As JavaObject
options.InitializeNewInstance("com.google.mlkit.vision.segmentation.subject.SubjectSegmenterOptions.Builder", Null)
options = options.RunMethodJO("enableForegroundBitmap", Null).RunMethodJO("enableForegroundConfidenceMask", Null).RunMethod("build", Null)
Dim SubjectSegmentation As JavaObject
SubjectSegmentation.InitializeStatic("com.google.mlkit.vision.segmentation.subject.SubjectSegmentation")
segmenter = SubjectSegmentation.RunMethod("getClient", Array(options))
End Sub

Private Sub CreateInputImage(bmp As B4XBitmap) As Object
Dim InputImage As JavaObject
InputImage.InitializeStatic("com.google.mlkit.vision.common.InputImage")
Return InputImage.RunMethod("fromBitmap", Array(bmp, 0))
End Sub

Public Sub Process (bmp As B4XBitmap, getForeGroundMask As Boolean, maskColor As Int) As ResumableSub
Dim image As JavaObject = CreateInputImage(bmp)
Dim width, height As Int
width = image.RunMethod("getWidth", Null)
height = image.RunMethod("getHeight", Null)
Dim colores(width & height) As Int
Dim task As JavaObject = segmenter.RunMethod("process", Array(image))
Do While task.RunMethod("isComplete", Null).As(Boolean) = False
Sleep(50)
Loop
Dim res As SegmentationResult
res.Initialize
If task.RunMethod("isSuccessful", Null) Then
res.Success = True
Dim SubjectSegmentationResult As JavaObject = task.RunMethod("getResult", Null)
res.ForegroundBitmap = SubjectSegmentationResult.RunMethod("getForegroundBitmap", Null)

If getForeGroundMask Then
Dim ForegroundMask As JavaObject
ForegroundMask = SubjectSegmentationResult.RunMethod("getForegroundConfidenceMask", Null)
Dim bitmapMask As BitmapCreator
bitmapMask.Initialize(width, height)
For i = 0 To (width * height) - 1
If (ForegroundMask.RunMethod("get", Null)) > 0.5 Then
colores(i) = maskColor
Else
colores(i) = xui.Color_White
End If
Next
Dim bmpx As JavaObject
Dim foregroundBitmap As B4XBitmap
bmpx = bmpx.InitializeStatic("android.graphics.Bitmap")
Dim config As JavaObject
config = config.InitializeStatic("android.graphics.Bitmap.Config")
bmpx = bmpx.RunMethodJO("createBitmap", Array(colores, width, height, config.GetField("ARGB_8888")))
foregroundBitmap = bmpx
res.ForegroundMask = foregroundBitmap
End If
End If
Return res
End Sub[/CODE]

The Type SegmentationResult will return the foreGroundBitmap and the ForegroundMask as well.

The added code is based on example code found on this link.

Cheers,
Walter

 

[jkhazraji]

Modified this class a little to serve my purpose, thanks @Erel for this useful class, I had the need to retrieve the ForegroundMask so I modified the class a little bit, here are the changes made.
[CODE lang="b4x" title="Segmentation" highlight="3, 11, 24, 25, 26, 27, 28, 40, 41, 42, 43, 44, 45, 46, 47, 48, 49, 50, 51, 52, 53, 54, 55, 56, 57, 58, 59, 60"]'v1.01
Sub Class_Globals
Type SegmentationResult (Success As Boolean, ForegroundBitmap As B4XBitmap, ForegroundMask As B4XBitmap)
Private segmenter As JavaObject
Private xui As XUI
End Sub

Public Sub Initialize
Dim options As JavaObject
options.InitializeNewInstance("com.google.mlkit.vision.segmentation.subject.SubjectSegmenterOptions.Builder", Null)
options = options.RunMethodJO("enableForegroundBitmap", Null).RunMethodJO("enableForegroundConfidenceMask", Null).RunMethod("build", Null)
Dim SubjectSegmentation As JavaObject
SubjectSegmentation.InitializeStatic("com.google.mlkit.vision.segmentation.subject.SubjectSegmentation")
segmenter = SubjectSegmentation.RunMethod("getClient", Array(options))
End Sub

Private Sub CreateInputImage(bmp As B4XBitmap) As Object
Dim InputImage As JavaObject
InputImage.InitializeStatic("com.google.mlkit.vision.common.InputImage")
Return InputImage.RunMethod("fromBitmap", Array(bmp, 0))
End Sub

Public Sub Process (bmp As B4XBitmap, getForeGroundMask As Boolean, maskColor As Int) As ResumableSub
Dim image As JavaObject = CreateInputImage(bmp)
Dim width, height As Int
width = image.RunMethod("getWidth", Null)
height = image.RunMethod("getHeight", Null)
Dim colores(width & height) As Int
Dim task As JavaObject = segmenter.RunMethod("process", Array(image))
Do While task.RunMethod("isComplete", Null).As(Boolean) = False
Sleep(50)
Loop
Dim res As SegmentationResult
res.Initialize
If task.RunMethod("isSuccessful", Null) Then
res.Success = True
Dim SubjectSegmentationResult As JavaObject = task.RunMethod("getResult", Null)
res.ForegroundBitmap = SubjectSegmentationResult.RunMethod("getForegroundBitmap", Null)

If getForeGroundMask Then
Dim ForegroundMask As JavaObject
ForegroundMask = SubjectSegmentationResult.RunMethod("getForegroundConfidenceMask", Null)
Dim bitmapMask As BitmapCreator
bitmapMask.Initialize(width, height)
For i = 0 To (width * height) - 1
If (ForegroundMask.RunMethod("get", Null)) > 0.5 Then
colores(i) = maskColor
Else
colores(i) = xui.Color_White
End If
Next
Dim bmpx As JavaObject
Dim foregroundBitmap As B4XBitmap
bmpx = bmpx.InitializeStatic("android.graphics.Bitmap")
Dim config As JavaObject
config = config.InitializeStatic("android.graphics.Bitmap.Config")
bmpx = bmpx.RunMethodJO("createBitmap", Array(colores, width, height, config.GetField("ARGB_8888")))
foregroundBitmap = bmpx
res.ForegroundMask = foregroundBitmap
End If
End If
Return res
End Sub[/CODE]

The Type SegmentationResult will return the foreGroundBitmap and the ForegroundMask as well.

The added code is based on example code found on this link.

Cheers,
Walter

After replacing the code you recommended and using getForeGroundMask as True and maskColor as colors.White, I did not get it working. Can you supply an example please.