B4J Library ABMaterial - a robust(?) camera component

JackKirk · Apr 11, 2024

INSTALLATION

Step 1

Install ABMMini on your D: drive - see:

https://www.b4x.com/android/forum/t...-absolute-beginners-update-2024-05-17.117237/

Make sure you download ABMMini240517.zip (note the 240517).

It is assumed that you end up with a folder [D:\ABMMini] containing:

Library folder
Template folder
LICENSE.TXT
README.txt

and these additions to your B4J [Additional Libraries] folder:

ABMaterial.jar
...
thumbnailator-0.4.8.jar

Step 2

Copy [D:\ABMMini\Template] to [D:\ABMMini\Template - Copy]

Step 3

Rename [D:\ABMMini\Template - Copy] as [D:\ABMMini\Cam_4_ABM]

Step 4

Delete [D:\ABMMini\Cam_4_ABM\Template.b4j]
Delete [D:\ABMMini\Cam_4_ABM\Template.b4j.meta
Delete [D:\ABMMini\Cam_4_ABM\ABMPageTemplate.bas]

Step 5

Download the zip file [Cam_4_ABM.zip] unzip it and copy the folder Cam_4_ABM into [D:\ABMMini].

[D:\ABMMini\Cam_4_ABM] should now contain:

Files folder
Objects folder
ABMCacheV3.bas
Cam_4_ABM.b4j
Cam_4_ABM.b4j.meta
Cam_4_ABM_Object.bas
Cam_4_ABM_Page.bas
LetsEncrypt.bas

and the Files folder should have files:
cameraclick.mp3
camerarotate.png

and the Objects folder should have a file:
[D:\ABMMini\Cam_4_ABM\Objects\www\.well-known\acme-challenge\dummy.txt]

Step 6

CAM_4_ABM is a B4J non-UI app that generates a photo in Base64 string form which needs to be converted to a .png.

To do this you need the B4J Image object - which is only available in B4J UI apps.

So there is a separate B4J UI app (Cam_4_ABM_Converter) that is slaved to CAM_4_ABM via the B4j jShell library.

Download the zip file [Cam_4_ABM_Converter.zip] unzip it and copy the folder Cam_4_ABM_Converter into [D:\ABMMini].

It is essential that CAM_4_ABM and Cam_4_ABM_Converter folders are in the same main folder (in this case [D:\ABMMini]) otherwise the slaving will not work.

[D:\ABMMini\Cam_4_ABM_Converter] should now contain:

Cam_4_ABM_Converter.b4j
Cam_4_ABM_Converter.b4j.meta

Step 7

Launch the B4J project [D:\ABMMini\Cam_4_ABM_Converter\Cam_4_ABM_Converter.b4j]

[Project] > [Build Standalone Package]

There should now be a file [D:\ABMMini\Cam_4_ABM_Converter\Objects\temp\build\Cam_4_ABM_Converter.exe] and a number of support files and folders.

Step 8

You almost have a complete runnable ABMaterial server with a camera component.

BUT YOU CAN NOT RUN IT YET - as already noted, browsers only allow access to cameras (i.e. mediaDevices API) when in a secure context (i.e. https://...) - see RUN WITH SELF-SIGNED SSL and/or RUN WITH CERTIFICATE AUTHORITY (LetsEncrypt) SIGNED SSL

JackKirk · Apr 11, 2024

RUN WITH SELF-SIGNED SSL ON LOCALHOST

Before running any new configuration always clear browsing data on the client device.

The following is based on:

https://eclipse.dev/jetty/documenta...rating-key-pairs-and-certificates-JDK-keytool

which describes how to make a self-signed keystore - sufficient for testing, putting server behind a https://localhost:...

On the local PC that is to run server (i.e. PC that will run CAM_4_ABM B4J project):

1. Right click Windows icon (bottom left)
2. Click [Command Prompt (Admin)]
3. In [Administrator Command Prompt] window conduct a dialog similar to:

C:\WINDOWS\system32>cd "C:\java\jdk-14.0.1\bin"

C:\java\jdk-14.0.1\bin>keytool -keystore keystoreselfsign -genkey -keyalg RSA

Enter keystore password:
Re-enter new password:
What is your first and last name?
[Unknown]: xxxx
What is the name of your organizational unit?
[Unknown]: xxxx
What is the name of your organization?
[Unknown]: xxxx
What is the name of your City Or Locality?
[Unknown]: xxxx
What is the name of your State Or Province?
[Unknown]: xxxx
What is the two-letter country code For this unit?
[Unknown]: xx
Is CN=xxxx, OU=xxxx, O=xxxx, L=xxxx, ST=xxxx, C=xx correct?
[no]: yes
Generating 2,048 bit RSA key pair and self-signed certificate (SHA256withRSA) with a validity of 90 days
for: CN=xxxx, OU=xxxx, O=xxxx, L=xxxx, ST=xxxx, C=xx

4. This will result in a file [C:\java\jdk-14.0.1\bin\keystoreselfsign, copy it to the CAM_4_ABM B4J project \Objects folder

5. Launch the B4J project [D:\ABMMini\Cam_4_ABM\Cam_4_ABM.b4j]

6. In the Main module change "yourpassword" to the password you used to generate keystoreselfsign.

7. In the Main/AppStart procedure of the Main module make sure there is a statement that reads:

CASignedKeystore = False

forcing the server to use a self signed SSL.

8. Ensure this statement in Main/AppStart:

Server.PortSSL = 51043

is not commented out - forcing the server to listen on port 51043 as https.

9. Run the B4J project in Debug mode - you should now have a running ABMaterial server with a camera component.

10. On a device (with a camera) on the same local network, launch a browser and enter the URL [https://localhost:51043/Cam_4_ABM]

11. If 10. does not work you may need to find the local IP address of the PC (something like 192.168.1.109) then use a URL like [https://192.168.1.109:51043/Cam_4_ABM]

12. Tap anywhere on the screen to take a photo then go look in [D:\Cam_4_ABM_Photos\PNG] to see the photo just taken.

JackKirk · Apr 11, 2024

RUN WITH CERTIFICATE AUTHORITY (LetsEncrypt) SIGNED SSL ON REMOTE PC

Before running any new configuration always clear browsing data on the client device.

Self-signed SSLs are fine for testing but are not adequate in a production environment - browser in use will throw up ugly "not safe" warnings.

The following assumes you are installing on a remote PC which is an AWS EC2 Windows instance (server edition) but you should fairly readily be able to translate it to other remote environments.

It is also assumed you have registered a domain (yourdomain.com) via the AWS Route 53 service that points to the AWS EC2 Windows instance.

The following is based in part on:

https://www.b4x.com/android/forum/threads/server-letsencrypt-ssl-certificates.159285/#content

which describes how to generate and maintain Certificate Authority signed SSLs via LetsEncrypt.

Step 1

Install CertBot - if you use the above URL you might get quite confused and you should also be aware of:

https://www.b4x.com/android/forum/threads/certbot-discontinuing-windows-beta-support-in-2024.160254/

At time of writing you need to get to:

https://certbot.eff.org/instructions?ws=other&os=windows

then scroll down to a URL:

https://github.com/certbot/certbot/...d/certbot-beta-installer-win_amd64_signed.exe

which will download [certbot-beta-installer-win_amd64_signed.exe]

Locate this file, launch it and follow breadcrumbs to install [C:\Program Files\Certbot\bin\certbot.exe]

Step 2

Install Git which includes OpenSSL:

https://gitforwindows.org/

This is straightforward, you download [Git-2.44.0-64-bit.exe]

Locate this file, launch it and follow breadcrumbs to install [C:\Program Files\Git\usr\bin\openssl.exe]

Step 3

When it is creating an SSL certificate, LetsEncrypt conducts a "challenge" dialog with PC on http (port 80) to confirm ownership of the domain.

To accommodate this there must be a server running on the PC that is listening on this port.

Ensure your AWS EC2 Windows instance has a security group rule http for port 80.

As you are using a server edition of Windows you can use IIS.

Step 4

Install IIS.

See:

Skip [Section 1 Configuring And Launching EC2 Instance]
Use [Section 2 Installation of web Server IIS on EC2 Instance]
Skip [Section 3 Creating webpage And hosting it on EC2 Instance]
Skip [Section 4 Deleting Instance]

Step 5

In the PC's Windows taskbar search box, search For "IIS" then click [Internet Information Services (IIS) Manager]

In left hand panel right click only entry (something like [EC2AMAZ-UJ82A3M...]) > [Add Website]
Site name: [.well-known] <<<<<<< without [ ], note the leading full stop (.) in [.well-known]
Physical path: navigate to [D:\ABMMini\Cam_4_ABM\Objects\www]
[Connect as...] > [Application user (pass-through authentication)] > [OK]
[OK]

Step 6

Following file should already exist, but if it doesn't...

Set up a test by using Notepad to create a file [D:\ABMMini\Cam_4_ABM\Objects\www\.well-known\acme-challenge\dummy.txt] containing some junk (e.g. "dummy junk")

Step 7

On another PC, test IIS works by launching a browser and using a URL of:

http://yourdomain.com/.well-known/acme-challenge/dummy.txt

which should display "dummy junk"

Step 8

Certbot must be run with administrative privileges, the easiest way to do this when running the B4J project in Debug mode is to give the B4J.exe administrative privileges:

1. Locate B4J.exe (for example: C:\Program Files\Anywhere Software\B4J\B4J.exe)
2. Right click it > [Properties] > [Compatibility] > [Run this program as an administrator] > [Apply] >[OK]
3. Do not forget to unwind this when finished.

Step 9

1. Launch the B4J project [D:\ABMMini\Cam_4_ABM\Cam_4_ABM.b4j]

2. In the Main module change "yourpassword" to the password you wish to use.

3. In the Main module change "yourdomain.com" to the name of the domain you have set up via the AWS Route 53 service that points to the AWS EC2 Windows instance.

4. In the AppStart procedure of the Main module make sure there is a statement that reads:

CASignedKeystore = True

5. Run the B4J project in Debug mode - the first time you run it it should generate the file keystore.jks in [D:\ABMMini\Cam_4_ABM\Cam_4_ABM\Objects] folder and then hang.

6. Again, run the B4J project in Debug mode - you should now have a running ABMaterial server with a camera component.

7. On a device (with a camera), launch a browser and enter the URL [https://yourdomain.com:51043/Cam_4_ABM], where yourdomain.com is as per 3.

8. Tap anywhere on the screen to take a photo then go look in [D:\Cam_4_ABM_Photos\PNG] on the AWS EC2 Windows instance to see the photo just taken.

JackKirk · Apr 11, 2024

CORRECTIONS AND UPDATES LOG

In post #2 added:

Make sure you download ABMMini240502.zip (note the 240502).

JackKirk · Apr 11, 2024

POSSIBLE ENHANCEMENTS

1. Replace Certbot - see:

https://www.b4x.com/android/forum/t...-windows-beta-support-in-2024.160254/#content

2. Lock screen orientation to portrait - tried a number of ways to do this but none worked.

3. Get Cam_4_ABM_Object/navigator.mediaDevices.enumerateDevices() working on iOS Safari.

4. Fix iOS Safari behavior - camera toggle, "Tap anywhere here to take a photo" momentarily visible before video when toggling cameras.

5. "Add to home screen" functionality - ended up doing this as a separate exercise, see:

https://www.b4x.com/android/forum/t...lly-for-android-chrome-and-ios-safari.160794/

JackKirk · Apr 11, 2024

KNOWN BUGS

1. Chrome mobile - see workaround at Cam_4_ABM_Page/Cam_4_ABM_Instance_ChromeBug - suspect it is ignoring javascript [stream.getTracks().forEach(track => track.stop());]

JackKirk · Apr 11, 2024

BROWSER TESTING TO DATE

The title says "a robust(?) camera component" - which is based on the following testing:

-Successful-
Chrome mobile 123.0.6312.99 on Pixel 3 running Android 12
Chrome mobile 123.0.6312.99 on vivo running Android 10
Safari mobile on iPhone 7 running iOS 15.8.2
Safari mobile on iPhone 12 running iOS 17.3.1
Desktop Chrome 123.0.6312.59 on Windows 10 build 19045.4170
Desktop Edge 123.0.2420.81 on Windows 10 build 19045.4170
Desktop Firefox 124.0.2 on Windows 10 build 19045.4170

If you test it on a browser not in the above lists please tell me and I will update this list - please include device, browser and version, OS and version.

JackKirk · Apr 11, 2024

ADDITIONAL NOTES

All of this is basically wrappers around Cam_4_ABM_Object which is the essence of it.

Note this does not directly access cameras, as can be done in B4A/i apps - it is taking a snapshot of a video stream provided by the HTML <video> element which in turn is accessing the cameras..

Testing on various mobile and laptop devices always seems to result in a photo 640px x 480px or vice versa - this seems to be result of the statement in Cam_4_ABM_Object:

cam_context.drawImage(cam_video, 0, 0, cam_video.videoWidth, cam_video.videoHeight);

where videoWidth and videoHeight are read only "intrinsic" values, explained here:

https://developer.mozilla.org/en-US.../videoHeight#about_intrinsic_width_and_height

I can't say I fully understand this - maybe it has something to do with HTML <video> element actually being a video player and deep down it has fixed width and height? - when src is a stream? - any help understanding this better would be appreciated.

Adequate for selfies for some purposes (my planned use), inadequate for capturing high quality photos.

JackKirk · Apr 11, 2024

USE OF Windows Copilot

If you bury yourself in the Cam_4_ABM_Object module you will notice some quite heavy (well at least for me) HTML, CSS and javascript.

B4X:

'Loosely based on REFERENCE (2)
Sub Class_Globals

    Public ABM As ABMaterial
    Public ABMComp As ABMCustomComponent

    Private Video_Width As String
    Private Video_Height As String
   
    Private Parent_Page As ABMPage
    Private Parent_Instance_Name As String

    Public BorderColor As String
    Public BorderIntensity As String
    Public BorderStyle As String
    Public BorderWidth As String
    Public BorderRadius As String
    Public PaddingBottom As String
    Public PaddingLeft As String
    Public PaddingRight As String
    Public PaddingTop As String
    Public ZDepth As String
   
End Sub

Public Sub Initialize(Passed_Page As ABMPage, Passed_Instance_Name As String)
   
    Parent_Page = Passed_Page
   
    Parent_Instance_Name = Passed_Instance_Name.ToLowerCase

    Video_Width = "100%"
    Video_Height = "100%" 'or "auto"
   
    'Set property defaults
    BorderColor = ABM.COLOR_LIGHTBLUE
    BorderIntensity = ABM.INTENSITY_NORMAL
    BorderStyle = ABM.BORDER_NONE
    BorderWidth = "0px"
    BorderRadius = "0px"
    PaddingBottom = "0px"
    PaddingLeft = "0px"
    PaddingRight = "0px"
    PaddingTop = "0px"
    ZDepth = ABM.ZDEPTH_REMOVE

    ABMComp.Initialize("ABMComp", Me, Parent_Page, Parent_Instance_Name, "")
   
End Sub

Sub ABMComp_Build(Passed_page As ABMPage, Passed_Instance_Name As String) As String
   
    'Passed_page is just Parent_page
    'Passed_Instance_Name is just Parent_Instance_Name

    Private cam_div_css As String

    If PaddingTop <> "" Then cam_div_css = cam_div_css & "padding-top: " & PaddingTop & ";"
    If PaddingBottom <> "" Then cam_div_css = cam_div_css & "padding-bottom: " & PaddingBottom & ";"
    If PaddingLeft <> "" Then cam_div_css = cam_div_css & "padding-left: " & PaddingLeft & ";"
    If PaddingRight <> "" Then cam_div_css = cam_div_css & "padding-right: " & PaddingRight & ";"
    If ZDepth <> "" Then cam_div_css = cam_div_css & ZDepth & ";"
    cam_div_css = cam_div_css & "width:" & Video_Width & ";height:" & Video_Height & _
                  ";background:#ffffff;border-radius:0px;"
   
    Private video_css As String

    video_css = "display: none;border-style:" & BorderStyle & ";border-color:" & BorderColor & _
                ";border-width:" & BorderWidth & ";border-radius:" & BorderRadius & ";" & _
                "width:" & Video_Width & ";height:" & Video_Height & ";"

    'Note trick employed in Main.AppStart that allows
    '<audio id="cam_click" src="../camerabits/cameraclick.mp3"></audio>
    '<img src="../camerabits/camerarotate.png"></img>
    'to work

    Private div_str As String

    'Note: if 'autoplay muted playsinline' is not included it will not
    'work on iOS Safari
    div_str = $"
<div id="cam_div" style="${cam_div_css}">
    <video id="cam_video" autoplay muted playsinline style="${video_css}"></video>
    <canvas id="cam_canvas" width=1, height=1 style="display:none;"></canvas>
    <audio id="cam_click" src="../camerabits/cameraclick.mp3"></audio>
    <style>
        #cam_message {
            position: absolute;
            top: 10vh;
            left: 0;
            width: 100%;
            height: 100%;
            display: flex;
            align-items: center;
            justify-content: center;
            background-color: rgba(0, 0, 0, 0.0);
            color: red;
            font-size: min(calc(0rem + 5vw), calc(0rem + 5vh)); /* Adjust scaling as needed -
                                                                   1rem is base font size (which is
                                                                   typically equivalent to 16px),
                                                                   5vw means 5% of viewport width
                                                                   5vh means 5% of viewport height) */
            text-align: center;
            font-weight: bold;
            padding-top: calc(${PaddingTop} + 10vh);
            padding-right: calc(${PaddingRight} + 10vw);
            padding-bottom: calc(${PaddingBottom} + 10vh);
            padding-left: calc(${PaddingLeft} + 10vw);
        }
        #cam_overlay {
            position: absolute;
            top: 0;
            left: 0;
            width: 100%;
            height: 100%;
            align-items: center;
            justify-content: center;
            background-color: rgba(0, 0, 0, 0.0);
            color: black;
            font-size: min(calc(0rem + 8vw), calc(0rem + 8vh));
            text-align: center;
            padding-top: calc(${PaddingTop} + 10vh);
            padding-right: calc(${PaddingRight} + 10vw);
            padding-bottom: calc(${PaddingBottom} + 10vh);
            padding-left: calc(${PaddingLeft} + 10vw);
            display: none;
        }
        #cam_toggle {
            position: absolute;
            top: calc(0px + 0px/2 + max(1vh, 1vw));
            left: 50%;
            transform: translateX(-50%);
            background-color: rgba(0, 0, 0, 0.0);
            border: none;
            cursor: pointer;
            display: none;
        }
        #cam_toggle img {
            width: min(calc(0rem + 12vw), calc(0rem + 12vh));
            height: calc(width * 448/512);
        }
    </style>
    <div id="cam_message"></div>
    <div id="cam_overlay">
        Tap anywhere here to take a photo
    </div>
    <div>
        <button id="cam_toggle">
            <img src="../camerabits/camerarotate.png"></img>
        </button>
    </div>
</div>
"$

'    Log("<<<<<<<< ABMComp_Build " & div_str)
   
    Return div_str
   
End Sub

Sub ABMComp_FirstRun(Passed_page As ABMPage, Passed_Instance_Name As String)
   
    'Passed_page is just Parent_page
    'Passed_Instance_Name is just Parent_Instance_Name
   
End Sub

Sub ABMComp_Refresh(Passed_page As ABMPage, Passed_Instance_Name As String)
   
    'Passed_page is just Parent_page
    'Passed_Instance_Name is just Parent_Instance_Name

    Private script_str As String = $"
var cam_video = document.getElementById('cam_video');
var cam_overlay = document.getElementById('cam_overlay');
var cam_toggle = document.getElementById('cam_toggle');
var cam_message = document.getElementById('cam_message');
var cam_toggle_display = 'flex'
var constraints =
{
    audio: false,
    video:
    {
        //Set camera inital value
        facingMode: 'user'
        //facingMode: 'environment'
    }
}
//
//This does not work in iOS Safari = videoDevices.length always comes back as 1
//on tested iPhones when it should be more
//
//navigator.mediaDevices.enumerateDevices()
//.then(function(devices)
//{
//    var videoDevices = devices.filter(device => device.kind === 'videoinput');
//    if (videoDevices.length > 1) {
//        cam_toggle_display = 'flex';
//    } else
//    {
//        cam_toggle_display = 'none';
//   }
//}),
navigator.mediaDevices.getUserMedia(constraints)
.then(function (stream)
{
    cam_video.srcObject=stream;
    cam_video.play();
    cam_video.style.display = 'flex';
    cam_overlay.style.display = 'flex';
    cam_toggle.style.display = cam_toggle_display;
    //cam_message.textContent = 'mediaDevices API supported';
    //b4j_raiseEvent('${Passed_Instance_Name}_Ready', {'value': 'mediaDevices API supported'});
    cam_overlay.addEventListener('click', function()
    {
        var cam_click = document.getElementById('cam_click');
        cam_click.play();
        var cam_canvas = document.getElementById('cam_canvas');
        var cam_context = cam_canvas.getContext('2d');
        cam_canvas.width = cam_video.videoWidth;
        cam_canvas.height = cam_video.videoHeight;
        cam_context.drawImage(cam_video, 0, 0, cam_video.videoWidth, cam_video.videoHeight);
        var cam_dataURL = cam_canvas.toDataURL('image/jpeg');
        b4j_raiseEvent('${Parent_Instance_Name}_PictureTaken', {'value': cam_dataURL});
    });
    cam_toggle.addEventListener('click', function()
    {
        cam_toggle.style.display = 'none';
        cam_overlay.style.display = 'none';
        cam_video.style.display = 'none';
        if (constraints.video.facingMode == 'user')
        {
            constraints.video.facingMode = 'environment';
            //cam_message.textContent = 'video.facingMode environment';
            //b4j_raiseEvent('${Passed_Instance_Name}_Error', {'value': 'environment'});
        } else
        {
            constraints.video.facingMode = 'user';
            //cam_message.textContent = 'video.facingMode user';
            //b4j_raiseEvent('${Passed_Instance_Name}_Error', {'value': 'user'});
        }
        //Reinitialize camera stream with updated constraints
        if (stream)
        {
            stream.getTracks().forEach(track => track.stop());
            cam_video.pause();
            cam_video.srcObject = null;
        }
        navigator.mediaDevices.getUserMedia(constraints)
        .then(function(stream)
        {
            cam_video.srcObject = stream;
            cam_video.play();
            cam_video.style.display = 'flex';
            cam_overlay.style.display = 'flex';
            cam_toggle.style.display = cam_toggle_display;
        })
        .catch(function(error)
        {
            b4j_raiseEvent('${Passed_Instance_Name}_ChromeBug', {'value': error.message});
        })
    });
},
function(error)
{
    cam_toggle.style.display = 'none';
    cam_overlay.style.display = 'none';
    cam_video.style.display = 'none';
    cam_message.textContent = "Your device does not have a camera or your device's camera is in use by another application or your browser does not support mediaDevices API - suggest you use another device or browser.";
    //b4j_raiseEvent('${Passed_Instance_Name}_Error', {'value':"Your device does not have a camera or your device's camera is in use by another application or your browser does not support mediaDevices API - suggest you use another device or browser."});
});   
"$   

'   Log("<<<<<<<< ABMComp_Refresh " & script_str)
   
    Parent_Page.ws.Eval(script_str, Array As Object(Passed_Instance_Name))
   
End Sub

Sub ABMComp_CleanUp(Passed_page As ABMPage, Passed_Instance_Name As String)
   
    'Passed_page is just Parent_page
    'Passed_Instance_Name is just Parent_Instance_Name
   
End Sub

This was mainly done by myself with some guidance from Mashiane's original effort - mainly by supplying various bits to Copilot and asking narrow questions.

I am super impressed.

I haven't tried it but I suspect I would get similar results from Google Gemini or many of the other LLMs now available.

JackKirk · Apr 11, 2024

A TRICK I HAVE FOUND THAT MAY HAVE GENERAL USE IN ABMaterial

Have a look in Cam_4_ABM/AppStart at the point where cameraclick.mp3 and camerarotate.png get copied.

Well on further snooping around it maybe isn't such an unknown trick - looks like images are supplied via ../image folders etc.

JackKirk · Apr 11, 2024

RESERVED

B4J Library ABMaterial - a robust(?) camera component

JackKirk

Well-Known Member

Attachments

JackKirk

Well-Known Member

JackKirk

Well-Known Member

JackKirk

Well-Known Member

JackKirk

Well-Known Member

JackKirk

Well-Known Member

JackKirk

Well-Known Member

JackKirk

Well-Known Member

JackKirk

Well-Known Member

JackKirk

Well-Known Member

JackKirk

Well-Known Member

Similar Threads