Need B4A wrapper for Eink SDK

[agraham]

getGrayImage42 and getGrayImage are implemented in Java in the MainActivity class of the demo. Driving the display looks doable in B4A. The only non-standard library I can see is this

import android.app.IT8951Manager;

Which seems to be in mysdk.jar

 

[JohnC]

My partner is shipping the whole display to me, so I should have it in 1-2 weeks to try your code

 

[JohnC]

OK, so I received the display and it is a custom Chinese version of android. I was able to switch it to English, but some items I still need to use google lens to read it.

So, does anyone have any ideas how I can create B4A code to send an image to the eink display?

Remember, this is a work project, so I have no problem paying for help as long as it works and is a reasonable cost

 

[JordiCP]

If you are using the same Android motherboard, it should be quite straightforward as suggested by @OliverA and @agraham, since the service is built-in. If for whatever reason you'd like to use another android board, the approach would be to implement a driver according to the Datasheet.

With the first option, it "should" be quite simple with B4A: You need to add mysdk.jar as AdditionalJar, since it contains the service class methods (they are empty implementations, so seems to be that the role is just to define the service's functions interface.)
Regarding the OpenCV part in the main java program, it just converts to gray, scales and gets the resulting pixel buffer, so OpenCV for B4A can be used, or you can simply avoid it and implement a B4A method that does so (getPixels + a bit of math).

 

[agraham]

they are empty implementations, so seems to be that the role is just to define the service's functions interface.

Correct, they are there just for compilation on the assumption that the class exists on the device. You do NOT have to include mysdk.jar in your apk as an additional jar. It should be quite simple judging from the demo program you supplied.

 

[OliverA]

This may get you started. The additional libraries needed and their locations are in the comments of the source. No math. No OpenCV (not knocking OpenCV). It's close to a straight rip as the source project, except I'm using combo BitmapCreator/BitmapCreator effects to create gray images and the byte array of those images and ByteConverter to merge the byte array of two images.
Note: You'll have to add the test1.png through test4.png files to the Files directory of the test app. They are located in the resource folder of the example app of the display.

 

[JohnC]

Ok, when I powered on the display and connected an external Monitor to the HDMI port, it displayed a generic android homescreen.

I then noticed there was an app on it called "Opencvtest", which seems to be the sample app in the demo.

When I run it, it displays four buttons "Picture_1", 2,3,4", however if I click on any of them I get dialog that says "Prompt - no such device".

This indicates to me that not even their demo app can see the eink controller. So first thing is to get their demo working because I would be chasing a shadow if I tried creating and debugging an app for a display that is not even working.

So, I reached out to them and here is there response:

Dear Mr John,

As i had helping you double check with our software supplier, it is related to the operation problem, below is the answer for how to find the device in the demo app:

1. Please check the USB whether already detect the E-ink device, whether ls /dev/eink* is already see the device node

If not, please check the connection of the hardware and the related cables, reboot the device for check the /dev/eink* again, if not having this device file, you can not start debugging

2. Try to open the device again in onClick , add OpenDevice()

If still not so understanding, you can see the below image which is showing some command for it.

My next thought is to figure out a way to get a list of devices and see if /dev/eink* is listed, and if not, figure out how to get it working, then rerun the demo app.

So, I tried downloading a few USB diag/device info apps from the play store, but they seems like they are just listing the USB ports themselves, not the devices connected to them.

Anyone have an idea how I can get an enumerated list of devices to see if /dev/eink* is on it?

 

[JordiCP]

What happens when you remove the usb cable from both sides and plug it again? Is there any kind of blinking in the USB interface board green led? (If not at that moment, after rebooting your Android motherboard)
Also worth checking what happens if you connect the USB cable to USBHost 2 or USB Host3 ports on the Android board.

 

[OliverA]

This is an updated version. I've moved opening and closing the device to their own respective buttons. I've also added a button that should list the USB devices available. Take all the code in this one with a grain of salt, since I've not even installed it on a device (non at hand) to even test if it runs. And don't forget to put the images in the Files folder.

 

[JohnC]

OK,

(Oliver, I did not see your first app version, sorry for that)

But I did compile and install the second version and here is what I got:

When I click USB devices I get:

(sorry for the 3rd party images - I didn't know that B4a can't take screen shots through B4A-Bridge)

And when I try to open I got:

So, it looks like it can't see the /dev/eink* device for some reason.

 

[JohnC]

What happens when you remove the usb cable from both sides and plug it again? Is there any kind of blinking in the USB interface board green led? (If not at that moment, after rebooting your Android motherboard)
Also worth checking what happens if you connect the USB cable to USBHost 2 or USB Host3 ports on the Android board.

OK, found something new...

1) When I went to move the white USB cable (the one that goes to the eink controller), android rebooted. I traced it down to the power plug of the android board is bad - it is not providing ground (-) to the board, so when I unplugged the white USB cable, it powered off the main board. But when I then plug back the eink controller, that provides the ground to the android board so it boots up.

2) As you noticed, there is a green LED on the eink controller board. When I apply power to it, the green LED doesn't come on for a few seconds, so that tells me its not simply connected to the power lines - and that it is controlled by software. When it comes on, it stays on.

 

[JohnC]

Another Note - Oliver's app shows that there are three USB devices.

So, I do want to mention what I have connected to the USB ports in case it will help in some way:

First Port - Has the white usb cable that connects to eink controller

Second Port - Has an unpowered USB hub with two devices plugged in: a Memory stick (to copy files) and a 2.5ghz receiver for a combo wireless keyboard/touchpad (to navigate android).

*** And if I unplug the white cable, it still reads 3 devices.
*** And if I unplug the memory stick, then it goes down to 2 devices
*** And if I remove the usb hub and plug the receiver directly into the second port, it still says 2 devices.

 

[JohnC]

UPDATE - The Display is working now!

When I didn't notice the usb device count change when I disconnected the white usb going to the eink controller, I decided to replace the cable since I already found one bad cable (the power plug), so maybe he had a friend......and sure enough, now I see 4 devices.

I then ran the Chinese demo app and it displayed an image!

I then ran Olivers app, and it opened the eink device without an error! Then I clicked "Image 1" with my fingers crossed, but got a "Display image status: -1", so I need to see whats up with that.

But many Kudos to Oliver for creating code that mostly worked without even having the hardware to test it on

That was very strange that the USB cable was bad, especially because it did power the board, but I guess the data lines were crap.

 

[JohnC]

Ok, I narrowed it down to this sub in Oliver's code:

Public Sub getGrayImage(bmp As B4XBitmap) As Byte()
    Dim gray As B4XBitmap = effects.GreyScale(bmp)
    Dim creator As BitmapCreator
    creator.Initialize(gray.width, gray.Height)
    creator.DrawBitmap(gray, creator.TargetRect, True)
    Return creator.Buffer
End Sub

It appears the eink driver's "DisplayImageAPI" routine doesn't like the way the byte data is formatted in this sub for some reason.

 

[JordiCP]

If the input image is W*H, the resulting buffer length should also be W*H.
This doesn't happen here because of 2 reasons

Public Sub getGrayImage(bmp As B4XBitmap) As Byte()
    Dim gray As B4XBitmap = effects.GreyScale(bmp)     '<-- if you look at gray bitmap  dimensions, they are not the same as bmp, but seem divided depending on device density
    Dim creator As BitmapCreator
    creator.Initialize(gray.width, gray.Height)
    creator.DrawBitmap(gray, creator.TargetRect, True)
    Return creator.Buffer           '<-- The buffer continues to be 4 byes per pixel, when the app needs 1
End Sub

There's plenty of options.

The simplest, without libs, you can directly call Klaus's getPixels from the bitmap. You will get a RGBA buffer with 4 bytes per pixel, and get the 2nd of each 4-pack as a rough estimation, or perform a color to gray conversion ( gray = (0.3 * R) + (0.59 * G) + (0.11 * B) ) into another buffer

Another option is this

Sub GLobals
   Dim ocl as OpenCVLoader
   '...
End Sub

Public Sub getGrayImage(bmp As B4XBitmap) As Byte()
    Dim mImgProc As OCVImgproc
    Dim mUtils As OCVUtils
    Dim myMat, myGrayMat As OCVMat
    mUtils.bitmapToMat1(bmp, myMat)
    mImgProc.cvtColor1(myMat, myGrayMat, mImgProc.COLOR_RGBA2GRAY) 
    Dim buffer(myGrayMat.cols*myGrayMat.rows) As Byte
    myGrayMat.get(0,0,buffer)
    Log(myGrayMat.toString)
    Return buffer
End Sub

 

[JohnC]

I had to make one small change to the declare:

Dim ocl As OCVOpenCVLoader

And it worked!!!!

Thanks guys

 

[JohnC]

I again just wanted to thank agraham, JordiCP and OliverA for all your help.

You guys just jumped in and helped solve another headache for me.

This forum is very fortunate to have members like you.