B4J Question Canvas.DrawImage2 fails with large pixel images

[max123]

Hi all,

in one of my libraries I wrote I've this line:

cvs.DrawImage2(img, 0, 0, W, H, 0, 0, W, H)

... where W and H are 200 pixels.

The image I process is a big horizontal stripped image of 20000x200 pixels where I just use a portion 200x200 on it, in the case of this line I've posted it just draw the first frame because the sourceX is zero, to show all frames I just increase the StartX every 200 pixels, so 200 show a second frame, 400 the third frame and so....

This always worked on old system with Win XP but still return a null pointer exception on a new system with Win 11.

In new system I compile with Java11, on old system I compile with Java8, maybe is this the problem ?

I even add #VirtualMachineArgs: -Xms256m -Xmx1024m so is not a memory Heap problem, the full image is 570Kb in jpg format or 2.8Mb in png format.

I tried to reproduce it with a small code and have the same issue, with smaller images (in pixels) works, with larger images (in pixels) does not works.

Is not a problem of image size in memory consume, it works with a 5.5Mb jpg file of 4250x3800 pixels, probably it do not like images with big width or height dimensions in this case 20000.

I cannot post the image because a upload limit and the smaller is 570Kb.

Please any advice to get it working is welcome......
Thanks!!!

Here the simple code I've used to reproduce it:

Sub Process_Globals
    Private fx As JFX
    Private MainForm As Form
    Private xui As XUI
    Private Button1 As B4XView
End Sub

Sub AppStart (Form1 As Form, Args() As String)
    MainForm = Form1
    MainForm.RootPane.LoadLayout("Layout1")
    MainForm.Show
 
    Dim cvs As Canvas
    Dim img As Image
    img.Initialize(File.DirAssets, "Speaker1.jpg")
 
    Dim W As Int = 200
    Dim H As Int = 200

    cvs.Initialize("")
    Form1.RootPane.AddNode(cvs, 0, 0, 500, 500)
 
    cvs.DrawImage2(img, 0, 0, W, H, 0, 0, W, H)
End Sub

..... and here is the log with 20000x200 image:

java.lang.NullPointerException
at javafx.graphics/com.sun.javafx.sg.prism.NGCanvas$RenderBuf.validate(NGCanvas.java:213)
at javafx.graphics/com.sun.javafx.sg.prism.NGCanvas.initCanvas(NGCanvas.java:641)
at javafx.graphics/com.sun.javafx.sg.prism.NGCanvas.renderContent(NGCanvas.java:604)
at javafx.graphics/com.sun.javafx.sg.prism.NGNode.doRender(NGNode.java:2072)
at javafx.graphics/com.sun.javafx.sg.prism.NGNode.render(NGNode.java:1964)
at javafx.graphics/com.sun.javafx.sg.prism.NGGroup.renderContent(NGGroup.java:270)
at javafx.graphics/com.sun.javafx.sg.prism.NGRegion.renderContent(NGRegion.java:578)
at javafx.graphics/com.sun.javafx.sg.prism.NGNode.doRender(NGNode.java:2072)
at javafx.graphics/com.sun.javafx.sg.prism.NGNode.render(NGNode.java:1964)
at javafx.graphics/com.sun.javafx.tk.quantum.ViewPainter.doPaint(ViewPainter.java:479)
at javafx.graphics/com.sun.javafx.tk.quantum.ViewPainter.paintImpl(ViewPainter.java:328)
at javafx.graphics/com.sun.javafx.tk.quantum.PresentingPainter.run(PresentingPainter.java:91)
at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:515)
at java.base/java.util.concurrent.FutureTask.runAndReset(FutureTask.java:305)
at javafx.graphics/com.sun.javafx.tk.RenderJob.run(RenderJob.java:58)
at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128)
at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628)
at javafx.graphics/com.sun.javafx.tk.quantum.QuantumRenderer$PipelineRunnable.run(QuantumRenderer.java:125)
at java.base/java.lang.Thread.run(Thread.java:834)
java.lang.NullPointerException
at javafx.graphics/com.sun.javafx.sg.prism.NGCanvas$RenderBuf.validate(NGCanvas.java:213)
at javafx.graphics/com.sun.javafx.sg.prism.NGCanvas.initCanvas(NGCanvas.java:641)
at javafx.graphics/com.sun.javafx.sg.prism.NGCanvas.renderContent(NGCanvas.java:604)
at javafx.graphics/com.sun.javafx.sg.prism.NGNode.doRender(NGNode.java:2072)
at javafx.graphics/com.sun.javafx.sg.prism.NGNode.render(NGNode.java:1964)
at javafx.graphics/com.sun.javafx.sg.prism.NGGroup.renderContent(NGGroup.java:270)
at javafx.graphics/com.sun.javafx.sg.prism.NGRegion.renderContent(NGRegion.java:578)
at javafx.graphics/com.sun.javafx.sg.prism.NGNode.doRender(NGNode.java:2072)
at javafx.graphics/com.sun.javafx.sg.prism.NGNode.render(NGNode.java:1964)
at javafx.graphics/com.sun.javafx.tk.quantum.ViewPainter.doPaint(ViewPainter.java:479)
at javafx.graphics/com.sun.javafx.tk.quantum.ViewPainter.paintImpl(ViewPainter.java:328)
at javafx.graphics/com.sun.javafx.tk.quantum.PresentingPainter.run(PresentingPainter.java:91)
at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:515)
at java.base/java.util.concurrent.FutureTask.runAndReset(FutureTask.java:305)
at javafx.graphics/com.sun.javafx.tk.RenderJob.run(RenderJob.java:58)
at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128)
at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628)
at javafx.graphics/com.sun.javafx.tk.quantum.QuantumRenderer$PipelineRunnable.run(QuantumRenderer.java:125)
at java.base/java.lang.Thread.run(Thread.java:834)

 

[max123]

Searching on the web seem related to this:
https://stackoverflow.com/questions/23467797/javafx-nullpointerexception-when-rendering-big-image
so probalbly the max size is 8k.

Adding this worked but still rendering software and is not good for my library that is a graphic controller library:

#VirtualMachineArgs: "-Dprism.order=sw"

Please any advice to mantain HW rendering while use big images (in pixels), this works on old system.

Note that I do not need to render a full big image, just a portion of it, 200x200 pixels.

Because I've a lots of graphic control libraries that I've to release and works the same way, this is a big problem if I do not solve it.

 

[Erel]

Is not a problem of image size in memory consume, it works with a 5.5Mb jpg file of 4250x3800

Not exactly related but there is no relation between the file size and the memory required to load the image. The memory required is about 4 * width * height.

I don't have any good workaround to suggest other than to use small image files or disable hardware acceleration. My guess is that you will not see any difference with sw acceleration.

 

[max123]

Thanks Erel,

my library should render as fast possible, this is a speaker I will move in realtime with music.
I reduced the image from 400x400 to 200x200 to be as fast possible.

I will try your advices, so with software render just to test, then maybe I put 100 frames just not one horozzontally row, but maybe on 25 any row on 4 rows, or more... so the final image size is smaller.

The problem here is that the program (KnobMan) that render this stripped image from consecutive frames only renders horizzontally or vertically or single frames every file.

I think I will use a graphic tool to manage it row-column.

 

[kimstudio]

Another way is to draw knob by code, draw the background and indicator/pointer by code in real-time.

 

[max123]

@kimstudio I do not know you, this is a graphical control, based on consecutive image frames.

 

[kimstudio]

Hi max123, maybe I took for granted. I used to use knobman to make knob controls, it generates a long image with each knob position of knob as one part of image, so for 100 angles it has 100 image parts combined to be a long image.

I mean if it is for something like a knob control, then it can be drawn by code, instead of loading the long image and copy each part of it. But maybe it is not your case.

 

[max123]

Ok, thanks kimstudio. Yes now I know it.
Yes this is one control of a long series of graphical controls I made, like sliders, leds, swithes, speakers, button pads, Vmeters and more, I attach a screenshot so you, Erel and other users can see on what I working on. Here there are even speakers.
Instead of using CSS I will use images.

 

[max123]

Ok, I want create from my original 20000x200 image (so one line 100 columns) a 2000x2000 image (so 10 lines, 10 columns) but manually with a photoeditor I still have a problems, even I need to process a lot of images.

Every image (frame) is 200x200 pixels.

At this point seem a good choice make a B4J program where I select an input image, put original row and line counts and width and height for a frame, and final rows, so it calculate the new image and save to a new file.... maybe using BitmapCreator.

This way I automatize the process and without human errors, eg. wrong crop and position of cutted images.

Can someone help me to write this ?

 

[kimstudio]

Hi max123, that's cool! looking forward to your controls and demo app.

Some other words: image based controls are good for B4J. However, looking at new fasion it seems flat-controls drawn by code are popular in these days, because there are many different screen size and DPIs so image based ones will cause blur after rescaling. Actually I like all those realistic image based controls from all those great music making softwares.

 

[kimstudio]

At this point seem a good choice make a B4J program

I can make one for you.

 

[max123]

Thanks kimstudio, I still study BitmapCreator, I used it in past to make a video encoder by reading frame by frame (every frame one image file) and save to file, but now I do not remember how the API works, used 2 years ago....

I'm still sure is not difficult using BitmapCreator, but I need to study a bit, need to disable the blur effect, just crop images and save to a new file. New file extension should match the input file extention, by default I use JPG and PNG, maybe put 'New' in the output file name is a good option, just to no overwrite original file in the same directory.

 

[kimstudio]

hi max123, I made one and put in share sub-forum as someone may also need this. I am not sure whether it can manage your huge images. You can change the source code to fit your needs or correct any bugs.

Link:

Split & merge image tool

To split an image into grids then merge them into a new image. Made in a fast and dirty way so just correct bugs or fit for your own needs with attached source code. I think it could be useful to manipulate game sprite sheet.

www.b4x.com

 

[max123]

Thanks kimstudio, I will try it.... ?
Sounds like this is what I will try to do it now this exact moment.
I'm here to manage with BitmapCreator, FileChooser, canvas ans some things, but for now just loaded an image.

 

[max123]

@kimstudio a BIG THANK YOU.

This is exactly what I started and I wanted to do.....
I've tried it with some files and seem to works, it is very simple and fast too.

It works but I found a very strange issue, the 20000x200 input file is 541Kb, the output file that is 2000x2000 and has the same content but disposed in a different way is 2.48Mb, here something seem wrong, I still search to know it, sould result the same file size.

I attached a screenshot of result of another file, I cannot post speaker file result because is 2.48Mb.
Seem something is wrong in the code, the next image file (that is for another library controllerer) is originally sticked vertically, original size is 180x11000, the resulting image is 1800x1100 (on 10 lines, 10 columns) that is good, but the input file is 904Kb and the output file is 305Kb. This is very strange.... You do not touched color resolution.

I attacked some sticked images to test with...

 

[OliverA]

It works but I found a very strange issue, the 20000x200 input file is 541Kb, the output file that is 2000x2000 and has the same content but disposed in a different way is 2.48Mb, here something seem wrong, I still search to know it, sould result the same file size.

If I look at post #1 here, your input image file is a JPG file. The tool that @kimstudio created saves the output as a PNG file at 100% quality. That will make the output file size larger. Even if saved back as JPG, if using 100% quality, there is a good chance that the file will be larger than the original source image file. If your goal is to save as much space on the image file that you use in B4J, then I would suggest the following image processing flow:

1) Create your original master sheet and save it in PNG.
2) Use @kimstudio's application to "re-arrange" the image file. Since the tool saves the resulting image as PNG with 100% quality, no image quality loss should occur.
3) Use another tool to save the re-arranged image file in JPG format at a size/quality ratio that you can live with.

In this pipeline, you keep image quality until the last step, otherwise, each step will degrade image quality, and the output, quality-wise, is not as expected.

 

[max123]

Hi @OliverA, thanks

yes you are absolutely right I wanted to try it just to know the difference in file size and saving an input JPG to out JPG increase the file from 541Kb to 1.38Mb, tested just now.

here my last code from kimstudio code:

Private Sub Button2_Click
 
    Dim wg As Int = TextField3.Text
    Dim hg As Int = TextField4.Text
    Dim nx As Int = TextField5.Text
 
    Dim w2 As Int = nx * wg
    Dim h2 As Int = ww * hh / w2
    Dim ny As Int = h2 / hg
 
    Dim sf As Object = xui.Msgbox2Async($"Frame width: ${wg} Pixels${CRLF}Frame height: ${hg} Pixels${CRLF}Rows: ${ny}${CRLF}Cols: ${ny}${CRLF}Final image width: ${w2} Pixels${CRLF}Final image height: ${h2} Pixels${CRLF & CRLF}Do you want continue with these parameters ???${CRLF & CRLF}"$, "Processing image ...", "OK", "", "NO", Null)
 
    Wait For (sf) Msgbox_Result (Result As Int)
 
    If Result = xui.DialogResponse_Positive Then
 
        Dim T1 As Long = DateTime.Now

        bc1.Initialize(ww, hh)
        bc1.CopyPixelsFromBitmap(img)
        bc2.Initialize(w2, h2)
 
        Dim rr As B4XRect
        rr.Initialize(0, 0, wg, hg)
 
        Dim dx As Int = 0
        Dim dy As Int = 0
 
        frame = 0
        For y = 0 To ny-1
            For x = 0 To nx-1
                frame = frame + 1
                Log("WRITING FRAME " & frame)
                bc2.DrawBitmapCreator(bc1, rr, dx, dy, False)
                dx = dx + wg
                rr.Left = rr.Left + wg  ' Increment rect left
                rr.Right = rr.Right + wg ' Increment rect right
                If rr.Left >= ww Then
                    rr.Left = 0
                    rr.Right = wg
                    rr.Top = rr.Top + hg  ' Increment rect top
                    rr.Bottom = rr.Bottom + hg  ' Increment rect bottom
                End If
            Next
            dx = 0
            dy = dy + hg
        Next
 
        Log("BUFFER LENGTH: " & bc2.Buffer.Length)
 
        Dim fDir As String = File.GetFileParent(path)
        Dim fName As String = File.GetName(path)
 
        Dim Out As OutputStream = File.OpenOutput(fDir, "New_" & fName, False)
 
        If fName.ToLowerCase.EndsWith("png") Then
            Log("SAVE AS PNG FILE")
            bc2.Bitmap.WriteToStream(Out, 100, "PNG")
        else If fName.ToLowerCase.EndsWith("jpg") Then
            Log("SAVE AS JPG FILE")
            bc2.Bitmap.WriteToStream(Out, 100, "JPEG")
'        else If fName.ToLowerCase.EndsWith("bmp") Then
'            Log("SAVE AS BMP FILE")
'            bc2.Bitmap.WriteToStream(Out, 100, "BMP")
        End If
 
        Out.Close
        xui.MsgboxAsync("SAVED TO:" & CRLF & File.Combine(fDir, "New_" & fName), "Process Done!  Required " & (DateTime.Now-T1) & " ms")
    End If
End Sub

Oliver, I will follow your advices and manage the file in PNG format, just at end I use a photoeditor or FFMPEG to open it and save as JPG by setting the compression ratio, by default 75%.

The best way I think should be use bitmap format (or png if alpha channel is required) and then as final stage convert to jpg to decrease file size or leave png as unchanged and always manage as png (only for transparent background).

To @kimstudio, your code works great, there is only the output file size that increases, this is not related to your code, but probably in the way B4X manage data, just please correct your code with the final IF that check if file has to be saved as PNG or JPG, otherwise while manage JPG files it save as .jpg but with PNG data, here I do not know what really happen, but the file size will be increased as it is treated as PNG file but have .jpg extension.

At this point I've some questions:
- How to manage real bitmap files (BMP) ?
- I need random access file, make the header and write data ?
- Is that possible in a simple way like PNG and JPG files ?

Attached my last project version of kimstudio tool, where I changed just some small things. Now a messagebox will show parameters data and ask to user if want to continue or abort. Now show the elapsed time required to do the conversion.