B4J Question Conversion list of float values into byte array

[agb2008]

I am performing data conversion: list of float values into byte array (to store data as BLOB into SQLite database). As it was suggested earlier - I am trying to used approach similar to one used in KeyValueStore. But I am trying to avoid additional file operation that exist in KeyValueStore precedures when for similar type of process list values first saved to disk using jRandomAccessFile library and only after result converted to byte array and saved to SQLite database.
In jRandomAccessFile lib there is an option that would allow to use byte array instead of disk file:

Dim raf As RandomAccessFile
Dim bf(2000000) As Byte
raf.Initialize3(bf,True)

But I have to specify size of byte array otherwise, if I just use definition like:

Dim bf() As Byte

application failing while trying to write data to such 'randomaccessfile' object.

In my case I am using arrays (lists) of the same size: 10000 float values. I've tried to do just straight
math conversion: size of float in bytes * 10000... but that was not correct answer and resulting byte array
was too small. I've tested as well with disk file and size of that file was bigger than expected value. Could you lease advice what's the proper conversion procedure.

P.S. I as well found a way to apply additional compression of my data in binary format before storing it into SQLite DB - but still would like to get this issue solved.

 

[narek adonts]

try

Dim bf(0) as Byte

 

[agb2008]

narek adonts:

Unfortunately that did not help - still getting ArrayIndexOutOfBoundsException...

 

[stevel05]

See here: https://docs.oracle.com/javase/tutorial/java/nutsandbolts/datatypes.html

A float is represented in 32bits, so you would need a byte array 4 times the size of the float array.

 

[agb2008]

stevel05:

Initially I also thought that it would be that simple: 10000 x 32 = 320000 bit / 8 = 40000 bytes... But... In reality that's what I've got:

If I use following defiition:

Dim raf1,raf2 As RandomAccessFile

raf1.Initialize(File.DirApp,"test_uncomp.dat",False)
raf2.Initialize(File.DirApp,"test_comp.dat",False)

'case #1

raf1.WriteObject(lst,False,0)

'case #2

raf2.WriteObject(lst,True,0)

I've got following results:

Case #1: file test_uncomp.dat size is 170081 bytes
Case #2: file test_comp.dat size is 26075 bytes

So... How do I estimate proper Dim buffer(??????) As Bytes length required ?

P.S. As Side note with help of additional gzip compression I was able to reduce size of my data to 4459 bytes.

 

[stevel05]

I suggest that you create a simple example showing exactly how you are going about the conversion, you only need 10 or so values to see what is happening. That way anybody trying to help will know exactly where the data has come from and is going to. In a lot of cases where I have tried to create a simple example, I have found the resolution in doing so.

Otherwise I , or anyone else that want's to help will have to guess at some aspects.

You should also look at the ByteConverter library that will return an array of bytes from an array of floats. That may be all you need.

https://www.b4x.com/android/forum/threads/byteconverter-library.6787/

 

[agb2008]

stevel05:

I thought that information in my previous posts was enough - but if not:

Dim raf1,raf2 As RandomAccessFile
' code would fail if I define bf as: Dim bf() As Byte
Dim bf(2000000) As Byte
Dim i,i1 As Int
Dim lst As List

lst.Initialize

For i=1 To 10000
    lst.Add(i+0.54321)
Next

raf.Initialize3(bf,True)
raf1.Initialize(File.DirApp,"test_uncomp.dat",False)
raf2.Initialize(File.DirApp,"test_comp.dat",False)

'case #1

raf1.WriteObject(lst,False,0)

'case #2

raf2.WriteObject(lst,True,0)

'case #3
' Would fail if byte array would be small enough !!!
'or if it would be defined as Dim bf() As Byte
raf.WriteObject(lst,True,0)

Could anyone comment on this code ?

 

[stevel05]

OK, there was enough information, but it's always easier to help when a little preparation is done in advance and examples are provided rather than just a description.

Does this not achieve what you want?

    Dim Floats(10000) As Float
    For i=0 To 9999
        Floats(i) = i+0.54321
    Next
   
    Dim Bytes() As Byte = BC.FloatsToBytes(Floats)
    Log(Bytes.Length)

Using the byte converter library,

 

[agb2008]

stevel05:

As I've noted in my post I have to perform conversion of List type of object. Of cause I could convert to
List of Floats to Array of Floats and then convert to Bytes array but I do not think that it would be the best solution... :-(

Anyway I've tested ByteConverter solution as a replacement. Now I am facing different issue - I am defining array of floats,
and then assigning values in a loop:

For i=0 To 9999
      Floats(i) = i+1.54321
Next

so I've exptected Floats(0) to be equal: 1.54321 , but in reality Floats(0) reported as: 1.5432100296020508
quite similar value, but not exactly the same. As a result initial buffer size indeed equal 40000, but gzip compression
results give me just 19709 bytes (comparing with 4459 bytes in previous case) - that is due to these differences in
source data definition...