Android Tutorial [B4X] CloudKVS - synchronized key / value store

christiantric

Member
Licensed User
Hi all,
I'm a novice in B4X and looking for a client-server/cloud data store solution.
Is this component still valid and supported?
I tried to start the B4JClient (attached in the first post) pointing to the demo server but is seems that doesn't respond.

Thanks in advance.
Christian
 

christiantric

Member
Licensed User
I'm testing CloudKVS to store values with image data (Byte array).
With inserts and updates everything's ok.
If I insert a new object with image I see the db size increasing accordinglly but if I delete the object the db size remains unchanged.

I delete with
B4X:
KVSUtils.Client.Put(KVSUtils.User, itemKey, Null)
Does CloudKVS implement a kind of "soft deletion"?
Isn't storing images a good idea? What's the best approach?
How could I delete data from database (local and remote)?
 

christiantric

Member
Licensed User
Integration to my previous post:
I looked into the db and noticed something weird: after deletion the values are null but the db size is high (see attached image).
How could we explain that?
upload_2019-11-22_12-9-2.png
 

Erel

Administrator
Staff member
Licensed User
Always better to start a new thread for your questions.

This is how SQLite behaves by default. You need to run this command on the server:
B4X:
sql1.ExecNonQuery("VACUUM")
I will update the server code to enable auto vacuum.
 

christiantric

Member
Licensed User
Always better to start a new thread for your questions.

This is how SQLite behaves by default. You need to run this command on the server:
B4X:
sql1.ExecNonQuery("VACUUM")
I will update the server code to enable auto vacuum.
Oh sorry, I will start a new thread the next time for my questions about CloudKVS, if any.
Thanks, I didn't know about this SQLite behaviour.
A "Vacuum" function to call from time to time would be usefull even for the client in my opinion.
 

OliverA

Expert
Licensed User
I looked into the db and noticed something weird: after deletion the values are null but the db size is high (see attached image).
How could we explain that?
This is how SQLite behaves by default.
I'm going to go one step further and say that almost all on disk databases behave that way. It is normal operating procedure. Disk access is slow and if a DB would compact (remove empty spaces) every time a record is deleted, it would kill its performance. Please note that freed up space is usually marked in such a way that a DB will reuse it. Of course there may be times when you would want to compact/de-frag/vacuum a DB and SQLite's site gives a good breakdown of these times:
  • Unless SQLite is running in "auto_vacuum=FULL" mode, when a large amount of data is deleted from the database file it leaves behind empty space, or "free" database pages. This means the database file might be larger than strictly necessary. Running VACUUM to rebuild the database reclaims this space and reduces the size of the database file.

  • Frequent inserts, updates, and deletes can cause the database file to become fragmented - where data for a single table or index is scattered around the database file. Running VACUUM ensures that each table and index is largely stored contiguously within the database file. In some cases, VACUUM may also reduce the number of partially filled pages in the database, reducing the size of the database file further.

  • When content is deleted from an SQLite database, the content is not usually erased but rather the space used to hold the content is marked as being available for reuse. This can allow deleted content to be recovered by a hacker or by forensic analysis. Running VACUUM will clean the database of all traces of deleted content, thus preventing an adversary from recovering deleted content. Using VACUUM in this way is an alternative to setting PRAGMA secure_delete=ON.

  • Normally, the database page_size and whether or not the database supports auto_vacuum must be configured before the database file is actually created. However, when not in write-ahead log mode, the page_size and/or auto_vacuum properties of an existing database may be changed by using the page_size and/or pragma auto_vacuum pragmas and then immediately VACUUMing the database. When in write-ahead log mode, only the auto_vacuum support property can be changed using VACUUM.
I will update the server code to enable auto vacuum.
Please note SQLite's doc on this feature
An alternative to using the VACUUM command to reclaim space after data has been deleted is auto-vacuum mode, enabled using the auto_vacuum pragma. When auto_vacuum is enabled for a database free pages may be reclaimed after deleting data, causing the file to shrink, without rebuilding the entire database using VACUUM. However, using auto_vacuum can lead to extra database file fragmentation. And auto_vacuum does not compact partially filled pages of the database as VACUUM does.
So I think that auto_vacuum should be an OPTION for CloudKVS, in such a way that a user can specify its usage and let the user decide if the need the benefits of auto_vacuum.
 

Erel

Administrator
Staff member
Licensed User
After further reading it looks like auto vacuum is not strictly needed. SQLite will reuse the deleted sections.
 
Top