madru Active Member Licensed User Longtime User Mar 10, 2021 #1 Morning, we developed some kind of photo app in B4A for a painting line some time ago, ... long story short ... at the end of the line we take a photo of the painted part. (via a integrated camera running Android). This picture will be uploaded to a DB incl. a smaller thumbnail and a lot of Metadata via RDC. This process works really well BUT the amount of pictures taken per day has massively grown from ~100 to ~800 (and growing). As a result the DB is getting bigger and bigger. Handling etc is getting difficult and the DB-Admin is not really amused I have to say that the original requirement was ~50 to ~70 images per day I am thinking to change the storage process and upload the pictures (not the thumbnails) to a NAS etc via HTTP or FTP. Storage should be like: B4X: Copy YEAR / \ Month Month / \ Day ... / \ PictureID .... and I am wondering how to implement this now as the server site should manages the tree structure and return the link to the Android App. for storage in the DB..... or is this all nonsense what I have in mind? has somebody a better idea? THX
Morning, we developed some kind of photo app in B4A for a painting line some time ago, ... long story short ... at the end of the line we take a photo of the painted part. (via a integrated camera running Android). This picture will be uploaded to a DB incl. a smaller thumbnail and a lot of Metadata via RDC. This process works really well BUT the amount of pictures taken per day has massively grown from ~100 to ~800 (and growing). As a result the DB is getting bigger and bigger. Handling etc is getting difficult and the DB-Admin is not really amused I have to say that the original requirement was ~50 to ~70 images per day I am thinking to change the storage process and upload the pictures (not the thumbnails) to a NAS etc via HTTP or FTP. Storage should be like: B4X: Copy YEAR / \ Month Month / \ Day ... / \ PictureID .... and I am wondering how to implement this now as the server site should manages the tree structure and return the link to the Android App. for storage in the DB..... or is this all nonsense what I have in mind? has somebody a better idea? THX
KMatle Expert Licensed User Longtime User Mar 10, 2021 #2 I use php as a middleware because you can add some logic here (RDC just handles DB requests). It's always good to store images on the file system (as you mentioned the db get's too big and it's hard to backup). Only indexes are stored in the db. The connection between the db entry and the image file could be the filename + filedate (beneath other indexes pointing to the user or similar). Store the files in folders like you have written (year/month/day/file) and store the path in the db. Upvote 0
I use php as a middleware because you can add some logic here (RDC just handles DB requests). It's always good to store images on the file system (as you mentioned the db get's too big and it's hard to backup). Only indexes are stored in the db. The connection between the db entry and the image file could be the filename + filedate (beneath other indexes pointing to the user or similar). Store the files in folders like you have written (year/month/day/file) and store the path in the db.
Erel B4X founder Staff member Licensed User Longtime User Mar 10, 2021 #3 KMatle said: I use php as a middleware because you can add some logic here (RDC just handles DB requests) Click to expand... ?????????????? You can of course add any logic you like to jRDC2. It is a regular jServer solution. 1. Change DBCommand: Code: Copy Type DBCommand (Name As String, Parameters() As Object, Image() As Byte, GUID As String) 2. Modify jRDC2 to write the image with the GUID as its file name and also insert the GUID to the database. Upvote 0
KMatle said: I use php as a middleware because you can add some logic here (RDC just handles DB requests) Click to expand... ?????????????? You can of course add any logic you like to jRDC2. It is a regular jServer solution. 1. Change DBCommand: Code: Copy Type DBCommand (Name As String, Parameters() As Object, Image() As Byte, GUID As String) 2. Modify jRDC2 to write the image with the GUID as its file name and also insert the GUID to the database.
Erel B4X founder Staff member Licensed User Longtime User Mar 10, 2021 #4 Random GUID: https://www.b4x.com/android/forum/threads/b4x-guid.99529/#content Upvote 0
udg Expert Licensed User Longtime User Mar 10, 2021 #5 I would consider the option to add to your tree the customer's ID. This way, in the event a customer asks for its raw data (files) you could easily pack them and mail them (eventually evne delete thme to comply with privacy/copyright/anything legislation). Obviously this depend on the nature of the stored data and the overall business scheme. Upvote 0
I would consider the option to add to your tree the customer's ID. This way, in the event a customer asks for its raw data (files) you could easily pack them and mail them (eventually evne delete thme to comply with privacy/copyright/anything legislation). Obviously this depend on the nature of the stored data and the overall business scheme.
madru Active Member Licensed User Longtime User Mar 10, 2021 #6 or just something like this https://github.com/jayesh-prajapati/django-multiple-fileupload ? Upvote 0
Erel B4X founder Staff member Licensed User Longtime User Mar 10, 2021 #7 Only if you want to create a complex solution with more possible points of failure. Upvote 0
madru Active Member Licensed User Longtime User Mar 10, 2021 #8 not sure if the attached is correctly done but it does work Attachments jRDC_http.zip jRDC_http.zip 9.9 KB · Views: 246 Upvote 0
Erel B4X founder Staff member Licensed User Longtime User Mar 10, 2021 #9 I don't think that you need HttpJob + HttpUtils2Service modules. I would have done everything in RDCHandler. It will probably make the program flow simpler. Send a single request with all the data. The server saves the file wherever you like and puts the other data in the database. But I'm not familiar with all your requirements so maybe it makes sense to split it. Upvote 0
I don't think that you need HttpJob + HttpUtils2Service modules. I would have done everything in RDCHandler. It will probably make the program flow simpler. Send a single request with all the data. The server saves the file wherever you like and puts the other data in the database. But I'm not familiar with all your requirements so maybe it makes sense to split it.
madru Active Member Licensed User Longtime User Mar 10, 2021 #10 I tried and I failed do you have an example ? Upvote 0
R roerGarcia Active Member Licensed User Longtime User Mar 10, 2021 #11 The tree structure where the images are saved can be a root directory with the year and its subfolders or this structure can be simulated through the GUID or other #TAGS in the name of the images or use a GUID and describe in other fields from the database the year, month, day, consecutive of each image. in example image001.png --- TAGS [ 2021 | 03 | 09 | 001 ] GUID 055AF933-59E8-C059-7291-E3BA80BD9804 Upvote 0
The tree structure where the images are saved can be a root directory with the year and its subfolders or this structure can be simulated through the GUID or other #TAGS in the name of the images or use a GUID and describe in other fields from the database the year, month, day, consecutive of each image. in example image001.png --- TAGS [ 2021 | 03 | 09 | 001 ] GUID 055AF933-59E8-C059-7291-E3BA80BD9804
Erel B4X founder Staff member Licensed User Longtime User Mar 11, 2021 #12 1. Add Image() As Byte to DBCommand. 2. Modify the client code and add the image bytes to DBCommand. 3. Modify RDCHandler. Generate a random GUID. Save the image with File.WriteBytes, and continue with adding the GUID and other information to the database. You can also generate the GUID on the client side and send it as another field. Upvote 0
1. Add Image() As Byte to DBCommand. 2. Modify the client code and add the image bytes to DBCommand. 3. Modify RDCHandler. Generate a random GUID. Save the image with File.WriteBytes, and continue with adding the GUID and other information to the database. You can also generate the GUID on the client side and send it as another field.