B4J Question MQTT With Large Files

[walterf25]

Hi All, i was wondering if anyone could give me some insight on whether it would be feasible or advisable to use MQTT to Send and Receive large files, maybe a database file with around 10000 records, i'm just looking to implement a system where a RFID Scanner will be used to do Inventory in a large warehouse, the RFID tags read will be stored in a database file, i'm just thinking out loud and looking for a quick and easy solution, I know I could probably set up a Server on a local P.C. and just store the RFID tags directly on the online database, has anyone used MQTT to do something like this, based on the examples floating around the forum using the MQTT broker library i don't really see a problem implementing something like this, but wanted to hear from someone else maybe there's a reason why I shouldn't implement something like this?

Thanks All,
Walter

 

[BillMeyer]

@walterf25

Before I make some way out assumptions, please tell me: "Is your Scanner Connected all the time and is it programmable ?"

If you are going to store your tags in a database and your scanner is connected, then use MQTT to send the messages ones at a time at the point of scanning and have a back end (the place where the database resides) read the message and act on the payload to store it in the database.

If your scanner is one of those that can collect data and needs a docking station to "download" the collected data then the PC that it is connected to via the docking station can then send the data via a normal connection to a MySQL or MSSQL or whatever database. If you want to use MQTT here, it would be possible, bearing mind that the max packet size is 256mb, and I would take the data and convert it to JSON and then send it as the payload. Once it reaches it's destination, you have a sub that reads the payload and then re-acts on it - like reading the JSON records and inserting/updating (possibly even use the REPLACE syntax) to manipulate your database.

I hope this helps somewhat and I have not missed the point.

 

[Erel]

What is the expected file size?

 

[walterf25]

@walterf25

Before I make some way out assumptions, please tell me: "Is your Scanner Connected all the time and is it programmable ?"

If you are going to store your tags in a database and your scanner is connected, then use MQTT to send the messages ones at a time at the point of scanning and have a back end (the place where the database resides) read the message and act on the payload to store it in the database.

If your scanner is one of those that can collect data and needs a docking station to "download" the collected data then the PC that it is connected to via the docking station can then send the data via a normal connection to a MySQL or MSSQL or whatever database. If you want to use MQTT here, it would be possible, bearing mind that the max packet size is 256mb, and I would take the data and convert it to JSON and then send it as the payload. Once it reaches it's destination, you have a sub that reads the payload and then re-acts on it - like reading the JSON records and inserting/updating (possibly even use the REPLACE syntax) to manipulate your database.

I hope this helps somewhat and I have not missed the point.

Hi BillMeyer, thanks for your reply.

The scanner i will be using is Android Based Scanner, it can scan Barcodes and RFID tags, the users will actually scan the barcodes and RFID tags and the information will be stored on a local database, once they are done performing an inventory they will go back to their offices, connect the scanner to their WiFi connection where they will be able to send the information collected, initially They had thought about just transferring the information using a simple FTP server, but then I suggested creating full B4A and B4J solution.

This is the Scanner I will be using, at the moment i found an example in the B4J forums where Erel implements a DataCollection Server, I am using that as a template and I have so far been able to read the records inserted into the DB and create a JSON String including the RFID Tag Number and Location number, then this Json String is Sent to the B4J DataCollection Server, in the B4J app i receive that JSON String, parse the JSON String and insert the Records into a the Online DataBase.

So Far this seems the way to go, although I'm not too sure how this will work once there Are 1000s of records, at the moment i am only sending around 50 records.

Regards,
Walter

 

[walterf25]

What is the expected file size?

Is hard to say, however there will only be two tables one for Barcode information and another for RFID Tags information, each table will only have at the most 3 columns, i'm guessing the DB file won't be bigger than a few MB.

Walter

 

[BillMeyer]

Walter,

This takes me back to my days at Cummins with Track and Trace.

A few things from your answer.
1. The scanner - it has 4 things to your advantage, a. It's Android based, b. It has WiFi, c. It also has cellular, d. It has the ability to run custom apps.

Now the question is one of process flow. You can stick with the "docking station upload method" you describe - and it will work - but there are a few steps involved here in your process.

2. I do not know if there is WiFi in the Warehouse. If this is the case then I would write an app on the Scanner, that would transfer either:

a. Directly to your Database (if it is MySQL). This would eliminate the intermediate steps of first docking the unit, processing the
data from it, turning it into whatever format you require, sending it to the database server, decoding it there and inserting/updating your online database. You do however have a security risk here with MITM attacks.

b. Send it to a jRDC2 Server (that resides on the same Server as where your database is) and use jRDC2 Client in your scanner to send the data direct to the database. With this method you can do read/process/respond in real time, AND it is much safer security wise.
​

If you do not have WiFi in the warehouse, then I suppose the exercise would be one of determining cost of WiFi Range extenders as opposed to providing cellular data as opposed to how quickly (such as in FMCG) you need to have the data available for a next step.

There are so many ways of doing this that at the end of the day it will come down to what you are most comfortable with and what resources you have available to you when you commence with this project.

If you have private questions, PM me - I'll try and impart my experience with you like that if it is not for general public consumption.