B4J Question Filewatcher: limits ?

[wl]

Hi,

I inted to use a file system watch in combination with the B4J webserver (websockets).

For each logged in user I would send a message to the browser if there is any change in a specific folder designated and private for each user.

When 100+ users would be logged in at the same time, it would mean 100+ file watches.

Would this be possible ?

Thanks


https://www.b4x.com/android/forum/threads/jfilewatcher-watch-system-file-events.56613/#content

 

[Harris]

When 100+ users would be logged in at the same time, it would mean 100+ file watches.

Probably...

You could try first copying file to a common dir, with a structured file name ("user1_file.txt")
Watch this folder for new files and parse file name for "user1" - do what you need to - then move this file to users folder...

Just a thought...

 

[Erel]

I'm not familiar with jFileWatcher implementation so I cannot say whether it will be problematic or not. It seems like it uses a polling mechanism on Linux.
Also note that it raises the events on the main thread. This means that you need to use CallSubDelayed to run a sub in the handler class with the correct thread.

You can also implement your own solution with a background worker that runs every few seconds and scans the file system for changes. The advantage is that you will have full control.

 

[Roycefer]

Attached is a little stress test program. In the command line arguments, you can set the number of watchers, the frequency of file creations and the number of seconds you want the program to run. I've run it with up to 100 watchers and 10 file creations/sec for 30 seconds. There doesn't seem to be any drop-off in performance. This holds for a Core i7 Windows computer as well as a Raspberry Pi 2. You can run it on your hardware to see if the FileWatcher will work for your purposes. I suspect it will.

My preliminary research indicates that this library doesn't use polling on Linux (at least after kernel 2.6.13) but rather native file system events. This is tentative but, either way, the latency between file creations and the corresponding events being raised was often around 1-2 ms on the RasPi2.

 

[Haris Hafeez]

A better approach would be to capture the event(s) that result in the files being modified/added. I have in the (rather distant) past (~10 yrs) run against issues with file watchers where the kernel raises the 'changed' event before the file change was committed to the disk and as a result reading in those changes caused issues. That was solved by introducing a delay among other hacks. Things may have improved though on this front.

Also this suggestion will work if you have control over the source of the events. That is, if the other system is simply writing to files then file monitoring is a decent option.