B4J Library jFileWatcher - watch system file events

Discussion in 'B4J Libraries & Classes' started by Roycefer, Jul 26, 2015.

  1. Roycefer

    Roycefer Well-Known Member Licensed User

    This library wraps many capabilities from the java.nio.file package and a few from the java.io.File package. It allows you to set certain directories to be watched, raising events when files or folders within those directories are created, deleted or modified. Additionally, this library enables you to get and set last access time, last modified time, creation time, read-only state and hiddenness of files. Beware, some operating systems won't support all of those attributes.

    On Windows, this library performs quite well, using the system's native file events and consuming only a minuscule amount of system resources. The latency was also extremely low. On systems that don't have native file events, the library will poll the file system for changes, periodically. This might introduce a latency in receiving file change events and might consume more resources, though I haven't tested it on any such systems.

    The watching is performed off the main thread however events are raised on the main thread.

    Example code:
    Code:
    Sub Process_Globals
        
    Dim fw As FileWatcher
        
    Dim t as Timer
    End Sub

    Sub AppStart(Args() as String)
         t.Initialize(
    "t"60*1000)   'Stop file watching in 1 minute
         fw.Initialize("fw")     'Initialize with the event name
         fw.SetWatchList(Array as String(File.DirApp))   'Set the current dir to be watched
         fw.StartWatching       'Start Watching
         t.Enabled = true        'Start the timer

         
    Log("creation time: " & fw.GetCreationTime("C:\Test\test.txt"))   'Log creation time of a file
         fw.SetCreationTime("C:\Test\test.txt"DateTime.Now)    'Set the creation time to now
         Log("creation time: " & fw.GetCreationTime("C:\Test\test.txt"))   'Log the new creation time of file
         StartMessageLoop
    End Sub

    Sub fw_CreationDetected(FileName As String)
        
    Log("CreationDetected: " & FileName)    'Logs the creation of a new file or folder
    End Sub

    Sub fw_DeletionDetected(FileName As String)
        
    Log("DeletionDetected: " & FileName)   'Logs the deletion of a file or folder
    End Sub

    Sub fw_ModificationDetected(FileName As String)
        
    Log("ModificationDetected: " & FileName)   'Logs the modification of a file or folder
    End Sub

    Sub fw_WatchingTerminated   'Logs the termination of the FileWatcher
        Log("WatchingTerminated")   'The FileWatcher can still be used, it just has to be started again.
        StopMessageLoop     'With the main message loop ended and the FileWatcher
                                   'watcher thread ended, this application will close now.
    End Sub

    Sub t_Tick
         fw.StopWatching
         
    Log("Timer Ticked, watching stopped")
    End Sub
    This library was compiled against jdk 8.

    EDIT(26JUL2015): Updated to version 1.1. See post #3 for details.
    EDIT(2AUG2015): Updated to version 1.2. See post #8 for details.
     

    Attached Files:

    Last edited: Jun 7, 2017
    swissmade, jmon, rwblinn and 5 others like this.
  2. giga

    giga Well-Known Member Licensed User

    Haven't tried it yet but nice work. Thanks for sharing!!
     
  3. Roycefer

    Roycefer Well-Known Member Licensed User

    Version 1.1 adds the ability to get and set the read-only status of a file. This should make it possible to write handy little Java apps that automate the creation of all those extra read-only high-resolution drawables, each in their own folder, that you need to fool Google into thinking your app was designed for tablets.
     
  4. Roycefer

    Roycefer Well-Known Member Licensed User

    Some experiments:
    • On a Core i7 Windows machine, steady state event latency is under 1 ms. I timed the interval between a File.WriteString call and the _CreationDetected event that responded to that new file. It was about 27 ms on average during the JVM warm-up phase and under 1 ms once the JVM was warmed up (a few seconds after program start). Once the JVM is in its steady state phase, this library is really snappy.
    • On systems that support native file events (like Windows, but probably others as well), the watcher thread consumes essentially 0 CPU resources while it's waiting for events. Only when an event is detected does the thread start running. Even then, the resource usage is quite low as the thread is only calling the events subs and then going back to its waiting mode.
    • If you have Windows computers A and B on the same network and A has a shared folder, visible from B, then a B4J program using this library running on B can watch that shared directory on A. This opens up some possibilities for intra-network communications without the hassle of setting up Servers and Sockets.
    I suspect the last point probably applies to non-Windows computers as well. If any users could report back on their success or failure, that would be appreciated. Also, I wonder if the FileWatcher library can watch a NAS. Any users with a NAS should feel free to report on their results, as well.
     
    Last edited: Jul 27, 2015
  5. alienhunter

    alienhunter Active Member Licensed User

    Thanks for sharing works well localy (would be nice if it would show the new created file name :) )
    it is working on a NAS or Mapped Networkdrive ..
     
  6. Roycefer

    Roycefer Well-Known Member Licensed User

    The newly created file name is in the "FilePath As String" parameter in the _CreationDetected event. See the example code in the original post. It reports the full file path, though, so you'll have to use File.GetName(FilePath) to extract just the file name.

    Thanks for reporting on the NAS. I suspected that it would work, I just don't have a NAS on which to test it. In principle, I think it should work on any directory where the operating system's file explorer reports changes to the directory in real time. That should include NAS, network-shared directories, external storage, RAM drives, etc... After all, on most systems, the underlying Java library is making use of the native file events which, presumably, is the same API used by the OS's file explorer.
     
  7. alienhunter

    alienhunter Active Member Licensed User

    :) thanks
    and you are welcome
     
  8. Roycefer

    Roycefer Well-Known Member Licensed User

    There doesn't seem to be a way to get the full file path from the JVM so the event subs' signatures have been changed to show a parameter of FileName instead of FilePath. Just note that you aren't getting the full file path in the event subs, just the name of the file (or folder) that was created/deleted/modified. However, you should be able to infer the full file path since it's going to be one of the directories you are watching.

    To that end, I've tested creating more than one FileWatcher object in an app and it seems to tolerate it. This means that if you need to know the full file path and you can't infer it, you can create one FileWatcher object per watched directory and that way you'll always know what the directory was that raised an event. You can even have all your FileWatchers with the same eventName and get which one raised an event with Sender.

    To make it up to you, I've changed the SetWatchList sub to return the Me FileWatcher object so you can write concise code:
    Code:
    Dim fw as FileWatcher
    fw.Initialize(
    "fw").SetWatchList(Array As String(File.DirApp)).StartWatching
    Also fixed a few bugs.
     
  9. wl

    wl Well-Known Member Licensed User

    Hi,

    Any feedback on the event latency on Linux based systems, like Ubuntu ?
    And also: how does it behave when many concurrent actions happen in the same folder (while watching this folder). Will the event be called for each of these changes (even if happen -approximately- at the same time) ?


    Thanks
     
  10. Roycefer

    Roycefer Well-Known Member Licensed User

    I haven't tested it on Ubuntu but, from what I understand, most Linux systems have file system events in which case this library should perform similarly on Linux as it does on Windows. If you have a Linux system, feel free to test it yourself and report back on your findings. They will be of interest to other users.

    Yes, the event will be called for each action for which there is an active listener. The events are raised in the main thread so they won't execute concurrently.
     
  11. wl

    wl Well-Known Member Licensed User

    Hi,

    Meanwhile I tested it on Ubuntu 14 and it seems to be working as expected. Events are being called nearly instantly. Thanks
     
  12. alienhunter

    alienhunter Active Member Licensed User

    Hi ,
    how do i clear the fw.GetWatchList ? it seems the fw.GetWatchList.Clear does not do it
     
  13. Roycefer

    Roycefer Well-Known Member Licensed User

    Call
    Code:
    fw.SetWatchList(Array As String())
    Basically, you're changing the WatchList to an empty list.

    If you want to stop watching, you can also just call
    Code:
    fw.StopWatching
     
    alienhunter likes this.
  14. alienhunter

    alienhunter Active Member Licensed User


    thank you ,
    :oops::oops: .... fw.SetWatchList(Array As String())
    i created a small autoupload for mysql and the clear list is needed when the user changes to
    manual mode the list has to be clear , if not he cannot switch to manual mode
    AH
     
  15. rboeck

    rboeck Well-Known Member Licensed User

    Hi,

    i have a problem after the use of jFilewatch - in ~ one of twenty usecases maybe i am to fast or to early to open the found file. I want to use textreader to open a downloaded file.
    The use of timers seems unprofessionell for me. How can i prevent this exception?
     
  16. Roycefer

    Roycefer Well-Known Member Licensed User

    Can you post the stack trace of your Exception and the code causing it?

    Which event are you using to trigger your TextReader opening the file? The _CreationDetected event might fire before the file is completely written to disk. That is, there might be still writing to disk underway when you attempt to open the file. That might be causing your problem.

    Why are Timers unprofessional? They may be a bit tedious to use. In that case, I suggest using CallSubPlus. But it's hard to know if that would solve this problem without seeing the stack trace and code that is causing the problems.
     
  17. rboeck

    rboeck Well-Known Member Licensed User

    Currently i changed my design in this way, that i included a file.exist before opening with textreader. If the troubles exists further, i will copy the stack trace and repost.
    I will try to make an stress test for the code.
    Thanks for helping!
     
  18. TomDuncan

    TomDuncan Active Member Licensed User

    I have a folder which adds an image every 30 seconds.
    This image in in a date created folder.
    Can I just watch the root folder for any changes. Thus giving me the folder and file added.
    I have tried checking for a date change then vw.StopWatching
    then set new Dir with StartWatching.
    But does not like to work.

    Basically have can I either check whole folder for additions or change the folder on a daily basis.

    Tom
     
  19. Roycefer

    Roycefer Well-Known Member Licensed User

    Let me see if I understand your question: you have a folder structure that looks like this:
    Code:
    Root/Subdir/img1.png
    and you want to just watch Root and catch file creations occurring in Subdir? Anything that happens inside Subdir will raise a _ModificationDetected event for a FileWatcher that is watching Root. "Subdir" will be the FileName parameter passed to that event. In short, if you want to retrieve the file name of the new image file, you'll need to watch Subdir directly.

    Note that you can have two FileWatchers, one dedicated to watching Root and one dedicated to watching Subdir. Everytime a new sub-directory is added to Root, you'll get an event. You can use the FileName parameter in that event to update the Subdir watcher's target. Example:
    Code:
    Sub Process_Globals
       
    Dim RootWatcher As FileWatcher  'this will be watching Root
       Dim SubWatcher As FileWatcher   'this will be watching a sub-directory of Root
    End Sub

    Sub RootWatcher_CreationDetected(FileName As String)
       
    If File.IsDirectory(Root, FileName) Then
          SubWatcher.SetWatchList(
    Array(File.Combine(Root,FileName)))
          
    'SubWatcher is now watching the newest sub-directory in Root
       End If
    End Sub
     
    TomDuncan likes this.
  20. TomDuncan

    TomDuncan Active Member Licensed User

    Thanks for that.
    I can see what you are doing.
    Will test this afternoon.
    Tom
     
Loading...