B4J Tutorial [ABMonitor] Profiling your B4J/B4A apps 'live'

Discussion in 'B4J Tutorials' started by alwaysbusy, May 31, 2017.

  1. alwaysbusy

    alwaysbusy Well-Known Member Licensed User

    It has been some time since I could give my donators a new goodie ;), and this time it will be a library/tool to profile and monitor your B4J + B4A (7.01+) apps (not limited to ABMaterial WebApps!) because I needed something like this for some time for my own projects.

    Using a very simple API, you can track how long the code execution time is, the times hit, average time, memory usage etc for nearly everything you want.

    NOTE: Do not forget to set your DONATORKEY in the viewer params.txt + in the apps you are monitoring!

    [​IMG]


    How it works:
    ABMonitor uses the Jamon library, which has a extremely low overhead on your code. Just by disabling it (using the SetActive method), you can actually leave it in your production apps if you want (or use B4Js conditional compiling if you want to get rid of it in a production app).

    ABMonitor consists of two parts:
    1. The 'live' ABMonitor viewer

    This viewer shows all the stuff you are monitoring with the API. It shows e.g. how many times some part of your code was hit, how long it took, what the average time was, when it was last accessed, memory consumption etc...

    2. The ABMonitor library.

    This API connects your own apps with the monitor. It basically consists of a Start and a Stop method.

    First, we have to make the connection with the Viewer. Thanks to Erels new Resumable Subs, doing this is a breeze:

    In main make some declarations:

    Code:
    Sub Process_Globals
      
    Public Monitor As ABMonitor

       
    Private port As Int = 10090
       
    Private ip As String = "127.0.0.1" ' <-- Set your IP!
       Private abmonitor As AsyncStreams
       
    Private client As Socket

       
    Public TRACKMONITOR As Boolean = True
    End Sub
    Next add the following resumable sub (the ABMonitor part is the SetActive() method):
    Code:
    Sub ConnectMonitor()
       
    Dim c As Socket
       c.Initialize(
    "client")
       c.Connect(ip, port, 
    5000)
       
    Wait For Client_Connected (Successful As Boolean)
       
    If Successful Then
         client = c
         abmonitor.InitializePrefix(client.InputStream, 
    False, client.OutputStream, "abmonitor")
         
    Log("ABMonitor connected")
         Monitor.SetActive(
    "Template"True,abmonitor, 1)
       
    Else
         
    Log("ABMonitor disconnected")
       
    End If
    End Sub

    Sub abmonitor_Error
       Monitor.SetActive(
    "Template"False,Null0)
       
    Log("ABMonitor disconnected")
    End Sub
    Finally, call the sub when your app starts (a good place is e.g. before StartMessageLoop):
    Code:
    ...
    Monitor.Initialize(
    "YOURDONATORKEY")

    ConnectMonitor

    StartMessageLoop
    Ready to do some monitoring!

    There are two ways to do this:

    a. Monitor some code:
    If for example you want to monitor a query, or a whole sub, ... In general this is a complete block of code.

    Good practice is using the class/module name as the Group parameter, and the method name as the Label, but you can put whatever you want. This will later be used in the Viewer to group stuff. (Group and Label are the first and second parameters in the calls).

    Example:
    Code:
    Private Sub WebSocket_Connected (WebSocket1 As WebSocket)
       
    If Main.TRACKMONITOR Then Main.Monitor.Start("ABMPageTemplate""WebSocket_Connected""")

       
    '   ... the code you want to monitor

       
    If Main.TRACKMONITOR Then Main.Monitor.Stop("ABMPageTemplate""WebSocket_Connected""")
    End Sub
    Or tracking a query:
    Code:
    ...
    If Main.TRACKMONITOR Then Main.Monitor.Start("ABMPageTemplate""MySlowQuery""")

    Dim SQL_str As String
    SQL_str = 
    "SELECT cases.CaseID, cases.CaseUserID, cases.CaseType, cases.CaseSummary FROM tCases WHERE cases.CaseStatus=1;"
    Dim cases As List = DBM.SQLSelect(SQL, SQL_str, Null)

    If Main.TRACKMONITOR Then Main.Monitor.Stop("ABMPageTemplate""MySlowQuery""")
    ...
    b. Monitor methods which are used in multiple places, and you want to know where it was called.
    You have for example a page.Refresh method, which is called in multiple places. You can use the third parameter to set the 'caller'. In general there will only be one line of code between the start() and stop().

    This caller will later be used in the Viewer to build a call tree (stack trace)

    Example:
    Code:
    Private Sub WebSocket_Connected (WebSocket1 As WebSocket)
         ...

         
    If Main.TRACKMONITOR Then Main.Monitor.Start("ABMPageTemplate""page.Refresh""WebSocket_Connected")
         
    page.Refresh
         
    If Main.TRACKMONITOR Then Main.Monitor.Stop("ABMPageTemplate""page.Refresh""WebSocket_Connected")

         ...
    End Sub

    public Sub ConnectPage()
       ...

       
    ' refresh the page
       If Main.TRACKMONITOR Then Main.Monitor.Start("ABMPageTemplate""page.Refresh""ConnectPage")
       
    page.Refresh
       
    If Main.TRACKMONITOR Then Main.Monitor.Stop("ABMPageTemplate""page.Refresh""ConnectPage")

        ...
    End Sub
    As you can see, you are totally free to monitor anything you want.

    Note: This is DonationWare only!

    Alain
     
    Last edited: Jun 26, 2017
    DMW, DonManfred, PCastagnetti and 7 others like this.
  2. Widget

    Widget Active Member Licensed User

    All I can say is "WOW!". I'm impressed. :D
    Can it also monitor the amount of memory consumed by each sub? Or by the app in general?

    Will you eventually have a demo app we can download and try it out on?
     
    joulongleu and Erel like this.
  3. mindful

    mindful Active Member Licensed User

    This is really really usefull ! Great work as usual !
     
    joulongleu and Erel like this.
  4. alwaysbusy

    alwaysbusy Well-Known Member Licensed User

    I will build-in something, however how exact it will be is something different. The Java garbage collector does not guarantee it cleans everything (when you would expect it would). It may be somewhat useful to see a 'trend' over a longer period (e.g. on a graph), but it will never be exact.

    It will be a available for all donators of any of my libraries. These kind of libraries are a extra 'Thank You' from me for their support.
     
    DonManfred and joulongleu like this.
  5. Widget

    Widget Active Member Licensed User

    That's great new. I think some monitoring of memory usage is important. It could monitor the memory when the app has been running for several hours.

    That's understandable. I started to write profiling code for the app that I'm working on now so you have impeccable timing.;)
    Will it also work for B4A? Please let us know when and where your profiler is available.

    BTW, there was a profiler for Delphi called "Sampling Profiler" that also used a web server to display the profiling results in a browser. It worked really well and was incredibly easy to use.
     
    joulongleu likes this.
  6. alwaysbusy

    alwaysbusy Well-Known Member Licensed User

    YES! Just wrote a modified version of the lib (works exactly the same as the B4J version) and it worked :)

    It will need B4A 7.00+ (because it uses resumable subs).

    From my Android phone, tracing how long a sync takes:

    [​IMG]

    When I have something, I'll post it in the annoucements. Libraries work, just need to finish the monitor app.
     
    DMW, joulongleu, Erel and 2 others like this.
  7. Widget

    Widget Active Member Licensed User

    It is looking good. :D

    Can a "Mem Used" column be added? (Or is that what "Mem Last", "Mem Min", "Mem Max" are calculated as?)
    I'm thinking it might be useful if you calculated the difference between the memory available when the sub started and when it ended, then you have the amount of memory that the sub consumed. It will ignore the garbage collection after the sub exits, which is fine because this "memory used" tells me if the sub is too fat memory-wise. In other words, if the sub requires a lot of memory even if it is temporary, it could be a concern to the programmer.

    I noticed one minor display problem that is probably already on your to-do list. The numbers are left justified. Making them right justified (with a little padding on the right) will make them easier to read when you have several rows because the commas will line up properly. Like I said, it is a minor point but will make the results easier to read.

    Can I assume the columns are sortable? That way it is easier to find the slowest subs, or the subs that are called the most etc.
    Thanks for all the work you've put into it. A lot of people will find it useful.

    TIA
     
    joulongleu and mindful like this.
  8. alwaysbusy

    alwaysbusy Well-Known Member Licensed User

    Yes, the mem variables are calculated as you describe.

    This is just my test screen, not how the final app will look like.
     
    joulongleu and mindful like this.
  9. alwaysbusy

    alwaysbusy Well-Known Member Licensed User

    Some progress on the profiler/monitor view. As there was no TreeTableView component available yet for B4J, I had to write my own first.

    [​IMG]

    Still lots to do, but I'm getting there...
     
    joulongleu, Harris, Widget and 4 others like this.
  10. alwaysbusy

    alwaysbusy Well-Known Member Licensed User

    Donators can now download ABMonitor from the feedback app! :)

    I had some trouble with the jCharts library (sometimes it did no draw the charts as the data may be have come to fast), so I wrote my own charts in pure B4J. I was able to reproduce the original look of the jCharts.

    You'll see, next to ABMaterial and ABMonitor in the download page some other extra libraries for you. So please consider the thousands of hours I've put into all of this when you make a donation. ;)

    [​IMG]

    Alain
     
    Last edited: Jun 26, 2017
Loading...