Android Question REST API APP - optimization

[luke2012]

Ho all,
I'm dealing with the optimization of an app that uses the rest api of a remote CMS (Drupal) that downloads about 40 nodes (all content on a Drupal website is stored and treated as "nodes". A node is a piece of individual content).
So for this case I'm using the OKHttpUtils2 from @Erel https://www.b4x.com/.../b4x-okhttputils2-with-wait-for.../ (i have to say that the code has a clean syntax and works very well).

DOWNLOAD TIME
The time to download all this nodes (that my app use) is about 10 secs when app start for the first time.
In order not to make the user impatient for waiting for the download before being able to use the app, I have implemented a main screen with an animated gif, but unfortunately we are realizing that it is not enough, the download time has increased (we are investigating what is the cause on the server side).

THE ISSUE
in this case the user waits 8 / 10 seconds (timer) in the first screen of the animated gif then when he touches the button to view the data, not being completely downloaded, a waiting message is displayed (until the data has been completely downloaded), inhibiting the opening of the data list screen and for the end user this is frustrating.

SOLUTION ?
in this case what is the most correct approach to make the app well usable for the end user, without doing so waits every time he opens the app? To save data locally on the phone (a kind of cache)?

Public Sub Initialize 'Class
    Download1
End sub

Sub Download1 As ResumableSub
    For Each nid As Int In nidsList
        Wait For (GetContent (nid, user, psw)) Complete (Content1 As Map)
    Next
End Sub

Sub GetContent (nid As Int, aUser As String, aPsw As String) As ResumableSub

    Wait For (GetJSONByNID (nid, aUser, aPsw)) Complete (WSResult As String)
    Private WSResponse As String = WSResult
    '......
 
end sub

Sub GetJSONByNID (nid As Int, aUser As String, aPsw As String) As ResumableSub

    Dim j As HttpJob : j.Initialize("j", Me)
    j.Username = aUser
    j.Password = aPsw
    Private WSNodeURL As String = WebHostURL & "/node/"
    Private WSFormat As String = "?_format=json"
    Private WSUrl As String = WSNodeURL & nid & WSFormat

    j.Download (WSUrl)
    'Log ("GetJSONByNID | WSUrl: " & WSUrl)

    Wait For (j) JobDone(j As HttpJob)
    If j.Success Then
        Private WSResponse As String = j.GetString2("UTF8")
        'Log ("GetJSONByNID | WSResponse " & WSResponse)
    End If
    j.Release

   'Parse JSON....

End Sub

 

[FrostCodes]

I think the optimizations you need to make depend on what the app allows the end-user to do.
For example if the app only allows the end-user to see the data and it's not necessary that the data is always in its latest form, you can do a local cache but it it allows the user to edit data, then this method most likely won't work.

Try explaining a bit more about the app. Also, you might need to tune your DB if its MySQL for example...

 

[Andrew (Digitwell)]

My first suggestion would be to pinpoint the areas of delay.

Use DateTime.Now to print the timestamps at various points in the download.

e.g.

Sub GetContent (nid As Int, aUser As String, aPsw As String) As ResumableSub
    dim dur as long = datetime.now
    Wait For (GetJSONByNID (nid, aUser, aPsw)) Complete (WSResult As String)
    Private WSResponse As String = WSResult
    '......
     log($"GetContent  - node ${nid} took ${(datetime.now-dur)/datetime.tickspersecond} seconds"$)
end sub

That should help pinpoint the performance issue to either processing code or download code.

 

[luke2012]

My first suggestion would be to pinpoint the areas of delay.

Use DateTime.Now to print the timestamps at various points in the download.

e.g.

Sub GetContent (nid As Int, aUser As String, aPsw As String) As ResumableSub
    dim dur as long = datetime.now
    Wait For (GetJSONByNID (nid, aUser, aPsw)) Complete (WSResult As String)
    Private WSResponse As String = WSResult
    '......
     log($"GetContent  - node ${nid} took ${(datetime.now-dur)/datetime.tickspersecond} seconds"$)
end sub

That should help pinpoint the performance issue to either processing code or download code.

Hi Andrew,
to pinpoint the areas of delay I'm using Postman set to "GET" (basic auth) and with url format "https://domain.com//node/ <X>? _Format = json". The remote server is a Drupal (9.x) CMS.

The nodes that I download within the app are logically divided into 3 categories and therefore I am doing 3 separate .download () groups.
Based on the times in ms that Postman gives me, I got the total times for each category:

POSTMAN TEST
Test with Postman - url format: https://dominio.com//node/<X>?_format=json
T (average) for 1 node = 550 ms
D (average) for 1 node = 2,6 kb

CATEGORY A (20 nodes - 52 kb - 11 seconds)
---------------------------
drupal nodes: 40,42,44,45,46,47,48,49,51,52,54,66,90,125,127,130,131,151,169,173
550 (ms) x 20 (nodes) = 11.000 ms (11 seconds) to download (get) 52 kb

CATEGORY B (8 nodes - 21 kb - 5 seconds)
-------------------------
drupal nodes: 32,33,34,35,37,38,41,104
550 (ms) x 8 (nodes) = 4.400 ms (almost 5 seconds) to download (get) 21 kb

CATEGORY C (14 nodes - 36 kb - 8 seconds)
---------------------------
drupal nodes: 18,19,22,24,93,106,108,123,126,128,132,149,155,171
550 (ms) x 14 (nodes) = 7.700 (almost 8 seconds) to download (get) 36 kb

 

[FrostCodes]

If I am right you are sending up to 20 API requests at once, right?
I think this might be the issue...

 

[luke2012]

If I am right you are sending up to 20 API requests at once, right?
I think this might be the issue...

"If I am right...": yes you are right:
- For CATEGORY A (20 nodes) 20 api requests
- For CATEGORY B (8 nodes) 8 api requests
- For CATEGORY C (14 nodes) 14 api requests

 

[DonManfred]

"If I am right...": yes you are right:
- For CATEGORY A (20 nodes) 20 api requests
- For CATEGORY B (8 nodes) 8 api requests
- For CATEGORY C (14 nodes) 14 api requests

Why not change the php-script on serverside and return all Data with ONE api-request?
Or do a save of the returned data and only request data which has changed on serverside. I´m positive the node data has such properties (at least i am guessing!)

 

[FrostCodes]

Why not change the php-script on serverside and return all Data with ONE api-request?
Or do a save of the returned data and only request data which has changed on serverside. I´m positive the node data has such properties (at least i am guessing!)

Exactly what I was going to suggest... Also, try and filter out the data columns you don't need. Your program would perform really well.
You can also Tune your server if you have root access.

 

[luke2012]

@DonManfred and @FrostCodes thanks for your suggestions.

Unfortunately for this project I'm in charge only for the client (app) development.
Hosting and the drupal instance is manager by my client.

But I can suggest also this to him.