Android Tutorial Large & Searchable List with SearchView + B4XSerializator

Discussion in 'Tutorials & Examples' started by Erel, Dec 31, 2015.

  1. Erel

    Erel Administrator Staff Member Licensed User

    Two very important methods were added to B4XSerializator in RandomAccessFile v2.20:
    ConvertObjectToBytesAsync and ConvertBytesToObjectAsync.

    With these methods you can asynchronously convert objects to bytes and vice versa. This allows doing things that previously were not possible.
    RDC2 for example is based on these methods: https://www.b4x.com/android/forum/threads/61801/#content

    SearchView was introduced in 2012. SearchView shows a list with an EditText that acts as a filter.
    It uses an in-memory index to find the matching items. SearchView is very useful, however it was limited to about 500 items as it takes time to build the index and the UI is frozen until the index is ready (the main thread is busy building the index).

    Now with B4XSerializator and a small B4J program we can easily load 10,000 items or more to SearchView.
    The B4J program will create the index and save it into two files. The first file holds the items list and the prefix based index
    The second file holds the secondary index (full text index) which is much larger.

    In the B4A program we will first load the smaller index and then the larger index (they were added to the Files tab). Both are loaded asynchronously so the UI is not affected.
    Until the second index is loaded the user will only be able to search for items based on the prefix. It is highly unlikely that the user will notice this at all.

    upload_2015-12-31_15-53-41.png


    Why are the new async methods so important here?

    On a Nexus 5 it takes 1.5 seconds until the first index is ready and another 5-6 seconds for the second index.
    As everything is done in the background, the UI is fully responsive while the data is loaded.
    If we were using the non-async methods then the UI would have been frozen for 7.5+ seconds. This is a very bad UI and is also likely to be killed by the OS.

    Notes

    - This solution is also compatible with B4i as B4XSerializator is cross platform (with some changes to work with iTableView).
    - The SearchView class is a modified version.
    - The data in this example is Alexa's top 10,000 sites.


    I'm sure that going forward we will see more and more usages for B4XSerializator.
     

    Attached Files:

    AnandGupta, hibrid0, Procesor and 7 others like this.
  2. shashkiranr

    shashkiranr Active Member Licensed User

    Hi Erel,

    If we want to use this searchview with data coming from the server db, then should we install the b4j program in the server,get the first and last index files an use that in the android program?

    Or is there a better way to do it? I am using the searchview to search for names downloaded from a mysql db. How can i implement this?

    Best,
    SK
     
  3. valentino s

    valentino s Active Member Licensed User

    Thanks Erel.
     
  4. Erel

    Erel Administrator Staff Member Licensed User

    The best way way is to implement your server with B4J and then convert the data inside the server code. Another option is to create a small B4J program and call it from your server.
     
  5. Mahares

    Mahares Well Known Member Licensed User

    Please clarify this brief tutorial by @Erel for me. Based on my understanding of this tutorial, here is the procedure I implemented to use this SearchView from a SQLite database:
    a. Created a database on the PC via B4J and inserted records.
    b. Copied all records to a list via B4J that I subsequently used to create the two .dat index files.
    c. Next, copied the 2 files manually to the asset folder of the B4A application and started using the B4A app.

    My questions are:
    1. Are the above steps correct? It appears that there are too many steps just to get to the index. Is there a better way?
    2. What happens if the database from which the two .dat files are created change frequently? Do I have to go through the same lengthy process outlined above.

    Thank you
     
  6. Erel

    Erel Administrator Staff Member Licensed User

    Your steps are correct.

    All three steps could be executed by the same B4J program.
     
  7. Mahares

    Mahares Well Known Member Licensed User

    How would you copy the 2 .dat files programmatically from B4J app folder to the files (assets) folder of the B4A program?
    Thank you.
     
  8. Erel

    Erel Administrator Staff Member Licensed User

    This is equivalent to using File.Copy from the B4J program.

    My guess is that you want to update the database at runtime. In that case you need to implement the same solution inside the B4J server.
     
  9. Mahares

    Mahares Well Known Member Licensed User

    Are we talking about something as simple as this, which seems to work?
    Code:
    File.Copy(File.DirApp,"second.dat","C:\Temp\Junk\B4JindexBuilder\Searchview\Files","second.dat")
    File.Copy(File.DirApp,"first.dat","C:\Temp\Junk\B4JindexBuilder\Searchview\Files","first.dat")
     
  10. Erel

    Erel Administrator Staff Member Licensed User

    Yes.
     
    Mahares likes this.
  11. Roberto P.

    Roberto P. Well-Known Member Licensed User


    Erel, would be very important to your example, certainly well done and optimized, server-side with b4j to provide the real-time data, such as a web server, to the list.
    Thank you in advance.
     
  12. Erel

    Erel Administrator Staff Member Licensed User

    This server code will convert the index to an array of bytes:
    Code:
    Dim search As SearchView
    search.Initialize
    Dim obj() As Object = search.SetItems(dataToIndex)
    Dim ser As B4XSerializator
    Dim DataToSend() As Byte = ser.ConvertObjectToBytes(obj)
    Now you should send DataToSend to the client.

    On the client you should read it with ser.ConvertBytesToObjectAsync and set it with this code:
    Code:
    Private Sub ser_BytesToObject (Success As Boolean, NewObject As Object)
       
    If Success Then
         
    Dim obj() As Object = NewObject
         firstIndex = 
    Array (obj(0), obj(1))
         secondIndex = obj(
    2)
         sv.LoadFirst(firstIndex)
         sv.LoadSecont(secondIndex)
       
    End If
    End Sub
    In this case, where the data comes from a remote server, it is less important to split the task as done in first post example.
     
    DonManfred likes this.
  13. Anser

    Anser Well-Known Member Licensed User

    I avoided the use of SearchView because I face the problem specified in the first post of this thread ie "The searchView takes time to build the index and the UI is frozen until the index is ready (the main thread is busy building the index)."

    I would like to use this new faster SearchView in my B4A app so that my UI doesn't freeze.

    My Challenge
    I am using jRDC2. The data in the SearchView has to come from a Remote MySQL DB via jRDC2 and not from a file that resides in the File.DirAssets Folder

    I may be really dump, I can't understand where to start to get this done.

    Do I need to run the B4J app (B4J_IndexBuilder.zip) attached in the First post of this thread on my VPS Server ? I mean the same server where my jRDC2 is running. ? If it is to run on the same VPS server, then on which port should it run ?. How do I call this from my B4A app.

    As I already said, my data comes from a remote MySQL DB.

    So far I understand that in the B4J app. I should make the following changes
    Code:
    'Non-UI application (console / server application)
    #Region  Project Attributes
        
    #CommandLineArgs:
        
    #MergeLibraries: True
    #End Region

    Sub Process_Globals

    End Sub

    Sub AppStart (Args() As String)

        
    Dim dataToIndex As List = File.ReadList(File.DirAssets, "sites.txt")
        
    Dim search As SearchView
        search.Initialize
        
    Dim obj() As Object = search.SetItems(dataToIndex)
        
    Dim ser As B4XSerializator
        
    ' The following lines commented by Anser
        'WriteFile("first.dat", ser.ConvertObjectToBytes(Array(obj(0), obj(1))))
        'WriteFile("second.dat", ser.ConvertObjectToBytes(obj(2)))

        
    'Added the following line as per Erel's post
        Dim DataToSend() As Byte = ser.ConvertObjectToBytes(obj)
    End Sub

    'May be the following Sub is not required at all
    Sub WriteFile(FileName As String, b() As Byte)
        
    Dim out As OutputStream = File.OpenOutput(File.DirApp, FileName, False)
        out.WriteBytes(b, 
    0, b.Length)
        out.Close
    End Sub
    And in the B4A example (LargeSearchView.zip) available in the fisrt post of this thread, the following Sub
    Code:
    Private Sub ser_BytesToObject (Success As Boolean, NewObject As Object)
        ProgressBar1.Visible = 
    False
        
    If Success Then
            
    Dim ser As B4XSerializator = Sender
            
    Log($"BytesToObject: ${ser.Tag}"$)
            
    If ser.Tag = 1 Then
                firstIndex = NewObject
                sv.LoadFirst(firstIndex)
                ser.Tag = 
    2
                
    'read the secondary index
                ser.ConvertBytesToObjectAsync( _
                    
    Bit.InputStreamToBytes(File.OpenInput(File.DirAssets, "second.dat")), "ser")
            
    Else if ser.Tag = 2 Then
                secondIndex = NewObject
                sv.LoadSecond(secondIndex)
            
    End If
        
    End If
    End Sub
    The above Sub should be changed as follows
    Code:
    Private Sub ser_BytesToObject (Success As Boolean, NewObject As Object)
       
    If Success Then
         
    Dim obj() As Object = NewObject
         firstIndex = 
    Array (obj(0), obj(1))
         secondIndex = obj(
    2)
         sv.LoadFirst(firstIndex)
         sv.LoadSecont(secondIndex)
       
    End If
    End Sub
    So what would be the equivalent code for the following Sub in the B4A sample
    Code:
    Private Sub LoadIndices
        
    Dim ser As B4XSerializator
        ser.Tag = 
    1
        ser.ConvertBytesToObjectAsync( _
            
    Bit.InputStreamToBytes(File.OpenInput(File.DirAssets, "first.dat")), "ser")
    End Sub
    In my case, the above said data does not reside on File.DirAssets, this has to come from the Remote MySQL DB via jRDC. I am confused, how is this going to work. May be I didn't understand the concept well.
    Is that right what I have understood so far ?
    Doesn't that B4J app run on some specific port on the same VPS server where I run my jRDC ?

    Any push in the right direction will be appreciated.
     
    Last edited: Jun 14, 2016
  14. Erel

    Erel Administrator Staff Member Licensed User

    B4J_IndexBuilder example is implemented as a non-ui program. You need to learn its code (the relevant code is about 10 lines) and add it to your server together with the SearchView module.
     
  15. Anser

    Anser Well-Known Member Licensed User

    Any plans to incorporate this feature in jRDC in its future versions ?

    My knowledge level in developing Tools is limited. I still don't know from where in the RDCHandler.bas in jRDC should I call the following function.


    Code:
    Sub CreateSvIndex (dataToIndex As List)
       
        
    Dim search As SearchView
        search.Initialize
        
    Dim obj() As Object = search.SetItems(dataToIndex)
        
    Dim ser As B4XSerializator
        
    Dim DataToSend() As Byte = ser.ConvertObjectToBytes(obj)
    End Sub
    A List should also be generated.

    Can anyone help me on this ?

    Regards

    Anser
     
    Last edited: Jun 15, 2016
  16. Erel

    Erel Administrator Staff Member Licensed User

    No. It is a very specific feature. The requirements will be different in each project.

    It will be simpler to add another handler in the server that is responsible for building DataToSend and call it with HttpUtils2. For further discussion please start a new thread.
     
    Anser likes this.
  17. Roberto P.

    Roberto P. Well-Known Member Licensed User

    Hello Erel,
    I would understand if and how you can use this new object to read data from the local DB tables for put data in SearchView.
    In other words, I would like to find a way to read data from a simple table, for example consisting of code and description, and transfer the data in SearchView.

    You can indicate the best way to do this using B4XSerializator

    Thank you
     
  18. Erel

    Erel Administrator Staff Member Licensed User

    Where does the data come from? Is it static?
     
  19. Roberto P.

    Roberto P. Well-Known Member Licensed User

    the data are in the local database
     
  20. Roberto P.

    Roberto P. Well-Known Member Licensed User

    Hello Erel
    I created a small example db to populate SearchView. I restored the setItems function to populate the SV.

    I ask you a kind help to optimize the use of the new object with the Db.

    Thanks so much
     

    Attached Files:

Loading...
  1. This site uses cookies to help personalise content, tailor your experience and to keep you logged in if you register.
    By continuing to use this site, you are consenting to our use of cookies.
    Dismiss Notice