Big JSON Data

[mdp]

hello
how to parse big json data?
i have to download around 40000 record via HTTP with JSON format and insert it to SQL Lite database on my Android. for around 2000 - 3000 record is ok, but above that just fail
thanks in advance

 

[Erel]

Where does the data come from?

 

[mdp]

from my Apache server which get is from the MySQL server

 

[Erel]

You can use LIMIT and OFFSET in your query to get the data in several parts (several queries).

 

[mdp]

so the basic idea is split the upcoming data and insert the data chunk by chunk
not as 1 big data? correct?

 

[Erel]

Yes.

 

[mdp]

ok sir
it seem that the http service on the android can get the big data, but cannot assign it to a string, since the size of the cache as i debug is around 9.89MB
the next question is : how to process the json data via stream not as string?
this is the method that return the data from your HttpUtils :

'Get methods should be called only after the JobDone event or the UrlDone event.
Sub GetString(URL As String) As String
If IsSuccess(URL) = False Then
Log("Task not completed successfully.")
Return ""
End If
Return File.GetText(HttpUtilsService.TempFolder, SuccessfulUrls.Get(URL)) 'it return as string, can i return it as stream and process the json data from it?
End Sub

 

[Erel]

The JSON library works with strings not streams.

 

[mdp]

ok sir
already done
i split the data by 5000 record, still doable with string in my android :icon_clap: