Best way to move large data from xml to sqlite db?

[netchicken]

I have to download large xml data files then parse them and add them to an sqlite database.

I have it working OK, but there doesn't seem to be a way to use async so it runs in the background, nor can I find a way to trigger a notification when the file has finished loading into the db (so I could run progreesdialoghide) or to let it load in the background.

So the program looks like its hanging while the file loads up, which can take some minutes. I suppose I could use a doevents to see an output, but thats going to make it go slower.

Any ideas?

   Sub LoadNewData
   'load the datafile into the database
   'parse the xml file
   Dim In As InputStream
   If File.Exists(File.DirAssets, "aaa.xml") Then
   ToastMessageShow("Finished Downloading now installing ...", True)
   
   In = File.OpenInput(File.DirAssets, "aaa.xml")
   parser.Parse(In, "Parser")
   'Log(parser)
   In.Close
   ProgressDialogShow("Working ...")   
   
   Else
   ToastMessageShow("File not found",True) 
   End If   
   
End Sub

Sub Parser_StartElement (Uri As String, Name As String, Attributes As Attributes)
'pull out the xml fields. These are inside of the < > tags which makes them Attributes and pulled out at the start element section becuase, well, they are in the start element.

'NOTE THAT THE NAMES ARE CASE SENSITIVE (caused me lots of issues until I realised it)
If Name = "Platform" Then
     'data is passed to the fields in the type named thisbusstop (type is a primitive Class)
    
        ThisBusStop.PlatformTag = Attributes.GetValue2("", "PlatformTag")
        ThisBusStop.PlatformNo = Attributes.GetValue2("", "PlatformNo")   
        ThisBusStop.PlatformName  = Attributes.GetValue2("", "Name")
      ThisBusStop.BearingToRoad = Attributes.GetValue2("", "BearingToRoad")
         ThisBusStop.RoadName = Attributes.GetValue2("", "RoadName")
End If

If Name = "Position" Then
      ThisBusStop.Lat = Attributes.GetValue2("", "Lat")
      ThisBusStop.Lon = Attributes.GetValue2("", "Long")
End If 

If  ThisBusStop.Lat <0 Then 'get data if there is any
'insert the data into the database
'crashes if there are ' in the name O'Brians
   ThisBusStop.RoadName =    ThisBusStop.RoadName.Replace("'", "")
ThisBusStop.PlatformName =    ThisBusStop.PlatformName.Replace("'", "")

Main.SQLmetroInfo.ExecNonQuery("INSERT INTO platforms VALUES('" & ThisBusStop.PlatformTag & "','" & ThisBusStop.PlatformNo & "','" &  ThisBusStop.PlatformName & "','" & ThisBusStop.RoadName & "','"& ThisBusStop.Lat &"', '" & ThisBusStop.Lon & "')")

Log(ThisBusStop.PlatformTag & " Road " & ThisBusStop.RoadName &" " & " Platname " & ThisBusStop.PlatformName )'&" " & " number " & ThisBusStop.PlatformNo)
End If

End Sub

 

[NJDude]

You could do the parsing in a service and when done trigger a notification.

 

[agraham]

You could do the parsing in a service and when done trigger a notification.

Services run on the main thread so it will still hang the application.

Service | Android Developers

Note that services, like other application objects, run in the main thread of their hosting process. This means that, if your service is going to do any CPU intensive (such as MP3 playback) or blocking (such as networking) operations, it should spawn its own thread in which to do that work

 

[Erel]

I think that your best option is to show a progress dialog and call DoEvents from one of the parser subs. You do not need to call DoEvents every time.

 

[netchicken]

Thanks Guys I will follow those ideas up