Android Question Remote/local db syncing, best practice

james_sgp

Active Member
Licensed User
Longtime User
Hi, im working on a large scale app (20,000+ users) for a client which is going to have a fair amount of data in a remote DB.

I'm getting a mind block on how to efficiently get the new entries from the remote DB, to insert into the local db.

I was initally just accessing data remotely, but locations with poor internet are causing the app to crash/be unresponsive. So want to move towards moving some of the data locally.

Clients server doeant have VPS so im running thro PHP, which is all working well so far.

Tips and suggestions, would be greatly appreciated.

Thanks, James
 

walterf25

Expert
Licensed User
Longtime User
Upvote 0

james_sgp

Active Member
Licensed User
Longtime User
Would be worth taking a look at this post
Thread 'Syncing local databases using PHP to access a database on a hosted server' https://www.b4x.com/android/forum/t...-access-a-database-on-a-hosted-server.142126/
Yeah, this is my current stratergy:

Routine in 'Starter'

> get all records with timestamp greater than 'last sync' (KVS field)
> if record exists, update it
> if record doesnt exist, insert it
> if all sucessful, put above timestamp into 'last sync'

Call the routine at different intervals, depending on table data importance
 
Upvote 0

emexes

Expert
Licensed User
How large is the remote database? How frequently does it change? Is it time-critical, or is an hourly or daily update good enough?
 
Upvote 0

james_sgp

Active Member
Licensed User
Longtime User
No deletions?
Im only downloading the datavthat doeant change much, like ads data (can change hourly), products for sale, etc. Other data like user profile i'm putting in a B4XOrderedmap, so its easy/fast to access it.

Mmmm, deletions...there shouldn't be any but will build in the ability jiat on case.

The whole database could be many 10's of Mb, as its also recording tracking information of 10,000+ users on 30 second intervals when they are inside a building.
 
Upvote 0

emexes

Expert
Licensed User
recording tracking information of 10,000+ users on 30 second intervals

That sounds entirely achievable with data held in memory and using UDP packets. šŸ»

You might need to keep an early and close eye on throughput capacity, rather than risk ending up with a beautiful system that works great with 100 simulated users but dies when faced with 10,000.
 
Upvote 0

Erel

B4X founder
Staff member
Licensed User
Longtime User
Upvote 0
Top