Android Question How to handle big data for use by an App?

[cyiwin]

I want to make an App that gives crypto-currency traders useful data. I will use this info page http://pubapi.cryptsy.com/api.php?method=marketdatav2. The problem is the info on this page is huge! I think it is about 2 MB of characters. Is it correct that 1,835,165 characters require almost 2 MB? To keep the info up to date, I want to download it every minute or 2, but that would kill the clients data usage plan.

This is a new area for me so I'm hoping someone can start me in the right direction. I would like to stay in the Basic4Android world as much as possible.

My idea...
The only thing I can think of is to modify the info before the B4A App downloads it. I would have to download this page to my hosted site, make the page smaller by getting rid of the information that I don't need (I don't know what software does this), have the client download the smaller version and the App will do the math.

Any advise on how to do this or a better way to do it will be much appreciated!

 

[thedesolatesoul]

Looks like an API server. Can you ask them for a different API method that only returns the data you need?
Do you need the full page? If you do, then you have no alternative.
2MB is not a lot if you are downloading once every few hours. But for every minute it means data will be invalid by the time it is downloaded.
Otherwise as you said, maybe download it to your server and parse it with a php script or something like that.

 

[thedesolatesoul]

See here for more info: https://www.cryptsy.com/pages/api

 

[cyiwin]

Thanks for the info!

So it looks like I will have to learn another programming language to convert the huge string into a smaller, more useful string on my own web site. Is php the recommended language to use for this kind of project?

Edit: Is php or something else good at sorting though text for key words and pulling out those key words and surrounding text?

 

[thedesolatesoul]

What do you want to convert though? What kind of information can be removed?

I'm no webdeveloper but most people use PHP. But if your webhost supports it maybe you can use B4J (no need to learn a new language!).

 

[cyiwin]

B4J looks interesting, I'll have to take a look at that.

I need to take the info from here http://pubapi.cryptsy.com/api.php?method=marketdatav2. Remove the data that I don't need and repost it on my web site.

Even if I were to do this, my website would be downloading the info at the rate of 1 MB/minute!

I think I'm getting way over my head with this one. I'm starting to see instant info apps like this may not be possible. The constant downloading of even text uses too much data.