Android Question Read only last line in a file

[pazzokli]

Hi forum,
I' ve a big text file with a lot of line. I want read only last line or last x lines without read all the lines.
This file store date and room temperature every 10 minutes and add a line each time. I want display a graph of some hours only, but problem is that I don't know how many lines there are in this file and I implemented a cicle to read all the line and use last 144 (for show 24h) only.
This take a lot of time.
There is a fastest way?

Thanks

 

[Mahares]

I want read only last line or last x lines without read all the lines

There is a fastest way?

If you are comfortable with SQLite, it is the way to go instead a large text file. You can import the text file into a table.
Suppose you have a table named: tblTemp with 2 cols: TEMP_DATE And TEMP, you can sort the table by any column in any direction and extract any number of records. Example to extract the most recent 144 records:

Dim strQuery As String="SELECT * FROM tblTemp ORDER BY TEMP_DATE DESC LIMIT 144"
cursor=Starter.sql1.ExecQuery(strQuery)

 

[mangojack]

If your not up with SQLite .. you could experiment with File.ReadList method .. writing your text file to a list first for easier interaction.

this is only basic hint code .. unsure how and what your writing to the text file. * You would have to include some checks for list size etc.

Dim List1 As List
    List1 = File.ReadList(File.DirDefaultExternal, "Text.txt")    
    Log($"List size =  ${List1.Size}"$)
    Log($"This is the last item  ...  ${List1.Get(List1.Size -1}"$))
   
    For i = List1.Size - 144 To List1.Size -1
        Log($"This is item #   ${List1.Get(i}"$))   
    Next

But Personally I would be following @Mahares suggestion above and go the SQLite Database approach.

 

[pazzokli]

I don't know database and SQL. I used access on vb.net only with simple interaction. This is why I store in a text file.
But database is the best solution. I know...
I will try the second solution

 

[Erel]

Tip: You should never use File.DirDefaultExternal.
See the runtime permissions tutorial.

 

[Sandman]

Using a database for this seems like a huge overkill to me. Just run a shell command:

tail -n 5 path/to/file

Where you obviously change the path to be correct, and change the 5 to the number of lines you want from the end.

Sorry, can't give better code example now, writing this using mobile, on the go. Search forum for how to run shell commands and get result back.

 

[Erel]

Shell commands in an Android app?! Much better to use a database or just read the whole file with File.ReadList assuming that it is not too large.

 

[Sandman]

Shell commands in an Android app?!

Sure, why not? Is that so bad?

assuming that it is not too large

The poster said it was a big file, so I assumed it was big.

 

[Erel]

Sure, why not? Is that so bad?

Shell commands can cause all kinds of problems including permission problems, missing commands and others. It will also be slower than reading the file directly from the app.

 

[Sandman]

I'm not going to argue with you about this, you most likely know this a lot better than me. But just to explain my reasoning...

missing commands

That might be correct as a general rule, but tail is such a standard command that I assumed it to always be present.

It will also be slower than reading the file directly from the app.

This surprised me. I would have imagined that tail would be a lot faster considering it's specialized for the job of getting lines from the end, especially for very large files.

permission problems

Yeah, I didn't think of that.

 

[Erel]

would have imagined that tail would be a lot faster

Remember that the OS needs to start a new process for this. An efficient implementation inside your app will be faster. Though this is not a real issue if you are reading the file every 10 minutes.

Android <> Linux. Accessing the underlying shell commands is not something that should normally be done.

 

[MarkusR]

just save the last 144 rows (for the last 24h) in a separated file from your .net app.
u can use the same file name or for each day the date.
in .net you can store the data in a list.

 

[Sandman]

Accessing the underlying shell commands is not something that should normally be done.

Well, ok then.

 

[pazzokli]

just save the last 144 rows (for the last 24h) in a separated file from your .net app.
u can use the same file name or for each day the date.
in .net you can store the data in a list.

I Made the sample of 24h but I could choose 3 days or a week so number of lines is not a fix parameter.

 

[mangojack]

While using SQL db for this might be slight overkill ... it just opens so many possible interactions with your data on a time / date basis.

Trust me .. compared with VB / .Net ... using a SQL database within B4x is an absolute breeze.

Chapter 5 / Page22 of the user guide has an excellent write up to get you started ... and there are many excellant startup tutorials within the forum.

B4A Users Guide ....

 

[klaus]

Chapter 5 / Page22 of the user guide has an excellent write up to get you started

The User's Guide is a bit outdated.
You may have a look at the B4X SQLite Database Booklet.
It has examples with source code.

 

[udg]

Another way could be to open the file and skip the first "x" records.
On the assumption that each row is of equal length, you could adapt the attached code where a couple of alternatives are showed. You should time each one against your real data file in order to understand which one is faster and less resource consuming. Along with the already suggested solutions.

Happy new year!

@Erel : I think that the addition of the "skip" method to the standard File I/O methods could be a nice one; I'm thinking about chunk-reading operations..

 

[Erel]

You can use RandomAccessFile to do it. Though it will be an overkill for a file with a few thousand lines.

 

[udg]

Hi Erel, I must admit that at first I didn't fully understand your comment.
Yesterday I wrote a quick and dirty hexeditor in B4J and the light shed on me again.
So, InputStream is OK for one-way only reading (and skip could be of some help like in the OP), while RAF is a block-oriented, rich featurs mechanism although "heavier" than the former (FileChannel?).

 

[Erel]

So, InputStream is OK for one-way only reading (and skip could be of some help like in the OP), while RAF is a block-oriented, rich featurs mechanism although "heavier" than the former (FileChannel?).

That's true.