Android Question Read only last line in a file

pazzokli

Active Member
Licensed User
Longtime User
Hi forum,
I' ve a big text file with a lot of line. I want read only last line or last x lines without read all the lines.
This file store date and room temperature every 10 minutes and add a line each time. I want display a graph of some hours only, but problem is that I don't know how many lines there are in this file and I implemented a cicle to read all the line and use last 144 (for show 24h) only.
This take a lot of time.
There is a fastest way?

Thanks
 

Mahares

Expert
Licensed User
Longtime User
I want read only last line or last x lines without read all the lines
There is a fastest way?
If you are comfortable with SQLite, it is the way to go instead a large text file. You can import the text file into a table.
Suppose you have a table named: tblTemp with 2 cols: TEMP_DATE And TEMP, you can sort the table by any column in any direction and extract any number of records. Example to extract the most recent 144 records:
B4X:
Dim strQuery As String="SELECT * FROM tblTemp ORDER BY TEMP_DATE DESC LIMIT 144"
cursor=Starter.sql1.ExecQuery(strQuery)
 
Upvote 0

mangojack

Well-Known Member
Licensed User
Longtime User
If your not up with SQLite .. you could experiment with File.ReadList method .. writing your text file to a list first for easier interaction.

this is only basic hint code .. unsure how and what your writing to the text file. * You would have to include some checks for list size etc.
B4X:
Dim List1 As List
    List1 = File.ReadList(File.DirDefaultExternal, "Text.txt")    
    Log($"List size =  ${List1.Size}"$)
    Log($"This is the last item  ...  ${List1.Get(List1.Size -1}"$))
   
    For i = List1.Size - 144 To List1.Size -1
        Log($"This is item #   ${List1.Get(i}"$))   
    Next

But Personally I would be following @Mahares suggestion above and go the SQLite Database approach.
 
Upvote 0

pazzokli

Active Member
Licensed User
Longtime User
I don't know database and SQL. I used access on vb.net only with simple interaction. This is why I store in a text file.
But database is the best solution. I know...
I will try the second solution
 
Upvote 0

Sandman

Expert
Licensed User
Longtime User
Using a database for this seems like a huge overkill to me. Just run a shell command:

B4X:
tail -n 5 path/to/file

Where you obviously change the path to be correct, and change the 5 to the number of lines you want from the end.

Sorry, can't give better code example now, writing this using mobile, on the go. Search forum for how to run shell commands and get result back.
 
Upvote 0

Sandman

Expert
Licensed User
Longtime User
I'm not going to argue with you about this, you most likely know this a lot better than me. But just to explain my reasoning...

missing commands
That might be correct as a general rule, but tail is such a standard command that I assumed it to always be present.

It will also be slower than reading the file directly from the app.
This surprised me. I would have imagined that tail would be a lot faster considering it's specialized for the job of getting lines from the end, especially for very large files.

permission problems
Yeah, I didn't think of that. :)
 
Upvote 0

Erel

B4X founder
Staff member
Licensed User
Longtime User
would have imagined that tail would be a lot faster
Remember that the OS needs to start a new process for this. An efficient implementation inside your app will be faster. Though this is not a real issue if you are reading the file every 10 minutes.

Android <> Linux. Accessing the underlying shell commands is not something that should normally be done.
 
Upvote 0

MarkusR

Well-Known Member
Licensed User
Longtime User
just save the last 144 rows (for the last 24h) in a separated file from your .net app.
u can use the same file name or for each day the date.
in .net you can store the data in a list.
 
Upvote 0

pazzokli

Active Member
Licensed User
Longtime User
just save the last 144 rows (for the last 24h) in a separated file from your .net app.
u can use the same file name or for each day the date.
in .net you can store the data in a list.
I Made the sample of 24h but I could choose 3 days or a week so number of lines is not a fix parameter.
 
Upvote 0

mangojack

Well-Known Member
Licensed User
Longtime User
While using SQL db for this might be slight overkill ... it just opens so many possible interactions with your data on a time / date basis.

Trust me .. compared with VB / .Net ... using a SQL database within B4x is an absolute breeze.

Chapter 5 / Page22 of the user guide has an excellent write up to get you started ... and there are many excellant startup tutorials within the forum.

B4A Users Guide ....
 
Upvote 0

klaus

Expert
Licensed User
Longtime User
Chapter 5 / Page22 of the user guide has an excellent write up to get you started
The User's Guide is a bit outdated.
You may have a look at the B4X SQLite Database Booklet.
It has examples with source code.
 
Upvote 0

udg

Expert
Licensed User
Longtime User
Another way could be to open the file and skip the first "x" records.
On the assumption that each row is of equal length, you could adapt the attached code where a couple of alternatives are showed. You should time each one against your real data file in order to understand which one is faster and less resource consuming. Along with the already suggested solutions.

Happy new year!

@Erel : I think that the addition of the "skip" method to the standard File I/O methods could be a nice one; I'm thinking about chunk-reading operations..
 

Attachments

  • FBR.zip
    8.7 KB · Views: 166
Upvote 0

udg

Expert
Licensed User
Longtime User
Hi Erel, I must admit that at first I didn't fully understand your comment.
Yesterday I wrote a quick and dirty hexeditor in B4J and the light shed on me again.
So, InputStream is OK for one-way only reading (and skip could be of some help like in the OP), while RAF is a block-oriented, rich featurs mechanism although "heavier" than the former (FileChannel?).
 
Upvote 0
Top