Format to read in CSV files

[Stellaferox]

Hi,

Does anyone have a routine that reads CSV files of certain length? Moreover, I 'd like to know what the fastest way is to read these files. Do you have to use SQL?
thnx
Marc

 

[RandomCoder]

The easiest way is to load the file into a table.
From there you can extract the data you want.
You can also use SQL which is faster for large databases, however I'm yet to learn how to use SQL (maybe some time soon I'll give it a try).

Regards,
RandomCoder

 

[Stellaferox]

thnx,

I was wondering whether anyone had a routine which regardless of the parameters (which are read at runtime and passed through) could read a csv file. I am not too familiar with SQL myself, so I hope someone is.....
Marc

 

[RandomCoder]

You don't need to understand SQL to use the table control.
The table control is very simple to use and will load the CSV file directly into the table cells.
All you then need to do is either filter the table or just extract the desired cells.

Regards,
RandomCoder

 

[Stellaferox]

Well, I dont need a table actually. i want to store over a few million data in the CSV file to retrieve with a search function matching one or two entries.

 

[RandomCoder]

Well, I dont need a table actually. i want to store over a few million data in the CSV file to retrieve with a search function matching one or two entries.

A few million :sign0137:

You would be best to use SQL, although my understanding is that a table will still be required for SQL to utilise.
My approach is usually to try and understand a few examples of other peoples work and then tailor it to my own needs.
Here is a link to an example that Erel did some time ago, maybe it will help you... http://www.b4x.com/forum/showthread.php?t=843&highlight=SQLite

Regards,
RandomCoder

 

[Stellaferox]

hmmmmmm.... missed this little gem, though I thought I skimmed the forum daily.
Thnx!
Marc

 

[Erel]

What type of data do you need to store?

 

[Stellaferox]

@ Erel

Hi erel,

I want to store a table containing numbers. The program computes the rank of a hand in poker with 7 cards, taken the best 5 out of it (Texas Holdem). Because there are 133784560 combinations with 7 cards out of 52, it takes too long to compute the hands, especially because I want to run simulations with dealing a lot of hands (Monte Carlo simulation) and compute the probability of winning with that certain hand against other random AI-hands. Therefore I want to generate a table and compare the hand to an index (with a unique longinteger) and retrieve the value of the hand.
Marc

 

[Erel]

I recommend you to use the BinaryFile to store the data in a raw file.
Write with WriteInt64 and read with ReadInt64.
Every number occupies 8 bytes so it is very easy to navigate the file.

 

[Stellaferox]

Thanks Erel, I will do so.
Marc

 

[LineCutter]

I want to store a table containing numbers. The program computes the rank of a hand in poker with 7 cards, taken the best 5 out of it (Texas Holdem). Because there are 133784560 combinations with 7 cards out of 52, it takes too long to compute the hands, especially because I want to run simulations with dealing a lot of hands (Monte Carlo simulation) and compute the probability of winning with that certain hand against other random AI-hands.

You're going to have fun sorting out which hands are valid after you drew the first cards & the subsequent interactions that get you to 7 cards drawn per hand...

 

[Stellaferox]

Yeah I know. I worked out to check all 21 combinations of a 5-hand from 7 cards. That works faster then evaluating a 7-hand card I think.

 

[LineCutter]

I might have the wrong end of the stick - are you simply calculating the probable outcomes for a hand played all the way through, or are you including the role of the evolving shared cards in determining whether you play to the end?

 

[Stellaferox]

The first part of the program is to caculate the outplayed hand (7 cards). With that I can simulate dealing let's say 200000 hands and calcuate probabilities and distribution. Then for the second part I want to calculate probabilities to reach a certain hand with the ongoing play after dealing the several rounds.
The idea is whether I can create a table with pointers after each card to a next table with remaining possibilities. I haven't got it quite clear yet, but the first idea is to build the "mothertable" of all 133784560 combinations.
Marc

 

[Stellaferox]

But can I get back to the original question? What is the general format/ routine to read in CSV files. There are a number of CSV files on the net that I can use to train my program. After that I can write my own table in BinaryFile.
thnx
Marc

 

[agraham]

What is the general format/ routine to read in CSV files

Open one with Wordpad or Notepad. They are just normal text files.

If FileExist ("text.csv") = true Then
  FileOpen (c1,"text.csv",cRead ,, cASCII)
  r = FileRead (c1)
  Do Until r = EOF
    fields() = StrSplit(r,",")
    Msgbox(fields(0) & " - " & fields (1)) ' do whatever with the fields
   r = FileRead (c1)
  Loop
FileClose (c1)

 

[Erel]

If you want to parse CSV files you should use Table.LoadCSV.

 

[s_ostap]

Do not show russian letters

I have a problem:

Table1.LoadCSV("Csv_file.csv",";",False,True)

File of "Csv_file.csv" contains information on Russian language. But the Russian letters are not represented in table cells.

 

[Bakhtiyor]

Open your csv in Notepad and save it in UTF-8. Russian letters should appear.