serious number crunching

cpc6128

Member
Licensed User
I was wondering whether someone could take a look at my code and give me some points as to how I could optimise it? It is fairly slow when it runs on my desktop so I haven't even bothered running it on my PPC yet!

Basically the program takes some initial lat/long data from a GPS (but for the purposes of testing, from 2 text boxes on the form) and compares this to a list of UK postcodes and their lat/longs read from the data file. It then works out which postcode is closest to the initial lat/long data.

I have only included a very small part of the data file as an example as I do not think I am allowed to distribute the full data set.

I am reading the text file a line at a time. Firstly, I couldn't see a way in the help to read part of a line (as in a comma delimited file) so my program has to iterate through each character in the line to check for commas. I'm sure there is a much more efficient way of doing this!

Secondly, I kept getting errors when I tried to peform maths functions on the data I read in. Ideally I need the numeric data to be stored as a Double, but since there are no functions to convert data types (that I could see) I think my program was treating the data as a String which was causing me errors. To overcome this I have used the Decimal library throughout my code which I'm sure is horribly inefficient given the number of calculations involved.

The program works as-is but as I said is incredibly slow... any help in speeding it up would be much appreciated!
 

Attachments

LineCutter

Active Member
Licensed User
Sort the file by Lat
Break it up into convenient sized chunks (index the range of lat in each file somehow)
[This saves memory space for the PPC version]

Read the Lat (& long) from your textboxes
Load the correct file (as a whole) into a database structure (table or SQL)
Use a binary search routine to home in on the correct Lat
Read "up" & "down" to find the limits of that Lat in your database
Search these to find the Long (you could sort by long, within each lat value when you create the data files,& do a binary search through these if there are lots)
Read the postcode.
 

Leginus

Member
Licensed User
I agree with linecutter that I think the speed problem comes from reading the file line by line and not the number crunching. Processing from memory is much quicker
 

cpc6128

Member
Licensed User
I think the main point of linecutters post was that I could speed up the program by making the search algorithm more efficient e.g. by pre_sorting some of the data so I don't have to search the entire file every time.
However I think you also have a very good point about file access slowing things down and I agree that loading the data into memory first is sensible. I will implement both ideas, hopefully together they will make a big difference. thanks!
 
Top