Figuring out an old "dos style" database table

MarrowZero

MarrowZero

Member
Licensed User
Hello peoples,

I've got a small application that stores the application data in some files, they seem to be in a group of three (I assume DAT is DATA, NDX some INDEX, and DIA? No idea) This type of format predates my computer experince, so I wonder if there's any older (cough; I mean "experienced") people out there that could help identify the format and most importantly how I might use B4J to read and hopefully write to the file structure. I've attached a sample of the files (zipped).


1623352856229.png



The best idea I have so far is from analsying the program executable with TRID (https://mark0.net/soft-trid-e.html) RESULTS:

83.3% (.EXE) Win32 Executable Borland Delphi 5 (451463/56/28)
7.9% (.EXE) InstallShield setup (43053/19/16)
2.6% (.EXE) Win32 Executable Delphi generic (14182/79/4)
1.9% (.EXE) Win64 Executable (generic) (10523/12/4)
1.8% (.EXE) DOS Borland compiled Executable (generic) (10000/1/2)


Thanks for any help.

(I've been reading the forum and playing with B4J for about a year.... Loving it as a replacment for VB6 and MS Access! Thanks Erel)

--Paul
 

Attachments

I doubt very much that you'll be able to read these files. Just because a file extension ends in .DIA or .DAT does not mean that they are created and saved in the same format all the time.

Let's take your .DIA extension. That file could be saved as good old fashioned plain text format or has been compressed (in multiple ways), encrypted (in multiple ways) or dare I say it all of the above in that one file, or none of the above which I doubt very much.

I personally believe that you're going to run into huge difficulties figuring out the file formats above, let alone reading the data out of them, but I also could be wrong about that.

What is the name of the program that these files belong too?
 
I doubt it'll help anything, but here's how the Linux command file sees the files:
Code: 
> file *
oldfilessample.zip: Zip archive data, at least v2.0 to extract
S_RESO.DAT:         DOS executable (block device driverC)
S_RESO.DIA:         data
S_RESO.NDX:         DOS executable (block device driver \307 )
 
Did you try to open it with Notepad++ (to see if it's cleartext or binary). If it is cleartex, you could open and reformat it (at least).
 
They are not, unfortunately dBase files which is the first DOS database that comes to mind.

The important file is the .DAT file that looks like it is a database record file with a 0x166 byte header and subsequent 0x143 byte long records that appear to be road repair items (including Dumper trucks!) The item description is a string but the remainder of the record seems to be binary. It you have the application that creates these files, or can read them, perhaps we can get an idea of the fields which are in each record and reverse engineer some of the data.

The .NDX file is probably a structured index for for rapidly accessing the database records. It's contents will be relevant only to the operation of the original application and will contain nothing useful.

I can't guess what the .DIA file is but it will also not be relevant to accessing the database data.

EDIT: Added the hex prefix to the byte contents. There is also an 0x120 byte trailing record that appears to be a duplicate of the preceding full size record which also appears to be a duplicate so the trailing records may be left over garbage.
 
Last edited:
Each record seems to start (or the previous record ends) with a byte value of 0x4F (letter O) following by a byte count for the item identification string followed by an ANSII encoded string. It looks to me like each field is probably a fixed length field with string fields having a length indicator prefix. I can't find anything in those files which looks like a field definition table so maybe that knowledge is hard coded into the application. Knowing what the fields are and having some sample record contents should make it possible to recreate the record layout and read the data.
 
Last edited:
can be also a custom made "database"... ..actually will be custom made "table" (those days every table was different file...)

remember Open "file.dat" For Random As #num Len = Len(TYPE)
that will more difficult to be read...
as you must know the TYPE and also all the possible variables... that may be string,byte,currency,integer,long,double...array with any type..

Also ndx and dia could be then Index files for better and faster search results... you never know how a custom "maker" programmer thinks ?
 
can files be damaged and therefore any identification attempt would be useless?
 
I have to say that I opened files...

.dat file seems to be the data (table and not damaged)
.dia file seems to be just like the number of latest record and if file .dat is using
.ndx file seems to be index of records...

I think that if you open with notepad ++ the .dat file ... count the bytes (every character) before the ...180 Excavatorer panelne)quipment... (this is a field of a record in string)
or better if you have a screenshot of program or a printed page with known records....

seperate the records when the same string of counted characters...
count the characters (bytes) if for example are 4 ---> what is 4-byte numbers ?? integer,long
...if 8-byte... what can be ?
nulls can be just strings...

You must make tries...

or if you have the old dos app ----> run it into DOSBOX.... somewhere will be export (as ascii) or print the data will be easier to ocr... :)
 
Last edited:
Start from seperating the records.... will be a nice division... every record will be the same size :) ... The strings seems to be showed without decryption... so is just numbers the prob...
 
Last edited:
Thankyou for all your comments, here are some answers below, sorry i could not respond yesterday work was a crazy day.

What is the name of the program that these files belong too?
Click to expand...
@Peter Simpson
I don't think I can say about the program or give full access to the data as there is commerically sensitive data.
The purpose of this discovery is to figure out how this program should be replaced, I'm keen to see a new program build in B4J.
I was inititally wondering what data was in those files, what structure, how do these older dos/binary file work and how they could be read with B4J (personal techincal interest)

It you have the application that creates these files, or can read them, perhaps we can get an idea of the fields which are in each record and reverse engineer some of the data.
Click to expand...
@agraham Yes I have the application and can make it run. It seems you are excited by the dumper trucks, so I thought you might like this screenshot

1623489918999.png



They are mostly likey DBase files (DOS), I suspect dbase (I-III). / Foxpro
Click to expand...
@Jmu5667 I was wondering if using "Borland Delphi 5" then what would be the goto method for storing data? Would someone writing in Delphi use dBase or Foxpro ?

actually will be custom made "table" (those days every table was different file...)
Click to expand...
@Magma oh no! This is going to be hard work right?!?! (could be fun though?) Here are some Excavators for you :)

1623490617835.png



Would anybody be kind enough to offer an example of how the DAT file could be read byte for byte with B4J ?

Thanks to all of you!
--Paul
 
I found two methods (by searching forum) which allow the file to read - Now I just need to figure out the structure


Code: 
Private Sub Button1_Click
    
    Dim b() As Byte = Bit.InputStreamToBytes(File.OpenInput(File.DirAssets, "R_RESO.DAT"))
    Log(BytesToString(b, 16508,32, "utf8")) 'Read 15 bytes starting from 10
    ' RESULT     180 Excavatorer panelne)quipment
    
End Sub


Private Sub Button2_Click
    
    Dim b() As Byte = File.ReadBytes(File.DirAssets, "R_RESO.DAT")
    Log(BytesToString(b, 16508,32, "utf8")) 'Read 15 bytes starting from 10
    ' RESULT     180 Excavatorer panelne)quipment
    
End Sub
 
I've had some fun and hand-picked the locations to recreate the first two columns of the PLANT/EXCAVATORS results.


Code: 
Private Sub Button1_Click
    
    Dim CS As String = "ASCII"
    Dim b() As Byte = Bit.InputStreamToBytes(File.OpenInput(File.DirAssets, "R_RESO.DAT"))
    
    Log("=1=============")
    'Log(BytesToString(b, 16506,1, CS)) '4F         "O"
    Log(BytesToString(b, 16508,13, CS)) '            "180 Excavator"   
    Log(BytesToString(b, 16590,4, CS)) '             "Hour"
    Log("=2============")
    'Log(BytesToString(b, 597260,1, CS)) '4F         "O"
    Log(BytesToString(b, 597262,23, CS)) '            "360 Excavator (8 Tonne)"
    Log(BytesToString(b, 597344,4, CS)) '             "Hour"
    Log("=3============")
    'Log(BytesToString(b, 2941,1, CS)) '4F             "O"
    Log(BytesToString(b, 2942,24, CS)) '            "360 Excavator (13 Tonne)"
    Log(BytesToString(b, 3024,4, CS)) '             "Hour"
    Log("=4============")
    'Log(BytesToString(b, 10692,1, CS)) '4F         "O"
    Log(BytesToString(b, 10694,24, CS)) '            "360 Excavator (20 Tonne)"
    Log(BytesToString(b, 10776,4, CS)) '             "Hour"
    Log("=5============")
    'Log(BytesToString(b, 192541,1, CS)) '4F         "O"
    Log(BytesToString(b, 192543,24, CS)) '            "360 Excavator (40 Tonne)"
    Log(BytesToString(b, 192625,4, CS)) '             "Hour"
    Log("=6============")
    'Log(BytesToString(b, 573681,1, CS)) '4F         "O"
    Log(BytesToString(b, 573683,33, CS)) '            "360 Excavator (45 Tonne) Operated"
    Log(BytesToString(b, 573765,4, CS)) '             "Hour"
    Log("=7============")
    'Log(BytesToString(b, 574004,1, CS)) '4F         "O"
    Log(BytesToString(b, 574006,26, CS)) '            "Transport costs (45 Tonne)"
    Log(BytesToString(b, 574088,4, CS)) '             "Item"
    Log("=8============")
    'Log(BytesToString(b, 442543,1, CS)) '4F         "O"
    Log(BytesToString(b, 442545,24, CS)) '            "Mini Excavator (3 Tonne)"
    Log(BytesToString(b, 442627,4, CS)) '             "Item"
    Log("=9============")
    'Log(BytesToString(b, 54620,1, CS)) '4F         "O"
    Log(BytesToString(b, 54622,8, CS)) '            "Tractair"
    Log(BytesToString(b, 54704,4, CS)) '             "Hour
    Log("=10===========")
    'Log(BytesToString(b, 74969,1, CS)) '4F         "O"
    Log(BytesToString(b, 74971,14, CS)) '            "Loading shovel"
    Log(BytesToString(b, 75053,4, CS)) '             "Hour"
    Log("=11===========")
    'Log(BytesToString(b, 75292,1, CS)) '4F         "O"
    Log(BytesToString(b, 75294,15, CS)) '            "Transport costs"
    Log(BytesToString(b, 75376,4, CS)) '             "Trip"
    Log("=12===========")
    'Log(BytesToString(b, 75615,1, CS)) '4F         "O"
    Log(BytesToString(b, 75617,10, CS)) '            "Dump truck"
    Log(BytesToString(b, 75699,4, CS)) '             "Hour"
    
End Sub
 
Obviously you'll have checked the program has not any menu to export data as cvs, text or similar...
 
José J. Aguilar said:
Obviously you'll have checked the program has not any menu to export data as cvs, text or similar...
Click to expand...
Indeed I have looked for export functions, sadly there is nothing useful. Even printing and OCR might not be good as those printed reports are for the summary of infomation, not the table data.
 
You must log in or register to reply here.
Back
Top