Hi All
I have added polar value interpolation to speedy spline.
This takes angles in the range -180 to +180 with an assumed magnitude of one, transforms these into their x (real) and y (imaginary) components interpolates the x vector values and y vector values separately then calculates the angle using atan2(y/x).
Doing the interpolation this way resolves the problem of interpolation where values straddle the circular boundary +/- 180 degrees.
i.e taking a small data input stream:
160, 170, 178, -178, 170, 160
Interpolated in the Cartesian domain this is approx:
160, 165, 170, 178, 0, 178, 170, 160
Note the 0 value which wouldn't make much sense if this were wind direction.
Whereas in the polar domain this is approx:
150, 165, 170, 178, 178, 170, 160.
To use it for compas values simply subtract 180 from the compass value, interpolate it and then add 180 to the interpolated vale to get it back to compass values.
The screenshots below show the difference in the calcs for compass reading that straddle the 0/360 degree boundary.
You can see in the first one using just the vector of angles (not polar) the interpolation ends up a bit of a mess with negative angles and angles greater than 360.
The second one below uses polar interpolation and does the job.
To use the polar calc the command to the server is:
3 => general interp
4 => 2 x interp
The data format remains the same: a vector of doubles being angles in the range +/- degrees. Usually the range would be limited to +/- 180, but it doesn't have to be. But remember for compass values subtract 180 from the input vales then add 180 to the result.
I have attached the updated version of speedy spline and the test program.
I have left the original version as not everyone would need a polar transform.
Best regards
Rob
Hi All,
I needed an interpolator to convert weather data stored at ten minute intervals to 5 minute intervals to match some power monitoring data I get from an sqlite db that aggregates
two second power data to five minutes. Five minutes is the trading interval of the Australian Electricity market.
I could have got sufficient accuracy with a linear interpolation as most of the weather variables are pretty slow with the exception of wind speed. However I always though that before I die I would like to develop an app using cubic splines. I plagerised some C code off the internet and converted it into B4J, very easy to do. This worked well except it was too slow for bulk data processing. I asked about how I could use a C++ dll called by B4J to implement a faster spline, but dll's are too hard and there is a better way; per MathiasM to whom I am indebted. MM suggested that I use the network local host to transfer data between B4J and a server written in C++ or some other language. Being a sick sort of person I went for C++.
The communications I am using is TCP over localhost and a selectable port no.
So I did all of this an viola I now have a speedySpline.exe mini server that takes a block of doubles and interpolates them according to the ratio you want. (There are even some stake knives for early adopters in the form of csv file output)
To talk to the server, B4J uses prefixed asynch transfers.
Basically the system works thus:
For larger data sets over 2000 double speedySpline is over 100 times faster than native B4J. For a 4000 set of doubles B4J takes 70 seconds, speedySpline takes 0.6 to 0.7 seconds.
For data-base crunching this is a very useful speed increase and it's relatively easy to do.
I have posted a zip of the splineTest program that compares B4J with speedySpline and also the source and project file for VS 2019 that I used to develop it.
A copy of the speedySpline.exe file has been included in the B4J app Objects folder. speedySpline is run from there. The original is in the visual studio project under the 64 Bit directory.
There are command line arguments that can be used when speedySpline is run to do the following:
You can run speedySpline from a command line console with "h" and the cmd line options will be output to the console.
The B4J test app is fairly straight forward so I won't bang on about it.
If anyone finds this useful and needs help getting it going just post a request.
I intend to provide spline and linear interpolation of polar data such as wind direction that will be commands 3 and 4 probably. Once youve done the code infratsructure and logic adding extra capability is a breeze.
The next project is to develop a fast random number generator which will produce accurate normal and other distributions.
I'm also thinking of large, multi channel FFT's with selectable window functions and bin averaging. The network interface is ideal for this, as data is nicely sized and consistent from one frame to the next. The FFTW library would be ideal for this.
Best regards
Rob
PS I forgot to write a sub for the communications timeout in ipcComms
It's now fixed!
I have added polar value interpolation to speedy spline.
This takes angles in the range -180 to +180 with an assumed magnitude of one, transforms these into their x (real) and y (imaginary) components interpolates the x vector values and y vector values separately then calculates the angle using atan2(y/x).
Doing the interpolation this way resolves the problem of interpolation where values straddle the circular boundary +/- 180 degrees.
i.e taking a small data input stream:
160, 170, 178, -178, 170, 160
Interpolated in the Cartesian domain this is approx:
160, 165, 170, 178, 0, 178, 170, 160
Note the 0 value which wouldn't make much sense if this were wind direction.
Whereas in the polar domain this is approx:
150, 165, 170, 178, 178, 170, 160.
To use it for compas values simply subtract 180 from the compass value, interpolate it and then add 180 to the interpolated vale to get it back to compass values.
The screenshots below show the difference in the calcs for compass reading that straddle the 0/360 degree boundary.
You can see in the first one using just the vector of angles (not polar) the interpolation ends up a bit of a mess with negative angles and angles greater than 360.
The second one below uses polar interpolation and does the job.
To use the polar calc the command to the server is:
3 => general interp
4 => 2 x interp
The data format remains the same: a vector of doubles being angles in the range +/- degrees. Usually the range would be limited to +/- 180, but it doesn't have to be. But remember for compass values subtract 180 from the input vales then add 180 to the result.
I have attached the updated version of speedy spline and the test program.
I have left the original version as not everyone would need a polar transform.
Best regards
Rob
Hi All,
I needed an interpolator to convert weather data stored at ten minute intervals to 5 minute intervals to match some power monitoring data I get from an sqlite db that aggregates
two second power data to five minutes. Five minutes is the trading interval of the Australian Electricity market.
I could have got sufficient accuracy with a linear interpolation as most of the weather variables are pretty slow with the exception of wind speed. However I always though that before I die I would like to develop an app using cubic splines. I plagerised some C code off the internet and converted it into B4J, very easy to do. This worked well except it was too slow for bulk data processing. I asked about how I could use a C++ dll called by B4J to implement a faster spline, but dll's are too hard and there is a better way; per MathiasM to whom I am indebted. MM suggested that I use the network local host to transfer data between B4J and a server written in C++ or some other language. Being a sick sort of person I went for C++.
The communications I am using is TCP over localhost and a selectable port no.
So I did all of this an viola I now have a speedySpline.exe mini server that takes a block of doubles and interpolates them according to the ratio you want. (There are even some stake knives for early adopters in the form of csv file output)
To talk to the server, B4J uses prefixed asynch transfers.
Basically the system works thus:
- B4J starts the speedySpline server using jShell
- B4J checks that the server is running.
- It then creates a socket to the server
- If all good then you can initiate a transaction using:
- - Sends a command to the server which has, command number 1 or 2, the number of values to be interpolated and the interpolation ratio.
- - The server echos this back to B4J
- - Using asynch_New Data B4J receives this, checks it and if all good sends the input doubles to be interpolated to the server
- - the server does the calc and sends the resulting interpolated frame of doubles back (all in the blink of an eye)
- - B4J capture the data and can then process or display it.
- When the B4J app is shut down the socket connection to the speedySpline server is lost and the speedySpline server process shuts down.
For larger data sets over 2000 double speedySpline is over 100 times faster than native B4J. For a 4000 set of doubles B4J takes 70 seconds, speedySpline takes 0.6 to 0.7 seconds.
For data-base crunching this is a very useful speed increase and it's relatively easy to do.
I have posted a zip of the splineTest program that compares B4J with speedySpline and also the source and project file for VS 2019 that I used to develop it.
A copy of the speedySpline.exe file has been included in the B4J app Objects folder. speedySpline is run from there. The original is in the visual studio project under the 64 Bit directory.
There are command line arguments that can be used when speedySpline is run to do the following:
- No parameters, the server defaults to port 27105
- 1 Par. The port number 0 to 99999 choose your poison so to speak!
- 7 Pars Port Number, y/n for input file write to csv, y/n for output file write to csv, input csv full path, output csv full path, precision 0-15 for the decimal places output to the csv files.
You can run speedySpline from a command line console with "h" and the cmd line options will be output to the console.
The B4J test app is fairly straight forward so I won't bang on about it.
If anyone finds this useful and needs help getting it going just post a request.
I intend to provide spline and linear interpolation of polar data such as wind direction that will be commands 3 and 4 probably. Once youve done the code infratsructure and logic adding extra capability is a breeze.
The next project is to develop a fast random number generator which will produce accurate normal and other distributions.
I'm also thinking of large, multi channel FFT's with selectable window functions and bin averaging. The network interface is ideal for this, as data is nicely sized and consistent from one frame to the next. The FFTW library would be ideal for this.
Best regards
Rob
PS I forgot to write a sub for the communications timeout in ipcComms
It's now fixed!
Last edited:
