Share My Creation jAudioTrack2 - Mixer Full

I have been looking for the best way to mix audio tracks with B4x for quite a while. Java basically provides a number of methods which I have tried. The audio Clip class allows skipping through the track, but it does not allow access to the raw byte data necessary to perform audio mixing. Then AudioInputStream (and related classes) do allow access to the raw byte so mixing can be performed, but it does not allow moving to any other positions in the track easily.

I came up with the idea of creating a WavRandomAccessFile class, which is basically a RandomAccessFile that is wav audio aware. It gets the all important AudioFormat from the file and parses the Wav header to find the start and end of the audio 'data' chunk. This is important as wav files can contain many different chunks, and the data chunk is not necessarily at the end of the file.

The WavRandomAccessFile class can be found in jAudioTrack2 v1.2+. There is also a B4xLib version which doesn't rely on JavaxSound so should be cross platrofm.

This example app provides 2 code modules that do the mixing of the audio files. TrackMixer_G mixes the tracks on the Gui thread and as you can see from the image below, can mix quite a few tracks. I have not found it's limit, but if you know anything about digital audio, you will know that the audio buffer size is very important. TrackMixer_T mixes the tracks on a separate thread. This allows the same performance with a lower buffer size or potentially more tracks. The choice is up to you, if you are only mixing a few tracks and the responsiveness is not an issue then basing your mixer on TrackMixer_G will be more than adequate, and you won't need the Threading Library. If you are mixing in the background when other things are going on, then base your mixer on TrackMixer_T to provide the best performance.

They are named as they are so that you can do a global replace on the names in the app to switch between the two methods. I have not provided a b4xlib with either of them as it would be a lot more work, and creating a complex app requires more classes to do things specific to your requirements. Feel free to copy the Mixer code and use it as you need.

1661254924454.png


Demo App

The JAudioTrack2-Mixer-Full demo app gives an example of creating a mixer with classes that manage the Gui for each track. It is complex by necessity. I have also created a simple mixer example with no Gui posted separately.

Both mixer classes require the audio to be 16bit stereo files. A utility app FFMpegConvert that will convert a folder of wav files to the required format is attached. A Sample Rate of 44100 is recommended as most sound cards can process at that rate. You may be able to use other sample rates if your sound card and Java supports them. You will need to download and set up FFMPeg if you haven't already (https://ffmpeg.org/) and point the converter at the ffmpeg.exe.

Depends On
  • jAudioTrack2 V0.12+
  • DragDrop2-b4xlib (attached) It may be slightly different from Erels b4xlib, so download this one. It won't cause any conflicts as it is named differently.
  • Threading
  • ByteConverter (Internal)
  • jDateUtils (Internal)
  • jRandomaccessFile (Internal)
  • jXUI and XUI Views (Internal)
  • JavaObject (internal)

Try it out
To play tracks in the demo app, drag a folder of wav files (44100 16bit stereo files) onto the Label in between the Folder label and the open folder button, or press the open folder button and load the folder.​
There are two folders with tracks available on my google drive for you to try it out. one with 3 Tracks (my creation) and one with 15 which was downloaded from Then Mixing Secrets and converted with the attached FFMpegconvert utility. Download either or both, unzip then and drag or open the folder. These folders are too large to include within the app so unfortunately you will have to download them separately.​

Limitations
  • The demo app only plays on the default output device, it is complex enough as it is for a demo. Changing outputs can easily be accomplished with the JAudioTrackLibrary which is required for the mixer.
  • There are no effects to add to the tracks. That would require a whole new project on it's own.
  • Files should be 16bit Stereo and preferably 44100hz sample rate. It may work with other sample rates if Java and your soundcard support them.


Caveat
This is not for the feint hearted. Audio programming is a complex topic, although the code that does the mixing is only a few lines long there are many things that have to be just right for it to work.

How well it works will also depend on the speed of your PC. It is unlikely to provide good performance on older slow PC's. Mine is a 3 year old i5-7400 3.00Ghz. definitely not high spec and anything approaching that should be fine. If you're not sure, just try it.

Enjoy and let me know how you get on with it.
 

Attachments

  • FFMPegConvert.zip
    7.6 KB · Views: 156
  • JAudioTrack2-Mixer-Full.zip
    27.5 KB · Views: 179
  • DragDrop2-b4xlib.b4xlib
    6.7 KB · Views: 161
Last edited:

Magma

Expert
Licensed User
Longtime User
just WOW!!!!!!...

Fruity Loops were terrified !!!! .... I am sure that your next project will be a Tracker... and I am waiting for it !!!!! .........................

is there a way to find the bpm of a track (wav... mp3) ????
 

stevel05

Expert
Licensed User
Longtime User
I am sure that your next project will be a Tracker.
All things are possible, there will be huge problems of syncing to overcome to accomplish this, but you never know. A few simple effects is my next goal, more out of curiosity than anything else.
is there a way to find the bpm of a track (wav... mp3)
That is more difficult than it sounds. There is no guarantee that the bpm of an audio track stays the same throughout the track. I have a feeling that the looping apps (Fruityloops etc) rely on the bpm specified in the file name for syncing, or more likely a value stored in the file as metadata, I haven't checked the file specs. It would be possible to get an estimate of the bpm, and there are some libraries to do this, but it would be by no means a guaranteed value.
 

kimstudio

Active Member
Licensed User
Longtime User
Great to have this! 👍 I compiled and it works perfectly.
BTW: What's the difference between the vertical slider and gain? haven't looked into the code.
 

stevel05

Expert
Licensed User
Longtime User
Thanks for your comment, it's always nerve wracking when you post a new projects, especially one as complex as this. Plenty of things to go wrong. I'm happy that it worked first time. Thanks for letting me know :) .

What's the difference between the vertical slider and gain?
I knew that question would come up. The answer is not much.

The gain control has a greater range than the volume slider and can boost/cut the level of the track making it useful to get a coarse mix if some of the track levels are particularly low. The downsides are: 1) if the track is noisy, the noise get's boosted as well and 2) it is pretty easy to put too much gain on and introduce clipping later during mixing. The master bus is the sum of all of the other tracks so too much gain on too many tracks can cause clipping on the master bus. Then you pretty much have to start again and reduce the gain on all tracks.
 

kimstudio

Active Member
Licensed User
Longtime User
I knew that question would come up. The answer is not much.

Thanks for explanation. I got the idea. I checked the views and they have different max values and the Gain has much larger dynamic range.

When the setting dialog shows, the playing stops. I first thought that was caused by the Gui mixing then I changed to threading mode and figured out that's not the reason :). By commenting TrackMixer_G.Stop in btnSettings_Click, I found the GUI mode will actually also not block the playing when the modal dialog shows, interesting.
 

stevel05

Expert
Licensed User
Longtime User
When the setting dialog shows
That is by design, I didn't test changing the buffer size while the audio is playing. Thinking about it it may be OK. Let me know if you have any problems with it not blocking.
 

stevel05

Expert
Licensed User
Longtime User
Looking at the code, you would need to move the Dim Dat(Buffersize) as Byte line (currently 143) inside the while loop, or it will cause problems if you change the buffer size. If that doesn't introduce clicks, it should be fine.
 

stevel05

Expert
Licensed User
Longtime User
Nope, it's a bit more convoluted than that. The buffersize is only read when the playing starts. A method would be required called when the value changes in the Gui to update the buffersize. Either that or just disable the Buffersize combobox when the track is playing. I'll need to do some more tests. I don't want to put any more code than is necessary within the play loop as it will slow it down and cause popping issues.
 

kimstudio

Active Member
Licensed User
Longtime User
any problems with it not blocking.

No problems at all. I am glad to know that Gui mixing will not block internal processing, as this makes its value almost equal to use threading lib. I guess internally Erel does some dirty job for us by converting sleep into threading.

Dynamically change the buffer size should stop or at least pause for a very short time then replay, I think your design has no problem at all.
 

stevel05

Expert
Licensed User
Longtime User
Changing the Clip Indicator Threshold or Sound Indicator Threshold values would be useful while it's running. I'll probably just disable the Buffersize combobox while the track is running with an appropriate message or just pause the track and restart if it's changed. I'm favouring the latter.
 

kimstudio

Active Member
Licensed User
Longtime User
steve, the variable for mixing accumulator should be dimed as Int as I found Short overflow will not be truncated but wrapped around. Could you confirm this?

Dim NewSample, Mixed As Short 'Int

I made an audio mixing demo before in B4J seems also has this problem and I want to modify it.
 

stevel05

Expert
Licensed User
Longtime User
the variable for mixing accumulator should be dimed as Int as I found Short overflow will not be truncated but wrapped around

This is true, the problem is that the value has to get back to a short at some point as that is the how the value is stored in a wav file and played with the SourceDataLine.

A bit further down the code the Mixed variable is put into the short mixbuffer. I just tested this and it has the same issues.

Hopefully the clip warnings will keep most values under the shorts Max value, but we obviously cant rely on this. The only way I can see is to hard limit the value when returning it to the buffer.

My preference would be a to use a float(4 byte) value to hold anything that is going to be calculated because most of the effects processing I am looking at do their calculations in floats and wouldn't then need multiple casting, hopefully saving a few clock cycles.
 

kimstudio

Active Member
Licensed User
Longtime User
My preference would be a to use a float(4 byte) value to hold anything that is going to be calculated because most of the effects processing I am looking at do their calculations in floats and wouldn't then need multiple casting, hopefully saving a few clock cycles.

For effects I think float is a must. I am now tring to make a simple synth app and using double for processing, not sure about the performance comparing to float. 64bit machine should have no problem? hate all these float to double conversions and for each number to put f behind like 0.5f as a float...
 
Top