Java Question Pitch Detection

techknight

Well-Known Member
Licensed User
Longtime User
I saw that a small section of the TarsosDSP library was wrapped for pitch detection and released here on the forums. I just now found it and it is the answer to one of my long standing issues I have had for a while now.

However, Could someone wrap the rest of the pitch detection support/algorithms? I need the Goertzel detection so I can pick a range of frequencies. I am looking for 3 specific frequencies at the same time among the rest of the "noise"

Kinda like DTMF but not.

Thanks
 

techknight

Well-Known Member
Licensed User
Longtime User
This is my halphazard attempt at editing the existing library to make it work:

B4X:
package smm.dspp;

import anywheresoftware.b4a.AbsObjectWrapper;
import anywheresoftware.b4a.BA;
import anywheresoftware.b4a.BA.ActivityObject;
import anywheresoftware.b4a.BA.Author;
import anywheresoftware.b4a.BA.Hide;
import anywheresoftware.b4a.BA.ShortName;
import anywheresoftware.b4a.BA.Version;
import anywheresoftware.b4a.BA.Permissions;
import anywheresoftware.b4a.BA.DependsOn;
import anywheresoftware.b4a.BA.Events;
import be.tarsos.dsp.AudioDispatcher;
import be.tarsos.dsp.AudioEvent;
import be.tarsos.dsp.io.android.AudioDispatcherFactory;
import be.tarsos.dsp.pitch.PitchDetectionHandler;
import be.tarsos.dsp.pitch.PitchDetectionResult;
import be.tarsos.dsp.pitch.PitchProcessor;
import be.tarsos.dsp.pitch.PitchProcessor.PitchEstimationAlgorithm;
import be.tarsos.dsp.pitch.Goertzel;
import be.tarsos.dsp.pitch.Goertzel.FrequenciesDetectedHandler;

@Author("SMM")
@Version(0.02f)
@Permissions(values={"android.permission.RECORD_AUDIO"})
@ActivityObject
@ShortName("DSP")
@Events(values={"pitchdata(data as float)", "GoertzelData(DetectedFrequencies() as Double, DetectedLevels() as Double"})
@DependsOn(values={"TarsosDSP-Android-2.0-bin"})

public class dspba extends AbsObjectWrapper<AudioDispatcher> {
    private BA ba;
    private String eventName;
    private AudioDispatcher obj;

    public void Initialize(BA ba, String EventName) {
  
    }
  
    /**
    * Initializes the DSP Object as a Single Pitch Frequency Detector
    *EventName = Name of Object/Event that will be raised with the frequency value
    *SampleRate = Sampling frequency of the audio
    *AudioBufferSize = Size of the buffer where the sampled audio will be stored
    *bufferOverlap = Amount of space where the buffers are allowed to overlap
    */  
    public void StartPitchDetect(BA ba, String EventName, int sampleRate, int audioBufferSize, int bufferOverlap) {
        _initializepitch(ba, EventName, sampleRate, audioBufferSize, bufferOverlap);
    }

    /**
    * Initializes the DSP Object as a Goertzel Frequency Detector
    *EventName = Name of Object/Event that will be raised with the frequency value
    *SampleRate = Sampling frequency of the audio
    *AudioBufferSize = Size of the buffer where the sampled audio will be stored
    *bufferOverlap = Amount of space where the buffer is allowed to overlap
    *DetectFrequencies = Array of frequencies at which to be detected
    */
    public void StartGoertzelDetect(BA ba, String EventName, int sampleRate, int audioBufferSize, int bufferOverlap, double[] DetectFrequencies) {
        _initializegoertzel(ba, EventName, sampleRate, audioBufferSize, bufferOverlap, DetectFrequencies);
    }

    @Hide
    public void _initializepitch(final BA ba, String EventName, int sampleRate, int audioBufferSize, int bufferOverlap) {
        this.ba = ba;
        this.eventName = EventName.toLowerCase(BA.cul);
        this.obj = AudioDispatcherFactory.fromDefaultMicrophone(sampleRate, audioBufferSize, bufferOverlap);
        setObject(this.obj);
        ((AudioDispatcher) getObject()).addAudioProcessor(new PitchProcessor(PitchEstimationAlgorithm.FFT_YIN, (float) sampleRate, audioBufferSize, new PitchDetectionHandler() {
            public void handlePitch(PitchDetectionResult pitchDetectionResult, AudioEvent audioEvent) {
                final float pitch = pitchDetectionResult.getPitch();
                ba.activity.runOnUiThread(new Runnable() {
                    public void run() {
                        if (ba.subExists(dspba.this.eventName + "_pitchdata")) {
                            ba.raiseEvent2(ba, false, dspba.this.eventName + "_pitchdata", true, new Object[]{Float.valueOf(pitch)});
                        }
                    }
                });
            }
        }));
        new Thread((Runnable) getObject(), "Audio Dispatcher").start();
        if (this.eventName.length() <= 0) {
        }
    }
  
    @Hide
    public void _initializegoertzel(final BA ba, String EventName, int sampleRate, int audioBufferSize, int bufferOverlap, double[] DetectFrequencies) {
        this.ba = ba;
        this.eventName = EventName.toLowerCase(BA.cul);
        this.obj = AudioDispatcherFactory.fromDefaultMicrophone(sampleRate, audioBufferSize, bufferOverlap);
        setObject(this.obj);
        ((AudioDispatcher) getObject()).addAudioProcessor(new Goertzel((float) sampleRate, audioBufferSize, DetectFrequencies, new FrequenciesDetectedHandler() {
            @Override
            public void handleDetectedFrequencies(final double[] frequencies, final double[] powers, final double[] allFrequencies, final double allPowers[]) {
                //final float pitch = pitchDetectionResult.getPitch();
                ba.activity.runOnUiThread(new Runnable() {
                    public void run() {
                        if (ba.subExists(dspba.this.eventName + "_goertzeldata")) {
                            ba.raiseEvent2(ba, false, dspba.this.eventName + "_goertzeldata", true, frequencies);
                        }
                    }
                });
            }
        }));
        new Thread((Runnable) getObject(), "Audio Dispatcher").start();
        if (this.eventName.length() <= 0) {
        }
    }
}

However, I keep getting "Object should first be initialized" when I call my added Goertzel routine.

If I call the Pitch routine, it works fine. I dont get it.

any ideas?
 

techknight

Well-Known Member
Licensed User
Longtime User
bump.

Tried everything I can think of including deleting the pitch code to get it out of the way. Same problem.

I tried to simplify things the best I could, and now I have this:
B4X:
package smm.dspp;

import anywheresoftware.b4a.AbsObjectWrapper;
import anywheresoftware.b4a.BA;
import anywheresoftware.b4a.BA.ActivityObject;
import anywheresoftware.b4a.BA.Author;
import anywheresoftware.b4a.BA.Hide;
import anywheresoftware.b4a.BA.ShortName;
import anywheresoftware.b4a.BA.Version;
import anywheresoftware.b4a.BA.Permissions;
import anywheresoftware.b4a.BA.DependsOn;
import anywheresoftware.b4a.BA.Events;
import be.tarsos.dsp.AudioDispatcher;
import be.tarsos.dsp.AudioProcessor;
import be.tarsos.dsp.AudioEvent;
import be.tarsos.dsp.io.android.AudioDispatcherFactory;
import be.tarsos.dsp.pitch.PitchDetectionHandler;
import be.tarsos.dsp.pitch.PitchDetectionResult;
import be.tarsos.dsp.pitch.PitchProcessor;
import be.tarsos.dsp.pitch.PitchProcessor.PitchEstimationAlgorithm;
import be.tarsos.dsp.pitch.Goertzel;
import be.tarsos.dsp.pitch.Goertzel.FrequenciesDetectedHandler;

@Author("SMM")
@Version(0.02f)
@Permissions(values={"android.permission.RECORD_AUDIO"})
@ActivityObject
@ShortName("DSP")
@Events(values={"pitchdata(data as float)",
        "GoertzelData(DetectedFrequencies() as Double)"})
@DependsOn(values={"TarsosDSP-Android-2.0-bin"})

public class dspba extends AbsObjectWrapper<AudioDispatcher> {
    private BA ba;
    private String eventName;
    private AudioDispatcher obj;

    public void Initialize(BA ba, String EventName) {
  
    }
  
    /**
    * Initializes the DSP Object as a Single Pitch Frequency Detector
    *EventName = Name of Object/Event that will be raised with the frequency value
    *SampleRate = Sampling frequency of the audio
    *AudioBufferSize = Size of the buffer where the sampled audio will be stored
    *bufferOverlap = Amount of space where the buffers are allowed to overlap
    */  
 //   public void StartPitchDetect(BA ba, String EventName, int sampleRate, int audioBufferSize, int bufferOverlap) {
 //       _initializepitch(ba, EventName, sampleRate, audioBufferSize, bufferOverlap);
 //   }

    /**
    * Initializes the DSP Object as a Goertzel Frequency Detector
    *EventName = Name of Object/Event that will be raised with the frequency value
    *SampleRate = Sampling frequency of the audio
    *AudioBufferSize = Size of the buffer where the sampled audio will be stored
    *bufferOverlap = Amount of space where the buffer is allowed to overlap
    *DetectFrequencies = Array of frequencies at which to be detected
    */
    public void StartGoertzelDetect(BA ba, String EventName, int sampleRate, int audioBufferSize, int bufferOverlap, double[] DetectFrequencies) {
        _initializegoertzel(ba, EventName, sampleRate, audioBufferSize, bufferOverlap, DetectFrequencies);
    }
/*
    @Hide
    public void _initializepitch(final BA ba, String EventName, int sampleRate, int audioBufferSize, int bufferOverlap) {
        this.ba = ba;
        this.eventName = EventName.toLowerCase(BA.cul);
        this.obj = AudioDispatcherFactory.fromDefaultMicrophone(sampleRate, audioBufferSize, bufferOverlap);
        setObject(this.obj);
        ((AudioDispatcher) getObject()).addAudioProcessor(new PitchProcessor(PitchEstimationAlgorithm.FFT_YIN, (float) sampleRate, audioBufferSize, new PitchDetectionHandler() {
            public void handlePitch(PitchDetectionResult pitchDetectionResult, AudioEvent audioEvent) {
                final float pitch = pitchDetectionResult.getPitch();
                ba.activity.runOnUiThread(new Runnable() {
                    public void run() {
                        if (ba.subExists(dspba.this.eventName + "_pitchdata")) {
                            ba.raiseEvent2(ba, false, dspba.this.eventName + "_pitchdata", true, new Object[]{Float.valueOf(pitch)});
                        }
                    }
                });
            }
        }));
        new Thread((Runnable) getObject(), "Audio Dispatcher").start();
        if (this.eventName.length() <= 0) {
        }
    }
*/  
    @Hide
    public void _initializegoertzel(final BA ba, String EventName, int sampleRate, int audioBufferSize, int bufferOverlap, double [] DetectFrequencies) {
        this.ba = ba;
        this.eventName = EventName.toLowerCase(BA.cul);
        this.obj = AudioDispatcherFactory.fromDefaultMicrophone(sampleRate, audioBufferSize, bufferOverlap);
        setObject(this.obj);
        ((AudioDispatcher) getObject()).addAudioProcessor(goertzelAudioProcessor);
        new Thread((Runnable) getObject(), "Audio Dispatcher").start();
        if (this.eventName.length() <= 0) {
        }
    }
  
    private final AudioProcessor goertzelAudioProcessor = new Goertzel(44100, 256, new double[]{ 1,2,3,4,5,6,7,8,9,10 }, new FrequenciesDetectedHandler() {
        @Override
        public void handleDetectedFrequencies(final double[] frequencies, final double[] powers, final double[] allFrequencies, final double allPowers[]) {
            //ba.activity.runOnUiThread(new Runnable() {
                //public void run() {
                    if (ba.subExists(dspba.this.eventName + "_goertzeldata")) {
                        ba.raiseEvent2(ba, false, dspba.this.eventName + "_goertzeldata", true, frequencies.length);
                    }
                //}
            //});
        }
    });
}

But I am still getting this:
B4X:
** Activity (main) Pause, UserClosed = true **
*** Service (starter) Create ***
** Service (starter) Start **
** Activity (main) Create, isFirst = true **
** Activity (main) Resume **
Error occurred on line: 59 (Main)
java.lang.RuntimeException: Object should first be initialized (DSP).
    at anywheresoftware.b4a.AbsObjectWrapper.getObject(AbsObjectWrapper.java:50)
    at smm.dspp.dspba._initializegoertzel(dspba.java:94)
    at smm.dspp.dspba.StartGoertzelDetect(dspba.java:62)
    at b4a.example.main._activity_permissionresult(main.java:418)
    at java.lang.reflect.Method.invoke(Native Method)
    at anywheresoftware.b4a.shell.Shell.runMethod(Shell.java:732)
    at anywheresoftware.b4a.shell.Shell.raiseEventImpl(Shell.java:348)
    at anywheresoftware.b4a.shell.Shell.raiseEvent(Shell.java:255)
    at java.lang.reflect.Method.invoke(Native Method)
    at anywheresoftware.b4a.ShellBA.raiseEvent2(ShellBA.java:144)
    at anywheresoftware.b4a.BA$2.run(BA.java:370)
    at android.os.Handler.handleCallback(Handler.java:873)
    at android.os.Handler.dispatchMessage(Handler.java:99)
    at android.os.Looper.loop(Looper.java:193)
    at android.app.ActivityThread.main(ActivityThread.java:6718)
    at java.lang.reflect.Method.invoke(Native Method)
    at com.android.internal.os.RuntimeInit$MethodAndArgsCaller.run(RuntimeInit.java:493)
    at com.android.internal.os.ZygoteInit.main(ZygoteInit.java:858)

The line is this: dp.StartGoertzelDetect("dp", 44100, 256, 0, Array As Double(1024))

I think ive met my match here considering I am trying to work with a language I have maybe 2% familiarity with.

Whether I call dp.initialize or not makes no difference. I wish I knew what B4A looks for to know something is "initialized" because I think its hiding the real error.
 
Last edited:

techknight

Well-Known Member
Licensed User
Longtime User
If I change the setObject(This.obj); to simply setObject(obj);

the problem goes away. Why is that?

Also even though it doesnt crash anymore, the event never gets raised/nothing happens. So theres still that to figure out.
 

techknight

Well-Known Member
Licensed User
Longtime User
I didnt think so either. but it runs one way, and not the other.

Also I discovered if I pass an unsupported sample rate setting, the same thing happens. the object never gets initialized.
 

Jim-WI

Member
Licensed User
The TarsosDSP library was wrapped for pitch detection

Can you send me a link to the pitch detection wrapper? A working example would really help. I'm looking to determine musical notes from a mandolin. There is also a crepe algorithm the is in an ml5 javascript library that appears to give good results.
 
Cookies are required to use this site. You must accept them to continue using the site. Learn more…