B4A Library SpeechToText - Continuous Offline Voice Recognition

This is a wrapper of Acephei VOSK , With this, you can add continuous offline speech recognition feature to your application,

NOTE:

1. As it works offline the app should be complied with the voice model. It will increase the app size by 30-40Mb.
2. The accuracy depends on the voice model. You can train your own voice model. For more details check the models download link below.
3. Remember to add RECORD_AUDIO permission.

How to use:

1. Download the required voice model from here.
2. Change the file name to a simple one like "model.zip"
3. Copy it to the Files folder of your project.
4. Now to use that model check the attached example.

SpeechToText

Author: @Biswajit
Version: 1.5

• SpeechToText
  • Events:
    • Error (message As String)
    • FinalResult (text As String)
    • MicrophoneBuffer (buffer() As Byte)
    • PartialResult (text As String)
    • Paused (paused As Boolean)
    • ReadyToListen
    • ReadyToListenEx new
    • ReadyToRead
    • Restarted
    • Result (text As String)
  • Fields:
    • sampleRate As Int
      Default 16000
  • Functions:
    • cancel As Boolean
      Cancel microphone recognition. Do not post any new events, simply cancel processing.
      Does nothing if recognition is not active.
      Return type: @return:true if recognition was actually stopped
    • FeedExternalBuffer (ExBuffer As Byte()) new
      For recognizing the external audio buffer, feed the buffer here.
      ExBuffer: The external audio byte buffer.
    • Initialize (eventName As String, modelPath As String)
      Initialize the object.
      eventName: The event name prefix.
      modelPath: The model folder path.
    • pause (pause As Boolean)
      Pause microphone recognition.
      pause: Pass true to pause and false to continue.
    • prepareAudioFile (audioPath As String, predefinedWords As String)
      Prepare the audio file for recognition. On success Eventname_ReadyToRead event will be raised.
      Call startReading to start reading the file.
      audioPath: Audio file path.
      predefinedWords: Add some predefined words/phrase as JSON string. Can be blank.
    • prepareListenerEx (predefinedWords As String) new
      Prepare the listener for external audio buffer. On success Eventname_ReadyToListenEx event will be raised.
      Call startListeningEx to start listening.
      predefinedWords: Add some predefined words/phrase as JSON string. Can be blank.
    • prepareMicrophone (predefinedWords As String)
      Prepare the microphone for listening. On success Eventname_ReadyToListen event will be raised.
      Call startListening to start listening.
      predefinedWords: Add some predefined words/phrase as JSON string. Can be blank.
    • reset
      Resets microphone recognizer in a thread, starts microphone recognition over again
    • shutdown
      Shutdown the microphone recognizer and release the recorder.
      Call this on activity or service closing event.
    • startListening (timeout As Int) As Boolean
      Starts microphone recognition. After specified timeout listening stops and the
      endOfSpeech signals about that. Does nothing if recognition is active.
      timeout: timeout in milliseconds to listen. -1 = infinite;
      Return type: @return:true if recognition was actually started
    • startListeningEx As Boolean new
      Starts external audio buffer recognition.
      Return type: @return:true if recognition was actually started
    • startReading (timeout As Int) As Boolean
      Starts file recognition. After specified timeout listening stops and the
      endOfSpeech signals about that. Does nothing if recognition is active.
      timeout: timeout in milliseconds to listen. -1 = infinite;
      Return type: @return:true if recognition was actually started
    • stop As Boolean
      Stops microphone/file recognition. Listener should receive final result if there is
      any. Does nothing if recognition is not active.
      Call this on activity or service closing event.
      Return type: @return:true if recognition was actually stopped

Downloads:

1. Library
2. Example
3. Voice Model
4. Test app

Update:

• Version 1.1:
  1. Added audio file to text functionality. (For now only WAV format is supported)
  2. Added predefined word/phrase detection functionality.
  3. Merged startListening and startListening2 together. Pass -1 for continuous recognition.
• Version 1.2:
  1. Added MicrophoneBuffer event where you will receive the microphone audio buffer while using voice recognition.
• Version 1.3:
  1. Added method to change the sampling rate.
• Version 1.4:
  1. Fixed the app crashing issue while calling shutdown without stating the recognizer
• Version 1.5:
  1. Added option to feed external audio buffer. Instead of using the internal audio recorder you can feed external audio buffer from another audio source.
     (Check the latest example project)
  2. Updated VOSK and JNA library. (Please delete old dependencies before coping the new ones.)

If you like my work, please donate. Your donations will encourage me to add more features in the future.

 

[DonManfred]

I am still using an old B4A version (8.3)

Start with updating to the newest Version!
Delete B4A and the SDK.
Install both new based on the installationinstructions.

 

[Ricks Film Restoration]

OK. Just to say it is not the .so file itself that is the problem. The problem is that the .so file can not be found!
See my initial (path) error in #68 and #73: java.lang.UnsatisfiedLinkError: Native library (com/sun/jna/android-aarch64/libjnidispatch.so) Not found in resource path (.)
Please re-read my earlier problem description. That's why I was asking whether the AAR file maybe should be added to the project a.o. or that I should modify the jna.jar file (which I already tried).

 

[Biswajit]

Hello @Ricks Film Restoration , sorry for late reply.
I am not sure which architecture it is. But you can download the JNA library directly from Maven Repo and try it.

I just quickly checked that the JNA library available in Maven Repo also does not have any so file for aarch64.

https://repo1.maven.org/maven2/net/java/dev/jna/jna/5.13.0/jna-5.13.0.aar

 

[Ricks Film Restoration]

Thank you for your reply. What worries me is that only one other person reported the same problem. All others seem to have gotten the speech recognizer working.
The only explanation I have is that I don't use a recent B4A version (because I have not been active with B4A for a while), but I still wonder why this error message appears:
java.lang.UnsatisfiedLinkError: Native library (com/sun/jna/android-aarch64/libjnidispatch.so) Not found in resource path (.) which impies that there is something wrong with the path to the libjnidispatch.so file. By the way, your APK file contains a second .so file: libvosk.so Isn't that needed as well?
Any more help would be appreciated.

Update: After giving it more thought: Has the problem maybe got to do with my Android device: A Samsung Tab S7 with Android 13?
Does that tablet use a different processor such that a (less common) special .so type of file is needed (that has to be stored in .../android-aarch64/... )?

 

[OliverA]

Update: After giving it more thought: Has the problem maybe got to do with my Android device: A Samsung Tab S7 with Android 13?

I don't think so. I just now downloaded the source from post #1, the libraries from post #1 (placed them in the additional library folder for B4A), Archiver from this forum (placed into the additional library folder for B4A). I'm using B4A version 12.5 w/JDK 11.0.1. It compiled and installed w/o issues on a Pixel 7 running Android 17. The CPU is manufactured by Samsung, so it's probably as close as I can get to your configuration. BTW, wow, this is a pretty cool lib.

 

[Ricks Film Restoration]

Please check your type of CPU. An 'arm64' CPU is apparently not entirely the same as an 'aarch64' CPU (I Googled it).
See my error message in red above: it detects an 'aarch64', not an 'arm64' on my Samsung.
Update: I also checked the CPU with the Devinfo app. It reports an 'aarch64' CPU on my tablet and not on my new Samsung phone.

 

[Ricks Film Restoration]

The required .so files for each type of CPU can be found here: https://github.com/java-native-access/jna/tree/master/lib/native
I added them in newly created folders (under com/sun/jna/) to the jna.jar file (with zip), but still the path error occurs. Any ideas?

 

[Biswajit]

An 'arm64' CPU is apparently not entirely the same as an 'aarch64' CPU (I Googled it)

In that case, I think this library will not work for your CPU maybe because I checked that VOSK library aar file and could not find the libvosk.so file for aarch64.

 

[Ricks Film Restoration]

Key question: But why does your APK file work fine then on my Samsung?!
By the way, I see a lot of differences between your Example B4A project and the contents of your APK (after unzipping this APK).
Is your example up to date?

 

[Ricks Film Restoration]

PS: The required .so files for each type of CPU can be found here: https://github.com/java-native-access/jna/tree/master/lib/native

 

[Biswajit]

Is your example up to date?

Yes. Though compilation doesn't depends on the old or new example project. Your issue is related to cpu architecture.

The required .so files for each type of CPU can be found here

I know. But have you tried using those in your project? I have busy schedule these days. Once I get free time I will check. Until then try using the jna from the office GitHub

 

[Ricks Film Restoration]

Indeed I did try heaps of .so files, edited the jna.jar file and much more. Questions:
1. If the issue is related to my CPU architecture, then why does your APK file work fine on both my Samsungs (!), but not with your B4A project??
2. Also in your APK file I found a directory called "lib" which contains four folder: "arm64-v8a", "armeabi-v7a", "x86" and "x86_64".
This "lib" folder is not included in your B4A sample project.
3. I am pretty sure now that the problem is caused by something pointing to the wrong path and not caused by the .so file(s).
I have read that "arm64-v8a" is compatible with "aarch64".
4. Please provide me the correct link to the suggested jna file in Github, because I already tried some and obviously failed.
5. If you have a "new example project" kindly make it available to the B4A community (especially me)
Thanks in advance for your help!

 

[Jmu5667]

Hi

There seems to be a problem on Android 12+ when initialising the library. I cant get any debug information as the app just exit's. I am using the mod'd version you did for me.

Regards

John.

 

[Biswajit]

Hi

There seems to be a problem on Android 12+ when initialising the library. I cant get any debug information as the app just exit's. I am using the mod'd version you did for me.

Regards

John.

check the unfiltered log

 

[Biswajit]

Indeed I did try heaps of .so files, edited the jna.jar file and much more. Questions:
1. If the issue is related to my CPU architecture, then why does your APK file work fine on both my Samsungs (!), but not with your B4A project??
2. Also in your APK file I found a directory called "lib" which contains four folder: "arm64-v8a", "armeabi-v7a", "x86" and "x86_64".
This "lib" folder is not included in your B4A sample project.
3. I am pretty sure now that the problem is caused by something pointing to the wrong path and not caused by the .so file(s).
I have read that "arm64-v8a" is compatible with "aarch64".
4. Please provide me the correct link to the suggested jna file in Github, because I already tried some and obviously failed.
5. If you have a "new example project" kindly make it available to the B4A community (especially me)
Thanks in advance for your help!

1. Is it failing for Debug as well as Release mode?
2. That Lib folder inside APK comes from the vosk AAR file. When you compile the project it will be added to your APK
3. I just made the wrapper. Neither the VOSK nor the JNA library. If you think there is something related to the libraries you can always download those from the internet and use in your project
4. You can download those from https://mvnrepository.com/
5. Will check once I get some free time.

Thanks.

 

[Ricks Film Restoration]

Hi

There seems to be a problem on Android 12+ when initialising the library. I cant get any debug information as the app just exit's. I am using the mod'd version you did for me.

Regards

John.

Indeed, same problem here. My Samsungs have Android-13. It crashes with the path-error to the aarch64 folder that I described earlier.

 

[Ricks Film Restoration]

1. Is it failing for Debug as well as Release mode?
2. That Lib folder inside APK comes from the vosk AAR file. When you compile the project it will be added to your APK
3. I just made the wrapper. Neither the VOSK nor the JNA library. If you think there is something related to the libraries you can always download those from the internet and use in your project
4. You can download those from https://mvnrepository.com/
5. Will check once I get some free time.

Thanks.

Thank you for taking the time to take a look at this problem. In reply to your questions,
1. Yes.
2. OK understood.
3. The links in the JNA.jar file seems to be the problem I think (not sure). See the Java error that I reported a number of times now.
4. Thanks. I´ll try to find and try the jna.jar file from there.
5. Don´t work too hard! There´s a saying that `One should work to live, not live to work`....

 

[Jmu5667]

09/27/2023 09:38:46.415 - Atlas SOS - starter::set_stt::/data/user/0/com.is.vitllinkSOS/files/model
09/27/2023 09:38:46.416 - Atlas SOS - cls_speechmodel::Initialize
09/27/2023 09:38:46.416 - Atlas SOS - cls_speechmodel::extract_model::Model found
Model():model.cc:122) Folder '/data/user/0/com.is.vitllinkSOS/files/model' does not contain model files. Make sure you specified the model path properly in Model constructor. If you are not sure about relative path, use absolute path specification.
terminating with uncaught exception of type kaldi::KaldiFatalError: kaldi::KaldiFatalError

From further investigation this ar.AsyncUnZip(File.DirInternal,starter.model_zip_name,File.DirInternal,"unzip") is not unzipping. I have tested this on a Pixel 6a API 33 and and One Plus 10 API 33.

I enabled both the unzip_ZipProgression and unzip_UnZipDone events and neither of them fire. I will keep at this.

Regards

John.

 

[Jmu5667]

So, and update. I have a class that does the extract of the model. The async methods dont function correctly, however the no async method do. I eleive I have solved the issue.

 

[Ricks Film Restoration]

So, and update. I have a class that does the extract of the model. The async methods dont function correctly, however the no async method do. I eleive I have solved the issue.

Would you care to share your solution with us? Does it resolve my path error problem as reported earlier?
it seems you are describing a different problem than mine, related to (un)zipping.