Android Example Offline Continuous Speech Recognition

This is a proof of concept port of CMU PocketSphinx for Android demo app.

It's written in B4a using JavaObject to access the PocketSphinx library.

There is a wealth of information on the website : http://cmusphinx.sourceforge.net/ on tailoring the dictionaries, grammar and models.

If you want to control simple tasks or games with a few keywords, it doesn't look like it would be too difficult to implement. I'll let you know in a few weeks when I have integrated it into a current project.

I've used B4a 3.82, so if you're using an older version you will have to change some of the JavaObject calls relating to the array definitions i.e. change them to Array As Object(..) instead of Array(...)

In line with recommendations from CMU the setup copies the files from the assets directory to the apps default directory if it's not there, or the MD5 ID has changed. This is done in a thread in a separate class that has the #ExcludeFormDebugger: True directive, so you should be able to debug the rest of the app.



Download and unzip the library file and copy it to your additional libraries folder.

The project is located on my Google Drive as, with the dictionaries, it is too large (about 10 MB) to upload to the forum you can download it here : SpeechRecognizerContinuous


It's a bit confusing to download if you haven't done it before (as I just found out), click on the file, then there will be a download icon somewhere at the top middle of the screen.

The copyright.txt file contains the original copyright notice from the demo app.

Update to V1.1
  • Replaced threading with wait for
  • Replaced File.DirExternal with RuntimePermissions GetSafeDirDefaultExternal
  • Changed listener event handler (not quite sure why that needed changing)
  • Changed manifest to android:targetSdkVersion="29"
  • Recognizer object is passed back on configuration completion instead of being shared.

  • Updated link to project on Google drive due to their change in security.


Download the new project from SpeechRecognizerContinuous. The library zip is unchanged if you already have it.

Have fun, and let me know how you get on with it.
 

Attachments

  • copyright.txt
    1.5 KB · Views: 1,393
  • Library.zip
    403.1 KB · Views: 297
Last edited:

Rick Harris

Well-Known Member
Licensed User
I would love to use your Pocket Sphinx library, but regrettably it does not work (on a Nexus-7 with Android 4.3 and Samsung Tab S 8.4 with Android 5.1).
I get no errors and the "onBeginningOfSpeech" and "onEndOfSpeech" events fire correctly, but the "onResult" and "onPartialResult" events do NOT fire, so no speech is recognized. Any ideas how to solve this?
 
Last edited:

youjunjer

Member
Licensed User
dear stevel05
I have try your code by thread:
https://www.b4x.com/android/forum/threads/offline-continuous-speech-recognition.42898/
But the event alwayse do Case "onBeginningOfSpeech" and Case "onEndOfSpeech".No any recognition text output?
Is there any more information about this API?

That I want to make a machine like "echo", it will wake up on a special word like "computer" or "master".
And then I can use google Speech Recognition api to make some voice command.
 

stevel05

Expert
Licensed User
This was done 4 years ago, I'm sure Pocket sphinx has moved on since then, the latest info can be found here http://cmusphinx.sourceforge.net/. I'm afraid I haven't used it since the original post so haven't kept up to date with improvements.
 

Sargasso

New Member
Licensed User
Hi!
Im using the latest B4A (7.80) and Android 7+.
And I was searching for "exactly this". An offline (or online) Speech recognition for an app. For me Im going to use it as... Say a word or number and the app check if its correct! So I dont need to know so many words. (But hopefully in many languages. )

So the question: What do you think about this? Or do anyone have something else that's newer and better for 2018?
(I was thinking about starting a new thread but I saw that you have wrote in this thread today so..)

THanks for any idea!
 

DonManfred

Expert
Licensed User
I was thinking about starting a new thread
From the forum Guidelines...

Tips:
  • Post in the correct sub-forum.
  • Use
    B4X:
    code here...
    tags when posting code.
  • All questions should be open to all members.
  • Start a new thread for your question instead of posting in an existing thread.
  • Use the Like button as a gratitude for other members who try to help you.
  • The thread title should summarize the question.
  • Use the search tool.
 

petr4ppc

Well-Known Member
Licensed User
Dear stevel05,

After some time I saw your answer (your update of Cont.Speech.Rec.). Thank you so much for this.
I am trying your project and I get:
"An error has occured in Sub:setuprecognizer ResumableSub.ConfigureRecognizerresume(java line: 101) _keyphrase not found" Continue? Yes-no

If I click on YES, I see only white screen and then nothing happen (I am trying to say : computer, etc)..

I am trying it with Android 11, Manifest editor: targetsdkversion:29 (or 30)
android.jar, SDK=30

Please very much, do you know where I am doing mistake?
Best regards and thank you very much,
p4ppc
 

stevel05

Expert
Licensed User
Sorry, this is not an error I am familiar with, and don't have an android 11 device to test it on. Does anyone recognise the error?
 

petr4ppc

Well-Known Member
Licensed User
Thank you for answer stevel05. For certain I am sending full error (android sdk 30):
** Activity (main) Create, isFirst = true **
** Activity (main) Resume **
setuprecognizer$ResumableSub_ConfigureRecognizerresume (java line: 101)
java.lang.RuntimeException: Field: _keyphrase not found in: com.stevel05.speechrecognizer
at anywheresoftware.b4j.object.JavaObject$FieldCache.getField(JavaObject.java:307)
at anywheresoftware.b4j.object.JavaObject.GetField(JavaObject.java:182)
at com.stevel05.setuprecognizer$ResumableSub_ConfigureRecognizer.resume(setuprecognizer.java:101)
at com.stevel05.setuprecognizer._vvv2(setuprecognizer.java:60)
at com.stevel05.setuprecognizer$ResumableSub_DoConfiguration.resume(setuprecognizer.java:212)
at com.stevel05.setuprecognizer._doconfiguration(setuprecognizer.java:191)
at java.lang.reflect.Method.invoke(Native Method)
at anywheresoftware.b4a.BA.raiseEvent2(BA.java:213)
at anywheresoftware.b4a.keywords.Common$11.run(Common.java:1178)
at android.os.Handler.handleCallback(Handler.java:938)
at android.os.Handler.dispatchMessage(Handler.java:99)
at android.os.Looper.loop(Looper.java:246)
at android.app.ActivityThread.main(ActivityThread.java:8512)
at java.lang.reflect.Method.invoke(Native Method)
at com.android.internal.os.RuntimeInit$MethodAndArgsCaller.run(RuntimeInit.java:602)
at com.android.internal.os.ZygoteInit.main(ZygoteInit.java:1139)
java.lang.RuntimeException: Field: _keyphrase not found in: com.stevel05.speechrecognizer

and android sdk 28(here I have not installed all components I think):
ObfuscatorMap.txt file created in Objects folder.
Compiling layouts code. (0.00s)
Organizing libraries. (0.00s)
(AndroidX SDK)
Compiling resources (1.32s)
Linking resources Error
bin\extra\compiled_resources\androidx.core.zip@values-am_values-am.arsc.flat: error: invalid compiled table.
bin\extra\compiled_resources\androidx.core.zip@values-en-rCA_values-en-rCA.arsc.flat: error: invalid compiled table.
bin\extra\compiled_resources\androidx.core.zip@values-hu_values-hu.arsc.flat: error: invalid compiled table.
bin\extra\compiled_resources\androidx.core.zip@values-my_values-my.arsc.flat: error: invalid compiled table.
bin\extra\compiled_resources\androidx.core.zip@values-gu_values-gu.arsc.flat: error: invalid compiled table.
bin\extra\compiled_resources\androidx.core.zip@values-ky_values-ky.arsc.flat: error: invalid compiled table.
bin\extra\compiled_resources\androidx.core.zip@values-eu_values-eu.arsc.flat: error: invalid compiled table.
bin\extra\compiled_resources\androidx.core.zip@values-hy_values-hy.arsc.flat: error: invalid compiled table.
bin\extra\compiled_resources\androidx.core.zip@drawable-xhdpi-v4_notify_panel_notification_icon_bg.png.flat: error: failed to read compiled file header.
bin\extra\compiled_resources\androidx.core.zip@values-b+sr+Latn_values-b+sr+Latn.arsc.flat: error: invalid compiled table.
bin\extra\compiled_resources\androidx.core.zip@values-nb_values-nb.arsc.flat: error: invalid compiled table.
bin\extra\compiled_resources\androidx.core.zip@drawable-hdpi-v4_notification_bg_normal_pressed.9.png.flat: error: failed to read compiled file header.
bin\extra\compiled_resources\androidx.core.zip@drawable-mdpi-v4_notification_bg_normal_pressed.9.png.flat: error: failed to read compiled file header.
bin\extra\compiled_resources\androidx.core.zip@layout_notification_template_part_chronometer.xml.flat: error: failed to read compiled file header.
bin\extra\compiled_resources\androidx.core.zip@values-ur_values-ur.arsc.flat: error: invalid compiled table.
bin\extra\compiled_resources\androidx.core.zip@values-tr_values-tr.arsc.flat: error: invalid compiled table.
bin\extra\compiled_resources\androidx.core.zip@values-sr_values-sr.arsc.flat: error: invalid compiled table.
bin\extra\compiled_resources\androidx.core.zip@values-mn_values-mn.arsc.flat: error: invalid compiled table.
bin\extra\compiled_resources\androidx.core.zip@values-en-rAU_values-en-rAU.arsc.flat: error: invalid compiled table.
bin\extra\compiled_resources\androidx.core.zip@layout-v16_notification_template_custom_big.xml.flat: error: failed to read compiled file header.

All errors are detected here:
B4X:
    mRecognizer = Setup.RunMethodJO("defaultSetup",Null) _
    .RunMethodJO("setAcousticModel",Array(JavaFile(ModelsDir,"hmm/en-us-semi"))) _
    .RunMethodJO("setDictionary",Array(JavaFile(ModelsDir,"dict/cmu07a.dic"))) _    
    .RunMethodJO("setKeywordThreshold",Array(Val)) _
    .Runmethod("getRecognizer",Null)
    Log ("cr4")
    'Create Keyword-activation search
   
    Dim ModuleJO As JavaObject = mModule
    Log ("cr41 ***** here")
    mRecognizer.RunMethod("addKeyphraseSearch",Array(ModuleJO.GetField("_kws_search"), ModuleJO.GetField("_keyphrase")))
    Log ("cr5 *****")
    'Create Grammar based searches
    mRecognizer.RunMethod("addGrammarSearch",Array(ModuleJO.GetField("_menu_search"),JavaFile(ModelsDir,"grammar/menu.gram")))
    mRecognizer.RunMethod("addGrammarSearch",Array(ModuleJO.GetField("_digits_search"),JavaFile(ModelsDir,"grammar/digits.gram")))
    mRecognizer.RunMethod("addNgramSearch",Array(ModuleJO.GetField("_forecast_search"),JavaFile(ModelsDir,"lm/weather.dmp")))
I am using Android 11, tested with Android SDK 28 and 30, B4A 10.70
Best regards
p4ppc
 
Last edited:

stevel05

Expert
Licensed User
OK it looks like you are running in obsfuscated mode, you could try changing the references to KEYPHRASE to KEY_PHRASE, including the JavaObject call in the SetupRecognizer module:

B4X:
mRecognizer.RunMethod("addKeyphraseSearch",Array(ModuleJO.GetField("_kws_search"), ModuleJO.GetField("_key_phrase")))
 

petr4ppc

Well-Known Member
Licensed User
OK it looks like you are running in obsfuscated mode, you could try changing the references to KEYPHRASE to KEY_PHRASE, including the JavaObject call in the SetupRecognizer module:

B4X:
mRecognizer.RunMethod("addKeyphraseSearch",Array(ModuleJO.GetField("_kws_search"), ModuleJO.GetField("_key_phrase")))

Dear Stevel05, thank you very much for your advice. In release mode is everything ok, as you wrote.
Best regards
p4ppc
 

Beja

Expert
Licensed User
Hi Steve,

I tried the example but couldn't have it running.. (unfortunately stopped....)
This is the logs:

SpeechRecognizer - 49: Not all code paths return a value. (warning #2)
File 'cmu07a.dic' is not used. (warning #15)
File 'digits.gram' is not used. (warning #15)
File 'feat.params' is not used. (warning #15)
File 'mdef' is not used. (warning #15)
File 'means' is not used. (warning #15)
File 'menu.gram' is not used. (warning #15)
File 'noisedict' is not used. (warning #15)
File 'readme' is not used. (warning #15)
File 'sendump' is not used. (warning #15)
File 'transition_matrices' is not used. (warning #15)
File 'variances' is not used. (warning #15)
File 'weather.dmp' is not used. (warning #15)
Library 'Reflection' is not used. (warning #32)

I set the mic permission.
Any meaning appreciated
 

stevel05

Expert
Licensed User
Updated link to project on Google drive in first post due to their change in security.
 

stevel05

Expert
Licensed User
Sorry @Beja, I missed your post, did you get it running?
 
Top