B4A Class Android Speech Recognition API Wrapper

stevel05

Expert
Licensed User
Here is a wrapper for the Android speech recognition API using JavaObject complete with Source code.

It's based on the code found here: http://www.truiton.com/2014/06/android-speech-recognition-without-dialog-custom-activity/

There is not much to it, if you create a new project, you just need to add the Record Audio permission to your Manifest:

B4X:
AddPermission(android.permission.RECORD_AUDIO)
It supposedly works offline if you download a language pack, but I haven't tried that yet.

A very basic Gui is provided, but you can soon get rid of that if you prefer.

You can see in the code where to change the Language Preference. I'm afraid the best list I could find is on this page: http://developer.android.com/reference/java/util/Locale.html

Full documentation is here: http://developer.android.com/reference/android/speech/SpeechRecognizer.html

Let me know how you get on with it.
 

Attachments

Last edited:

JohnC

Well-Known Member
Licensed User
Steve,

Great job - I just happen to find your wrapper for something I was looking into today - I wanted to be able to end up with a file of recorded audio and the text of what was spoken in that audio file.

I figured there are basically two possible ways to do this:

1) Record audio to a file, then somehow pass the audio stream from the file to the voice recognizer so it will generate the text of what was spoken in the audio.
2) Alternatively, I was hoping that that "onBufferReceived" event in your recognizer wrapper was going to provide the audio data (buffer()) that I could then save into an audio file so I could also end up with an audio file and the text spoken in that audio file.

But it looks like the "onBufferReceived" event never gets triggered :(
 

stevel05

Expert
Licensed User
Yes, the API documentation says that there is no guarantee that the method will be called unfortunately.
 

BobsYourUncle

Member
Licensed User
Steve, nice library! I noticed it works on some of my phones and not others. If the latest version of the Google App is installed, the first recognition attempt works, the second fails with a Client Error. If I then uninstall the update to the Google App, recognition works every time! Is the latest Google App forcing Recognizer to be fussier? Is there some object left open by the library that's causing the problem for the second recognition? Any thoughts?
 

stevel05

Expert
Licensed User
The library is very light, just the code in the SpeechRecognition class. It appears to work OK with my Nexus 7 and Android 6.0.1. There are occasional clientside errors, but it generally works OK.

Which Google app did you update that causes the problem?
 

peacemaker

Expert
Licensed User
Thanks for this class!
Here is my modification for the background speech recognition (SR) in the service. But after debugging week i could not make my app well that
  • listens for silence
  • activates SR with suppressing start\stop beeps
  • recognizes speech for checking the command
  • gives message to the user by text-to-speech (TTS)
  • records voice message from user
  • play messages
All parts can work well separately - but together, one interferes to another part breaking the work :(

Here the class:
B4X:
'Class module
'v.2 by Pomelov Vlad aka Peacemaker radioa@elec.ru
Sub Class_Globals
    Private JO As JavaObject
    Private RecognizerIntent As Intent
    Private Initialized As Boolean
    Public Busy As Boolean
    Private SpeechRecognizer As JavaObject
    Private Target As Object
    Private Lang As String
    Private SpeechRecognition_Name As String
End Sub

'Initializes the object. You can add parameters to this method if needed.
'ObjectName = name of this SpeechRecognition object
Public Sub Initialize(TargetModule As Object, RecognizeLanguage As String, ObjectName As String)

 
    SpeechRecognizer.InitializeStatic("android.speech.SpeechRecognizer")
    JO = SpeechRecognizer.RunMethod("createSpeechRecognizer",Array(JO.InitializeContext))
 
    If Not(IsRecognitionAvailable) Then
        Log("Speech Recognition Not Available")
        ToastMessageShow("Speech Recognition is not Available", True)
        Return
    End If
    SpeechRecognition_Name = ObjectName
    Lang = RecognizeLanguage
    Target = TargetModule
    RecognizerIntent.Initialize("android.speech.action.VOICE_SEARCH_HANDS_FREE", "")
    RecognizerIntent.PutExtra("calling_package",Application.PackageName)
    RecognizerIntent.PutExtra("android.speech.extra.LANGUAGE_MODEL", "free_form")
    RecognizerIntent.PutExtra("android.speech.extra.MAX_RESULTS",3)
    RecognizerIntent.PutExtra("android.speech.extras.SPEECH_INPUT_POSSIBLY_COMPLETE_SILENCE_LENGTH_MILLIS", 1000)
    RecognizerIntent.PutExtra("android.speech.extras.EXTRA_SPEECH_INPUT_COMPLETE_SILENCE_LENGTH_MILLIS", 2000)
    RecognizerIntent.PutExtra("android.speech.extras.SPEECH_INPUT_MINIMUM_LENGTH_MILLIS", 1000 * 10)
 
    If Lang <> "" Then
        RecognizerIntent.PutExtra("android.speech.extra.LANGUAGE", Lang)
    End If
     
    Dim Event As Object = JO.CreateEvent("android.speech.RecognitionListener","Received","")
    JO.RunMethod("setRecognitionListener",Array(Event))
    Initialized = True
End Sub

Public Sub IsInitialized As Boolean
    Return Initialized
End Sub


Public Sub IsRecognitionAvailable As Boolean
    Dim JO1 As JavaObject
    JO1.InitializeContext
    Return JO.RunMethod("isRecognitionAvailable",Array(JO1))
End Sub
Public Sub StartListening
    Busy = True
    JO.RunMethod("startListening",Array(RecognizerIntent))
End Sub
Public Sub StopListening
    JO.RunMethod("stopListening",Null)
    Busy = False
End Sub

Public Sub Destroy
    JO.RunMethod("destroy",Null)
    Busy = False
End Sub

Public Sub cancel
    JO.RunMethod("cancel",Null)
    Busy = False
End Sub

Private Sub Received_Event (MethodName As String, Args() As Object) As Object
    Select MethodName
        Case "onBeginningOfSpeech"
        Case "onEndOfSpeech"
            Busy = False
            If SubExists(Target, SpeechRecognition_Name & "_onEndOfSpeech") Then
                CallSubDelayed(Target, SpeechRecognition_Name & "_onEndOfSpeech")
            End If
        Case "onError"
            Busy = False
            'Dim ErrorMsg As String = GetErrorText(Args(0))
            If SubExists(Target, SpeechRecognition_Name & "_onError") Then
                CallSubDelayed2(Target, SpeechRecognition_Name & "_onError", Args(0))
            End If
        Case "onResults"
            Busy = False
            Dim Results As JavaObject = Args(0)
            Dim Matches As List = Results.RunMethod("getStringArrayList",Array("results_recognition"))
            If SubExists(Target, SpeechRecognition_Name & "_onResults") Then
                CallSubDelayed2(Target, SpeechRecognition_Name & "_onResults", Matches)
            End If
        Case "onRmsChanged"
            Busy = True
            If SubExists(Target, SpeechRecognition_Name & "_onRmsChanged") Then
                CallSubDelayed2(Target, SpeechRecognition_Name & "_onRmsChanged", Args(0))
            End If
    End Select
 
End Sub

Sub GetErrorText(ErrorCode As Int) As String
    Select ErrorCode
        Case SpeechRecognizer.GetField("ERROR_AUDIO")
            Return "Audio Recording error"
         Case SpeechRecognizer.GetField("ERROR_CLIENT")
             Return "Client side error"
         Case SpeechRecognizer.GetField("ERROR_INSUFFICIENT_PERMISSIONS")
            Return "Insufficient permissions"
         Case SpeechRecognizer.GetField("ERROR_NETWORK")
             Return "Network error"
         Case SpeechRecognizer.GetField("ERROR_NETWORK_TIMEOUT")
             Return "Network timeout"
          Case SpeechRecognizer.GetField("ERROR_NO_MATCH")
             Return "No match"
          Case SpeechRecognizer.GetField("ERROR_RECOGNIZER_BUSY")
             Return "RecognitionService busy"
          Case SpeechRecognizer.GetField("ERROR_SERVER")
             Return "error from server"
         Case SpeechRecognizer.GetField("ERROR_SPEECH_TIMEOUT")
            Return "No speech input"
         Case Else
             Return "Didn't understand, please try again."
    End Select
End Sub
Working project is enclosed also.
 

Attachments

JackKirk

Well-Known Member
Licensed User
stevel05,

Thanks for wrapping this API - looks like the most palatable way to get speech recognition.

Except for one problem - when I run your sr.zip project straight out of the box on my Samsung S5 I get "Speech Recognition Not Available" - do I have to fiddle something in the phone? - I have googled and generally rutted around without success.

Any advice would be appreciated...
 

JackKirk

Well-Known Member
Licensed User
peacemaker,

In your sample project I get:

Starter.sr.IsRecognitionAvailable = False

in Main.butStart_Click

I must have some external setting wrong - but what?

Regards...
 

JackKirk

Well-Known Member
Licensed User
I've sorted it.

After some considerable googling it turns out that at some stage I must have uninstalled "Google voice typing" - if it ever was installed.

I went to the Playstore, searched for "google" and installed the first app (called rather mysteriously "Google").

Now all sample apps mentioned in this thread work!

Also, I now have a microphone icon on any keyboard that is raised - which is what I was really after - so I don't have to modify my app - bliss!

Sorry for inconveniencing everyone...
 
Last edited:

peacemaker

Expert
Licensed User
I meant - did butSetup not help to do this without googling ?
 

JackKirk

Well-Known Member
Licensed User
peacemaker,

When I installed your example I tapped butSetup but it had no obvious effect.

Once I sorted it as per my post #11 everything worked.

Thanks for your interest in my problem...
 

ArminKH

Well-Known Member
great job @stevel05 works fine on my Android 4.2.2 huawei g610
i tried it with 4 language and accuracy is awesome
thank u
 

Rick Harris

Well-Known Member
Licensed User
Steve, nice library! I noticed it works on some of my phones and not others. If the latest version of the Google App is installed, the first recognition attempt works, the second fails with a Client Error. If I then uninstall the update to the Google App, recognition works every time! Is the latest Google App forcing Recognizer to be fussier? Is there some object left open by the library that's causing the problem for the second recognition? Any thoughts?
Yes, it seems Google has messed up their speech recognizer API since beginning of March 2016. I am having similar problems. Errors 5 and 7 occur nearly always and the API only returns speech RMS values after the first recognition attempt, thus the speech volume indicator in my apps no longer works.
The solution I used was to go back to Google App version 5.8 (february 2016).
These older APK's can be found among other on www.apkmirror.com.
Everything now works fine again on my Nexus-7 (2012) with Android 4.2.
Google App versions 5.10, 5.11 and 5.12 are not OK.
Don't forget to remove the "automatic update" tick in the Google Playstore for the Google App!
 
Last edited:

Rick Harris

Well-Known Member
Licensed User
The library is very light, just the code in the SpeechRecognition class. It appears to work OK with my Nexus 7 and Android 6.0.1. There are occasional clientside errors, but it generally works OK.

Which Google app did you update that causes the problem?
Hello Steve. Can you please take a look at Google's brand new cloud based speech recognizer API (which is still in bèta).
Ref: https://www.b4x.com/android/forum/threads/new-google-cloud-speech-recognition.66135/
The API is described here: https://cloud.google.com/speech/
 

Rick Harris

Well-Known Member
Licensed User
Try my sample project. Above.
You forgot to add: StartService(Starter) under Activity_Create (because butStart_Click flags an error).
Sadly it only works once. A second speech attempt fails. This applies also to the original project (on my Nexus-7 2012).

If you do get it to work (on most devices) then I suggest to add an on_partial_results event.
 

Rick Harris

Well-Known Member
Licensed User
Yes, it seems Google has messed up their speech recognizer API since beginning of March 2016. I am having similar problems. Errors 5 and 7 occur nearly always and the API only returns speech RMS values after the first recognition attempt, thus the speech volume indicator in my apps no longer works.
The solution I used was to go back to Google App version 5.8 (february 2016).
These older APK's can be found among other on www.apkmirror.com.
Everything now works fine again on my Nexus-7 (2012) with Android 4.2.
Google App versions 5.10, 5.11 and 5.12 are not OK.
Don't forget to remove the "automatic update" tick in the Google Playstore for the Google App!
Update: There is a crude solution to this problem, namely by adding:
B4X:
Activity_Pause(False)       
Activity_Resume
'SR.startListening(SI) or CallSub(RecognizerService, "StartListening"), depending on which SR library you are using...
just before giving the StartListening command! This will make the speechrecognizer raise the RMS event again (needed to show the speech volume) and it also seems to reduce the number of errors 6, 7 and 8!
I have a feeling that the Google engineers have overlooked the fact that since recently they open a separate activity while the speech is being dictated. After the EndOfSpeech event the spoken text is sent to the calling activity.
 

Rick Harris

Well-Known Member
Licensed User
Update: There is a crude solution to this problem, namely by adding:
B4X:
Activity_Pause(False)      
Activity_Resume
'SR.startListening(SI) or CallSub(RecognizerService, "StartListening"), depending on which SR library you are using...
just before giving the StartListening command! This will make the speechrecognizer raise the RMS event again (needed to show the speech volume) and it also seems to reduce the number of errors 6, 7 and 8!
I have a feeling that the Google engineers have overlooked the fact that since recently they open a separate activity while the speech is being dictated. After the EndOfSpeech event the spoken text is sent to the calling activity.
Update: Since Google released Google App version 6.0 this trick also no longer works. I now find myself advising customers to delete the Google update and go back to the original version (before version 5.08).
 

Rusty

Well-Known Member
Licensed User
Any solutions to this?
I'm having the same problems...
First time works fast and well; second and forward end up with an error 5 client side error...a very long delay and finally a return of my speech/text.
Unusable with the delays/errors.
Thanks,
Rusty
 
Top