B4A Library [Chargeable] SpeechRecognizer Library

SpeechRecognizer




I have received a lot of requests lately for my SpeechRecognizer library.
This is why I made a separate topic for SpeechRecognizer.
With SpeechRecognizer, you'll be able to recognize text from a service! No popups or anything else!

A2ae6l.png


Requirements:
- Android 2.1+ (API 8)



What will you get from me:
- The library files
- A sample project
- A readme file. (Disclaimer.)


Payment

The price of the library is 15 Euros.
Payment is done through PayPal.
If you are interested in the library, you can send me a private message here on the forums or send me an email to rootsoftllc@gmail.com.

Sample

You can find a sample here written by NJDude:
http://www.basic4ppc.com/forum/andr...eelance-library-development-6.html#post134332


Special thanks and credits goes to NJDude for the sample.

Kind regards,
Tomas
 
Last edited:

Kevin

Well-Known Member
Licensed User
Excellent stuff! Does this require the latest update to his lib? I bought it a while back but wound up not using it because of the limitations that the past few posts here seem to fix. I tried PMing Tomas several days ago but haven't yet heard from him.

Tomas: If you see this message, please find my PM with my email address inside. :)
 

Rick Harris

Well-Known Member
Licensed User
in the official documentation it speak of activity "Starts an activity that will prompt the user for speech without...."

you bieleve that is possible to run also into a service module?
Yes, I have already done it with the code provided in thread #37. It is working great.
 

Rick Harris

Well-Known Member
Licensed User
Excellent stuff! Does this require the latest update to his lib? I bought it a while back but wound up not using it because of the limitations that the past few posts here seem to fix. I tried PMing Tomas several days ago but haven't yet heard from him.

Tomas: If you see this message, please find my PM with my email address inside. :)
I have had similar problems and frustrations and nearly abandoned his library due to lacking features like language selection and VU meter implementation. In the end it turned out that these features only required half a dozen of additional lines of code in the SpeechLibrary bas module. I wasted two working days trying to resolve all problems... Tomas is busy with his studies and it usually takes nearly a week before he responds. I have written to him in detail and, among other, have requested/advised him to give better support if he continues to sell code via this forum.
Only Tomas can answer your question, but I did read somewhere that somebody recently received an upgrade from him. No idea whether that only involved the module (in which case you can use the code provided by me) or that the Java library was changed as well.
 

Kevin

Well-Known Member
Licensed User
I suppose it depends on whether or not the library I have includes the "CreateRecognizeIntent2" method. I can check later when I have a chance. Hopefully I can get an updated library soon.
 

Rick Harris

Well-Known Member
Licensed User
I suppose it depends on whether or not the library I have includes the "CreateRecognizeIntent2" method. I can check later when I have a chance. Hopefully I can get an updated library soon.
Yes I can confirm that the library version I received from Tomas does have a CreateRecognizeIntent2 method.
No idea what I can use it for. The documentation is rather Spartanic...
 

Kevin

Well-Known Member
Licensed User
Just checked my lib and I do have this function! :D

Hopefully I'll get some time to try this again in a few days. I wanted it to constantly listen for a 'keyword' and always be ready/listening... I previously didn't think it was possible but it's certainly looking possible now.
 

mcmanu

Active Member
Licensed User
Fortunately No!! You should ignore the SpeechRecognition_EndofSpeech event. As long as this handsfree mode is active, the SpeechRecognition_Results event keeps firing after something has been spoken and Google Voice detects a brief moment of silence. For that reason I also inhibit Stop_listening which is called by some parts of my software in non-handsfree mode. Also in this mode, no Google Voice bleep tones are heared while the handsfree mode is active (except once when the function is started).
This is also an ideal mode to keep the microphone open all the time. No voice activated microphone on/off switch is needed (which I had to use before)! Only after a keyword is said, like "Computer", or "James" will my software respond by doing something and then continues listening for the next occasion whereby the keyword is spoken, i.e. all speech in the room is simply ignored unless the keyword is spoken. I make software for blind and visually impaired users. This is a very welcome new feature.



I dont get it work, it listens the first time and then nothing happens anymore.
Do you have a sample project?

my code


B4X:
Sub Service_Create


    sr.Initialize("SpeechRecognition")
    isreginitialized=True
    i.Initialize("android.speech.action.VOICE_SEARCH_HANDS_FREE", "")
    i.PutExtra("android.speech.extra.LANGUAGE_MODEL", "free_form")
    i.PutExtra("calling_package", "test.package")
    i.PutExtra("android.speech.extra.MAX_RESULTS", 5)
    i.PutExtra("android.speech.extra.LANGUAGE", "de")
    obj= sr.CreateRecognizeIntent2(i)

End Sub

Sub Service_Start (StartingIntent As Intent)
StartingIntent.Action="android.speech.action.VOICE_SEARCH_HANDS_FREE"
StartListening
End Sub

Sub Service_Destroy
End Sub

Sub StartListening
    sr.StartListening(obj)
    Log("----- Started -----")
End Sub

Sub StopListening
    sr.StopListening
    isreginitialized=False
End Sub
 

Rick Harris

Well-Known Member
Licensed User
I dont get it work, it listens the first time and then nothing happens anymore.
Do you have a sample project?

my code


B4X:
Sub Service_Create
 
 
    sr.Initialize("SpeechRecognition")
    isreginitialized=True
    i.Initialize("android.speech.action.VOICE_SEARCH_HANDS_FREE", "")
    i.PutExtra("android.speech.extra.LANGUAGE_MODEL", "free_form")
    i.PutExtra("calling_package", "test.package")
    i.PutExtra("android.speech.extra.MAX_RESULTS", 5)
    i.PutExtra("android.speech.extra.LANGUAGE", "de")
    obj= sr.CreateRecognizeIntent2(i)
 
End Sub
 
Sub Service_Start (StartingIntent As Intent)
StartingIntent.Action="android.speech.action.VOICE_SEARCH_HANDS_FREE"
StartListening
End Sub
 
Sub Service_Destroy
End Sub
 
Sub StartListening
    sr.StartListening(obj)
    Log("----- Started -----")
End Sub
 
Sub StopListening
    sr.StopListening
    isreginitialized=False
End Sub
Don't put anything inder Service_Start and trigger Start_listening from within your source code. Also Make sure that Stop_listening is not called from within your code anywhere. Also take a closer look at the code I provided. Service_create is wrong too. Use this (entire contents of the BAS module):
B4X:
Sub Process_Globals
    'These global variables will be declared once when the application starts.
    'These variables can be accessed from all modules.
   
    'Declares the SpeechRecognizer; You should be able to call this in the activity too.
    Dim sr As SpeechRecognizer
    Dim obj As Object
End Sub
 
Sub Service_Create
    'Initializing the speech recogniter. We are using a default intent.
    'You can create your own intent with the desired parameters from :
    'http://developer.android.com/reference/android/speech/RecognizerIntent.html
    sr.Initialize("SpeechRecognition")
    obj = sr.CreateRecognizerIntent(sr.LANGUAGE_MODEL_FREE_FORM, "voice.recognition.test", 5)
    'If sr.isRecognitionAvailable = True  Then ToastMessageShow("Spraakherkenning is beschikbaar! ", False)
    'If sr.isRecognitionAvailable = False Then ToastMessageShow("Spraakherkenning is NIET beschikbaar! ", False)
End Sub
 
Sub Service_Start (StartingIntent As Intent)
End Sub
 
Sub Service_Destroy
End Sub
 
Sub StartListening            'Door mij aangepast n.a.v. mailtje Tomas Verhelst
    Dim i As Intent
    'Initialize intent door middel van constants van de api
    'http://developer.android.com/reference/android/speech/RecognizerIntent.html
   
    If Main.MicAuto1 = 0 Then i.Initialize("android.speech.action.RECOGNIZE_SPEECH", "")
    If Main.MicAuto1 = 1 Then i.Initialize("android.speech.action.VOICE_SEARCH_HANDS_FREE", "")  'In plaats van ...action.RECOGNIZE_SPEECH  !!
    i.PutExtra("android.speech.extra.LANGUAGE_MODEL", "free_form")
    i.PutExtra("calling_package", "your package name")
    ' i.PutExtra("android.speech.extras.SPEECH_INPUT_COMPLETE_SILENCE_LENGTH_MILLIS",500)  'Heb ik toegevoegd
    i.PutExtra("android.speech.extra.MAX_RESULTS", 5)
    i.PutExtra("android.speech.extra.LANGUAGE", "nl")
    sr.StartListening(sr.CreateRecognizeIntent2(i))
End Sub
 
Sub StopListening
    If Main.micauto1=1 Then Return
    sr.StopListening
    Log("----- Stopped -----")
End Sub
 
'Beginning of the speech.
Sub SpeechRecognition_ReadyForSpeech
    Log("Ready for Speech")
    'ToastMessageShow("Zeg iets alsjeblief", False)
End Sub
 
'Beginning of the speech.
Sub SpeechRecognition_BeginningOfSpeech
    Log("Spraak begin")
End Sub
 
'End of the speech.
Sub SpeechRecognition_EndOfSpeech
    Log("End of Speech")
    'ToastMessageShow("Gereed!", False)
End Sub
 
'Buffer received
Sub SpeechRecognition_BufferReceived (buffer() As Byte)
    Log("Buffer received")
End Sub
 
'An error has occured.
Sub SpeechRecognition_Error (ErrorType As Int)
    Log("Error: " & ErrorType)
    CallSub2(Main, "ShowError", ErrorType)    'Dit heb ik toegevoegd!
End Sub
 
'An event has been done
Sub SpeechRecognition_Event (EventType As Int)
    Log("Event: " & EventType)
End Sub
 
'PartialResults arrived
Sub SpeechRecognition_PartialResults
    Log("PartialResults arrived")
End Sub
 
'The Results has arrived.
Sub SpeechRecognition_Results (Words As List)
    Log("RESULTS: " & Words)
    CallSub2(Main, "ShowWords", Words)
End Sub
 
'Rms has changed
Sub SpeechRecognition_RmsChanged (rms As Float)
    Log("RMS: " & rms)
    CallSub2(Main, "ShowVU", rms)    'Dit heb ik toegevoegd!
End Sub
 

Kevin

Well-Known Member
Licensed User
I finally got around to playing with this a bit. The only issue I have is that occasionally an error event fires and it seems to kill the SR "service", meaning that my log stops showing any further SR events such as RMS levels.

I take it this is normal and the only recourse is to call StartListening again? Sometimes that doesn't seem to restart it though. o_O

EDIT: I'm not sure I want it to work this way, but is there a way for it to only respond after a keyword? For example, I have an app named Carma and it only wakes up if I say Carma. No beeping or anything else in the meantime. When I say "carma", it says "Yes" and then I hear a beep waiting for a command. Really, I was hoping to just listen for commands and ignore anything it doesn't recognize but it just keeps erroring out (i.e. No speech input, etc) after which I need to restart it and then it beeps again.

By the way, for anyone interested I have added error descriptions to the error sub:

B4X:
Sub SpeechRecognition_Error (ErrorType As Int)
        Dim sErrorMsg As String
      
        Select Case ErrorType
            Case 1
              sErrorMsg = "Network operation timed out"
            Case 2
              sErrorMsg = "Network error"
            Case 3
              sErrorMsg = "Audio recording error"
            Case 4
              sErrorMsg = "Server sends error status"
            Case 5
              sErrorMsg = "Client-side error"
            Case 6
                sErrorMsg = "No speech input"
            Case 7
                sErrorMsg = "No recognition result matched"
            Case 8
                sErrorMsg = "RecognitionService busy"
            Case 9
                sErrorMsg = "Insufficient permissions"
            Case Else
                sErrorMsg = "Unknown error"
        End Select

        'StartListening  ' Do this here?
    Log("VR Error: " & ErrorType & " - " & sErrorMsg)
    'CallSub2(Main, "ShowError", ErrorType)    ' Currently not using this part
End Sub
 
Last edited:

Rick Harris

Well-Known Member
Licensed User
I finally got around to playing with this a bit. The only issue I have is that occasionally an error event fires and it seems to kill the SR "service", meaning that my log stops showing any further SR events such as RMS levels.

I take it this is normal and the only recourse is to call StartListening again? Sometimes that doesn't seem to restart it though. o_O

EDIT: I'm not sure I want it to work this way, but is there a way for it to only respond after a keyword? For example, I have an app named Carma and it only wakes up if I say Carma. No beeping or anything else in the meantime. When I say "carma", it says "Yes" and then I hear a beep waiting for a command. Really, I was hoping to just listen for commands and ignore anything it doesn't recognize but it just keeps erroring out (i.e. No speech input, etc) after which I need to restart it and then it beeps again.

By the way, for anyone interested I have added error descriptions to the error sub:

B4X:
Sub SpeechRecognition_Error (ErrorType As Int)
        Dim sErrorMsg As String
 
        Select Case ErrorType
            Case 1
              sErrorMsg = "Network operation timed out"
            Case 2
              sErrorMsg = "Network error"
            Case 3
              sErrorMsg = "Audio recording error"
            Case 4
              sErrorMsg = "Server sends error status"
            Case 5
              sErrorMsg = "Client-side error"
            Case 6
                sErrorMsg = "No speech input"
            Case 7
                sErrorMsg = "No recognition result matched"
            Case 8
                sErrorMsg = "RecognitionService busy"
            Case 9
                sErrorMsg = "Insufficient permissions"
            Case Else
                sErrorMsg = "Unknown error"
        End Select
 
        'StartListening  ' Do this here?
    Log("VR Error: " & ErrorType & " - " & sErrorMsg)
    'CallSub2(Main, "ShowError", ErrorType)    ' Currently not using this part
End Sub
Yes Errors do occur, especially 6 and 7. In your Main module you can intercept errors with 'ShowError' sub.
Simply retrigger Start_listening when these errors occur.
To use a Keyword, define a Proces Global: Dim ListenOnce as Int. Set it to 0. In the sub 'Showwords' do the following:
B4X:
Sub ShowWords(Words1 AsList)
  Dim MyWord1 as string
  MyWord1 = Words1.get(0)
  If ListenOnce = 1 then
    If MyWord1 = "Hello"  then ....  'do something
    If MyWord1 = "Goodbye"  then ... 'do something else
    ListenOnce = 0
    return
  end if
  If MyWord1= "carma" and ListenOnce=0 then
    ListenOnce =1         'This will force this sub to process what will be spoken the next time
    'You can add a beep sound here too
  end if
end sub
 
Last edited:

Kevin

Well-Known Member
Licensed User
I meant to listen for a keyword before it even beeps and starts the service, although the more I think of it this is outside of the scope of this library. A really good example would be the newer Google Search that has been out for a while on Android. When you open it, it presents you with a search box that you can type into as well as your "cards". You can also do a voice search by saying "Okay Google". It is only after you say that phrase that the phone beeps (the start of the speech recognition service). You can say anything else or even just "Google" but it is absolutely silent and does nothing until you say the full phrase or at least something very close... (Funny story: the first time I found out about this was when I was looking at my "cards" and there was some kind of glitch and I said "F**k*n' Google!", at which time the phone beeped awaiting my voice search. :rolleyes:)

I'm guessing that how they did it is to constantly keep the microphone turned on and recording the last few seconds of audio and repeatedly analyze it (locally as opposed to sending it to a server) to look for a matching sound pattern. When it finds a match, then it triggers the speech recognition service.

That is more what I had in mind. I think this could probably be done using the audio recording library and somehow comparing the incoming audio to a pre-recorded file containing the keyword. Of course this would have language limitations unless it was something somewhat universally pronounced such as "okay" and "Google". :D

It could just be that I need to re-think how I want to do speech recognition in my app. I had envisioned it always listening but that constant beeping gets old after about 30 seconds. So instead I guess I will have to have the user press a button in order to start listening for a command unless I can get the above scenario working (similar to "okay Google").
 

mcmanu

Active Member
Licensed User
Thank you Rick Harris, i will try it today ;) maybe there is a Stop_listening in my code ;)
 

Rick Harris

Well-Known Member
Licensed User
I meant to listen for a keyword before it even beeps and starts the service, although the more I think of it this is outside of the scope of this library. A really good example would be the newer Google Search that has been out for a while on Android. When you open it, it presents you with a search box that you can type into as well as your "cards". You can also do a voice search by saying "Okay Google". It is only after you say that phrase that the phone beeps (the start of the speech recognition service). You can say anything else or even just "Google" but it is absolutely silent and does nothing until you say the full phrase or at least something very close... (Funny story: the first time I found out about this was when I was looking at my "cards" and there was some kind of glitch and I said "F**k*n' Google!", at which time the phone beeped awaiting my voice search. :rolleyes:)

I'm guessing that how they did it is to constantly keep the microphone turned on and recording the last few seconds of audio and repeatedly analyze it (locally as opposed to sending it to a server) to look for a matching sound pattern. When it finds a match, then it triggers the speech recognition service.

That is more what I had in mind. I think this could probably be done using the audio recording library and somehow comparing the incoming audio to a pre-recorded file containing the keyword. Of course this would have language limitations unless it was something somewhat universally pronounced such as "okay" and "Google". :D

It could just be that I need to re-think how I want to do speech recognition in my app. I had envisioned it always listening but that constant beeping gets old after about 30 seconds. So instead I guess I will have to have the user press a button in order to start listening for a command unless I can get the above scenario working (similar to "okay Google").
Yes I understand what you want to do. I think that the method whereby you would try to compare a spoken keyword with a pre-recorded file won't work reliably. Spoken words will never match due to variations in volume, intonation, pitch, speed etc. Neither will it be voice-independent.
I assume that Google is listening constantly for somebody to say "Okay Google". It is unclear to me whether this is a local feature or handled on line. If the latter is the case then this implies that the microphone is constantly open and sending everything it hears to Google. Nice feature for the US government! Google does have an offline voice recognition service (in case no Internet is available), but it is not on by default. It has to be switched on via the Android "Language and input" settings.
There is no need for the user to press a button. Simply add Start_listening under Activity_Create or start it with a timer or use sound detection (using the audiorecorder library).
 

Kevin

Well-Known Member
Licensed User
There is no need for the user to press a button. Simply add Start_listening under Activity_Create or start it with a timer or use sound detection (using the audiorecorder library).

I like the idea of just listening for sound using the audio recorder library. It's so obvious I'm embarrassed to have not thought of it myself! Perhaps if I feel really adventurous I will experiment with keyword detection.

I like to think that the Google Search app is only listening locally for the "okay Google", because I have thought about that as well - the possibility of it sending everything it hears to a Google server. Not a very warm and fuzzy feeling, to say the least! If I recall correctly, that app is excessively large to download, so it is conceivable that they could have included whatever is necessary to detect that phrase in varying forms inside the app itself. One reason I feel this way is because it only beeps after saying the phrase. But Google being Google, I'm sure there are plenty of private functions that only they can use to circumvent how things may normally work in most apps. With that in mind, they could certainly be using a special method of the speech recognition service that does not beep. I think one way to find out for sure would be to run Wireshark and watch all of the traffic involved with the phone before and during the use of the Google search app.
 

Rick Harris

Well-Known Member
Licensed User
Thank you Rick Harris, i will try it today ;) maybe there is a Stop_listening in my code ;)
Here is a trick to suppress Google Voice's Beep sound that is generated when StartListening needs to be re-triggered after Error 6 or Error 7 in order to maintain a continuous handsfree listening mode.
B4X:
Sub ShowError(Error1 As Int)
    Dim Volu1 As Int
    Dim Wticks1 as long
    Dim Wticks2 as long
    If Error1 = 6 OR Error1 = 7 Then
        Dim p As Phone              'Suppress Google Voice Beep
        Volu1 = p.GetVolume(p.VOLUME_MUSIC)
        p.SetVolume(p.VOLUME_MUSIC, 0, False)
        CallSub(RecognizerService, "StartListening")
        WTicks1 = DateTime.Now
        WTicks2 = 0
        Do While WTicks2 < (WTicks1+800)
          WTicks2 = DateTime.Now
          DoEvents
        Loop
        p.SetVolume(p.VOLUME_MUSIC, Volu1, False)
      End If
   End Sub

In Tomas Verhelst's SpeechRecognizer module you should add the following error handler:
B4X:
'An error has occured.
Sub SpeechRecognition_Error (ErrorType As Int)
    Log("Error: " & ErrorType)
    CallSub2(Main, "ShowError", ErrorType) 
End Sub
The solution is a bit crude because it turns down the loudspeaker volume to 0 for 800 milliseconds and then restores it to its original volume. A similar routine can be used directly after calling StartListening in order to suppress the initial Google Voice beep.
 
Last edited:

Kevin

Well-Known Member
Licensed User
I can't look at your code right now but you likely need to call StartListening again. This is the case after speech is processed or when some (or any?) errors occur.

This was my complaint about the constant beeping when restarting the recognition service but a work-around has been posted above for that.
 
Top