B4i Library [class] Voice Recognition using api.ai service

iOS internal voice recognition features are not open to developers. This makes it more difficult to add voice recognition to your apps.

This class captures the microphone input when the user holds the button, sends it to api.ai service and then returns the resolved text:


In order to use this service you need to register with api.ai and set the two keys in the main module.

The code to consume this service is simple:

Private Sub Application_Start (Nav As NavigationController)
   NavControl = Nav
   Page1.Initialize("Page1")
   Page1.RootPanel.LoadLayout("Main")
   NavControl.ShowPage(Page1)
   Speak.Initialize(Me, "speak") 'initialize the object
   Speak.AddToPanel(Panel1)  'add it to the layout
End Sub

public Sub Speak_Result(Success As Boolean, Resolved As String)
   If Success Then
     TextView1.Text = $"Resolved: ${Resolved}"$
   Else
     TextView1.Text = $"Error: ${Resolved}"$
   End If
End Sub

Note that the project requires B4i v2.0+

 

[JohnC]

Erel,

Why are there '$' symbols when setting the text?

 

[Erel]

[B4X] Smart String Literal

 

[John Woodsmall]

I am trying to get the "dob" value from the json string that comes back from api.ai:
response success
{
"id": "317d4e1c-dc32-4144-841b-16d2cad7d114",
"timestamp": "2015-08-25T07:58:48.768Z",
"result": {
"source": "agent",
"resolvedQuery": "december eighteenth nineteen fourty",
"speech": "",
"action": "dob",
"parameters": {
"dob": "2015-12-19"
},
"metadata": {
"intentId": "e3ec76de-c450-44ba-b3af-646e032a4a92",
"inputContexts": [],
"outputContexts": [],
"contexts": [],
"intentName": "Date of Birth"
}
},
"status": {
"code": 200,
"errorType": "success"
},
"asr": {
"december eighteenth nineteen fourty": 0.940715,
"december eighth nineteen fourty": 0.9389066,
"december eighth nineteen four": 0.9383301
}
}

how do i extract it?

 

[JanPRO]

Hi, use iJSON:

Dim Parser As JSONParser
Dim JSONMap As Map

Parser.Initialize(js)
JSONMap = Parser.NextObject

JSONMap = JSONMap.Get("result")
Dim action As String = JSONMap.Get("action")

Log(action)

 

[John Woodsmall]

note: this web function is no longer working.
it worked for awhile, but sadly is not correctly displaying the voice input.

 

[Erel]

Which error do you get? Have you tried to contact their support?

 

[John Woodsmall]

hi...i did contact them and they said they were having a problem
that they would fix.
the problem has to do with "action" a function under "dob" that i have setup
as an intent. It just returns blank.

Also the text return is just wrong. an example: I say december 8 1942 it
says: 2015-12-08.
I have tried this at least 30 times and I get various results all wrong.

so on to other solutions.

any word on xcode natively doing the voice to text deal?

 

[Erel]

There is no native support for voice recognition in iOS (exposed to developers).

 

[John Woodsmall]

is there a way to state the intent when calling the api.ai web page?
it seems my "intent" is not working remotely with the program above.
i wonder if is because the "intent" is not being called out.
do you know of a way to include the intent name in the call?

my intent works correctly on their site and produces the birth numbers, but when i call it remotely
it just returns the birth date spelled out...no dob as numbers?

 

[Erel]

I think that you should ask them this question.

 

[John Woodsmall]

trying to join this up with my main program and I get the error for
SpeakButton unknown type speakbutton are you missing a Lib reference?
I have the libs you had in yours:
icore 2.01
ihttp 1.01
iJson 1.00
iMedia 1.20
iStringutils 1.00

 

[Erel]

Please start a new thread for this question.

 

[valentino s]

Very interesting, also if you want to know how to send on the web a wav audio file.

Problem: I've created an italian agent and updated the keys with the 2 ones needed.

If I try recognize the speech in english, the agent returns an english text even if it's configured as italian.

If I try to change from en to it, it doesn't work. This is the line from the speakbutton module, line 37, I've customized changing EN to IT.

The timezone added works also in english, it's not relevant.

jg.Initialize(CreateMap("v": "20150910", "timezone":"Europe/Paris", "lang": "it", "sessionId": sessionId))

api.ai should reject my request when in "en" mode because my agent keys are for italian language, instead it works in english and it isn't working in italian.

I think I've missed something on their service.

In b4i code I've changed is the two keys and the lang option. There's something else ?

This the error:

Application_Start
Application_Active
response success
{
"id": "f944dbe8-2973-47af-96fe-a7a0e157266f",
"timestamp": "2015-12-02T10:47:27.472Z",
"status": {
"code": 400,
"errorType": "bad_request",
"errorDetails": "Could not recognize the text from the voice data."
}
}
Class (b4i_httpjob) instance released.

 

[Erel]

I'm not sure that they support Italian. It is worth asking their support group.

 

[valentino s]

I'll do it. But their online panel support configuration for italian language (now, some months ago they didn't).
Thank you,

v.

UPDATE:

I can config ... but they only support english and french: "The problem appears because of our current recognition engine is not support Italian language.Italian recognition is in the plans, but not nearest…"

https://support.api.ai/customer/portal/questions/14364988-unity-api-doesn-t-work-for-italian


FINALLY:

Perhaps Google can help me: https://gist.github.com/alotaiba/1730160

There's also: https://github.com/gillesdemey/google-speech-v2

No: google removed the speech api.

 

[Erel]

Yes. It should be simple to take the code from SpeakButton class and port it to B4A.

Make sure that api.ai hasn't changed their API.