B4i Library [class] Voice Recognition using api.ai service

iOS internal voice recognition features are not open to developers. This makes it more difficult to add voice recognition to your apps.

This class captures the microphone input when the user holds the button, sends it to api.ai service and then returns the resolved text:


In order to use this service you need to register with api.ai and set the two keys in the main module.

The code to consume this service is simple:
B4X:
Private Sub Application_Start (Nav As NavigationController)
   NavControl = Nav
   Page1.Initialize("Page1")
   Page1.RootPanel.LoadLayout("Main")
   NavControl.ShowPage(Page1)
   Speak.Initialize(Me, "speak") 'initialize the object
   Speak.AddToPanel(Panel1)  'add it to the layout
End Sub

public Sub Speak_Result(Success As Boolean, Resolved As String)
   If Success Then
     TextView1.Text = $"Resolved: ${Resolved}"$
   Else
     TextView1.Text = $"Error: ${Resolved}"$
   End If
End Sub

Note that the project requires B4i v2.0+
 

Attachments

  • VoiceRecognition.zip
    7 KB · Views: 143

JohnC

Expert
Licensed User
Longtime User
Erel,

Why are there '$' symbols when setting the text?
 

John Woodsmall

Active Member
Licensed User
Longtime User
I am trying to get the "dob" value from the json string that comes back from api.ai:
response success
{
"id": "317d4e1c-dc32-4144-841b-16d2cad7d114",
"timestamp": "2015-08-25T07:58:48.768Z",
"result": {
"source": "agent",
"resolvedQuery": "december eighteenth nineteen fourty",
"speech": "",
"action": "dob",
"parameters": {
"dob": "2015-12-19"
},
"metadata": {
"intentId": "e3ec76de-c450-44ba-b3af-646e032a4a92",
"inputContexts": [],
"outputContexts": [],
"contexts": [],
"intentName": "Date of Birth"
}
},
"status": {
"code": 200,
"errorType": "success"
},
"asr": {
"december eighteenth nineteen fourty": 0.940715,
"december eighth nineteen fourty": 0.9389066,
"december eighth nineteen four": 0.9383301
}
}

how do i extract it?
 

JanPRO

Well-Known Member
Licensed User
Longtime User
Hi, use iJSON:
B4X:
Dim Parser As JSONParser
Dim JSONMap As Map

Parser.Initialize(js)
JSONMap = Parser.NextObject

JSONMap = JSONMap.Get("result")
Dim action As String = JSONMap.Get("action")

Log(action)
 

John Woodsmall

Active Member
Licensed User
Longtime User
note: this web function is no longer working.
it worked for awhile, but sadly is not correctly displaying the voice input.
 

John Woodsmall

Active Member
Licensed User
Longtime User
hi...i did contact them and they said they were having a problem
that they would fix.
the problem has to do with "action" a function under "dob" that i have setup
as an intent. It just returns blank.

Also the text return is just wrong. an example: I say december 8 1942 it
says: 2015-12-08.
I have tried this at least 30 times and I get various results all wrong.

so on to other solutions.

any word on xcode natively doing the voice to text deal?
 

John Woodsmall

Active Member
Licensed User
Longtime User
is there a way to state the intent when calling the api.ai web page?
it seems my "intent" is not working remotely with the program above.
i wonder if is because the "intent" is not being called out.
do you know of a way to include the intent name in the call?

my intent works correctly on their site and produces the birth numbers, but when i call it remotely
it just returns the birth date spelled out...no dob as numbers?
 

John Woodsmall

Active Member
Licensed User
Longtime User
trying to join this up with my main program and I get the error for
SpeakButton unknown type speakbutton are you missing a Lib reference?
I have the libs you had in yours:
icore 2.01
ihttp 1.01
iJson 1.00
iMedia 1.20
iStringutils 1.00
 

valentino s

Active Member
Licensed User
Longtime User
Very interesting, also if you want to know how to send on the web a wav audio file.

Problem: I've created an italian agent and updated the keys with the 2 ones needed.

If I try recognize the speech in english, the agent returns an english text even if it's configured as italian.

If I try to change from en to it, it doesn't work. This is the line from the speakbutton module, line 37, I've customized changing EN to IT.

The timezone added works also in english, it's not relevant.

B4X:
jg.Initialize(CreateMap("v": "20150910", "timezone":"Europe/Paris", "lang": "it", "sessionId": sessionId))

api.ai should reject my request when in "en" mode because my agent keys are for italian language, instead it works in english and it isn't working in italian.

I think I've missed something on their service.

In b4i code I've changed is the two keys and the lang option. There's something else ?


2015-12-02_113733-jpg.39393


This the error:

Application_Start
Application_Active
response success
{
"id": "f944dbe8-2973-47af-96fe-a7a0e157266f",
"timestamp": "2015-12-02T10:47:27.472Z",
"status": {
"code": 400,
"errorType": "bad_request",
"errorDetails": "Could not recognize the text from the voice data."
}
}
Class (b4i_httpjob) instance released.
 

Attachments

  • 2015-12-02_113733.jpg
    2015-12-02_113733.jpg
    24.4 KB · Views: 54
Last edited:

valentino s

Active Member
Licensed User
Longtime User
I'll do it. But their online panel support configuration for italian language (now, some months ago they didn't).
Thank you,

v.

UPDATE:

I can config ... but they only support english and french: "The problem appears because of our current recognition engine is not support Italian language.Italian recognition is in the plans, but not nearest…"

https://support.api.ai/customer/portal/questions/14364988-unity-api-doesn-t-work-for-italian


FINALLY:

Perhaps Google can help me: https://gist.github.com/alotaiba/1730160

There's also: https://github.com/gillesdemey/google-speech-v2


No: google removed the speech api.
 
Last edited:
Top