B4i Library [class] Voice Recognition using api.ai service

Discussion in 'iOS Libraries' started by Erel, Jul 6, 2015.

  1. Erel

    Erel Administrator Staff Member Licensed User

    iOS internal voice recognition features are not open to developers. This makes it more difficult to add voice recognition to your apps.

    This class captures the microphone input when the user holds the button, sends it to api.ai service and then returns the resolved text:



    In order to use this service you need to register with api.ai and set the two keys in the main module.

    The code to consume this service is simple:
    Code:
    Private Sub Application_Start (Nav As NavigationController)
       NavControl = Nav
       Page1.Initialize(
    "Page1")
       Page1.RootPanel.LoadLayout(
    "Main")
       NavControl.ShowPage(Page1)
       Speak.Initialize(Me, 
    "speak"'initialize the object
       Speak.AddToPanel(Panel1)  'add it to the layout
    End Sub

    public Sub Speak_Result(Success As Boolean, Resolved As String)
       
    If Success Then
         TextView1.Text = 
    $"Resolved: ${Resolved}"$
       
    Else
         TextView1.Text = 
    $"Error: ${Resolved}"$
       
    End If
    End Sub
    Note that the project requires B4i v2.0+
     

    Attached Files:

  2. JohnCody

    JohnCody Active Member Licensed User

    Erel,

    Why are there '$' symbols when setting the text?
     
  3. Erel

    Erel Administrator Staff Member Licensed User

  4. John Woodsmall

    John Woodsmall Active Member Licensed User

    I am trying to get the "dob" value from the json string that comes back from api.ai:
    response success
    {
    "id": "317d4e1c-dc32-4144-841b-16d2cad7d114",
    "timestamp": "2015-08-25T07:58:48.768Z",
    "result": {
    "source": "agent",
    "resolvedQuery": "december eighteenth nineteen fourty",
    "speech": "",
    "action": "dob",
    "parameters": {
    "dob": "2015-12-19"
    },
    "metadata": {
    "intentId": "e3ec76de-c450-44ba-b3af-646e032a4a92",
    "inputContexts": [],
    "outputContexts": [],
    "contexts": [],
    "intentName": "Date of Birth"
    }
    },
    "status": {
    "code": 200,
    "errorType": "success"
    },
    "asr": {
    "december eighteenth nineteen fourty": 0.940715,
    "december eighth nineteen fourty": 0.9389066,
    "december eighth nineteen four": 0.9383301
    }
    }

    how do i extract it?
     
  5. JanPRO

    JanPRO Well-Known Member Licensed User

    Hi, use iJSON:
    Code:
    Dim Parser As JSONParser
    Dim JSONMap As Map

    Parser.Initialize(js)
    JSONMap = Parser.NextObject

    JSONMap = JSONMap.Get(
    "result")
    Dim action As String = JSONMap.Get("action")

    Log(action)
     
  6. John Woodsmall

    John Woodsmall Active Member Licensed User

    note: this web function is no longer working.
    it worked for awhile, but sadly is not correctly displaying the voice input.
     
  7. Erel

    Erel Administrator Staff Member Licensed User

    Which error do you get? Have you tried to contact their support?
     
  8. John Woodsmall

    John Woodsmall Active Member Licensed User

    hi...i did contact them and they said they were having a problem
    that they would fix.
    the problem has to do with "action" a function under "dob" that i have setup
    as an intent. It just returns blank.

    Also the text return is just wrong. an example: I say december 8 1942 it
    says: 2015-12-08.
    I have tried this at least 30 times and I get various results all wrong.

    so on to other solutions.

    any word on xcode natively doing the voice to text deal?
     
  9. Erel

    Erel Administrator Staff Member Licensed User

    There is no native support for voice recognition in iOS (exposed to developers).
     
  10. John Woodsmall

    John Woodsmall Active Member Licensed User

    is there a way to state the intent when calling the api.ai web page?
    it seems my "intent" is not working remotely with the program above.
    i wonder if is because the "intent" is not being called out.
    do you know of a way to include the intent name in the call?

    my intent works correctly on their site and produces the birth numbers, but when i call it remotely
    it just returns the birth date spelled out...no dob as numbers?
     
  11. Erel

    Erel Administrator Staff Member Licensed User

    I think that you should ask them this question.
     
  12. John Woodsmall

    John Woodsmall Active Member Licensed User

    trying to join this up with my main program and I get the error for
    SpeakButton unknown type speakbutton are you missing a Lib reference?
    I have the libs you had in yours:
    icore 2.01
    ihttp 1.01
    iJson 1.00
    iMedia 1.20
    iStringutils 1.00
     
  13. Erel

    Erel Administrator Staff Member Licensed User

    Please start a new thread for this question.
     
  14. valentino s

    valentino s Active Member Licensed User

    Very interesting, also if you want to know how to send on the web a wav audio file.

    Problem: I've created an italian agent and updated the keys with the 2 ones needed.

    If I try recognize the speech in english, the agent returns an english text even if it's configured as italian.

    If I try to change from en to it, it doesn't work. This is the line from the speakbutton module, line 37, I've customized changing EN to IT.

    The timezone added works also in english, it's not relevant.

    Code:
    jg.Initialize(CreateMap("v""20150910""timezone":"Europe/Paris""lang""it""sessionId": sessionId))
    api.ai should reject my request when in "en" mode because my agent keys are for italian language, instead it works in english and it isn't working in italian.

    I think I've missed something on their service.

    In b4i code I've changed is the two keys and the lang option. There's something else ?


    [​IMG]

    This the error:

    Application_Start
    Application_Active
    response success
    {
    "id": "f944dbe8-2973-47af-96fe-a7a0e157266f",
    "timestamp": "2015-12-02T10:47:27.472Z",
    "status": {
    "code": 400,
    "errorType": "bad_request",
    "errorDetails": "Could not recognize the text from the voice data."
    }
    }
    Class (b4i_httpjob) instance released.
     

    Attached Files:

    Last edited: Dec 2, 2015
  15. Erel

    Erel Administrator Staff Member Licensed User

    I'm not sure that they support Italian. It is worth asking their support group.
     
  16. valentino s

    valentino s Active Member Licensed User

    I'll do it. But their online panel support configuration for italian language (now, some months ago they didn't).
    Thank you,

    v.

    UPDATE:

    I can config ... but they only support english and french: "The problem appears because of our current recognition engine is not support Italian language.Italian recognition is in the plans, but not nearest…"

    https://support.api.ai/customer/portal/questions/14364988-unity-api-doesn-t-work-for-italian


    FINALLY:

    Perhaps Google can help me: https://gist.github.com/alotaiba/1730160

    There's also: https://github.com/gillesdemey/google-speech-v2


    No: google removed the speech api.
     
    Last edited: Dec 3, 2015
  17. Syd Wright

    Syd Wright Active Member Licensed User

    Is it possible to make something similar for B4A (i.e. to access api.ai) ?
     
  18. Erel

    Erel Administrator Staff Member Licensed User

    Yes. It should be simple to take the code from SpeakButton class and port it to B4A.

    Make sure that api.ai hasn't changed their API.
     
    Syd Wright likes this.
  19. Syd Wright

    Syd Wright Active Member Licensed User

    Yes, that's the problem with Google nowadays. They keep changing API's.
     
Loading...