Android Question How to use Google Actions with B4A

Syd Wright

Well-Known Member
Licensed User
Google allows its Home Assistant to interact with other apps, such that the app can perform certain tasks or answer spoken questions given via the Google Assistent.
For this Google uses so called "Actions".
See: https://developers.google.com/assistant/app/overview
Has anyone tried to create such an interaction using B4A? Is there any B4A example to get started?
Thanks in advance.

PS
I know this question was asked in the Forum 2 years ago by "Aaronk", but nobody has responded. In the mean time, Google Home devices have become very popular.
 

DonManfred

Expert
Licensed User
If i understand it correctly one only needs to define the actions.xml to support it.
You find more details on how to build this XML in the Link you provided. This and the following sublinks are interesting in this regard.
I´m positive that you can find an example project on github. At least you need to understand the actions.xml.
The rest is handling the startingintent.
 

Syd Wright

Well-Known Member
Licensed User
If i understand it correctly one only needs to define the actions.xml to support it.
You find more details on how to build this XML in the Link you provided. This and the following sublinks are interesting in this regard.
I´m positive that you can find an example project on github. At least you need to understand the actions.xml.
The rest is handling the startingintent.
Thank you. You've hit the nail on the head because these are precisely the issues that I don't understand enough. Ofcourse I did read the Google information in the link, but I don't fully understand the actions.xml nor how to handle the starting intent. Is this intent accompanied by "extras" that need to be handled? How does my app determine which question or command was raised via the Google Assistant or via a Google Home device? etc.

What I want is to be able to extend the Google Assistant by letting my app answer a large range of questions. For instance if the assistant replies that no answer can be given, I would like the user to say for example: "What can [my App name] tell me about [this]?" or "What does [my App name] know about [this]?"
The word "this" would be sufficient if somehow the original question by the user to the Google Assistant can be passed on to my app (such that my app can analyse it).

I already have a very elaborate personal assistant app that I started in 2007 for blind people (in Dutch and English). In the mean time, Google, Siri and Bixby have overtaken me. "If you can't beat them, then join them" is the saying...
 

DonManfred

Expert
Licensed User

Syd Wright

Well-Known Member
Licensed User
Maybe use Android Studio to use the Plugin.
It is surprising that so few B4A developers are engaged in Google Assistant or Artificial Intelligence in general. This also applies to Cortana, Bixby and Houndify solutions with B4A. I was hoping to avoid using Android Studio which I have never used before. Thank you anyhow for your thoughts.
 

Syd Wright

Well-Known Member
Licensed User
I´ll try to build a small example (have bought a Google thingy recently). Unfortunately time is what i am missing...
Great! I'm looking forward to what you might be able to create. Did you buy a Google Home mini? I also have one of those. It is quite clear that such a Google Home shares data with the Google Assistant app on my tablet that I have running in parallel to the Mini (i.e. some kind of synchronisation). If the Google Home mini is switched on then the tablet does not respond to "Hey Google". Maybe there is a way to intercept this data-interchange.
 

DonManfred

Expert
Licensed User
How does my app determine which question or command was raised via the Google Assistant or via a Google Home device?
through the definitions in the actions.xml.

you setup, for example, your app to extends the excercise feature. You define actions for start excercise together with the type you are using. Running, Walking, ect.
the user tells google to start a excercise in your app. Your app then gets started with the intent for actions.intent.START_EXERCISE
As per definition your app must handle

Parameters your app must handle
exercise.name

so exercise.name is part of the intent.
Possible are
exercise.name (Aerobics, Bench Press, Biking, Burpee, Climbing, Cross Fit, Crunch, Hiking, Kayaking, Lunge, Other, Plank, Pull-up, Push-up, Rowing, Running, Sit-up, Skiing, Snow boarding, Squat, Stretching, Surfing, Swimming, Walking, Yoga)

There are some build-in intents. You need to implement one or more of these Intents.
I would like the user to say for example: "What can [my App name] tell me about [this]?" or "What does [my App name] know about [this]?"
As per my understanding you are not able to build such a function. You are limited to the built-in intents which you need to implement (or not).

Plus the ability to start your app with a specific feature using the built-in intent actions.intent.OPEN_APP_FEATURE. But i guess you can not use any additional queries to this intent. I may be wrong though.
 

DonManfred

Expert
Licensed User
The big problem i have while trying to build an example.
I do not have any app on playstore.
But this is a requirement for these actions to appear.
Google extracts the Info (actions.xml) when you upload your app to the playstore.

So i did not come further on this as of now. I was too lazy to prepare an app on the Playstore console, upload images, setup description and so on.
I did it once in the past (2013 :D) but i don´t know if it has changed. I´m not familar with it.

Back to the actions. I found an Video describing to build actions using api.ai (resp. DialogFlow). Here one can setup everything using an web-ui. Not sure if this is another implementation-path....
 

Syd Wright

Well-Known Member
Licensed User
Google extracts the Info (actions.xml) when you upload your app to the playstore.
I did not know that. This makes building an app even less attractive: the app I want to adapt for the Google Assistant I already sell outside of the Playstore (after being fed-up with all of Googles restrictions, requirements, removal of apps for no valid reason and other crap that cripples many apps nowadays). Thank you anyhow for looking into this matter.

PS, have you seen this:
https://actions-on-google.github.io/actions-on-google-nodejs/2.12.0/index.html ?
 
Last edited:
Top