Once you have entered the fulfillment url, you can test either in the dialogflow console on the right side of the console or in google assistant.
To test in google assistant, click on ‘Integrations’ in the dialogflow console and select ‘Google Assistant’. On the next screen select ‘test’.
It will open up a test window like this.
We can test each of your intent here. We can also see the request and response and any errors we might encounter.
For e.g. in this app, when I ask to ‘Talk to LitInspire’ , it will retrieve a quote and display on the simulator. It will also read the quote with appropriate pauses as specified in the SSML. It will also show the suggested replies. We can click on the suggested replies to test the next part of the conversation.
We can also view logs by following the link on the simulator. We can see the request and response and exception stack traces also. In the google cloud platform logs console, you can select the right application (actions on google app or the cloud function) to see the logs at the appropriate level.
Once we have verified the happy paths, the fallback paths and conversations and we are satisfied with the result, we can submit the app for approval.
Submit the app for approval
To prepare the app for approval, we have to provide more details so that the app can be visible in the directory. We can provide a short description, more details and invocation phrases to talk with the app.
We also have to provide images in two different sizes. I used the canva app to create these images. We have to provide a banner image and also a logo
Next we have to create a privacy policy.
Google provides a sample doc which we can copy and edit to create our own privacy policy. You can check my privacy policy here.
Lit Inpsire Privacy Policy.
Next we can select the category the app belongs to and other details helpful for review. We can also select the countries where the app is available and also the devices which can use the app. Then we can submit the app for review.
The app will be reviewed and if there are no issues, the app will be deployed to production. We will get an email informing us of the same. That completes the process of building and deployment.
Summary
In summary we have seen how to design, build and deploy an actions on google app. To summarize the steps, they are
Design a conversation
Create a project and define actions
Build a fulfillment service
Test the actions
Prepare for deployment and submit.
Thanks for staying with me. Hope this was helpful. Feel free to reach out if you have questions. You can comment here or reach out to me via twitter @thisisananth
Install firebase-cli. In many cases, new features and bug fixes are available only with the latest version of the Firebase CLI and thefirebase-functions SDK. So first install the latest versions of firebase-functions and firebase-admin
2. Once the dependencies are installed, do the following to initialize the project
a. Run firebase login to log in via the browser and authenticate the firebase tool.
b. Go to your Firebase project directory.
c. Run firebase init. The tool gives you an option to install dependencies with npm. We will select functions for the function and hosting for any public files. The init tool will guide through various options to choose the language, and the initialization options for functions and hosting
myproject
+- .firebaserc # Hidden file that helps you quickly switch between
| # projects with `firebase use`
|
+- firebase.json # Describes properties for your project
|
+- functions/ # Directory containing all your functions code
|
+- .eslintrc.json # Optional file containing rules for JavaScript linting.
|
+- package.json # npm package file describing your Cloud Functions code
|
+- index.js # main source file for your Cloud Functions code
|
+- node_modules/ # directory where your dependencies (declared in
# package.json) are installed
Once the project is initialized, we can develop our functions.
This is the package.json for the dependencies we have. We are using actions-on-google, firebase-admin, firebase-functions and google cloud datastore. We are using es-lint for linting. Code linting is a type of static analysis that is frequently used to find problematic patterns or code that doesn’t adhere to certain style guidelines.
Next let us look at the functions. When user invokes the app by name or google matches the request to any of our intents, then it sends a request to our app with the intent and other parameters. In our function we will implement the callback function for each intent.
// Instantiate a datastore client
const datastore = Datastore();
const app = dialogflow({debug: true});
app.middleware((conv) => {
});
//Setup contexts
const Contexts = {
ONE_MORE: 'one_more'
};
app.intent('quit_app', (conv) => {
conv.close("Have a good day! come back again. Bye!");
});
app.intent('start_app', (conv) => {
conv.contexts.set(Contexts.ONE_MORE,5);
const initMessage = ` Welcome to LitInspire. With great quotes and inspiring passages, I will inspire you.`;
app.intent('one_more_yes', (conv) => {
conv.contexts.set(Contexts.ONE_MORE,3);
const initMessage = `Great! Here is another one.`;
return getQuote().then((entity)=>{
return getMessageFromQuote(entity,initMessage,conv);
});
});
app.intent('one_more_no', (conv) => {
conv.close("Hope you're inspired and ready to take on your challenges. Have a good day and come back for more.");
});
app.intent('Default Fallback Intent', (conv) => {
console.log(conv.data.fallbackCount);
if (typeof conv.data.fallbackCount !== 'number') {
conv.data.fallbackCount = 0;
}
conv.data.fallbackCount++;
// Provide two prompts before ending game
if (conv.data.fallbackCount === 1) {
conv.contexts.set(Contexts.ONE_MORE,2);
return conv.ask(new Suggestions('Yes Please', 'No thanks'), new SimpleResponse("Would you like to hear a quote?"));
}else if(conv.data.fallbackCount === 2){
return conv.ask(new Suggestions('Yes Please', 'No thanks'), new SimpleResponse("Welcome to LitInspire. With great quotes and inspiring passages, I will inspire you.Would you like to hear a quote?"));
}
return conv.close("This isn't working.Have a good day. Bye! ");
});
function buildReadableQuoteFromEntity(entity){
let readableQuote = entity.quote +
` This was said by ` + entity.author + ` ` ;
if(entity.comments){
readableQuote += entity.comments + ` `;
}
return readableQuote;
}
function getViewableQuote(entity){
let viewableQuote = entity.quote +
`.This was said by ` + entity.author + ` ` ;
if(entity.comments){
viewableQuote += entity.comments + ` `;
}
return viewableQuote;
}
function getEndingMessage(){
return `
" clipBegin="10s" clipEnd="13s">Consider the quote!
Do you want to listen to another quote?`;
}
function getEndingMessageText(){
return `.Do you want to listen to another quote?`;
}
function getMessageFromQuote(entity,initMessage,conv){
return conv.ask(new Suggestions('Yes Please', 'No thanks'), new SimpleResponse(initMessage),
new SimpleResponse( {text: getViewableQuote(entity) + getEndingMessageText(),
speech: ` ` + buildReadableQuoteFromEntity(entity) + getEndingMessage() + ` ` }));
}
function getQuote(){
return new Promise(((resolve,reject) => {
let randomQuoteNum = getRandomNumber();
console.log("the id of the quote is: quote_"+randomQuoteNum);
const key = datastore.key(['quote', 'quote_'+randomQuoteNum]);
console.log("Querying datastore for the quote..."+key);
let readableQuote = '';
datastore.get(key,(err,entity) => {
if(!err){
console.log('entity:'+entity.quote);
resolve(entity);
}else{
reject(console.log('Error occured'));
}
});
}));
}
// HTTP Cloud Function for Firebase handler
exports.InspireMe = functions.https.onRequest(app);
Let us look at the intent handler for the start_app. First since we are in the app, we want the user to be able to ask for multiple quotes. Since it is in the one_more context, we will set the context to one_more. Next we need to get a quote and return it. We have the quotes stored in google cloud datastore. So we make a call to the datastore to return a random quote.
app.intent('start_app', (conv) => {
conv.contexts.set(Contexts.ONE_MORE,5);
const initMessage = ` Welcome to LitInspire. With great quotes and inspiring passages, I will inspire you.`;
The getQuote() function returns a promise with the quote. And in the intent handler, we use the .then() function of the promise to build the message from quote and return it. In the getMessageFromQuote() method, we can see join together multiple responses and return it. Here we use the ask method, which tells the user our quote and waits for the user response. And in this method we can pass suggestions, and atmost two simple responses. The simple response converts the text to speech. We are using the SSML (speech synthesis markup lanuguage) to specify how to generate the speech. With SSML, we can specify where to pause, and add music to the text. There are other types of responses also like Basic Card, Image, Button and List (Carousel) responses.
function getMessageFromQuote(entity,initMessage,conv){
return conv.ask(new Suggestions('Yes Please', 'No thanks'), new SimpleResponse(initMessage),
new SimpleResponse( {text: getViewableQuote(entity) + getEndingMessageText(),
speech: ` ` + buildReadableQuoteFromEntity(entity) + getEndingMessage() + ` ` }));
}
The two other things to see are how to end the conversation and how to handle unknown input. In the below function you can see that when the user says no, we use the conv.close() function to end the conversation with a message.
app.intent('one_more_no', (conv) => {
conv.close("Hope you're inspired and ready to take on your challenges. Have a good day and come back for more.");
});
When the user provides unknown input, google invokes the default fallback function. Lets look at that. We are giving options to the user two times whether to listen to a quote or not. And if the user doesn’t give a valid response even after two times, we are ending the conversation.
app.intent('Default Fallback Intent', (conv) => {
console.log(conv.data.fallbackCount);
if (typeof conv.data.fallbackCount !== 'number') {
conv.data.fallbackCount = 0;
}
conv.data.fallbackCount++;
// Provide two prompts before ending game
if (conv.data.fallbackCount === 1) {
conv.contexts.set(Contexts.ONE_MORE,2);
return conv.ask(new Suggestions('Yes Please', 'No thanks'), new SimpleResponse("Would you like to hear a quote?"));
}else if(conv.data.fallbackCount === 2){
return conv.ask(new Suggestions('Yes Please', 'No thanks'), new SimpleResponse("Welcome to LitInspire. With great quotes and inspiring passages, I will inspire you.Would you like to hear a quote?"));
}
return conv.close("This isn't working.Have a good day. Bye! ");
});
Now that’s it. We have written intent handlers for all possible intents and fallback handler also. Now this is ready for testing.
Use the firebase deploy command to deploy the function to firebase functions.
firebase deploy
Once deployed you will get a functions url. Go to the dialogflow console and go to the fulfilment menu item and provide the function url as the fulfilment url.
This is part 2 of the series. For part 1 see here.
Design a conversation
Here is a sample conversation.
Default flow:
User : Hey google tell me some inspiring quotes/motivational quotes/inspirational quotes
OR if the user knows the app already, he can ask by name
User: Hey google, talk to LitInspire
App: Welcome to Lit Inspire. With great quotes and inspiring passages, I will inspire you.
Here is a quote: Real Artists ship — This was said by Steve Jobs asking us to share our work with the world and not hold back.
Do you want to hear another quote?
User: Yes Please
App: here is another one . . Do you want to hear another one ?
User: Sure
App: Great. Here is another quote. . Do you want to hear another one.
User: No Thanks.
App: Hope you are inspired to take on your challenges. Have a good day!
The user could cancel at any time by saying cancel.
Create a project and define actions
To create the project you should first setup a project in the actions on google console.
Then choose a category which your app belongs to. It will show a form asking you to the select the default invocation phrase, i.e the name of the app the user can explicitly ask for and save.
Next click on the ‘actions’ menu and you will be prompted to build your first action. When you click on that you will be take to a menu with many built in actions and custom action. Built in actions help you to build apps based on template or you can customize later. Here we will proceed with a custom action. Select ‘custom action’ and click Build.
You will be redirected to the dialogflow console.
Dialogflow console
This is the agent which maps user queries to intents and invokes a specific action.
If you see the conversation above, the user first starts the app by asking for a specific quote or asking for the app by name.
So the initial intent could be start_app which is triggered when either asked by name of if google matches the invocation phrase (tell me an inspiring quote) to our app.
So create an intent with name ‘start_app’ and save
It gives options below to enter
contexts: Context is a way to link the conversation. Let us come back to this later.
events: Events are a way to enter the app without matched text. i.e a way to enter the app by calling by name. Select ‘Welcome’ and ‘Google Assistant Welcome’ which says that when users ask for this app by name, start_app intent is triggered.
actions and parameters: Specifies the action that should be triggered when this intent is specified. Dialogflow can also get ask questions to get all the parameters needed before fulfilling the action. So in this screen you can specify the action and also the parameter names to use later. For e.g if you want to listen a quote in a specific category, you can have that as a parameter. Currently the app just tells a quote in any category, so there is not paremeter in this app. So just specify an action name.
training phrases: Training pharses are phrases which the user can say to match this intent without asking for the app by name. for e.g tell me a motivating quote, tell me an inspiring quote, give me some inspiration, tell me a quote etc. Enter different varieties of training phrases all of the different ways a user can ask for quotes etc.
Responses: This section can be used to provide hardcoded responses. You can enter a few quotes here and they will be used if our service fails. If you want to end the conversation here, you can select, end this conversation slider. In this example we don’t want hardcoded response, so don’t select this.
Fulfillment: This is used if we want to create a fulfillment service which can provide user response instead of hard coding. Here in this example we want to use the fulfillment service. So select ‘Enable webhook call for this intent’. Later when we have the fulfillment service ready we can enter the fulfillment url in the fulfillment section.
This action will play the introduction and also return a quote. After that it asks user either to listen to another quote or quit. So we need two more intents one to handle when the user says ‘yes’ and one when user says ‘no’.
Contexts:Though there is only one question for which the user might answer yes here, the user might say ‘yes’ to different questions in conversation, but our app should know in which context the user said yes to. So when the user says ‘yes’ since the app knows the context the user is in, it will only match the action for that context not all the intents which have yes as a triggering phrase.
So create new intents one for one_more_yes and one for one_more_no.
In the one_more_yes, we want to listen to process the yes only if he says it in the one_more context i.e after he has listened to the first quote and wants to listen more. So in the contexts section, set the input context to one_more and save. Set the training phrases to be variations of the yes response like sure, absolutely, yes,please etc. Also enable webhook call for fulfillment. Set the action to be same as the intent name.
Do the same thing for the one_more_no intent. Set the input context to one_more and set the training phrases to variations of no and enable webhook call for fulfillment. Set the action to be same as the intent name.
Fallback intent: Next there may be times where the user might say something totally unrelated to the app and we should have a way of handling that. Google already provides a ‘Default Fallback Intent’. Open that and set an action name like input.unknown and enable fulfillment so that we can handle it in our service.
This should cover all the intents which are necessary for our app. Next lets build the fulfillment service.
Build fulfillment
The fulfillment service should accept a POST request from Dialogflow with the matched intent details. Your service should handled the request and return a response as per the Dialogflow V2 specification.
We could use any language for creating the service, but Dialogflow provides a NodeJS based fulfillment SDK which will make our job very easier. So we will be using the NodeJS SDK.
Our fulfillment service will be a Google Cloud fucntions for firebase . Cloud Functions for Firebase lets you automatically run backend code in response to events triggered by Firebase features and HTTPS requests. Your code is stored in Google’s cloud and runs in a managed environment. There’s no need to manage and scale your own servers.
Hunter S Thomson is an american journalist. In a letter he wrote to his friend Hume Logan, he gives excellent advice about finding what you want to do in life.
Basically the advice boils down to this -
You can either float with the tide or swim towards a goal. But since our goals change as we get more experience and insight, the goals are never constant. They keep changing so our focus shouldn't be on the goals. We don't want to be a fireman or banker or teacher etc., we want to be ourselves. So the idea is not to reach a pre-defined goal but to choose a way of life you ENJOY. So choose your lifestyle and not the goal. Next if there are eight pre-defined paths laid down for you in life and you feel they are all not satisfying to you, then it is upto you to design the ninth path for yourself. But if you procrastinate in this CHOOSING you will have circumstances making the choice for you. What this means is that you don't just have to take the handed down paths to you but you can look for new paths that fit your lifestyle. This is an amazing point.
I like learning new things, reading and sharing stuff. I should choose a way that allows me to keep doing that!
This is from a book called letters of note via farnum street.
Lose an hour in the morning and chase it all day - Yiddish saying via Tim Ferris, Jason Fried
Morning:
The linked medium post got me thinking about my morning routine. There were days where I followed my routine and it worked wonders in my day and there were times when I got distracted and it ended up ruining my day. So I'm thinking back to see what was the routine on the day that worked well and what was the routine on the day that didn't work so well so that I can redesign my ideal routine and commit to it.
News:
The biggest ruiner of my day is reading news online first thing in the day. I know that has ruined my day many times and sometimes and just fool myself into thinking that I can do it for five minutes and close it. But it reality I have never closed it before 20 minutes. I only have one hour a day to study for myself and I waste 20 minutes out of that. So this is the first thing I have to avoid at any cost. On the contrary, the best start for the day was when I started working directly on my most important priority of the day.
Journalling:
Next thing that made a good day was to journal - just think about yesterday and today and to clearly spell out what I'm doing today, how I'm going to accomplish as said by Ben Hardy will really make me very productive.
Work:
Deadlines:
Next is to overcome Parkinson's law by scheduling minimum amount of time for any work. Deadlines help. I become more focused when I have deadlines and when the deadlines are near.
Next is the good water and food rule. When I'm well rested and had good food, I don't get distracted often.
Gratitude:
Whenever I feel grateful for what I have, I'm motivated. Also when I'm excited about starting something new at work, I'm excited too - especially new ideas.
Overexcitement:
When I start reading any exciting news article - it is so exciting to me that I will do that at the expense of other things - so this is something which is very damaging to my scheduled tasks and productivity. But these are the things that are very very interesting to me that if I stop doing them completely it feels something is missing and I will crash hard if I just abstain from it for a few days. I have to schedule sometime for these daily so that I can feel that I'm aware of the latest happenings around the world and exciting things in the areas I'm interested in.
Also when I'm working on something with full focus, there is less chance of missing something or skipping something. So I'm fully present in my work and do my best work which is of course obvious.
Distractions masked as focus:
Another thing that distracts is when I'm working on some development which needs more focus and my brain thinks it is easy to get distracted so I try to focus by listening to something but to identify that something takes time and most of the times it is podcasts or youtube videos. Sometimes these are boring and I again change it and try changing them etc. Also when I'm listening to podcasts or other prosy stuff on youtube, my attention is obviously distracted. The only safe thing that works is classical or lyric less music - so there should be an easy and simple way to always select and listen to that in one click so that I don't get distracted on youtube or google play music trying to find the right music.
Fear:
Another thing that distracts is if I find something which scares me which is generally about money, health or work. This kind of triggers a fight or flight response which overwhelms anything else I know of. So it is obviously better if I don't know of things that trigger this kind of response.
Escaping effort:
Also the main reason for getting is distracted is the inability to wait - inability to wait while project is building, inability to wait for a few seconds to put some mental effort and the concept of avoiding effort. Our brain wants to put minimal effort into things and each time we have to think to make an effort, it becomes a challenge. So if we develop it into a habit it will be easier. So this should be added to my do not do list.
Evening:
Then after I come home, it will be great to teach something to my son. If I spent time with him it is very satisfactory. One of the worst days is when just switch on the TV and sit infront of it mindlessly playing things from youtube there. Often times just scheduling something in the evening can get me out of this trap.
Summarizing the best things in my routine are
- Starting the work directly
- Journalling
- Scheduling minimum amount of time to complete the work
- working without distractions
- Playing study music (classical or lyric less)
- Getting used to waiting
- Scheduling something in the evening
- Count your blessings
The do not do items are
- Do not Open a browser first
- Do not browse while waiting for application to come up
- Do not get the music from youtube or play podcasts
- Do not research on non office work especially exciting news or scary things in office.
- Do not work without deadlines or a plan
If we think of everything we have to do, we feel overwhelmed. If we do the one thing we need to do, we make progress.
