Monday, June 18, 2018

A home robot - what does it need ?

In our personal life, we spend a lot of time organizing, cooking and cleaning. If it is possible to automate these on any level even if it is not intelligent, it can save at least one hour a day easily.

So I want to work on making robots do these things in narrow domains. The state of the art ai is just now learning to pick all sorts of things.

For organizing, an ai has to identify the homes for each thing, grab the thing and put it there. If the space is full, it has to think of an alternate space and put it there. It has to understand when something can be discarded and then discard it. So it should be able to move and access all the storage spaces and also identify what thing is important/ unimportant and make decisions accordingly.

For the cleaning ai, it should also have capabilities to handle solids and liquids. It has to understand how tightly or softly it has to grip stuff - what is clean and what is unclean and how to get from unclean to clean. It should be able to access cleaning utilities and also know how to use them. It should also know where such stuff is available in a home and should be able to access them.

Cooking is even more complex and would need to understand the basics of cooking. It should also handle different kinds of stuff (hard, soft, wet, dry, powdery, thin, thick etc). It should also be able to identify textures to know if something is done or not even to follow existing recipes. It should be able to cut, mix, saute and other cooking related actions.

All of these should understand the humans in the house, interact with them, follow their directions but not do them any harm.

I think each of these increase in complexity and it will be interested to see how each will be approached and solved. I am interested in these use cases and will be keeping an out on the advances here. 

Monday, June 11, 2018

How to do good scientific research work - Advice from Richard Hamming

Richard Hamming is an american mathematician who worked in Bell labs and discovered error correcting codes and digital filters among other things. He gave a talk called 'You and Your Research' about how to do great work. It is worth reading the original fully so go ahead and read it here.

Here are some interesting points from that lecture.

He talks about luck here.. 

Once in a while a person does only one thing in his whole life, and we'll talk about that later, but a lot of times there is repetition. I claim that luck will not cover everything. And I will cite Pasteur who said, ``Luck favors the prepared mind.'' And I think that says it the way I believe it. There is indeed an element of luck, and no, there isn't. The prepared mind sooner or later finds something important and does it. So yes, it is luck. The particular thing you do is luck, but that you do something is not.


The importance of courage and confidence. 


One of the characteristics of successful scientists is having courage. Once you get your courage up and believe that you can do important problems, then you can. If you think you can't, almost surely you are not going to. 

On turning defects to assets


I think that if you look carefully you will see that often the great scientists, by turning the problem around a bit, changed a defect to an asset. For example, many scientists when they found they couldn't do a problem finally began to study why not. They then turned it around the other way and said, ``But of course, this is what it is'' and got an important result. 
Talking about compounding of learning and how having the drive does it


``Knowledge and productivity are like compound interest.'' Given two people of approximately the same ability and one person who works ten percent more than the other, the latter will more than twice outproduce the former. The more you know, the more you learn; the more you learn, the more you can do; the more you can do, the more the opportunity - it is very much like compound interest. I don't want to give you a rate, but it is a very high rate. Given two people with exactly the same ability, the one person who manages day in and day out to get in one more hour of thinking will be tremendously more productive over a lifetime. 

On being committed and thinking about a problem till you find a solution


when you have a real important problem you don't let anything else get the center of your attention - you keep your thoughts on the problem. Keep your subconscious starved so it has to work on your problem, so you can sleep peacefully and get the answer in the morning, free.


On thinking and attacking the important problems in the field. 


If you do not work on an important problem, it's unlikely you'll do important work. It's perfectly obvious.You can't always know exactly where to be, but you can keep active in places where something might happen. And even if you believe that great science is a matter of luck, you can stand on a mountain top where lightning strikes; you don't have to hide in the valley where you're safe. 
Most great scientists know many important problems. They have something between 10 and 20 important problems for which they are looking for an attack. And when they see a new idea come up, one hears them say ``Well that bears on this problem.'' They drop all the other things and get after it.The great scientists, when an opportunity opens up, get after it and they pursue it. They drop all other things. They get rid of other things and they get after an idea because they had already thought the thing through. Their minds are prepared; they see the opportunity and they go after it. Now of course lots of times it doesn't work out, but you don't have to hit many of them to do some great science. It's kind of easy. One of the chief tricks is to live a long time!

On generalizing the results of your work


You should do your job in such a fashion that others can build on top of it, so they will indeed say, ``Yes, I've stood on so and so's shoulders and I saw further.'' The essence of science is cumulative. By changing a problem slightly you can often do great work rather than merely good work.

You have to sell your work


You have to learn to write clearly and well so that people will read it, you must learn to give reasonably formal talks, and you also must learn to give informal talks.

on self-delusion


 self-delusion in humans is very, very common. There are enumerable ways of you changing a thing and kidding yourself and making it look some other way. When you ask, ``Why didn't you do such and such,'' the person has a thousand alibis. If you look at the history of science, usually these days there are 10 people right there ready, and we pay off for the person who is there first. The other nine fellows say, ``Well, I had the idea but I didn't do it and so on and so on.'' There are so many alibis. Why weren't you first? Why didn't you do it right? Don't try an alibi. Don't try and kid yourself. You can tell other people all the alibis you want. I don't mind. But to yourself try to be honest.

Summarizing 

In summary, I claim that some of the reasons why so many people who have greatness within their grasp don't succeed are: they don't work on important problems, they don't become emotionally involved, they don't try and change what is difficult to some other situation which is easily done but is still important, and they keep giving themselves alibis why they don't. They keep saying that it is a matter of luck. 

Go read the whole thing. It is very interesting and packed with gems! 

Monday, June 4, 2018

Improving the design of google home app

In the current design of the app, the number of quotes is hardcoded in the app. But because of that each time, I add new quotes I will have to redeploy the app. This will cause reluctance in me to redeploy and change the quotes - which means it has difficult maintenance.

To fix this, I want to modify it such that either I will set the total count as a runtime configuration which the app can pick up automatically. I tried using the firebase runtime configuration. But it seems for this config change to take effect I will need to redeploy the function. This seems to not fix any of the issue.

Researching further I found a solution to this problem by adding a random float to each quote and when querying just querying for quote greater than a specific random number. That way I will get a different random quote each time and I don't need to redploy the app each time. I haven't made the changes yet and redeployed the app but this should fix the maintenance nightmare and will make it easy for me to add new quotes to the app. 

Thursday, May 31, 2018

Developing a Google Assistant app with NodeJs, Dialogflow & Firebase Functions - Part 4

This is part of a multipart series. 

Part 1

Part 2

Part 3

This is part 4. 


Test the app

Once you have entered the fulfillment url, you can test either in the dialogflow console on the right side of the console or in google assistant.
To test in google assistant, click on ‘Integrations’ in the dialogflow console and select ‘Google Assistant’. On the next screen select ‘test’.
It will open up a test window like this.


We can test each of your intent here. We can also see the request and response and any errors we might encounter.
For e.g. in this app, when I ask to ‘Talk to LitInspire’ , it will retrieve a quote and display on the simulator. It will also read the quote with appropriate pauses as specified in the SSML. It will also show the suggested replies. We can click on the suggested replies to test the next part of the conversation.
We can also view logs by following the link on the simulator. We can see the request and response and exception stack traces also. In the google cloud platform logs console, you can select the right application (actions on google app or the cloud function) to see the logs at the appropriate level.
Once we have verified the happy paths, the fallback paths and conversations and we are satisfied with the result, we can submit the app for approval.
Submit the app for approval
To prepare the app for approval, we have to provide more details so that the app can be visible in the directory. We can provide a short description, more details and invocation phrases to talk with the app.
We also have to provide images in two different sizes. I used the canva app to create these images. We have to provide a banner image and also a logo


Next we have to create a privacy policy.
Google provides a sample doc which we can copy and edit to create our own privacy policy. You can check my privacy policy here.

Lit Inpsire Privacy Policy.

Next we can select the category the app belongs to and other details helpful for review. We can also select the countries where the app is available and also the devices which can use the app. Then we can submit the app for review.
The app will be reviewed and if there are no issues, the app will be deployed to production. We will get an email informing us of the same. That completes the process of building and deployment.
Summary
In summary we have seen how to design, build and deploy an actions on google app. To summarize the steps, they are
Design a conversation
Create a project and define actions
Build a fulfillment service
Test the actions
Prepare for deployment and submit.
Thanks for staying with me. Hope this was helpful. Feel free to reach out if you have questions. You can comment here or reach out to me via twitter @thisisananth

Wednesday, May 30, 2018

Developing a Google Assistant app with NodeJs, Dialogflow & Firebase Functions - Part 3

This is part of a multipart series. 
This is part 3. 

Setting up the project.
  1. Install firebase-cli. In many cases, new features and bug fixes are available only with the latest version of the Firebase CLI and thefirebase-functions SDK. So first install the latest versions of firebase-functions and firebase-admin
npm install firebase-functions@latest firebase-admin@latest --save
npm install -g firebase-tools
2. Once the dependencies are installed, do the following to initialize the project
a. Run firebase login to log in via the browser and authenticate the firebase tool.
b. Go to your Firebase project directory.
c. Run firebase init. The tool gives you an option to install dependencies with npm. We will select functions for the function and hosting for any public files. The init tool will guide through various options to choose the language, and the initialization options for functions and hosting

myproject
 +- .firebaserc    # Hidden file that helps you quickly switch between
 |                 # projects with `firebase use`
 |
 +- firebase.json  # Describes properties for your project
 |
 +- functions/     # Directory containing all your functions code
      |
      +- .eslintrc.json  # Optional file containing rules for JavaScript linting.
      |
      +- package.json  # npm package file describing your Cloud Functions code
      |
      +- index.js      # main source file for your Cloud Functions code
      |
      +- node_modules/ # directory where your dependencies (declared in
                       # package.json) are installed


Once the project is initialized, we can develop our functions.
This is the package.json for the dependencies we have. We are using actions-on-google, firebase-admin, firebase-functions and google cloud datastore. We are using es-lint for linting. Code linting is a type of static analysis that is frequently used to find problematic patterns or code that doesn’t adhere to certain style guidelines.
{
  "name": "functions",
  "description": "Cloud Functions for Firebase",
  "scripts": {
    "lint": "eslint --fix \"**/*.js\"",
    "start": "firebase serve",
    "deploy": "firebase deploy",
    "test": "npm run lint"
  },
  "dependencies": {
    "@google-cloud/datastore": "1.1.0",
    "actions-on-google": "^2.0.0",
    "ajv": "^5.0.0",
    "firebase-admin": "^5.11.0",
    "firebase-functions": "^1.0.0"
  },
  "devDependencies": {
    "eslint": "^4.19.1",
    "eslint-config-google": "^0.9.1"
  },
  "private": true,
  "version": "0.0.1"
}
Next let us look at the functions. When user invokes the app by name or google matches the request to any of our intents, then it sends a request to our app with the intent and other parameters. In our function we will implement the callback function for each intent. 
'use strict';
//Initialize libraries
const {dialogflow} = require('actions-on-google');
const functions = require('firebase-functions');
const Datastore = require('@google-cloud/datastore');
const {
  SimpleResponse,
  BasicCard,
  Image,
  Suggestions,
  Button
} = require('actions-on-google');
// Instantiate a datastore client
const datastore = Datastore();
  
  const app = dialogflow({debug: true});
app.middleware((conv) => {
    
  });
//Setup contexts
const Contexts = {
    ONE_MORE: 'one_more'
  };
app.intent('quit_app', (conv) => {
    conv.close("Have a good day! come back again. Bye!");
  });
app.intent('start_app', (conv) => {
    conv.contexts.set(Contexts.ONE_MORE,5);
    const initMessage = ` Welcome to LitInspire. With great quotes and inspiring passages, I will inspire you.`;
return  getQuote().then((entity)=>{
         return getMessageFromQuote(entity,initMessage,conv);
    });
      
  });
app.intent('one_more_yes', (conv) => {
    conv.contexts.set(Contexts.ONE_MORE,3);
      const initMessage = `Great! Here is another one.`;
         
    return  getQuote().then((entity)=>{
      return getMessageFromQuote(entity,initMessage,conv);
  });
      
  });
app.intent('one_more_no', (conv) => {
    conv.close("Hope you're inspired and ready to take on your challenges. Have a good day and come back for more.");
});
app.intent('Default Fallback Intent', (conv) => {
    console.log(conv.data.fallbackCount);
    if (typeof conv.data.fallbackCount !== 'number') {
      conv.data.fallbackCount = 0;
    }
    conv.data.fallbackCount++;
    // Provide two prompts before ending game
    if (conv.data.fallbackCount === 1) {
      conv.contexts.set(Contexts.ONE_MORE,2);
      return conv.ask(new Suggestions('Yes Please', 'No thanks'), new SimpleResponse("Would you like to hear a quote?"));
    }else if(conv.data.fallbackCount === 2){
      return conv.ask(new Suggestions('Yes Please', 'No thanks'), new SimpleResponse("Welcome to LitInspire. With great quotes and inspiring passages, I will inspire you.Would you like to hear a quote?"));
    }
   return conv.close("This isn't working.Have a good day. Bye! ");
});
function getRandomNumber(){
return  Math.floor((Math.random()*num_quotes)+1);
}

function buildReadableQuoteFromEntity(entity){
  let readableQuote =  entity.quote + 
     ` This was said by ` + entity.author + ` `  ;
     if(entity.comments){
       readableQuote +=  entity.comments + ` `;
     }
     return readableQuote;
}
function getViewableQuote(entity){
  let viewableQuote =  entity.quote + 
     `.This was said by ` + entity.author + ` `  ;
     if(entity.comments){
      viewableQuote +=  entity.comments + ` `;
     }
     return viewableQuote;
}
function getEndingMessage(){
return `  
" clipBegin="10s" clipEnd="13s">Consider the quote!
Do you want to listen to another quote?`;
}
function getEndingMessageText(){
  return `.Do you want to listen to another quote?`;
  }
function getMessageFromQuote(entity,initMessage,conv){
  return conv.ask(new Suggestions('Yes Please', 'No thanks'), new SimpleResponse(initMessage),
  new SimpleResponse( {text: getViewableQuote(entity) + getEndingMessageText(),
speech: ` ` +  buildReadableQuoteFromEntity(entity)   + getEndingMessage() + `   ` }));
 }
function getQuote(){
  return new Promise(((resolve,reject) => {
    let randomQuoteNum = getRandomNumber();
  console.log("the id of the quote is: quote_"+randomQuoteNum);
  const key = datastore.key(['quote', 'quote_'+randomQuoteNum]);
  console.log("Querying datastore for the quote..."+key);
  let readableQuote = '';
  datastore.get(key,(err,entity) => {
    if(!err){
      console.log('entity:'+entity.quote);
    resolve(entity);
    }else{
     reject(console.log('Error occured'));
    }
  });
  }));
}
// HTTP Cloud Function for Firebase handler
exports.InspireMe = functions.https.onRequest(app);
Let us look at the intent handler for the start_app. First since we are in the app, we want the user to be able to ask for multiple quotes. Since it is in the one_more context, we will set the context to one_more. Next we need to get a quote and return it. We have the quotes stored in google cloud datastore. So we make a call to the datastore to return a random quote. 
app.intent('start_app', (conv) => {
    conv.contexts.set(Contexts.ONE_MORE,5);
    const initMessage = ` Welcome to LitInspire. With great quotes and inspiring passages, I will inspire you.`;
return  getQuote().then((entity)=>{
         return getMessageFromQuote(entity,initMessage,conv);
    });
      
  });
The getQuote() function returns a promise with the quote. And in the intent handler, we use the .then() function of the promise to build the message from quote and return it. In the getMessageFromQuote() method, we can see join together multiple responses and return it. Here we use the ask method, which tells the user our quote and waits for the user response. And in this method we can pass suggestions, and atmost two simple responses. The simple response converts the text to speech. We are using the SSML (speech synthesis markup lanuguage) to specify how to generate the speech. With SSML, we can specify where to pause, and add music to the text. There are other types of responses also like Basic Card, Image, Button and List (Carousel) responses. 
function getMessageFromQuote(entity,initMessage,conv){
  return conv.ask(new Suggestions('Yes Please', 'No thanks'), new SimpleResponse(initMessage),
  new SimpleResponse( {text: getViewableQuote(entity) + getEndingMessageText(),
speech: ` ` +  buildReadableQuoteFromEntity(entity)   + getEndingMessage() + `   ` }));
 }
The two other things to see are how to end the conversation and how to handle unknown input. In the below function you can see that when the user says no, we use the conv.close() function to end the conversation with a message. 
app.intent('one_more_no', (conv) => {
    conv.close("Hope you're inspired and ready to take on your challenges. Have a good day and come back for more.");
});
When the user provides unknown input, google invokes the default fallback function. Lets look at that. We are giving options to the user two times whether to listen to a quote or not. And if the user doesn’t give a valid response even after two times, we are ending the conversation. 
app.intent('Default Fallback Intent', (conv) => {
    console.log(conv.data.fallbackCount);
    if (typeof conv.data.fallbackCount !== 'number') {
      conv.data.fallbackCount = 0;
    }
    conv.data.fallbackCount++;
    // Provide two prompts before ending game
    if (conv.data.fallbackCount === 1) {
      conv.contexts.set(Contexts.ONE_MORE,2);
      return conv.ask(new Suggestions('Yes Please', 'No thanks'), new SimpleResponse("Would you like to hear a quote?"));
    }else if(conv.data.fallbackCount === 2){
      return conv.ask(new Suggestions('Yes Please', 'No thanks'), new SimpleResponse("Welcome to LitInspire. With great quotes and inspiring passages, I will inspire you.Would you like to hear a quote?"));
    }
   return conv.close("This isn't working.Have a good day. Bye! ");
});
Now that’s it. We have written intent handlers for all possible intents and fallback handler also. Now this is ready for testing. 
Use the firebase deploy command to deploy the function to firebase functions. 
firebase deploy
Once deployed you will get a functions url. Go to the dialogflow console and go to the fulfilment menu item and provide the function url as the fulfilment url. 

Enter fulfilment

Monday, May 28, 2018

Developing a Google Assistant app with NodeJs, Dialogflow & Firebase Functions - Part 2

This is part 2 of the series. For part 1 see here.

Design a conversation

Here is a sample conversation.
Default flow:
User : Hey google tell me some inspiring quotes/motivational quotes/inspirational quotes 
OR if the user knows the app already, he can ask by name
User: Hey google, talk to LitInspire

App: Welcome to Lit Inspire. With great quotes and inspiring passages, I will inspire you. 
Here is a quote: Real Artists ship — This was said by Steve Jobs asking us to share our work with the world and not hold back. 
Do you want to hear another quote? 
User: Yes Please
App: here is another one . . Do you want to hear another one ?
User: Sure
App: Great. Here is another quote. . Do you want to hear another one. 
User: No Thanks. 
App: Hope you are inspired to take on your challenges. Have a good day! 
The user could cancel at any time by saying cancel. 

Create a project and define actions 

To create the project you should first setup a project in the actions on google console. 



Then choose a category which your app belongs to. It will show a form asking you to the select the default invocation phrase, i.e the name of the app the user can explicitly ask for and save. 
Next click on the ‘actions’ menu and you will be prompted to build your first action. When you click on that you will be take to a menu with many built in actions and custom action. Built in actions help you to build apps based on template or you can customize later. Here we will proceed with a custom action. Select ‘custom action’ and click Build. 
You will be redirected to the dialogflow console. 


Dialogflow console

This is the agent which maps user queries to intents and invokes a specific action. 
If you see the conversation above, the user first starts the app by asking for a specific quote or asking for the app by name. 
So the initial intent could be start_app which is triggered when either asked by name of if google matches the invocation phrase (tell me an inspiring quote) to our app.
So create an intent with name ‘start_app’ and save
It gives options below to enter

contexts: Context is a way to link the conversation. Let us come back to this later.

events: Events are a way to enter the app without matched text. i.e a way to enter the app by calling by name. Select ‘Welcome’ and ‘Google Assistant Welcome’ which says that when users ask for this app by name, start_app intent is triggered.

actions and parameters: Specifies the action that should be triggered when this intent is specified. Dialogflow can also get ask questions to get all the parameters needed before fulfilling the action. So in this screen you can specify the action and also the parameter names to use later. For e.g if you want to listen a quote in a specific category, you can have that as a parameter. Currently the app just tells a quote in any category, so there is not paremeter in this app. So just specify an action name.

training phrases: Training pharses are phrases which the user can say to match this intent without asking for the app by name. for e.g tell me a motivating quote, tell me an inspiring quote, give me some inspiration, tell me a quote etc. Enter different varieties of training phrases all of the different ways a user can ask for quotes etc.

Responses: This section can be used to provide hardcoded responses. You can enter a few quotes here and they will be used if our service fails. If you want to end the conversation here, you can select, end this conversation slider. In this example we don’t want hardcoded response, so don’t select this.

Fulfillment: This is used if we want to create a fulfillment service which can provide user response instead of hard coding. Here in this example we want to use the fulfillment service. So select ‘Enable webhook call for this intent’. Later when we have the fulfillment service ready we can enter the fulfillment url in the fulfillment section.

This action will play the introduction and also return a quote. After that it asks user either to listen to another quote or quit. So we need two more intents one to handle when the user says ‘yes’ and one when user says ‘no’.

Contexts:Though there is only one question for which the user might answer yes here, the user might say ‘yes’ to different questions in conversation, but our app should know in which context the user said yes to. So when the user says ‘yes’ since the app knows the context the user is in, it will only match the action for that context not all the intents which have yes as a triggering phrase. 
So create new intents one for one_more_yes and one for one_more_no. 
In the one_more_yes, we want to listen to process the yes only if he says it in the one_more context i.e after he has listened to the first quote and wants to listen more. So in the contexts section, set the input context to one_more and save. Set the training phrases to be variations of the yes response like sure, absolutely, yes,please etc. Also enable webhook call for fulfillment. Set the action to be same as the intent name. 
Do the same thing for the one_more_no intent. Set the input context to one_more and set the training phrases to variations of no and enable webhook call for fulfillment. Set the action to be same as the intent name.

Fallback intent: Next there may be times where the user might say something totally unrelated to the app and we should have a way of handling that. Google already provides a ‘Default Fallback Intent’. Open that and set an action name like input.unknown and enable fulfillment so that we can handle it in our service. 
This should cover all the intents which are necessary for our app. Next lets build the fulfillment service. 

Build fulfillment 

The fulfillment service should accept a POST request from Dialogflow with the matched intent details. Your service should handled the request and return a response as per the Dialogflow V2 specification. 
We could use any language for creating the service, but Dialogflow provides a NodeJS based fulfillment SDK which will make our job very easier. So we will be using the NodeJS SDK. 
Our fulfillment service will be a Google Cloud fucntions for firebase . Cloud Functions for Firebase lets you automatically run backend code in response to events triggered by Firebase features and HTTPS requests. Your code is stored in Google’s cloud and runs in a managed environment. There’s no need to manage and scale your own servers.
Cloud Functions runs Node v6.14.0, so install node v6.14 or higher version. 
Once nodejs is installed, set up the firebase cli following the instructions here
The steps are 
  1. Install firebase-cli
  2. Initialize project firebase init and download dependencies 
Once the project is initialized, we can develop our functions. Lets see how to install firebase-cli and how to develop fulfilment in  the next part. 

Monday, May 21, 2018

How to find what you want to do in life - Excellent advice from Hunter S Thomson

Hunter S Thomson is an american journalist. In a letter he wrote to his friend Hume Logan, he gives excellent advice about finding what you want to do in life.

Basically the advice boils down to this -

You can either float with the tide or swim towards a goal. But since our goals change as we get more experience and insight, the goals are never constant. They keep changing so our focus shouldn't be on the goals. We don't want to be a fireman or banker or teacher etc., we want to be ourselves.

So the idea is not to reach a pre-defined goal but to choose a way of life you ENJOY. So choose your lifestyle and not the goal.

Next if there are eight pre-defined paths laid down for you in life and you feel they are all not satisfying to you, then it is upto you to design the ninth path for yourself. But if you procrastinate in this CHOOSING you will have circumstances making the choice for you.

What this means is that you don't just have to take the handed down paths to you but you can look for new paths that fit your lifestyle.  This is an amazing point.

I like learning new things, reading and sharing stuff. I should choose a way that allows me to keep doing that!

This is from a book called letters of note via farnum street