Dialogflow Mega Agent Tutorial

In this tutorial, I explain how to get started with Dialogflow Mega Agents.

The mega agent is based on two sub-agents:

  • a sub-agent based on the articles on my website
  • a sub-agent based on the course lessons

You can interact with the mega agent I built by clicking on the chat widget at the bottom right of this page. You will get links to articles and individual course lessons on this website. If the prefix is Article – it is an article from my main website. If the prefix is Free Course – it is a lesson from one of my courses here.

Note: the agents are up-to-date till the articles published till end of January 2021.

Create the mega agent

We will first create the mega agent.

Inside your Dialogflow Console, click on New agent. I created a new project inside my Google Cloud account (called megademo here), but you can also just leave it empty and allow Dialogflow to create a new Google Cloud project to be associated with this agent. Make sure you select the toggle called “Set as Mega Agent”.

In case you weren’t aware, Dialogflow now creates a new Google Cloud project for each new agent. It wasn’t like this previously, but Google has been limiting the number of agents you can create with a given Google account. (If you need more agents, you can also send an email to Google support to increase the number of projects and thus the number of agents, but you can only do this after you add a credit card).

Notice two things:

  • there is no Default Welcome Intent in your mega agent
  • you cannot add new intents into your mega agent either

Create the first subagent

Let us first create the sub-agent for the articles on my website. Click on Create new agent.

I will call the first sub-agent mbdmain to indicate that it is about the main website.

In the sub-agent, you do see both Default Welcome Intent and Default Fallback Intent as expected.

Bulk upload intents to the first subagent

I use my BotFlo app to bulk upload intents to the mbdmain sub-agent.

You can also manually add the corresponding intents for your use case if you don’t want to use my app.

I use WordPress for my website, so I use a plugin which allows me to export a list of all URLs and page titles. If your website isn’t using WordPress, you will still likely have some kind of tool to generate the sitemap for it. You can use that tool to get a similar export file.

Inside the plugin, I select all my blog posts to export and select Titles, URLs and categories as the fields to export.

The export produces a CSV file. You can see what this CSV file looks like below. (I have opened it in Mac Numbers).

I imported the CSV file into my Airtable account. This just makes it easier for me to manage the formulas (compared to Mac Numbers).

Inside Airtable, I make sure I convert the URL into a format which is compatible with my app. If you want to bulk upload Dialogflow Messenger responses in BotFlo, you should provide the links in Markdown format. You can read more about this format here.

After I prepared the CSV file in the appropriate format, I logged in to BotFlo and uploaded the CSV file. BotFlo shows you a preview of the CSV file contents before it does the conversion. Once you click on the “Convert” button, you will be able to download the agent ZIP file for all the intents you uploaded in your CSV file.

Inside Dialogflow, use the RESTORE agent feature and use the ZIP file you downloaded in the previous step.

Now click on the Intents tab in the left pane. You will be able to see all the intents imported into your Dialogflow agent.

There are 150+ intents in this agent. Typing out all the intents one by one would have taken me many hours. By using BotFlo, I was able to do the whole thing in a few minutes.

Add the subagent to the Mega agent

After creating all the intents in your sub-agent, you can add it to the Mega agent.

Select the Mega Agent. On the left hand menu click on Sub Agents link.

Click on “Add the first one” link.

Select the sub-agent mbdmain on this page. Leave the Environment and Knowledge Base dropdown selections with default values.

Test the first subagent in the Dialogflow simulator

You will now be able to test your changes in the Dialogflow simulator. I test the phrase “slot filling” to see if it matches one of my articles on this topic. And you can see that it does.

Open “Diagnostic Info” to see the JSON payload coming back in the response. You can see the payload corresponding to the Dialogflow Messenger clickable hyperlink. You can also see an autogenerated implicit output context called “__mega_agent_context__” with a very large lifespan count.

You also see the detected intent as well as the intentDetectionConfidence score.

Add the second subagent to the Mega Agent

Now I will create a second sub-agent based on the course lessons.

Using the same Export All URLs plugin, this time I exported all the Lessons and Topics post types. These are the post types used by the learning management system I use (LearnDash).

You can see the exported URLs in Airtable below.

Next, you see the CSV file preview after it has been uploaded to BotFlo. Then I clicked on the Convert button to download the agent ZIP file for the courses sub-agent.

Then I created a new agent called mbdcourses.

Then I used the RESTORE ZIP feature and used the agent ZIP file I downloaded.

Then I selected mbdcourses as the second sub-agent in the Mega Agent.

How the mega_agent_context affects the detected intent

Now we have two sub-agents under our Mega agent. Let us see how this works in the Dialogflow simulator.

The presence of the __mega_agent_context__ produces some interesting results when it comes to the actual intent which is mapped.

Phrase 1 “runkit”

First, I will try the phrase “runkit” inside the Dialogflow simulator.

The selected intent is from the mbdmain sub-agent.

Phrase 2 “create runkit”

Next I try the phrase “create runkit”. This maps to an intent called “How to create new Dialogflow agent programmatically”. At first glance, that doesn’t seem very relevant. It seems to have matched the keyword “create” and ignored the keyword “runkit”.

Phrase 3 “create runkit account”

Now I test with the phrase “create runkit account”. Notice that it matches an intent which is quite close to the phrase.

And when we look at the “Diagnostic info”, we can see that the intent was selected from the mbd-courses sub-agent.

Phrase 4 “create runkit”

Here it gets interesting. Let us repeat phrase 2 and try “create runkit” once again. This time we get a different result from phrase 2.

And when we look at the “Diagnostic Info” we notice that the mbdcourses sub-agent was selected for the same phrase. So the most recently selected sub-agent has some impact on the result of this intent mapping.

I cannot tell you what you can do to make sure a specific sub-agent gets selected for a specific training phrase (plus, it sort of beats the purpose of having a Mega agent). But it is clear that you need to take into account the impact of the most recent agent which was selected for the previous training phrase used in the session.

Integrating Dialogflow Messenger with your Mega Agent

While the Mega Agent works in the Dialogflow simulator, you need to do a few more things to get the mega agent working with the Dialogflow Messenger integration.

Go to your Google Cloud Console.

Select the mega agent project.

Select IAM* from the left menu

Select the checkbox which says “Include Google-provided role grants”

You will see a service account which was automatically created when you created your Dialogflow Mega Agent. Notice that the service account has exactly one role, called “Dialogflow Service Agent”.

Now click on the pencil icon to the right of the service account. This will allow you to edit the roles assigned to this service account.

In the new dialog window, click to add a role and type the word dialogflow. You will see that the dropdown now filters down to all roles which are related to Dialogflow.

Select the Dialogflow API Client role.

Make sure to save your changes.

Back in your IAM page, you will now see that there are two roles for the service account. The Dialogflow API Client has been added as a role to your service account.

Now change the Google cloud project.

Select the project for your mbdmain agent.

Click on the ADD link below the project name.

In the new dialog window which opens, first add the exact service account email address associated with the Mega Agent. If you make a typo, you will usually see an error in this step. So make sure you don’t have any extra trailing white spaces etc.

Add the Dialogflow API Client role.

Effectively, what you have done here is provided Dialogflow API Client role (the ability to issue “detect intent” queries) on the mbdmain agent from your Mega agent.

Remember to save your changes.

Now you will see that the service account of the Mega Agent is added as a member with the Dialogflow API Client role.

Now repeat the same steps for the second sub-agent.

When you are done, you should see something similar to the image below.

Back in Dialogflow, select the Mega Agent.

Now test your Dialogflow Messenger integration inside this mega-agent. (Make sure you don’t accidentally end up testing the sub-agent, which would have worked anyway without all these steps).

* If you are not very clear on the concepts of service accounts, IAM, roles and permissions in Dialogflow, I would recommend my Dialogflow REST API v2 course. In fact, the course will also be useful to get a good idea of how Google APIs work in general.


Check out my free Udemy course: Zoho Deluge Script Quickstart for Programmers

This website contains affiliate links. See the disclosure page for more details. 
"The magic key I needed as a non-programmer"

The custom payload generator was the magic key I needed (as a non-programmer) to build a good demo with rich responses in DialogFlow Messenger. I've only used it for 30 minutes and am thrilled. I've spent hours trying to figure out some of the intricacies of DialogFlow on my own. Over and over, I kept coming back to Aravind's tutorials available on-line. I trust the other functionalities I learn to use in the app will save me additional time and heartburn.

- Kathleen R
Cofounder, gathrHealth
In this free course, I provide some tips for managing large Dialogflow ES bots without compromising on accuracy.