04 How to create an LLM powered custom website search using Vertex AI

Byaravindmc

LLM Powered Search

Vertex AI Search is very different from Vertex AI Agent Builder.

You can think of Vertex AI Search as a way to provide chat functionality inside a search widget, while Vertex AI Agent Builder is a way to provide search functionality inside a chat widget.

Here is another way to think about Vertex AI Search – it is like adding an advanced Google Custom Search Engine based on your website content, which has additional features like

  • LLM generated summary response
  • ability to ask follow up questions (i.e. chat with the response)
  • filters for structured data
  • built-in support for Document AI

In my opinion, Vertex AI Search has the potential to be something of a paradigm shift when it comes to implementing custom search applications.

First create an app.

Choose Search app.

I kept the default values and provided a name for my search app.

Connect the appropriate data store (I have only one).

Go to the Search preview and test the search widget

This is the full response (the search results are implemented as an infinite scroll, so I selected the top 10 results, and I also added a separator between the summary response and the list of results to make it easier to read)


## Converting a Mindomo Flowchart to a Dialogflow ES Bot:

Mindomo offers a convenient way to design and manage your Dialogflow ES bot's flowchart. [ 4 ] Here's a step-by-step guide to convert your Mindomo flowchart:

1. Write the Conversation Script:

Start by writing out the conversation script directly within your Mindomo map. [ 3 ] This script should follow the decision tree structure of your bot, similar to the Dichotomous Key bot example.

2. Create the Flowchart:

In Mindomo, click "Create" and hover over "Organigram" to create a flowchart. [ 3 ] Use rectangles to represent both the training phrase and the bot's response, as they form a single block in Dialogflow. [ 2 ]

3. Organize and Manage:

Mindomo's features make it easy to organize and manage your flowchart. You can:

Use multiline nodes: Format different lines within a single node with different styles. [ 5 ]
Support different shapes: Use different shapes for different nodes to visually distinguish them.
Automatically reformat: Mindomo automatically reformats your layout to an org-tree structure for better organization.
Drill down: Drill down into individual branches for a closer look at specific parts of your flowchart.
Add notes: Add notes to individual nodes for additional information or context.
Share and collaborate: Easily share your flowchart with others and collaborate on its development. [ 5 ]

4. Migrate to Dialogflow ES:

Once your flowchart is complete, you can migrate it to Dialogflow ES. To keep track of your progress, add a "Completed Icon" to each intent in Mindomo. [ 1 ] This allows you to check off intents as you migrate them to Dialogflow ES.

5. Use Conversation Analytics:

Dialogflow's Conversation Analytics tool can help you analyze your bot's performance and identify areas for improvement. You can use the Search tool to convert your agent ZIP files into a CSV format and import them into Airtable for further analysis. [ 4 ]

<-------------------------------------------------------------------------------->

[ 1 ] https://botflo.com/04-the-easiest-way-to-migrate-your-dialogflow-es-bot-to-dialogflow-cx/

04 The easiest way to migrate your Dialogflow ES bot to Dialogflow CX
You can migrate the bot step by step by adding icons into "intents" Inside each Mindomo intent, you can add a Completed Icon (like the Green one I use, but you can use any icon you want) and then check off the intent as soon as you migrate it over to ES.

[ 2 ] https://botflo.com/01-a-guide-to-dialogflow-flowcharts/

01 A Guide to Dialogflow Flowcharts
Use a rectangle for representing training phrase and bot response You should put both the training phrase and the bot response inside a single rectangle, since it is conceptually a single block in Dialogflow. For example, here is what it looks like in the Dichotomous Key bot.
[ 3 ] https://botflo.com/03-write-the-conversation-script/

03 Write the conversation script
Dialogflow ES Flowcharts The first step is to write out the conversation script into your mindmap directly. This is based on the decision tree bot which can identify a vertebrate class. The flowchart of the decision tree bot looks like this: Inside Mindomo, click on Create Hover your mouse over the Organigram.
[ 4 ] https://botflo.com/dialogflow-conversation-analytics-tips-2/

Dialogflow Conversation Analytics Tips
I used the Search tool to convert my agent ZIP files into a CSV, which I then imported into Airtable. Design a Flowchart I used Mindomo to design the bot flowchart and maintain versions. It made it much easier for me to both design the bot and to translate it into a Dialogflow ES agent.
[ 5 ] https://botflo.com/02-why-mindomo-is-the-best-tool-for-creating-a-dialogflow-flowchart/

02 Why Mindomo is the best tool for creating a Dialogflow flowchart
I have compiled a list based on this article Features draw.io XMind desktop Mindomo Multiline nodes Yes Yes Yes Format different lines inside a single node with different styles Yes No Yes Support different shapes for different nodes Yes Yes Yes Automatically reformat layout to org-tree structure No Yes Yes Support drill down into individual branches No Yes Yes Node labels No Yes No Support adding notes to individual nodes No Yes Yes Easy to share Yes No Yes Easy to collaborate Yes No Yes Aesthetics Possible Good Good Price Free One time payment Recurring payment You can draw 3 free mindmaps using Mindomo and decide for yourself if it works for you.

[ 6 ] https://botflo.com/16-how-to-create-a-quiz-bot-in-dialogflow-es/

16 How to create a Quiz Bot in Dialogflow ES
Finally, when user answers question 5 correctly, we need to call the webhook. Similarly we also call the webhook if user provides the incorrect answer for question 5. This is the core of the Python webhook code def calculate_score(req): query_result = req.get('queryResult') session_vars = get_session_vars(query_result) sum_of_scores = get_sum_of_scores(session_vars) fulfillment_messages = json.dumps(query_result.get('fulfillmentMessages')) return_msg = fulfillment_messages.replace('_SCORE_', str(sum_of_scores)) json_formatted = json.loads(return_msg) result = { "fulfillmentMessages": json_formatted } with open('result.json', 'w', encoding='utf-8') as f: json.dump(result, f, ensure_ascii=False, indent=4) return result The get_sum_of_scores just calculates the sum of the 5 scores def get_sum_of_scores(session_vars): session_vars_params = session_vars.get('parameters') score1 = int(session_vars_params.get('score1')) score2 = int(session_vars_params.get('score2')) score3 = int(session_vars_params.get('score3')) score4 = int(session_vars_params.get('score4')) score5 = int(session_vars_params.get('score5')) return score1 + score2 + score3 + score4 + score5 Here is what the bot looks like in Dialogflow Messenger.

[ 7 ] https://botflo.com/06-translate-the-flowchart-into-dialogflow-es-agent/

06 Translate the flowchart into Dialogflow ES agent
Here is the intent definition. In the case of intent 3, we have both an input and an output context. Here is the intent definition You can do the same for the rest of the intents and fully translate the flowchart into a Dialogflow agent.

[ 8 ] https://botflo.com/dialogflow-tutorial-setting-context-from-your-inline-webhook-using-contextout/

How to manage context from your Python webhook in Dialogflow ES
On the other hand, if it captured the complete birthday, DialogFlow will simply send a message prompting for the next response. The flowchart As I mentioned in previous posts, I suggest creating flowcharts for your chatbots using Mindomo. So here is the Mindomo flowchart which describes this flow.

[ 9 ] https://botflo.com/14-how-to-manage-context-from-your-python-webhook-in-dialogflow-es/

14 How to manage context from your Python webhook in Dialogflow ES
On the other hand, if it captured the complete birthday, DialogFlow will simply send a message prompting for the next response. The flowchart As I mentioned in previous posts, I suggest creating flowcharts for your chatbots using Mindomo. So here is the Mindomo flowchart which describes this flow.

[ 10 ] https://botflo.com/dialogflow-knowledge-connectors/

Dialogflow Knowledge Connectors
You need to do a couple of things to prepare the CSV file for importing into Dialogflow. a) delete the third colum (categories) b) delete the top header row as it is not required Enable beta features Now, inside Dialogflow, create a new agent and make sure you have enabled Beta features.

As you can see, Vertex AI search finds the list of relevant articles, and then also provides a high level summary so that in addition to a list of search results, the end user can also get an answer (with relevant citations).

And the nice thing about Vertex AI search is that you can ask follow up questions (in other words, you can chat with the search results)

Note: This is my old website and is in maintenance mode. I am publishing new articles only on my new website. 

If you are not sure where to start on my new website, I recommend the following article:

Is Dialogflow still relevant in the era of Large Language Models?

Leave a Reply