I have created a chatbot to search across the autogenerated captions (subtitles) for all my YouTube videos. Unfortunately, YouTube does not actually generate subtitles for all videos for some reason, so I should qualify it by saying that it allows you to search across all video subtitles where they exist.
You can interact with the bot at the bottom right of this page.
The bot uses Dialogflow ES to do its search. I split the video subtitles into 60 second segments, and created one intent per segment.
The bot uses Zoho SalesIQ + Dialogflow ES integration.
I then added some custom code to the BotFlo FAQ Bot Generator to generate the agent ZIP file with the click of a button.
While this might not be the best search engine, it is actually a very interesting application of Dialogflow ES. 🙂
This is what the autogenerated subtitles looks like for the video above
All right so i wanted to make a video which uh gave a demonstration of a chatbot that i have added to my website and i think that it’s actually a really cool bot so what this bot does it actually allows you to search inside the auto generated youtube transcript that is the subtitles of all the videos in my youtube channel and it uses dialog flow for the search itself it uses dialogflow yes so i’ve created this bot it’s sort of a way for me to showcase a few things at once the first thing is that of course if people come to my site they will see a live example of dialogflow in action because this bot is after all it’s powered by dialogflow and what i’m going to do now is i’m just going to type something and search for it so what i’m going to try is embed dialog flow messenger widget in website okay so let’s see what it comes back with so it will take a couple of seconds and you see that it came back with a youtube result and of course you can see that the title of the video is embed chat widget into your website it’s talking about dialogue messenger and what you see here is the text is it’s actually a segment of the video which has all these words and these words are the auto generated subtitles okay but the best part is that when you click on this for the website and in you notice that it actually starts in the middle of the video in fact it’s right towards the end because it jumps directly to the segment of the video where you can see these subtitles okay so this is one way to see that you know this is a real dialog flow bot in the sense that it’s it’s nlu powered bot it’s an nlu powered bot because it tries to parse the words the user has used and then it’s going to come up with some result based on that so that’s the first thing that this bot is showcasing and the second thing is that this actually uses zoho sales iq chatbot integration with dialogflow and it’s actually a one click integration in other words to actually integrate zoho’s sales iq live chat widget with your dialog flow bot and this only works for es at the moment all you need to do is just do this connect with dialog flow button and it actually connects to your dialogflow account and it pulls all the chat bots into connected with your account into the zoho’s console and you can just choose the bot that you want to use so that’s all i had to do to get this integration to work and the third thing that i wanted to show you is that this bot so this is the actual all the intents in this bot and you notice that there are 70 pages worth of interns so given that each page has 20 intents that actually comes to about 70 into 20 that’s about 1400 total intents right that’s a lot of intense and i did not have to type out all this information naturally i used my botflow app to create a agent zip file where i was able to just get all this into a csv format and then i made it so that i was able to convert it into an agent zip file now i had to do some custom modifications to the existing bot flow app to be able to do that but what i wanted to show you is that it just takes all the so let’s go into some other intent so that i can show you a better example so let’s see if this is better okay so there are a few too few training phrases i’m looking for something which has more training phrases i guess let’s see this is looking better so you can see that there are a lot of training phrases and then the response is a custom payload which is a format which zoho sales iq understands okay so that’s that’s a bunch of things that this bot is automatically to showcase showcasing all these things in this single bot and that’s kind of why i created the spot now i want to talk a minute about the search quality of a bot like this now obviously this is not this is not exactly a great search engine for all the obvious reasons first of all the quality of the bot is inherently dependent on the quality of the auto generated subtitles in youtube and even though youtube’s subtitles are really good you know that they have some issues with more technical terms and of course my site is you know almost entirely technical stuff so it it does suffer from identifying the technical terms in the uh videos when they are spoken in the video so that’s one thing where the underlying text quality itself is not so great which which is going to reduce the search quality the other thing that you’ll notice is that longer queries actually lead to better search results more pinpointed results and that’s just what you would expect because of the way that i created this agent where all the intern training phrases are just text from the inside the transcript right so that’s the second thing and the other big problem with this search bot is that of course it doesn’t show multiple results so even though it gives you what it thinks is the best result if that’s not something that is what you want if you’re not satisfied by that result obviously you’re out of luck because it doesn’t show you anything else it’s just got one result and that’s just the way dialogflow is works at the moment right but i just wanted to show you all these things because this is a way to uh even though this you might say you know this some people might say yeah but you know this is not going to be very practical uh i kind of agree with that i don’t think that this is what you will turn to if you want to search across all the videos in a given youtube channel but this is a very interesting application of the dialogflow chatbot framework and also it it’s a good way to understand like how nlu that is natural language understanding how it works in general because building a search bot like this will give you a lot of insight into what is going on under the hood so anyway i just wanted to give you this little demo and it will be great if you can go and check out this chat bot on my website
This website contains affiliate links. See the disclosure page for more details.
"The magic key I needed as a non-programmer" The custom payload generator was the magic key I needed (as a non-programmer) to build a good demo with rich responses in DialogFlow Messenger. I've only used it for 30 minutes and am thrilled. I've spent hours trying to figure out some of the intricacies of DialogFlow on my own. Over and over, I kept coming back to Aravind's tutorials available on-line. I trust the other functionalities I learn to use in the app will save me additional time and heartburn. - Kathleen R Cofounder, gathrHealth