Getting Started with Google DialogFlow and Custom Search Engine using .NET Core.

Photo by Volodymyr Hryshchenko on Unsplash

Hello! In this post, I’m going to cover how I built my first chatbot, StanLeeBot, with Google DialogFlow which retrieves information from Marvel and DC Comics’ website using Google’s Custom Search Engine. In some integrations, the chatbot can shorten urls too!

I’m going to make a couple assumptions before moving forward:

If you wish to skip through everything and check out the code, you can view it on GitHub. However, I apologize it cannot be run locally without adding your own user secrets or appSettings.json file.

For the purposes of this tutorial and to keep things simple, I’m going to cover the first step: Gathering data from a website (I’m just going to do Marvel for this tutorial) so that we can eventually display it to our users when they request it via our Chatbot.

First, we need to create a custom search engine. That can be done by going to https://cse.google.com/cse/create/new. Fill the form out to be like below:

Once you click “Create” the page will refresh and you’ll see three buttons on the right: Get Code, Public URL, and Control Panel. Let’s click “Public URL” to test things out.

Great! Looks like it’s working. Before we head to the next step, you’ll notice in the address bar a “cx” querystring. You’ll want to save that value someplace for future reference.

Alright, so we have a custom search engine, but we need to interact with it in such a way that allows us (developers) to get the information we want. So, the next step in our journey is to set up our Google Custom Search JSON API. Head on over to https://developers.google.com/custom-search/v1/overview. Scroll downwards to the “API key” section and click “Get a Key” fill out the form and click “Next”.

After some time, you’ll receive an API Key. Save that someplace secure. For the purposes of this tutorial, we won’t be going over best practices for saving API Keys or “secrets.”

Now, we have the information available to view the search results in a JSON format.

Here’s the format of the URL for you to paste in your own values: https://www.googleapis.com/customsearch/v1?cx={cxKey}&key={apiKey}&q={searchTerm}

And BAM! Now we have something we can work with. Here’s where things get a bit more technical, but most of the heavy lifting is already done for you!

The first thing we need to do is parse the JSON. Using a HttpClient we’re going to query the URL and get data back as a string asynchronously. You can view the code I have here. Then, using JsonConvert, we take the result and parse it into a nice model called GoogleSearchResponse.

The next steps I took for implementation took several rounds of development, but I finally came up with a solution I’m happy with (for now). In essence, I have a MarvelSearchBuilder class that takes the GoogleSearchResponse object and parses out the title, bio, image url, and website. (Note that I’m using a lot of extension methods in this class from another project I work on called BabouExtensions.) This information is then passed into a DialogFlowResponse Card object. All we’re doing at this point is taking information from the website and then building some objects to be used by DialogFlow. On the first iteration, don’t worry about custom payloads. Get the default DiaglogFlow screen working first.

Ok, so we have the custom search engine, we’re parsing the JSON search results, and building a response card ready for consumption. The next step is building out the API interface that DialogFlow can speak with. You can check out my implementation which verifies the header name and value matches. At the end of the day, there isn’t much to this API endpoint. Checkout the logic for how I handle returning the objects here. Once you have the endpoint configured, you can run your API locally using a nifty tool called ngrok. This allows you to have your application run locally, but accessible from the internet.

The next step is to setup DialogFlow. Go to https://dialogflow.cloud.google.com/ and proceed with the agreement. Click “Create Agent” and give it a name and click create and wait a few seconds.

DialogFlow automatically created a “Default Welcome Intent” (I renamed it to Hello), so let’s go there. For the responses, clear them out and have one question: Would you like to lookup Marvel or DC? Next, we need some follow up intents. Add an intent in case someone replies “Marvel” and then another intent in case someone replies “DC”. Be sure to add training phrases as well for each. Since we’re not going to cover DC, if someone replies back “DC” set the response to “Sorry, this feature is under construction.”

For the “Marvel” training intent, let’s prompt the user with something like, “Alright, Marvel. What do you want to lookup? Try Spiderman or Wolverine vs Deadpool…”. Finally, add another follow up intent which expects the user’s query.

It should look like this:

In the final intent we want the training phrase to be anything, so enter a training phrase (literally type anything), highlight it, and select “@sys.any”

This is also where you’re going to fulfill the conversation with your response card object that we built earlier. Your screen should look similar to

Theoretically, you can put in a text response there and test out the workflow before proceeding to the next step which is the web fulfillment response message.

Alright, save the intent and go to “Fulfillment” on the left hand side and put in the details from ngrok, your header name, and your header value.

After saving the agent, it should all come together. When you walk through DialogFlow on the right hand side, type, “Hi”. Then “Marvel” and then a search term and voila!

If you want to test out the StanLeeBot,the easiest way is probably via Telegram as the Slack version is currently waiting reapproval (was using slash commands).

Thank you for reading!

Be sure to check out my blog, AJ Knows Tech, and if you enjoyed this article, please consider signing up for my newsletter.

I’m a geek and software developer with a passion for the human side of technology. On Medium, you’ll find me discussing everything from life to .NET 🚀.

Get the Medium app

A button that says 'Download on the App Store', and if clicked it will lead you to the iOS App store
A button that says 'Get it on, Google Play', and if clicked it will lead you to the Google Play store