The people who proclaim that chatbots are the future haven’t seen the first IRC chatbots from the 1990’s. Especially the project called Eggdrop from 1993.
I have been looking at ways to create awesome chatbots for a plethora of projects. I had chosen to focus specifically on Telegram.
Why Telegram?
Why telegram and not something else? Telegram has an open API which just
works. It’s simple and effective. Pretty complex bots can be created.
Telegram has groups and channels which can host thousands of people at
the same time. Bots can join those channels and groups and provide
functionality there.
What’s great about Telegram is that they provide you with 2 options to get updates for your bot.
You can either use the getUpdates function which asks the Telegram
servers for updates.
Or we can set up a webhook. Meaning that we have our own domain name and
Telegram sends us the updates for each bot.
The difference is that the getupdates version can make use of "long
polling" in which we can set it up to get updates once every few
seconds.
The benefits of getupdates and polling is that you do NOT need your own
webserver with a static IP, domain and SSL certificate.
It can run from a small raspberry pi. The downside is that it’s
constantly going to ask for updates even if there are none.
For big bots which have hundreds of requests per second a webhook might be better suited.
Another cool thing which I loved about Telegram is that your updates are
kept for a few hours up to 24 hours in the cloud.
Meaning that if your bot for any reason can’t process the update from
Telegram, say it’s offline, crashed or you’re upgrading it. You won’t
lose ANY message. I find this extremely useful as a 99.99% functionality
uptime can be ensured.
Why the need for Alexia a new Telegram bot library?
There are at least 4 Telegram libraries for Elixir. They each seem to
work in a similar way. The one I liked and seemed to work directly out
of the box was Nadia.
Nadia is full featured because you can send and download files. Send
images, messages, documents.
Joining groups and answering to groups. Setting up inline answers.
Since telegram is so extensible it supports a full range of Telegram
options.
I haven’t been able to find a full featured library for Facebook as Nadia is for Telegram. However I’ve created yet another Telegram bot library for Elixir. It’s called Alexia. Long story short after 2 weeks form sending a pull request and receiving no answer I decided to fork Nadia and added my own enhancements and improvements.
ONe thing I wanted to do is add multibot functionality. I don’t want to
create a new application for each bot I need. Maybe I want to create 5
different bots.
Or I want to create one bot which has a similar functionality which can
be replicated.
I played around with supervisors. i created one supervisor which
started. First I needed to modify everything underlying. To send a
token.
Then I added the supervisor system . Supevisors for registry. I want to
hvae each bot’s name for the polller and matcher.
Then I started to look at the poller supevisor. For each bot there is a
poller. From within the poller I dynamically started the process with
the correct supervisor for the matcher.
Whenever a bot/polller crashes I wanted to be sure it doesn’t crash everything. Plus, each matcher runs it’s own Task as a separate process. This way if I want to process a video to audio conversion It won’t lag the whole system and will make use of real concurrency.
I wanted to be able to add bots at runtime not at compiletime. This
system made it extremely flexible to add new bots anytime.
Each bot can have it’s own module for commands and it can dispatch it’s
own set of commands.
Getting Started and using Alexia to Create Bots!
[alexia]#
You can add this bot to any project. Be it a simple elixir app or even a phoenix app. Umbrella or simple? It doesn’t matter since it’s going to work anyway.
Installation
Add Alexia to your mix.exs dependencies:
def deps do [{:alexia, "~> 0.5.3"}]endAfter you run mix deps.get you 'll need to configure your
config/config.exs. I’d recommend putting your tokens in
config/dev.secret.ex and config/prod.secret.ex. You can even consider
encrypting them to keep them safe.
You can add as many bots as you like.
config :alexia, bots: [ %{bot_name: "AlexandraBot", commands_module: YourAppModule.AlexandraBot.Commands, token: Base.decode64!("KFEDOSKYG5KUQSZVK5CDKU22INEUUWCZGRLDIRKPJ5BEUQZWK5IA"), webhook: "https://yourdomain.example.com/your-telegram-update-link/"}, %{bot_name: "MegaCoolBot", commands_module: YourAppModule.MegaCoolBot.Commands, token: Base.decode64!("KZDU2VBUJZJUQQJSIFLVMT2SGZCVMWKYKFKUOQ2OKNHU4QSXIU2Q")}], secret_mix: "MOq8cDjlwEBoLi88TXfGY+HeQllySLgEuObNUr006Ug"
And then, in mix.exs, list :alexia as an application inside
application/0:
def application do [applications: [:alexia]]end
Now Mix will guarantee the :alexia application is started before your
application is started. This is "required" because we’ve set up the
:alexia config for
the bots. Otherwise we get an warning that there is no such app.
Using the built-in Governor "framework" to handle multiple bots
The Governor framework is just a functionality enhancement which adds
supervisor trees to handle the process Registry, bot Pollers and Bot
Matchers.
Taking care of the fault tolerance part so you can focus on programming
and creating useful bots. This makes it easy to have multiple bots
within the same application.
The Alexia.Supervisor.BotSupervisor starts 2 registries
Registry.BotPoller and Registry.BotMatcher to keep track of the
Poller and Matcher processes. It also starts 2 supervisors
+Alexia.Supervisor.Matcher +`and+ Alexia.Supervisor.Poller.+`
Using the Governor Supervisors edit YourApp.Application and add it to the supervisor children list.
To make use of those Governor Supervisors add this to your application supervisor
{Alexia.Supervisor.BotSupervisor, Application.get_env(:alexia,:bots)},
What happens is on Startup the settings are read from
:alexia, bots: [...]. For each bot which doesn’t have the
webhook: setting there is a Poller process started which polls the
Telegram servers every 1 second.
Each poller starts it’s own Matcher process which allows you to specify
which module your application must run the code in order to process the
messages/updates. This is done via the commands_module: option on a
per bot basis. The matcher runs the command under `+Task.start +`so that
each task is independent and can take it’s time to process the request.
This way on a multicore server we can ensure true parallelism.
Should we decide to use multiple nodes and clusters for the bots then a
Task.Supervisor can be used instead.
Webhooks
For webhooks to work you need to follow some steps. Webhooks allow your
bots to process your data as it comes in. Setting up webhooks under
Phoenix is extremely simple as you will see soon enough.
In case you have given the webhook: key for a bot then it will NOT
start a poller. It will instead only start a Worker which sets up the
webhook link accordingly and creates a hashed token which can be used
for identification.
Setting up webhooks requires that you have the following:
A valid domain name
A valid SSL certificate
A phoenix webserver set up to acceot incoming POSTs from 149.154.167.197-233 on port 443,80,88 or 8443.
Setting up a route in router.ex and
Setting up the proper commands in the controller
Review this guide about telegram webhooks to get an idea of what you actually need to do. I’ll only show the last 2 steps and hope that the first 3 steps have been completed.
setting up router.ex
We’re setting up 1 POST action in the router so that all our bots can
use it
scope "/someapi", YourAppWeb do pipe_through :api post "/telegram_notify/:botkey", TelegramBotUpdateController, :updateend
TelegramBotUpdateController.ex
Our Controller will have only one function which will handle the command actions for all started bots.
defmodule LbaWeb.TelegramBotUpdateController do use LbaWeb, :controller require Logger alias Alexia.Model.Update defstruct botkey: nil #See https://core.telegram.org/bots/webhooks # Telegram IP range is 149.154.167.197-233 @telegram_ips Enum.map(197..233, fn (x) -> "149.154.167." <> Integer.to_string(x) end) def update(conn, params) do [ip | _] = Plug.Conn.get_req_header(conn, "x-real-ip") # Logger.info "Allow ip? #{inspect ip} #{ip in @telegram_ips} " handle_update(conn,params,ip in @telegram_ips) end defp handle_update(conn,%{"botkey" => botkey} =params,true) do update_message = Utilities.snake_case_map(params) |> Alexia.Parser.parse_result("getUpdate") bot_matcher = Alexia.Governor.get_bot_info(botkey) if !is_nil(bot_matcher) do Alexia.Governor.Matcher.match(bot_matcher, update_message) conn |> json %{status: true} else conn |> json %{status: false, error: "Unknown Bot/Command"} end end defp handle_update(conn,_params,false) do conn |> json %{status: false, error: "Disallowed. Nope, not in the Telegram IP range!"} endenddefmodule Utilities dodef snake_case_map(map) when is_map(map) do Enum.reduce(map, %{}, fn {key, value}, result -> Map.put(result, String.to_existing_atom(Macro.underscore(key)), snake_case_map(value)) end) end def snake_case_map(list) when is_list(list), do: Enum.map(list, &snake_case_map/1) def snake_case_map(value), do: valueendThe snake_case_map converts all "strings" to atom keys: based on the fact that it expects the atom to already exist. This is why i’ve set up defstruct botkey: In case you catch any errors of new functionalities that exist then it’s indicated to add those atoms somewhere in your app.
Creating YourBotApp.Commands modules for each bot
Each bot has a commands_module: key which specifies the module you want to use for that specific bot. There is only one function which is dispatched, namely command(). With pattern matching you can make it match almost anything.
Below is a code example of the usage. The token is passed so you can send an answer using that token. I did not use macro’s to keep the code clean since my intention is to add new bots at runtime.
defmodule YourBot.Commands do #alias Alexia.Model #Handling Messages from chats to their own command def command(%{message: %{text: text} } = update, token) do text_command(text,update,token) end #Handy when using the Up arrow! and editing the previous message def command(%{edited_message: %{text: text} } = update, token) do text_command(text,update,token) end #Inline queries see documentation def command(%{inline_query: %{query: query}} = update,token) do inline_query_command(query,update,token) end #Callback queries, see documentation def command(%{callback_query: %{data: data}} = update,token) do callback_query_command(data,update,token) end #Catchall, can be used for debugging/Testing OR #Showing your default help/answer for unknown requests def command(update, token), do: default_reply(update,token) #Example of externalizing to another module def inline_query_command("troll" <> _,update,token), do: Testing.inline_query_command("troll",update,token) def inline_query_command(_query,update,token) do #Default end def callback_query_command("/choose" <> _,update,token), do: Testing.callback_query_command("/choose",update,token) def callback_query_command(_data,update,token) do Alexia.answer_callback_query token,update.callback_query.id, text: "Default callback." end def text_command("hello",update,token) do Alexia.send_message(token,Alexia.Governor.get_chat_id(update),"Well, hello there! #{update.message.from.first_name}") end #Externalize to a different module def text_command("/yourcommand",update,token), do: Testing.text_command("/yourcommand",update,token) def text_command(_,update,token) , do: default_reply(update,token) #The default reply def default_reply(update,token) do Alexia.send_message(token,Alexia.Governor.get_chat_id(update), "Sorry, that command is NOT yet implemented!") endend
Each bot’s worker will have the token in it’s own unique process.
Whenever there is a message either from a poller or a webhook then it
sends the command toYourBot.Commands +`and it tries to match a specific `+command()
function. You’re free to match whatever you want. The above
demonstration is just an example of what can be achieved.
I’ve built some pretty complex bots which I will detail in a future post.
Did it work? Do you have some questions? Want something specific? Comment below or send me a message via the contact page.