Alexia Telegram Bot Library for Elixir Multi Bot Environments and Supervisor

The people who proclaim that chatbots are the future haven't seen the first IRC chatbots from the 1990's. Especially the project called Eggdrop from 1993.

Want to get started? Jump over the introduction and start creating your own Telegram bots with Alexia.

I have been looking at ways to create awesome chatbots for a plethora of projects. I had chosen to focus specifically on Telegram.

Why Telegram?

Why telegram and not something else? Telegram has an open API which just works. It's simple and effective. Pretty complex bots can be created.
Telegram has groups and channels which can host thousands of people at the same time. Bots can join those channels and groups and provide functionality there.

What's great about Telegram is that they provide you with 2 options to get updates for your bot.

You can either use the getUpdates function which asks the Telegram servers for updates.
Or we can set up a webhook. Meaning that we have our own domain name and Telegram sends us the updates for each bot.

The difference is that the getupdates version can make use of "long polling" in which we can set it up to get updates once every few seconds.
The benefits of getupdates and polling is that you do NOT need your own webserver with a static IP, domain and SSL certificate.
It can run from a small raspberry pi. The downside is that it's constantly going to ask for updates even if there are none.

For big bots which have hundreds of requests per second a webhook might be better suited.

Another cool thing which I loved about Telegram is that your updates are kept for a few hours up to 24 hours in the cloud.
Meaning that if your bot for any reason can't process the update from Telegram, say it's offline, crashed or you're upgrading it. You won't lose ANY message. I find this extremely useful as a 99.99% functionality uptime can be ensured.

Why the need for Alexia a new Telegram bot library?

There are at least 4 Telegram libraries for Elixir. They each seem to work in a similar way. The one I liked and seemed to work directly out of the box was Nadia.
Nadia is full featured because you can send and download files. Send images, messages, documents.
Joining groups and answering to groups. Setting up inline answers.
Since telegram is so extensible it supports a full range of Telegram options.

I haven't been able to find a full featured library for Facebook as Nadia is for Telegram. However I've created yet another Telegram bot library for Elixir. It's called Alexia. Long story short after 2 weeks form sending a pull request and receiving no answer I decided to fork Nadia and added my own enhancements and improvements.

ONe thing I wanted to do is add multibot functionality. I don't want to create a new application for each bot I need. Maybe I want to create 5 different bots.
Or I want to create one bot which has a similar functionality which can be replicated.
I played around with supervisors. i created one supervisor which started. First I needed to modify everything underlying. To send a token.
Then I added the supervisor system . Supevisors for registry. I want to hvae each bot's name for the polller and matcher.
Then I started to look at the poller supevisor. For each bot there is a poller. From within the poller I dynamically started the process with the correct supervisor for the matcher.

Whenever a bot/polller crashes I wanted to be sure it doesn't crash everything. Plus, each matcher runs it's own Task as a separate process. This way if I want to process a video to audio conversion It won't lag the whole system and will make use of real concurrency.

I wanted to be able to add bots at runtime not at compiletime. This system made it extremely flexible to add new bots anytime.
Each bot can have it's own module for commands and it can dispatch it's own set of commands.

Getting Started and using Alexia to Create Bots!

You can add this bot to any project. Be it a simple elixir app or even a phoenix app. Umbrella or simple? It doesn't matter since it's going to work anyway.

Installation

Add Alexia to your `mix.exs` dependencies:

def deps do  [{:alexia, "~> 0.5.3"}]end

After you run mix deps.get you 'll need to configure your config/config.exs. I'd recommend putting your tokens in config/dev.secret.ex and config/prod.secret.ex. You can even consider encrypting them to keep them safe.
You can add as many bots as you like.

config :alexia, bots: [  %{bot_name: "AlexandraBot", commands_module: YourAppModule.AlexandraBot.Commands,  token:  Base.decode64!("KFEDOSKYG5KUQSZVK5CDKU22INEUUWCZGRLDIRKPJ5BEUQZWK5IA"),    webhook: "https://yourdomain.example.com/your-telegram-update-link/"},  %{bot_name: "MegaCoolBot", commands_module: YourAppModule.MegaCoolBot.Commands,  token: Base.decode64!("KZDU2VBUJZJUQQJSIFLVMT2SGZCVMWKYKFKUOQ2OKNHU4QSXIU2Q")}],  secret_mix: "MOq8cDjlwEBoLi88TXfGY+HeQllySLgEuObNUr006Ug"

And then, in `mix.exs`, list `:alexia` as an application inside `application/0`:

def application do  [applications: [:alexia]]end

Now Mix will guarantee the `:alexia` application is started before your application is started. This is "required" because we've set up the :alexia config for the bots. Otherwise we get an warning that there is no such app.

Using the built-in Governor "framework" to handle multiple bots

The Governor framework is just a functionality enhancement which adds supervisor trees to handle the process Registry, bot Pollers and Bot Matchers.
Taking care of the fault tolerance part so you can focus on programming and creating useful bots. This makes it easy to have multiple bots within the same application.
The Alexia.Supervisor.BotSupervisor starts 2 registries Registry.BotPoller and Registry.BotMatcher to keep track of the Poller and Matcher processes. It also starts 2 supervisors Alexia.Supervisor.Matcher and Alexia.Supervisor.Poller.

Using the Governor Supervisors edit YourApp.Application and add it to the supervisor children list.

To make use of those Governor Supervisors add this to your application supervisor

  {Alexia.Supervisor.BotSupervisor, Application.get_env(:alexia,:bots)},

What happens is on Startup the settings are read from :alexia, bots: [...]. For each bot which doesn't have the webhook: setting there is a Poller process started which polls the Telegram servers every 1 second.
Each poller starts it's own Matcher process which allows you to specify which module your application must run the code in order to process the messages/updates. This is done via the commands_module: option on a per bot basis. The matcher runs the command under Task.start so that each task is independent and can take it's time to process the request. This way on a multicore server we can ensure true parallelism.
Should we decide to use multiple nodes and clusters for the bots then a Task.Supervisor can be used instead.

Webhooks

For webhooks to work you need to follow some steps. Webhooks allow your bots to process your data as it comes in. Setting up webhooks under Phoenix is extremely simple as you will see soon enough.
In case you have given the webhook: key for a bot then it will NOT start a poller. It will instead only start a Worker which sets up the webhook link accordingly and creates a hashed token which can be used for identification.

Setting up webhooks requires that you have the following:

• A valid domain name
• A valid SSL certificate
• A phoenix webserver set up to acceot incoming POSTs from 149.154.167.197-233 on port 443,80,88 or 8443.
• Setting up a route in router.ex and
• Setting up the proper commands in the controller

Review this guide about telegram webhooks to get an idea of what you actually need to do. I'll only show the last 2 steps and hope that the first 3 steps have been completed.

setting up router.ex

We're setting up 1 POST action in the router so that all our bots can use it
 

scope "/someapi", YourAppWeb do  pipe_through :api  post "/telegram_notify/:botkey", TelegramBotUpdateController, :updateend

TelegramBotUpdateController.ex

Our Controller will have only one function which will handle the command actions for all started bots.

defmodule LbaWeb.TelegramBotUpdateController do  use LbaWeb, :controller  require Logger  alias Alexia.Model.Update  defstruct botkey: nil    #See https://core.telegram.org/bots/webhooks  # Telegram IP range is 149.154.167.197-233   @telegram_ips Enum.map(197..233, fn (x) -> "149.154.167." <>  Integer.to_string(x)  end)  def update(conn,  params) do      [ip | _] =  Plug.Conn.get_req_header(conn, "x-real-ip")   #   Logger.info "Allow ip? #{inspect ip} #{ip in  @telegram_ips} "  handle_update(conn,params,ip in  @telegram_ips)  end  defp handle_update(conn,%{"botkey" => botkey} =params,true) do    update_message = Utilities.snake_case_map(params)     |> Alexia.Parser.parse_result("getUpdate")      bot_matcher = Alexia.Governor.get_bot_info(botkey)    if !is_nil(bot_matcher) do      Alexia.Governor.Matcher.match(bot_matcher, update_message)      conn      |> json %{status: true}    else      conn |> json %{status: false, error: "Unknown Bot/Command"}    end  end  defp handle_update(conn,_params,false) do    conn |> json %{status: false, error: "Disallowed. Nope, not in the Telegram IP range!"}  endenddefmodule Utilities dodef snake_case_map(map) when is_map(map) do    Enum.reduce(map, %{}, fn {key, value}, result ->      Map.put(result, String.to_existing_atom(Macro.underscore(key)), snake_case_map(value))    end)  end  def snake_case_map(list) when is_list(list), do: Enum.map(list, &snake_case_map/1)  def snake_case_map(value), do: valueend

The snake_case_map converts all "strings" to atom keys: based on the fact that it expects the atom to already exist. This is why i've set up defstruct botkey: In case you catch any errors of new functionalities that exist then it's indicated to add those atoms somewhere in your app.

Creating YourBotApp.Commands modules for each bot

Each bot has a commands_module: key which specifies the module you want to use for that specific bot. There is only one function which is dispatched, namely command(). With pattern matching you can make it match almost anything.

Below is a code example of the usage. The token is passed so you can send an answer using that token. I did not use macro's to keep the code clean since my intention is to add new bots at runtime.

defmodule YourBot.Commands do    #alias Alexia.Model  #Handling Messages from chats to their own command  def command(%{message: %{text: text} } = update, token) do    text_command(text,update,token)  end  #Handy when using the Up arrow! and editing the previous message    def command(%{edited_message: %{text: text} } = update, token) do      text_command(text,update,token)    end  #Inline queries see documentation  def command(%{inline_query: %{query: query}} = update,token) do    inline_query_command(query,update,token)  end  #Callback queries, see documentation  def command(%{callback_query: %{data: data}} = update,token) do    callback_query_command(data,update,token)  end  #Catchall, can be used for debugging/Testing OR  #Showing your default help/answer for unknown requests  def command(update, token), do: default_reply(update,token)  #Example of externalizing to another module  def inline_query_command("troll" <> _,update,token), do: Testing.inline_query_command("troll",update,token)  def inline_query_command(_query,update,token) do    #Default  end  def callback_query_command("/choose" <> _,update,token), do: Testing.callback_query_command("/choose",update,token)  def callback_query_command(_data,update,token) do      Alexia.answer_callback_query token,update.callback_query.id,      text: "Default callback."  end  def text_command("hello",update,token)  do    Alexia.send_message(token,Alexia.Governor.get_chat_id(update),"Well, hello there! #{update.message.from.first_name}")  end  #Externalize to a different module  def text_command("/yourcommand",update,token),  do: Testing.text_command("/yourcommand",update,token)  def text_command(_,update,token) , do: default_reply(update,token)  #The default reply    def default_reply(update,token) do      Alexia.send_message(token,Alexia.Governor.get_chat_id(update), "Sorry, that command is NOT yet implemented!")    endend

Each bot's worker will have the token in it's own unique process. Whenever there is a message either from a poller or a webhook then it sends the command to
YourBot.Commands and it tries to match a specific command() function. You're free to match whatever you want. The above demonstration is just an example of what can be achieved.

I've built some pretty complex bots which I will detail in a future post.

Did it work? Do you have some questions? Want something specific? Comment below or send me a message via the contact page.