Simple, Distributed and Scalable PubSub in Erlang
This blog post is about how to build high scalable and distributed messaging-based applications using ErlBus, which is a lightweight and simple library to enable what we want here.
Since current release
0.2.0 (in progress), ErlBus was improved substantially. The current PubSub implementation was taken from the original, remarkable, and proven Phoenix PubSub Layer,
but re-written in Erlang.
You may be wondering ¿why not to include Phoenix as dependency and call it from Erlang?. Well, there are some thoughts about it:
Phoenix is a whole framework with different several modules, not only PubSub, so as a dependency, all sub-dependencies will be fetched too. Probably it isn’t a big deal, but seems like we got all Phoenix and we only use a 5% (rest will be wasted). Besides, PubSub is a simple, small, and great piece of Software (architecture and design is pretty good), so the goal was to have only that single and specific module to handle messaging, not the whole web framework.
Maintainability. In this case, we wanted to have the change to maintain an lightweight Erlang PubSub version and evolve independently.
Build gets more complex, you will need not only Erlang but also Elixir, and make sure that all dependencies that Phoenix brings with it and yours get compiled well. Deal with a single platform is easier than deal with two or more.
As we explained before, ErlBus is the Erlang clone of Phoenix PubSub, so the architecture and design are exactly the same.
Due to Phoenix PubSub architecture, ErlBus scales out pretty well, not only globally (in cluster) but also locally. When we start an ErlBus instance, it starts N number of shards, and this value is set in the
pool_size config parameter (by default
pool_size = 1). In order to have a better idea of this, we should run ErlBus.
Let’s build and start ErlBus:
Now we should have
ebus (or ErlBus) running into an Erlang console. So now within the console let’s run the
And, you should see a process tree like this:
In the image above, we see the main supervisor
ebus_ps_pg2, which supervises
ebus_ps_pg2_server and the
ebus_ps_local_sup supervises the local shards, and each shard has its own supervisor
ebus_supervisor which supervises the garbage collector
ebus_ps_gc and the local PubSub server
ebus_ps_local. Whole this runs on each instance of
Next step may be setup an Erlang Cluster using Distributed Erlang, and then we start
ebus on each one, a PG2 group is created and each local
ebus_ps_pg2_server is joined in order to received all broadcasted messages and then forward them locally. This is a pretty nice and highly scalable architecture.
Now let’s see some examples!
Open an Erlang shell running
Within the Erlang shell:
Now, let’s make it more fun, start two Erlang consoles, first one:
The second one:
Then what we need to do is put these Erlang nodes in cluster, so from any of them send a ping to the other:
Repeat the same thing above in
Once you have handlers subscribed to the same channel in both nodes, publish some messages from any node:
And in the other node you will see those messages have arrived too:
Let’s check subscribers, so from any Erlang console:
To learn more about it you can go HERE.
So far, so good! Let’s continue!
The great thing here is that you don’t need something special to implement a point-to-point behavior. It is as simple as this:
Let’s see an example:
Extremely easy isn’t?
ErlBus is a simple and usable library that make thousands times easier to build: soft-real-time and highly scalable messaging-based applications.
It was implemented taken as base a remarkable and proven framework like Phoenix Framework.
Next step is create a coarse-grained interface on top of ErlBus, like WebSockets, using Cowboy. So may be something similar to Phoenix Channels, isn’t?. See WEST.
Finally, to learn more check ErlBus, and ErlBus Examples.