ErlBus: Erlang Message Bus
IMPORTANT:: This blog post was written based on ErlBus 0.1.0. Now there is a new release in progress (
0.2.0), with a totally different implementation and substantial improvements. To read more about it, you can go to this Blog Post or directly to the GitHub Repo.
Messaging in Erlang is easy by default, because of its nature. But, why not make it even easier? ridiculously easy, and fun. Well, this is where ErlBus comes in; an Open Source project created to provide all what you need around messaging, including quality attributes such as: high performance and scale-out.
Let’s start doing an implementation of a simple but very powerful messaging pattern: Publish/Subscribe. In Erlang there are different ways to do it, but I’m going to mention only two (the common ones):
- Using the native tool/library pg2.
- Using an external tool/library like gproc.
Pub/Sub with pg2
pg2 doesn’t provides explicit Pub/Sub functions, so we have to implement that logic ourselves, using the given primitives.
Things to highlight here:
It’s easy, but you have to code more and deal with some annoying things like errors or exceptions.
We have to code the logic for receive incoming messages (a listener), in addition to the message handling logic, and we should only be concerned for handle/process our messages.
You will be violating the DRY principle soon, because you’ll end up coding the same thing over and over again.
Pub/Sub with gproc
With gproc happen almost the same situation, but at least provides a pub/sub module
gproc_ps to abstract that logic better.
Almost the same thing is it? with some minor differences of course. Again, we could highlight exactly the same points as previously with pg2.
Either with gproc or pg2 we’re limited to the pub/sub basics. We don’t have:
Event-driven consumers that allow us implement the message handling logic.
Task executors or worker pools, in case that we need distribute load across of a pool of workers.
Different scaling strategies, different to
pg2global model, or
And other things, but don’t worry, we’ll cover all these things later.
Pub/Sub with ErlBus
Now we are on the heart of all this, ErlBus is a very lightweight and simple approach to build messaging-based apps. Messaging infrastructure that provides: Publish/Subscribe, Point-To-Point, Event Driven Consumer (Message Handler), Task Executor, etc., all that is missing in the previously explored options, all what you really need and expect from a messaging tool.
ErlBus has two main modules:
ebuswhich provides all pub/sub and messaging basics in general.
ebus_handlerthat provides all needed functions to create and manage message handlers.
As we mentioned, one of the big differences and also strength, is the ability to have real event-driven consumer or message handlers, so let’s start creating our handler.
Also, if you don’t like to create a module to implement
ebus_hanlder behavior, ErlBus gives you the chance to pass an anonymous function to act as callback. In the example below you can see the mentioned ways to create and manage message handlers (anonymous funs or implementing
How was it? awesome isn’t? Well, let’s finish giving another interesting example.
Suppose now that you have a handler that takes a while processing each message/event, so it will be blocked until complete the task, and for some scenarios would be unthinkable. Therefore,
ebus_handler module gives you the option to create a pool of workers attached to your handler, and is totally transparent to you.
Once the pool is subscribed to a channel and some message arrives, it will be processed only by one worker of that pool; this is one way to have a Point-To-Point behavior.
ErlBus is distributed by nature, inherits all properties of Distributed Erlang and pg2. But
pg2 has some limitations, distribution model works with full replication, which can cause problem when we have a considerable amount of subscribers, and at the same time the amount of messages sent is too high. For these scenarios ErlBus provides another option:
ebus_dist, which is built on top of riak_core and gproc.
There are more nice features, so I strongly recommend you visit ErlBus 0.0.1 GitHub, you will find more documentation and examples.
Is extremely important let something clear, ErlBus is not a Message Queue System or MoM like RabbitMQ. In the MoM, channels are represented by queues and messages always cross the MoM. With ErlBus, channels are just a logical mechanism to allow communicate two or more endpoints (processes) each other, and messages are directly delivered from senders/publishers to receivers/subscribers; messages arrives to the buffer of each process. According to that, ErlBus and RabbitMQ drives different problems and applies to different scenarios.
ErlBus is just a simple and lightweight tool to build messaging-based applications, which provides an easy and usable interface on top of known and proven libs/tools like pg2, gproc and riak_core, enabling a clearer and more powerful semantics for messaging patterns such as: Publish/Subscribe, Point-To-Point, Event-Driven Consumers, Task Executors, etc.
The original idea was build something pretty simple, ErlBus embodies K.I.S.S principle. Because of this, things like Durable Subscriber, Store-And-Forward, etc., are not supported here, but the good thing is that you can go adding more modules and functions on top of ErlBus in order to provide missing things like mentioned before, depending on your needs –it’s a kind of backbone to build more complex systems, tools, etc.