12.2. STOMP
This text elaborates the previous chapter A Transport Layer for the PDR. In it we describe in some detail the technical design decisions underlying the implementation of the message layer.
12.2.1. Technology chosen
The implementation is built on RabbitMQ (https://www.rabbitmq.com/documentation.html) and Stompjs (http://jmesnil.net/stomp-websocket/doc/). The latter library caters for Stomp version 1.0 and 1.1, not 1.2. The Stomp web plugin for RabbitMQ handles all versions.
12.2.2. Exchange type
While peers may use the Perspective User Identity (PUI) to send Transactions to, only the intended receiver must be able to subscribe to the relevant queue on the AMQP server. To achieve this end, we use a Topic Exchange where
-
the routing keys are PUIs;
-
the binding keys are PUIs, but only the PDR for a particular PUI knows the identification of the queue that binds that PUI.
Note that not even the server administrator has to know the queue that uses a particular PUI as binding key. The subscribing PDR can keep it to itself.
A Topic Exchange matches routing keys to binding keys, where the latter may include wildcards. We have the simplest possible situation, where both keys are equal. It is the binding rule that connects the key to a particular queue, that protects the receiver from others marauding his post box!
12.2.3. Creating topic queues
The web client using Stomp can create a queue with a particular binding key in a Topic Exchange; we don’t need the RabbitMQ administrator for that.
It turns out that when a new vhost is created using the management console of RabbitMQ, all types of Exchanges are created for it. Stomp sends a frame with a destination string that starts with “/topic” automatically to the amq.topic Exchange of that vhost.
A queue with a particular binding key and queue identification can be created from the client as follows:
const \{id, unsubscribe} = client.subscribe(
"/topic/" + topic,
function(message)\{…}, // handle the message
\{ durable: true
, "auto-delete": false
, id: "secred-id" // the secret queue identification.
});
Notice the fields in the object that is provided as last argument to subscribe. They specify that, apart from the queue identification, the queue is not to be deleted when no one subscribes to it and will be available after the server restarts, too.
This behaviour is governed partially by the semantics attributed to the destination string that, by default, Stomp assigns neither structure nor semantics to. For RabbitMQ this is described in https://www.rabbitmq.com/stomp.html.
12.2.4. Acknowledgements
We don’t want Transactions to get lost. To prevent the RabbitMQ server from deleting a Transaction before it has been handled by the receiver, we make the receiver send explicit acknowledgements.
By default the server removes a message after it has been delivered. To change that behaviour we give the object supplied as third argument to subscribe with another key:
ack: “client”
Now the subscribing client has to acknowledge the message, using the function that is the value of the field ack on the message that is received.
12.2.5. Heartbeat
By default, the Stomp server sets up a heartbeat (RabbitMQ by default sends a beat every 10.000 milliseconds). However, as we have the client send explicit acknowledgement, it seems not necessary to have a heartbeat on the socket level. This is how to disable it:
const client = Stomp.client(url);
client.heartbeat = \{incoming: 0, outgoing: 0};
12.2.6. User accounts
The RabbitMQ manager must create user accounts; there is no self-registration.
12.2.7. Multiple RabbitMQ services
In principle, a single RabbitMQ service provider could cater for the entire Perspectives Universe. However, that certainly is not in the spirit of distribution and would mean a big lock-in with a single provider. Instead, we would like it to be possible for many providers to provide the messaging infrastructure in a distributed way.
It turns out that RabbitMQ has at least two technologies to be used to that purpose: federation and shovel. In this section I focus on shovel, as it seems the most apt to our situation. Shovel can be used to move messages from one server (or cluster) to another (note: this is different from load balancing in a cluster of RabbitMQ nodes). Let’s assume there are two clusters, C1 and C2. What we want is that users of C1 can use their account to send messages to other users who have registered with C2. Were that not possible, a user would need an account with each cluster that caters for one of his peers. This would explode the number of RabbitMQ accounts that a single Perspectives user needs.
In cluster C1, we need to set up a queue for cluster C2: let’s call it C2queue_in_C1. We configure shovel such that any message sent to C2queue_in_C1 is forwarded to the amq.topic exchange in C2. Now lets delve into addressing messages.
The PDR publishes messages using the receivers Perspectives identification as topic (and that matches with a binding key on the receiver’s secret queue). However, both topics and binding keys may consist of a series of words, separated by dots. In a binding key, we can use wildcards. We’ll give C2queue_in_C1 the binding key C2.*. Thus, any message whose topic starts with C2 will end up in C2queue_in_C1 and consequently will be forwarded to amq.topic of C2.
We will configure a users' queue with a binding key that ignores the first (cluster identification) part: *.<userId>. As cluster C2 will not have a queue with a binding key that matches its own identification, only the intended receivers' binding key matches the topic of the message arriving from cluster C1 and so the message will end up in the queue of the receiver on C2.
Notice that this approach requires a directed shovel link between any two clusters that have subscribers who are Perspectives peers. The number of links thus is quadratic, which does not seem to be prohibitive.