If it’s not obvious, we love Elixir here at Gigalixir and one of the reasons is distributed clustering. Clustering allows you to connect your app servers together and makes it simple for them to communicate with one another. Here are 5 reasons we love Elixir clustering.
-
Pub/Sub
If you’ve ever needed to publish/subscribe all the things, you probably had to spin up Redis or use something like AWS SNS. With clustering, you can broadcast the message directly to the other app servers, eliminating the extra dependency. In fact, Phoenix Channels uses clustering by default.
-
Replaces Redis (in some cases)
If your app keeps data in Redis or a database, the reason is probably either because it’s shared data that all the app servers need access to or it’s persistent data that needs to survive app crashes and restarts. In the case of shared data that doesn’t need to be persistent, clustering makes it simple to keep this data in-memory directly on your app servers accessible by all the other servers, eliminating Redis.
-
Eliminates Cron
If you need to run a job once and only once every night, you might have had a single server somewhere running cron to ensure multiple copies of the job weren’t spawned at once. Or if you were on Heroku, you might have used the Heroku Scheduler which spins up a new dyno for every job you want to run, costing you extra money. With clustering and libraries like Quantum, you can run the jobs directly in your app server using app code.
Aside from the cost savings, it’s nice to keep all your code together in one place. Development is easier without code separated across crontabs, scheduler configuration, and bash scripts. And I always forget where the cron jobs send their logs since it’s usually not the same place as your app logs.
-
Smooths Out Load
If your app has slow requests, some servers might become over-utilized and backlogged while other servers are sitting idle.
With clustering, you can break each request into smaller chunks to distribute to the rest of the machines. This should smooth the workload out.
You can even run the chunks in parallel. You could have done something like this with a work queue like Resque, but that adds overhead, complexity, and cost.
-
Avoids Sticky Sessions
If you need requests from a client to always hit the same server, you might be tempted to use sticky sessions. For example, if you are writing a game, you might want to keep the state of the game in-memory on the server. Each time the player makes a move, you’ll want them to hit the same server each time. With clustering, you can let the request go to any of your app servers and the app server can redirect the request to the right server. It’s better to keep as much code as you can in the app. Configuring your load balancer to have knowledge of player ids leaks app-specific data to your network layer.
In conclusion, when you use Elixir clustering, you save money and complexity as well as improve performance. You can drop Redis in some cases, avoid dedicated cron machine(s), smooth out load, and keep more state in-memory. That’s not even mentioning boosts in developer productivity by keeping more of your code in the app.
If you want to give clustering a try, consider Gigalixir, which makes clustering not only possible but easy to set up.