A comparison of popular message queues
A comparison of popular message queues: SQS, RabbitMQ, ActiveMQ, IronMQ, and RedisMessage queues provide a means of fast asynchronous communication between software components. They come handy in solving some common problems that arise while building software, e.g.
- loosely couple components by delegating message delivery and retry logic to the message queues.
- split long running tasks into multiple sub tasks (without having to use threading or concurrency libraries).
- "hold" messages when destination is unavailable or overwhelmed.
- provide asynchronous updates of long running transactions.
- scheduling messages for future delivery.
At Better.com, we tried Beanstalk and Amazon SWF and realized that they weren't a fit for our stack. For example, Beanstalk doesn't have message persistence or authentication. SWF doesn't scale well at high frequencies and it is built for complex asynchronous multi-step workflows. So it isn't the right product for high frequency simple async communications.
Message queue vs Message broker
A Message queue simply enqueues and dequeues messages
A Message broker can perform additional operations on messages before dequeueing them. You can find more information regarding these transforms here
Comparison
Our criteria for a message broker were thus:
Must have: Delivery retry, CI/CD friendly, Fast, High Availability Nice to have: Admin console, PubSub
Here are the contenders
- ActiveMQ: Open source Apache project written in Java. 15+ years old and very mature.
- Amazon SQS: AWS product for message queueing.
- RabbitMQ: Open source message broker written in Erlang.
- IronMQ: Closed source message broker.
- Redis: In memory cache that can be used as a queue using redis operations.
Core features
SQS | RabbitMQ | ActiveMQ | IronMQ | Redis | |
---|---|---|---|---|---|
Message Broker | ✓ | ✓ | ✓ | ||
PubSub | ✓ | ✓ | ✓ | ✓ | |
Queues | ✓ | ✓ | ✓ | ✓ | ✓ |
FIFO Queues | ✓ | ✓ | ✓ | ✓ | ✓ |
Message retrieval | pull | push | push | push and pull | push |
Redelivery | ✓ | ✓ | |||
Supported protocols | http | multiple | multiple | https | custom(RESP) |
Max message size | 256KB | 2GB | >1GB | 256KB | 512MB |
Management features
SQS | Rabbit | ActiveMQ | IronMQ | Redis | |
---|---|---|---|---|---|
Dockerized | ✓ | ✓ | ✓ | ✓ | |
Admin UI | ✓ | ✓ | ✓ | ✓ | |
Auto scaling | ✓ | ✓ | |||
Scaling mechanism | Managed by AWS | Clustering | Resize | Managed | Clustering |
High availabilty mechanism | Managed by AWS | Failover | Failover | Clustering |
SQS
Pros:
- Easy setup and API.
- Outstanding client libraries for popular language.
- Managed
- "Infinite" scalability
Cons:
- Consumers pull messages, but long polling is available on the clients.
- Max batch size of 10 messages per consumer
- No pub/sub. You'd have to use SNS for pub/sub.
RabbitMQ
Pros:
- Self hosted
- Cheap and fast
- Outstanding client libraries for popular language.
- Supports multiple protocols (amqp, mqtt, stomp)
Cons:
- AMQP is complex
- RabbitMQ is written in erlang. The configurations are in erlang.
ActiveMQ
Pros:
- Extensive broker capabilities link
- Free
- Fast
- Supports multiple protocols (amqp, openwire, mqtt, jmx, stomp)
- Java client support and feature set is impeccable.
Cons:
- Client library support for non java languages is mediocre
- High availability is achieved via a failover mechanism, which is not supported out of the box by non-jms libraries.
- Requires reboots for configuration changes.
Redis
Pros:
- Free
- Fast
- Outstanding client libraries for popular languages
Cons:
- Advanced message queue features like atleast-once delivery need to be implemented in the clients.
- 3rd party Admin UIs need to be used.
Winner
None of the queue implementations check all the boxes. We went with ActiveMQ since it was the best fit for our stack and traffic volume. in production, ActiveMQ on AWS was very easy to setup. The ability to spin up ActiveMQ instances in Docker ties in neatly into our CI/CD pipelines and lets us run proper integration tests without the need for mocking. It was significant work to implement and test client libraries in Javascript and Python, but this was a good tradeoff for the ease of use and speed provided by ActiveMQ.
Final Impressions
We've been running ActiveMQ for over a year in production. It's proven to be lightweight, fast and requires low maintenence. On the other hand, writing client libraries in Javascript and Python has not been as easy as we thought. It required a lot of effort to write these libraries and get them production ready. If this is not something you are up for, RabbitMQ might be a very good alternative.
- ActiveMQ
- Redis

Our thinking
Engineering a Diverse Team: Taffy Chen and Jimmy Farillo
Software engineers Taffy Chen and Jimmy Farillo launch a new blog series to showcase different perspectives on the Better engineering team, and the ways they’re working to make it even more diverse and inclusive.Fri May 14 2021—by Taffy Chen and Jimmy Farillo2 min read- diversity
- inclusion
- perspectives
Are iBuyers manipulating the real estate market?
A preliminary analysis with Better's transaction datasetThu Oct 21 2021—by Yasha Sheynin6 min readUsing ImageMagick for PDF redaction
Automating document redaction with a classic command line toolThu Aug 06 2020—by Robert Brownstein3 min read