Chronicle Queue and Aeron
This article compares Chronicle Queue and Aeron, two high-performance messaging systems.
Chronicle Queue:
Chronicle Queue is a persisted, low-latency messaging and persistence engine designed for high-throughput applications. Its key features include:
- Microsecond Latency: Chronicle Queue excels in ultra-low latency scenarios, often achieving sub-microsecond performance within a single machine or cluster. This makes it suitable for even the most demanding high-frequency trading (HFT) systems.
- Persistence: All messages are persisted to disk, enabling durable message storage and replay from any point in time. This supports “record everything” functionality, valuable for debugging, auditing, and replaying market events.
- Scalability: Chronicle Queue can handle terabyte-sized queues and millions of messages per second. It’s designed for high-volume data ingestion and processing.
- Replication: Chronicle Queue Enterprise offers reliable replication with notifications to writers or readers upon successful replication. This enhances data durability and availability.
- Multi-Language Support: Chronicle Queue supports Java, C++, Rust, and Python, offering flexibility and interoperability across different technology stacks.
- Compression: Chronicle Queue Enterprise supports real-time compression, reducing message size and storage requirements.
- No Flow Control: Chronicle Queue’s “producer-centric” design avoids flow control, ensuring producers are never slowed down by consumers. This is crucial for scenarios like market data processing and compliance systems where continuous, high-speed data ingestion is essential.
- Use Cases: Chronicle Queue is ideal for a wide range of applications, including:
- High-Frequency Trading (HFT)
- Real-time Analytics
- Log Replacement
- Market Data Gateways
- Compliance Systems
- Risk Management
- Microservices
- Big Data Solutions
Companies Using Chronicle Queue:
- Abbey Cross: Uses Chronicle Queue for reliable and high-performance market data distribution. (https://chronicle.software/abbeycross-case-study/)
- Vantage Markets: Employs Chronicle Queue to enhance the speed and efficiency of their online trading platform. (https://chronicle.software/vantage-markets-case-study/)
- Unnamed US Investment Bank: Leverages Chronicle Queue for HFT, capturing and analyzing market data with extremely low latency. (https://chronicle.software/us-investment-bank-case-study/)
- StoneX: Uses Chronicle Queue to improve the performance and scalability of their risk management systems. (https://chronicle.software/stonex-case-study/)
- Boost High-Frequency Trading Performance on Amazon EC2 with Chronicle Tune: This case study demonstrates how Chronicle Queue can be optimized for high-frequency trading on Amazon EC2 instances. (https://aws.amazon.com/blogs/apn/boost-high-frequency-trading-performance-on-amazon-ec2-with-chronicle-tune/)
Aeron:
Aeron is designed for high-performance messaging between machines over a network. Its core features include:
- Low Latency: Aeron achieves low millisecond latency for network communication, making it suitable for distributed systems.
- High Throughput: Aeron supports high message throughput, often exceeding hundreds of millions of messages per second.
- Reliable UDP Transport: Aeron uses a reliable UDP transport protocol for efficient and reliable message delivery.
- Persistence: Aeron supports message persistence for data durability and recovery.
- Use Cases: Aeron is well-suited for applications requiring high-performance inter-machine communication, such as distributed systems and microservices.
Companies Using Aeron:
- Man Group: Employs Aeron in their Special FX execution system for foreign exchange trading. (https://www.man.com/special-fx-execution-system-on-aeron)
- CME Group: Leverages Aeron, likely for market data distribution and trading infrastructure.
- Bullish: Uses Aeron in their cryptocurrency exchange platform.
- Kepler Cheuvreux: Building a low-latency, cloud-enabled trade execution platform.
- SIX Interbank Clearing: Uses Aeron Cluster for Swiss Instant Payments.
Comparison Table:
| Feature | Chronicle Queue | Aeron |
|---|---|---|
| Latency | Very low (<1 microsecond) | Low (<1 millisecond) |
| Throughput | High (up to 100 million messages per second) | Very high (up to 1 billion messages per second) |
| Persistence | Yes | Yes |
| Transport protocol | Shared memory | Reliable UDP |
| Use cases | Low latency messaging within a single machine or cluster | High-performance messaging between machines over a network |
Choosing Between Chronicle Queue and Aeron:
Select Chronicle Queue when ultra-low latency within a single machine or cluster is paramount. Choose Aeron when high-performance messaging between machines over a network is the primary requirement. Consider factors like throughput needs, persistence requirements, and multi-language support when making your decision. Always conduct thorough performance testing with realistic workloads to determine the best solution for your specific application.