CHRONICLE QUEUE TUTORIAL: GETTING STARTED WITH ULTRA-LOW LATENCY MESSAGING
If you’re building a trading system, analytics platform, or any application where microseconds matter, you’ve likely heard of Chronicle Queue. But getting started can feel overwhelming — the documentation assumes you already understand the architecture.
This tutorial walks you through your first Chronicle Queue implementation. By the end, you’ll understand the core concepts, have working code, and know when to use Chronicle versus alternatives like Aeron or Kafka.
Why Chronicle Queue?
Before diving in, let’s understand why Chronicle Queue exists:
- Sub-microsecond latency using memory-mapped files
- Disk-backed persistence without the latency penalty
- Single-file storage for easy backup and replay
- No network overhead for local IPC
- Multiple language support: Java, C++, Rust, Python
Setting Up Chronicle Queue
First, add the dependency:
<dependency>
<groupId>net.openhft</groupId>
<artifactId>chronicle-queue</artifactId>
<version>5.24.0</version>
</dependency>
For Gradle:
implementation 'net.openhft:chronicle-queue:5.24.0'
Your First Chronicle Queue
Let’s create a simple producer-consumer example:
Creating a Queue
import net.openhft.chronicle.queue.ChronicleQueue;
import net.openhft.chronicle.queue.ExcerptAppender;
import net.openhft.chronicle.queue.ExcerptReader;
public class SimpleQueueExample {
public static void main(String[] args) {
// Create a queue stored on disk
ChronicleQueue queue = ChronicleQueue.builder()
.path(Path.of("/tmp/my-queue"))
.build();
// Get an appender (producer)
ExcerptAppender appender = queue.acquireAppender();
// Write a message
appender.writeDocument(w -> w
.writeEventName("message")
.writeText("Hello, Chronicle!")
);
// Read the message
try (ExcerptReader reader = queue.createReader()) {
reader.readDocument(r -> {
String message = r.readEventName();
System.out.println("Received: " + message);
});
}
queue.close();
}
}
Understanding the Excerpt Model
Chronicle Queue uses “excerpts” — think of them as self-contained records:
- ExcerptAppender: Writes data to the queue
- ExcerptReader: Reads data from the queue
- ExcerptTailer: Reads from the end (for streaming)
Writing Multiple Messages
public class MarketDataProducer {
private final ExcerptAppender appender;
public MarketDataProducer(ChronicleQueue queue) {
this.appender = queue.acquireAppender();
}
public void sendTick(String symbol, double price, long timestamp) {
appender.writeDocument(w -> {
w.writeEventName("tick")
.writeString("symbol", symbol)
.writeFloat("price", (float) price)
.writeLong("timestamp", timestamp);
});
}
}
Reading Messages
public class MarketDataConsumer {
private final ExcerptReader reader;
public MarketDataConsumer(ChronicleQueue queue) {
this.reader = queue.createReader();
}
public void processTicks() {
while (reader.readDocument(r -> {
String eventName = r.readEventName();
String symbol = r.readString("symbol");
float price = r.readFloat("price");
long timestamp = r.readLong("timestamp");
System.out.printf("Tick: %s @ %.2f%n", symbol, price);
})) {
// Continue reading
}
}
}
Persistent Queues
For systems requiring durability, configure Chronicle Queue for persistence:
ChronicleQueue queue = ChronicleQueue.builder()
.path(Path.of("/data/persistent-queue"))
.wireType(WireType.BINARY)
.build();
// Messages are persisted to disk immediately
// Recovery is automatic on restart
Configuring Roll Behavior
ChronicleQueue queue = ChronicleQueue.builder()
.path(Path.of("/data/roll-test"))
.rollCycle(RollCycle.DAILY) // New file each day
.maxFileSize(256 * 1024 * 1024) // Or when file hits 256MB
.build();
Performance Tuning
Chronicle Queue is fast by default, but here are optimizations for trading systems:
Use PREFER_HEAP
ChronicleQueue queue = ChronicleQueue.builder()
.path(Path.of("/data/queue"))
.bufferCapacity(1024) // Increase for throughput
.build();
Enable Rollbacks
ExcerptAppender appender = queue.acquireAppender();
appender.writeDocument(w -> {
w.writeEventName("trade")
.writeLong("id", tradeId)
.writeText("details", tradeDetails);
// If something goes wrong, rollback this entry
// appender.rollback();
});
Common Patterns
Request-Response
public class RequestResponseQueue {
private final ChronicleQueue requestQueue;
private final ChronicleQueue responseQueue;
public void request(String requestId, String payload) {
requestQueue.acquireAppender()
.writeDocument(w -> w
.writeString("id", requestId)
.writeString("payload", payload));
}
public String waitForResponse(String requestId, long timeoutMs) {
long deadline = System.currentTimeMillis() + timeoutMs;
try (ExcerptReader reader = responseQueue.createReader()) {
while (System.currentTimeMillis() < deadline) {
final String[] result = new String[1];
if (reader.readDocument(r -> {
String id = r.readString("id");
if (id.equals(requestId)) {
result[0] = r.readString("response");
}
}) && result[0] != null) {
return result[0];
}
Thread.sleep(1);
}
}
throw new TimeoutException("No response within " + timeoutMs);
}
}
Tailer for Streaming
public class StreamProcessor {
private final ExcerptTailer tailer;
public StreamProcessor(ChronicleQueue queue) {
this.tailer = queue.createTailer();
}
public void process() {
// Only reads NEW messages (like a stream consumer)
tailer.readDocument(r -> {
String event = r.readEventName();
handleEvent(event, r);
});
}
}
When to Use Chronicle Queue
Use Chronicle Queue when:
- Single-machine latency matters more than distributed features
- You need persistence without sacrificing speed
- Your use case is within a data center (not cross-geo)
- You’re building HFT, trading platforms, or real-time analytics
Consider alternatives when:
- You need cross-datacenter replication (use Kafka, Redpanda)
- You need massive horizontal scaling (use Kafka, Pulsar)
- You prefer network-based communication over shared memory (use Aeron)
Next Steps
Now that you understand the basics:
- Experiment with the code — Run the examples above
- Explore Chronicle Queue Enterprise — For additional features like replication
- Read the performance benchmarks — Compare with Aeron and Kafka
- Build a real system — Try Chronicle Queue in your trading platform
For a deeper comparison with Aeron, see my article on Chronicle vs Aeron /.