AERON QUEUE: PRACTICAL IMPLEMENTATION GUIDE FOR LOW-LATENCY SYSTEMS
If you’ve been researching low-latency messaging systems, you’ve likely encountered Aeron as one of the top contenders. But setting up Aeron’s queue system isn’t always straightforward, and understanding when to use its various features can be confusing.
This guide walks you through implementing Aeron’s queue functionality with practical code examples. We’ll cover the core concepts, configuration options, and real-world patterns used in trading systems.
Understanding Aeron’s Architecture
Before diving into code, it’s worth understanding how Aeron differs from traditional message queues.
Aeron uses a pub/sub model with three core concepts:
- Publication - The sender side that pushes messages to a stream
- Subscription - The receiver side that listens for messages on a stream
- MediaDriver - The underlying transport layer handling UDP/IPC communication
Unlike Chronicle Queue which excels at single-machine shared memory, Aeron’s strength is networked communication with its reliable UDP protocol.
Setting Up Your First Aeron Queue
Let’s start with a minimal working example. You’ll need these dependencies:
<dependency>
<groupId>io.aeron</groupId>
<artifactId>aeron-all</artifactId>
<version>1.43.0</version>
</dependency>
Creating the MediaDriver
The MediaDriver is the backbone of Aeron’s communication:
public class AeronServer {
private final MediaDriver driver;
private final Aeron client;
private Publication publication;
private Subscription subscription;
public AeronServer() {
// Configure for low-latency
driver = MediaDriver.launch(
MediaDriver.Context()
.dirsDeleteOnStart(true)
.termBufferSparseBitSet(true)
.conductorBufferLength(1024)
.senderBufferLength(256)
.receiverBufferLength(256)
);
client = Aeron.connect();
}
}
Creating a Publication
Publications are how you send messages:
public void startPublisher(String channel, int streamId) {
publication = client.addPublication(channel, streamId);
// Wait for connection
while (!publication.isConnected()) {
Thread.yield();
}
}
public boolean sendMessage(String message) {
final byte[] data = message.getBytes(StandardCharsets.UTF_8);
final Buffer buffer = Buffer.allocate(data.length);
buffer.putBytes(data);
return publication.offer(buffer, 0, data.length) > 0;
}
Creating a Subscription
Subscriptions receive messages:
public void startSubscriber(String channel, int streamId, MessageHandler handler) {
subscription = client.addSubscription(channel, streamId,
(buffer, offset, length, header) -> {
byte[] data = new byte[length];
buffer.getBytes(offset, data);
handler.handle(new String(data, StandardCharsets.UTF_8));
}
);
}
Persistent Queue Configuration
For systems requiring message durability, Aeron’s archive subsystem provides persistence:
public class PersistentAeronQueue {
private final Archive archive;
private final Archive.WriterPosition writerPosition;
public void setupPersistentQueue(String archiveDirectory) {
// Configure archive for persistence
archive = Archive.launch(
Archive.Context()
.archiveDir(new File(archiveDirectory))
.segmentFileLength(128 * 1024 * 1024)
.maxArchiveFileLength(512 * 1024 * 1024)
);
// Create a recording
long recordingId = archive.startRecording(
"aeron:udp?endpoint=localhost:40123",
streamId
);
}
}
Performance Tuning for Trading Systems
Trading systems demand the lowest latency. Here are the key tuning parameters:
Buffer Sizes
MediaDriver.Context()
.termBufferLength(64 * 1024 * 1024) // Larger for higher throughput
.senderBufferLength(16 * 1024) // Match MTU for zero-copy
.receiverBufferLength(16 * 1024);
Network Optimization
// Disable Nagle's algorithm for lower latency
// Use kernel bypass with AERON_IPC_CHANNEL
// For local IPC (faster than UDP)
String ipcChannel = "aeron:ipc";
Memory Mapped Files
MediaDriver.Context()
.mtuLength(16384) // Match network MTU
.termBufferSparseBitSet(true) // Sparse allocation
.journalBufferLength(16 * 1024);
Common Patterns in Production
Request-Response Pattern
public class AeronRpc {
private final Map<String, Subscription> responseHandlers = new ConcurrentHashMap<>();
public String requestResponse(String channel, int requestStream,
int responseStream, String request) {
String correlationId = UUID.randomUUID().toString();
// Set up one-time response handler
Subscription responseSub = client.addSubscription(
channel, responseStream,
(buffer, offset, length, header) -> {
// Handle response
}
);
responseHandlers.put(correlationId, responseSub);
// Send request
publication.offer(correlationId + ":" + request);
// Wait for response with timeout
return waitForResponse(correlationId, 5000);
}
}
Back-Pressure Handling
public class BackPressureAwarePublisher {
public long offerWithBackPressure(Buffer buffer) {
long result;
do {
result = publication.offer(buffer);
if (result < 0) {
// Back-pressure: try again after small delay
LockSupport.parkNanos(1000); // 1 microsecond
}
} while (result < 0);
return result;
}
}
When to Use Aeron’s Queue
Aeron excels in these scenarios:
- Networked systems where messages cross machine boundaries
- Multi-site replication requiring geo-distributed messaging
- FIX protocol transport in trading platforms
- Microservices communication requiring low jitter
- Systems needing multi-language support (Java, C++, Go, Rust)
For single-machine ultra-low latency, consider Chronicle Queue / which uses shared memory for sub-microsecond IPC.
Common Pitfalls
- Not waiting for connection - Always verify
publication.isConnected()before sending - Buffer too small - Undersized buffers cause back-pressure and dropped messages
- Ignoring flow control - Without proper flow control, fast senders overwhelm slow receivers
- No heartbeats - Connections can appear alive but be silently dead; add application-level heartbeats
- Missing error handling - Register an ErrorHandler to catch and log issues
Conclusion
Aeron provides a powerful foundation for low-latency messaging, particularly in networked trading systems. The key is understanding when to use its persistent archive versus in-memory streaming, and tuning buffer sizes for your specific throughput requirements.
For most trading systems, the combination of Aeron for network distribution and Chronicle Queue for local IPC provides the best of both worlds - sub-microsecond local latency with robust network capabilities.