Kafka vs Google Cloud Pub/Sub: 2025 Comparison
| Capability | Confluent Cloud (Kafka) | Google Cloud Pub/Sub |
|---|---|---|
| Retention | Configurable per topic (hours → infinite via log compaction). | Default 7 days; up to 31 days for subscription retention after configuration. Export to Cloud Storage for long-term replay. |
| Replay | Offset-based; consumers can seek to any committed offset within retention. | Seek/Replay using snapshots or timestamps within retention window; Cloud Storage + Dataflow for archival replay. |
| Ordering | Partition ordering guaranteed; use multiple partitions for parallelism. | Ordering keys guarantee per-key ordering; without keys ordering is best-effort. |
| Delivery Semantics | At-least-once; exactly-once semantics available for transactions and idempotent producers. | At-least-once by default; exactly-once achievable with Dataflow or idempotent subscribers. |
| Latency | Low-latency (sub-50 ms) when clients reside near the cluster; depends on partition placement. | Typically 60–100 ms median within a region; no latency SLA but scales automatically. |
| Availability SLA | 99.95% per region. Multi-region clusters available (see pricing). | 99.95% per region; regional or multi-region topics available. |
| Schema Support | Managed Schema Registry (Avro/JSON/Protobuf). | Pub/Sub topic schemas (Avro, Protobuf, JSON) with validation. |
| Security & IAM | API keys, role-based access, service accounts; integrate with AWS/GCP/Azure networking (PrivateLink, VPC Peering). | Native Google IAM, CMEK encryption, VPC Service Controls (push/pull except legacy push). |
| Stream Processing | Kafka Streams, ksqlDB, Connect ecosystem. | Cloud Dataflow (Apache Beam), Data Fusion, or 3rd-party stream processors. |
| Pricing (illustrative) | Charged by throughput (GB ingested/egressed), partitions, and storage. | Pay per request plus data volume; sustained use discounts for commitment tiers. |
Picking the Right Service
Choose Confluent Cloud when you need Kafka protocol compatibility, fine-grained control over partitions/retention, or plan to reuse Kafka Streams/Connect tooling across clouds.
Choose Google Cloud Pub/Sub when you prefer a fully managed service tightly integrated with GCP IAM, Cloud Logging, and Dataflow, or when infrastructure teams want global topic replication without managing brokers.
Migration Tips
- Map producers/consumers to equivalent client SDKs; Kafka clients cannot talk to Pub/Sub without a connector layer.
- For long-term replay on Pub/Sub, schedule Dataflow jobs to archive topics to Cloud Storage and reload via seek when needed.
- Evaluate egress costs: cross-region consumers on Confluent or Pub/Sub can trigger additional charges.