Pub/Sub has launched several new features and enhancements to help meet the demands of modern streaming workloads, across three key data analytics patterns - Streaming ingestion, Streaming analytics, and Stream sharing and export, to help leverage real-time data for actionable insights and improved decision-making.
This year, we brought simplification at every step of the journey to ingest streaming data directly into Pub/Sub from various sources, including AWS Kinesis Data Streams.
Another typical streaming ingestion use case is to ingest batch data into Pub/Sub. Now you can create a subscription to get the data to your choice of sink for downstream processing.
Customers use Pub/Sub in conjunction with stream processing engines to power streaming-analytics use cases such as anomaly detection, personalization, etc.
To simplify streaming real-time data from BigQuery to external systems and vendors, you can use BigQuery continuous queries with Pub/Sub.
New support for OpenTelemetry in Pub/Sub provides a detailed trace of your message lifecycle, including the ability to see a distributed trace.
We will be launching cross-cloud Kafka sources with Import Topics in early 2025 and plan to further simplify streaming analytics architectures by providing native, lightweight, single-message transformations.
Get started with Pub/Sub today and start exploring these new features to solve your hardest business challenges.