Stream analytics solution

Integrated and open stream analytics

Stream analytics has emerged as a simpler, faster alternative to batch ETL for getting maximum value from user-interaction events and application and machine logs. Ingesting, processing, and analyzing these data streams quickly and efficiently is critical in fraud detection, clickstream analysis, and online recommendations, among many examples. For such use cases, Google Cloud offers an integrated and open stream analytics solution that is easy to adopt, scale, and manage.

Respond to events as they happen

Ingest millions of streaming events per second from anywhere in the world with Cloud Pub/Sub, powered by Google's unique, high-speed private network. Process the streams with Cloud Dataflow to ensure reliable, exactly-once, low-latency data transformation. Stream the transformed data into BigQuery, the cloud-native data warehousing service, for immediate analysis via SQL or popular visualization tools. Finally, bring predictive analytics to fraud detection, real-time personalization and similar use cases by integrating TensorFlow-based Cloud Machine Learning models and APIs into your streaming data pipelines.

Accelerate development, with no compromises

Stream analytics on GCP simplifies ETL pipelines without compromising robustness, accuracy, or functionality. Cloud Dataflow supports fast pipeline development via expressive Java and Python APIs in the Apache Beam SDK, which provides a rich set of windowing and session analysis primitives as well as an ecosystem of source and sink connectors. Plus, Beam’s unique, unified development model lets you reuse more code across streaming and batch pipelines.

Keep your favorite tools and systems

Stream analytics on GCP is open and interoperable by design. Cloud Pub/Sub’s open API and multiple clients enable multi-cloud and hybrid deployments. For Apache Kafka users, Confluent is Google’s recommended way to run managed Kafka, and a Cloud Dataflow connector makes do-it-yourself integration with GCP easy. BigQuery works seamlessly with the ETL and BI tools you know and love via standard SQL. Data processing pipelines written with the Beam-based Cloud Dataflow 2.x SDK are portable across Cloud Dataflow, Apache Spark, and Apache Flink. Finally, Spark support is available via Cloud Dataproc for streaming and batch workloads.