Share
AI Amplified Insights
RDBMS to BQ streaming using Debezium
•
This podcast based on a Medium article by Ravish Garg that describes how to use the Debezium Server to stream data changes from a SQL Server database directly to Google Cloud Pub/Sub. The article provides a step-by-step guide on configuring both the Debezium Server and the Pub/Sub service, including details on how to enable change data capture (CDC) on the SQL Server database. It also highlights the advantages of using Debezium Server, such as eliminating the need for Apache Kafka and reducing operational overhead. The author notes that Debezium Server is still in incubation, meaning some aspects of it might change in the future.
More episodes
View all episodes
BigQuery Pipe Syntax: Streamlining Your SQL
09:40|This is an introduction to BigQuery’s Pipe Syntax, a new way of writing SQL queries that streamlines the process by breaking it down into sequential operations. It contrasts traditional SQL, which often involves nested subqueries and complex clauses, with Pipe Syntax, which allows for simpler, more readable queries by using the pipe { | } operator to chain operations.
Next-Gen Data Lakehouses with BigQuery and Iceberg
08:41|BigQuery Iceberg Tables are an open-format data lakehouse solution on Google Cloud. These tables combine the flexibility of Cloud Storage with BigQuery's managed analytics, allowing users to work with data in Parquet format using the open-source Apache Iceberg table format. Key features of BigQuery Iceberg Tables include schema evolution, unified batch and streaming data handling, automatic storage optimization, and enhanced security features. The article explains how to create and use Iceberg tables in BigQuery, including data ingestion methods and a real-world case study of an e-commerce company using BigQuery Iceberg Tables to improve their data management and analytics capabilities.