Member-only story
Google introduced Workflows for BigQuery
How to execute Code in Sequence at a Scheduled Time
Google announced that you can now create workflows to execute code assets in sequence at a scheduled time — for now this feature is in Preview[1].
This new functionality is based on Data Form — a feature that was launched in 2023. Dataform offers an end-to-end experience to develop, version control, and deploy SQL pipelines in BigQuery for Data Engineers and Data Analysts of all skill levels. You can build production-grade SQL pipelines in BigQuery while following software engineering best practices such as version control with Git, CI/CD, and code lifecycle management[2].
Dataform is a meaningful addition to BigQuery and provides single unified UI and API with which to build, version control, and operationalize scalable SQL pipelines[1][3] according to Google. Data Engineers and other users will develop new tables faster, ensure data quality, and operationalize their pipelines with minimal effort, making data more accessible across their organization[3].
