Posts

Multiple Geographic Regions with single Dataform repo

tldr: Use seperate git branches to set your workflow_settings.yaml with the regions you want to deploy to and then setup release configurations using each different branches. Finally anytime you want to execute a pipeline in a particular region use the appropriate release configuration.

Do you have data requirements where you need to run the same pipelines in particular regions? Naively, we could copy the same code into region-specific repositories, or maintain separate configuration files for each region in the same repository. Ultimately, when we need to make a change to the pipeline, we run into the issue of needing to update the same code in multiple places, increasing the risk of making mistakes along the way.