dataprep / dataflow jobs region
TLDR: dataprep creates jobs in the us only. This is probably a bug.
I'm trying to prep data from BQ - EU to BQ -
But dataprep creates dataflow in the us, and because of that I get an error:
Cannot read and write in different locations: source: EU, destination: US, error: Cannot read and write in different locations: source: EU, destination: US
Pat Baker commented
Changing the gs:// locations to eu works but I'm guessing that the job running in the us will be slower and include extra charges for the cross region data transfer.
Can't find where to submit dataflow job to eu, any suggestions welcome
Ole Dallerup commented
I had the same problem. If you go to https://clouddataprep.com/profile and change the directories to some that are located in EU it should work. At least it did for me.