Ability to Downscale Google Provided Templates
We have a job which the basic templated DataFlow job works fairly well for, but so far we cannot see a way to have the machine use fewer machines. Our data ingestion is large and growing, but not yet extremely large. The 3 4 vcpu 15 gig ram machines that are started to process our volume of data are very overkill. I do not see any way to use these basic templates, but to also set the max_workers setting.
Hi, they added the maxNumWorkers parameter on provided templates. You can see it in the web console while trying to create new job from template.