Log GCE worker setup process and exception
We had trouble to do even simple word count job with the DataflowRunner.
The job was stuck for 1 hour, but log said nothing.
INFO:root:2017-08-28T07:26:07.709Z: JOBMESSAGEBASIC: (5296cce74062ca91): Starting 1 workers in asia-northeast1-c...
INFO:root:2017-08-28T07:26:07.731Z: JOBMESSAGEDEBUG: (f4c12c649707e205): Value "group/Session" materialized.
INFO:root:2017-08-28T07:26:07.745Z: JOBMESSAGEBASIC: (f4c12c649707e994): Executing operation read/Read+split+pairwithone+group/Reify+group/Write
**** Stuck in 1 hour, so cancelled ****
The reason was we need to set [network] and [subnetwork] option explicitly and correctly.
After that, the job was going to work.
Its so good to know what is happening, or what is stuck the job when setting up the workers. (quota? wrong networks? etc..)