dataprep/Causing a job to fail if there is an error in the recipe
TL;DR - I want to be able to get notified in case a recipe is not valid due to errors in the data source/recipe that can happen from a change in the source tables or any other reason, and see that the transformation has failed
Example Scenario - I have a dataset (e.g. a salesforce table) that is being extracted on a daily basis and saved in GCS. A scheduled dataprep job is being invoked every day on the new table and transforms it with a pre-defined recipe. When a column is added/removed from the data source, the recipe might needs to be updated as well, as it now might contains errors (e.g. trying to transform missing columns). Today if there is an error in the recipe it is being ignored and the job ends up successfully, resulting in corrupted data and hurts the integrity of the entire process.
Alternatively, another solution to this problem might be enabling the user to fail the job by adding a function to the recipe that explicitly fails the job on a given condition (e.g. a column is all nulls)