Execute the batch pipeline

  1. Once the update is completed and the application is running again, inspect the updated job graph. You can see how the structure of the application graph changed, as the Beam pipeline has generated a different job graph for the batch application.

    KDA batch job graph

  2. Navigate to the Flink dashboard by clicking on Open Apache Flink dashboard at the top of the page

  3. Select the only job from the Runing Job List to track the progress of the execution

    KDA batch job graph

  4. Once the pipeline has completed, you can inspect the metrics for the boroughs that have been generated in Amazon CloudWatch. The graph shows count of taxi trips that are backfilled.