Deploy Streaming Pipeline

In this chapter you will compile the Beam pipeline to a Jar file. The Beam pipeline is configured with the Apache Flink Runner, so that it can be executed with Kinesis Data Analytics in a managed Apache Flink environment. You will then create a Kinesis Data Analytics application and execute the Beam pipeline so that it populates an Amazon CloudWatch dashboard with the total number of taxi trips made in NYC in a certain time interval.

One the application is producing output, you change the output of the Beam pipeline to generate more fine grained metrics on the number of trips per borough.