Replay historic taxi trips

We will use a Java application to replay a historic set of taxi trips made in NYC that are stored in objects in Amazon S3 into the Kinesis stream. The Java application has already been compiled and loaded onto the EC2 instance. If you are interested in the details of the application, you can obtain the sources, including a CloudFormation template to build the Java application with Amazon CodePipeline, from GitHub.

  1. Click on the Terminal icon in the bottom of Intellij to open a shell

  2. Enter the following command into the terminal prompt to start producing events into the Kinesis data stream created earlier, beam-workshop.

    java -jar C:\Users\Administrator\Desktop\workshop-resources\amazon-kinesis-replay-0.1.0.jar -streamName beam-workshop -objectPrefix artifacts/kinesis-analytics-taxi-consumer/taxi-trips-partitioned.json.lz4/dropoff_year=2018/ -speedup 720

    You should see lines similar to the one below in the terminal if the above operation was successful. Please keep the ingestion running while you continue with the next step.

    INFO   StreamPopulator - all events with dropoff time until 2018-01-01T04:36:21Z have been sent (3442.0 events/sec, PT-1S replay lag)