How to write TrafficMaxLaneFlow pipeline and deploy it on the google dataflow platform.
In this workshop we will write together a pipeline that will reads and analyses data from traffic sensors.
The TrafficMaxLaneFlow pipeline analyses data from traffic sensors. This pipeline can run in both batch and streaming modes. In batch mode, the pipeline reads traffic sensor data from an input file. In streaming mode, the data constantly flows in from a Cloud Pub/Sub topic.
TrafficMaxLaneFlow analyzes the incoming data stream using Windowing, specifically Sliding Time Windows. Sliding time Windows use time intervals in the data stream to define bundles of data with windows that overlap.
TrafficMaxLaneFlow uses a custom Combine transform to extract lane information and calculate the max lane flow found for a given station for each Window. A custom combine transform is necessary because the combination is not a simple Max combination it needs to retain some additional information along with the flow value.
The pipeline formats and writes the max values along with the auxiliary information to a BigQuery table.
We will contact you as soon as possible.