ETExamTower
Q13Data Ingestion and Transformation

A manufacturing company collects sensor data from its factory floor to monitor and enhance operational efficiency. The company uses Amazon Kinesis Data Streams to publish the data that the sensors collect to a data stream. Then Amazon Kinesis Data Firehose writes the data to an Amazon S3 bucket. The company needs to display a real-time view of operational efficiency on a large screen in the manufacturing facility. Which solution will meet these requirements with the LOWEST latency?

← → navigate · a answer
Community votes
A
90% (9)
C
10% (1)
B
0% (0)
D
0% (0)
Discussion · 19
A 14
Selected Answer: A https://aws.amazon.com/blogs/database/near-real-time-processing-with-amazon-kinesis-amazon-timestream-and-grafana/ Check the architecture diagram
A 9
Selected Answer: A real time -> no Quicksight. And bookmarks to read sensor data real time is just as dumb as the flat earth theory. A it is.
A 8
Selected Answer: A Firehose cannot use Timestream as a destination. Answer is A
A 5
Selected Answer: A Grafana: Real-time Performance: Grafana is well known for its excellent real-time data visualization capabilities. It's often used for operational dashboards that need frequent updates. Integration: Works well with time-series databases and streaming data sources. [2]
A 4
Selected Answer: A Amazon QuickSight is mainly built for business intelligence and data visualization, and it can provide near real-time views depending on the data refresh rate. However, it is not usually used for real-time streaming data visualization with very low latency. For real-time dashboards with very low latency, services like Grafana are more suitable. You can use Amazon Managed Grafana to set up the dashboard so you're using an AWS service which is always preferible on these exams.
2
C. - **Processing Sensor Data with Amazon Flink**: Like option A, this approach uses Amazon Managed Service for Apache Flink to process sensor data, giving real-time analytics or transformation capabilities. - **Data Firehose Delivery Stream to Timestream**: Creates a new Amazon Data Firehose delivery stream to publish processed data directly to Amazon Timestream. Data Firehose is a fully managed service for delivering real-time streaming data to destinations such as data lakes, databases, and analytics services. - **Timestream Database as a Source for QuickSight Dashboard**: Like option B, the data stored in Amazon Timestream acts as the data source for creating an Amazon QuickSight dashboard.
2
Based on this it should be C, why use an open source app when you can an AWS Service https://community.amazonquicksight.com/t/real-time-data-visualization-capabilities-of-amazon-quicksight/24007
1
Considerations: Option A uses Amazon Managed Service for Apache Flink to process sensor data and then writes the processed data to Amazon Timestream. From there, the Timestream database acts as a source to create a Grafana dashboard. So the data passes through Apache Flink for processing, then to Timestream, and finally to Grafana. "Each additional step introduces potential latency". Option C processes sensor data using Amazon Managed Service for Apache Flink and then publishes data directly to Amazon Timestream via a Data Firehose delivery stream. Finally, it uses Timestream as a source to create an Amazon QuickSight dashboard. So, in terms of latency, both options involve processing data in real-time using Apache Flink. However, Option C has a more direct data flow by publishing data directly to Timestream, which could reduce latency compared to Option A, where the data has to go through an extra step of writing to Timestream.
1
A. - **Processing Sensor Data**: Uses Amazon Managed Service for Apache Flink, a fully managed service for real-time data processing. This service is used to process the sensor data, which likely means real-time analysis or transformation of incoming data streams. - **Connector for Apache Flink to Amazon Timestream**: Adds a connector for Apache Flink to write processed data into Amazon Timestream, a fully managed time-series database. Timestream is optimized for IoT and time-series data. - **Timestream Database as a Source for Grafana Dashboard**: The data stored in Timestream acts as the data source for creating a Grafana dashboard. Grafana is a popular open-source analytics and monitoring platform that visualizes time-series data.
1
Option A: - Involves extra steps: Option A requires writing data to Amazon Timestream after processing with Apache Flink, which could add latency compared to a more direct path like Option C. - Grafana integration: While Grafana is a strong visualization tool, setting up and configuring Grafana dashboards might require more effort compared to using Amazon QuickSight, which offers more straightforward integration with AWS services like Amazon Timestream.
1
I would go with Option A. Kinesis Data Firehose can connect to 3 AWS destinations so far S3, Redshift and OpenSearch.
1
The Question is: Which solution will meet these requirements with the LOWEST latency? So only A can be the right answer "lowest latency!!!!"
1
No! :-) Because: The company needs to show a REAL-TIME view of operational efficiency on a large screen in the manufacturing facility. So it's definitely C
C 1
Selected Answer: C Because: The company needs to show a real-time view of operational efficiency on a large screen in the manufacturing facility.
1
Option A is for processing data in Flink and then sending it to Timestream. This is useful when complex data processing is needed in Flink, but the processing step where complex analytics are handled can tolerate extra latency. Option C does data processing in Flink, sends the data straight to Timestream without any extra steps, and provides dashboards via QuickSight. Since data can be started immediately after arriving in Timestream, latency is likely to be higher. Therefore, option C is preferable because it can handle latency by doing data processing, publishing data directly to Timestream, and provides fast dashboards using QuickSight.
1
V0811 - and new firehose, ofc lowers latency, khomon man
A 1
Selected Answer: A A is right
A 1
Selected Answer: A Use Managed Apache Flink to process Kinesis streams in real time and write to Timestream. Timestream + Grafana gives the lowest-latency live dashboard.
A 1
Selected Answer: A A: Kinesis data stream --> kinesis data analytics (Flink) --> Timestream --> Quicksight/Graphana