Q33Design High-Performing Architectures
A company’s application uses Network Load Balancers, Auto Scaling groups, Amazon EC2 instances, and databases deployed in an Amazon VPC. The company wants to capture information about traffic to and from the network interfaces in near real time in its Amazon VPC. The company also wants to send this information to Amazon OpenSearch Service for analysis. Which solution will meet these requirements?
← → navigate · a answer
Community votes
Discussion · 8
B 10
CloudTrail is for logging administrative actions, we need CloudWatch. We want the data in another AWS service (OpenSearch), not Kinesis, thus we need Firehose, not Streams.
B 5
Amazon CloudWatch Logs and VPC Flow Logs (Option B): VPC Flow Logs capture information about the IP traffic going to and from network interfaces in a VPC. By configuring VPC Flow Logs to send the log data to a log group in Amazon CloudWatch Logs, you can then use Amazon Kinesis Data Firehose to stream the logs from the log group to Amazon OpenSearch Service for analysis. This approach provides near real-time streaming of logs to the analytics service.
B 5
OpenSearch patterns for CloudWatch Logs:
1) "Near Real Time": CloudWatch logs --> Subscription Filter --> Kinesis Data Firehose --> Amazon OpenSearch (option *B*)
2) "Real Time": CloudWatch logs --> Subscription Filter --> Lambda --> Amazon OpenSearch
2
log analysis place= aws cloudwatch log
data capturing on the entire vpc=aws flow log
near real time data analysis and send to OpenSearch service= kinesis data fire hose
2
While Amazon Kinesis Data Streams can stream data, it requires additional setup (e.g., Lambda functions) to process and send logs to OpenSearch Service. This adds complexity compared to Kinesis Data Firehose, which is purpose-built for this use case.
A 1
Currently, Firehose does not support the delivery of CloudWatch Logs to Amazon OpenSearch Service destination because Amazon CloudWatch combines multiple log events into one Firehose record and Amazon OpenSearch Service cannot accept multiple log events in one record. As an alternative, you can consider Using subscription filter for Amazon OpenSearch Service in CloudWatch Logs.
https://docs.aws.amazon.com/firehose/latest/dev/writing-with-cloudwatch-logs.html
1
base on the research, it should be Answer A, because question is asking for a "near real time" which Kinesis Data Stream is offering the data with less than 1 second latency. But Kinese Data Firehost is offering the data with more than 1 second.
https://docs.aws.amazon.com/opensearch-service/latest/developerguide/integrations.html#integrations-kinesis
https://stackoverflow.com/questions/44608274/is-there-any-difference-in-processing-times-between-aws-kinesis-firehose-and-str
A 1
base on the research, it should be Answer A, because question is asking for a "near real time" which Kinesis Data Stream is offering the data with less than 1 second latency. But Kinese Data Firehost is offering the data with more than 1 second.
https://docs.aws.amazon.com/opensearch-service/latest/developerguide/integrations.html#integrations-kinesis
https://stackoverflow.com/questions/44608274/is-there-any-difference-in-processing-times-between-aws-kinesis-firehose-and-str
https://docs.aws.amazon.com/streams/latest/dev/using-other-services-cw-logs.html