Option 2 - Event Driven Data Lake
The purpose of this lab is to demonstrate an event-driven data lake ingestion strategy, as well as how to use AWS services to transform the raw data, query the data lake and create visualizations of the data.
In this lab you will perform the following tasks:
- Validate sample data
- Configure an ETL pipeline using S3 event triggers and AWS Lambda
- Catalog ingested data using AWS Glue
- Transform ingested data into Parquet format using an AWS Glue ETL job
- Execute ad-hoc queries on transformed data using Amazon Athena
- Visualize formatted data using Amazon QuickSight
Services Used
Click below to be taken to the product page for services used in this lab.
- Amazon Simple Storage Service (S3)
- AWS Glue
- AWS Lambda
- Amazon CloudWatch
- Amazon Athena
- Amazon QuickSight
- Amazon SNS
Duration : Approx. 2-2.5 hours