📊
Data Analytics Pipeline
Kinesis → Firehose → S3 data lake → Glue ETL → Redshift with Athena.
What you can build with this
Ingest high-volume event streams in real time, store raw data in a data lake on S3, run ETL jobs to transform the data, then query it ad-hoc with Athena or load it into Redshift for BI dashboards and reporting. The Redshift cluster deploys privately (no public access) with credentials auto-generated into AWS Secrets Manager.
Deployment timing
After you run the deploy command, resources come online at different times. Check the validation steps in order.
| Resource | Ready in |
|---|---|
| Kinesis Stream | 1–2 min |
| Firehose | 1–2 min |
| S3 Buckets | Immediate |
| First data in S3 (via Firehose) | Up to 5 min after first record sent (128 MB / 300 s buffering) |
| Redshift Cluster | 8–12 min after stack starts |
| Glue crawler you create (schema discovery) | 2–5 min per run |
CloudFormation parameters
These are the values you will fill in when deploying the exported template. Changing these does not break the template — that is the point.
| Parameter | What to provide | Example |
|---|---|---|
| Environment | Environment tag applied to resources (does not rename them) | dev |