CloudWatch, CloudTrail, alarms and operational evidence

Why and what

Metrics are time-series measurements; logs are event records; traces follow requests across components. CloudWatch collects/queries operational telemetry. CloudTrail records supported AWS API activity; it is not a replacement for an Nginx access log. AWS Config records supported resource configuration/history and compliance evaluations.

Lab: build an operational view

1. Open EC2 metrics and inspect CPUUtilization and status checks. Choose a correct period/statistic.
2. Install/configure the CloudWatch Agent if memory, filesystem usage or custom logs are required. Basic EC2 metrics do not automatically include guest memory usage.
3. Give the instance role only required log/metric permissions. Collect Nginx error logs into a named log group with a defined retention period.
4. Create an alarm for an appropriate threshold and evaluation window. Configure missing-data treatment intentionally.
5. Subscribe a notification destination you control and confirm any required subscription.
6. Produce a small, authorized test error and verify the log, metric/alarm and notification path.

Logs Insights example

fields @timestamp, @message
| filter @message like /error/
| sort @timestamp desc
| limit 20

Select the correct log groups and time window first. Queries can incur scan charges; narrow the scope. Structured JSON logs make fields easier to query than free text, but avoid logging passwords, tokens and full sensitive request bodies.

Audit experiment

Change a disposable security-group rule and find the corresponding CloudTrail event. Check actor, time, Region and request parameters. Event history has scope/retention limits; trails or event data stores are needed for longer-term/custom audit requirements. Data events such as object access may require explicit selection and incur charges.

Incident method

Record impact, timeline, evidence, hypothesis, mitigation and follow-up. An alarm is a signal requiring context. High CPU with healthy latency may need capacity planning rather than emergency restart. A sudden 5xx rise after deployment should be correlated with release time and target health.

Cleanup and interview

Remove lab alarms/notification subscriptions and set log retention rather than letting logs grow indefinitely. Why can CloudTrail show no event for a Linux process crash? The crash is an OS/application event, not necessarily an AWS API call.

Official references

CloudWatch Agent CloudTrail guide

Interview and revision check

Why might CloudWatch lack guest memory metrics?

Basic EC2 metrics do not automatically collect guest memory. An agent or other explicit collector must publish it.