CloudWatch Insights

Advanced Log Analytics & Container Monitoring

CloudWatch Logs Insights

CloudWatch Logs Insights enables you to interactively search and analyze log data using a purpose-built query language. It provides fast, interactive log analytics.

Key Features
  • Query multiple log groups simultaneously
  • Automatic field discovery
  • Visualization with charts and graphs
  • Save and share queries

Query Language

Basic Commands
  • fields: Select fields to display
  • filter: Filter log events
  • stats: Calculate aggregate statistics
  • sort: Sort results
  • limit: Limit number of results
Example Query

fields @timestamp, @message | filter @message like /ERROR/ | sort @timestamp desc | limit 20

Query Functions

Aggregate Functions
  • count(), sum(), avg(), min(), max()
  • stddev() - Standard deviation
  • pct(field, percent) - Percentile
String Functions
  • strlen(), concat(), trim()
  • replace(), substr()
Date Functions
  • bin() - Group by time intervals
  • datefloor(), dateceil()

Container Insights

Container Insights collects, aggregates, and summarizes metrics and logs from containerized applications and microservices.

Supported Platforms
  • Amazon ECS (EC2 and Fargate)
  • Amazon EKS (Kubernetes)
  • Kubernetes on EC2
Metrics Collected

CPU, memory, disk, network metrics at cluster, node, pod, task, and service levels

Lambda Insights

Lambda Insights provides enhanced monitoring for AWS Lambda functions with system-level metrics and diagnostics.

Metrics Provided
  • CPU time, memory utilization
  • Cold starts and initialization time
  • Network activity
  • Disk I/O
Implementation

Add Lambda Insights layer to your function and grant CloudWatch permissions

Contributor Insights

Contributor Insights analyzes log data and creates time series that display contributor data, helping identify top talkers and understand who or what is impacting system performance.

Use Cases
  • Find top IP addresses making requests
  • Identify heaviest network users
  • Discover URLs generating most errors
  • Analyze DynamoDB throttling sources

Application Insights

CloudWatch Application Insights facilitates observability for .NET and SQL Server applications, automatically detecting and monitoring application components.

Capabilities
  • Automatic problem detection
  • Built-in monitoring templates
  • Anomaly detection using ML
  • Integration with AWS Systems Manager OpsCenter
Supported Technologies

.NET, SQL Server, IIS, Java, custom applications

Query Performance

Pricing Model

Charged based on data scanned by queries (per GB)

Optimization Tips
  • Use specific time ranges
  • Filter early in queries
  • Select only needed fields
  • Use log group filters
Query Limits

Can query up to 20 log groups at once, 10,000 log streams per query

Best Practices

✓ Save Common Queries

Save frequently used queries for quick access and sharing with team

✓ Use Visualizations

Leverage built-in visualizations to identify trends and patterns

✓ Enable Container Insights

Enable at cluster creation for comprehensive container monitoring

✓ Monitor Query Costs

Track data scanned to optimize query costs

Exam Tips

Key Point

CloudWatch Logs Insights is for interactive log analysis with a query language, while metric filters are for extracting metrics from logs

Remember

Container Insights requires CloudWatch agent (for ECS) or Fluent Bit/Fluentd (for EKS) to be deployed

Important

Lambda Insights is implemented as a Lambda layer and provides enhanced monitoring beyond standard CloudWatch metrics

Back: Dashboards Next: Agent