Unveiling the Secrets of SQS Monitoring with Cloud Watch
Simple Queue Service: Amazon Simple Queue Service (SQS) is a fully managed message queuing service provided by Amazon Web Services (AWS). It enables you to decouple and scale microservices, distributed systems, and serverless applications by allowing components to communicate asynchronously. SQS helps in building fault-tolerant and loosely coupled systems, where different parts of an application can communicate with each other without direct dependencies.
CloudWatch alarms benefits for Amazon SQS : Cloud Watch Alarms help in detecting faults or abnormalities in the SQS environment. For example, we can set up an alarm to trigger when the number of messages in a queue exceeds a certain threshold, indicating a potential issue that requires investigation and resolution.
CloudWatch metrics for SQS queues are collected and pushed to CloudWatch at one-minute intervals automatically. These metrics are provided at no charge in CloudWatch for both standard and FIFO queues. These metrics are essential for understanding how your queues are behaving and for setting up alarms to detect and respond to issues.
Important CloudWatch metrics for SQS queues:
- ApproximateNumberOfMessagesVisible:
Description: Represents the approximate number of messages available for retrieval from the queue.
Use Case: Monitor the current workload of the queue and adjust resources accordingly.
- ApproximateNumberOfMessagesNotVisible:
Description: Represents the approximate number of messages that are in flight, i.e., not yet deleted or timed out.
Use Case: Helps track messages that are being processed and ensures that they are not accumulating.
- ApproximateNumberOfMessagesDelayed:
Description: Represents the approximate number of messages in the queue that are delayed and not available for processing.
Use Case: Useful for tracking delayed messages and ensuring they are processed after the delay period. This will happen when the queue is configured as a delay queue or when a message has been sent with a delay parameter.
- NumberOfMessagesSent:
Description: Represents the total number of messages sent to the queue during the specified time period.
Use Case: Provides insights into the overall message load on the queue.
- NumberOfMessagesReceived:
Description: Represents the total number of messages received from the queue during the specified time period.
Use Case: Helps monitor the rate at which messages are produced to the queue.
- NumberOfMessagesDeleted:
Description: Represents the total number of messages deleted from the queue during the specified time period.
Use Case: Useful for tracking the rate of message processing and ensuring messages are deleted after processing.
- NumberOfEmptyReceives:
Description: Represents the total number of ReceiveMessage API calls that return no messages.
Use Case: Can indicate periods of low activity or help optimize resource allocation.
- SentMessageSize:
Description: Represents the size of messages sent to the queue during the specified time period.
Use Case: Helps monitor the size of messages being sent, which is crucial for resource planning and optimization.
- ReceiveMessageSize:
Description: Represents the size of messages received from the queue during the specified time period.
Use Case: Useful for monitoring the size of messages being processed by consumers.
Step by Step process to create CloudWatch Alarm for SQS :
In this article, I will create an alarm for the " ApproximateNumberOfMessagesVisible " metric and integrate with SNS to get email notification when the queue is holding more than 50 messages in the period of 1 minute.
Step 1 : Login to AWS Account and navigate to CloudWatch then Choose Alarms and Click Create alarm
Step 2: Under Specify metric and conditions, Click Select metric and from the lists of available metrics , Choose SQS and then click Queue Metrics
Step 3: Select the QueueName and the Metric Name ApproximateNumberOfMessagesVisible and then click Select metric
Step 4: Under Statistic Choose Minimum , Under Period Choose 1 minute and Under Conditions Threshold type should be static and whenever ApproximateNumberOfMessagesVisible is Greater/Equal to 50
Step 5: Add SNS ARN to get email alert .When the queue is holding more than 50 messages in the period of 1 minute we will be alerted.
Step 6: Add name and description then Review the configuration and click on the "Create alarm" button which will create an alarm.
Step 7 : Now you can see the alarm we just created. The state of the alarm will change based on the threshold value and condition.
The alarm we created is triggered Whenever ApproximateNumberOfMessagesVisible is Greater/Equal to 50 for Period 1 minutes and when it is triggered a notification is sent to the SNS topic like below.
SNS email alert :
Conclusion : In this article, we created an alarm for SQS queue. The alarm we created is triggered Whenever ApproximateNumberOfMessagesVisible isGreater/Equal to 50 for Period 1 minutes and when it is triggered a notification is sent to the SNS topic. You can now follow the same steps to create alarms for any other available metric, with different statistics and the threshold values.