# AWS Service - SQS

**SQS:**

* **Standard Queue**
    
    * Oldest offering (over 10 years old)
        
    * Fully managed service, used to decouple applications
        
    * Attributes:
        
        * Unlimited throughput, unlimited number of messages in queue
            
        * Default retention of messages: 4 days, maximum of 14 days
            
        * Low latency (<10 ms on publish and receive)
            
        * Limitation of 256KB per message sent
            
    * Can have duplicate messages (at least once delivery, occasionally)
        
    * Can have out of order messages (best effort ordering)
        
* **Producing Messages**
    
    * Produced to SQS using the SDK (SendMessage API)
        
    * The message is persisted in SQS until a consumer deletes it
        
    * Message retention: default 4 days, up to 14 days
        
    * Message size up to 256KB
        
    * Example: send an order to be processed
        
        * Order id; Customer id
            
        * Any attributes you want
            
    * SQS standard: unlimited throughput
        
* **Consuming Messages**
    
    * Consumers (running on EC2 instances, servers, or AWS Lambda)
        
    * Poll SQS for messages (receive up to 10 messages at a time)
        
    * Process the messages (example: insert the message into an RDS database)
        
    * Delete the messages using the DeleteMessage API
        

![](https://cdn.hashnode.com/res/hashnode/image/upload/v1727720830202/9d480b3a-eebc-4c7b-938f-7eec6048b0df.png align="center")

* **Multiple EC2 Instances Consumers**
    
    * Consumers receive and process messages in parallel
        
    * At least once delivery
        
    * Best-effort message ordering
        
    * Consumers delete messages after processing them
        
    * We can scale consumers horizontally to improve throughput of processing
        

![](https://cdn.hashnode.com/res/hashnode/image/upload/v1727721196081/8fc2c401-b1a2-4656-880c-caf3f16c25bb.png align="center")

* **SQS to decouple between application tiers**
    

![](https://cdn.hashnode.com/res/hashnode/image/upload/v1727721293419/2947793d-659a-46a2-9b42-af62b83f7b9d.png align="center")

* **Security**
    
    * Encryption
        
        * In-flight encryption using HTTPS API
            
        * At-rest encryption using KMS keys
            
        * Client-side encryption if the client wants to perform encryption and decryption itself
            
    * Access Controls
        
        * IAM policies to regulate access to the SQS API
            
    * SQS Access Policies
        
        * Useful for cross-account access to SQS queues
            
        * Useful for allowing other services (SNS, S3) to write to an SQS queue
            
* **Message Visibility Timeout**
    
    * After a message is polled by a consumer, it becomes invisible to other consumers
        
    * By default, the `message visibility timeout` is 30 seconds
        
    * That means the message has 30 seconds to be processed
        
    * After the message visibility timeout is over, the message is “visible” in SQS
        
    * If a message is not processed within the visibility timeout, it will be processed twice
        
    * A consumer could call the ChangeMessageVisibility API to get more time
        
    * If visibility timeout is high (hours), and consumer crashes, re-processing will take time
        
    * If visibility timeout is too low (seconds), we may get duplicates
        

![](https://cdn.hashnode.com/res/hashnode/image/upload/v1727721566735/aa244fc7-6274-4b2a-9e8f-9a79935a823f.png align="center")

* **Long Polling**
    
    * When a consumer requests messages from the queue, it can optionally `wait` for messages to arrive if there are none in the queue
        
    * This is called Long Polling
        
    * LongPolling decreases the number of API calls made to SQS while increasing the efficiency and reducing latency of your application
        
    * The wait time can be between 1 sec to 20 sec (20 sec preferable)
        
    * Long Polling is preferable to Short Polling
        
    * Long polling can be enabled at the queue level or at the API level using WaitTimeSeconds
        

![](https://cdn.hashnode.com/res/hashnode/image/upload/v1727721944834/f8664d59-9ec9-45db-b8df-dc99ac7bd037.png align="center")

* **FIFO Queue**
    
    * FIFO - First In First Out (ordering of messages in the queue)
        
    * Limited throughput: 300 message per sec without batching, 3000 message per sec with
        
    * Exactly-once send capability (by removing duplicates)
        
    * Messages are processed in order by the consumer
        
* **SQS with Auto Scaling Group (ASG)**
    

![](https://cdn.hashnode.com/res/hashnode/image/upload/v1727722160319/c8d0e2f9-ff62-49aa-93ca-f368d385c668.png align="center")

* If the load is too big, some transactions may be lost so in this cases SQS as a buffer to database writes would help
    

![](https://cdn.hashnode.com/res/hashnode/image/upload/v1727722242651/462a8312-765c-482f-839c-8d78fc882a1d.png align="center")

![](https://cdn.hashnode.com/res/hashnode/image/upload/v1727722265875/a7018386-c867-42d1-8d7b-0358dc0bc481.png align="center")

* **SQS to decouple between application tiers**
    

![](https://cdn.hashnode.com/res/hashnode/image/upload/v1727722370599/b35fb920-7bbb-462b-800b-1524e7386ff3.png align="center")

**Relevant Doc:**

[https://docs.aws.amazon.com/AWSSimpleQueueService/latest/SQSDeveloperGuide/welcome.html](https://docs.aws.amazon.com/AWSSimpleQueueService/latest/SQSDeveloperGuide/welcome.html)

**Disclaimer:** This is a personal blog that might come in handy when I suffer from Dementia in future.