AWS SQS

Amazon SQS is a message queue service used by distributed applications to exchange messages through a polling model, and can be used to decouple sending and receiving components.

Amazon Simple Queue Service (SQS) is a fully managed message queuing service that enables us to decouple and scale microservices, distributed systems, and serverless applications. SQS eliminates the complexity and overhead associated with managing and operating message oriented middleware. Using SQS, we can send, store, and receive messages between software components at any volume, without losing messages or requiring other services to be available.

Features of SQS:

• Messages can be sent, received or deleted in batches of up to 10 messages or 256KB. Batches cost the same amount as single messages, meaning SQS can be even more cost effective for customers that use batching.
• Messages can be sent and read simultaneously.
• The message payload can contain up to 256KB of text in any format. Each 64KB ‘chunk’ of payload is billed as 1 request. For example, a single API call with a 256KB payload will be billed as four requests.
• Persistent message queue with high durability and availability.
• Messages are stored across multiple AZs.
• Messages retained until deleted — upto 14 days.
• Provides nearly unlimited throughput.

SQS queues come in two distinct types:

• Standard SQS queues are able to scale to enormous throughput with at-least-once delivery.
• FIFO queues are designed to guarantee that messages are processed exactly once in the exact order that they are received and have a default rate of 300 transactions per second.

Kind of use-cases that can be solved using SQS:

Suppose that we run a major auction platform where people buy and sell a wide range of products, platform requires that transactions from buyers and sellers get processed in exactly the order received. Here’s how a FIFO queue helps you keep all your transactions in one straight flow.

A seller currently is holding an auction for a laptop, and three different bids are received for the same price. Ties are awarded to the first bidder at that price so it is important to track which arrived first. Our auction platform receives the three bids and sends them to a FIFO queue before they are processed.

Now observe how messages leave the queue. When our consumer asks for a batch of up to 10 messages, SQS starts filling the batch with the oldest message (bid A1). It keeps filling until either the batch is full or the queue is empty. In this case, the batch contains the three messages and the queue is now empty. After a batch has left the queue, SQS considers that batch of messages to be “in-flight” until the consumer either deletes them or the batch’s visibility timer expires.

Industry Use-Case of AWS SQS:

The National Aeronautics and Space Administration (NASA) is an independent agency of the U.S. federal government responsible for the civilian space program, as well as aeronautics and space research.

NASA was established in 1958, succeeding the National Advisory Committee for Aeronautics (NACA). The new agency was to have a distinctly civilian orientation, encouraging peaceful applications in space science.

Development of the new NASA Image and Video Library was handled by the Web Services Office within NASA’s Enterprise Service and Integration Division. Technology selection, solution design, and implementation was managed by InfoZen, the WESTPrime contract service provider. As an Advanced Consulting Partner of the AWS Partner Network (APN), InfoZen chose to build the solution on Amazon Web Services (AWS). Amazon was the largest cloud services provider, had a strong government cloud presence, and offered the most suitable cloud in terms of elasticity,

NASA formally launched its Image and Video Library in March 2017.

Architecture OF NASA’S Image and Video Library:

The NASA Image and Video Library is a cloud-native solution, with the front-end web app separated from the backend API. It runs as immutable infrastructure in a fully automated environment, with all infrastructure defined in code to support continuous integration and continuous deployment (CI/CD).

In building the solution, ManTech International took advantage of the following AWS services:

• Amazon Elastic Compute Cloud (Amazon EC2), which provides secure, resizable compute capacity in the cloud. This enables NASA to scale up under load and scale down during periods of inactivity to save money, and pay for only what it uses.

• Elastic Load Balancing (ELB), which is used to distribute incoming traffic across multiple Amazon EC2 instances, as required to achieve redundancy and fault-tolerance.

• Amazon Simple Storage Service (Amazon S3), which supports object storage for incoming (uploaded) media, metadata, and published assets.

• Amazon Simple Queue Service (Amazon SQS), which is used to decouple incoming jobs from pipeline processes.

• Amazon Relational Database Service (Amazon RDS), which is used for automatic synchronization and failover.

• Amazon DynamoDB, a fast and flexible NoSQL database service, which is used to track incoming jobs, published assets, and users.

• Amazon Elastic Transcoder, which is used to transcode audio and video to various resolutions.

• Amazon CloudSearch, which is used to support searching by free text or fields.

• Amazon Simple Notification Service (Amazon SNS), which is used to trigger the processing pipeline when new content is uploaded.

• AWS CloudFormation, which enables automated creation, updating, and destruction of AWS resources.

• Amazon CloudWatch, which provides a monitoring service for AWS cloud resources and the applications running on AWS.

Key features include:

• Built-in Scalability: All components of the NASA Image and Video Library are built to scale on demand, as needed to handle usage spikes.

• Good Use of Taxpayer Dollars: By building its Image and Video Library in the cloud, NASA avoided the costs associated with deploying and maintaining server and storage hardware in-house. Instead, the agency can simply pay for the AWS resources it uses at any given time.

#vimaldaga #righteducation #educationredefine#rightmentor #worldrecordholder #linuxworld #makingindiafutureready #righeducation #arthbylw#aws #awssqs #amazonwebservice