Saturday, April 18, 2009

Amazon Simple Queue Service (Amazon SQS)

From Amazon .. 

Amazon Simple Queue Service (Amazon SQS) offers a reliable, highly scalable, hosted queue for storing messages as they travel between computers. By using Amazon SQS, developers can simply move data between distributed components of their applications that perform different tasks, without losing messages or requiring each component to be always available. Amazon SQS makes it easy to build an automated workflow, working in close conjunction with the Amazon Elastic Compute Cloud (Amazon EC2) and the other AWS infrastructure web services.

Amazon SQS works by exposing Amazon’s web-scale messaging infrastructure as a web service. Any computer on the Internet can add or read messages without any installed software or special firewall configurations. Components of applications using Amazon SQS can run independently, and do not need to be on the same network, developed with the same technologies, or running at the same time.

Amazon SQS Functionality

  • Developers can create an unlimited number of Amazon SQS queues with an unlimited number of messages.
  • A queue can be created in the United States or in Europe. Queue names and message stores are independent of other regions.
  • The message body can contain up to 8 KB of text in any format.
  • Messages can be retained in queues for up to 4 days.
  • Messages can be sent and read simultaneously.
  • When a message is received, it becomes “locked” while being processed. This keeps other computers from processing the message simultaneously. If the message processing fails, the lock will expire and the message will be available again. In the case where the application needs more time for processing, the “lock” timeout can be changed dynamically via the ChangeMessageVisibility operation.
  • Developers can access Amazon SQS through standards-based SOAP and Query interfaces.
  • Developers can securely share Amazon SQS queues with others. Queues can be shared with other AWS accounts and Anonymously. Queue sharing can also be restricted by IP address and time-of-day.

Service Highlights

Amazon SQS provides significant advantages over the complexity of home-grown messaging systems or the ongoing expense of licensed or hosted systems.

Reliable – Amazon SQS runs within Amazon’s high-availability data centers, so queues will be available whenever applications need them. To prevent messages from being lost or becoming unavailable, all messages are stored redundantly across multiple servers and data centers.

Simple – Developers can get started with Amazon SQS by using only five APIs: CreateQueue, SendMessage, ReceiveMessage, ChangeMessageVisibility, and DeleteMessage. Additional APIs are available to provide advanced functionality. In all cases, the SOAP and query APIs can be used with virtually any language and platform.

Scalable – Amazon SQS was designed to enable an unlimited number of computers to read and write an unlimited number of messages at any time.

Secure – Authentication mechanisms are provided to ensure that messages stored in Amazon SQS queues are secured against unauthorized access.

Inexpensive – No up-front or fixed expenses. The only costs of sending messages through Amazon SQS are small per-request handling fees and data transfer fees.


Using Amazon SQS with Other AWS Infrastructure Web Services

Amazon SQS can be used with Amazon EC2, as well as Amazon S3 and Amazon SimpleDB, to make applications more flexible and scalable. A common use case is to create an integrated and automated workflow, where multiple components or modules need to communicate with each other, but can’t all process the same amount of work simultaneously. In this case, SQS queues carry messages to be processed in an orderly fashion by the user’s application running on Amazon EC2instances. The Amazon EC2 instances can read the queue, process the job, and then post the results as messages to another SQS queue (possibly for further processing by another application). Because Amazon EC2 allows applications to scale up and down dynamically, application developers can easily vary the number of compute instances based on the amount of work in the SQS queues, to ensure that jobs are executed in a timely manner.

For example, here is how a video transcoding website uses Amazon EC2, Amazon SQS, Amazon S3, and Amazon SimpleDB together. End users submit videos to be transcoded to the website. The videos are stored in Amazon S3, and a message (“the request message”) is placed in an Amazon SQS queue (“the incoming queue”) with a pointer to the video and to the target video format in the message. The transcoding engine, running on a set of Amazon EC2 instances, reads the request message from the incoming queue, retrieves the video from Amazon S3 using the pointer, and transcodes the video into the target format. The converted video is put back into Amazon S3 and another message (“the response message”) is placed in another Amazon SQS queue (“the outgoing queue”) with a pointer to the converted video. At the same time, metadata about the video (e.g., format, date created and length) can be indexed into Amazon SimpleDB for easy querying. During this whole workflow, a dedicated Amazon EC2 instance can constantly monitor the incoming queue and, based on the number of messages in the incoming queue, is able to dynamically adjust the number of transcoding Amazon EC2 instances to meet customers’ response time requirements.


Pricing

Pay only for what you use. There is no minimum fee. Estimate your monthly bill using AWS Simple Monthly Calculator.

Requests

  • $0.01 per 10,000 Amazon SQS Requests ($0.000001 per Request)

Amazon SQS requests are CreateQueue, ListQueues, DeleteQueue, SendMessage, ReceiveMessage, ChangeMessageVisibility, DeleteMessage, SetQueueAttributes, GetQueueAttributes, AddPermision, and RemovePermission

Data Transfer

  • $0.100 per GB – all data transfer in

  • $0.170 per GB – first 10 TB / month data transfer out
  • $0.130 per GB – next 40 TB / month data transfer out
  • $0.110 per GB – next 100 TB / month data transfer out
  • $0.100 per GB – data transfer out / month over 150 TB

Data transfer “in” and “out” refers to transfer into and out of Amazon SQS. Data transferred between Amazon SQS and Amazon EC2 in the US region, or between Amazon SQS and Amazon EC2 in the EU region is free of charge (i.e., $0.00 per GB). Data transferred between Amazon SQS and Amazon EC2 in different regions will be charged at Internet Data Transfer rates on both sides of the transfer.

(Amazon SQS is sold by Amazon Web Services LLC.)