Overview of Queue

Oracle Cloud Infrastructure (OCI) Queue is a fully managed serverless service that helps decouple systems and enable asynchronous operations. Queue handles high-volume transactional data that requires independently processed messages without loss or duplication. Queue supports transparent, automatic scaling based on throughput for producers and consumers. Queue uses open standards to support communication with any client or producer with minimal effort.

The OCI Queue service is built on four principles:

Publishing
Messages can be published to a queue by one or more producers, each with a retention period. If retention isn't specified, the message expires using a retention period defined at the queue level. A message contains a payload in the form of a string.
Consuming
Multiple consumers can consume messages from a single queue. Consumer count can scale along with the rate of messages being published. After a message is delivered to a consumer, the message is hidden from other consumers for a pre-defined amount of time, which is known as its visibility timeout.
Updating
If processing a message takes longer than expected, consumers can extend the visibility timeout of a message. Extending the visibility timeout prevents the message from being returned to the queue and being delivered to another consumer.
Deleting
After a message has been delivered to and processed by a consumer, the message must be deleted to prevent redelivery to another consumer.

Benefits

The Queue service provides the following benefits.

Application Decoupling

Queue helps decouple applications and systems by using event-driven architecture. Decoupling ensures that individual application components can scale independently and that as new application components get built, they can publish or subscribe to the queue.

A diagram representing a producer sending messages to multiple queues consumed by a consumer.

Reliable Message Processing

Queue guarantees that a message is never lost, even if the consumer is unavailable for consumption. A published message is persistent until it's either deleted or expired.

If a message isn't consumed successfully, it's sent to a dead letter queue (DLQ). Dead letter queues let you isolate problematic messages to determine why they're failing. Isolating and consuming problematic messages in this way can guarantee successful delivery to a consumer application at least once. See Dead Letter Queues for more information.

Open Standards

Queue can be called using RESTful API (with an Open API specification) definition or by using the industry standard STOMP protocol.

Concepts

The Queue service uses the following concepts.

message
A message is an element in a queue that contains a payload in the form of a string. The string can be in any format, including XML, JSON, CSV, a Base64-encoded binary message, and even compressed formats such as gzip. Producers and consumers should agree upon the message format. Each message is processed independently.
producer
A client which sends messages to the queue.
consumer
A client which receives messages from a queue. The consumer is also responsible for deleting messages from the queue after the messages are received.
channel
An ephemeral destination within a queue that can be created on demand. Messages can be published to a specific channel within a queue, and consumers can retrieve messages from specific channels. For more information, see Channels.
maximum retention period
The length of time that a queue retains a message until the message is automatically deleted by the system, if not deleted by a consumer. Maximum retention period is configurable to values of 10 seconds to 7 days at the queue level. The default value is 1 day.
delivery count
The number of times that a message is delivered to a consumer upon request.
maximum delivery attempts
The number of times that a message is delivered to a consumer, but not updated or deleted, before it's sent to a dead letter queue (DLQ). Maximum delivery attempts is configurable to values of 1-20 at the queue level. For more information, see delivery count.
polling timeout
The length of time that a consumer will wait for messages to consume. Increasing the polling timeout reduces the number of times a consumer requests messages from the queue but the response indicates that there are no available messages to consume. Polling timeout is configurable to values of 0 to 30 seconds at the queue level, and consumers can set the value when requesting messages. The default value is 30 seconds. For more information, see long polling.
visibility timeout
The length of time during which a message received from the queue by one consumer isn't visible to other consumers. Visibility timeout is configurable to values of 1 second to 12 hours at the queue level, and consumers can set the value when requesting messages. The default value is 30 seconds. For more information, see message locking.
visible messages
The number of messages currently in a queue that are available for consumption.
in-flight messages
The number of messages delivered to a consumer but not yet deleted. In-flight messages are unavailable for redelivery until their visibility timeout has passed.
dead letter queue
If a message isn't consumed successfully, and has more delivery attempts than the configured maximum delivery attempts, the message is transferred to a dead letter queue (DLQ). For more information, see Dead Letter Queues.

Guarantees

The Queue service provides the following guarantees.

  • A successfully published message is guaranteed to be durable until it's either deleted or its retention period has passed. The publication of a message is considered successful when the Queue service sends back an acknowledgment to the producer. It does not matter whether the response was received.
  • A message within the visibility timeout is guaranteed not to be delivered to another consumer until that timeout expires.
  • A message will not be deleted by the Queue service before its retention period is over. A consumer can process and delete a message during its retention period.

Authentication and Authorization

Each service in Oracle Cloud Infrastructure integrates with IAM for authentication and authorization, for all interfaces (the Console, SDK or CLI, and REST API).

An administrator in your organization needs to set up groups, compartments , and policies  that control which users can access which services, and which resources, and the type of access they have. For example, policies control who can create users, groups, and compartments, or who can create and manage virtual deployments.

Ways to Access Queue

You can access Queue by using the Console (a browser-based interface), Oracle Cloud Infrastructure CLI, or REST APIs.

Instructions for all three methods are included throughout this guide.

Service Limits

When you sign up for Oracle Cloud Infrastructure, a set of service limits is configured for your tenancy. The service limit is the quota or allowance set on a resource. Review the following service limits for Queue resources.

Resource Details
Queues 10 per tenancy per region

See Service Limits to learn more about service limits and find instructions for requesting a limit increase. To set compartment-specific limits on a resource or resource family, administrators can use compartment quotas.

Functional Limits

In addition to service resource limits, specific Queue limits exist for features and functionality.

Limit Details
Channels per queue 256 per queue
Maximum PutMessage request size 512 KB and 20 messages
Maximum GetMessage response size 2 MB and 20 messages
Maximum message size 256 KB
Maximum number of in-flight messages 100,000 per queue
Maximum messages per queue Unlimited
Message retention Maximum: 7 days

Minimum: 10 seconds

Default: 1 day

Message visibility timeout Maximum: 12 hours

Minimum: 0 seconds at message level

Minimum: 1 second at queue level

Default: 30 seconds

Maximum concurrent GET requests 1,000 requests per second per queue
Maximum message operations 1,000 requests per seconds per API per queue
Maximum data rate Ingress per queue: 10 MB/s

Egress per queue: 10 MB/s

Polling timeout Maximum: 30 seconds

Minimum: 0 seconds

STOMP throughput 10 MB/s per STOMP connection
Storage 20 GB per tenancy

2 GB per queue