Azure Service Bus

📘

Check out the official Azure Service Bus docs here

Set up

The connector authenticates using Shared Access Signature (SAS) tokens, which are generated from Shared Access Policies configured in Azure Service Bus. More info can be found here and here

To set up a connector Source for Azure Service Bus, you will need:

• Namespace: The Service Bus namespace hostname (e.g. example.servicebus.windows.net)
• Access Policy Name: The name of a Shared Access Policy with Listen permissions
• Access Policy Key: The primary or secondary key associated with the policy

For each Extract you want to set up, you will need the Queue Path (this is the name of the Service Bus queue to read from)

Features

Reports detail

📘

Assumptions & constraints

• The queue is dedicated to this connector (no competing consumers)
• Message ordering is not guaranteed
• Duplicate messages may occur and should be handled downstream if applicable

📘

Incremental behaviour

The connector processes messages until the queue is empty. Incremental extraction is based on queue semantics:

• New messages added after a successful run remain in the queue
• On the next run, only those new messages are extracted

📘

Data Format

• Each queue message must contain valid JSON
• Each JSON field is mapped to a column in the destination data warehouse table
• One queue corresponds to one extract and to one table in the data warehouse
• One message corresponds to one row in the warehouse

🚧

Destructive operation

The connector uses a destructive read operation. The Receive and Delete Message endpoint retrieves a message and immediately removes it from the queue.

If the connector fails partway through a run (before data is written to the Data Warehouse table), any messages already read will not be available for reprocessing unless the producer republishes them.