Skip to main content

Azure Table Storage

Overview

The Azure table storage supports Full Refresh and Incremental syncs. You can choose which tables you want to replicate.

Output schema

This Source have generic schema for all streams. Azure Table storage is a service that stores non-relational structured data (also known as structured NoSQL data). There is no efficient way to read schema for the given table. We use data property to have all the properties for any given row.

  • data - This property contains all values
  • additionalProperties - This property denotes that all the values are in data property.
{
"$schema": "http://json-schema.org/draft-07/schema#",
"type": "object",
"properties": {
"data": {
"type": "object"
},
"additionalProperties": {
"type": "boolean"
}
}
}

Data type mapping

Azure Table Storage uses different property types and Airbyte uses internally (string, date-time, object, array, boolean, integer, and number). We don't apply any explicit data type mappings.

Features

FeatureSupported?
Full Refresh SyncYes
Incremental - Append SyncYes
Incremental - Dedupe SyncNo
SSL connectionYes
NamespacesNo

Performance considerations

The Azure table storage connector should not run into API limitations under normal usage. Please create an issue if you see any rate limit issues that are not automatically retried successfully.

Getting started

Requirements

  • Azure Storage Account
  • Azure Storage Account Key
  • Azure Storage Endpoint Suffix

Setup guide

Visit the Azure Portal. Go to your storage account, you can find :

  • Azure Storage Account - under the overview tab
  • Azure Storage Account Key - under the Access keys tab
  • Azure Storage Endpoint Suffix - under the Endpoint tab

We recommend creating a restricted key specifically for Airbyte access. This will allow you to control which resources Airbyte should be able to access. However, shared access key authentication is not supported by this connector yet.

Reference

Config fields reference

Field
Type
Property name
string
storage_account_name
string
storage_access_key
string
storage_endpoint_suffix

Changelog

Expand to review
VersionDatePull RequestSubject
0.1.82024-06-0438968[autopull] Upgrade base image to v1.2.1
0.1.72024-06-0338915Replace AirbyteLogger with logging.Logger
0.1.62024-06-0338915Replace AirbyteLogger with logging.Logger
0.1.52024-05-2038443[autopull] base image + poetry + up_to_date
0.1.42024-01-2634576Migrate to per-stream/global state
0.1.32022-08-1215591Clean instantiation of AirbyteStream
0.1.22021-12-2314212Adding incremental load capability
0.1.12021-12-238434Update fields in source-connectors specifications