Microsoft Azure Azure Data Tables Client Library for Python
Azure Tables is a NoSQL data storage service that can be accessed from anywhere in the world via authenticated calls using HTTP or HTTPS.
Tables scales as needed to support the amount of data inserted, and allow for the storing of data with non-complex accessing.
The Azure Tables client can be used to access Azure Storage or Cosmos accounts. This document covers azure-data-tables
.
Please note, this package is a replacement for azure-cosmosdb-tables
which is now deprecated. See the migration guide for more details.
Source code | Package (PyPI) | Package (Conda) | API reference documentation | Samples
Azure SDK Python packages support for Python 2.7 has ended 01 January 2022. For more information and questions, please refer to https://github.com/Azure/azure-sdk-for-python/issues/20691 Python 3.7 or later is required to use this package. For more details, please refer to Azure SDK for Python version support policy.
The Azure Tables SDK can access an Azure Storage or CosmosDB account.
Install the Azure Tables client library for Python with pip:
pip install azure-data-tables
The Azure Tables library allows you to interact with two types of resources:
endpoint
can be found on the page for your storage account in the Azure Portal under the "Access Keys" section or by running the following Azure CLI command:# Get the table service URL for the account
az storage account show -n mystorageaccount -g MyResourceGroup --query "primaryEndpoints.table"
Once you have the account URL, it can be used to create the service client:
from azure.data.tables import TableServiceClient
service = TableServiceClient(endpoint="https://<my_account_name>.table.core.windows.net/", credential=credential)
For more information about table service URL's and how to configure custom domain names for Azure Storage check out the official documentation
The credential
parameter may be provided in a number of different forms, depending on the type of authorization you wish to use. The Tables library supports the following authorizations:
To use an account shared key (aka account key or access key), provide the key as a string. This can be found in your storage account in the Azure Portal under the "Access Keys" section or by running the following Azure CLI command:
az storage account keys list -g MyResourceGroup -n MyStorageAccount
Use the key as the credential parameter to authenticate the client:
from azure.core.credentials import AzureNamedKeyCredential
from azure.data.tables import TableServiceClient
credential = AzureNamedKeyCredential("my_account_name", "my_access_key")
service = TableServiceClient(endpoint="https://<my_account_name>.table.core.windows.net", credential=credential)
Depending on your use case and authorization method, you may prefer to initialize a client instance with a connection string instead of providing the account URL and credential separately. To do this, pass the
connection string to the client's from_connection_string
class method. The connection string can be found in your storage account in the Azure Portal under the "Access Keys" section or with the following Azure CLI command:
az storage account show-connection-string -g MyResourceGroup -n MyStorageAccount
from azure.data.tables import TableServiceClient
connection_string = "DefaultEndpointsProtocol=https;AccountName=<my_account_name>;AccountKey=<my_account_key>;EndpointSuffix=core.windows.net"
service = TableServiceClient.from_connection_string(conn_str=connection_string)
To use a shared access signature (SAS) token, provide the token as a string. If your account URL includes the SAS token, omit the credential parameter. You can generate a SAS token from the Azure Portal under Shared access signature or use one of the generate_*_sas()
functions to create a sas token for the account or table:
from datetime import datetime, timedelta
from azure.data.tables import TableServiceClient, generate_account_sas, ResourceTypes, AccountSasPermissions
from azure.core.credentials import AzureNamedKeyCredential, AzureSasCredential
credential = AzureNamedKeyCredential("my_account_name", "my_access_key")
sas_token = generate_account_sas(
credential,
resource_types=ResourceTypes(service=True),
permission=AccountSasPermissions(read=True),
expiry=datetime.utcnow() + timedelta(hours=1),
)
table_service_client = TableServiceClient(endpoint="https://<my_account_name>.table.core.windows.net", credential=AzureSasCredential(sas_token))
Common uses of the Table service included:
The following components make up the Azure Tables Service:
The Azure Tables client library for Python allows you to interact with each of these components through the use of a dedicated client object.
Two different clients are provided to interact with the various components of the Table Service:
TableServiceClient
-
TableClient
to access a specific table using the get_table_client
method.TableClient
-
Entities are similar to rows. An entity has a PartitionKey
, a RowKey
, and a set of properties. A property is a name value pair, similar to a column. Every entity in a table does not need to have the same properties. Entities can be represented as dictionaries like this as an example:
entity = {
'PartitionKey': 'color',
'RowKey': 'brand',
'text': 'Marker',
'color': 'Purple',
'price': '5'
}
UpdateMode.MERGE
will add new properties to an existing entity it will not delete an existing propertiesUpdateMode.REPLACE
will replace the existing entity with the given one, deleting any existing properties not included in the submitted entityUpdateMode.MERGE
will add new properties to an existing entity it will not delete an existing propertiesUpdateMode.REPLACE
will replace the existing entity with the given one, deleting any existing properties not included in the submitted entityThe following sections provide several code snippets covering some of the most common Table tasks, including:
Create a table in your account and get a TableClient
to perform operations on the newly created table:
from azure.data.tables import TableServiceClient
table_service_client = TableServiceClient.from_connection_string(conn_str="<connection_string>")
table_name = "myTable"
table_client = table_service_client.create_table(table_name=table_name)
Create entities in the table:
from azure.data.tables import TableServiceClient
from datetime import datetime
PRODUCT_ID = u'001234'
PRODUCT_NAME = u'RedMarker'
my_entity = {
u'PartitionKey': PRODUCT_NAME,
u'RowKey': PRODUCT_ID,
u'Stock': 15,
u'Price': 9.99,
u'Comments': u"great product",
u'OnSale': True,
u'ReducedPrice': 7.99,
u'PurchaseDate': datetime(1973, 10, 4),
u'BinaryRepresentation': b'product_name'
}
table_service_client = TableServiceClient.from_connection_string(conn_str="<connection_string>")
table_client = table_service_client.get_table_client(table_name="myTable")
entity = table_client.create_entity(entity=my_entity)
Querying entities in the table:
from azure.data.tables import TableClient
my_filter = "PartitionKey eq 'RedMarker'"
table_client = TableClient.from_connection_string(conn_str="<connection_string>", table_name="myTable")
entities = table_client.query_entities(my_filter)
for entity in entities:
for key in entity.keys():
print("Key: {}, Value: {}".format(key, entity[key]))
Optional keyword arguments can be passed in at the client and per-operation level. The azure-core reference documentation describes available configurations for retries, logging, transport protocols, and more.
Use the following keyword arguments when instantiating a client to configure the retry policy:
retry_total=0
if you do not want to retry on requests. Defaults to 10.False
.Other optional configuration keyword arguments that can be specified on the client or per-operation.
Client keyword arguments:
Per-operation keyword arguments:
headers={'CustomValue': value}
Azure Tables clients raise exceptions defined in Azure Core.
When you interact with the Azure table library using the Python SDK, errors returned by the service respond ot the same HTTP status codes for REST API requests. The Table service operations will throw a HttpResponseError
on failure with helpful error codes.
For examples, if you try to create a table that already exists, a 409
error is returned indicating "Conflict".
from azure.data.tables import TableServiceClient
from azure.core.exceptions import HttpResponseError
table_name = 'YourTableName'
service_client = TableServiceClient.from_connection_string(connection_string)
# Create the table if it does not already exist
tc = service_client.create_table_if_not_exists(table_name)
try:
service_client.create_table(table_name)
except HttpResponseError:
print("Table with name {} already exists".format(table_name))
This library uses the standard logging library for logging. Basic information about HTTP sessions (URLs, headers, etc.) is logged at INFO level.
Detailed DEBUG level logging, including request/response bodies and unredacted
headers, can be enabled on a client with the logging_enable
argument:
import sys
import logging
from azure.data.tables import TableServiceClient
# Create a logger for the 'azure' SDK
logger = logging.getLogger('azure')
logger.setLevel(logging.DEBUG)
# Configure a console output
handler = logging.StreamHandler(stream=sys.stdout)
logger.addHandler(handler)
# This client will log detailed information about its HTTP sessions, at DEBUG level
service_client = TableServiceClient.from_connection_string("your_connection_string", logging_enable=True)
Similarly, logging_enable
can enable detailed logging for a single operation,
even when it is not enabled for the client:
service_client.create_entity(entity=my_entity, logging_enable=True)
Get started with our Table samples.
Several Azure Tables Python SDK samples are available to you in the SDK's GitHub repository. These samples provide example code for additional scenarios commonly encountered while working with Tables.
These code samples show common scenario operations with the Azure Tables client library. The async versions of the samples (the python sample files appended with _async) show asynchronous operations.
For more extensive documentation on Azure Tables, see the Azure Tables documentation on docs.microsoft.com.
A list of currently known issues relating to Cosmos DB table endpoints can be found here.
This project welcomes contributions and suggestions. Most contributions require you to agree to a Contributor License Agreement (CLA) declaring that you have the right to, and actually do, grant us the rights to use your contribution. For details, visit https://cla.microsoft.com.
When you submit a pull request, a CLA-bot will automatically determine whether you need to provide a CLA and decorate the PR appropriately (e.g., label, comment). Simply follow the instructions provided by the bot. You will only need to do this once across all repos using our CLA.
This project has adopted the Microsoft Open Source Code of Conduct. For more information see the Code of Conduct FAQ or contact opencode@microsoft.com with any additional questions or comments.
container
in account SAS access.isodate
to <1.0.0,>=0.6.1
.msrest
requirement.isodate
with version range >=0.6.0
(isodate
was required by msrest
).typing-extensions
with version range >=4.3.0
.azure-core
to >=1.24.0
.msrest
to >=0.7.1
.yarl
with version range <2.0,>=1.0
.query_entities
APIs (#23235)azure-core
to >=1.15.0
Warning This release involves a bug fix that may change the behaviour for some users. Partition and Row keys that contain a single quote character ('
) will now be automatically escaped for upsert, update and delete entity operations. Partition and Row keys that were already escaped, or contained duplicate single quote char (''
) will now be treated as unescaped values.
msrest
to >=0.6.21
TableClient
and TableServiceClient
s can now use azure-identity
credentials for authentication. Note: A TableClient
authenticated with a TokenCredential
cannot use the get_table_access_policy
or set_table_access_policy
methods.Breaking
bytes
in Python 3 and str
in Python 2, rather than an EdmProperty
instance. Likewise on serialization, bytes
in Python 3 and str
in Python 2 will be interpreted as binary (this is unchanged for Python 3, but breaking for Python 2, where str
was previously serialized as EdmType.String)TableClient.create_table
now returns an instance of TableItem
.Table
, including
TableAccessPolicy
, TableMetrics
, TableRetentionPolicy
, TableCorsRule
TableServiceClient.set_service_properties
are now keyword-only.credential
parameter for all Clients is now keyword-only.TableClient.get_access_policy
will now return None
where previously it returned an "empty" access policy object.TableAccessPolicy
instances returned from TableClient.get_access_policy
will now be deserialized to datetime
instances.Fixes
TableClient.from_table_url
classmethod.account_name
attribute on clients will now be pulled from an AzureNamedKeyCredential
if used.prefer
header is added in the create_entity
operation, the echo will be returned.azure.core.exceptions.ResourceModifiedError
.EdmType.DOUBLE
values are now explicitly typed in the request payload.TableCorsRule
.Breaking
account_url
parameter in the client constructors has been renamed to endpoint
.TableEntity
object now acts exclusively like a dictionary, and no longer supports key access via attributes.TableEntity.metadata
attribute rather than a method.LinearRetry
and ExponentialRetry
in favor of keyword parameter.filter
parameter in query APIs to query_filter
.location_mode
attribute on clients is now read-only. This has been added as a keyword parameter to the constructor.TableItem.table_name
has been renamed to TableItem.name
.TableClient.create_batch
method along with the TableBatchOperations
object. The transactional batching is now supported via a simple Python list of tuples.TableClient.send_batch
has been renamed to TableClient.submit_transaction
.BatchTransactionResult
object in favor of returning an iterable of batched entities with returned metadata.EntityProperty
is now a NampedTuple, and can be represented by a tuple of (entity, EdmType)
.EntityProperty.type
to EntityProperty.edm_type
.BatchErrorException
has been renamed to TableTransactionError
.location_mode
is no longer a public attribute on the Clients.AzureNamedKeyCredential
, AzureSasCredential
, or authentication by connection stringdate
and api_version
from the TableItem
class.Fixes
RequestTooLargeError
on transaction requests that return a 413 error codeselect
keyword parameter to TableClient.get_entity()
.update_entity
and delete_entity
if no etag
is supplied via kwargs, the etag
in the entity will be used if it is in the entity.azure-core
to 1.10.00.6.10
to 0.6.19
.query_entities
kwarg parameters
would not work with multiple parameters or with non-string parameters. This now works with multiple parameters and numeric, string, boolean, UUID, and datetime objects.delete_entity
will return a ClientAuthenticationError
when the '@' symbol is included in the entity.list_tables
and query_tables
where TableItem.table_name
was an object instead of a string.TypeError
.This is the first beta of the azure-data-tables
client library. The Azure Tables client library can seamlessly target either Azure Table storage or Azure Cosmos DB table service endpoints with no code changes.