Microsoft Azure Schema Registry Client Library for Python
Azure Schema Registry is a schema repository service hosted by Azure Event Hubs, providing schema storage, versioning, and management. The registry is leveraged by serializers to reduce payload size while describing payload structure with schema identifiers rather than full schemas.
Source code | Package (PyPi) | API reference documentation | Samples | Changelog
Azure SDK Python packages support for Python 2.7 has ended on 01 January 2022. For more information and questions, please refer to https://github.com/Azure/azure-sdk-for-python/issues/20691
Install the Azure Schema Registry client library for Python with pip:
pip install azure-schemaregistry
To use this package, you must have:
Interaction with Schema Registry starts with an instance of SchemaRegistryClient class. The client constructor takes the fully qualified namespace and an Azure Active Directory credential:
The fully qualified namespace of the Schema Registry instance should follow the format: <yournamespace>.servicebus.windows.net
.
An AAD credential that implements the TokenCredential protocol should be passed to the constructor. There are implementations of the TokenCredential
protocol available in the
azure-identity package. To use the credential types provided by azure-identity
, please install the Azure Identity client library for Python with pip:
pip install azure-identity
pip install aiohttp
Create client using the azure-identity library:
from azure.schemaregistry import SchemaRegistryClient
from azure.identity import DefaultAzureCredential
credential = DefaultAzureCredential()
# Namespace should be similar to: '<your-eventhub-namespace>.servicebus.windows.net/'
fully_qualified_namespace = '<< FULLY QUALIFIED NAMESPACE OF THE SCHEMA REGISTRY >>'
schema_registry_client = SchemaRegistryClient(fully_qualified_namespace, credential)
Schema: Schema is the organization or structure for data. More detailed information can be found here.
Schema Group: A logical group of similar schemas based on business criteria, which can hold multiple versions of a schema. More detailed information can be found here.
SchemaRegistryClient: SchemaRegistryClient
provides the API for storing and retrieving schemas in schema registry.
The following sections provide several code snippets covering some of the most common Schema Registry tasks, including:
Use SchemaRegistryClient.register_schema
method to register a schema.
import os
from azure.identity import DefaultAzureCredential
from azure.schemaregistry import SchemaRegistryClient
token_credential = DefaultAzureCredential()
fully_qualified_namespace = os.environ['SCHEMA_REGISTRY_FULLY_QUALIFIED_NAMESPACE']
group_name = os.environ['SCHEMA_REGISTRY_GROUP']
name = "your-schema-name"
format = "Avro"
definition = """
{"namespace": "example.avro",
"type": "record",
"name": "User",
"fields": [
{"name": "name", "type": "string"},
{"name": "favorite_number", "type": ["int", "null"]},
{"name": "favorite_color", "type": ["string", "null"]}
]
}
"""
schema_registry_client = SchemaRegistryClient(fully_qualified_namespace=fully_qualified_namespace, credential=token_credential)
with schema_registry_client:
schema_properties = schema_registry_client.register_schema(group_name, name, definition, format)
id = schema_properties.id
Get the schema definition and its properties by schema id.
import os
from azure.identity import DefaultAzureCredential
from azure.schemaregistry import SchemaRegistryClient
token_credential = DefaultAzureCredential()
fully_qualified_namespace = os.environ['SCHEMA_REGISTRY_FULLY_QUALIFIED_NAMESPACE']
schema_id = 'your-schema-id'
schema_registry_client = SchemaRegistryClient(fully_qualified_namespace=fully_qualified_namespace, credential=token_credential)
with schema_registry_client:
schema = schema_registry_client.get_schema(schema_id)
definition = schema.definition
properties = schema.properties
Get the schema definition and its properties by schema version.
import os
from azure.identity import DefaultAzureCredential
from azure.schemaregistry import SchemaRegistryClient
token_credential = DefaultAzureCredential()
fully_qualified_namespace = os.environ['SCHEMA_REGISTRY_FULLY_QUALIFIED_NAMESPACE']
group_name = os.environ["SCHEMAREGISTRY_GROUP"]
name = "your-schema-name"
version = int("<your schema version>")
schema_registry_client = SchemaRegistryClient(fully_qualified_namespace=fully_qualified_namespace, credential=token_credential)
with schema_registry_client:
schema = schema_registry_client.get_schema(group_name=group_name, name=name, version=version)
definition = schema.definition
properties = schema.properties
Get the schema id of a schema by schema definition and its properties.
import os
from azure.identity import DefaultAzureCredential
from azure.schemaregistry import SchemaRegistryClient
token_credential = DefaultAzureCredential()
fully_qualified_namespace = os.environ['SCHEMA_REGISTRY_FULLY_QUALIFIED_NAMESPACE']
group_name = os.environ['SCHEMA_REGISTRY_GROUP']
name = "your-schema-name"
format = "Avro"
definition = """
{"namespace": "example.avro",
"type": "record",
"name": "User",
"fields": [
{"name": "name", "type": "string"},
{"name": "favorite_number", "type": ["int", "null"]},
{"name": "favorite_color", "type": ["string", "null"]}
]
}
"""
schema_registry_client = SchemaRegistryClient(fully_qualified_namespace=fully_qualified_namespace, credential=token_credential)
with schema_registry_client:
schema_properties = schema_registry_client.register_schema(group_name, name, definition, format)
id = schema_properties.id
Schema Registry clients raise exceptions defined in Azure Core.
This library uses the standard logging library for logging. Basic information about HTTP sessions (URLs, headers, etc.) is logged at INFO level.
Detailed DEBUG level logging, including request/response bodies and unredacted
headers, can be enabled on a client with the logging_enable
argument:
import sys
import logging
from azure.schemaregistry import SchemaRegistryClient
from azure.identity import DefaultAzureCredential
# Create a logger for the SDK
logger = logging.getLogger('azure.schemaregistry')
logger.setLevel(logging.DEBUG)
# Configure a console output
handler = logging.StreamHandler(stream=sys.stdout)
logger.addHandler(handler)
credential = DefaultAzureCredential()
# This client will log detailed information about its HTTP sessions, at DEBUG level
schema_registry_client = SchemaRegistryClient("your_fully_qualified_namespace", credential, logging_enable=True)
Similarly, logging_enable
can enable detailed logging for a single operation,
even when it isn't enabled for the client:
schema_registry_client.get_schema(schema_id, logging_enable=True)
Please take a look at the samples directory for detailed examples of how to use this library to register and retrieve schema to/from Schema Registry.
This project welcomes contributions and suggestions. Most contributions require you to agree to a Contributor License Agreement (CLA) declaring that you have the right to, and actually do, grant us the rights to use your contribution. For details, visit https://cla.microsoft.com.
When you submit a pull request, a CLA-bot will automatically determine whether you need to provide a CLA and decorate the PR appropriately (e.g., label, comment). Simply follow the instructions provided by the bot. You will only need to do this once across all repos using our CLA.
This project has adopted the Microsoft Open Source Code of Conduct. For more information see the Code of Conduct FAQ or contact opencode@microsoft.com with any additional questions or comments.
This version and all future versions will require Python 3.7+, Python 3.6 is no longer supported.
group_name
, name
, and version
have been added as optional parameters to the get_schema
method on the sync and async SchemaRegistryClient
.version
has been added to SchemaProperties
.SchemaRegistryClient
.This version and all future versions will require Python 3.6+. Python 2.7 is no longer supported.
group_name
and name
have been added as instance variables to SchemaProperties
.Note: This is the first stable release of our efforts to create a user-friendly and Pythonic client library for Azure Schema Registry.
SchemaRegistryClient
is the top-level client class interacting with the Azure Schema Registry Service. It provides three methods:
register_schema
: Store schema in the service by providing schema group name, schema name, schema definition, and schema format.get_schema
: Get schema definition and its properties by schema id.get_schema_properties
: Get schema properties by providing schema group name, schema name, schema definition, and schema format.SchemaProperties
has the following instance variables: id
and format
:
format
has been changed from str
to SchemaFormat
.Schema
has the following properties: properties
and definition
.SchemaFormat
provides the schema format to be stored by the service. Currently, the only supported format is Avro
.api_version
has been added as a keyword arg to the sync and async SchemaRegistryClient
constructors.version
instance variable in SchemaProperties
has been removed.schema_definition
instance variable in Schema
has been renamed definition
.id
parameter in get_schema
method on sync and async SchemaRegistryClient
has been renamed schema_id
.schema_definition
parameter in register_schema
and get_schema_properties
methods on sync and async SchemaRegistryClient
has been renamed definition
.serializer
namespace has been removed from azure.schemaregistry
.get_schema_id
method on sync and async SchemaRegistryClient
has been renamed get_schema_properties
.schema_id
parameter in get_schema
method on sync and async SchemaRegistryClient
has been renamed id
.register_schema
and get_schema_properties
methods on sync and async SchemaRegistryClient
now take in the following parameters in the given order:
group_name
, which has been renamed from schema_group
name
, which has been renamed from schema_name
schema_definition
, which has been renamed from schema_content
format
, which has been renamed from serialization_type
endpoint
parameter in SchemaRegistryClient
constructor has been renamed fully_qualified_namespace
location
instance variable in SchemaProperties
has been removed.Schema
and SchemaProperties
no longer have positional parameters, as they will not be constructed by the user.This version and all future versions will require Python 2.7 or Python 3.6+, Python 3.5 is no longer supported.
Version 1.0.0b1 is the first preview of our efforts to create a user-friendly and Pythonic client library for Azure Schema Registry.
New features
SchemaRegistryClient
is the top-level client class interacting with the Azure Schema Registry Service. It provides three methods:
register_schema
: Store schema into the service.get_schema
: Get schema content and its properties by schema id.get_schema_id
: Get schema id and its properties by schema group, schema name, serialization type and schema content.