Microsoft Azure Cognitive Services Computer Vision Client Library for Python
The Computer Vision service provides developers with access to advanced algorithms for processing images and returning information. Computer Vision algorithms analyze the content of an image in different ways, depending on the visual features you're interested in.
You can use Computer Vision in your application to:
Looking for more documentation?
If you need a Computer Vision API account, you can create one with this Azure CLI command:
RES_REGION=westeurope
RES_GROUP=<resourcegroup-name>
ACCT_NAME=<computervision-account-name>
az cognitiveservices account create \
--resource-group $RES_GROUP \
--name $ACCT_NAME \
--location $RES_REGION \
--kind ComputerVision \
--sku S1 \
--yes
Install the Azure Cognitive Services Computer Vision SDK with pip, optionally within a virtual environment.
Although not required, you can keep your base system and Azure SDK environments isolated from one another if you use a virtual environment. Execute the following commands to configure and then enter a virtual environment with venv, such as cogsrv-vision-env:
python3 -m venv cogsrv-vision-env
source cogsrv-vision-env/bin/activate
Install the Azure Cognitive Services Computer Vision SDK for Python package with pip:
pip install azure-cognitiveservices-vision-computervision
Once you create your Computer Vision resource, you need its region, and one of its account keys to instantiate the client object.
Use these values when you create the instance of the ComputerVisionClient client object.
Use the Azure CLI snippet below to populate two environment variables with the Computer Vision account region and one of its keys (you can also find these values in the Azure portal). The snippet is formatted for the Bash shell.
RES_GROUP=<resourcegroup-name>
ACCT_NAME=<computervision-account-name>
export ACCOUNT_REGION=$(az cognitiveservices account show \
--resource-group $RES_GROUP \
--name $ACCT_NAME \
--query location \
--output tsv)
export ACCOUNT_KEY=$(az cognitiveservices account keys list \
--resource-group $RES_GROUP \
--name $ACCT_NAME \
--query key1 \
--output tsv)
Once you've populated the ACCOUNT_REGION and ACCOUNT_KEY environment variables, you can create the ComputerVisionClient client object.
from azure.cognitiveservices.vision.computervision import ComputerVisionClient
from azure.cognitiveservices.vision.computervision.models import VisualFeatureTypes
from msrest.authentication import CognitiveServicesCredentials
import os
region = os.environ['ACCOUNT_REGION']
key = os.environ['ACCOUNT_KEY']
credentials = CognitiveServicesCredentials(key)
client = ComputerVisionClient(
endpoint="https://" + region + ".api.cognitive.microsoft.com/",
credentials=credentials
)
Once you've initialized a ComputerVisionClient client object, you can:
For more information about this service, see What is Computer Vision?.
The following sections provide several code snippets covering some of the most common Computer Vision tasks, including:
You can analyze an image for certain features with analyze_image. Use the visual_features property to set the types of analysis to perform on the image. Common values are VisualFeatureTypes.tags and VisualFeatureTypes.description.
url = "https://upload.wikimedia.org/wikipedia/commons/thumb/1/12/Broadway_and_Times_Square_by_night.jpg/450px-Broadway_and_Times_Square_by_night.jpg"
image_analysis = client.analyze_image(url,visual_features=[VisualFeatureTypes.tags])
for tag in image_analysis.tags:
print(tag)
Review the subject domains used to analyze your image with list_models. These domain names are used when analyzing an image by domain. An example of a domain is landmarks.
models = client.list_models()
for x in models.models_property:
print(x)
You can analyze an image by subject domain with analyze_image_by_domain. Get the list of supported subject domains in order to use the correct domain name.
domain = "landmarks"
url = "https://images.pexels.com/photos/338515/pexels-photo-338515.jpeg"
language = "en"
analysis = client.analyze_image_by_domain(domain, url, language)
for landmark in analysis.result["landmarks"]:
print(landmark["name"])
print(landmark["confidence"])
You can get a language-based text description of an image with describe_image. Request several descriptions with the max_description property if you are doing text analysis for keywords associated with the image. Examples of a text description for the following image include a train crossing a bridge over a body of water, a large bridge over a body of water, and a train crossing a bridge over a large body of water.
domain = "landmarks"
url = "http://www.public-domain-photos.com/free-stock-photos-4/travel/san-francisco/golden-gate-bridge-in-san-francisco.jpg"
language = "en"
max_descriptions = 3
analysis = client.describe_image(url, max_descriptions, language)
for caption in analysis.captions:
print(caption.text)
print(caption.confidence)
You can get any handwritten or printed text from an image. This requires two calls to the SDK: read and get_read_result. The call to read is asynchronous. In the results of the get_read_result call, you need to check if the first call completed with OperationStatusCodes before extracting the text data. The results include the text as well as the bounding box coordinates for the text.
# import models
from azure.cognitiveservices.vision.computervision.models import OperationStatusCodes
url = "https://github.com/Azure-Samples/cognitive-services-python-sdk-samples/raw/master/samples/vision/images/make_things_happen.jpg"
raw = True
numberOfCharsInOperationId = 36
# SDK call
rawHttpResponse = client.read(url, language="en", raw=True)
# Get ID from returned headers
operationLocation = rawHttpResponse.headers["Operation-Location"]
idLocation = len(operationLocation) - numberOfCharsInOperationId
operationId = operationLocation[idLocation:]
# SDK call
result = client.get_read_result(operationId)
# Get data
if result.status == OperationStatusCodes.succeeded:
for line in result.analyze_result.read_results[0].lines:
print(line.text)
print(line.bounding_box)
You can generate a thumbnail (JPG) of an image with generate_thumbnail. The thumbnail does not need to be in the same proportions as the original image.
This example uses the Pillow package to save the new thumbnail image locally.
from PIL import Image
import io
width = 50
height = 50
url = "http://www.public-domain-photos.com/free-stock-photos-4/travel/san-francisco/golden-gate-bridge-in-san-francisco.jpg"
thumbnail = client.generate_thumbnail(width, height, url)
for x in thumbnail:
image = Image.open(io.BytesIO(x))
image.save('thumbnail.jpg')
When you interact with the ComputerVisionClient client object using the Python SDK, the ComputerVisionErrorResponseException class is used to return errors. Errors returned by the service correspond to the same HTTP status codes returned for REST API requests.
For example, if you try to analyze an image with an invalid key, a 401 error is returned. In the following snippet, the error is handled gracefully by catching the exception and displaying additional information about the error.
domain = "landmarks"
url = "http://www.public-domain-photos.com/free-stock-photos-4/travel/san-francisco/golden-gate-bridge-in-san-francisco.jpg"
language = "en"
max_descriptions = 3
try:
analysis = client.describe_image(url, max_descriptions, language)
for caption in analysis.captions:
print(caption.text)
print(caption.confidence)
except HTTPFailure as e:
if e.status_code == 401:
print("Error unauthorized. Make sure your key and region are correct.")
else:
raise
While working with the ComputerVisionClient client, you might encounter transient failures caused by rate limits enforced by the service, or other transient problems like network outages. For information about handling these types of failures, see Retry pattern in the Cloud Design Patterns guide, and the related Circuit Breaker pattern.
Several Computer Vision Python SDK samples are available to you in the SDK's GitHub repository. These samples provide example code for additional scenarios commonly encountered while working with Computer Vision:
For more extensive documentation on the Computer Vision service, see the Azure Computer Vision documentation on docs.microsoft.com.
Breaking changes
Features
Breaking changes
Features
Features
Breaking changes
Features
Breaking changes
Breaking changes
Bugfix
Features
Breaking changes
Features
Bug fixes
Breaking changes
General Breaking changes
This version uses a next-generation code generator that might introduce breaking changes.
NameOfEnum.stringvalue. Format syntax
should be prefered.msrestazure.azure_operation.AzureOperationPoller to
msrest.polling.LROPoller. External API is the same.msrest.polling.LROPoller,
regardless of the optional parameters used.raw=True. Instead of
returning the initial call result as ClientRawResponse,
without polling, now this returns an LROPoller. After polling,
the final resource will be returned as a ClientRawResponse.polling parameter. The default behavior is
Polling=True which will poll using ARM algorithm. When
Polling=False, the response of the initial call will be
returned without polling.polling parameter accepts instances of subclasses of
msrest.polling.PollingMethod.add_done_callback will no longer raise if called after
polling is finished, but will instead execute the callback right
away.