Project: azureml-fsspec

Access datastore uri with fsspec

Project Details

Latest version
1.2.0
Home Page
https://docs.microsoft.com/python/api/overview/azure/ml/?view=azure-ml-py
PyPI Page
https://pypi.org/project/azureml-fsspec/

Project Popularity

PageRank
0.00229081606617341
Number of downloads
72378

Filesystem interface to Azure Machine Learning defined URIs

Getting started

This package can be installed using:

pip install azureml-fsspec

Examples

Accepted uri format is Azure Machcine Learning defined datastore uri: azureml://subscriptions/([^/]+)/resourcegroups/([^/]+)/workspaces/([^/]+)/datastores/([^/]+)/paths/([^/]+)

# load parquet file to pandas
import pandas
df = pandas.read_parquet('azureml://subscriptions/{sub_id}/resourcegroups/{rs_group}/workspaces/{ws}
                          /datastores/workspaceblobstore/paths/myfolder/mydata.parquet')

# load csv file to pandas
import pandas
df = pandas.read_csv('azureml://subscriptions/{sub_id}/resourcegroups/{rs_group}/workspaces/{ws}
                          /datastores/workspaceblobstore/paths/myfolder/mydata.csv')

# load parquet file to dask
import dask.dataframe as dd
df = dd.read_parquet('azureml://subscriptions/{sub_id}/resourcegroups/{rs_group}/workspaces/{ws}
                          /datastores/workspaceblobstore/paths/myfolder/mydata.parquet')

# load csv file to dask
import dask.dataframe as dd
df = dd.read_csv('azureml://subscriptions/{sub_id}/resourcegroups/{rs_group}/workspaces/{ws}
                          /datastores/workspaceblobstore/paths/myfolder/mydata.csv')

Release History

1.2.0 (08-14-2023)

Features Added

  • added auth dict support (SP and managedIdentity) for fsspec.
  • added user exception mapping

1.1.0 (06-01-2023)

Features Added

  • upload/download performance improvement

1.0.0 (04-07-2023)

Features Added

  • Added upload/download API
  • bugfix(root behavior)

0.1.0b1

Features Added

  • Initial public preview release of FileSystem for Azure Machine Learning defined datastore uri

0.1.0b2

Features Added

  • Fix release notes

0.1.0b3

Features Added

  • Add mode in open API
  • Add validation
  • Upgrade dataprep to 4.8