ClearML - Auto-Magical Experiment Manager, Version Control, and MLOps for AI
ClearML - Auto-Magical Suite of tools to streamline your ML workflow Experiment Manager, MLOps and Data-Management
Formerly known as Allegro Trains
ClearML is a ML/DL development and production suite. It contains FIVE main modules:
Instrumenting these components is the ClearML-server, see Self-Hosting & Free tier Hosting
Sign up & Start using in under 2 minutes
Friendly tutorials to get you started
Step 1 - Experiment Management | |
Step 2 - Remote Execution Agent Setup | |
Step 3 - Remotely Execute Tasks |
Adding only 2 lines to your code gets you the following
argparse
/Click/PythonFire for command line parameters with currently used valuesSign up for free to the ClearML Hosted Service (alternatively, you can set up your own server, see here).
ClearML Demo Server: ClearML no longer uses the demo server by default. To enable the demo server, set the
CLEARML_NO_DEFAULT_SERVER=0
environment variable. Credentials aren't needed, but experiments launched to the demo server are public, so make sure not to launch sensitive experiments if using the demo server.
Install the clearml
python package:
pip install clearml
Connect the ClearML SDK to the server by creating credentials, then execute the command below and follow the instructions:
clearml-init
Add two lines to your code:
from clearml import Task
task = Task.init(project_name='examples', task_name='hello world')
And you are done! Everything your process outputs is now automagically logged into ClearML.
Next step, automation! Learn more about ClearML's two-click automation here.
The ClearML run-time components:
ClearML is our solution to a problem we share with countless other researchers and developers in the machine learning/deep learning universe: Training production-grade deep learning models is a glorious but messy process. ClearML tracks and controls the process by associating code version control, research projects, performance metrics, and model provenance.
We designed ClearML specifically to require effortless integration so that teams can preserve their existing methods and practices.
We believe ClearML is ground-breaking. We wish to establish new standards of true seamless integration between experiment management, MLOps, and data management.
ClearML is supported by you and the clear.ml team, which helps enterprise companies build scalable MLOps.
We built ClearML to track and control the glorious but messy process of training production-grade deep learning models. We are committed to vigorously supporting and expanding the capabilities of ClearML.
We promise to always be backwardly compatible, making sure all your logs, data, and pipelines will always upgrade with you.
Apache License, Version 2.0 (see the LICENSE for more information)
If ClearML is part of your development process / project / publication, please cite us :heart: :
@misc{clearml,
title = {ClearML - Your entire MLOps stack in one open-source tool},
year = {2023},
note = {Software available from http://github.com/allegroai/clearml},
url={https://clear.ml/},
author = {ClearML},
}
For more information, see the official documentation and on YouTube.
For examples and use cases, check the examples folder and corresponding documentation.
If you have any questions: post on our Slack Channel, or tag your questions on stackoverflow with 'clearml' tag (previously trains tag).
For feature requests or bug reports, please use GitHub issues.
Additionally, you can always find us at info@clear.ml
PRs are always welcome :heart: See more details in the ClearML Guidelines for Contributing.
May the force (and the goddess of learning rates) be with you!