A lightweight library to help with training neural networks in PyTorch.
Ignite is a high-level library to help with training and evaluating neural networks in PyTorch flexibly and transparently.
Less code than pure PyTorch while ensuring maximum control and simplicity
Library approach and no program's control inversion - Use ignite where and when you need
Extensible API for metrics, experiment managers, and other components
Ignite is a library that provides three high-level features:
No more coding for/while
loops on epochs and iterations. Users instantiate engines and run them.
from ignite.engine import Engine, Events, create_supervised_evaluator
from ignite.metrics import Accuracy
# Setup training engine:
def train_step(engine, batch):
# Users can do whatever they need on a single iteration
# Eg. forward/backward pass for any number of models, optimizers, etc
# ...
trainer = Engine(train_step)
# Setup single model evaluation engine
evaluator = create_supervised_evaluator(model, metrics={"accuracy": Accuracy()})
def validation():
state = evaluator.run(validation_data_loader)
# print computed metrics
print(trainer.state.epoch, state.metrics)
# Run model's validation at the end of each epoch
trainer.add_event_handler(Events.EPOCH_COMPLETED, validation)
# Start the training
trainer.run(training_data_loader, max_epochs=100)
The cool thing with handlers is that they offer unparalleled flexibility (compared to, for example, callbacks). Handlers can be any function: e.g. lambda, simple function, class method, etc. Thus, we do not require to inherit from an interface and override its abstract methods which could unnecessarily bulk up your code and its complexity.
trainer.add_event_handler(Events.STARTED, lambda _: print("Start training"))
# attach handler with args, kwargs
mydata = [1, 2, 3, 4]
logger = ...
def on_training_ended(data):
print(f"Training is ended. mydata={data}")
# User can use variables from another scope
logger.info("Training is ended")
trainer.add_event_handler(Events.COMPLETED, on_training_ended, mydata)
# call any number of functions on a single event
trainer.add_event_handler(Events.COMPLETED, lambda engine: print(engine.state.times))
@trainer.on(Events.ITERATION_COMPLETED)
def log_something(engine):
print(engine.state.output)
# run the validation every 5 epochs
@trainer.on(Events.EPOCH_COMPLETED(every=5))
def run_validation():
# run validation
# change some training variable once on 20th epoch
@trainer.on(Events.EPOCH_STARTED(once=20))
def change_training_variable():
# ...
# Trigger handler with customly defined frequency
@trainer.on(Events.ITERATION_COMPLETED(event_filter=first_x_iters))
def log_gradients():
# ...
Events can be stacked together to enable multiple calls:
@trainer.on(Events.COMPLETED | Events.EPOCH_COMPLETED(every=10))
def run_validation():
# ...
Custom events related to backward and optimizer step calls:
from ignite.engine import EventEnum
class BackpropEvents(EventEnum):
BACKWARD_STARTED = 'backward_started'
BACKWARD_COMPLETED = 'backward_completed'
OPTIM_STEP_COMPLETED = 'optim_step_completed'
def update(engine, batch):
# ...
loss = criterion(y_pred, y)
engine.fire_event(BackpropEvents.BACKWARD_STARTED)
loss.backward()
engine.fire_event(BackpropEvents.BACKWARD_COMPLETED)
optimizer.step()
engine.fire_event(BackpropEvents.OPTIM_STEP_COMPLETED)
# ...
trainer = Engine(update)
trainer.register_events(*BackpropEvents)
@trainer.on(BackpropEvents.BACKWARD_STARTED)
def function_before_backprop(engine):
# ...
Metrics for various tasks: Precision, Recall, Accuracy, Confusion Matrix, IoU etc, ~20 regression metrics.
Users can also compose their metrics with ease from existing ones using arithmetic operations or torch methods.
precision = Precision(average=False)
recall = Recall(average=False)
F1_per_class = (precision * recall * 2 / (precision + recall))
F1_mean = F1_per_class.mean() # torch mean method
F1_mean.attach(engine, "F1")
From pip:
pip install pytorch-ignite
From conda:
conda install ignite -c pytorch
From source:
pip install git+https://github.com/pytorch/ignite
From pip:
pip install --pre pytorch-ignite
From conda (this suggests to install pytorch nightly release instead of stable version as dependency):
conda install ignite -c pytorch-nightly
Pull a pre-built docker image from our Docker Hub and run it with docker v19.03+.
docker run --gpus all -it -v $PWD:/workspace/project --network=host --shm-size 16G pytorchignite/base:latest /bin/bash
Base
pytorchignite/base:latest
pytorchignite/apex:latest
pytorchignite/hvd-base:latest
pytorchignite/hvd-apex:latest
pytorchignite/msdp-apex:latest
Vision:
pytorchignite/vision:latest
pytorchignite/hvd-vision:latest
pytorchignite/apex-vision:latest
pytorchignite/hvd-apex-vision:latest
pytorchignite/msdp-apex-vision:latest
NLP:
pytorchignite/nlp:latest
pytorchignite/hvd-nlp:latest
pytorchignite/apex-nlp:latest
pytorchignite/hvd-apex-nlp:latest
pytorchignite/msdp-apex-nlp:latest
For more details, see here.
Few pointers to get you started:
Inspired by torchvision/references, we provide several reproducible baselines for vision tasks:
Features:
The easiest way to create your training scripts with PyTorch-Ignite:
GitHub issues: questions, bug reports, feature requests, etc.
Discuss.PyTorch, category "Ignite".
PyTorch-Ignite Discord Server: to chat with the community
GitHub Discussions: general library-related discussions, ideas, Q&A, etc.
We have created a form for "user feedback". We appreciate any type of feedback, and this is how we would like to see our community:
Thank you!
Please see the contribution guidelines for more information.
As always, PRs are welcome :)
See other projects at "Used by"
If your project implements a paper, represents other use-cases not covered in our official tutorials, Kaggle competition's code, or just your code presents interesting results and uses Ignite. We would like to add your project to this list, so please send a PR with brief description of the project.
If you use PyTorch-Ignite in a scientific publication, we would appreciate citations to our project.
@misc{pytorch-ignite,
author = {V. Fomin and J. Anmol and S. Desroziers and J. Kriss and A. Tejani},
title = {High-level library to help with training neural networks in PyTorch},
year = {2020},
publisher = {GitHub},
journal = {GitHub repository},
howpublished = {\url{https://github.com/pytorch/ignite}},
}
PyTorch-Ignite is a NumFOCUS Affiliated Project, operated and maintained by volunteers in the PyTorch community in their capacities as individuals (and not as representatives of their employers). See the "About us" page for a list of core contributors. For usage questions and issues, please see the various channels here. For all other questions and inquiries, please send an email to contact@pytorch-ignite.ai.