Extending PyYAML with a custom constructor for including YAML files within YAML files
An extending constructor of PyYAML: include other YAML files into YAML document.
pip install pyyaml-include
Consider we have such YAML files:
├── 0.yml
└── include.d
├── 1.yml
└── 2.yml
1.yml
's content:
name: "1"
2.yml
's content:
name: "2"
To include 1.yml
, 2.yml
in 0.yml
, we shall add YamlIncludeConstructor
to PyYAML's loader, then add an !include
tag in 0.yaml
:
import yaml
from yamlinclude import YamlIncludeConstructor
YamlIncludeConstructor.add_to_loader_class(loader_class=yaml.FullLoader, base_dir='/your/conf/dir')
with open('0.yml') as f:
data = yaml.load(f, Loader=yaml.FullLoader)
print(data)
If 0.yml
was:
file1: !include include.d/1.yml
file2: !include include.d/2.yml
We'll get:
file1:
name: "1"
file2:
name: "2"
If 0.yml
was:
files:
- !include include.d/1.yml
- !include include.d/2.yml
We'll get:
files:
- name: "1"
- name: "2"
ℹ Note:
File name can be either absolute (like
/usr/conf/1.5/Make.yml
) or relative (like../../cfg/img.yml
).
File name can contain shell-style wildcards. Data loaded from the file(s) found by wildcards will be set in a sequence.
That is to say, a list will be returned when including file name contains wildcards. Length of the returned list equals number of matched files:
If 0.yml
was:
files: !include include.d/*.yml
We'll get:
files:
- name: "1"
- name: "2"
ℹ Note:
- For
Python>=3.5
, ifrecursive
argument of!include
YAML tag istrue
, the pattern“**”
will match any files and zero or more directories and subdirectories.- Using the
“**”
pattern in large directory trees may consume an inordinate amount of time because of recursive search.
In order to enable recursive
argument, we shall set it in Mapping
or Sequence
arguments mode:
Arguments in Sequence
mode:
!include [tests/data/include.d/**/*.yml, true]
Arguments in Mapping
mode:
!include {pathname: tests/data/include.d/**/*.yml, recursive: true}
This extending constructor can now load data from non YAML files, supported file types are:
json
toml
(only available when toml installed)ini
The constructor read non YAML files by different readers according to a pattern table defined in src/yamlinclude/readers.py
.
Default reader table can be replaced by a custom reader_map
when call add_to_loader_class
.