0

One of my packages (tinject), currently using ruamel.yaml 0.17.x, installs a custom constructor to implement a !include directive allowing me to write a document like:

foo:
  - bar: 42

---

bar:
  - foo: 42
  - text: !include 'test.txt'

When I run the following simplified script:

from pathlib import Path
from pprint import pprint

from ruamel import yaml


def include(loader, node):
    path = loader.construct_scalar(node)
    fullpath = include.basedir / path
    return fullpath.read_text('utf-8')


yaml.add_constructor('!include', include, Loader=yaml.Loader)


def load(fname):
    include.basedir = fname.parent
    with fname.open() as stream:
        content = yaml.load_all(stream, Loader=yaml.Loader)
        pprint(list(content))


print("ruamel.yaml.__version__ ==", yaml.__version__)
load(Path('/tmp/ruamel-test/test.yml'))

with a test.txt file containing

This
is
the
text

I obtain the following output:

ruamel.yaml.__version__ == 0.17.32
[{'foo': [{'bar': 42}]},
 {'bar': [{'foo': 42}, {'text': 'This\nis\nthe\ntext\n'}]}]

Now I have the need to upgrade it to 0.18.x, where the top level load_all() functions have been dropped, and indeed running under that version I get the following instead:

ruamel.yaml.__version__ == 0.18.6
Traceback (most recent call last):
  File "/tmp/ruamel-test/test.py", line 24, in <module>
    load(Path('/tmp//ruamel-test/test.yml'))
  File "/tmp/ruamel-test/test.py", line 19, in load
    content = yaml.load_all(stream, Loader=yaml.Loader)
              ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/nix/store/8zvc8cgw8bpi1i3c0zpd4xhjcg6f9shr-python3-3.11.9-env/lib/python3.11/site-packages/ruamel/yaml/main.py", line 1096, in load_all
    error_deprecation('load_all', 'load_all', arg=_error_dep_arg, comment=_error_dep_comment)
  File "/nix/store/8zvc8cgw8bpi1i3c0zpd4xhjcg6f9shr-python3-3.11.9-env/lib/python3.11/site-packages/ruamel/yaml/main.py", line 1039, in error_deprecation
    raise AttributeError(s, name=None)
AttributeError: 
"load_all()" has been removed, use

  yaml = YAML(typ='rt')
  yaml.load_all(...)

and register any classes that you use, or check the tag attribute on the loaded data,
instead of file "/tmp/ruamel-test/test.py", line 19

        content = yaml.load_all(stream, Loader=yaml.Loader)

After several attempts, I eventually found a solution that apparently gives me the expected result, and I'm asking if what I'm doing is the right thing or if instead I'm missing some (even) simpler way.

At first, I modified my script like the following:

from pathlib import Path
from pprint import pprint

from ruamel import yaml


def include(loader, node):
    path = loader.construct_scalar(node)
    fullpath = include.basedir / path
    return fullpath.read_text('utf-8')


yaml.add_constructor('!include', include)


def load(fname):
    include.basedir = fname.parent
    with fname.open() as stream:
        content = yaml.YAML().load_all(stream)
        pprint(list(content))


print("ruamel.yaml.__version__ ==", yaml.__version__)
load(Path('/tmp//ruamel-test/test.yml'))

and this emits this:

ruamel.yaml.__version__ == 0.18.6
[{'foo': [{'bar': 42}]},
 {'bar': [{'foo': 42}, {'text': TaggedScalar(value='test.txt', style="'", tag=Tag('!include'))}]}]

where the include() constructor is never called.


What eventually worked is something like this:

from pathlib import Path
from pprint import pprint

from ruamel import yaml


class MyConstructor(yaml.Constructor):
    pass


def include(loader, node):
    path = loader.construct_scalar(node)
    fullpath = include.basedir / path
    return fullpath.read_text('utf-8')


MyConstructor.add_constructor('!include', include)


def load(fname):
    include.basedir = fname.parent
    with fname.open() as stream:
        y = yaml.YAML()
        y.Constructor = MyConstructor
        content = y.load_all(stream)
        pprint(list(content))


print("ruamel.yaml.__version__ ==", yaml.__version__)
load(Path('/tmp//ruamel-test/test.yml'))

that produces:

ruamel.yaml.__version__ == 0.18.6
[{'foo': [{'bar': 42}]},
 {'bar': [{'foo': 42}, {'text': 'This\nis\nthe\ntext\n'}]}]

Is this (that is, creating my own Constructor subclass and explicitly adding the constructor there) the recommended way now?

Thanks in advance & bye.

2
  • Yes this is the proper way to go. With the deprecated function load_all() you would use instances that where globally modified by add_constructor chaning the class variables. With the creation of the YAML() instance, I wanted to move away from these global changes, so you can have different instance doing different things (e.g. dumping to different widths, having different tags registered etc).
    – Anthon
    Commented Jul 10 at 10:43
  • Thank you, maybe this deserves an example/explanation in the doc? Commented Jul 10 at 11:27

0

Browse other questions tagged or ask your own question.