Get package version for conda meta.yaml from source file

Question

I'm trying to reorganize my python package versioning so I only have to update the version in one place, preferably a python module or a text file. For all the places I need my version there seems to be a way to load it from the source from mypkg import __version__ or at least parse it out of the file as text. I can't seem to find a way to do it with my conda meta.yaml file though. Is there a way to load the version from an external source in the meta.yaml file?

I know there are the git environment variables, but I don't want to tag every alpha/beta/rc commit that gets tested through out local conda repository. I could load the python object using !!python/object in pyyaml, but conda doesn't support arbitrary python execution. I don't see a way to do it with any other jinja2 features. I could also write a script to update the version number in more than one place, but I was really hoping to only modify one file as the definitive version number. Thanks for any help.

Answer 1

As of conda-build-3.16.1 (Nov-2018) here is what works to programmatically setup version inside the conda recipe.

The examples are a part of meta.yaml that you pass to conda-build, as explained here.

A. Tap into setup.py's version:

This recipe is perfect if you build a python package, since setup.py needs it anyway, so you must have figured that one out already.

{% set data = load_setup_py_data() %}

package:
  name: mypackage
  version: {{ data.get('version') }}

note that sometimes you have to tell the conda recipe explicitly where to find it, if it's not in the same dir as setup.py:

{% set data = load_setup_py_data(setup_file='../setup.py', from_recipe_dir=True) %}

and now proceed with:

$ conda-build conda-recipe

---

B. Git env variables

This recipe is good if your project is tagged in git, and you use a tag format that conda accepts as a valid version number (e.g. 2.5.1 or v2.5.1).

package:
  name: hub
  version: {{ GIT_DESCRIBE_TAG }}

and now proceed with:

$ conda-build conda-recipe

---

C. Pass env variable:

This one is useful for non-python conda packages, where the version comes from a variety of different places, and you can perfect its value - e.g. convert v2.5.1 into 2.5.1.

package:
  name: mypkg
  version: {{ environ.get('MYPKG_VERSION', '') }}

Then create an executable script that fetches the version, let's call it script-to-get-mypkg-version

and now proceed with loading the env var that will set the version:

$ MYPKG_VERSION=`script-to-get-mypkg-version` conda-build conda-recipe

Depending on the conda-build version, you may have to use os.environ.get instead of environ.get. The docs use the latter.

---

This doesn't work

Note that if this used to work in the past, as described in one of the answers from 2016, it doesn't work now.

package:
  name: mypkg
build:
  script_env:
    - VERSION

$ VERSION=`script-to-get-mypkg-version` conda-build conda-recipe

conda-build ignores env var VERSION in this case.

source.

Answer 2

There are lots of ways to get to your endpoint. Here's what conda itself does...

The source of truth for conda's version information is __version__ in conda/__init__.py. It can be loaded programmatically within python code as from conda import __version__ as you suggest. It's also hard-wired into setup.py here (note this code too), so from the command line python setup.py --version is the canonical way to get that information.

In 1.x versions of conda-build, putting a line

$PYTHON setup.py --version > __conda_version__.txt

in build.sh would set the version for the built package using our source of truth. The __conda_version__.txt file is deprecated, however, and it will likely be removed with the release of conda-build 2.0. In recent versions of conda-build, the preferred way to do this is to use load_setup_py_data() within a jinja2 context, which will give you access to all the metadata from setup.py. Specifically, in the meta.yaml file, we'd have something like this

package:
  name: conda
  version: "{{ load_setup_py_data().version }}"

---

Now, how the __version__ variable is set in conda/__init__.py...

What you see in the source code is a call to the auxlib.packaging.get_version() function. This function does the following in order

1. look first for a file conda/.version, and if found return the contents as the version identifier
2. look next for a VERSION environment variable, and if set return the value as the version identifier
3. look last at the git describe --tags output, and return a version identifier if possible (must have git installed, must be a git repo, etc etc)
4. if none of the above yield a version identifier, return None

Now there's just one more final trick. In conda's setup.py file, we set cmdclass for build_py and sdist to those provided by auxlib.packaging. Basically we have

from auxlib import packaging
setup(
    cmdclass={
        'build_py': packaging.BuildPyCommand,
        'sdist': packaging.SDistCommand,
    }
)

These special command classes actually modify the conda/__init__.py file in built/installed packages so the __version__ variable is hard-coded to a string literal, and doesn't use the auxlib.packaging.get_version() function.

---

In your case, with not wanting to tag every release, you could use all of the above, and from the command line set the version using a VERSION environment variable. Something like

VERSION=1.0.0alpha1 conda build conda.recipe

In your build section meta.yaml recipe, you'll need add a script_env key to tell conda-build to pass the VERSION environment variable all the way through to the build environment.

build:
  script_env:
    - VERSION

Answer 3

Manual __version__

If you have the version in a separate _version.py that you can import without loading the whole package.

# coding: utf-8
# file generated by setuptools_scm
# don't change, don't track in version control
version = '0.0.9.post2+g6481728.d20200518.dirty'

In my case this gets automatically generated, but the next step stays the same.

in __init__.py you have a line from ._version import version as __version__

and then in setup.py you could do something like this. This is also how I import the version in my sphinx conf.py

source_dir = Path("src/<my_package>")
sys.path.insert(0, str(source_dir))

from _version import  version

setup(version=version)
...

Alternatively, instead of importing the _version file, you can try to parse it manually, so you don't have to add something to sys.path

and then in meta.yaml

{% set data = load_setup_py_data() %}
{% set version = data.get('version')  %}


package:
  name: <my_package>
  version: {{ version }}

---

I had the reverse issue. I forgot to update my version from time to sime, so was looking for a way to have the git repository as single source of the package version. I used setuptools_scm

I've tried a lot of things, with and without pep517 compliant pyproject.toml etcetera, but eventually, this is the one that works for me.

The advantage of this is you don't need that huge versioneer.py, but it gets written to _version.py at build time

setup.py

from setuptools import setup
import setuptools_scm


def my_local_scheme(version: setuptools_scm.version.ScmVersion) -> str:
    """My local node and date version."""
    node_and_date = setuptools_scm.version.get_local_node_and_date(version)
    dirty = ".dirty" if version.dirty else ""
    return str(node_and_date) + dirty

version = setuptools_scm.get_version(
    write_to="src/<my_package>/_version.py",
    version_scheme="post-release",
    local_scheme=my_local_scheme,
)
setup(version=version,)

The rest of the setup() metadata and options in in setup.cfg. One that needs to be there is:

[options]
package_dir=
    =src
packages = <my_package>
install_requires = setuptools_scm

src/<my_package>/_version.py

gets generated:

# coding: utf-8
# file generated by setuptools_scm
# don't change, don't track in version control
version = '0.0.3.post0+g887e418.d20200518.dirty'

and I add it to my .gitignore

src/<my_package>/__init__.py

"""<package_description>"""
from ._version import version as  __version__

meta.yaml

{% set data = load_setup_py_data() %}
{% set version = data.get('version')  %}


package:
  name: capacity_simulation
  version: {{ version }}

source:
  path: .

build:
  noarch: python
  number: {{ environ.get('GIT_DESCRIBE_NUMBER', 0) }}
  script: python -m pip install --no-deps --ignore-installed .
  include_recipe: False

requirements:
  build:
    - setuptools_scm
...

pyproject.toml

To also be able to use pip wheel .

you need this section in pyproject.toml

[build-system]
requires = ["setuptools>=34.4", "wheel", "setuptools_scm"]
build-backend = "setuptools.build_meta"