zhangyiss/spack
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

Improve version, version range, and version list syntax and behavior (#36273)

Browse Source 
## Version types, parsing and printing

- The version classes have changed: `VersionBase` is removed, there is now a
  `ConcreteVersion` base class. `StandardVersion` and `GitVersion` both inherit
  from this.

- The public api (`Version`, `VersionRange`, `ver`) has changed a bit:
  1. `Version` produces either `StandardVersion` or `GitVersion` instances.
  2. `VersionRange` produces a `ClosedOpenRange`, but this shouldn't affect the user.
  3. `ver` produces any of `VersionList`, `ClosedOpenRange`, `StandardVersion`
     or `GitVersion`.

- No unexpected type promotion, so that the following is no longer an identity:
  `Version(x) != VersionRange(x, x)`.

- `VersionList.concrete` now returns a version if it contains only a single element
  subtyping `ConcreteVersion` (i.e. `StandardVersion(...)` or `GitVersion(...)`)

- In version lists, the parser turns `@x` into `VersionRange(x, x)` instead
  of `Version(x)`.

- The above also means that `ver("x")` produces a range, whereas
  `ver("=x")` produces a `StandardVersion`. The `=` is part of _VersionList_
  syntax.

- `VersionList.__str__` now outputs `=x.y.z` for specific version entries,
  and `x.y.z` as a short-hand for ranges `x.y.z:x.y.z`.

- `Spec.format` no longer aliases `{version}` to `{versions}`, but pulls the
  concrete version out of the list and prints that -- except when the list is
  is not concrete, then is falls back to `{versions}` to avoid a pedantic error.
  For projections of concrete specs, `{version}` should be used to render
  `1.2.3` instead of `=1.2.3` (which you would get with `{versions}`).
  The default `Spec` format string used in `Spec.__str__` now uses
  `{versions}` so that `str(Spec(string)) == string` holds.

## Changes to `GitVersion`

- `GitVersion` is a small wrapper around `StandardVersion` which enriches it
   with a git ref. It no longer inherits from it.

- `GitVersion` _always_ needs to be able to look up an associated Spack version
  if it was not assigned (yet). It throws a `VersionLookupError` whenever `ref_version`
  is accessed but it has no means to look up the ref; in the past Spack would
  not error and use the commit sha as a literal version, which was incorrect.
   
- `GitVersion` is never equal to `StandardVersion`, nor is satisfied by it. This
  is such that we don't lose transitivity. This fixes the following bug on `develop`
  where `git_version_a == standard_version == git_version_b` does not imply
  `git_version_a == git_version_b`. It also ensures equality always implies equal
  hash, which is also currently broken on develop; inclusion tests of a set of
  versions + git versions would behave differently from inclusion tests of a
  list of the same objects.

- The above means `ver("ref=1.2.3) != ver("=1.2.3")` could break packages that branch
  on specific versions, but that was brittle already, since the same happens with
  externals: `pkg@1.2.3-external` suffixes wouldn't be exactly equal either. Instead,
  those checks should be `x.satisfies("@1.2.3")` which works both for git versions and
  custom version suffixes.

- `GitVersion` from commit will now print as `<hash>=<version>` once the
  git ref is resolved to a spack version. This is for reliability -- version is frozen
  when added to the database and queried later. It also improves performance
  since there is no need to clone all repos of all git versions after `spack clean -m`
  is run and something queries the database, triggering version comparison, such
  as potentially reuse concretization.

- The "empty VerstionStrComponent trick" for `GitVerison` is dropped since it wasn't
  representable as a version string (by design). Instead, it's replaced by `git`,
  so you get `1.2.3.git.4` (which reads 4 commits after a tag 1.2.3). This means
  that there's an edge case for version schemes `1.1.1`, `1.1.1a`, since the
  generated git version `1.1.1.git.1` (1 commit after `1.1.1`) compares larger
  than `1.1.1a`, since `a < git` are compared as strings. This is currently a
  wont-fix edge case, but if really required, could be fixed by special casing
  the `git` string.

- Saved, concrete specs (database, lock file, ...) that only had a git sha as their
  version, but have no means to look the effective Spack version anymore, will
  now see their version mapped to `hash=develop`. Previously these specs
  would always have their sha literally interpreted as a version string (even when
  it _could_ be looked up). This only applies to databases, lock files and spec.json
  files created before Spack 0.20; after this PR, we always have a Spack version
  associated to the relevant GitVersion).

- Fixes a bug where previously `to_dict` / `from_dict` (de)serialization would not
  reattach the repo to the GitVersion, causing the git hash to be used as a literal
  (bogus) version instead of the resolved version. This was in particularly breaking
  version comparison in the build process on macOS/Windows.


## Installing or matching specific versions

- In the past, `spack install pkg@3.2` would install `pkg@=3.2` if it was a
  known specific version defined in the package, even when newer patch releases
  `3.2.1`, `3.2.2`, `...` were available. This behavior was only there because
  there was no syntax to distinguish between `3.2` and `3.2.1`. Since there is
  syntax for this now through `pkg@=3.2`, the old exact matching behavior is
  removed. This means that `spack install pkg@3.2` constrains the `pkg` version
  to the range `3.2`, and `spack install pkg@=3.2` constrains it to the specific
  version `3.2`.

- Also in directives such as `depends_on("pkg@2.3")` and their when
  conditions `conflicts("...", when="@2.3")` ranges are ranges, and specific
  version matches require `@=2.3.`.

- No matching version: in the case `pkg@3.2` matches nothing, concretization
  errors. However, if you run `spack install pkg@=3.2` and this version
  doesn't exist, Spack will define it; this allows you to install non-registered
  versions.

- For consistency, you can now do `%gcc@10` and let it match a configured
  `10.x.y` compiler. It errors when there is no matching compiler.
  In the past it was interpreted like a specific `gcc@=10` version, which
  would get bootstrapped.

- When compiler _bootstrapping_ is enabled, `%gcc@=10.2.0` can be used to
  bootstrap a specific compiler version.

## Other changes

- Externals, compilers, and develop spec definitions are backwards compatible.
  They are typically defined as `pkg@3.2.1` even though they should be
  saying `pkg@=3.2.1`. Spack now transforms `pkg@3` into `pkg@=3` in those cases.

- Finally, fix strictness of `version(...)` directive/declaration. It just does a simple
  type check, and now requires strings/integers. Floats are not allowed because
  they are ambiguous `str(3.10) == "3.1"`.
This commit is contained in:
Harmen Stoppels
2023-05-06 06:04:41 +02:00
committed by GitHub
parent f6497972b8
commit fa7719a031
91 changed files with 2059 additions and 1870 deletions
Download Patch File Download Diff File Expand all files Collapse all files

72
lib/spack/docs/basic_usage.rst
View File

@@ -1103,16 +1103,31 @@ Below are more details about the specifiers that you can add to specs.
Version specifier
^^^^^^^^^^^^^^^^^
A version specifier comes somewhere after a package name and starts
with ``@``. It can be a single version, e.g. ``@1.0``, ``@3``, or
``@1.2a7``. Or, it can be a range of versions, such as ``@1.0:1.5``
(all versions between ``1.0`` and ``1.5``, inclusive). Version ranges
can be open, e.g. ``:3`` means any version up to and including ``3``.
This would include ``3.4`` and ``3.4.2``. ``4.2:`` means any version
above and including ``4.2``. Finally, a version specifier can be a
set of arbitrary versions, such as ``@1.0,1.5,1.7`` (``1.0``, ``1.5``,
or ``1.7``). When you supply such a specifier to ``spack install``,
it constrains the set of versions that Spack will install.
A version specifier ``pkg@<specifier>`` comes after a package name
and starts with ``@``. It can be something abstract that matches
multiple known versions, or a specific version. During concretization,
Spack will pick the optimal version within the spec's constraints
according to policies set for the particular Spack installation.
The version specifier can be *a specific version*, such as ``@=1.0.0`` or
``@=1.2a7``. Or, it can be *a range of versions*, such as ``@1.0:1.5``.
Version ranges are inclusive, so this example includes both ``1.0``
and any ``1.5.x`` version. Version ranges can be unbounded, e.g. ``@:3``
means any version up to and including ``3``. This would include ``3.4``
and ``3.4.2``. Similarly, ``@4.2:`` means any version above and including
``4.2``. As a short-hand, ``@3`` is equivalent to the range ``@3:3`` and
includes any version with major version ``3``.
Notice that you can distinguish between the specific version ``@=3.2`` and
the range ``@3.2``. This is useful for packages that follow a versioning
scheme that omits the zero patch version number: ``3.2``, ``3.2.1``,
``3.2.2``, etc. In general it is preferable to use the range syntax
``@3.2``, since ranges also match versions with one-off suffixes, such as
``3.2-custom``.
A version specifier can also be a list of ranges and specific versions,
separated by commas. For example, ``@1.0:1.5,=1.7.1`` matches any version
in the range ``1.0:1.5`` and the specific version ``1.7.1``.
For packages with a ``git`` attribute, ``git`` references
may be specified instead of a numerical version i.e. branches, tags
@@ -1121,36 +1136,35 @@ reference provided. Acceptable syntaxes for this are:
.. code-block:: sh
# branches and tags
foo@git.develop # use the develop branch
foo@git.0.19 # use the 0.19 tag
# commit hashes
foo@abcdef1234abcdef1234abcdef1234abcdef1234 # 40 character hashes are automatically treated as git commits
foo@git.abcdef1234abcdef1234abcdef1234abcdef1234
Spack versions from git reference either have an associated version supplied by the user,
or infer a relationship to known versions from the structure of the git repository. If an
associated version is supplied by the user, Spack treats the git version as equivalent to that
version for all version comparisons in the package logic (e.g. ``depends_on('foo', when='@1.5')``).
# branches and tags
foo@git.develop # use the develop branch
foo@git.0.19 # use the 0.19 tag
The associated version can be assigned with ``[git ref]=[version]`` syntax, with the caveat that the specified version is known to Spack from either the package definition, or in the configuration preferences (i.e. ``packages.yaml``).
Spack always needs to associate a Spack version with the git reference,
which is used for version comparison. This Spack version is heuristically
taken from the closest valid git tag among ancestors of the git ref.
Once a Spack version is associated with a git ref, it always printed with
the git ref. For example, if the commit ``@git.abcdefg`` is tagged
``0.19``, then the spec will be shown as ``@git.abcdefg=0.19``.
If the git ref is not exactly a tag, then the distance to the nearest tag
is also part of the resolved version. ``@git.abcdefg=0.19.git.8`` means
that the commit is 8 commits away from the ``0.19`` tag.
In cases where Spack cannot resolve a sensible version from a git ref,
users can specify the Spack version to use for the git ref. This is done
by appending ``=`` and the Spack version to the git ref. For example:
.. code-block:: sh
foo@git.my_ref=3.2 # use the my_ref tag or branch, but treat it as version 3.2 for version comparisons
foo@git.abcdef1234abcdef1234abcdef1234abcdef1234=develop # use the given commit, but treat it as develop for version comparisons
If an associated version is not supplied then the tags in the git repo are used to determine
the most recent previous version known to Spack. Details about how versions are compared
and how Spack determines if one version is less than another are discussed in the developer guide.
If the version spec is not provided, then Spack will choose one
according to policies set for the particular spack installation. If
the spec is ambiguous, i.e. it could match multiple versions, Spack
will choose a version within the spec's constraints according to
policies set for the particular Spack installation.
Details about how versions are compared and how Spack determines if
one version is less than another are discussed in the developer guide.

2
lib/spack/docs/conf.py
View File

@@ -215,7 +215,7 @@ def setup(sphinx):
("py:class", "spack.repo._PrependFileLoader"),
("py:class", "spack.build_systems._checks.BaseBuilder"),
# Spack classes that intersphinx is unable to resolve
("py:class", "spack.version.VersionBase"),
("py:class", "spack.version.StandardVersion"),
("py:class", "spack.spec.DependencySpec"),
]

56
lib/spack/docs/packaging_guide.rst
View File

@@ -851,16 +851,16 @@ Version comparison
^^^^^^^^^^^^^^^^^^
Most Spack versions are numeric, a tuple of integers; for example,
``apex@0.1``, ``ferret@6.96`` or ``py-netcdf@1.2.3.1``. Spack knows
how to compare and sort numeric versions.
``0.1``, ``6.96`` or ``1.2.3.1``. Spack knows how to compare and sort
numeric versions.
Some Spack versions involve slight extensions of numeric syntax; for
example, ``py-sphinx-rtd-theme@0.1.10a0``. In this case, numbers are
example, ``py-sphinx-rtd-theme@=0.1.10a0``. In this case, numbers are
always considered to be "newer" than letters. This is for consistency
with `RPM <https://bugzilla.redhat.com/show_bug.cgi?id=50977>`_.
Spack versions may also be arbitrary non-numeric strings, for example
``@develop``, ``@master``, ``@local``.
``develop``, ``master``, ``local``.
The order on versions is defined as follows. A version string is split
into a list of components based on delimiters such as ``.``, ``-`` etc.
@@ -918,6 +918,32 @@ use:
will be used.
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Ranges versus specific versions
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
When specifying versions in Spack using the ``pkg@<specifier>`` syntax,
you can use either ranges or specific versions. It is generally
recommended to use ranges instead of specific versions when packaging
to avoid overly constraining dependencies, patches, and conflicts.
For example, ``depends_on("python@3")`` denotes a range of versions,
allowing Spack to pick any ``3.x.y`` version for Python, while
``depends_on("python@=3.10.1")`` restricts it to a specific version.
Specific ``@=`` versions should only be used in exceptional cases, such
as when the package has a versioning scheme that omits the zero in the
first patch release: ``3.1``, ``3.1.1``, ``3.1.2``. In this example,
the specifier ``@=3.1`` is the correct way to select only the ``3.1``
version, whereas ``@3.1`` would match all those versions.
Ranges are preferred even if they would only match a single version
defined in the package. This is because users can define custom versions
in ``packages.yaml`` that typically include a custom suffix. For example,
if the package defines the version ``1.2.3``, the specifier ``@1.2.3``
will also match a user-defined version ``1.2.3-custom``.
.. _cmd-spack-checksum:
^^^^^^^^^^^^^^^^^^
@@ -2388,21 +2414,29 @@ requires Python 2, you can similarly leave out the lower bound:
Notice that we didn't use ``@:3``. Version ranges are *inclusive*, so
``@:3`` means "up to and including any 3.x version".
What if a package can only be built with Python 2.7? You might be
inclined to use:
You can also simply write
.. code-block:: python
depends_on("python@2.7")
However, this would be wrong. Spack assumes that all version constraints
are exact, so it would try to install Python not at ``2.7.18``, but
exactly at ``2.7``, which is a non-existent version. The correct way to
specify this would be:
to tell Spack that the package needs Python 2.7.x. This is equivalent to
``@2.7:2.7``.
In very rare cases, you may need to specify an exact version, for example
if you need to distinguish between ``3.2`` and ``3.2.1``:
.. code-block:: python
depends_on("python@2.7.0:2.7")
depends_on("pkg@=3.2")
But in general, you should try to use version ranges as much as possible,
so that custom suffixes are included too. The above example can be
rewritten in terms of ranges as follows:
.. code-block:: python
depends_on("pkg@3.2:3.2.0")
A spec can contain a version list of ranges and individual versions
separated by commas. For example, if you need Boost 1.59.0 or newer,