builtin: use api v2.0 and update dir structure (#49275)

* Bump the package API of the `builtin` repo to `v2.0`
* Move `var/spack/repos/builtin` -> `var/spack/repos/spack_repo/builtin`
* Move test repos `var/spack/repos/{builtin.mock,tutorial,...}` -> `var/spack/test_repos/`
* Update package dir names to v2 format (`-` -> `_` etc)
* Change absolute imports `from spack.pkg.builtin.my_pkg ...` to relative imports `from ..my_pkg.package ...`

Users who have a repo on top of builtin should change imports from

```python
from spack.pkg.builtin.my_pkg import MyPkg
```

to

```python
from spack_repo.builtin.packages.my_pkg.package import MyPkg
```

and can configure their editors with

```
PYTHONPATH=$spack/lib/spack:$spack/var/spack/repos
```

[skip-verify-checksums]
This commit is contained in:
Harmen Stoppels
2025-05-06 12:05:44 +02:00
committed by GitHub
parent 285f95a4d8
commit b932c14008
11017 changed files with 506 additions and 351 deletions

View File

@@ -28,7 +28,7 @@ max-line-length = 99
# - F821: undefined name `name`
#
per-file-ignores =
var/spack/repos/*/package.py:F403,F405,F821
var/spack/*/package.py:F403,F405,F821
*-ci-package.py:F403,F405,F821
# exclude things we usually do not want linting for.

View File

@@ -42,17 +42,17 @@ jobs:
# built-in repository or documentation
filters: |
bootstrap:
- 'var/spack/repos/builtin/packages/clingo-bootstrap/**'
- 'var/spack/repos/builtin/packages/clingo/**'
- 'var/spack/repos/builtin/packages/python/**'
- 'var/spack/repos/builtin/packages/re2c/**'
- 'var/spack/repos/builtin/packages/gnupg/**'
- 'var/spack/repos/builtin/packages/libassuan/**'
- 'var/spack/repos/builtin/packages/libgcrypt/**'
- 'var/spack/repos/builtin/packages/libgpg-error/**'
- 'var/spack/repos/builtin/packages/libksba/**'
- 'var/spack/repos/builtin/packages/npth/**'
- 'var/spack/repos/builtin/packages/pinentry/**'
- 'var/spack/repos/spack_repo/builtin/packages/clingo-bootstrap/**'
- 'var/spack/repos/spack_repo/builtin/packages/clingo/**'
- 'var/spack/repos/spack_repo/builtin/packages/python/**'
- 'var/spack/repos/spack_repo/builtin/packages/re2c/**'
- 'var/spack/repos/spack_repo/builtin/packages/gnupg/**'
- 'var/spack/repos/spack_repo/builtin/packages/libassuan/**'
- 'var/spack/repos/spack_repo/builtin/packages/libgcrypt/**'
- 'var/spack/repos/spack_repo/builtin/packages/libgpg-error/**'
- 'var/spack/repos/spack_repo/builtin/packages/libksba/**'
- 'var/spack/repos/spack_repo/builtin/packages/npth/**'
- 'var/spack/repos/spack_repo/builtin/packages/pinentry/**'
- 'lib/spack/**'
- 'share/spack/**'
- '.github/workflows/bootstrap.yml'

View File

@@ -34,7 +34,7 @@ jobs:
vermin --backport importlib --backport argparse --violations --backport typing -t=3.6- -vvv lib/spack/spack/ lib/spack/llnl/ bin/
- name: vermin (Repositories)
run: |
vermin --backport importlib --backport argparse --violations --backport typing -t=3.6- -vvv var/spack/repos
vermin --backport importlib --backport argparse --violations --backport typing -t=3.6- -vvv var/spack/repos var/spack/test_repos
# Run style checks on the files that have been changed
style:

View File

@@ -11,4 +11,4 @@
# ~/.spack/repos.yaml
# -------------------------------------------------------------------------
repos:
- $spack/var/spack/repos/builtin
- $spack/var/spack/repos/spack_repo/builtin

View File

@@ -1916,7 +1916,7 @@ diagnostics. Issues, if found, are reported to stdout:
PKG-DIRECTIVES: 1 issue found
1. lammps: wrong variant in "conflicts" directive
the variant 'adios' does not exist
in /home/spack/spack/var/spack/repos/builtin/packages/lammps/package.py
in /home/spack/spack/var/spack/repos/spack_repo/builtin/packages/lammps/package.py
------------

View File

@@ -83,7 +83,7 @@ packages. You can quickly find examples by running:
.. code-block:: console
$ cd var/spack/repos/builtin/packages
$ cd var/spack/repos/spack_repo/builtin/packages
$ grep -l QMakePackage */package.py

View File

@@ -27,10 +27,10 @@ it could use the ``require`` directive as follows:
Spack has a number of built-in bundle packages, such as:
* `AmdAocl <https://github.com/spack/spack/blob/develop/var/spack/repos/builtin/packages/amd-aocl/package.py>`_
* `EcpProxyApps <https://github.com/spack/spack/blob/develop/var/spack/repos/builtin/packages/ecp-proxy-apps/package.py>`_
* `Libc <https://github.com/spack/spack/blob/develop/var/spack/repos/builtin/packages/libc/package.py>`_
* `Xsdk <https://github.com/spack/spack/blob/develop/var/spack/repos/builtin/packages/xsdk/package.py>`_
* `AmdAocl <https://github.com/spack/spack/blob/develop/var/spack/repos/spack_repo/builtin/packages/amd_aocl/package.py>`_
* `EcpProxyApps <https://github.com/spack/spack/blob/develop/var/spack/repos/spack_repo/builtin/packages/ecp_proxy_apps/package.py>`_
* `Libc <https://github.com/spack/spack/blob/develop/var/spack/repos/spack_repo/builtin/packages/libc/package.py>`_
* `Xsdk <https://github.com/spack/spack/blob/develop/var/spack/repos/spack_repo/builtin/packages/xsdk/package.py>`_
where ``Xsdk`` also inherits from ``CudaPackage`` and ``RocmPackage`` and
``Libc`` is a virtual bundle package for the C standard library.

View File

@@ -199,7 +199,7 @@ a variant to control this:
However, not every CMake package accepts all four of these options.
Grep the ``CMakeLists.txt`` file to see if the default values are
missing or replaced. For example, the
`dealii <https://github.com/spack/spack/blob/develop/var/spack/repos/builtin/packages/dealii/package.py>`_
`dealii <https://github.com/spack/spack/blob/develop/var/spack/repos/spack_repo/builtin/packages/dealii/package.py>`_
package overrides the default variant with:
.. code-block:: python

View File

@@ -20,8 +20,8 @@ start is to look at the definitions of other build systems. This guide
focuses mostly on how Spack's build systems work.
In this guide, we will be using the
`perl <https://github.com/spack/spack/blob/develop/var/spack/repos/builtin/packages/perl/package.py>`_ and
`cmake <https://github.com/spack/spack/blob/develop/var/spack/repos/builtin/packages/cmake/package.py>`_
`perl <https://github.com/spack/spack/blob/develop/var/spack/repos/spack_repo/builtin/packages/perl/package.py>`_ and
`cmake <https://github.com/spack/spack/blob/develop/var/spack/repos/spack_repo/builtin/packages/cmake/package.py>`_
packages as examples. ``perl``'s build system is a hand-written
``Configure`` shell script, while ``cmake`` bootstraps itself during
installation. Both of these packages require custom build systems.

View File

@@ -96,9 +96,9 @@ there are any other variables you need to set, you can do this in the
env.set("BLASLIB", spec["blas"].libs.ld_flags)
`cbench <https://github.com/spack/spack/blob/develop/var/spack/repos/builtin/packages/cbench/package.py>`_
`cbench <https://github.com/spack/spack/blob/develop/var/spack/repos/spack_repo/builtin/packages/cbench/package.py>`_
is a good example of a simple package that does this, while
`esmf <https://github.com/spack/spack/blob/develop/var/spack/repos/builtin/packages/esmf/package.py>`_
`esmf <https://github.com/spack/spack/blob/develop/var/spack/repos/spack_repo/builtin/packages/esmf/package.py>`_
is a good example of a more complex package.
""""""""""""""""""""""
@@ -129,7 +129,7 @@ If you do need access to the spec, you can create a property like so:
]
`cloverleaf <https://github.com/spack/spack/blob/develop/var/spack/repos/builtin/packages/cloverleaf/package.py>`_
`cloverleaf <https://github.com/spack/spack/blob/develop/var/spack/repos/spack_repo/builtin/packages/cloverleaf/package.py>`_
is a good example of a package that uses this strategy.
"""""""""""""
@@ -152,7 +152,7 @@ and a ``filter`` method to help with this. For example:
makefile.filter(r"^\s*FC\s*=.*", f"FC = {spack_fc}")
`stream <https://github.com/spack/spack/blob/develop/var/spack/repos/builtin/packages/stream/package.py>`_
`stream <https://github.com/spack/spack/blob/develop/var/spack/repos/spack_repo/builtin/packages/stream/package.py>`_
is a good example of a package that involves editing a Makefile to set
the appropriate variables.
@@ -192,7 +192,7 @@ well for storing variables:
inc.write(f"{key} = {config[key]}\n")
`elk <https://github.com/spack/spack/blob/develop/var/spack/repos/builtin/packages/elk/package.py>`_
`elk <https://github.com/spack/spack/blob/develop/var/spack/repos/spack_repo/builtin/packages/elk/package.py>`_
is a good example of a package that uses a dictionary to store
configuration variables.
@@ -213,7 +213,7 @@ them in a list:
inc.write(f"{var}\n")
`hpl <https://github.com/spack/spack/blob/develop/var/spack/repos/builtin/packages/hpl/package.py>`_
`hpl <https://github.com/spack/spack/blob/develop/var/spack/repos/spack_repo/builtin/packages/hpl/package.py>`_
is a good example of a package that uses a list to store
configuration variables.

View File

@@ -39,7 +39,7 @@ for "CRAN <package-name>" and you should quickly find what you want.
If it isn't on CRAN, try Bioconductor, another common R repository.
For the purposes of this tutorial, we will be walking through
`r-caret <https://github.com/spack/spack/blob/develop/var/spack/repos/builtin/packages/r-caret/package.py>`_
`r-caret <https://github.com/spack/spack/blob/develop/var/spack/repos/spack_repo/builtin/packages/r_caret/package.py>`_
as an example. If you search for "CRAN caret", you will quickly find what
you are looking for at https://cran.r-project.org/package=caret.
https://cran.r-project.org is the main CRAN website. However, CRAN also
@@ -337,7 +337,7 @@ Non-R dependencies
^^^^^^^^^^^^^^^^^^
Some packages depend on non-R libraries for linking. Check out the
`r-stringi <https://github.com/spack/spack/blob/develop/var/spack/repos/builtin/packages/r-stringi/package.py>`_
`r-stringi <https://github.com/spack/spack/blob/develop/var/spack/repos/spack_repo/builtin/packages/r_stringi/package.py>`_
package for an example: https://cloud.r-project.org/package=stringi.
If you search for the text "SystemRequirements", you will see:
@@ -352,7 +352,7 @@ Passing arguments to the installation
Some R packages provide additional flags that can be passed to
``R CMD INSTALL``, often to locate non-R dependencies.
`r-rmpi <https://github.com/spack/spack/blob/develop/var/spack/repos/builtin/packages/r-rmpi/package.py>`_
`r-rmpi <https://github.com/spack/spack/blob/develop/var/spack/repos/spack_repo/builtin/packages/r_rmpi/package.py>`_
is an example of this, and flags for linking to an MPI library. To pass
these to the installation command, you can override ``configure_args``
like so:

View File

@@ -104,10 +104,10 @@ Finding available options
The first place to start when looking for a list of valid options to
build a package is ``scons --help``. Some packages like
`kahip <https://github.com/spack/spack/blob/develop/var/spack/repos/builtin/packages/kahip/package.py>`_
`kahip <https://github.com/spack/spack/blob/develop/var/spack/repos/spack_repo/builtin/packages/kahip/package.py>`_
don't bother overwriting the default SCons help message, so this isn't
very useful, but other packages like
`serf <https://github.com/spack/spack/blob/develop/var/spack/repos/builtin/packages/serf/package.py>`_
`serf <https://github.com/spack/spack/blob/develop/var/spack/repos/spack_repo/builtin/packages/serf/package.py>`_
print a list of valid command-line variables:
.. code-block:: console
@@ -177,7 +177,7 @@ print a list of valid command-line variables:
More advanced packages like
`cantera <https://github.com/spack/spack/blob/develop/var/spack/repos/builtin/packages/cantera/package.py>`_
`cantera <https://github.com/spack/spack/blob/develop/var/spack/repos/spack_repo/builtin/packages/cantera/package.py>`_
use ``scons --help`` to print a list of subcommands:
.. code-block:: console

View File

@@ -226,9 +226,9 @@ If all is well, you'll see something like this:
Modified files:
var/spack/repos/builtin/packages/hdf5/package.py
var/spack/repos/builtin/packages/hdf/package.py
var/spack/repos/builtin/packages/netcdf/package.py
var/spack/repos/spack_repo/builtin/packages/hdf5/package.py
var/spack/repos/spack_repo/builtin/packages/hdf/package.py
var/spack/repos/spack_repo/builtin/packages/netcdf/package.py
=======================================================
Flake8 checks were clean.
@@ -236,9 +236,9 @@ However, if you aren't compliant with PEP 8, flake8 will complain:
.. code-block:: console
var/spack/repos/builtin/packages/netcdf/package.py:26: [F401] 'os' imported but unused
var/spack/repos/builtin/packages/netcdf/package.py:61: [E303] too many blank lines (2)
var/spack/repos/builtin/packages/netcdf/package.py:106: [E501] line too long (92 > 79 characters)
var/spack/repos/spack_repo/builtin/packages/netcdf/package.py:26: [F401] 'os' imported but unused
var/spack/repos/spack_repo/builtin/packages/netcdf/package.py:61: [E303] too many blank lines (2)
var/spack/repos/spack_repo/builtin/packages/netcdf/package.py:106: [E501] line too long (92 > 79 characters)
Flake8 found errors.
Most of the error messages are straightforward, but if you don't understand what
@@ -280,7 +280,7 @@ All of these can be installed with Spack, e.g.
.. warning::
Sphinx has `several required dependencies <https://github.com/spack/spack/blob/develop/var/spack/repos/builtin/packages/py-sphinx/package.py>`_.
Sphinx has `several required dependencies <https://github.com/spack/spack/blob/develop/var/spack/repos/spack_repo/builtin/packages/py-sphinx/package.py>`_.
If you're using a ``python`` from Spack and you installed
``py-sphinx`` and friends, you need to make them available to your
``python``. The easiest way to do this is to run:

View File

@@ -131,7 +131,7 @@ creates a simple python file:
It doesn't take much python coding to get from there to a working
package:
.. literalinclude:: _spack_root/var/spack/repos/builtin/packages/libelf/package.py
.. literalinclude:: _spack_root/var/spack/repos/spack_repo/builtin/packages/libelf/package.py
:lines: 5-
Spack also provides wrapper functions around common commands like

View File

@@ -369,9 +369,9 @@ If you have a collection of software expected to work well together with
no source code of its own, you can create a :ref:`BundlePackage <bundlepackage>`.
Examples where bundle packages can be useful include defining suites of
applications (e.g, `EcpProxyApps
<https://github.com/spack/spack/blob/develop/var/spack/repos/builtin/packages/ecp-proxy-apps/package.py>`_), commonly used libraries
(e.g., `AmdAocl <https://github.com/spack/spack/blob/develop/var/spack/repos/builtin/packages/amd-aocl/package.py>`_),
and software development kits (e.g., `EcpDataVisSdk <https://github.com/spack/spack/blob/develop/var/spack/repos/builtin/packages/ecp-data-vis-sdk/package.py>`_).
<https://github.com/spack/spack/blob/develop/var/spack/repos/spack_repo/builtin/packages/ecp_proxy_apps/package.py>`_), commonly used libraries
(e.g., `AmdAocl <https://github.com/spack/spack/blob/develop/var/spack/repos/spack_repo/builtin/packages/amd_aocl/package.py>`_),
and software development kits (e.g., `EcpDataVisSdk <https://github.com/spack/spack/blob/develop/var/spack/repos/spack_repo/builtin/packages/ecp_data_vis_sdk/package.py>`_).
These versioned packages primarily consist of dependencies on the associated
software packages. They can include :ref:`variants <variants>` to ensure
@@ -443,7 +443,7 @@ lives in:
.. code-block:: console
$ spack location -p gmp
${SPACK_ROOT}/var/spack/repos/builtin/packages/gmp/package.py
${SPACK_ROOT}/var/spack/repos/spack_repo/builtin/packages/gmp/package.py
but ``spack edit`` provides a much simpler shortcut and saves you the
trouble of typing the full path.
@@ -457,19 +457,19 @@ live in Spack's directory structure. In general, :ref:`cmd-spack-create`
handles creating package files for you, so you can skip most of the
details here.
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
``var/spack/repos/builtin/packages``
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
``var/spack/repos/spack_repo/builtin/packages``
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
A Spack installation directory is structured like a standard UNIX
install prefix (``bin``, ``lib``, ``include``, ``var``, ``opt``,
etc.). Most of the code for Spack lives in ``$SPACK_ROOT/lib/spack``.
Packages themselves live in ``$SPACK_ROOT/var/spack/repos/builtin/packages``.
Packages themselves live in ``$SPACK_ROOT/var/spack/repos/spack_repo/builtin/packages``.
If you ``cd`` to that directory, you will see directories for each
package:
.. command-output:: cd $SPACK_ROOT/var/spack/repos/builtin/packages && ls
.. command-output:: cd $SPACK_ROOT/var/spack/repos/spack_repo/builtin/packages && ls
:shell:
:ellipsis: 10
@@ -479,7 +479,7 @@ package lives in:
.. code-block:: none
$SPACK_ROOT/var/spack/repos/builtin/packages/libelf/package.py
$SPACK_ROOT/var/spack/repos/spack_repo/builtin/packages/libelf/package.py
Alongside the ``package.py`` file, a package may contain extra
directories or files (like patches) that it needs to build.
@@ -492,7 +492,7 @@ Packages are named after the directory containing ``package.py``. So,
``libelf``'s ``package.py`` lives in a directory called ``libelf``.
The ``package.py`` file defines a class called ``Libelf``, which
extends Spack's ``Package`` class. For example, here is
``$SPACK_ROOT/var/spack/repos/builtin/packages/libelf/package.py``:
``$SPACK_ROOT/var/spack/repos/spack_repo/builtin/packages/libelf/package.py``:
.. code-block:: python
:linenos:
@@ -520,7 +520,7 @@ these:
$ spack install libelf@0.8.13
Spack sees the package name in the spec and looks for
``libelf/package.py`` in ``var/spack/repos/builtin/packages``.
``libelf/package.py`` in ``var/spack/repos/spack_repo/builtin/packages``.
Likewise, if you run ``spack install py-numpy``, Spack looks for
``py-numpy/package.py``.
@@ -686,7 +686,7 @@ https://www.open-mpi.org/software/ompi/v2.1/downloads/openmpi-2.1.1.tar.bz2
In order to handle this, you can define a ``url_for_version()`` function
like so:
.. literalinclude:: _spack_root/var/spack/repos/builtin/packages/openmpi/package.py
.. literalinclude:: _spack_root/var/spack/repos/spack_repo/builtin/packages/openmpi/package.py
:pyobject: Openmpi.url_for_version
With the use of this ``url_for_version()``, Spack knows to download OpenMPI ``2.1.1``
@@ -787,7 +787,7 @@ of GNU. For that, Spack goes a step further and defines a mixin class that
takes care of all of the plumbing and requires packagers to just define a proper
``gnu_mirror_path`` attribute:
.. literalinclude:: _spack_root/var/spack/repos/builtin/packages/autoconf/package.py
.. literalinclude:: _spack_root/var/spack/repos/spack_repo/builtin/packages/autoconf/package.py
:lines: 9-18
^^^^^^^^^^^^^^^^^^^^^^^^
@@ -1995,7 +1995,7 @@ structure like this:
.. code-block:: none
$SPACK_ROOT/var/spack/repos/builtin/packages/
$SPACK_ROOT/var/spack/repos/spack_repo/builtin/packages/
mvapich2/
package.py
ad_lustre_rwcontig_open_source.patch
@@ -2133,7 +2133,7 @@ handles ``RPATH``:
.. _pyside-patch:
.. literalinclude:: _spack_root/var/spack/repos/builtin/packages/py-pyside/package.py
.. literalinclude:: _spack_root/var/spack/repos/spack_repo/builtin/packages/py_pyside/package.py
:pyobject: PyPyside.patch
:linenos:
@@ -2201,7 +2201,7 @@ using the ``spack resource show`` command::
$ spack resource show 3877ab54
3877ab548f88597ab2327a2230ee048d2d07ace1062efe81fc92e91b7f39cd00
path: /home/spackuser/src/spack/var/spack/repos/builtin/packages/m4/gnulib-pgi.patch
path: /home/spackuser/src/spack/var/spack/repos/spack_repo/builtin/packages/m4/gnulib-pgi.patch
applies to: builtin.m4
``spack resource show`` looks up downloadable resources from package
@@ -2219,7 +2219,7 @@ wonder where the extra boost patches are coming from::
^boost@1.68.0%apple-clang@9.0.0+atomic+chrono~clanglibcpp cxxstd=default +date_time~debug+exception+filesystem+graph~icu+iostreams+locale+log+math~mpi+multithreaded~numpy patches=2ab6c72d03dec6a4ae20220a9dfd5c8c572c5294252155b85c6874d97c323199,b37164268f34f7133cbc9a4066ae98fda08adf51e1172223f6a969909216870f ~pic+program_options~python+random+regex+serialization+shared+signals~singlethreaded+system~taggedlayout+test+thread+timer~versionedlayout+wave arch=darwin-highsierra-x86_64
$ spack resource show b37164268
b37164268f34f7133cbc9a4066ae98fda08adf51e1172223f6a969909216870f
path: /home/spackuser/src/spack/var/spack/repos/builtin/packages/dealii/boost_1.68.0.patch
path: /home/spackuser/src/spack/var/spack/repos/spack_repo/builtin/packages/dealii/boost_1.68.0.patch
applies to: builtin.boost
patched by: builtin.dealii
@@ -2930,7 +2930,7 @@ this, Spack provides four different methods that can be overridden in a package:
The Qt package, for instance, uses this call:
.. literalinclude:: _spack_root/var/spack/repos/builtin/packages/qt/package.py
.. literalinclude:: _spack_root/var/spack/repos/spack_repo/builtin/packages/qt/package.py
:pyobject: Qt.setup_dependent_build_environment
:linenos:
@@ -2958,7 +2958,7 @@ variables to be used by the dependent. This is done by implementing
:meth:`setup_dependent_package <spack.package_base.PackageBase.setup_dependent_package>`. An
example of this can be found in the ``Python`` package:
.. literalinclude:: _spack_root/var/spack/repos/builtin/packages/python/package.py
.. literalinclude:: _spack_root/var/spack/repos/spack_repo/builtin/packages/python/package.py
:pyobject: Python.setup_dependent_package
:linenos:
@@ -3785,7 +3785,7 @@ It is usually sufficient for a packager to override a few
build system specific helper methods or attributes to provide, for instance,
configure arguments:
.. literalinclude:: _spack_root/var/spack/repos/builtin/packages/m4/package.py
.. literalinclude:: _spack_root/var/spack/repos/spack_repo/builtin/packages/m4/package.py
:pyobject: M4.configure_args
:linenos:
@@ -4110,7 +4110,7 @@ Shell command functions
Recall the install method from ``libelf``:
.. literalinclude:: _spack_root/var/spack/repos/builtin/packages/libelf/package.py
.. literalinclude:: _spack_root/var/spack/repos/spack_repo/builtin/packages/libelf/package.py
:pyobject: Libelf.install
:linenos:
@@ -4901,7 +4901,7 @@ the one passed to install, only the MPI implementations all set some
additional properties on it to help you out. E.g., in openmpi, you'll
find this:
.. literalinclude:: _spack_root/var/spack/repos/builtin/packages/openmpi/package.py
.. literalinclude:: _spack_root/var/spack/repos/spack_repo/builtin/packages/openmpi/package.py
:pyobject: Openmpi.setup_dependent_package
That code allows the ``openmpi`` package to associate an ``mpicc`` property
@@ -6001,16 +6001,16 @@ with those implemented in the package itself.
* - Parent/Provider Package
- Stand-alone Tests
* - `C
<https://github.com/spack/spack/blob/develop/var/spack/repos/builtin/packages/c>`_
<https://github.com/spack/spack/blob/develop/var/spack/repos/spack_repo/builtin/packages/c>`_
- Compiles ``hello.c`` and runs it
* - `Cxx
<https://github.com/spack/spack/blob/develop/var/spack/repos/builtin/packages/cxx>`_
<https://github.com/spack/spack/blob/develop/var/spack/repos/spack_repo/builtin/packages/cxx>`_
- Compiles and runs several ``hello`` programs
* - `Fortran
<https://github.com/spack/spack/blob/develop/var/spack/repos/builtin/packages/fortran>`_
<https://github.com/spack/spack/blob/develop/var/spack/repos/spack_repo/builtin/packages/fortran>`_
- Compiles and runs ``hello`` programs (``F`` and ``f90``)
* - `Mpi
<https://github.com/spack/spack/blob/develop/var/spack/repos/builtin/packages/mpi>`_
<https://github.com/spack/spack/blob/develop/var/spack/repos/spack_repo/builtin/packages/mpi>`_
- Compiles and runs ``mpi_hello`` (``c``, ``fortran``)
* - :ref:`PythonPackage <pythonpackage>`
- Imports modules listed in the ``self.import_modules`` property with defaults derived from the tarball
@@ -6031,7 +6031,7 @@ maintainers provide additional stand-alone tests customized to the package.
One example of a package that adds its own stand-alone tests to those
"inherited" by the virtual package it provides an implementation for is
the `Openmpi package
<https://github.com/spack/spack/blob/develop/var/spack/repos/builtin/packages/openmpi/package.py>`_.
<https://github.com/spack/spack/blob/develop/var/spack/repos/spack_repo/builtin/packages/openmpi/package.py>`_.
Below are snippets from running and viewing the stand-alone test results
for ``openmpi``:

View File

@@ -9,7 +9,7 @@ Package Repositories (repos.yaml)
=================================
Spack comes with thousands of built-in package recipes in
``var/spack/repos/builtin/``. This is a **package repository** -- a
``var/spack/repos/spack_repo/builtin/``. This is a **package repository** -- a
directory that Spack searches when it needs to find a package by name.
You may need to maintain packages for restricted, proprietary or
experimental software separately from the built-in repository. Spack
@@ -69,7 +69,7 @@ The default ``etc/spack/defaults/repos.yaml`` file looks like this:
.. code-block:: yaml
repos:
- $spack/var/spack/repos/builtin
- $spack/var/spack/repos/spack_repo/builtin
The file starts with ``repos:`` and contains a single ordered list of
paths to repositories. Each path is on a separate line starting with
@@ -78,16 +78,16 @@ paths to repositories. Each path is on a separate line starting with
.. code-block:: yaml
repos:
- /opt/local-repo
- $spack/var/spack/repos/builtin
- /opt/repos/spack_repo/local_repo
- $spack/var/spack/repos/spack_repo/builtin
When Spack interprets a spec, e.g., ``mpich`` in ``spack install mpich``,
it searches these repositories in order (first to last) to resolve each
package name. In this example, Spack will look for the following
packages and use the first valid file:
1. ``/opt/local-repo/packages/mpich/package.py``
2. ``$spack/var/spack/repos/builtin/packages/mpich/package.py``
1. ``/opt/repos/spack_repo/local_repo/packages/mpich/package.py``
2. ``$spack/var/spack/repos/spack_repo/builtin/packages/mpich/package.py``
.. note::
@@ -101,14 +101,15 @@ Namespaces
Every repository in Spack has an associated **namespace** defined in its
top-level ``repo.yaml`` file. If you look at
``var/spack/repos/builtin/repo.yaml`` in the built-in repository, you'll
``var/spack/repos/spack_repo/builtin/repo.yaml`` in the built-in repository, you'll
see that its namespace is ``builtin``:
.. code-block:: console
$ cat var/spack/repos/builtin/repo.yaml
$ cat var/spack/repos/spack_repo/builtin/repo.yaml
repo:
namespace: builtin
api: v2.0
Spack records the repository namespace of each installed package. For
example, if you install the ``mpich`` package from the ``builtin`` repo,
@@ -217,15 +218,15 @@ Suppose you have three repositories: the builtin Spack repo
repo containing your own prototype packages (``proto``). Suppose they
contain packages as follows:
+--------------+------------------------------------+-----------------------------+
| Namespace | Path to repo | Packages |
+==============+====================================+=============================+
| ``proto`` | ``~/proto`` | ``mpich`` |
+--------------+------------------------------------+-----------------------------+
| ``llnl`` | ``/usr/local/llnl`` | ``hdf5`` |
+--------------+------------------------------------+-----------------------------+
| ``builtin`` | ``$spack/var/spack/repos/builtin`` | ``mpich``, ``hdf5``, others |
+--------------+------------------------------------+-----------------------------+
+--------------+-----------------------------------------------+-----------------------------+
| Namespace | Path to repo | Packages |
+==============+===============================================+=============================+
| ``proto`` | ``~/my_spack_repos/spack_repo/proto`` | ``mpich`` |
+--------------+-----------------------------------------------+-----------------------------+
| ``llnl`` | ``/usr/local/repos/spack_repo/llnl`` | ``hdf5`` |
+--------------+-----------------------------------------------+-----------------------------+
| ``builtin`` | ``$spack/var/spack/repos/spack_repo/builtin`` | ``mpich``, ``hdf5``, others |
+--------------+-----------------------------------------------+-----------------------------+
Suppose that ``hdf5`` depends on ``mpich``. You can override the
built-in ``hdf5`` by adding the ``llnl`` repo to ``repos.yaml``:
@@ -233,8 +234,8 @@ built-in ``hdf5`` by adding the ``llnl`` repo to ``repos.yaml``:
.. code-block:: yaml
repos:
- /usr/local/llnl
- $spack/var/spack/repos/builtin
- /usr/local/repos/spack_repo/llnl
- $spack/var/spack/repos/spack_repo/builtin
``spack install hdf5`` will install ``llnl.hdf5 ^builtin.mpich``.
@@ -243,9 +244,9 @@ If, instead, ``repos.yaml`` looks like this:
.. code-block:: yaml
repos:
- ~/proto
- /usr/local/llnl
- $spack/var/spack/repos/builtin
- ~/my_spack_repos/spack_repo/proto
- /usr/local/repos/spack_repo/llnl
- $spack/var/spack/repos/spack_repo/builtin
``spack install hdf5`` will install ``llnl.hdf5 ^proto.mpich``.
@@ -326,8 +327,8 @@ files, use ``spack repo list``.
$ spack repo list
==> 2 package repositories.
myrepo ~/myrepo
builtin ~/spack/var/spack/repos/builtin
myrepo v2.0 ~/my_spack_repos/spack_repo/myrepo
builtin v2.0 ~/spack/var/spack/repos/spack_repo/builtin
Each repository is listed with its associated namespace. To get the raw,
merged YAML from all configuration files, use ``spack config get repos``:
@@ -335,9 +336,9 @@ merged YAML from all configuration files, use ``spack config get repos``:
.. code-block:: console
$ spack config get repos
repos:srepos:
- ~/myrepo
- $spack/var/spack/repos/builtin
repos:
- ~/my_spack_repos/spack_repo/myrepo
- $spack/var/spack/repos/spack_repo/builtin
Note that, unlike ``spack repo list``, this does not include the
namespace, which is read from each repo's ``repo.yaml``.
@@ -351,66 +352,54 @@ yourself; you can use the ``spack repo create`` command.
.. code-block:: console
$ spack repo create myrepo
$ spack repo create ~/my_spack_repos myrepo
==> Created repo with namespace 'myrepo'.
==> To register it with spack, run this command:
spack repo add ~/myrepo
spack repo add ~/my_spack_repos/spack_repo/myrepo
$ ls myrepo
$ ls ~/my_spack_repos/spack_repo/myrepo
packages/ repo.yaml
$ cat myrepo/repo.yaml
$ cat ~/my_spack_repos/spack_repo/myrepo/repo.yaml
repo:
namespace: 'myrepo'
api: v2.0
By default, the namespace of a new repo matches its directory's name.
You can supply a custom namespace with a second argument, e.g.:
Namespaces can also be nested, which can be useful if you have
multiple package repositories for an organization. Spack will
create the corresponding directory structure for you:
.. code-block:: console
$ spack repo create myrepo llnl.comp
$ spack repo create ~/my_spack_repos llnl.comp
==> Created repo with namespace 'llnl.comp'.
==> To register it with spack, run this command:
spack repo add ~/myrepo
spack repo add ~/my_spack_repos/spack_repo/llnl/comp
$ cat myrepo/repo.yaml
$ cat ~/my_spack_repos/spack_repo/llnl/comp/repo.yaml
repo:
namespace: 'llnl.comp'
You can also create repositories with custom structure with the ``-d/--subdirectory``
argument, e.g.:
.. code-block:: console
$ spack repo create -d applications myrepo apps
==> Created repo with namespace 'apps'.
==> To register it with Spack, run this command:
spack repo add ~/myrepo
$ ls myrepo
applications/ repo.yaml
$ cat myrepo/repo.yaml
repo:
namespace: apps
subdirectory: applications
api: v2.0
^^^^^^^^^^^^^^^^^^
``spack repo add``
^^^^^^^^^^^^^^^^^^
Once your repository is created, you can register it with Spack with
``spack repo add``:
``spack repo add``. You nee to specify the path to the directory that
contains the ``repo.yaml`` file.
.. code-block:: console
$ spack repo add ./myrepo
$ spack repo add ~/my_spack_repos/spack_repo/llnl/comp
==> Added repo with namespace 'llnl.comp'.
$ spack repo list
==> 2 package repositories.
llnl.comp ~/myrepo
builtin ~/spack/var/spack/repos/builtin
llnl.comp v2.0 ~/my_spack_repos/spack_repo/llnl/comp
builtin v2.0 ~/spack/var/spack/repos/spack_repo/builtin
This simply adds the repo to your ``repos.yaml`` file.
@@ -432,46 +421,43 @@ By namespace:
.. code-block:: console
$ spack repo rm llnl.comp
==> Removed repository ~/myrepo with namespace 'llnl.comp'.
==> Removed repository ~/my_spack_repos/spack_repo/llnl/comp with namespace 'llnl.comp'.
$ spack repo list
==> 1 package repository.
builtin ~/spack/var/spack/repos/builtin
builtin ~/spack/var/spack/repos/spack_repo/builtin
By path:
.. code-block:: console
$ spack repo rm ~/myrepo
==> Removed repository ~/myrepo
$ spack repo rm ~/my_spack_repos/spack_repo/llnl/comp
==> Removed repository ~/my_spack_repos/spack_repo/llnl/comp
$ spack repo list
==> 1 package repository.
builtin ~/spack/var/spack/repos/builtin
builtin ~/spack/var/spack/repos/spack_repo/builtin
--------------------------------
Repo namespaces and Python
--------------------------------
You may have noticed that namespace notation for repositories is similar
to the notation for namespaces in Python. As it turns out, you *can*
treat Spack repositories like Python packages; this is how they are
implemented.
Package repositories are implemented as Python packages. To be precise,
they are `namespace packages
<https://packaging.python.org/en/latest/guides/packaging-namespace-packages/>`_
with ``spack_repo`` the top-level namespace, followed by the repository
namespace as submodules. For example, the builtin repository corresponds
to the Python module ``spack_repo.builtin.packages``.
You could, for example, extend a ``builtin`` package in your own
This structure allows you to extend a ``builtin`` package in your own
repository:
.. code-block:: python
from spack.pkg.builtin.mpich import Mpich
from spack_repo.builtin.packages.mpich.package import Mpich
class MyPackage(Mpich):
...
Spack repo namespaces are actually Python namespaces tacked on under
``spack.pkg``. The search semantics of ``repos.yaml`` are actually
implemented using Python's built-in `sys.path
<https://docs.python.org/2/library/sys.html#sys.path>`_ search. The
:py:mod:`spack.repo` module implements a custom `Python importer
<https://docs.python.org/2/library/imp.html>`_.
Spack populates ``sys.path`` at runtime with the path to the root of your
package repository's ``spack_repo`` directory.

View File

@@ -1055,8 +1055,8 @@ def setup_dependent_build_environment(
) -> None:
# NB: This function is overwritten by 'mpi' provider packages:
#
# var/spack/repos/builtin/packages/intel-mpi/package.py
# var/spack/repos/builtin/packages/intel-parallel-studio/package.py
# var/spack/repos/spack_repo/builtin/packages/intel_mpi/package.py
# var/spack/repos/spack_repo/builtin/packages/intel_parallel_studio/package.py
#
# They call _setup_dependent_env_callback() as well, but with the
# dictionary kwarg compilers_of_client{} present and populated.

View File

@@ -10,11 +10,13 @@
import re
import sys
from html import escape
from typing import Type
import llnl.util.tty as tty
from llnl.util.tty.colify import colify
import spack.deptypes as dt
import spack.package_base
import spack.repo
from spack.cmd.common import arguments
from spack.version import VersionList
@@ -139,10 +141,10 @@ def name_only(pkgs, out):
tty.msg("%d packages" % len(pkgs))
def github_url(pkg):
def github_url(pkg: Type[spack.package_base.PackageBase]) -> str:
"""Link to a package file on github."""
url = "https://github.com/spack/spack/blob/develop/var/spack/repos/builtin/packages/{0}/package.py"
return url.format(pkg.name)
mod_path = pkg.__module__.replace(".", "/")
return f"https://github.com/spack/spack/blob/develop/var/spack/{mod_path}.py"
def rows_for_ncols(elts, ncols):

View File

@@ -59,7 +59,7 @@ def is_package(f):
packages, since we allow `from spack import *` and poking globals
into packages.
"""
return f.startswith("var/spack/repos/") and f.endswith("package.py")
return f.startswith("var/spack/") and f.endswith("package.py")
#: decorator for adding tools to the list

View File

@@ -56,8 +56,9 @@
# read-only things in $spack/var/spack
repos_path = os.path.join(var_path, "repos")
packages_path = os.path.join(repos_path, "builtin")
mock_packages_path = os.path.join(repos_path, "builtin.mock")
test_repos_path = os.path.join(var_path, "test_repos")
packages_path = os.path.join(repos_path, "spack_repo", "builtin")
mock_packages_path = os.path.join(test_repos_path, "builtin.mock")
#
# Writable things in $spack/var/spack

View File

@@ -15,7 +15,7 @@
@pytest.fixture()
def builder_test_repository(config):
builder_test_path = os.path.join(spack.paths.repos_path, "builder.test")
builder_test_path = os.path.join(spack.paths.test_repos_path, "builder.test")
with spack.repo.use_repositories(builder_test_path) as mock_repo:
yield mock_repo

View File

@@ -143,8 +143,8 @@ def test_list_count():
def test_list_repos():
with spack.repo.use_repositories(
os.path.join(spack.paths.repos_path, "builtin.mock"),
os.path.join(spack.paths.repos_path, "builder.test"),
os.path.join(spack.paths.test_repos_path, "builtin.mock"),
os.path.join(spack.paths.test_repos_path, "builder.test"),
):
total_pkgs = len(list().strip().split())
mock_pkgs = len(list("-r", "builtin.mock").strip().split())

View File

@@ -159,8 +159,12 @@ def test_bad_root(tmpdir):
def test_style_is_package(tmpdir):
"""Ensure the is_package() function works."""
assert spack.cmd.style.is_package("var/spack/repos/builtin/packages/hdf5/package.py")
assert spack.cmd.style.is_package("var/spack/repos/builtin/packages/zlib/package.py")
assert spack.cmd.style.is_package(
"var/spack/repos/spack_repo/builtin/packages/hdf5/package.py"
)
assert spack.cmd.style.is_package(
"var/spack/repos/spack_repo/builtin/packages/zlib/package.py"
)
assert not spack.cmd.style.is_package("lib/spack/spack/spec.py")
assert not spack.cmd.style.is_package("lib/spack/external/pytest.py")

View File

@@ -29,7 +29,7 @@ def _concretize_with_reuse(*, root_str, reused_str):
@pytest.fixture
def runtime_repo(mutable_config):
repo = os.path.join(spack.paths.repos_path, "compiler_runtime.test")
repo = os.path.join(spack.paths.test_repos_path, "compiler_runtime.test")
with spack.repo.use_repositories(repo) as mock_repo:
yield mock_repo

View File

@@ -2332,7 +2332,7 @@ def test_select_lower_priority_package_from_repository_stack(
from cli.
"""
# 'builtin.mock" and "duplicates.test" share a 'gmake' package
additional_repo = os.path.join(spack.paths.repos_path, "duplicates.test")
additional_repo = os.path.join(spack.paths.test_repos_path, "duplicates.test")
with spack.repo.use_repositories(additional_repo, override=False):
s = spack.concretize.concretize_one(spec_str)
@@ -2576,7 +2576,7 @@ def test_correct_external_is_selected_from_packages_yaml(self, mutable_config):
@pytest.fixture()
def duplicates_test_repository():
repository_path = os.path.join(spack.paths.repos_path, "duplicates.test")
repository_path = os.path.join(spack.paths.test_repos_path, "duplicates.test")
with spack.repo.use_repositories(repository_path) as mock_repo:
yield mock_repo
@@ -2811,7 +2811,7 @@ def test_adding_specs(self, input_specs, default_mock_concretization):
@pytest.fixture()
def edges_test_repository():
repository_path = os.path.join(spack.paths.repos_path, "edges.test")
repository_path = os.path.join(spack.paths.test_repos_path, "edges.test")
with spack.repo.use_repositories(repository_path) as mock_repo:
yield mock_repo

View File

@@ -46,7 +46,7 @@
@pytest.fixture
def test_repo(mutable_config, monkeypatch, mock_stage):
repo_dir = pathlib.Path(spack.paths.repos_path) / "flags.test"
repo_dir = pathlib.Path(spack.paths.test_repos_path) / "flags.test"
with spack.repo.use_repositories(str(repo_dir)) as mock_repo_path:
yield mock_repo_path

View File

@@ -28,7 +28,7 @@ def update_packages_config(conf_str):
@pytest.fixture
def test_repo(mutable_config, monkeypatch, mock_stage):
repo_dir = pathlib.Path(spack.paths.repos_path) / "requirements.test"
repo_dir = pathlib.Path(spack.paths.test_repos_path) / "requirements.test"
with spack.repo.use_repositories(str(repo_dir)) as mock_repo_path:
yield mock_repo_path

View File

@@ -914,7 +914,7 @@ def test_single_file_scope(config, env_yaml):
assert spack.config.get("config:checksum") is True
assert spack.config.get("config:checksum") is True
assert spack.config.get("packages:externalmodule:buildable") is False
assert spack.config.get("repos") == ["/x/y/z", "$spack/var/spack/repos/builtin"]
assert spack.config.get("repos") == ["/x/y/z", "$spack/var/spack/repos/spack_repo/builtin"]
def test_single_file_scope_section_override(tmpdir, config):
@@ -950,7 +950,7 @@ def test_single_file_scope_section_override(tmpdir, config):
# from the lower config scopes
assert spack.config.get("config:checksum") is True
assert not spack.config.get("packages:externalmodule")
assert spack.config.get("repos") == ["/x/y/z", "$spack/var/spack/repos/builtin"]
assert spack.config.get("repos") == ["/x/y/z", "$spack/var/spack/repos/spack_repo/builtin"]
def test_write_empty_single_file_scope(tmpdir):

View File

@@ -1,2 +1,2 @@
repos:
- $spack/var/spack/repos/builtin
- $spack/var/spack/repos/spack_repo/builtin

View File

@@ -219,7 +219,7 @@ def test_use_repositories_and_import():
"""Tests that use_repositories changes the import search too"""
import spack.paths
repo_dir = pathlib.Path(spack.paths.repos_path)
repo_dir = pathlib.Path(spack.paths.test_repos_path)
with spack.repo.use_repositories(str(repo_dir / "compiler_runtime.test")):
import spack.pkg.compiler_runtime.test.gcc_runtime

View File

@@ -99,12 +99,12 @@ archspec = ["archspec"]
llnl = ["llnl"]
[tool.ruff.lint.per-file-ignores]
"var/spack/repos/*/package.py" = ["F403", "F405", "F811", "F821"]
"var/spack/*/package.py" = ["F403", "F405", "F811", "F821"]
"*-ci-package.py" = ["F403", "F405", "F821"]
[tool.black]
line-length = 99
include = '(lib/spack|var/spack/repos)/.*\.pyi?$|bin/spack$'
include = '(lib/spack|var/spack/repos|var/spack/test_repos)/.*\.pyi?$|bin/spack$'
extend-exclude = 'lib/spack/external'
skip_magic_trailing_comma = true
@@ -126,8 +126,16 @@ src_paths = "lib"
honor_noqa = true
[tool.mypy]
files = ['lib/spack/llnl/**/*.py', 'lib/spack/spack/**/*.py', './var/spack/repos/builtin/packages/*/package.py']
mypy_path = ['bin', 'lib/spack', 'lib/spack/external', 'var/spack/repos/builtin']
files = [
"lib/spack/llnl/**/*.py",
"lib/spack/spack/**/*.py",
"var/spack/repos/spack_repo/builtin/packages/*/package.py"
]
mypy_path = [
"lib/spack",
"lib/spack/external",
"var/spack/repos",
]
allow_redefinition = true
# This and a generated import file allows supporting packages
@@ -144,7 +152,7 @@ ignore_missing_imports = true
ignore_missing_imports = false
[[tool.mypy.overrides]]
module = 'packages.*'
module = 'spack_repo.*'
ignore_errors = false
ignore_missing_imports = false
# we can't do this here, not a module scope option, in spack style instead
@@ -197,7 +205,7 @@ ignore_missing_imports = true
useLibraryCodeForTypes = true
reportMissingImports = true
reportWildcardImportFromLibrary = false
include = ['lib/spack', 'var/spack/repos']
include = ['lib/spack', 'var/spack/repos', 'var/spack/test_repos']
ignore = ['lib/spack/external']
extraPaths = ['lib/spack', 'lib/spack/external']

View File

@@ -40,7 +40,9 @@ spack -p --lines 20 spec mpileaks%gcc
$coverage_run $(which spack) bootstrap status --dev --optional
# Check that we can import Spack packages directly as a first import
$coverage_run $(which spack) python -c "import spack.pkg.builtin.mpileaks; repr(spack.pkg.builtin.mpileaks.Mpileaks)"
# TODO: this check is disabled, because sys.path is only updated once
# spack.repo.PATH.get_pkg_class is called.
# $coverage_run $(which spack) python -c "import spack.pkg.builtin.mpileaks; repr(spack.pkg.builtin.mpileaks.Mpileaks)"
#-----------------------------------------------------------
# Run unit tests with code coverage

View File

@@ -1 +0,0 @@
../../builtin/packages/compiler-wrapper/

View File

@@ -1 +0,0 @@
../../builtin/packages/gcc-runtime

View File

@@ -1 +0,0 @@
../../builtin/packages/compiler-wrapper/

View File

@@ -1 +0,0 @@
../../builtin/packages/compiler-wrapper/

View File

@@ -5,7 +5,8 @@
import numbers
from spack.package import *
from spack.pkg.builtin.boost import Boost
from ..boost.package import Boost
def is_multiple_32(x):

View File

@@ -5,9 +5,10 @@
import inspect
import os
import spack.pkg.builtin.openfoam as openfoam
from spack.package import *
from ..openfoam import package as openfoam
class Additivefoam(Package):
"""AdditiveFOAM is a heat and mass transfer software for Additive Manufacturing (AM)"""
@@ -39,7 +40,7 @@ class Additivefoam(Package):
def add_extra_files(self, common, local_prefix, local):
"""Copy additional common and local files into the stage.source_path
from the openfoam/common and the package/assets directories,
respectively. Modified from `spack.pkg.builtin.openfoam.add_extra_files()`.
respectively. Modified from `..openfoam.package.OpenFoam.add_extra_files()`.
"""
outdir = self.stage.source_path
indir = join_path(os.path.dirname(inspect.getfile(openfoam)), "common")

Some files were not shown because too many files have changed in this diff Show More