Compilers and linker optimize string constants for space by aliasing
them when one is a suffix of another. For gcc / binutils this happens
already at -O1, due to -fmerge-constants. This means that we have
to take care during relocation to always preserve a certain length
of the suffix of those prefixes that are C-strings.
In this commit we pick length 7 as a safe suffix length, assuming the
suffix is typically the 7 characters from the hash (i.e. random), so
it's unlikely to alias with any string constant used in the sources.
In general we now pad shortened strings from the left with leading
dir seperators, but in the case of C-strings that are much shorter
and don't share a common suffix (due to projections), we do allow
shrinking the C-string, appending a null, and retaining the old part
of the prefix.
Also when rewiring, we ensure that the new hash preserves the last
7 bytes of the old hash.
Co-authored-by: Harmen Stoppels <harmenstoppels@gmail.com>
A user may want to set some attributes on a package without actually modifying the package (e.g. if they want to git pull updates to the package without conflicts). This PR adds a per-package configuration section called "set", which is a dictionary of attribute names to desired values. For example:
packages:
openblas:
package_attributes:
submodules: true
git: "https://github.com/myfork/openblas"
in this case, the package will always retrieve git submodules, and will use an alternate location for the git repo.
While git, url, and submodules are the attributes for which we envision the most usage, this allows any attribute to be overridden, and the acceptable values are any value parseable from yaml.
Newer versions of the CrayPE for EX systems have standalone compiler executables for CCE and compiler wrappers for Cray MPICH. With those, we can treat the cray systems as part of the linux platform rather than having a separate cray platform.
This PR:
- [x] Changes cray platform detection to ignore EX systems with Craype version 21.10 or later
- [x] Changes the cce compiler to be detectable via paths
- [x] Changes the spack compiler wrapper to understand the executable names for the standalone cce compiler (`craycc`, `crayCC`, `crayftn`).
For some instances of externally-provided Python (e.g. Homebrew),
the LDLIBRARY/LIBRARY config variables don't actually refer to
libraries and should therefore be excluded from ".libs".
Only enable the hdf5-vfd-gds package if it can compile.
- hdf5-vfd-gds needs cuda@11.7.1+ to be able to `find_library` for cuFile.
- Only enable hdf5-vfd-gds in the sdk if cuda@11.7.1+ is available.
If an earlier version of cuda is being used, do not depend on the
hdf5-vfd-gds package at all.
* take two
* Add missing import statement
* Group dependencies together
* Extract libtiff arguments
* Extract libpng arguments
* Push preamble variable into png_args and tiff_args
* Extract setting args associated with the screenshot variant
* Inlined a few variables
* Modify only build targets and install targets
Co-authored-by: Massimiliano Culpo <massimiliano.culpo@gmail.com>
Whenever the rpath string actually _grows_, it falls back to patchelf,
when it stays the same length or gets shorter, we update it in-place,
padded with null bytes.
This PR only deals with absolute -> absolute rpath replacement. We don't
use `_build_tarball(relative=True)` in our CI. If `relative` then it falls
back to the old replacement code.
With this PR, relocation time goes down significantly, likely because patchelf
does some odd things with mmap, causing lots of overhead. Example:
- `binutils`: 700MB installed, goes from `1.91s` to `0.57s`, or `3.4x` faster.
Relocation time: 27% -> 10% of total install time
- `llvm`: 6.8GB installed, goes from `28.56s` to `5.38`, or `5.3x` faster.
Relocation time: 44% -> 13% of total install time
The bottleneck is now decompression.
Note: I'm somewhat confused about the "relative rpath" code paths. Right
now this PR only deals with absolute -> absolute replacement. As far as
I understand, if you embrace relative rpaths when uploading to the
buildcache, the whole point is you _don't_ want to patch rpaths on
install? So it seems fine to not expand `$ORIGIN` again imho.
When a package asks for non-parallel make, we need to force `make -j1` because just doing `make` will run in parallel under jobserver (e.g. `spack env depfile`).
We now always add `-j1` when asked for a non-parallel execution (even if there is no jobserver).
And each `MakeExecutable` can now ask for jobserver support or not. For example: the default `ninja` does not support jobserver so spack applies the default `-j`, but `ninja@kitware` or `ninja-fortran` does, so spack doesn't add `-j`.
Tips: you can run `SPACK_INSTALL_FLAGS=-j1 make -f spack-env-depfile.make -j8` to avoid massive job-spawning because of build tools that don't support jobserver (ninja).
* testing ssh key
* test
* LR : Creating the packge to install the gegelati app
* LR : Gegelati, a TPG C++ library added and fully tested
* LR : adjusting for fork
* LR: taking out the boilerplates
* LR: taking out the rest
We try to avoid non-default variant values in the concretizer, but this doesn't make
sense for variants forced to take some non-default value by variant propagation.
Counting this as a penalty effectively biases the concretizer for small specs dependency
graphs -- something we try very hard to avoid elsewhere because it can lead to very
strange decisions.
Example: with the penalty, `spack spec hdf5` will choose the default `openmpi` as its
`mpi` provider, but `spack spec hdf5 ~~shared` will choose `mpich` because it has to set
fewer non-default variant values because `mpich`'s DAG is smaller. That's not a good
reason to prefer a non-default virtual provider.
To fix this, if the user explicitly requests a non-default value to be propagated, there
shouldn't be a penalty. Variant values set on the CLI already don't count as default; we
just need to extend that to propagated values.
Adds another post install hook that loops over the install prefix, looking for shared libraries type of ELF files, and sets the soname to their own absolute paths.
The idea being, whenever somebody links against those libraries, the linker copies the soname (which is the absolute path to the library) as a "needed" library, so that at runtime the dynamic loader realizes the needed library is a path which should be loaded directly without searching.
As a result:
1. rpaths are not used for the fixed/static list of needed libraries in the dynamic section (only for _actually_ dynamically loaded libraries through `dlopen`), which largely solves the issue that Spack's rpaths are a heuristic (`<prefix>/lib` and `<prefix>/lib64` might not be where libraries really are...)
2. improved startup times (no library search required)
Untouched spec pruning was added to reduce the number of specs
developers see getting rebuilt in their PR pipelines that they
don't understand. Because the state of the develop mirror lags
quite far behind the tip of the develop branch, PRs often find
they need to rebuild things untouched by their PR.
Untouched spec pruning was previously implemented by finding all
specs in the environment with names of packages touched by the PR,
traversing in both directions the DAGS of those specs, and adding
all dependencies as well as dependents to a list of concrete specs
that should not be considered for pruning.
We found that this heuristic results in too many pruned specs, and
that dependents of touched specs must have all their dependencies
added to the list of specs that should not be considered for pruning.
* SEACAS: Update package.py to handle new SEACAS project name
The base project name for the SEACAS project has changed from
"SEACASProj" to "SEACAS" as of @2022-10-14, so the package
needed to be updated to use the new project name when needed.
The refactor also changes several:
"-DSome_CMAKE_Option:BOOL=ON"
to
define("Some_CMAKE_Option", True)
* SEACAS: Additional refactorings
* Replaced all cmake "-Dsomething=other" lines with either `define`
or `define_from_variant` functions.
Consolidated the application (fortran, legacy, all) enabling lines
into loops over the code names. Easier to see categorization of
applications and also to add/move/remove an application
Reordered some lines; general cleanup and restructuring.
* Address flake8 issues
* Remove trailing whitespace
* Reformat using black
* add new package: py-pylatex
* fix bugs
* add extras indicated in setup.py
* Update var/spack/repos/builtin/packages/py-pylatex/package.py
Co-authored-by: Adam J. Stewart <ajstewart426@gmail.com>
* Update var/spack/repos/builtin/packages/py-pylatex/package.py
Co-authored-by: Adam J. Stewart <ajstewart426@gmail.com>
* improvements
* remove git merge related lines
* tidy
* Update var/spack/repos/builtin/packages/py-pylatex/package.py
Co-authored-by: Adam J. Stewart <ajstewart426@gmail.com>
* remove variant
* [@spackbot] updating style on behalf of Sinan81
Co-authored-by: sbulut <sbulut@3vgeomatics.com>
Co-authored-by: Adam J. Stewart <ajstewart426@gmail.com>
Co-authored-by: Sinan81 <Sinan81@users.noreply.github.com>
This issue was introduced in #29761:
```
==> Installing ncurses-6.3-22hz6q6cvo3ep2uhrs3erpp2kogxncbn
==> No binary for ncurses-6.3-22hz6q6cvo3ep2uhrs3erpp2kogxncbn found: installing from source
==> Using cached archive: /spack/var/spack/cache/_source-cache/archive/97/97fc51ac2b085d4cde31ef4d2c3122c21abc217e9090a43a30fc5ec21684e059.tar.gz
==> No patches needed for ncurses
==> ncurses: Executing phase: 'autoreconf'
==> ncurses: Executing phase: 'configure'
==> ncurses: Executing phase: 'build'
==> ncurses: Executing phase: 'install'
==> Error: AttributeError: 'str' object has no attribute 'propagate'
The 'ncurses' package cannot find an attribute while trying to build from sources. This might be due to a change in Spack's package format to support multiple build-systems for a single package. You can fix this by updating the build recipe, and you can also report the issue as a bug. More information at https://spack.readthedocs.io/en/latest/packaging_guide.html#installation-procedure
/spack/lib/spack/spack/build_environment.py:1075, in _setup_pkg_and_run:
1072 tb_string = traceback.format_exc()
1073
1074 # build up some context from the offending package so we can
>> 1075 # show that, too.
1076 package_context = get_package_context(tb)
1077
1078 logfile = None
```
It turns out this was caused by a bug that had been around much longer, in which the flags were passed by reference to the flag_handler, and the flag_handler was modifying the spec object, not just the flags given to the build system. The scope of this bug was limited by the forking model in Spack, which is how it went under the radar for so long.
PR includes regression test.
* remove deptype_query remnants
* deptypes -> deptype
These arguments haven't existed since 2017, but `traverse` now fails on unknown **kwargs, so they have finally popped up.
The base project name for the SEACAS project has changed from
"SEACASProj" to "SEACAS" as of @2022-10-14, so the package
needed to be updated to use the new project name when needed.
The refactor also changes several:
"-DSome_CMAKE_Option:BOOL=ON"
to
define("Some_CMAKE_Option", True)
This updates the propagation logic used in `concretize.lp` to avoid rules with `path()`
in the body and instead base propagation around `depends_on()`.
Currently, compiler flags and variants are inconsistent: compiler flags set for a
package are inherited by its dependencies, while variants are not. We should have these
be consistent by allowing for inheritance to be enabled or disabled for both variants
and compiler flags.
- [x] Make new (spec language) operators
- [x] Apply operators to variants and compiler flags
- [x] Conflicts currently result in an unsatisfiable spec
(i.e., you can't propagate two conflicting values)
What I propose is using two of the currently used sigils to symbolized that the variant
or compiler flag will be inherited:
Example syntax:
- `package ++variant`
enabled variant that will be propagated to dependencies
- `package +variant`
enabled variant that will NOT be propagated to dependencies
- `package ~~variant`
disabled variant that will be propagated to dependencies
- `package ~variant`
disabled variant that will NOT be propagated to dependencies
- `package cflags==True`
`cflags` will be propagated to dependencies
- `package cflags=True`
`cflags` will NOT be propagated to dependencies
Syntax for string-valued variants is similar to compiler flags.
Fixes an issue on the RHEL8 UBI container where this test would fail because `gr_mem`
was empty for every entry in the `grp` DB.
You have to check *both* the `pwd` database (which has primary groups) and `grp` (which
has other gorups) to do this correctly.
- [x] update `llnl.util.filesystem.group_ids()` to do this
- [x] use it in the `sbang` test
This PR introduces breadth-first traversal, and moves depth-first traversal
logic out of Spec's member functions, into `traverse.py`.
It introduces a high-level API with three main methods:
```python
spack.traverse.traverse_edges(specs, kwargs...)
spack.traverse.traverse_nodes(specs, kwags...)
spack.traverse.traverse_tree(specs, kwargs...)
```
with the usual `root`, `order`, `cover`, `direction`, `deptype`, `depth`, `key`,
`visited` kwargs for the first two.
What's new is that `order="breadth"` is added for breadth-first traversal.
The lower level API is not exported, but is certainly useful for advanced use
cases. The lower level API includes visitor classes for direction reversal and
edge pruning, which can be used to create more advanced traversal methods,
especially useful when the `deptype` is not constant but depends on the node
or depth.
---
There's a couple nice use-cases for breadth-first traversal:
- Sometimes roots have to be handled differently (e.g. follow build edges of
roots but not of deps). BFS ensures that root nodes are always discovered at
depth 0, instead of at any depth > 1 as a dep of another root.
- When printing a tree, it would be nice to reduce indent levels so it fits in the
terminal, and ensure that e.g. `zlib` is not printed at indent level 10 as a
dependency of a build dep of a build dep -- rather if it's a direct dep of my
package, I wanna see it at depth 1. This basically requires one breadth-first
traversal to construct a tree, which can then be printed with depth-first traversal.
- In environments in general, it's sometimes inconvenient to have a double
loop: first over the roots then over each root's deps, and maintain your own
`visited` set outside. With BFS, you can simply init the queue with the
environment root specs and it Just Works. [Example here](3ec7304699/lib/spack/spack/environment/environment.py (L1815-L1816))
Currently, many tests hardcode to older versions of gcc for comparisons of
concretization among compiler versions. Those versions are too old to concretize for
`aarch64`-family targets, which leads to failing tests on `aarch64`.
This PR fixes those tests by updating the compiler versions used for testing.
Currently, many tests hardcode the expected architecture result in concretization to the
`x86_64` family of architectures.
This PR generalizes the tests that can be generalized, to cover multiple architecture
families. For those that test specific relationships among `x86_64`-family targets, it
ensures that concretization uses the `x86_64`-family targets in those cases.
Currently, many tests rely on the fact that `AutotoolsPackage` imposes no dependencies
on the inheriting package. That is not true on `aarch64`-family architectures.
This PR ensures that the fact `AutotoolsPackage` on `aarch64` pulls in a dependency on
`gnuconfig` is ignored when testing for the appropriate relationships among dependencies
Additionally, 5 tests currently prompt the user for input when `gpg` is available in the
user's path. This PR fixes that issue. And 7 tests fail currently when the user has a
yubikey available. This PR fixes the incorrect gpg argument causing those issues.
The `spack info <package>` command does not show the `Virtual Packages:` output unless the `--virtuals` command option is passed. Before this changes, the information that the command is supposed to be illustrating is not shown in the example and is confusing.
* julia: don't look for the openlibm libraries when unneeded
Cause spack to *not* check for the existence of the openlibm libraries (by adding it to the pkgs list) when ~openlibm is specified.
* [@spackbot] updating style on behalf of downloadico
Co-authored-by: downloadico <downloadico@users.noreply.github.com>
- the updated OpenFOAM wmake rules now allow multiple locations for
compiler flags:
* wmake/General/common/c++Opt [central]
* wmake/linux64Gcc/c++Opt [traditional]
- match both '=' and ':=' make rule lines
Co-authored-by: Mark Olesen <Mark.Olesen@esi-group.com>
Changes to improve locating shared libraries on Windows, which in
turn enables the use of Clingo. This PR attempts to establish a
proper distinction between linking on Windows vs. Linux/Mac: on
Windows, linking is always done with .lib files (never .dll files).
This somewhat complicates the model since the Spec.lib method could
return libraries that were used for both linking and loading, but
since these are not always the same on Windows, it was decided to
treat Spec.libs as being for link-time libraries. Additional functions
are added to help dependents locate run-time libraries.
* Clingo is now the default concretizer on Windows
* Clingo is now the concretizer used for unit tests on Windows
* Fix a permissions issue that can occur while moving Git files during
fetching/staging
* Packages can now implement "win_add_library_dependent" to register
files/directories that include libraries that would need to link
to dependency dlls
* Packages can now implement "win_add_rpath" to register the locations
of dlls that dependents would want to load
* "Spec.libs" on Windows is updated to return link-time libraries
(i.e. .lib files, rather than .dll files)
* PackageBase.rpath on Windows is now updated to return the most-likely
locations where .dlls will be found (which is generally in the bin/
directory)
* Nalu-Wind: Allow for standard versions of trilinos
This will allow us to utilize custom numeric versions for trilinos in `spack-manager` while we continue to develop `nalu-wind`.
Pinging @eugeneswalker @jrood-nrel @tasmith4
* Update var/spack/repos/builtin/packages/nalu-wind/package.py
Currently there's a slow sequential step in binary relocation where all
strings of a binary are collected, with rpaths removed, and then
filtered for the old install root.
This is completely unnecessary, and also incorrect, since we replace
more than just the old install root in the prefix to prefix mapping. And
in fact the prefix to prefix mapping is parallel, and a single pass. So
even as an optimization, this filter makes no sense anymore.
Therefor we remove it
- single pass over the binary data matching all prefixes
- collect offsets and replacement strings
- do in-place updates with `fseek` / `fwrite`, since typically our
replacement touch O(few bytes) while the file is O(many megabytes)
- be nice: leave the file untouched if some string can't be
replaced
* Added py-medaka and dependencies
* fixed py-parasail build error
* medaka still doesn't have correct linked libdeflate
* fixed pyspoa deps
* added htslib.patch, confirmed builds and runs
* fixed style
* Update var/spack/repos/builtin/packages/py-auditwheel/package.py
Co-authored-by: Adam J. Stewart <ajstewart426@gmail.com>
* made requested changes
* added targets for pyspoa dep
Co-authored-by: Adam J. Stewart <ajstewart426@gmail.com>
/home/xsdk/spack.x/lib/spack/env/oneapi/icx -DAdd_ -Dscalapack_EXPORTS -I/opt/intel/oneapi/mpi/2021.7.0/include -O3 -DNDEBUG -fPIC -MD -MT CMakeFiles/scalapack.dir/BLACS/SRC/dgamx2d_.c.o -MF CMakeFiles/scalapack.dir/BLACS/SRC/dgamx2d_.c.o.d -o CMakeFiles/scalapack.dir/BLACS/SRC/dgamx2d_.c.o -c /home/xsdk/spack.x/spack-stage/spack-stage-netlib-scalapack-2.2.0-uj3jepiowz5is4hmdmjrzjltetgdr3lx/spack-src/BLACS/SRC/dgamx2d_.c
/home/xsdk/spack.x/spack-stage/spack-stage-netlib-scalapack-2.2.0-uj3jepiowz5is4hmdmjrzjltetgdr3lx/spack-src/BLACS/SRC/igsum2d_.c:154:7: error: call to undeclared function 'BI_imvcopy'; ISO C99 and later do not support implicit function declarations [-Wimplicit-function-declaration]
BI_imvcopy(Mpval(m), Mpval(n), A, tlda, bp->Buff);
^
Currently the vasp package always enables the use of shmem to reduce algorithm memory usage (see
https://www.vasp.at/wiki/index.php/Precompiler_options). This is great,but on some systems gives compile errors with the interoperability of C and Fortran. This PR makes that shmem flag optional, but retains the
existing default on behavior.
* Added support for building the DiHydrogen package and LBANN extensions
to DiHydrogen with ROCm libraries.
Fixed a bug on Cray systems where CMake didn't try hard enough to find
an MPI-compatible compiler wrapper. Make it look more.
Added support for the roctracer package when using ROCm libraries.
* Fixed how ROCm support is defined for pre-v0.3 versions.
- hdf5-vfd-gds:
- Add new version 1.0.2 compatible with hdf5@1.13.
- CMake is a build dependency.
- Set `HDF5_PLUGIN_PATH` in the runtime environment, this plugin
is loaded dynamically.
- SDK:
- The VFD GDS driver only has utility when CUDA is enabled.
- Require hdf5-vfd-gds@1.0.2+ (1.0.1 and earlier do not compile).
* Add patches for building clingo with MSVC
* Help python find clingo DLL
* If an executable is located in "C:\Program Files", Executable was
running into issues with the extra space. This quotes the exe
to ensure that it is treated as a single value.
Signed-off-by: Kiruya Momochi <65301509+KiruyaMomochi@users.noreply.github.com>
This commit extends the DSL that can be used in packages
to allow declaring that a package uses different build-systems
under different conditions.
It requires each spec to have a `build_system` single valued
variant. The variant can be used in many context to query, manipulate
or select the build system associated with a concrete spec.
The knowledge to build a package has been moved out of the
PackageBase hierarchy, into a new Builder hierarchy. Customization
of the default behavior for a given builder can be obtained by
coding a new derived builder in package.py.
The "run_after" and "run_before" decorators are now applied to
methods on the builder. They can also incorporate a "when="
argument to specify that a method is run only when certain
conditions apply.
For packages that do not define their own builder, forwarding logic
is added between the builder and package (methods not found in one
will be retrieved from the other); this PR is expected to be fully
backwards compatible with unmodified packages that use a single
build system.
* Updating package file for osu-micro-benchmarks for the 6.2 release
* updating sha hash for 6.2 tarball
Co-authored-by: natshineman <shineman.5@osu.edu>
* Add netcdf-c 4.9.0 and netcdf-fortran 4.6.0
With v4.9.0 netcdf-c introduces zstandard compression option which is added as a variant.
* Fix when= in dependency
* Turn on variant zstd by default
Co-authored-by: kgerheiser <kgerheiser@icloud.com>
* alquimia, pflotran, plasma, py-mpi4py, strumpack - add in new versions
* Fix hip CI failure
Co-authored-by: eugeneswalker <eugenesunsetwalker@gmail.com>
Instead of looping over multiple regexes and the entire text file
contents, create a giant regex with all literal prefixes and do a single
pass over files to detect prefixes. Not only is a single pass faster,
it's also likely that the regex is compiled better, given that most
prefixes share a common ... prefix.
In the dfs code, flip edges so that `parent` means `from` and
`spec` means `to` in the direction of traversal. This makes it slightly
easier to write generic/composable code. For example when using visitors
where one visitor reverses direction, and another only cares about
accepting particular edges or not depending on whether the target node
is seen before, it would be good if the second visitor didn't have to
know whether the order was changed or not.
Use the same compression level as `gzip` (6) instead of what Python uses
(9).
The LLVM tarball takes 4m instead of 12m to create, and is <1% larger.
That's not worth the wait...
* udunits: Update download URL
* udunits: Deprecate older versions
Unidata now only provides the latest version of each X.Y branch. Older 2.2 versions have been deprecated accordingly but are still available in the build cache.
Co-authored-by: RemiLacroix-IDRIS <RemiLacroix-IDRIS@users.noreply.github.com>
Co-authored-by: Tamara Dahlgren <35777542+tldahlgren@users.noreply.github.com>
* Update wgrib2 from JCSDA/NOAA-EMC fork
* var/spack/repos/builtin/packages/wgrib2/package.py: fix typo in comment, add conflict for variants netcdf3, netcdf4
* wget hdf5/netcdf4 internal dependencies for wgrib2
* Black-format var/spack/repos/builtin/packages/wgrib2/package.py
* More format changes in var/spack/repos/builtin/packages/wgrib2/package.py
#32137 added an option to update() a BinaryCacheIndex with a
cooldown: repeated attempts within this cooldown would not
actually retry. However, the cooldown was not properly
tracked for failures (which is common when the mirror
does not store any binaries and therefore has no index.json).
This commit ensures that update(..., with_cooldown=True) will
also skip the update even if a failure has occurred within the
cooldown period.
Due to reuse concretization, we may generate DAGs with two occurrences
of the same package corresponding to distinct specs. This happens when
build dependencies are reused, since their dependencies are ignored in
concretization.
This caused a regression, for example: `spec['openssl']` would take the
'openssl' of the build dep `cmake`, instead of the direct `openssl`
dependency, simply because the edge to `cmake` was traversed first and
we do depth first traversal.
One solution that was discussed is to limit `spec[name]` to just direct
deps, or direct deps + transitive link deps, but this is too breaking.
Instead, this PR simply prioritizes transitive link and direct
build/run/test deps, and then falls back to a full DAG traversal. So,
it's just about order of iteration.
* Update py-sphinxcontrib-mermaid
* Add py-myst-parser
* Fix py-mdit-py-plugins and py-myst-parser dependencies
* Add py-exhale package
* Update var/spack/repos/builtin/packages/py-mdit-py-plugins/package.py
Co-authored-by: Adam J. Stewart <ajstewart426@gmail.com>
* Update var/spack/repos/builtin/packages/py-myst-parser/package.py
Co-authored-by: Adam J. Stewart <ajstewart426@gmail.com>
* Update var/spack/repos/builtin/packages/py-myst-parser/package.py
Co-authored-by: Adam J. Stewart <ajstewart426@gmail.com>
* Update var/spack/repos/builtin/packages/py-myst-parser/package.py
Co-authored-by: Adam J. Stewart <ajstewart426@gmail.com>
* Update py-exhale and py-myst-parser dependencies
* Add @svenevs as py-exhale maintainer
* Update var/spack/repos/builtin/packages/py-mdit-py-plugins/package.py
Co-authored-by: Adam J. Stewart <ajstewart426@gmail.com>
Co-authored-by: Adam J. Stewart <ajstewart426@gmail.com>
Scan the text files for relocatable prefixes *before* creating a tarball,
to reduce the amount of work to be done during install from binary
cache.
Co-authored-by: Harmen Stoppels <harmenstoppels@gmail.com>
* py-drep: new package
* fixed file extension
* added darwin conflict
* py-checkm-genome and py-pysam: bumped version and updated deps (#10)
added checkm and pysam deps
* added dep documentation and fixed style
* changed checkm and pysam back to dev version for upstreaming
* added url and perl run dep
* fixed style
Instead of showing
```
==> Error: Timed out waiting for a write lock.
```
show
```
==> Error: Timed out waiting for a write lock after 1.200ms and 4 attempts on file: /some/file
```
s.t. we actually get to see where acquiring a lock failed even when not
running in debug mode.
And use pretty time units everywhere, so we don't get 1.45e-9 seconds
but 1.450ns etc.
* backtraces without --debug
Currently `--debug` is too verbose and not-`--debug` gives to little
context about where exceptions are coming from.
So, instead, it'd be nice to have `spack --backtrace` and
`SPACK_BACKTRACE=1` as methods to get something inbetween: no verbose
debug messages, but always a full backtrace.
This is useful for CI, where we don't want to drown in debug messages
when installing deps, but we do want to get details where something goes
wrong if it goes wrong.
* completion
* acts: new versions
In the 20.x release line, these are the changes, https://github.com/acts-project/acts/compare/v20.0.0...v20.2.0
- `option(ACTS_SETUP_ACTSVG "Build ActSVG display plugin" OFF)` introduced in v20.1.0
- `option(ACTS_USE_SYSTEM_ACTSVG "Use the ActSVG system library" OFF)` introduced in v20.1.0
- `option(ACTS_BUILD_PLUGIN_ACTSVG "Build SVG display plugin" OFF)` introduced in v20.1.0
- `option(ACTS_USE_EXAMPLES_TBB "Use Threading Building Blocks library in examples" ON)` introduced in v20.1.0
- `option(ACTS_EXATRKX_ENABLE_ONNX "Build the Onnx backend for the exatrkx plugin" OFF)` introduced in v20.2.0
- `option(ACTS_EXATRKX_ENABLE_TORCH "Build the torchscript backend for the exatrkx plugin" ON)` introduced in v20.2.0
In the 19.x release line, these are the changes: https://github.com/acts-project/acts/compare/v19.7.0...v19.9.0
- `option(ACTS_USE_EXAMPLES_TBB "Use Threading Building Blocks library in examples" ON)` introduced in v19.8.0
The new build options have not been implemented in this commit but will be implemented next.
* acts: new variant svg
* actsvg: new package
* actsvg: style fixes
* acts: new versions 20.3.0 and 19.10.0
* astsvg: depends_on boost googletest
* actsvg: new version 0.4.26 (and style fix)
Includes fix to build issue when +examples, https://github.com/acts-project/actsvg/pull/23
* acts: new variant tbb when +examples @19.8:19 @20.1:
* acts: set ACTS_USE_EXAMPLES_TBB
* acts: no need for ACTS_SETUP_ACTSVG
* acts: move tbb variant to examples block
* acts: ACTS_USE_SYSTEM_ACTSDD4HEP removed in 20.3
* acts: use new ACTS_USE_SYSTEM_LIBS
* acts-dd4hep: new version 1.0.1, maintainer handle fixed
* acts: simplify variant tbb condition
* py-checkm-genome and py-pysam: bumped version and updated deps
* updated setuptools dep type
Co-authored-by: Adam J. Stewart <ajstewart426@gmail.com>
Co-authored-by: Adam J. Stewart <ajstewart426@gmail.com>
When we lose a running pod (possibly loss of spot instance) or encounter
some other infrastructure-related failure of this job, we need to retry
it. This retries the job the maximum number of times in those cases.
Currently `relocate_text` and `relocate_text_bin` are unsafe in the
sense that they run in parallel, and lead to races when modifying
different items pointing to the same inode.
This leads to the issue observed in #33453.
This PR:
1. Renames those functions to `unsafe_*` so people are aware
2. Adds logic to deal with hardlinks in current binary packages
3. Adds logic to deal with hardlinks when creating new binary tarballs,
so the install side doesn't have to de-dupe hardlinks.
4. Adds a test for 3
The assumption is that all our relocation logic preserves inodes. That
is, we should never copy a file, modify it, and then move it back. I
quickly verified, and its seems like this is true for (binary) text
relocation, as well as rpath patching in patchelf (even when the file
grows) and mach-o fixes.
* axom@0.7.0: require cmake@3.21:
* Update var/spack/repos/builtin/packages/axom/package.py
Co-authored-by: Chris White <white238@llnl.gov>
Co-authored-by: Chris White <white238@llnl.gov>
`reuse` and `when_possible` concretization broke the invariant that
`spec[pkg_name]` has unique keys. This invariant is relied on in tons of
places, such as when setting up the build environment.
When using `when_possible` concretization, one may end up with two or
more `perl`s or `python`s among the transitive deps of a spec, because
concretization does not consider build-only deps of reusable specs.
Until the code base is fixed not to rely on this broken property of
`__getitem__`, we should disable reuse in CI.
* fixed version numbers to python 2 and old biopython
* changed shortbred pacakge to pypi, removed python 2 version
* added package description
* re-added shortbred package with depreciated flag
* fixed style and removed unnecessary python dep (it can't build with python 2 anyway)
* removed whitespace and readded the python2.7.9+ dep
* fixed style
* gitlab: Do not use root_spec['pkg_name'] anymore
For a long time it was fine to index a concrete root spec with the name
of a dependency in order to access the concrete dependency spec. Since
pipelines started using `--use-buildcache dependencies:only,package:never`
though, it has exposed a scheduling issue in how pipelines are
generated. If a concrete root spec depends on two different hashes of
`openssl` for example, indexing that root with just the package name
is ambiguous, so we should no longer depend on that approach when
scheduling jobs.
* env: make sure exactly one spec in env matches hash
When installing some/all specs from a buildcache, build edges are pruned
from those specs. This can result in a much smaller effective DAG. Until
now, `spack env depfile` would always generate a full DAG.
Ths PR adds the `spack env depfile --use-buildcache` flag that was
introduced for `spack install` before. This way, not only can we drop
build edges, but also we can automatically set the right buildcache
related flags on the specific specs that are gonna get installed.
This way we get parallel installs of binary deps without redundancy,
which is useful for Gitlab CI.
When downloading from binary cache not only replace RPATHs to dependencies, but
also text references to dependencies.
Example:
`autoconf@2.69` contains a text reference to the executable of its dependency
`perl`:
```
$ grep perl-5 /shared/spack/opt/spack/linux-amzn2-x86_64_v3/gcc-7.3.1/autoconf-2.69-q3lo/bin/autoreconf
eval 'case $# in 0) exec /shared/spack/opt/spack/linux-amzn2-x86_64_v3/gcc-7.3.1/perl-5.34.1-yphg/bin/perl -S "$0";; *) exec /shared/spack/opt/spack/linux-amzn2-x86_64_v3/gcc-7.3.1/perl-5.34.1-yphg/bin/perl -S "$0" "$@";; esac'
```
These references need to be replace or any package using `autoreconf` will fail
as it cannot find the installed `perl`.
Co-authored-by: Stephen Sachs <stesachs@amazon.com>
"spack install" will not update the binary index if given a concrete
spec, which causes it to fall back to direct fetches when a simple
index update would have helped. For S3 buckets in particular, this
significantly and needlessly slows down the install process.
This commit alters the logic so that the binary index is updated
whenever a by-hash lookup fails. The lookup is attempted again with
the updated index before falling back to direct fetches. To avoid
updating too frequently (potentially once for each spec being
installed), BinaryCacheIndex.update now includes a "cooldown"
option, and when this option is enabled it will not update more
than once in a cooldown window (set in config.yaml).
Co-authored-by: Tamara Dahlgren <35777542+tldahlgren@users.noreply.github.com>
Without this patch one hits this error trying to compiler papi with Intel OneAPI:
icx: error: Note that use of '-g' without any optimization-level option will turn off most compiler optimizations similar to use of '-O0' [-Werror,-Wdebug-disables-optimization]
Signed-off-by: Howard Pritchard <howardp@lanl.gov>
Signed-off-by: Howard Pritchard <howardp@lanl.gov>
* Fixed two bugs in the HIP package recipe. The first is that the
HIP_PATH was being set to the actual spec, and not the spec prefix.
The second bug is that HIP is expected to be in /opt/rocm-x.y.z/hip
but it's libraries can exist at both /opt/rocm-x.y.z/hip/lib and
/opt/rocm-x.y.z/lib. This means that the external detection logic may
find it in either and it turns out that some modules only expose one
of those two locations. Logic is added to ensure that the internal
HIP_PATH and associated ROCM_PATH are correctly set in both cases.
* Added support for Aluminum to use the libfabric plugin with either
RCCL or NCCL.
* Add libpressio and dependencies; some of these packages are
maintained as forks of the original repositories and in those
cases the docstring mentions this.
* Add optional dependency in adios2 on libpressio
* cub package: set CUB_DIR environment variable for dependent
installations
* Clear R_HOME/R_ENVIRON before Spack installation (avoid sources
outside of Spack from affecting the installation in Spack)
* Rename dlib to dorian3d-dlib and update dependents; add new dlib
implementation. Pending an official policy on how to handle
packages with short names, reviewer unilaterally decided that
the rename was acceptable given that the new Spack dlib package
is referenced more widely (by orders of magnitude) than the
original
Co-authored-by: Samuel Li <shaomeng@users.noreply.github.com>
* Classic Intel compilers do not support gcc-toolchain
This fix removes `--gcc-toolchain=` from the ~.fcg` files for the classic Intel
compilers. AFAIK this option is only supported for Clang based compilers.
This lead to an issue when installing cmake. Reproducer:
```
spack install cmake@3.24.2%intel@2021.7.0~doc+ncurses+ownlibs~qt
build_type=Release arch=linux-amzn2-skylake_avx512
```
Tagging maintainer @rscohn2
* Add `-gcc-name` for icc
.. and `-gxx-name` for icpc.
AFAIK this is used for modern C++ support, so we can ignore `ifort`.
Co-authored-by: Stephen Sachs <stesachs@amazon.com>
When installing an individual spec `spack --only=package --cache-only /xyz`
from a buildcache, Spack currently issues tons of warnings about
missing build deps (and their deps) in the database.
This PR disables these warnings, since it's fine to have a spec without
its build deps in the db (they are just "missing").
Currently `traverse_dependencies` fixes deptypes to traverse once and
for all in the recursion, but this is incorrect, since deptypes depend
on the node (e.g. if it's a dependency and cache-only, don't follow
build type edges, even if the parent is build from sources and needs
build deps.)
Support spackbot rebuilding all specs from source when asked (with "rebuild everything")
- Allow overriding --prune-dag cli opt with env var
- Use job variable to optionally prevent rebuild jobs early exit behavior
- ci rebuild: Use new install argument to insist deps are always installed from binary, but
package is only installed from source.
- gitlab: fix bug w/ untouched pruning
- ci rebuild: install from hash rather than json file
- When doing a "rebuild everything" pipeline, make sure that each install job only consumes
binary dependencies from the mirror being populated by the current pipeline. This avoids
using, e.g. binaries from develop, when rebuilding everything on a PR.
- When running a pipeline to rebuild everything, do not die because we generated a hash on
the broken specs list. Instead only warn in that case.
- bugfix: Replace broken no-args tty.die() with sys.exit(1)
Print a message of the form
```
Fetch mm:ss. Build: mm:ss. Total: mm:ss
```
when installing from buildcache.
Previously this only happened for source builds.
Currently "spack ci generate" chooses the first matching entry in
gitlab-ci:mappings to fill attributes for a generated build-job,
requiring that the entire configuration matrix is listed out
explicitly. This unfortunately causes significant problems in
environments with large configuration spaces, for example the
environment in #31598 (spack.yaml) supports 5 operating systems,
3 architectures and 130 packages with explicit size requirements,
resulting in 1300 lines of configuration YAML.
This patch adds a configuraiton option to the gitlab-ci schema called
"match_behavior"; when it is set to "merge", all matching entries
are applied in order to the final build-job, allowing a few entries
to cover an entire matrix of configurations.
The default for "match_behavior" is "first", which behaves as before
this commit (only the runner attributes of the first match are used).
In addition, match entries may now include a "remove-attributes"
configuration, which allows matches to remove tags that have been
aggregated by prior matches. This only makes sense to use with
"match_behavior:merge". You can combine "runner-attributes" with
"remove-attributes" to effectively override prior tags.
* petsc,py-petsc4py,slepc,py-slepc4py: add version 3.18.0
* workaround for dealii build failure [with petsc version check]
* pism: add compatibility fix to for petsc@3.18
* add in hipsolver dependency
* Add checksum for py-gitpython 3.1.27
* Update package.py
* Update var/spack/repos/builtin/packages/py-gitpython/package.py
Co-authored-by: Adam J. Stewart <ajstewart426@gmail.com>
Co-authored-by: Adam J. Stewart <ajstewart426@gmail.com>
When a pipeline generation job is automatically failed because it
generated jobs for specs known to be broken on develop, print better
information about the broken specs that were encountered. Include
at a minimum the hash and the url of the job whose failure caused it
to be put on the broken specs list in the first place.
* new package + deps
* Update var/spack/repos/builtin/packages/py-about-time/package.py
Co-authored-by: Adam J. Stewart <ajstewart426@gmail.com>
* Update var/spack/repos/builtin/packages/py-alive-progress/package.py
Co-authored-by: Adam J. Stewart <ajstewart426@gmail.com>
* removed unnecessary python version dep
Co-authored-by: Adam J. Stewart <ajstewart426@gmail.com>
* add new lua releases
* split install phase and move it into a build phase, remove hardcoded standard flag
* revert back to the original hardcoded std flag, guard patch against versions above 5.4
* py-pyopenssl: add version 22.1.0
* Update var/spack/repos/builtin/packages/py-pyopenssl/package.py
Co-authored-by: Adam J. Stewart <ajstewart426@gmail.com>
Co-authored-by: Adam J. Stewart <ajstewart426@gmail.com>
* Add checksum for py-regex 2022.8.17
* Update var/spack/repos/builtin/packages/py-regex/package.py
Co-authored-by: Adam J. Stewart <ajstewart426@gmail.com>
Co-authored-by: Adam J. Stewart <ajstewart426@gmail.com>
* Add checksum for py-async-lru 1.0.3
* [@spackbot] updating style on behalf of iarspider
* Update var/spack/repos/builtin/packages/py-async-lru/package.py
Co-authored-by: Adam J. Stewart <ajstewart426@gmail.com>
* Update package.py
Co-authored-by: iarspider <iarspider@users.noreply.github.com>
Co-authored-by: Adam J. Stewart <ajstewart426@gmail.com>
* env depfile: allow deps only install
- Refactor `spack env depfile` to use a Jinja template, making it a bit
easier to follow as a human being.
- Add a layer of indirection in the generated Makefile through an
`<prefix>/.install-deps/<hash>` target, which allows one to specify
different options when installing dependencies. For example, only
verbose/debug mode on when installing some particular spec:
```
$ spack -e my_env env depfile -o Makefile --make-target-prefix example
$ make example/.install-deps/<hash> -j16
$ make example/.install/<hash> SPACK="spack -d" SPACK_INSTALL_FLAGS=--verbose -j16
```
This could be used to speed up `spack ci rebuild`:
- Parallel install of dependencies from buildcache
- Better readability of logs, e.g. reducing verbosity when installing
dependencies, and splitting logs into deps.log and current_spec.log
* Silence please!
* Add bioc attribute to r-do-db
* add version 1.38.1 to bioconductor package r-annotationforge
* add version 1.30.4 to bioconductor package r-biocparallel
* add version 2.64.1 to bioconductor package r-biostrings
* add version 4.4.4 to bioconductor package r-clusterprofiler
* add version 2.12.1 to bioconductor package r-complexheatmap
* add version 1.18.1 to bioconductor package r-delayedmatrixstats
* add version 3.22.1 to bioconductor package r-dose
* add version 3.38.4 to bioconductor package r-edger
* add version 1.16.2 to bioconductor package r-enrichplot
* add version 2.20.2 to bioconductor package r-ensembldb
* add version 1.32.4 to bioconductor package r-genomeinfodb
* add version 1.32.1 to bioconductor package r-genomicalignments
* add version 1.48.4 to bioconductor package r-genomicfeatures
* add version 1.44.1 to bioconductor package r-ggbio
* add version 3.4.4 to bioconductor package r-ggtree
* add version 1.24.2 to bioconductor package r-hdf5array
* add version 2.30.1 to bioconductor package r-iranges
* add version 1.36.3 to bioconductor package r-keggrest
* add version 3.52.4 to bioconductor package r-limma
* add version 1.8.1 to bioconductor package r-matrixgenerics
* update r-org-hs-eg-db
* add version 1.38.1 to bioconductor package r-organismdbi
* add version 1.36.1 to bioconductor package r-pathview
* add version 1.56.1 to bioconductor package r-rtracklayer
* add version 1.4.1 to bioconductor package r-scaledmatrix
* add version 1.24.1 to bioconductor package r-scran
* add version 1.6.3 to bioconductor package r-scuttle
* add version 1.18.1 to bioconductor package r-singlecellexperiment
* add version 1.20.2 to bioconductor package r-treeio
* Revert "Add bioc attribute to r-do-db"
This reverts commit 36be5c6072.
* Fix quotes on versions
Co-authored-by: Massimiliano Culpo <massimiliano.culpo@gmail.com>
Co-authored-by: Tamara Dahlgren <35777542+tldahlgren@users.noreply.github.com>
For older versions of intel-oneapi-compilers, running the compiler in
preprocessor / compilation mode would trigger warnings that
`-Wl,-rpath,...` flags were unused.
This in turn caused certain configure scripts to fail as they did not
expect output from the compiler (it's treated as an error). Notably
cmake's bootstrap phase failed to detect c++ features of the compiler.
As a workaround, add this flag to silence the warning, since I don't
think we can scope the flags to compile+link mode.
* bump version for libvterm, required by neovim
* bump version for neovim and add related dep constraints
see release note:
d367ed9b23
in particular:
'deps: Bump required libvterm to v0.3'
https://github.com/neovim/neovim/pull/20222
* spack.compiler.Compiler: introduce prefix property
We currently don't really have something that gives the GCC install
path, which is used by many LLVM-based compilers (llvm, llvm-amdgpu,
nvhpc, ...) to fix the GCC toolchain once and for all.
This `prefix` property is dynamic in the sense that it queries the
compiler itself. This is necessary because it's not easy to deduce the
install path from the `cc` property (might be a symlink, might be a
filename like `gcc` which works by having the compiler load a module
that sets the PATH variable, might be a generic compiler wrapper based
on environment variables like on cray...).
With this property introduced, we can clean up some recipes that have
the logic repeated for GCC.
* intel-oneapi-compilers: set --gcc-sysroot to %gcc prefix
Caches used by repositories don't reference the global spack.repo.path instance
anymore, but get the repository they refer to during initialization.
Spec.virtual now use the index, and computation done to compute the index
use Repository.is_virtual_safe.
Code to construct mock packages and mock repository has been factored into
a unique MockRepositoryBuilder that is used throughout the codebase.
Add debug print for pushing and popping config scopes.
Changed spack.repo.use_repositories so that it can override or not previous repos
spack.repo.use_repositories updates spack.config.config according to the modifications done
Removed a peculiar behavior from spack.config.Configuration where push would always
bubble-up a scope named command_line if it existed
Resolves#31782
With this change, if a spec is concrete after parsing (e.g. spec.yaml
or /hash-based), then it is not disambiguated (a process which requires
(a) that the spec be installed and (b) that it be part of the
currently-active environment).
This commit allows you to:
* Diff specs from an environment regardless of whether they have
been installed (more useful for projection/matrix-based envs)
* Diff specs read from .yaml files which may or may not be entirely
different installations of Spack
Co-authored-by: Massimiliano Culpo <massimiliano.culpo@gmail.com>
* hpctoolkit: add version 2022.10.01
1. add version 2022.10.01
2. remove version for master branch, develop is now the main branch
3. add CPATH and LD_LIBRARY_PATH to module run environment,
this is for apps that want to use the start/stop interface
4. cleanup style in variants, depends and conflicts
5. remove all-static variant, nothing uses it
6. deprecate more old versions
* [@spackbot] updating style on behalf of mwkrentel
* Add when(+level_zero) to the gtpin variant.
* Test commit to see if this passes E4S.
* Another test commit to see if E4S succeeds.
* Add temporary hack to ignore +mpi for version 2022.10.01 and issue a
warning instead.
Co-authored-by: mwkrentel <mwkrentel@users.noreply.github.com>
* Add checksum for py-msgpack 1.0.4
* Update var/spack/repos/builtin/packages/py-msgpack/package.py
Co-authored-by: Adam J. Stewart <ajstewart426@gmail.com>
Co-authored-by: Adam J. Stewart <ajstewart426@gmail.com>
* Add checksum for py-virtualenv 20.16.4
* Add checksum for py-werkzeug 2.2.2
* Restore py-virtualenv/package.py
* Update var/spack/repos/builtin/packages/py-werkzeug/package.py
Co-authored-by: Adam J. Stewart <ajstewart426@gmail.com>
Co-authored-by: Adam J. Stewart <ajstewart426@gmail.com>
* Docs: Getting Started Dependencies
Finally document what one needs to install to use Spack on
Linux and Mac :-)
With <3 for minimal container users and my colleagues with
their fancy Macs.
* Debian Update Packages: GCC, Python
- build-essential: includes gcc, g++ (thx Cory)
- Python: add python3-venv, python3-distutils (thx Pradyun)
* Add RHEL8 Dependencies
* Add checksum for py-skl2onnx 1.12
* Update var/spack/repos/builtin/packages/py-skl2onnx/package.py
Co-authored-by: Adam J. Stewart <ajstewart426@gmail.com>
* Update package.py
Co-authored-by: Adam J. Stewart <ajstewart426@gmail.com>
* Add checksum for py-tables 3.7.0
* Update package.py
* Update var/spack/repos/builtin/packages/py-tables/package.py
Co-authored-by: Adam J. Stewart <ajstewart426@gmail.com>
Co-authored-by: Adam J. Stewart <ajstewart426@gmail.com>
ROOT has a webgui which is available with the `+root7` variant. This is a fairly large part of a ROOT install (275MB out of 732MB on my system) which is not necessarily useful in all use cases (e.g. inside containers on network-restricted HPC/HTC compute nodes). This new variant adds the option to retain the ROOT7 functionality but not necessarily include the `webgui` aspects.
`__unused__` defined in `general.h` conflict with the one defined by libc headers,
so change it to `__attribute__unused__` according to s.zharkoff:
https://bugs.gentoo.org/828550#c11
cmd:
`grep -rl "__unused__" . | xargs -n1 sed -i -e 's/\b__unused__\b/__attribute__unused__/g' -e 's/(unused)/(__unused__)/g'`
Basic stack of ML packages we would like to test and generate binaries for in CI.
Spack now has a large CI framework in GitLab for PR testing and public binary generation.
We should take advantage of this to test and distribute optimized binaries for popular ML
frameworks.
This is a pretty extensive initial set, including CPU, ROCm, and CUDA versions of a core
`x96_64_v4` stack.
### Core ML frameworks
These are all popular core ML frameworks already available in Spack.
- [x] PyTorch
- [x] TensorFlow
- [x] Scikit-learn
- [x] MXNet
- [x] CNTK
- [x] Caffe
- [x] Chainer
- [x] XGBoost
- [x] Theano
### ML extensions
These are domain libraries and wrappers that build on top of core ML libraries
- [x] Keras
- [x] TensorBoard
- [x] torchvision
- [x] torchtext
- [x] torchaudio
- [x] TorchGeo
- [x] PyTorch Lightning
- [x] torchmetrics
- [x] GPyTorch
- [x] Horovod
### ML-adjacent libraries
These are libraries that aren't specific to ML but are still core libraries used in ML pipelines
- [x] numpy
- [x] scipy
- [x] pandas
- [x] ONNX
- [x] bazel
Co-authored-by: Jonathon Anderson <17242663+blue42u@users.noreply.github.com>
* Add checksum for py-scikit-build 0.15.0 and use sources from pypi
* Update var/spack/repos/builtin/packages/py-scikit-build/package.py
Co-authored-by: Adam J. Stewart <ajstewart426@gmail.com>
Co-authored-by: Adam J. Stewart <ajstewart426@gmail.com>
* Add checksum for py-terminado 0.15.0
* Update var/spack/repos/builtin/packages/py-terminado/package.py
Co-authored-by: Adam J. Stewart <ajstewart426@gmail.com>
Co-authored-by: Adam J. Stewart <ajstewart426@gmail.com>
* added metaphlan v4, cleaned up phylophlan
* added iqtree2
* fixed phylophlan, builds now
* changed config.yaml to default
* fixed style
* py-jsonschema: add 4.16.0 and new package py-hatch-fancy-pypi-readme (#32929)
* acfl: add v22.1 (#32915)
Co-authored-by: Annop Wongwathanarat <annop.wongwathanarat@arm.com>
* Fixup errors introduced by Clingo Pr: (#32905)
* re2c depends on cmake on Windows
* Winbison properly added to bootstrap package search list
* Set CMAKE_HIP_ARCHITECTURES with the value of amdgpu_target (#32901)
* libtiff: default to +zlib+jpeg (#32945)
* octave: add version 7.2.0 (#32943)
* simgrid new releases (#32920)
* [rocksdb] Added rtti variant (#32918)
* rvs binary path updated for 5.2 rocm release (#32892)
* Add checksum for py-pytest-runner 6.0.0 (#32957)
* py-einops: add v0.5.0 (#32959)
* Replace repo with the NVIDIA one (#32951)
* Add checksum for py-tomli 2.0.1 (#32949)
* QMCPACK: add @3.15.0 (#32931)
* Tidied up configure arguments to use special spack autotools features. (#32930)
* casper: old domain fell off, adding github repo (#32928)
* unifyfs: pin mercury version; add boost variant (#32911)
Mercury has a new version (v2.2) releasing soon that UnifyFS does not build with and hasn't been tested with. This pins UnifyFS to the last version of Mercury used/tested.
Add a variant to avoid building/using boost
Append -std=gnu99 to cflags if building with gcc@4. Needed for mochi-margo to compile
* trilinos: constrain superlu-dist version (#32889)
* trilinos: constrain superlu-dist version for 13.x
* syntax
* FEniCSx: Updates for 0.5.1 (#32665)
* Updates for DOLFINx 0.5.1 and associated packages
* xtensor needed on anything less than main
* Switch back to Python 3.7 minimum.
* Might be good to point out in our README how to fix Python version?
* Fix basix, xtensor dep
* Add numba feature
* Fix checksum
* Make slepc optional
Co-authored-by: Tamara Dahlgren <35777542+tldahlgren@users.noreply.github.com>
* simgrid: add variant and remove flag (#32797)
* simgrid: remove std c++11 flag
* simgrid: add msg variant
* Axom: bring in changes from axom repo (#32643)
* bring in changes from axom repo
Co-authored-by: white238 <white238@users.noreply.github.com>
Co-authored-by: Tamara Dahlgren <35777542+tldahlgren@users.noreply.github.com>
* Add checksum for py-pyparsing 3.0.9 (#32952)
* rdma-core: fix syntax for external discoverability (#32962)
* Add checksum for py-flatbuffers 2.0.7 (#32955)
* amrex: add v22.10 (#32966)
* Remove CMakePackage.define alias from most packages (#32950)
* Bug fix for `ca-certificates-mozilla/package.py` to enable `spack install --source` (#32953)
* made suggested changes to iqtree2, py-dendropy, py-metaphlan, and py-pkgconfig. Poetry install still broken
* reverted py-pkgconfig deps to poetry-core
* made iqtree2 less dedundant, changes to py-dendropy and py-pkgconfig deps
Co-authored-by: Manuela Kuhn <36827019+manuelakuhn@users.noreply.github.com>
Co-authored-by: Annop Wongwathanarat <annop.wongwathanarat@gmail.com>
Co-authored-by: Annop Wongwathanarat <annop.wongwathanarat@arm.com>
Co-authored-by: John W. Parent <45471568+johnwparent@users.noreply.github.com>
Co-authored-by: Auriane R <48684432+aurianer@users.noreply.github.com>
Co-authored-by: Adam J. Stewart <ajstewart426@gmail.com>
Co-authored-by: Kai Torben Ohlhus <k.ohlhus@gmail.com>
Co-authored-by: Vinícius <viniciusvgp@gmail.com>
Co-authored-by: Matthieu Dorier <mdorier@anl.gov>
Co-authored-by: renjithravindrankannath <94420380+renjithravindrankannath@users.noreply.github.com>
Co-authored-by: iarspider <iarspider@gmail.com>
Co-authored-by: Paul R. C. Kent <kentpr@ornl.gov>
Co-authored-by: Brian Van Essen <vanessen1@llnl.gov>
Co-authored-by: snehring <7978778+snehring@users.noreply.github.com>
Co-authored-by: Cameron Stanavige <stanavige1@llnl.gov>
Co-authored-by: Cody Balos <balos1@llnl.gov>
Co-authored-by: Jack S. Hale <mail@jackhale.co.uk>
Co-authored-by: Tamara Dahlgren <35777542+tldahlgren@users.noreply.github.com>
Co-authored-by: Lucas Nesi <lucas31nesi@hotmail.com>
Co-authored-by: Chris White <white238@llnl.gov>
Co-authored-by: white238 <white238@users.noreply.github.com>
Co-authored-by: Martin Pokorny <mpokorny@caltech.edu>
Co-authored-by: Weiqun Zhang <WeiqunZhang@lbl.gov>
Co-authored-by: Massimiliano Culpo <massimiliano.culpo@gmail.com>
Co-authored-by: Dom Heinzeller <dom.heinzeller@icloud.com>
* filter_file: introduce argument 'start_at'
* autotools: extend patching of the libtool script
* autotools: refactor _patch_usr_bin_file
* autotools: improve readability of the filtering
* autotools: keep the modification time of the configure scripts
* autotools: do not try to patch directories
* autotools: explain libtool patching for posterity
Co-authored-by: Massimiliano Culpo <massimiliano.culpo@gmail.com>
Co-authored-by: Harmen Stoppels <harmenstoppels@gmail.com>
* Add checksum for py-onnxmltools 1.11.0
* Add checksum for py-onnxmltools 1.11.0
* Fix patch name
* Update var/spack/repos/builtin/packages/py-onnx-runtime/package.py
Co-authored-by: Adam J. Stewart <ajstewart426@gmail.com>
Co-authored-by: Adam J. Stewart <ajstewart426@gmail.com>
* Add checksum for py-pkginfo 1.8.3
* Update var/spack/repos/builtin/packages/py-pkginfo/package.py
Co-authored-by: Adam J. Stewart <ajstewart426@gmail.com>
Co-authored-by: Adam J. Stewart <ajstewart426@gmail.com>
* trilinos and xyce: fix fortran library handling
xyce:
- add pymi_static_blas variant and logic
handles blas and hdf5 conflicts for Xyce-PyMi
- make +isorropia and +zoltan conditional on mpi
* xyce: clean up CMake options
* xyce: change pymi_static_blas to pymi_static_tpls
* xyce: made pymi_static_tpls only when +pymi
Remove `module-info mode load` condition that prevents auto-unloading when autoloading is enabled. It looks like this condition was added to work around an issue in environment-modules that is no longer necessary.
Add quotes to make is-loaded happy
* Add checksum for py-urllib3 1.26.12
* Update var/spack/repos/builtin/packages/py-urllib3/package.py
Co-authored-by: Adam J. Stewart <ajstewart426@gmail.com>
* Fixes from review
Co-authored-by: Adam J. Stewart <ajstewart426@gmail.com>
* Add checksum for py-soupsieve 2.3.2.post1
* Update package.py
* Update var/spack/repos/builtin/packages/py-soupsieve/package.py
Co-authored-by: Adam J. Stewart <ajstewart426@gmail.com>
Co-authored-by: Adam J. Stewart <ajstewart426@gmail.com>
* Update to PyFR v1.15.0
* allow unsupported compilers for cuda
* update requirement for py-gimmik version
* update requirement for cuda version
* update versions, style fix
* lib directory changed, style fixes
* PYFR_METIS_LIBRARY_PATH set
There is a new OpenBLAS release out that can be compiled w/o
a Fortran compiler.
macOS XCode developers, rejoice. Maybe at some point Spack
becomes a package manager that can be used without using
another package manager (to get gfortran) 🎉
phist: add conflict on reference netlib-lapack due to API change in lapack.h
Co-authored-by: Harmen Stoppels <harmenstoppels@gmail.com>
* Added a package for the aws-ofi-nccl plug-in from to enable NCCL to
use libfabric communication library as a network provider.
Co-authored-by: Harmen Stoppels <harmenstoppels@gmail.com>
When concurrent misc_cache provider index rebuilds happen, try to
rebuild it only once, so we don't exceed misc_cache lock timeout.
For example, when using `spack env depfile`, with no previous
misc_cache, running `make -f depfile -j8` could run at most 8 concurrent
`spack install` locking on misc_cache to rebuild the provider index. If
one rebuild takes 30s, before this fix, the "worst" lock could wait up
to 30s * 7, easily exceeding misc_cache lock timeout. Now, the "worst"
lock would take 30s * 1 + ~1s * 6.
* Add checksums for Rivet 3.1.7 and YODA 1.9.7
* Add checksum for fastjet 3.4.0
* Add v3.1.7b...
... in which version requirement for autoconf was lowered to 2.68
Currently, module changes from `setup_dependent_package` are applied only to the module of the package class, but not to any parent classes' modules between the package class module and `spack.package_base`.
In this PR, we create a custom class to accumulate module changes, and apply those changes to each class that requires it. This design allows us to code for a single module, while applying the changes to multiple modules as needed under the hood, without requiring the user to reason about package inheritance.
* find/list: display package counts last
We have over 6,600 packages now, and `spack list` still displays the number of packages
before it lists them all. This is useless for large sets of results (e.g., with no args)
as the number has scrolled way off the screen before you can see it. The same is true
for `spack find` with large installations.
This PR changes `spack find` and `spack list` so that they display the package count
last.
* add some quick testing
Co-authored-by: Danny McClanahan <1305167+cosmicexplorer@users.noreply.github.com>
* Updates for DOLFINx 0.5.1 and associated packages
* xtensor needed on anything less than main
* Switch back to Python 3.7 minimum.
* Might be good to point out in our README how to fix Python version?
* Fix basix, xtensor dep
* Add numba feature
* Fix checksum
* Make slepc optional
Co-authored-by: Tamara Dahlgren <35777542+tldahlgren@users.noreply.github.com>
Mercury has a new version (v2.2) releasing soon that UnifyFS does not build with and hasn't been tested with. This pins UnifyFS to the last version of Mercury used/tested.
Add a variant to avoid building/using boost
Append -std=gnu99 to cflags if building with gcc@4. Needed for mochi-margo to compile
Allow environment variables and spack-specific path substitution variables (e.g. `$spack`) to be
used in the paths associated with develop specs, while maintaining the ability to keep those
paths relative to the environment rather than the working directory.
* Added new versions
* New slate version
* Adding GPU support for lapackpp package
* Modified dependency on lapackpp
* Added rocblas and rocsolver to deps
* Testing with custom lapackpp repo
* Added chaining depends_on for +rocm
* Removing testing repo
* py-ipython: add 8.5.0 and py-win-unicode-console: new package
* Fix style
* Fix dependency version
* Deprecate version 2.3.1 and 3.1.0
* Always skip IPython.kernel
* Move skip_module definition to top
Install: Add use-buildcache option to install
* Allow differentiating between top level packages and dependencies when
determining whether to install from the cache or not.
* Add unit test for --use-buildcache
* Use metavar to display use-buildcache options.
* Update spack-completion
* Added a package for the aws-ofi-rccl plug-in from the ROCm software
stack. It allows RCCL to use the libfabric communication library.
Added support for using libfabric in Aluminum.
* Updated the run environment so that the plugin would get loaded.
* Added support for setting up the the LD_LIBRARY_PATH for dependent packages.
* Added package for RCCL tests to assess the impact of OFI libfabric RCCL plug-in.
* Add tag for xgboost 1.6.2
* Update py-xgboost as well
* Update package.py
* Update var/spack/repos/builtin/packages/py-xgboost/package.py
Co-authored-by: Adam J. Stewart <ajstewart426@gmail.com>
Co-authored-by: Adam J. Stewart <ajstewart426@gmail.com>
* intel-oneapi-compilers-classic: refactor setup_run_environment
* intel-oneapi-compilers-classic: extend setup_run_environment with the one from intel-oneapi-compilers
* py-interface-meta: add 1.3.0 (incl new dependency packages)
* Update var/spack/repos/builtin/packages/py-poetry-dynamic-versioning/package.py
Co-authored-by: Adam J. Stewart <ajstewart426@gmail.com>
Co-authored-by: Adam J. Stewart <ajstewart426@gmail.com>
superlu:oneapi-deal with non ISO C99 complianc in the package.
The Intel OneAPI compilers are based on LLVM 14. A recent enhancement to LLVM -
https://reviews.llvm.org/D122983
results in superlu-dist not compiling because of some non ISO C99 compliant stuff.
A workaround is to use an llvm compile line option noted in the above URL.
Signed-off-by: Howard Pritchard <howardp@lanl.gov>
Make it possible to install the Clingo package on Windows; this
also provides a means to use Clingo with Spack on Windows.
This includes
* A new "winbison" package: Windows has a port of bison and flex where
the two packages are grouped together. Clingo dependencies have been
updated to use winbison on Windows and bison elsewhere (this avoids
complicating the existin bison/flex packages until we can add support
for implied virtuals).
* The CMake build system was incorrectly converting CMAKE_INSTALL_PREFIX
to POSIX format.
* The re2c package has been modified to use CMake on Windows; for now
this is done by overloading the configure/build/install methods to
perform CMake-appropriate operations; the package should be refactored
once support for multiple build systems in one Package is available.
This commit fixes#27027.
The root cause of the issue is that the `SPACK_OLD_PROMPT` variable
was evaluated in string interpolation regardless of whether the
guard condition above evaluates to true or false. This commit uses
the `eval` keyword to defer evaluation until the command is executed.
Co-authored-by: Alexander Hornburg <alexande@xilinx.com>
The file rbg.txt is needed for many PGPLOT application, such as the MESA
stellar evolution code. This change installs the file to the PGPLOT_DIR,
where the library expects it.
PGPLOT_DIR was previously set to the prefix itself, which is an odd
place to install rgb.txt. This commit changes it to lib/pgplot5,
following the convention used by Debian.
Co-authored-by: Philipp Edelmann <edelmann@fs.tum.de>
* py-datalad-metalad: add 0.4.5
* Update var/spack/repos/builtin/packages/py-datalad-metalad/package.py
Co-authored-by: Adam J. Stewart <ajstewart426@gmail.com>
Co-authored-by: Adam J. Stewart <ajstewart426@gmail.com>
In addition to the new release, made intel-oneapi-compilers-classic version number match the compiler version number, instead of the version of the package that contains it.
Co-authored-by: Robert Cohn <robert.s.cohn@intel.com>
* root: make X11 really optional on macOS
* Update var/spack/repos/builtin/packages/root/package.py
* remove when clauses in provides
Co-authored-by: Hadrien G. <knights_of_ni@gmx.com>
This modifications breaks `develop` since it doesn't
pass audits with Python 2.7 It is to be investigated
why audits pass in CI for the PR and the issue is
revealed only when the package is pushed to develop.
PR #32615 deprecated Python versions up to 3.6.X. Since
the "build-systems" pipeline requires Python 3.6.15 to
build "tut", it will fail on the first rebuild that
involves Python.
The "tut" package is meant to perform an end-to-end
test of the "Waf" build-system, which is scarcely
used. The fix therefore is just to remove it from
the pipeline.
Spack currently depends on parsing filenames of downloaded files to
determine what type of archive they are and how to decompress them.
This commit adds a preliminary check based on magic numbers to
determine archive type (but falls back on name parsing if the
extension type cannot be determined).
As part of this work, this commit also enables decompression of
.tar.xz-compressed archives on Windows.
* Update py-poetry to 1.2.1
* Update py-xattr
* Apply style from review
* Apply suggestions from code review (part 1)
Co-authored-by: Adam J. Stewart <ajstewart426@gmail.com>
* Changes from review - 2
* Fix typo
* Fix style
* Add missing py-dulwich version
* Update var/spack/repos/builtin/packages/py-poetry-core/package.py
Co-authored-by: Adam J. Stewart <ajstewart426@gmail.com>
* Update var/spack/repos/builtin/packages/py-poetry/package.py
Co-authored-by: Adam J. Stewart <ajstewart426@gmail.com>
* Update var/spack/repos/builtin/packages/py-poetry/package.py
Co-authored-by: Adam J. Stewart <ajstewart426@gmail.com>
* Apply changes from review
* Add py-backports-cached-property and fix style
* Apply suggestions from code review
Co-authored-by: Adam J. Stewart <ajstewart426@gmail.com>
* Update package.py
Co-authored-by: Adam J. Stewart <ajstewart426@gmail.com>
* py-eccodes: ensure the minimal recommended shared version of libeccodes
* py-eccodes: set less general environment variables to enable location of libeccodes
* py-eccodes: add version 1.5.0
* py-eccodes: make flake8 happy
amazon linux 2 ships a glibc that is too old to work with cuda toolkit
for aarch64.
For example:
`libcurand.so.10.2.10.50` requires the symbol `logf@@GLIBC_2.27`, but
glibc is at 2.26.
So, these specs are removed.
* New packages: py-conan, py-node-semver, py-patch-ng
* Update var/spack/repos/builtin/packages/py-conan/package.py
Co-authored-by: Adam J. Stewart <ajstewart426@gmail.com>
* Update var/spack/repos/builtin/packages/py-conan/package.py
Co-authored-by: Adam J. Stewart <ajstewart426@gmail.com>
Co-authored-by: Adam J. Stewart <ajstewart426@gmail.com>
* py-babel: add 2.10.3
* Update var/spack/repos/builtin/packages/py-babel/package.py
Co-authored-by: Adam J. Stewart <ajstewart426@gmail.com>
Co-authored-by: Adam J. Stewart <ajstewart426@gmail.com>
* New package: py-jaraco-classes
* Update var/spack/repos/builtin/packages/py-jaraco-classes/package.py
Co-authored-by: Adam J. Stewart <ajstewart426@gmail.com>
Co-authored-by: Adam J. Stewart <ajstewart426@gmail.com>
* [py-tinydb] added package py-tinydb
* [py-tinydb] corrected version in dependency for py-tinydb
* [py-tinydb] update python dependency
Co-authored-by: Adam J. Stewart <ajstewart426@gmail.com>
* [py-tinydb] update dependency
Co-authored-by: Adam J. Stewart <ajstewart426@gmail.com>
Co-authored-by: Adam J. Stewart <ajstewart426@gmail.com>
Include exception info related to url retrieval in debug messages
which otherwise would be swallowed. This is intended to be useful
for detecting if CA configuration interferes with downloads from
HTTPS links.
* add the 2 variants OPENCL and HIP and their dependencies correctly
for OPENCL - rocm-opencl, miopengemm and miopen-opencl
for HIP - miopen-hip
Earlier this was adding both the dependencies -miopen-hip and miopen-opencl
for both the backends which did not seem correct.
Also corrected the miopen-hip or miopen-opencl config.h in patch() depending on the
backend
Also added libjpeg-turbo as it is required for building ROCAl .
the AMDRpp is still required for ROCAL inclusion but it currently does not build
AMDRpp will be added as a new spack recipe and the mivisionx will refer to that as a
dependency in future.
* fix style errors
* bump up the version for 5.2.3 release. tested +opencl, +hip and ~hip~opencl(cpu backend)
Co-authored-by: Tamara Dahlgren <35777542+tldahlgren@users.noreply.github.com>
* Initial opencatalyst proxy app spackage
Initial spackage for opencatalyst proxy app. Includes exposing of
versions in dependency spackages
* Verified Functionality and Spack Style
Verified build of mlperf-opencatalyst and fixed lingering spack style
issues
* Making requested changes to py-ocp
Making requested changed to spackage
* Further Requested Changes
Since fmt@9.0.0 and 9.1.0 were [added](6c4acfbf83) to spack a few days ago, gaudi fails to compile with default concretization. Since gaudi developers are usually paying attention to new versions of dependencies, I'm going to assume (perhaps optimistically) that the next bugfix version of gaudi will fix this (even though the issue has not been reported yet to Gaudi; I posted on the [key4hep public mirror](https://github.com/key4hep/Gaudi/issues/1)).
* Add fixed version 0.7.0 to cpu-features
* [@spackbot] updating style on behalf of iarspider
Co-authored-by: iarspider <iarspider@users.noreply.github.com>
* started updating multiqc package
* working now
* added py-rich-click
* fixed style
* changed py-matlibplot versions
Co-authored-by: Adam J. Stewart <ajstewart426@gmail.com>
* changed py-networkx versions
Co-authored-by: Adam J. Stewart <ajstewart426@gmail.com>
* changed py-coloredlogs versions
Co-authored-by: Adam J. Stewart <ajstewart426@gmail.com>
* changed python versions
Co-authored-by: Adam J. Stewart <ajstewart426@gmail.com>
* changed py-markdown versions
Co-authored-by: Adam J. Stewart <ajstewart426@gmail.com>
* changed py-pyyaml requirement
Co-authored-by: Adam J. Stewart <ajstewart426@gmail.com>
* changed py-requests requirements
Co-authored-by: Adam J. Stewart <ajstewart426@gmail.com>
* changed py-spectra requirements
Co-authored-by: Adam J. Stewart <ajstewart426@gmail.com>
Co-authored-by: Adam J. Stewart <ajstewart426@gmail.com>
* Bump up the version for rocm-5.2.1-initial commit
* Bump up the version for rocm-5.2.1 release
* Bump up the version for rocm-5.2.1 release
* correct the PROF_API_HEADER_PATH to include
* Bump up the version of rocm-openmp-extras for rocm-5.2.1 release
* bump up the version of rocwmma for 5.2.1
* imagemagick: new variant ghostscript
Ghostscript adds about 75 dependencies to an installation of
imagemagick (97 without, 172 with Ghostscript). This adds a
variant (defaulting to true for backward compatibility) that
allows users to turn off Ghostscript support.
* imagemagick: be explicit when `--with-gslib`
* imagemagick: fix suggestion fail
* imagemagick: use spec.prefix.share.font
* imagemagick: default ghostscript false
* imagemagick: no need for join_path anymore
Co-authored-by: Tamara Dahlgren <35777542+tldahlgren@users.noreply.github.com>
Co-authored-by: Tamara Dahlgren <35777542+tldahlgren@users.noreply.github.com>
This PR deprecates using Spack to install [EOL Python versions](https://endoflife.date/python), including Python 2.7, 3.1–3.6. It also deprecates running Spack with Python 2.7. Going forward, we expect Spack to have the following Python support timeline.
### Running Spack
* Spack 0.18 (spring 2022): Python 2.7, 3.5–3.10
* Spack 0.19 (fall 2022): Python 2.7 (deprecated), 3.6–3.11
* Spack 0.20 (spring 2023): Python 3.6–3.11
### Building with Spack
* Spack 0.18 (spring 2022): Python 2.7, 3.1–3.10
* Spack 0.19 (fall 2022): Python 2.7, 3.1–3.6 (deprecated), 3.7–3.11
* Spack 0.20 (spring 2023): Python 3.7–3.11
This is a reboot of #28003. See #31824 for a detailed discussion of the motivation for this PR.
If you have concerns about this change, please comment on #31824.
* py-datalad: add 0.17.5
* Fix description of py-types-urllib3
* Update var/spack/repos/builtin/packages/py-datalad/package.py
Co-authored-by: Adam J. Stewart <ajstewart426@gmail.com>
* Update var/spack/repos/builtin/packages/py-datalad/package.py
Co-authored-by: Adam J. Stewart <ajstewart426@gmail.com>
* Add colorama dependency for windows
* Fix importlib-metadata dependency
* Update var/spack/repos/builtin/packages/py-pytest/package.py
Co-authored-by: Adam J. Stewart <ajstewart426@gmail.com>
* Update var/spack/repos/builtin/packages/py-pytest/package.py
Co-authored-by: Adam J. Stewart <ajstewart426@gmail.com>
* Update var/spack/repos/builtin/packages/py-pytest/package.py
Co-authored-by: Adam J. Stewart <ajstewart426@gmail.com>
* Use conflict to avoid dependency duplication
Co-authored-by: Adam J. Stewart <ajstewart426@gmail.com>
Allow the nccl flag to be specified even for ROCm builds so that NCCL kernels are included in the build.
In this case the NCCL kernels will use RCCL as the backend implementation.
* Adding intel-oneapi-itac package
* Make black happy
* add rscohn2 as maintainer
* black prefers double quotes
Co-authored-by: Stephen Sachs <stesachs@amazon.com>
Co-authored-by: Robert Cohn <rscohn2@gmail.com>
* Don't run bootstrap on package only PRs
* Run bootstrap tests when ci.yaml is modified
* Test a package only PR
* Revert "Test a package only PR"
This reverts commit af96b1af60b0c31efcc9a2875ffb1a809ef97241.
Now that `podio` can support `cxxstd` variants 17 and 20, we can allow `edm4hep` to use `cxxstd=20` as well, but must ensure that `edm4hep` uses the same `cxxstd` variant as `podio`. Solution as in `celeritas`.
This fixes a bug where two installations that differ only by package hash will not show
up in `spack find`.
The bug arose because `_cmp_node` on `Spec` didn't include the package hash in its
yielded fields. So, any two `Spec` objects that were only different by package hash
would appear to be equal and would overwrite each other when inserted into the same
`dict`. Note that we could still *install* specs with different package hashes, and they
would appear in the database, but we code that needed to put them into data structures
that use `__hash__` would have issues.
This PR makes `Spec.__hash__` and `Spec.__eq__` include the `process_hash()`, and it
makes `Spec._cmp_node` include the package hash. All of these *should* include all
information in a spec so that we don't end up in a situation where we are blind to
particular field differences.
Eventually, we should unify the `_cmp_*` methods with `to_node_dict` so there aren't two
sources of truth, but this needs some thought, since the `_cmp_*` methods exist for
speed. We should benchmark whether it's really worth having two types of hashing now
that we use `json` instead of `yaml` for spec hashing.
- [x] Add `package_hash` to `Spec._cmp_node`
- [x] Add `package_hash` to `spack.solve.asp.spec_clauses` so that the `package_hash`
will show up in `spack diff`.
- [x] Add `package_hash` to the `process_hash` (which doesn't affect abstract specs
but will make concrete specs correct)
- [x] Make `_cmp_iter` report the dag_hash so that no two specs with different
process hashes will be considered equal.
* initial commit of 0.5.0 changes
* updated dependencies
* updated ffcx sha
* comment style
* llvm compilers
* introduce pugixml dependency for 0.5.0:
* update compilers to support C++20 features
* style fixes
* xtensor and xtl not needed for basix 0.5.0 and above
* Skip to Basix 0.5.1
The 0.5.1 release removes the C++ build dependency on Python that sneaked into the 0.5.0 build system.
* Improve depends on version ranges
* More dependency version improvements
Co-authored-by: Chris Richardson <chris@bpi.cam.ac.uk>
Co-authored-by: Garth N. Wells <gnw20@cam.ac.uk>
* added missing dependency for py-msgpack that breaks neovim
* Update var/spack/repos/builtin/packages/py-pynvim/package.py
Co-authored-by: Adam J. Stewart <ajstewart426@gmail.com>
Co-authored-by: Robert Underwood <runderwood@anl.gov>
Co-authored-by: Adam J. Stewart <ajstewart426@gmail.com>
The issue is that we are not not able to install (Fetch URL error) any
version of catalyst other than the specified in the spack package.py.
This very version is accessible only because it is cached by Spack. The
real URL does not exist anymore, I believe the reason is that there used
to be a tag in catalyst that does not exist anymore.
53a7b49 created a reference error which broke `.libs` (and
`find_libraries`) for many packages. This fixes the reference
error and improves the testing for `find_libraries` by actually
checking the extension types of libraries that are retrieved by
the function.
* catch json schema errors and reraise as property of SpackError
* no need to catch subclass of given error
* Builtin json library for Python 2 uses more generic type
* Correct instantiation of SpackError (requires a string rather than an exception)
* Use exception chaining (where possible)
* Add libraqm package
* py-pillow: Add optional raqm dependency/variant
* Use sha256
* Use " instead of '
* Use more explicit import
* Only add raqm from @8.4.0:
* Make the docstring shorter to satisfy flake
* Add conflict, silence warning, adjust version
Co-authored-by: Adam J. Stewart <ajstewart426@gmail.com>
Add a post-install step which runs (only) on Windows to modify an
install prefix, adding symlinks to all dependency libraries.
Windows does not have the same concept of RPATHs as Linux, but when
resolving symbols will check the local directory for dependency
libraries; by placing a symlink to each dependency library in the
directory with the library that needs it, the package can then
use all Spack-built dependencies.
Note:
* This collects dependency libraries based on Package.rpath, which
includes only direct link dependencies
* There is no examination of libraries to check what dependencies
they require, so all libraries of dependencies are symlinked
into any directory of the package which contains libraries
UnifyFS:
- Add 1.0 release
- Deprecate older, unsupported versions
- Set fortran variant to true by default
- Update gotcha and mochi-margo dependency versions for unifyfs@1.0
and unifyfs@develop
- Add conflict of unifyfs with libfabric 1.13.* versions
- Update configure_args to use helper functions
GOTCHA: Hasn't been under active development for a couple years but
the develop branch has some fixes UnifyFS uses. To avoid having
UnifyFS v1.0 depend on a develop branch of a dependency, this creates
a new release in the Gotcha Spack package based on the most recent
commit of the develop branch.
* Adds hyperqueue package
* Adds maintainers
* Adds missing dockstring
* Bump year for Copyright
Co-authored-by: Tamara Dahlgren <35777542+tldahlgren@users.noreply.github.com>
* Fixes checksum, selects preferred version
Don't use a release candidate as the default version
* Switch maintainer to one of the developers for HQ
* Update package.py
Co-authored-by: Tamara Dahlgren <35777542+tldahlgren@users.noreply.github.com>
* Add new intel-mpi-benchmarks version
* Add new versions of intel mpi benchmarks
* Fix style bugs
* Fix style bugs
* Switch to using url_for_version formatting and improve patch ranges
* p2p benchmark is not included on older versions
* Set patch to proper version
* Add url field, improve patch versioning, improve version detection
* Add url field, improve patch versioning, improve version detection
* Bug fix
Syntax fix
* Remove 2019 from valid version on reorder_benchmark_macros patch
* OpenMPI isn't supported on older versions of the benchmark. Prevents OpenMPI from being selected on those versions
* Add new requirement of gmake for older versions
* Require intel-mpi for older versions of benchmark
* Minor changes to build directory for older versions
* Remove repeated conflict
* Minor style changes
* Minor change
* Correct fix for intel-mpi-benchmarks
* Bug fix
* Bug fix
* Attempted fix for install bug
* Attempted fix for install bug
* Remove duplicate build_directory setting
* new packages: py-arm-pyart and dependencies
- py-arm-pyart
- py-cylp
- rsl
* Update var/spack/repos/builtin/packages/py-cylp/package.py
Co-authored-by: Adam J. Stewart <ajstewart426@gmail.com>
* Fix dependencies
- xarray is not optional
- pandas is needed
- pylab is needed
- new package, py-pylab-sdk
- setuptools is needed at run time
* Patch for import of StringIO
* Update var/spack/repos/builtin/packages/py-arm-pyart/package.py
Co-authored-by: Adam J. Stewart <ajstewart426@gmail.com>
* Fix call to `StringIO` in patch
Co-authored-by: Adam J. Stewart <ajstewart426@gmail.com>
* ci: restore coverage computation
* Mark "test_foreground_background" as xfail
* Mark "test_foreground_background_output" as xfail
* Make number of processes explicit, remove verbosity on linux
* Run coverage on just 3 Python jobs for linux
* Run coverage on just 3 Python jobs for linux
* Run coverage on just 2 Python jobs for linux
* Add back verbose, since before we didn't encounter the xdist internal error
* Reduce the workers to 2
* Try to use command line
* Fix a version cmp bug in asp.py
* Fix submodule bug for git refs
* Add branch in logic for submodules
* Fix git version comparisons
main does not satisfy git.foo=main
git.foo=main does satisfy main
* [py-python-bioformats] New package
* [py-python-bioformats] Added version 4.0.0
* [py-python-bioformats] Added types
* [py-python-bioformats] setuptools is build only
* [py-python-bioformats] fixup import
* [@spackbot] updating style on behalf of qwertos
Co-authored-by: James A Zilberman <jazrc@rit.edu>
Co-authored-by: qwertos <qwertos@users.noreply.github.com>
* Fixed the py-protobuf recipe so that when cpp support is require so
that it uses the same major and minor version range for the protobuf
package.
* Fixed the range bound for the 3.x py-protobuf packages.
Added mappings for 4.x py-protobuf packages to 3.x protobuf packages.
Removed a hash for v21.1 protobuf and replaced with v3.21.1 to keep a
standard versioning convention. Note that while Google has started
releasing both 3.x.y and a tag that dropped the leading 3. so it is
just x.y. This provides the appearance of a new major version, but
really is just a new minor version. These packages still report
versions as 3.x.y, so switching to versions and hashes with that
convention.
* Simplified constraints based on reviewer comments.
* Fixed flake8 errors
* Update var/spack/repos/builtin/packages/py-protobuf/package.py
* Fixed constraints on v2. versions and addressed Flake8 comments.
* Fixed flake8
* Fixed range dependencies for version 2.x
* Update var/spack/repos/builtin/packages/py-protobuf/package.py
Co-authored-by: Adam J. Stewart <ajstewart426@gmail.com>
* Fixed version ranges to skip unknown versions.
* Fixed the dependencies on protobuf to solve weird build issues.
Co-authored-by: Adam J. Stewart <ajstewart426@gmail.com>
* ci: remove !docs from "core" filters
Written like it is now it causes package only PRs
to run with coverage.
* Try to skip job under condition, see if the workflow proceed
* Try to cancel a running CI job
* Simplify linux unit-tests, skip windows unit-tests on package PRs
* Reduce the inputs to unit-tests workflow
* Move control logic to main workflow, remove inputs
* Revert "Move control logic to main workflow, remove inputs"
This reverts commit 0c46fece4c.
* Do not compute "with_coverage" since it's always == to "core"
* Remove workflow dispatch from unit tests
* Revert "Revert "Move control logic to main workflow, remove inputs""
This reverts commit dd4e4a4e61.
* Try to skip all from the main workflow
* Add back bootstrap to needed checks for "all"
* Restore the correct logic for conditionals
* Versions added for each dep, but I think I'll need to remove them
* py-tesseract now builds and will import in python
* Fixed flake style error as raised by pipeline
* changed to proper python dependency
* added pil as a dependency
* Fixed flake style errors
* [py-pytesseract] py-pillow and py-wheel are redundant
* [py-pytesseract]
- fixed spelling
- removed unneeded dependency
* [py-pytesseract] update import
Co-authored-by: Viv Eric Hafener <vehrc@sporcbuild.rc.rit.edu>
* Add two no-op jobs named "all-prechecks" and "all"
These are a suggestion from @tgamblin, they are stable named markers we
can use from gitlab and possibly for required checks to make CI more
resilient to refactors changing the names of specific checks.
* Enable parallel testing using xdist for unit testing in CI
* Normalize tmp paths to deal with macos
* add -u flag compatibility to spack python
As of now, it is accepted and ignored. The usage with xdist, where it
is invoked specifically by `python -u spack python` which is then passed
`-u` by xdist is the entire reason for doing this. It should never be
used without explicitly passing -u to the executing python interpreter.
* use spack python in xdist to support python 2
When running on python2, spack has many import cycles unless started
through main. To allow that, this uses `spack python` as the
interpreter, leveraging the `-u` support so xdist doesn't error out when
it unconditionally requests unbuffered binary IO.
* Use shutil.move to account for tmpdir being in a separate filesystem sometimes
update libflame for work with crayCC, craycc, crayftn compiler wrappers. These lightweight compiler drivers do not add the `-L<lib_path>` like the CC/cc/ftn compiler drivers do. I've made a slight change to add the lib directories.
This change adds support for building the rocthrust tests and adds the `amdgpu_target`
variant to the `rocthrust` package.
- [x] rocthrust: add amdgpu_target and spack build test
- [x] Drop numactl as it is not a direct dependency
* add workaround for broken behavior in HIP
Hip has a longstanding cmake issue where they calculate include paths
incorrectly, this works around it for raja and adds an explicit rocprim
dependency.
* propagate openmp requirement and workaround to camp
* refactor and include umpire
* propagate openmp option to camp in umpire and use main camp for main and develop raja and umpire
* bump camp to new patch release
This patchset refactors our GitHub actions into a single top-level ci workflow that
invokes a series of reusable actions. The main goal of this is to be able to easily
control which tests run and in what order based on the success or failure of top-level
prechecks. Our previous workflows ran in three sets:
* nix tests: style and verification first, then linux and macos tests if successful
* windows tests: style and verification first, then linux and macos tests if successful
* bootstrap tests
As a result, the bootstrap tests ran even if the style failed, and style and verification
had to run on two different platforms despite running identical checks. I'm relatively
sure that's because of the limitation on dependencies between steps in the jobs.
Reusable workflows allow us to run the style, verification and now audit checks once,
then depending on the results, and the files changed, run the appropriate nix, windows
and bootstrap tests. While it saves only a few minutes by itself, this makes it easier to
refactor checks to subset tests without having to replicate tests or other workflow
components in the future.
Co-authored-by: Massimiliano Culpo <massimiliano.culpo@gmail.com>
Fix GCC compiler warnings due to not using C99 mode
CC should be overriden with Spack's value, and the other flags needed
to be copied from the Makefile.
Move the copying of the buildcache to a root job that runs after all the child
pipelines have finished, so that the operation can be coordinated across all
child pipelines to remove the possibility of race conditions during potentially
simlutandous copies. This lets us ensure the .spec.json.sig and .spack files
for any spec in the root mirror always come from the same child pipeline
mirror (though which pipeline is arbitrary). It also allows us to avoid copying
of duplicates, which we now do.
If you have an environment like
```
$ cat spack.yaml
spack:
specs: [openmpi@4.1.0+cuda]
```
this PR provides a new command `spack change` that you can use to adjust environment specs from the command line:
```
$ spack change openmpi~cuda
$ cat spack.yaml
spack:
specs: [openmpi@4.1.0~cuda]
```
in other words, this allows you to tweak the details of environment specs from the command line.
Notes:
* This is only allowed for environments that do not define matrices
* This is possible but not anticipated to be needed immediately
* If this were done, it should probably only be done for "named"/not-anonymous specs (i.e. we can change `openmpi+cuda` but not spec like `+cuda` or `@4.0.1~cuda`)
The building of tests is optional [as of 2.42.9](801eef111d). This applies this option in the build.
The reason the option was added was to deal with test build failures in sandboxed environments and with certain glibc versions (caused by glib gresources). For example, with the latest version glibc and in the latest version of docker these tests [cannot be built](https://github.com/moby/moby/issues/43595).
* llvm: fix 15.0.0rc builds on MacOS with command-line-tools
Ref: https://github.com/llvm/llvm-project/issues/57037
i.e use -DBUILTINS_CMAKE_ARGS=-DCOMPILER_RT_ENABLE_IOS=OFF. But this needs switching "compiler-rt" from "projects" to "runtimes".
Also fixing the warnings below fixes compile errors
CMake Warning at CMakeLists.txt:101 (message):
Using LLVM_ENABLE_PROJECTS=libcxx is deprecated now, please use
-DLLVM_ENABLE_RUNTIMES=libcxx or see the instructions at
https://libcxx.llvm.org/BuildingLibcxx.html for building the runtimes.
CMake Warning at CMakeLists.txt:101 (message):
Using LLVM_ENABLE_PROJECTS=libcxxabi is deprecated now, please use
-DLLVM_ENABLE_RUNTIMES=libcxxabi or see the instructions at
https://libcxx.llvm.org/BuildingLibcxx.html for building the runtimes.
CMake Warning at CMakeLists.txt:101 (message):
Using LLVM_ENABLE_PROJECTS=libunwind is deprecated now, please use
-DLLVM_ENABLE_RUNTIMES=libunwind or see the instructions at
https://libcxx.llvm.org/BuildingLibcxx.html for building the runtimes.
/private/var/folders/nt/_m1t_x7j76q6sl3xt91tqgs00000gn/T/balay/spack-stage/spack-stage-llvm-15.0.0-rc2-h2t5bohzyy7exz2ub3m42pfycjcmbndk/spack-build-h2t5boh/include/c++/v1/cstdlib:135:9: error: no member named 'at_quick_exit' in the global namespace
using ::at_quick_exit _LIBCPP_USING_IF_EXISTS;
~~^
* Update var/spack/repos/builtin/packages/llvm/package.py
Co-authored-by: Tamara Dahlgren <35777542+tldahlgren@users.noreply.github.com>
Co-authored-by: Tamara Dahlgren <35777542+tldahlgren@users.noreply.github.com>
fixes#31484
Before this change if anything was matching an external
condition, it was considered "external" and thus something
to be "built".
This was happening in particular to external packages
that were re-read from the DB, which then couldn't be
reused, causing the problems shown in #31484.
This PR fixes the issue by excluding specs with a
"hash" from being considered "external"
* Test that users have a way to select a virtual
This ought to be solved by extending the "require"
attribute to virtual packages, so that one can:
```yaml
mpi:
require: 'multi-provider-mpi'
```
* Prevent conflicts to be enforced on specs that can be reused.
* Rename the "external_only" fact to "buildable_false", to better reflect its origin
* py-breathe: add new version and improve version constraints
* py-breathe: everyone loves versions
```
py-breathe, py-breathe in the air
don't be afraid to care
````
* Update var/spack/repos/builtin/packages/py-breathe/package.py
Co-authored-by: Adam J. Stewart <ajstewart426@gmail.com>
* add comment
Co-authored-by: Adam J. Stewart <ajstewart426@gmail.com>
* Preliminary support for include URLs in spack.yaml (environment) files
This commit adds support in environments for external configuration files obtained from a URL with a preference for grabbing raw text from GitHub and gitlab for efficient downloads of the relevant files. The URL can also be a link to a directory that contains multiple configuration files.
Remote configuration files are retrieved and cached for the environment. Configuration files with the same name will not be overwritten once cached.
Extend the semantics of package requirements to
allow using them also under a virtual package
attribute in packages.yaml
These requirements are enforced whenever that
virtual spec is present in the DAG.
Per #32214, the existing patch `92b2c...` for `gcc@11:` only applies
cleanly for `libzmq@4.3.3:4.3.4`. This PR adds a conflict for earlier
versions, as they cannot be patched due to different context.
For `gcc@11`, this leaves the most recent two versions available (a
satisfactory compromise).
For `gcc@12`, however, there is another existing conflict that makes
these most recent two versions unavailable. This PR adds an upstream
patch for the single most recent version that allows compilation with
`gcc@12` for that most recent version.
Starting point:
- `gcc@11` concretizes on all versions, attempts to apply patch on
`@4.2.3:4.3.4`, and only succeeds to apply patch on `@4.3.3:4.3.4`,
- `gcc@12` concretizes on `@:4.3.1` (and `@master`), attempts to apply
patch on `@4.2.3:4.3.1`, fails to apply patch on all.
Ending point:
- `gcc@11` concretizes on `@4.3.3:4.3.4` (and `@master`), attempts and
succeeds to apply patch on `@4.3.3:4.3.4`,
- `gcc@12` concretizes on `@4.3.4` (and `@master`), attempts and
succeeds to apply patch on `@4.3.4`.
Verified with environment build:
```yaml
spack:
specs:
- libzmq@4.3.4%gcc@12.1.0
- libzmq@4.3.4%gcc@11.3.0
- libzmq@4.3.3%gcc@11.3.0
view: false
```
which returns the following:
```console
16:14:47 wdconinc@menelaos ~/git/spack (libzmq-patch-gcc12 *+$%=) $
spack install --fresh
==> Installing environment libzmq
[+]
/opt/software/linux-ubuntu22.04-skylake/gcc-12.1.0/libmd-1.0.4-egpgd6eoaqtsl5fja2iwsl6gyc4o32p5
[+]
/opt/software/linux-ubuntu22.04-skylake/gcc-12.1.0/libsodium-1.0.18-af3rsfnvck6anxf7eeog3f2bph44tjia
[+]
/opt/software/linux-ubuntu22.04-skylake/gcc-12.1.0/pkgconf-1.8.0-z5of2hj2c6ygd3kxr4cwv7u7t42sxair
[+]
/opt/software/linux-ubuntu22.04-skylake/gcc-11.3.0/libmd-1.0.4-tec234gco2sd7n52dkwbrad43sdhaw4o
[+]
/opt/software/linux-ubuntu22.04-skylake/gcc-11.3.0/libsodium-1.0.18-uljf675u3yrn5c7fdjdpa5c7qnnkynke
[+]
/opt/software/linux-ubuntu22.04-skylake/gcc-11.3.0/pkgconf-1.8.0-l4hzc2g4pnn7dwyttphmxivt3xghvpoq
[+]
/opt/software/linux-ubuntu22.04-skylake/gcc-12.1.0/libbsd-0.11.5-fi3ri64moy45ksr4sf5pcwd6k23dsa4o
[+]
/opt/software/linux-ubuntu22.04-skylake/gcc-11.3.0/libbsd-0.11.5-2matmm7im7oygrr77k7wznttv4rbupfz
==> Installing libzmq-4.3.4-t7ad54q3atrnte4rzq7g7cfjcw5227pr
==> No binary for libzmq-4.3.4-t7ad54q3atrnte4rzq7g7cfjcw5227pr found:
installing from source
==> Fetching
c593001a89.tar.gz
==> Fetching
310b8aa57a
==> Fetching
https://github.com/zeromq/libzmq/pull/4334.patch?full_index=1
==> Applied patch
92b2c38a2c.patch?full_index=1
==> Applied patch
https://github.com/zeromq/libzmq/pull/4334.patch?full_index=1
==> libzmq: Executing phase: 'autoreconf'
==> libzmq: Executing phase: 'configure'
==> libzmq: Executing phase: 'build'
==> libzmq: Executing phase: 'install'
==> libzmq: Successfully installed
libzmq-4.3.4-t7ad54q3atrnte4rzq7g7cfjcw5227pr
Fetch: 0.61s. Build: 1m 31.57s. Total: 1m 32.18s.
[+]
/opt/software/linux-ubuntu22.04-skylake/gcc-12.1.0/libzmq-4.3.4-t7ad54q3atrnte4rzq7g7cfjcw5227pr
==> Installing libzmq-4.3.3-pxrd6piprucu65bkro2ixms6d3x2eudz
==> No binary for libzmq-4.3.3-pxrd6piprucu65bkro2ixms6d3x2eudz found:
installing from source
==> Fetching
9d9285db37.tar.gz
==> Using cached archive:
/home/wdconinc/.spack/cache/_source-cache/archive/31/310b8aa57a8ea77b7ac74debb3bf928cbafdef5e7ca35beaac5d9c61c7edd239
==> Applied patch
92b2c38a2c.patch?full_index=1
==> libzmq: Executing phase: 'autoreconf'
==> libzmq: Executing phase: 'configure'
==> libzmq: Executing phase: 'build'
==> libzmq: Executing phase: 'install'
==> libzmq: Successfully installed
libzmq-4.3.3-pxrd6piprucu65bkro2ixms6d3x2eudz
Fetch: 0.93s. Build: 11.55s. Total: 12.48s.
[+]
/opt/software/linux-ubuntu22.04-skylake/gcc-11.3.0/libzmq-4.3.3-pxrd6piprucu65bkro2ixms6d3x2eudz
==> Installing libzmq-4.3.4-hiil6dzy2reb4nk555zztwh44rpbyv4h
==> No binary for libzmq-4.3.4-hiil6dzy2reb4nk555zztwh44rpbyv4h found:
installing from source
==> Using cached archive:
/home/wdconinc/.spack/cache/_source-cache/archive/c5/c593001a89f5a85dd2ddf564805deb860e02471171b3f204944857336295c3e5.tar.gz
==> Using cached archive:
/home/wdconinc/.spack/cache/_source-cache/archive/31/310b8aa57a8ea77b7ac74debb3bf928cbafdef5e7ca35beaac5d9c61c7edd239
==> Applied patch
92b2c38a2c.patch?full_index=1
==> libzmq: Executing phase: 'autoreconf'
==> libzmq: Executing phase: 'configure'
==> libzmq: Executing phase: 'build'
==> libzmq: Executing phase: 'install'
==> libzmq: Successfully installed
libzmq-4.3.4-hiil6dzy2reb4nk555zztwh44rpbyv4h
Fetch: 0.01s. Build: 10.77s. Total: 10.78s.
[+]
/opt/software/linux-ubuntu22.04-skylake/gcc-11.3.0/libzmq-4.3.4-hiil6dzy2reb4nk555zztwh44rpbyv4h
```
* ddt: Initial commit
* ddt: Minor change to license date to appease the CI gods
* Get rid of unattractive extra line
* Switch to sha256 instead of md5
* Initial Draft of E3SM-Kernels Spackage
Initial draft of E3SM-Kernels. Currently no support for nested_loops due
to build system limitations
* Style Check and Fixed gfortran Check
Fixed style issues and changed gnu toolchain check to a gfortran check
due to hybrid compilers (e.g. clang+gfortran)
* Fixed Style Issues
* tixi: new variants (fortran,shared)
Since some tixi 3 versions, additional CMake flags are needed to build
tixi with shared libraries, respectively with Fortran support.
* tixi: fix style
* Changes for ROCm-5.2.0 changes and new recipe rocwmma
* modify the maintainers for hipify-clang
* address review comments
* update the rocwmma new recipe as per latest syntax
* fix style errors
* modify the patch file to provide the details about the patch
* fix style errors
* whizard: Fix passing of build options, update versions
The dependency of whizard on libtirpc is now correctly passed down as
an autotools option.
Update known versions of package with 3.0.2 and 3.0.3.
* Express path to headers via spec object methods
Allow users to express default requirements in packages.yaml.
These requirements are overridden if more specific requirements
are present for a given package.
* cpu-features: fix fetch failure
`master` branch was renamed to `main`
* Update var/spack/repos/builtin/packages/cpu-features/package.py
Co-authored-by: Adam J. Stewart <ajstewart426@gmail.com>
Co-authored-by: Adam J. Stewart <ajstewart426@gmail.com>
* [py-loguru] New package
* [py-loguru] Removed commented out line
* [py-loguru] Added types removed extra dependencies
* [py-loguru] missing windows dependency. listing windows as a conflict for now
* [py-loguru] depends on py-colorama when platform=windows
* [py-loguru] flake8
* [py-loguru] Import update
* [py-loguru]
- python is a runtime dependency
- setuptools is a build dependency
* [@spackbot] updating style on behalf of qwertos
Co-authored-by: James A Zilberman <jazrc@rit.edu>
Co-authored-by: qwertos <qwertos@users.noreply.github.com>
* libfabric: add new versions
1.15.0, 1.15.1, main (previously named master)
* Add OPX fabric option, with conflict for versions before v1.15.0
Co-authored-by: Tamara Dahlgren <35777542+tldahlgren@users.noreply.github.com>
* geant4: use define function
* geant4: Change new feature from conflicts to when
* geant4: add support/conflicts for nvhpc
* fixup! geant4: add support/conflicts for nvhpc
This support requires adding the '--tests' option to 'spack ci rebuild'.
Packages whose stand-alone tests are broken (in the CI environment) can
be configured in gitlab-ci to be skipped by adding them to
broken-tests-packages.
Highlights include:
- Restructured 'spack ci' help to provide better subcommand summaries;
- Ensured only one InstallError (i.e., installer's) rather than allowing
build_environment to have its own; and
- Refactored CI and CDash reporting to keep CDash-related properties and
behavior in a separate class.
This allows stand-alone tests from `spack ci` to run when the `--tests`
option is used. With `--tests`, stand-alone tests are run **after** a
**successful** (re)build of the package. Test results are collected
and report(able) using CDash.
This PR adds the following features:
- Adds `-t` and `--tests` to `spack ci rebuild` to run stand-alone tests;
- Adds `--fail-fast` to stop stand-alone tests after the first failure;
- Ensures a *single* `InstallError` across packages
(i.e., removes second class from build environment);
- Captures skipping tests for externals and uninstalled packages
(for CDash reporting);
- Copies test logs and outputs to the CI artifacts directory to facilitate
debugging;
- Parses stand-alone test results to report outputs from each `run_test` as
separate test parts (CDash reporting);
- Logs a test completion message to allow capture of timing of the last
`run_test` part;
- Adds the runner description to the CDash site to better distinguish entries
in CDash tables;
- Adds `gitlab-ci` `broken-tests-packages` to CI configuration to skip
stand-alone testing for packages with known issues;
- Changes `spack ci --help` so description of each subcommand is a single line;
- Changes `spack ci <subcommand> --help` to provide the full description of
each command (versus no description); and
- Ensures `junit` test log file ends in an `.xml` extension (versus default where
it does not).
Tasks:
- [x] Include the equivalent of the architecture information, or at least the host target, in the CDash output
- [x] Upload stand-alone test results files as `test` artifacts
- [x] Confirm tests are run in GitLab
- [x] Ensure CDash results are uploaded as artifacts
- [x] Resolve issues with CDash build-and test results appearing on same row of the table
- [x] Add unit tests as needed
- [x] Investigate why some (dependency) packages don't have test results (e.g., related from other pipelines)
- [x] Ensure proper parsing and reporting of skipped tests (as `not run`) .. post- #28701 merge
- [x] Restore the proper CDash URLand or mirror ONCE out-of-band testing completed
* gaudi: consistent test dependencies when +examples
Gaudi requires testing to be enabled for examples to be built, so all
test dependencies are also there for `+examples`. This PR fixes a
missing pytest dependency when `+examples` is used but no testing is
enabled. The construct with the loop is to ensure the identical
dependencies are always used, even as version ranges my start to differ.
Testing with gaudi_add_tests was added in v35r0. Some nosetests and
QMtests were in the tree before, but not accessible it seems. The
effective dependency since 35.0 is also applied for pytest, extending
the range that was there before disentangling `optional`, at
9d67d1e034/var/spack/repos/builtin/packages/gaudi/package.py
* gaudi: version 36.7 in other PR...
The "release" tarball provided by github lacks several files in
the SFconv/expat/xmlparse directory, including xmlparse. Using
tarballs based off of version tags solves the problem.
o Changes version() to use commits associated with version tags.
o Adds several additional versions.
o Adds myself as maintainer.
o Adds hook to execute autogen.sh.
o Adds autotools &ct dependences.
o Removes expat dependence.
* openmc: add v0.13.1
* Add @paulromano as maintainer of openmc and py-openmc
* Address review comments from @adamjstewart
* Add back MPI variant in openmc package
In a fast-moving project with as many forks as LLVM, it's difficult to
accurately determine if a function exists just by checking the version
number. The existing version check fails, for example, with llvm-amdgpu
from ROCm 4.5. It is more robust to directly check if the function
exists.
Added the SHA256 for version 6.3.2 and added logic to eal with the change of naming pattern for the makefile.include files that now appears to leave out the "linux_" prefix. (Changes should be. backwards compatible.)
Co-authored-by: Massimiliano Culpo <massimiliano.culpo@gmail.com>
Co-authored-by: alansill <alansill@users.noreply.github.com>
* modified list.py and added functionality for --tag
* Removed long and very long, shifted rest of code above return statement
* removed results variable
* added import statement at top
* added the line accidentally deleted
* added line accidentally deleted
* changed p.name to p, added line inside if statement
* line order switched
* [@spackbot] updating style on behalf of sparkyniner
* ran update completion command
* add tests
* Update lib/spack/spack/test/cmd/list.py
Co-authored-by: Adam J. Stewart <ajstewart426@gmail.com>
* [@spackbot] updating style on behalf of sparkyniner
* changed argument to mock_packages and moved code under filter by tag
* removed bad rebase code and added additional test
* [@spackbot] updating style on behalf of sparkyniner
* added line removed earlier
* added line removed earlier
* replaced function
* added more recommended changes
Co-authored-by: sairaj <sairaj@sairajs-MacBook-Pro.local>
Co-authored-by: Adam J. Stewart <ajstewart426@gmail.com>
* VTK-h: don't assume I have Fortran
Don't assume I have a working Fortran compiler in my toolchain :)
* Conduit: Do not Assume Fortran
* Ascent: Do not Assume Fortran
* fix style
* Build Tensorflow using the fork for rocm. Initial commit
* re-order the versions
* fix style errors
* address review comments
* add conflicts for rocm version
* address review comments
* remove rocm variant as its added by ROCmPackage
* add mothra tests
* py-globus-sdk: add new versions; unpin py-cryptography version constraint
* Update var/spack/repos/builtin/packages/py-globus-sdk/package.py
Co-authored-by: Adam J. Stewart <ajstewart426@gmail.com>
Co-authored-by: Adam J. Stewart <ajstewart426@gmail.com>
* py-pyjwt: fix py-cryptography version constraints
* Update var/spack/repos/builtin/packages/py-pyjwt/package.py
Co-authored-by: Adam J. Stewart <ajstewart426@gmail.com>
Co-authored-by: Adam J. Stewart <ajstewart426@gmail.com>
* ForTrilinos: new versions 2.0.1, 2.1.0
I also had to update the checksum for the released 2.0.0: see #32000 for
the explanation and solution to keep this from happening again.
* Soooo stylish
Assertions without messages if/when hit create a blank error message for users.
This PR adds error messages to all assertions in asp.py even
if it seems unlikely they will ever be needed.
* Sensei: Refactor package to work with v4.0.0
* Add missing MPI dependency
* Patch bug in libsim adapter
* Simplify conflicts with when-clauses
* Conflict variants that are incompatible (catalyst/libsim/ascent)
* Fix paraview version constraints to be more clear
* Add version constraints for VTK
* Drop unneeded visit restrictions
* Specify +vtkm dependency on ParaView's VTKm
* +hl is not needed for VTK, and is already specified in the VTK recipe
when it is needed
* Pass paths for adios2 and ascent packages
* ECP-SDK: Enable sensei
* CI: Add sensei to the data-vis-sdk pipeline
* Sensei: Change VISIT_DIR to work on linux
* Fixup: style check
* Sensei: Add patch for version detection
* CI: revert SDK pipeline in favor of new matrices
* Sensei: Formatting fixes
The argument is very likely a typo, and was meant to
be given to the fixture decorator. Since the value
being passed is the default, let's just remove it.
* metaeuk: new package
* sepp: new package
* busco: adding version 5.4.3
* busco: adding maintainers
* metauk: more accurate cmake dep
* sepp: adding missing dep and accurate python dep
* sepp: remove install_tree
* sepp: extend python
Ensure that build tools with module-level commands in spack use
the version built as part of their build graph if one exists.
This is now also required for mesa, scons, cmake and ctest, out
of graph versions of these tools in path will not be found unless
added as an external.
This bug appeared because a new version of rocprim needs cmake
3.16, while I have 3.14 in my path I had added an external for
cmake 3.20 to the dag, but 3.14 was still used to configure
rocprim causing it to fail. As far as I can tell, all the build
tools added in build_environment.py had this problem, despite the
fact that they should have been resolving these tools by name
with a path search and find the one in the dag that way. I'm
still investigating why the path searching and Executable logic
didn't do it, but this makes three of the build systems much more
explicit, and leaves only gmake and ninja as dependencies from
out in the system while ensuring the version in the dag is used
if there is one.
The additional sqlite version is to perturb the hash of python to
work around a relocation bug which will be fixed in a subsequent
PR.
The Python that comes with XCode command line tools is a bit weird.
This fixes two issues that were preventing it from bootstrapping:
- [x] CommandLineTools python does not come with a `python-config` executable. If you
don't have one of these, CMake's FindPython will, for some reason, assume that you
want Python 2, so you need to set `CLINGO_PYTHON_VERSION` to ensure that clingo
tells `find_package(Python <VERSION>)` the right thing.
- [x] We weren't setting `PYTHONHOME` correctly in Python's `package.py`. We were
setting it to the `prefix` from Python `sysconfig`, but `prefix` tells you where
the executable lives. CommandLineTools python reports its prefix as
/Applications/Xcode.app/Contents/Developer/Library/Frameworks/Python3.framework/Versions/3.8
but that doesn't exist. It looks like Apple builds the full python and just copies
pieces of it into command line tools. PYTHONHOME is supposed to tell you where the
default python library location is. So you have to look at the `base`, which
`sysconfig` reports as
/Library/Developer/CommandLineTools/Library/Frameworks/Python3.framework/Versions/3.8
On most systems this is probably the same as `prefix`, but not with
CommandLineTools, which has its system library in a different place.
The second change here was cherry-picked to 0d981a012d before merging and doesn't show
up here.
Co-authored-by: Adam J. Stewart <ajstewart426@gmail.com>
- [x] Rework `headers` to search a sequence of directories and to display all searched
locations on error, as opposed to handling each directory with a variable
- [x] Make `headers` and `libs` do the same type of search and raise the same sort of
errors.
Co-authored-by: Todd Gamblin <tgamblin@llnl.gov>
* [py-bitshuffle] New package
* [py-bitshuffle] compiling against spack hdf5
* [py-bitshuffle] flake8
* [py-bitshuffle] update import ine
* [@spackbot] updating style on behalf of qwertos
Co-authored-by: James A Zilberman <jazrc@rit.edu>
Co-authored-by: qwertos <qwertos@users.noreply.github.com>
We're seeing errors pop up with older versions of Kokkos and newer versions of
`oneapi`, specifically:
error: no member named 'ONEAPI' in namespace 'sycl'
This hapens because `sycl::ONEAPI` is `sycl::ext::oneapi` since oneapi `2022.0.0`.
`kokkos@3.6:` uses the new namespace. A conflict was present for this, but it was too
specific -- both to `dpcpp` and to the `2022.0.0` version.
- [x] Expand version ranges in `kokkos` conflict
- [x] Add conflict for `oneapi` in addition to `dpcpp`
* filesystem: use lstat in recursive mtime
When a `develop` path contains a dead symlink, the `os.stat` in the recursive `mtime` determination trips up over it.
Closes#32165.
`requirement_policy/3` is generated and may not be in Spack's inputs to Clingo.
Currently this is causing warnings like:
```
$ spack spec zlib
/global/u2/t/tgamblin/src/spack/lib/spack/spack/solver/concretize.lp:510:3-43: info: atom does not occur in any rule head:
requirement_policy(Package,X,"one_of")
/global/u2/t/tgamblin/src/spack/lib/spack/spack/solver/concretize.lp:517:3-43: info: atom does not occur in any rule head:
requirement_policy(Package,X,"one_of")
/global/u2/t/tgamblin/src/spack/lib/spack/spack/solver/concretize.lp:523:3-43: info: atom does not occur in any rule head:
requirement_policy(Package,X,"any_of")
/global/u2/t/tgamblin/src/spack/lib/spack/spack/solver/concretize.lp:534:3-43: info: atom does not occur in any rule head:
requirement_policy(Package,X,"any_of")
Input spec
--------------------------------
zlib
Concretized
--------------------------------
zlib@1.2.11%gcc@7.5.0+optimize+pic+shared arch=cray-sles15-haswell
```
- [x] Silence warning with `#defined requirement_policy/3`
Spack doesn't have an easy way to say something like "If I build
package X, then I *need* version Y":
* If you specify something on the command line, then you ensure
that the constraints are applied, but the package is always built
* Likewise if you `spack add X...`` to your environment, the
constraints are guaranteed to hold, but the environment always
builds the package
* You can add preferences to packages.yaml, but these are not
guaranteed to hold (Spack can choose other settings)
This commit adds a 'require' subsection to packages.yaml: the
specs added there are guaranteed to hold. The commit includes
documentation for the feature.
Co-authored-by: Massimiliano Culpo <massimiliano.culpo@gmail.com>
* celeritas: update checksum for git describe difference
Using `$Format:%h$` and the export-subst gitattribute property mean that
the release checksum can change post release...
```
$ diff -ru spack-stage-celeritas-0.1.0-hv7ewpjouekqws2y5iaql2cnp6tn76iz/ spack-stage-celeritas-0.1.0.wtf-ynyck3df2a7kkgkqrpwapgz3l2i62omz/
diff -ru spack-stage-celeritas-0.1.0-hv7ewpjouekqws2y5iaql2cnp6tn76iz/spack-src/cmake/CgvFindVersion.cmake spack-stage-celeritas-0.1.0.wtf-ynyck3df2a7kkgkqrpwapgz3l2i62omz/spack-src/cmake/CgvFindVersion.cmake
--- spack-stage-celeritas-0.1.0-hv7ewpjouekqws2y5iaql2cnp6tn76iz/spack-src/cmake/CgvFindVersion.cmake 2022-07-31 19:45:03.000000000 -0400
+++ spack-stage-celeritas-0.1.0.wtf-ynyck3df2a7kkgkqrpwapgz3l2i62omz/spack-src/cmake/CgvFindVersion.cmake 2022-07-31 19:45:03.000000000 -0400
@@ -75,7 +75,7 @@
# Get a possible Git version generated using git-archive (see the
# .gitattributes file)
- set(_ARCHIVE_TAG "HEAD -> master, tag: v0.1.0")
+ set(_ARCHIVE_TAG "tag: v0.1.0")
set(_ARCHIVE_HASH "04fe945d9")
set(_TAG_REGEX "v([0-9.]+)(-dev[0-9.]+)?")
Binary files spack-stage-celeritas-0.1.0-hv7ewpjouekqws2y5iaql2cnp6tn76iz/v0.1.0.tar.gz and spack-stage-celeritas-0.1.0.wtf-ynyck3df2a7kkgkqrpwapgz3l2i62omz/v0.1.0.tar.gz differ
```
* celeritas: Use exported and re-uploaded tar.gz for reproducibility
All PRs are failing the docs build on account of an error with
pygments. These errors coincide with a new release of pygments
(2.13.0) and restricting to < 2.13 allows the doc tests to pass,
so this commit enforces that constraint for the docs build.
Co-authored-by: Adam J. Stewart <ajstewart426@gmail.com>
A few attribute in packages are meant to be reserved for
Spack internal use. This audit checks packages to ensure none
of these attributes are overridden.
- [x] add additional audit check
In file included from /data/xsdkci/VS1mJQ1K/0/xsdk-project/spack-xsdk/spack-stage/balay/spack-stage-oce-0.18.3-rim5zhuacb7z4plxag52fjj3gbc4znv3/spack-src/src/OSD/OSD_Parallel.cxx:17:
/data/xsdkci/VS1mJQ1K/0/xsdk-project/spack-xsdk/spack-stage/balay/spack-stage-oce-0.18.3-rim5zhuacb7z4plxag52fjj3gbc4znv3/spack-src/src/OSD/OSD_Parallel.hxx: In static member function 'static void OSD_Parallel::ForEach(InputIterator, InputIterator, const Functor&, Standard_Boolean)':
/data/xsdkci/VS1mJQ1K/0/xsdk-project/spack-xsdk/spack-stage/balay/spack-stage-oce-0.18.3-rim5zhuacb7z4plxag52fjj3gbc4znv3/spack-src/src/OSD/OSD_Parallel.hxx:223:18: error: 'captured_exception' in namespace 'tbb' does not name a type
223 | catch ( tbb::captured_exception& anException )
| ^~~~~~~~~~~~~~~~~~
/data/xsdkci/VS1mJQ1K/0/xsdk-project/spack-xsdk/spack-stage/balay/spack-stage-oce-0.18.3-rim5zhuacb7z4plxag52fjj3gbc4znv3/spack-src/src/OSD/OSD_Parallel.hxx:225:38: error: 'anException' was not declared in this scope
225 | Standard_NotImplemented::Raise(anException.what());
| ^~~~~~~~~~~
/data/xsdkci/VS1mJQ1K/0/xsdk-project/spack-xsdk/spack-stage/balay/spack-stage-oce-0.18.3-rim5zhuacb7z4plxag52fjj3gbc4znv3/spack-src/src/OSD/OSD_Parallel.hxx: In static member function 'static void OSD_Parallel::For(Standard_Integer, Standard_Integer, const Functor&, Standard_Boolean)':
/data/xsdkci/VS1mJQ1K/0/xsdk-project/spack-xsdk/spack-stage/balay/spack-stage-oce-0.18.3-rim5zhuacb7z4plxag52fjj3gbc4znv3/spack-src/src/OSD/OSD_Parallel.hxx:272:18: error: 'captured_exception' in namespace 'tbb' does not name a type
272 | catch ( tbb::captured_exception& anException )
| ^~~~~~~~~~~~~~~~~~
/data/xsdkci/VS1mJQ1K/0/xsdk-project/spack-xsdk/spack-stage/balay/spack-stage-oce-0.18.3-rim5zhuacb7z4plxag52fjj3gbc4znv3/spack-src/src/OSD/OSD_Parallel.hxx:274:38: error: 'anException' was not declared in this scope
274 | Standard_NotImplemented::Raise(anException.what());
| ^~~~~~~~~~~
`setup_dependent_build_environment(self, env, dependent_spec)` does not have a variable `spec`.
This causes several issues right now:
```console
==> Installing gaudi-36.6-cjjrpjwpcqrtojyrdqml3jpzkbn55hpb
==> No binary for gaudi-36.6-cjjrpjwpcqrtojyrdqml3jpzkbn55hpb found: installing from source
==> Error: NameError: name 'spec' is not defined
/home/wdconinc/git/spack/var/spack/repos/builtin/packages/root/package.py:614, in setup_dependent_build_environment:
611 env.prepend_path("ROOT_INCLUDE_PATH", dependent_spec.prefix.include)
612 if "+rpath" not in self.spec:
613 env.prepend_path("LD_LIBRARY_PATH", self.prefix.lib.root)
>> 614 if "platform=darwin" in spec:
615 # Newer deployment targets cause fatal errors in rootcling
616 env.unset("MACOSX_DEPLOYMENT_TARGET")
```
On PR pipelines we need to override the buildcache destination to
point to the "spack-binaries-prs" bucket, otherwise, those pipelines
try to push to the default mirror in a bucket for which they don't
have write permission.
This PR fixes the performance regression reported in #31985 and a few
other issues found while refactoring the spack mirror create command.
Modifications:
* (Primary) Do not require concretization for
`spack mirror create --all`
* Forbid using --versions-per-spec together with --all
* Fixed a few issues when reading specs from input file (specs were
not concretized, command would fail when trying to mirror
dependencies)
* Fix issue with default directory for spack mirror create not being
canonicalized
* Add more unit tests to poke spack mirror create
* Skip externals also when mirroring environments
* Changed slightly the wording for reporting (it was mentioning
"Successfully created" even in presence of errors)
* Fix issue with colify (was not called properly during error
reporting)
* [py-pynndescent] New package
* [py-pynndescent] Removed white space
* [py-pynndescent] fixed dependencies
* [py-pynndescent] flake8
* [py-pynndescent] Import update
Co-authored-by: James A Zilberman <jazrc@rit.edu>
- [x] Add release v0.2 of the AML package, deprecate v0.1, and add support for
OpenCL, HIP, and CUDA variants of the library. Also update repo and
release URL, as the previous one is not accessible anymore.
- [x] aml: add oneapi-level-zero support
- [x] Change openmp flags to force compatibility when compiling with
intel-oneapi compilers.
`LD_LIBRARY_PATH` can break system executables (e.g., when an enviornment is loaded) and isn't necessary thanks to `RPATH`s. Packages that require `LD_LIBRARY_PATH` can set this in `setup_run_environment`.
- [x] Prefix inspections no longer set `LD_LIBRARY_PATH` by default
- [x] Document changes and workarounds for people who want `LD_LIBRARY_PATH`
These changes make many packages build on nixos where nearly nothing
comes from /bin or /usr/bin (the only things in "system locations" are
/bin/sh and /usr/bin/env, all the rest is found through PATH).
Many configuration scripts hardcode /usr/bin/file instead of using the
one from PATH. This patches them to use file from PATH.
* ADIOS2: ZFP<1.0
The tagged ADIOS2 releases in Spack (and develop) do not yet
work with ZFP 1.0. Express valid depedency range before someone
updates the ZFP package and triggers the incompatibility.
* Use semver
* flux packages used in a spack view will already be unshallow
and this particular command will error and break the entire build. Instead
we want to catch the ProcessError and allow for a regular fetch.
* final tweaks to add missing sqlite and fallback to fetch
Signed-off-by: vsoch <vsoch@users.noreply.github.com>
fixes#31736
Catch errors when concretizing specs and report them as
debug messages. The corresponding spec is skipped.
Co-authored-by: Greg Becker <becker33@llnl.gov>
* Extracted two functions in cmd/install.py
* Extracted a function to perform installation from the active environment
* Rename a few functions, remove args from their arguments
* Rework conditional in install_from_active_environment to reduce nesting in the function
* Extract functions to parsespecs from cli and files
* Extract functions to getuser confirmation for overwrite
* Extract functions to install specs inside and outside environments
* Rename a couple of functions
* Fix outdated comment
* Add missing imports
* Split conditional to dedent one level
* Invert check and exit early to dedent one level when requiring user confirmation
* tassel: add version 3.0.174
fix naming of 5.2.39 from "2017-07-22"
* changed single quotes to double quotes to appease the gods ;)
* fixed style issues
The current use of git ref's as a version requires a search algorithm to pick the right matching version based on the tags in the git history of the package.
This is less than ideal for the use case where users already know the specific version they want the git ref to be associated with. This PR makes a new version syntax [package]@[ref]=[version] to allow the users to specify the exact hash they wish to use.
* Add checksum for py-arror 1.2.1 and 1.2.2
* Update package.py
* Update var/spack/repos/builtin/packages/py-arrow/package.py
Co-authored-by: Adam J. Stewart <ajstewart426@gmail.com>
* Update package.py
* Update package.py
* Update package.py
* Update package.py
Co-authored-by: Adam J. Stewart <ajstewart426@gmail.com>
* [py-glymur] New Package: py-glymur
* [py-glymur] fixed copyright line
* [py-glumur] update import ine
* [@spackbot] updating style on behalf of qwertos
Co-authored-by: James A Zilberman <jazrc@rit.edu>
Co-authored-by: qwertos <qwertos@users.noreply.github.com>
There is a problem with the git repo for rhtslib that apparently led to
a bad version entry during the previous round of package updates. A git
checkout on the commit also fails so use branch for most recent version.
* new package: alphafold
and related dependencies, depends on #27138
* [@spackbot] updating style on behalf of aweits
* fix
Co-authored-by: aweits <aweits@users.noreply.github.com>
On some systems the shell in login mode wipes important parts of the
environment, such as PATH. This causes the build to fail since it can't
find `spack`.
For better robustness, don't use a login shell.
In a full CI job the final spack install is run in an environment formed by scripts running in this order:
export AWS_SECRET=... # 1. Load environment from GitLab project variables
source spack/share/spack/setup-env.sh # 2. Load Spack into the environment (PATH)
spack env activate -V concrete_env # 3. Activate the concrete environment
source /etc/profile # 4. Bash login shell (from -l)
spack install ...
Whereas when a user launches their own container with (docker|podman) run -it, they end up running spack install in an environment formed in this order:
source /etc/bash.bashrc # (not 4). Bash interactive shell (default with TTY)
export AWS_SECRET=... #~1. Manually load environment from GitLab project variables
source spack/share/spack/setup-env.sh # 2. Load Spack into the environment (PATH)
spack env activate -V concrete_env # 3. Activate the concrete environment
spack install ...
The big problem being that (4) has a completely different position and content (on Leap 15 and possibly other containers).
So in context, this PR removes (4) from the CI job case, leaving us with the simpler:
export AWS_SECRET=... # 1. Load environment from GitLab project variables
source spack/share/spack/setup-env.sh # 2. Load Spack into the environment (PATH)
spack env activate -V concrete_env # 3. Activate the concrete environment
spack install ...
* database: don't sort on return from query_local
* ASP-based solver: don't build the hash-lookup dictionary twice
Building this dictionary twice and traversing all the specs
might be time-consuming for large buildcaches.
This test relied on an old version of the `flake8_package` fixture that modified
the spack repository, but it doesn't do that anymore. There are other tests for
`changed_files()` that do a better job of mocking up a git repository with
changes, so we can just delete this one.
A GitHub rebase merge seems to rewrite commits even if it would be a
fast-forward, which means that the commit merged from #24718 is wrong.
- [x] update `.git-blame-ignore-revs` with real commit from `develop`
`spack style` tests were annoyingly brittle because we could not easily be
specific about which tools to run (we had to use `--no-black`, `--no-isort`,
`--no-flake8`, and `--no-mypy`). We should be able to specify what to run OR
what to skip.
Now you can run, e.g.:
spack style --tool black,flake8
or:
spack style --skip black,isort
- [x] Remove `--no-black`, `--no-isort`, `--no-flake8`, and `--no-mypy` args.
- [x] Add `--tool TOOL` argument.
- [x] Add `--skip TOOL` argument.
- [x] Allow either `--tool black --tool flake8` or `--tool black,flake8` syntax.
- [x] remove alignment spaces from tempaltes
- [x] replace single with double quotes
- [x] Makefile template now generates parsable code
(function body is `pass` instead of just a comment)
- [x] template checks now run black to check output
Previously we'd accept any version for bootstrapping black, but we need <= 21.
- [x] modify bootstrapping code to check black version before accepting an
executable from `PATH`.
- [x] add `.git-blame-ignore-revs` to ignore black reformatting
- [x] make `spack blame` respect `.git-blame-ignore-revs`
(even if the user hasn't configured git to do so)
Some of our tests rely on single vs. double quotes, and others rely on specific
line numbers in the source. These needed fixing after the switch to Black.
Black will automatically fix a lot of the exceptions we previously allowed for
directives, so we don't need them in our custom `flake8_formatter` anymore.
- [x] remove `E501` (long line) exceptions for directives from `flake8_formatter`,
as they won't help us now.
- [x] Refine exceptions for long URLs in the `flake8_formatter`.
- [x] Adjust the mock `flake8-package` to exhibit the exceptions we still allow.
- [x] Update style tests for new `flake8-package`.
- [x] Blacken style test.
Many noqa's in the code are no longer necessary now that the column limit is 99
characters. Others can easily be eliminated, and still more can just be made more
specific if they do not have to do with line length.
The only E501's still in the code are in the tests for `spack.util.path` and the tests
for `spack style`.
This adds necessary configuration for flake8 and black to work together.
This also sets the line length to 99, per the data here:
* https://github.com/spack/spack/pull/24718#issuecomment-876933636
Given the data and the spirit of black's 88-character limit, we set the limit to 99
characters for all of Spack, because:
* 99 is one less than 100, a nice round number, and all lines will fit in a
100-character wide terminal (even when the text editor puts a \ at EOL).
* 99 is just past the knee the file size curve for packages, and it means that packages
remain readable and not significantly longer than they are now.
* It doesn't seem to hurt core -- files in core might change length by a few percent but
seem like they'll be mostly the same as before -- just a bit more roomy.
- [x] set line length to 99
- [x] remove most exceptions from `.flake8` and add the ones black cares about
- [x] add `[tool.black]` to `pyproject.toml`
- [x] make `black` run if available in `spack style --fix`
Co-Authored-By: Tom Scogland <tscogland@llnl.gov>
* Add checksum for numba 0.55.2 and 0.56
* Add checksum for py-llvmlite 0.39.0
* Apply suggestions from code review
Co-authored-by: Adam J. Stewart <ajstewart426@gmail.com>
Co-authored-by: Adam J. Stewart <ajstewart426@gmail.com>
* OpenGL: Restructures the OpenGL packages
This provides concrete glx and osmesa packages which delegate to
virtual libglx and libosmesa packages provided by mesa. This was
necessary because GLX and OSMesa both provide gl implementations but
with mesa providing the girtual gl package there was no way to properly
distinguish which of the two opengl implementations was beiing requested
when querying the spec['gl'] dependency. This additional level of
indirection allows for that.
* OpenGL: Adjust downstream dependents of OpenGL for the restructure
This implements the necessary fixes in the packages that depend on
OpenGL to work with the restructuring. This also attempts to create a
consistent variant for specifying glx or osmesa.
the patch urls dynamically generate a diff, which includes metadata
about the git version used, meaning they are not content-addressable.
instead ship the patches with spack.
In #31618 the idea was to determine the file extension heuristically by dropping query params etc from a url and then consider it as a file path. That broke for URLs that only have query params like http://example.com/?patch=x as it would result in empty string as basename. This PR reverts to the old behavior of saving files as ?patch=x in that case.
Co-authored-by: Stephen Sachs <stesachs@amazon.com>
* herwig3: change lhapdfsets dependency type to build
These data sets are needed for a check during build, but due to the difficulty of versioning the datasets it is preferred not to keep the them in the run environment.
* herwig3: explicitly state needed boost libs
* thepeg: explicitly state needed boost libs
* style
* stylestyle
* Tau must get GCC path from environment on Cray
self.compiler doesn't provide the path to the gcc compiler when using cray cc and the spack internal compiler overrides the location in PATH. If possible get the location from the GCC_PATH variable instead.
* Fix flake8 issues
* Update package.py
* llvm: Use variant when clauses for many of the expressed conflicts
* llvm: Remove the shared variant as it wasn't really used
* llvm: Remove unnecessary deps and make explicit the ones that are
* llvm: Cleanup patch conditions
* pocl: Update for llvm cleanup
* unit-test: update unparse package hash with the updated llvm package
* llvm: Fix ppc long double patching and add clarifying comments
`self.archive_file` is (among others) a symlink to a tarball. `extension()` on a
symlink will result in no extension. This patch fixes the behavior introduced in
https://github.com/spack/spack/pull/31618.
Co-authored-by: Stephen Sachs <stesachs@amazon.com>
When
1. Spack installs libtool,
2. system libtool is installed too, and
3. system automake is used
Spack passes system automake's `-I <prefix>` flag to itself, even though
it's a default search path. This takes precedence over spack's libtool
prefix dir. This causes the wrong `libtool.m4` file to be used (since
system libtool is in the same prefix as system automake).
And that leads to error messages about incompatible libtool, something
something LT_INIT.
fixes#31627
spack.mirror.get_all_versions now uses the package class
instead of the package object in its implementation.
Ensure spec is concrete before staging for mirrors
* Update open-ce patches for py-torch to us immutable URLs. Update magma dependency specs to be more explicit.
* Address comments for PR regarding URLs and conflicting variants.
Co-authored-by: Nicholas Cameron Sly <sly1@llnl.gov>
* rocblas: make tensile dependencies conditional
* Remove rocm-smi from the rocblas dependency list
rocm-smi was added to the rocblas dependency list because Tensile was a
dependency of rocBLAS and rocm-smi was a dependency of Tensile. However,
that reasoning was not correct.
Tensile is composed of three components:
1. A command-line tool for generating kernels, benchmarking them, and
saving the parameters used for generating the best kernels
(a.k.a. "Solutions") in YAML files.
2. A build system component that reads YAML solution files, generates
kernel source files, and invokes the compiler to compile then into
code object files (*.co, *.hsco). An index of the kernels and their
associated parameters is also generated and stored in either YAML
or MessagePack format (TensileLibrary.yaml or TensileLibrary.dat).
3. A runtime library that will load the TensileLibrary and code object
files when asked to execute a GEMM and choose the ideal kernel for
your specific input parameters.
rocBLAS developers use (1) during rocBLAS development. This is when
Tensile depends on rocm-smi. The GPU clock speed and temperature must be
controlled to ensure consistency when doing the solution benchmarking.
That control is provided by rocm-smi. When building rocBLAS, Tensile is
used for (2). However, there is no need for control of the GPU at that
point and rocm-smi is not a dependency. At runtime, the rocBLAS library
uses Tensile for (3), but there is again no dependency on rocm-smi.
tl;dr: rocm-smi is a dependency of the tensile benchmarking tool,
which is not a build dependency or runtime dependency of rocblas.
This PR contains several fixes for the kallisto package.
- create hdf5 variant as hdf5 is optional beginning with 0.46.2
- provide patch for 0.43 to link against libz
- provide patch for older versions to build again gcc-11 and up
- patch and code to use autoconf-2.70 and up
Newer versions of botocore (>=1.23.47) support the full IOBase
interface, so the hacks added to supplement the missing attributes are
no longer needed. Conditionally disable the hacks if they appear to be
unnecessary based on the class hierarchy found at runtime.
* py-panaroo: new package
* moving panaroo to branch
* updated mizani, plotnine, and pystan versions and requirements
* made suggested fixes
* adding more requested fixes
* added new versions of statsmodels and httpstan
* py-torch: add version 0.23.0 and fix to built on aarch64
* Add newer versions, fix build issues
* Fix tests
Co-authored-by: Adam J. Stewart <ajstewart426@gmail.com>
* Add connection information to buildcache update command
Ensure that the s3 connection made when attempting to update the content
of a buildcache attempts to use the extra connection information
from the mirror creation.
* Add unique help for endpoint URL argument
Fix copy/paste error for endpoint URL help which was the same as
the access token
* Re-work URL checking for S3 mirrors
Due to the fact that nested bucket URLs would never match the string used
for checking that the mirror is the same, switch the check used.
Sort all mirror URLs by length to have the most specific cases first
and see if the desired URL "starts with" the mirror URL.
* Long line style fixes
Add execptions for long lines and fix other style errors
* Use format() function to rebuild URL
Use the format command to rebuild the url instead of crafing a
formatted string out of known values
* Add early exit for URL checking
When a valid mirror is found, break from the loop
For a long time the module configuration has had a few settings that use
`blacklist`/`whitelist` terminology. We've been asked by some of our users to replace
this with more inclusive language. In addition to being non-inclusive, `blacklist` and
`whitelist` are inconsistent with the rest of Spack, which uses `include` and `exclude`
for the same concepts.
- [x] Deprecate `blacklist`, `whitelist`, `blacklist_implicits` and `environment_blacklist`
in favor of `exclude`, `include`, `exclude_implicits` and `exclude_env_vars` in module
configuration, to be removed in Spack v0.20.
- [x] Print deprecation warnings if any of the deprecated names are in module config.
- [x] Update tests to test old and new names.
- [x] Update docs.
- [x] Update `spack config update` to fix this automatically, and include a note in the error
that you can use this command.
Python's built-in tarfile support doesn't address some general
cases of malformed tarfiles that are already handled by the system
'tar' utility; until these can be addressed, use that exclusively.
Add support for CMake builds while preserving autotools support
for older versions of GDAL
* Add GDAL 3.5.0
* Remove GDAL 1
* Add support for new CMake build system
* Change defaults to build all recommended dependencies
* Simplify Autotools flag handling
* Determine version range for drivers
The goal of this PR is to make clearer where we need a package object in Spack as opposed to a package class.
We currently instantiate a lot of package objects when we could make do with a class. We should use the class
when we only need metadata, and we should only instantiate and us an instance of `PackageBase` at build time.
Modifications:
- [x] Remove the `spack.repo.get` convenience function (which was used in many places, and not really needed)
- [x] Use `spack.repo.path.get_pkg_class` wherever possible
- [x] Try to route most of the need for `spack.repo.path.get` through `Spec.package`
- [x] Introduce a non-data descriptor, that can be used as a decorator, to have "class level properties"
- [x] Refactor unit tests that had to be modified to reduce code duplication
- [x] `Spec.package` and `Repo.get` now require a concrete spec as input
- [x] Remove `RepoPath.all_packages` and `Repo.all_packages`
* fixed the cgal recipe and added the latest release.
* Update var/spack/repos/builtin/packages/cgal/package.py
Co-authored-by: Adam J. Stewart <ajstewart426@gmail.com>
* updated cgal recipe to new URL for tarballs
Co-authored-by: Adam J. Stewart <ajstewart426@gmail.com>
There's a race condition in `remove()` as the lockfile is removed after
releasing the lock, which is a problem when another process acquires a
write lock during deletion.
Also simplify life a bit in multiprocessing when a file is possibly
removed multiple times, which currently is an error on the second
deletion, so the proposed fix is to make remove(...) idempotent and not
error when deleting non-existing cache entries.
Don't tests for existence of lockfile, cause windows/linux behavior is different
Oversight in #31433, the non-phony `env` target was missing a file being
created for it, which can cause make to infinitely loop when including
multiple generated makefiles.
* Metall package: add dependency to GCC for build test
* Package Metall: add v.017
* Package Metall: update the package file
* Update var/spack/repos/builtin/packages/metall/package.py
Co-authored-by: Bernhard Kaindl <43588962+bernhardkaindl@users.noreply.github.com>
* Metall package: add v0.18 and v0.19
* Metall Package: add v0.20
* Metall package: add v0.21
Co-authored-by: Bernhard Kaindl <43588962+bernhardkaindl@users.noreply.github.com>
When no default editor is installed and no environment variable is set,
which_string would return None and this would be passed to os.execv
resulting in a TypeError. The message presented to the user would be:
Error: execv: path should be string, bytes or os.PathLike,
not NoneType
This change checks that which_string has returned successfully before
attempting to execute the result, resulting in a new error message:
Error: No text editor found! Please set the VISUAL and/or EDITOR
environment variable(s) to your preferred text editor.
It's not strictly necessary, but I've also changed try_exec to catch
all errors rather than just OSErrors. This would have provided slightly
more context for the original error message.
Use 8 byte reals only when `precision=double` is set
The pre-defined compilation targets do not follow the usual behavior of
Makefiles. Compiler flags are set as strings (not Makefile variables) and as
such are not able to be overridden by environment variables. This patch changes
the behavior to the expected behavior of a Makefile such that `fflags` etc have
the desired effect.
Co-authored-by: Stephen Sachs <stesachs@amazon.com>
* [py-ipyparallel] setuptools.config.StaticModule moved
... in py-setuptools@61
* [py-ipyparallel] setuptools fix only added to release 8.2.1
https://github.com/ipython/ipyparallel/pull/680
* [py-ipyparallel] Fix typo
Co-authored-by: Stephen Sachs <stesachs@amazon.com>
There were two choices: 1) remove '-p' from '-a' or 2) allow monkeypatching
the cleaning of the python cache since clean's test_function_calls isn't
supposed to be actually cleaning anything.
This commit supports the latter and adds a test case for `-p`.
* add recipe for improved-rdock
* style: fix format
* style: fix format again
* Fix location of the directory
* Fix copyright
* Fix according to the reviewer's comments.
* Revert "Fix according to the reviewer's comments."
This reverts commit 4877877daf.
* Fix according to the reviewer's comments.
* style: fix format
* Update var/spack/repos/builtin/packages/improved-rdock/package.py
Co-authored-by: Tamara Dahlgren <35777542+tldahlgren@users.noreply.github.com>
Co-authored-by: Yuichi Otsuka <otsukay@riken.jp>
Co-authored-by: Tamara Dahlgren <35777542+tldahlgren@users.noreply.github.com>
* guppy: new package
* adding different hashs/urls
* working now
* guppy: new package
* ont-guppy: new package
* ont-guppy: new package
* ont-guppy: fixed style
- rocprim is a header-only library; its amdgpu_target variant is only
meaningful for its client executables, not the library itself
- comgr and llvm-amdgpu are merely indirect dependencies (via hip)
* Initial implementation of Omnitrace package
* Fix flake8 errors
* Fix run environment when +python
* String normalization and fix for build env when +tau
* Tweak to style
Release branches and tags run protected pipelines, and we noticed
that those pipelines were generating all jobs in the stack, even
when the mirror contained all the built specs and an up to date
index. The problem was caused because the override mirror was
not present in spacks mirror configuration at the time when the
binary_distribution.update() method was called. This fixes that
by always adding the mirror override, if present, to the list of
configured mirrors.
* py-altair: new package
* Update var/spack/repos/builtin/packages/py-altair/package.py
Co-authored-by: Adam J. Stewart <ajstewart426@gmail.com>
* Update var/spack/repos/builtin/packages/py-altair/package.py
Co-authored-by: Adam J. Stewart <ajstewart426@gmail.com>
* Update var/spack/repos/builtin/packages/py-altair/package.py
Co-authored-by: Adam J. Stewart <ajstewart426@gmail.com>
* py-altair: update
* Update var/spack/repos/builtin/packages/py-altair/package.py
Co-authored-by: Adam J. Stewart <ajstewart426@gmail.com>
Co-authored-by: Adam J. Stewart <ajstewart426@gmail.com>
* boost: do not access the context-impl variant for versions below 1.65.0
* boost: check if spec.variants contains context-impl
* boost: improve error message when the context-impl variant causes a conflict
Executing spack solve boost@1.63.0 +context context-impl=fcontext
triggers the following error message:
==> Error: No version for 'boost' satisfies '@1.63.0' and '@1.79.0'
With this change, the error message becomes the following:
==> Error: Cannot set variant 'context-impl' for package 'boost' because the variant condition cannot be satisfied for the given spec
We adopted the convention of putting binaries for each stack into
a dedicated mirror named after the directory in which the stack
(spack.yaml file) resides. This fixes the mirror url of the
radiuss-aws-aarch64 stack to follow that convention.
`make_target` can be used to instruct Spack to build one of the pre-defined make
targets in the MPAS makefile. Spack will guess the targret based on the value of
`spack_fc`, but the user can overwrite the target as variant. E.g.
```
spack install mpas-model make_target=pgi-nersc
```
`precision` is used to set the `PRECISION` flag in the Makefile to {single,
double}. Default is double.
Co-authored-by: Stephen Sachs <stesachs@amazon.com>
* remove unhelpful comment
* Filter compiler duplicates while reading manifest
* more-specific version matching edited to use module-specific version (to avoid an issue where a user might add a compiler with the same version to the initial test configuration
* Set R_LIBS_USER='' in dependent build environment
Despite R packages being installed with the --vanilla flag, which
ignores Rprofile and Renviron files, R will still set R_LIBS_USER if the
default directory exists. This could lead to pulling in packages from
that directory during the build which could cause the build to fail. To
avoid that, explicitly set R_LIB_USER='' to ensure that packages from
the HOME/R directory are not in the library path.
* Update var/spack/repos/builtin/packages/r/package.py
Co-authored-by: Adam J. Stewart <ajstewart426@gmail.com>
Co-authored-by: Adam J. Stewart <ajstewart426@gmail.com>
* PythonPackage: add default libs/headers attributes
* Style fix
* libs and headers should be properties
* Check both platlib and include
* Fix variable reference
Building on #24639, this allows versions to be prefixed by `git.`. If a version begins `git.`, it is treated as a git ref, and handled as git commits are starting in the referenced PR.
An exception is made for versions that are `git.develop`, `git.main`, `git.master`, `git.head`, or `git.trunk`. Those are assumed to be greater than all other versions, as those prefixed strings are in other contexts.
This commit adds some changes which improve use of Spack-installed
oneAPI packages with Spack-generated modules, but not within Spack
(e.g. if you install some of these packages with Spack, then load
their associated modules to build other packages outside of Spack).
The majority of the PR diff is refactoring. The functional changes
are:
* intel-oneapi-mkl:
* setup_run_environment: update Intel compiler flags to RPATH the
mkl libs
* intel-oneapi-compilers: update the compiler configuration to RPATH
libraries installed by this package (note that Spack already handled
this when installing dependent packages with Spack, but this is
specifically to use the intel-oneapi-compilers package outside
of Spack). Specifically:
* inject_rpaths: this modifies the binaries installed as part of
the intel-oneapi-compilers package to RPATH libraries provided
by this package
* extend_config_flags: this instructs the compiler executables
provided by this package to RPATH those same libraries
Refactoring includes:
* intel-oneapi-compilers: in addition to the functional changes,
a large portion of this diff is reorganizing the creation of
resources/archives downloaded for the install
* The base oneAPI package renamed component_path, so several packages
changed as a result:
* intel-oneapi-dpl
* intel-oneapi-dnn
* extrae
* intel-oneapi-mpi:
* Perform file filtering in one pass rather than two
Allow `spack external find` (with no extra args) to proceed if the manifest file exists but
without sufficient permissions; in that case, print a warning. Also add a test for that behavior.
TODOs:
- [x] continue past any exception raised during manifest parsing as part of `spack external find`,
except for CTRL-C (and other errors that indicate immediate program termination)
- [x] Semi-unrelated but came up when discussing this with the user who reported this issue to
me: the manifest parser now accepts older schemas
See: https://github.com/spack/spack/issues/31191
nvc doesn't know this flag: `-Wno-unknown-pragmas`, it doesn't hurt to
remove the specific warning flags, from:
```
-O3 -Wall -Wextra -Wno-unknown-pragmas -Wcast-qual
```
to:
```
-O3 -Wall
```
2022-06-27 14:41:40 +02:00
7809 changed files with 232143 additions and 177531 deletions
description:Some package in Spack had stand-alone tests that didn't pass
title:"Testing issue: "
labels:[test-error]
body:
- type:textarea
id:reproduce
attributes:
label:Steps to reproduce the failure(s) or link(s) to test output(s)
description:|
Fill in the test output from the exact spec that is having stand-alone test failures. Links to test outputs (e.g., CDash) can also be provided.
value:|
```console
$ spack spec -I <spec>
...
```
- type:textarea
id:error
attributes:
label:Error message
description:|
Please post the error message from spack inside the `<details>` tag below:
value:|
<details><summary>Error message</summary><pre>
...
</pre></details>
validations:
required:true
- type:textarea
id:information
attributes:
label:Information on your system or the test runner
description:Please include the output of `spack debug report` for your system.
validations:
required:true
- type:markdown
attributes:
value:|
If you have any relevant configuration detail (custom `packages.yaml` or `modules.yaml`, etc.) you can add that here as well.
- type:textarea
id:additional_information
attributes:
label:Additional information
description:|
Please upload test logs or any additional information about the problem.
- type:markdown
attributes:
value:|
Some packages have maintainers who have volunteered to debug build failures. Run `spack maintainers <name-of-the-package>` and **@mention** them here if they exist.
- type:checkboxes
id:checks
attributes:
label:General information
options:
- label:I have reported the version of Spack/Python/Platform/Runner
required:true
- label:I have run `spack maintainers <name-of-the-package>` and **@mentioned** any maintainers
required:true
- label:I have uploaded any available logs
required:true
- label:I have searched the issues of this repo and believe this is not a duplicate
@@ -1093,6 +1097,8 @@ could depend on ``mpich@1.2:`` if it can only build with version
Below are more details about the specifiers that you can add to specs.
.._version-specifier:
^^^^^^^^^^^^^^^^^
Version specifier
^^^^^^^^^^^^^^^^^
@@ -1108,6 +1114,37 @@ set of arbitrary versions, such as ``@1.0,1.5,1.7`` (``1.0``, ``1.5``,
or ``1.7``). When you supply such a specifier to ``spack install``,
it constrains the set of versions that Spack will install.
For packages with a ``git`` attribute, ``git`` references
may be specified instead of a numerical version i.e. branches, tags
and commits. Spack will stage and build based off the ``git``
reference provided. Acceptable syntaxes for this are:
..code-block::sh
# branches and tags
foo@git.develop # use the develop branch
foo@git.0.19 # use the 0.19 tag
# commit hashes
foo@abcdef1234abcdef1234abcdef1234abcdef1234 # 40 character hashes are automatically treated as git commits
foo@git.abcdef1234abcdef1234abcdef1234abcdef1234
Spack versions from git reference either have an associated version supplied by the user,
or infer a relationship to known versions from the structure of the git repository. If an
associated version is supplied by the user, Spack treats the git version as equivalent to that
version for all version comparisons in the package logic (e.g. ``depends_on('foo', when='@1.5')``).
The associated version can be assigned with ``[git ref]=[version]`` syntax, with the caveat that the specified version is known to Spack from either the package definition, or in the configuration preferences (i.e. ``packages.yaml``).
..code-block::sh
foo@git.my_ref=3.2 # use the my_ref tag or branch, but treat it as version 3.2 for version comparisons
foo@git.abcdef1234abcdef1234abcdef1234abcdef1234=develop # use the given commit, but treat it as develop for version comparisons
If an associated version is not supplied then the tags in the git repo are used to determine
the most recent previous version known to Spack. Details about how versions are compared
and how Spack determines if one version is less than another are discussed in the developer guide.
If the version spec is not provided, then Spack will choose one
according to policies set for the particular spack installation. If
the spec is ambiguous, i.e. it could match multiple versions, Spack
@@ -1193,6 +1230,23 @@ variants using the backwards compatibility syntax and uses only ``~``
for disabled boolean variants. The ``-`` and spaces on the command
line are provided for convenience and legibility.
Spack allows variants to propagate their value to the package's
dependency by using ``++``, ``--``, and ``~~`` for boolean variants.
For example, for a ``debug`` variant:
..code-block::sh
mpileaks ++debug # enabled debug will be propagated to dependencies
mpileaks +debug # only mpileaks will have debug enabled
To propagate the value of non-boolean variants Spack uses ``name==value``.
For example, for the ``stackstart`` variant:
..code-block::sh
mpileaks stackstart=4# variant will be propagated to dependencies
mpileaks stackstart==4# only mpileaks will have this variant value
^^^^^^^^^^^^^^
Compiler Flags
^^^^^^^^^^^^^^
@@ -1200,10 +1254,15 @@ Compiler Flags
Compiler flags are specified using the same syntax as non-boolean variants,
but fulfill a different purpose. While the function of a variant is set by
the package, compiler flags are used by the compiler wrappers to inject
flags into the compile line of the build. Additionally, compiler flags are
inherited by dependencies. ``spack install libdwarf cppflags="-g"`` will
install both libdwarf and libelf with the ``-g``flag injected into their
compile line.
flags into the compile line of the build. Additionally, compiler flags can
be inherited by dependencies by using ``==``.
``spack install libdwarf cppflags=="-g"``will install both libdwarf and
libelf with the ``-g`` flag injected into their compile line.
..note::
versions of spack prior to 0.19.0 will propagate compiler flags using
the ``=`` syntax.
Notice that the value of the compiler flags must be quoted if it
contains any spaces. Any of ``cppflags=-O3``, ``cppflags="-O3"``,
@@ -1405,7 +1464,7 @@ built.
You can see what virtual packages a particular package provides by
getting info on it:
..command-output:: spack info mpich
..command-output:: spack info --virtuals mpich
Spack is unique in that its virtual packages can be versioned, just
like regular packages. A particular version of a package may provide
Notice that this function, if you define it, requires a result object (generated by
``run()``, a monitor (if you want to send), and a boolean ``overwrite`` to be used
to check if a result exists first, and not write to it if the result exists and
overwrite is False. Also notice that since we already saved these files to the analyzer metadata folder, we return early if a monitor isn't defined, because this function serves to send results to the monitor. If you haven't saved anything to the analyzer metadata folder
yet, you might want to do that here. You should also use ``tty.info`` to give
the user a message of "Writing result to $DIRNAME."
.._writing-commands:
@@ -699,23 +446,6 @@ with a hook, and this is the purpose of this particular hook. Akin to
``on_phase_success`` we require the same variables - the package that failed,
the name of the phase, and the log file where we might find errors.
"""""""""""""""""""""""""""""""""
``on_analyzer_save(pkg, result)``
"""""""""""""""""""""""""""""""""
After an analyzer has saved some result for a package, this hook is called,
and it provides the package that we just ran the analysis for, along with
the loaded result. Typically, a result is structured to have the name
of the analyzer as key, and the result object that is defined in detail in
:ref:`analyzer_run_function`.
..code-block::python
defon_analyzer_save(pkg,result):
"""given a package and a result...
"""
print('Do something extra with a package analysis result here')
^^^^^^^^^^^^^^^^^^^^^^
Adding a New Hook Type
@@ -1233,8 +963,13 @@ completed, the steps to make the point release are:
$ git checkout releases/v0.15
#. If a pull request to the release branch named ``Backports vX.Y.Z`` is not already
in the project, create it. This pull request ought to be created as early as
possible when working on a release project, so that we can build the release
commits incrementally, and identify potential conflicts at an early stage.
#. Cherry-pick each pull request in the ``Done`` column of the release
project board onto the release branch.
project board onto the ``Backports vX.Y.Z`` pull request.
This is **usually** fairly simple since we squash the commits from the
vast majority of pull requests. That means there is only one commit
@@ -1259,7 +994,7 @@ completed, the steps to make the point release are:
It is important to cherry-pick commits in the order they happened,
otherwise you can get conflicts while cherry-picking. When
cherry-picking onto a point release, look at the merge date,
cherry-picking look at the merge date,
**not** the number of the pull request or the date it was opened.
Sometimes you may **still** get merge conflicts even if you have
@@ -1280,15 +1015,19 @@ completed, the steps to make the point release are:
branch if neither of the above options makes sense, but this can
require a lot of work. It's seldom the right choice.
#.Bump the version in ``lib/spack/spack/__init__.py``.
#.When all the commits from the project board are cherry-picked into
the ``Backports vX.Y.Z`` pull request, you can push a commit to:
#. Update ``CHANGELOG.md`` with a list of the changes.
1. Bump the version in ``lib/spack/spack/__init__.py``.
2. Update ``CHANGELOG.md`` with a list of the changes.
This is typically a summary of the commits you cherry-picked onto the
==> Installing "patchelf@0.16.1%gcc@10.2.1 ldflags="-static-libstdc++ -static-libgcc" build_system=autotools arch=linux-centos7-x86_64" from a buildcache
Some files were not shown because too many files have changed in this diff
Show More
Reference in New Issue
Block a user
Blocking a user prevents them from interacting with repositories, such as opening or commenting on pull requests or issues. Learn more about blocking a user.
