This patch adds license information for about 5,300 packages from automated sources.
The license information was obtained from Alpine Linux and PyPI and processed
using tooling available in https://github.com/boomanaiden154/spack-license-utils.
The license field was added in after all other directives in an automated fashion.
Note that while this license information is probably fairly accurate, it is not
guaranteed to be accurate. In addition some of the license strings from Alpine Linux
might not be valid SPDX license strings. Invalid SPDX identifiers can be picked up
and fixed once we have validation/parsing infrastructure in place for the solver,
and issues can be fixed as they come up.
* Updating the LBANN, Hydrogen, and DiHydrogen recipes for both new
variants and to make sure that RPATHs are properly setup.
Co-authored-by: bvanessen <bvanessen@users.noreply.github.com>
* Style: black 23, skip magic trailing commas
* isort should use same line length as black
* Fix unused import
* Update version of black used in CI
* Update new packages
* Update new packages
* Added support for building the DiHydrogen package and LBANN extensions
to DiHydrogen with ROCm libraries.
Fixed a bug on Cray systems where CMake didn't try hard enough to find
an MPI-compatible compiler wrapper. Make it look more.
Added support for the roctracer package when using ROCm libraries.
* Fixed how ROCm support is defined for pre-v0.3 versions.
Explicitly import package utilities in all packages, and corresponding fallout.
This includes:
* rename `spack.package` to `spack.package_base`
* rename `spack.pkgkit` to `spack.package`
* update all packages in builtin, builtin_mock and tutorials to include `from spack.package import *`
* update spack style
* ensure packages include the import
* automatically add the new import and remove any/all imports of `spack` and `spack.pkgkit`
from packages when using `--fix`
* add support for type-checking packages with mypy when SPACK_MYPY_CHECK_PACKAGES
is set in the environment
* fix all type checking errors in packages in spack upstream
* update spack create to include the new imports
* update spack repo to inject the new import, injection persists to allow for a deprecation period
Original message below:
As requested @adamjstewart, update all packages to use pkgkit. I ended up using isort to do this,
so repro is easy:
```console
$ isort -a 'from spack.pkgkit import *' --rm 'spack' ./var/spack/repos/builtin/packages/*/package.py
$ spack style --fix
```
There were several line spacing fixups caused either by space manipulation in isort or by packages
that haven't been touched since we added requirements, but there are no functional changes in here.
* [x] add config to isort to make sure this is maintained going forward
Recipes that are not actually required for LBANN or DiHydrogen to
build. These should be concretized within the same environment or
installed via PIP using the same Python that installed LBANN.
Removing these will help eliminate build time failures that are
actually associated with Python tools, not LBANN.
* Added support to LBANN, Hydrogen, DiHydrogen, and Aluminum to capture
a gcc-toolchain cxxflags argument and pass it to a CMAKE_CUDA_FLAG
argument when set. This helps deal with compiling with clang on
systems with old base gcc installations.
* Added a dependency on py-scipy when enabling tests on LBANN.
* Updated the C++ standard for Hydrogen to C++17.
* Added a new variant +apps to enable (or disable) python packages that
are used by applications in the LBANN repo, but are not strictly
required for building and using LBANN.
* Added a run time dependency for both py-pytest and py-scipy so that
they are activated in any environment.
* Added support for building LBANN, Hydrogen, and DiHydrogen with the
IBM ESSL BLAS library. This requires explicit identification of
additional LAPACK libraries, since ESSL does not implement LAPACK, but
is found by CMake.
* Fixed a bug in the LBANN dependency on OpenCV for Power architectures.
The +powerpc variant is only required for GCC toolchains and causes
Clang to break. Switched to only enabling when using %gcc on power.
* fix remaining flake8 errors
* imports: sort imports everywhere in Spack
We enabled import order checking in #23947, but fixing things manually drives
people crazy. This used `spack style --fix --all` from #24071 to automatically
sort everything in Spack so PR submitters won't have to deal with it.
This should go in after #24071, as it assumes we're using `isort`, not
`flake8-import-order` to order things. `isort` seems to be more flexible and
allows `llnl` mports to be in their own group before `spack` ones, so this
seems like a good switch.
These were deprecated when the custom cuda_arch list was
removed. Also fixed up the Aluminum dependencies for Hydrogen and
DiHydrogen. Turns out that Aluminum v0.6.0 didn't have a correct
version in CMake and thus the interaction with older versions of
Hydrogen and DiHydrogen needed to be corrected.
* Fixed a bug in the DiHydrogen package where the variant legacy was
changed to distconv and wasn't fully propagated. Cleaned up the
openmp variants on the blas library packages in DiHydrogen and
Elemental. Extended support for Aluminum v1.0 in LBANN, Hydrogen, and
DiHydrogen. Fixed a when clause in the LBANN dependencies.
* Removed the upper range limit for the Aluminum library dependence
* Update var/spack/repos/builtin/packages/dihydrogen/package.py
Co-authored-by: Adam J. Stewart <ajstewart426@gmail.com>
Co-authored-by: Adam J. Stewart <ajstewart426@gmail.com>
Updated the versions for DiHydrogen and Aluminum. Added new constraints on versions of Aluminum that are used across the software stack. Cleaned up the dependency on DiHydrogen for LBANN.
* Made DiHydrogen a required dependencies on newer versions of LBANN.
Added an explicit variant for enabling Boost-dependent callbacks.
Updated the separation for embedded Python and the Python front end
code and associated dependencies.
* Bugfix on ROCm include in DiHydrogen
* Also removed LBANN CUDA CMake flags that are set by the
version of Hydrogen that is compiled against.
* Updated recipes to use HWLOC 2.3 with ROCm to enable
topology awareness.
Co-authored-by: Harmen Stoppels <harmenstoppels@gmail.com>
Set the minimun C++ standard for LBANN, Hydrogen, and DiHydrogen to
C++17. The minumim C++ standard for Aluminum is C++14. Add new
versions for Aluminum, Hydrogen, and DiHydrogen. Added support for
high performance linkers in LBANN recipe (gold and lld). Added
variants to LBANN for enabling embedded Python support independently
from the Python front end.
- [x] add `concretize.lp`, `spack.yaml`, etc. to licensed files
- [x] update all licensed files to say 2013-2021 using
`spack license update-copyright-year`
- [x] appease mypy with some additions to package.py that needed
for oneapi.py
* Adding support for the CMake flags in LBANN that are missing.
* Added new flag to OpenCV dependency and removed negative variants
since OpenCV no longer turns on everything by default. Removed CMake
flags in LBANN that have been deprecated.
* Removed type='build' flags from dependencies so that they get linked
into a environment's view.
* Removed type='build' flags from dependencies so that they get linked
into a environment's view. Fixed DiHydrogen variant to enable
DistConv feature, renamed to +distconv from +legacy. Added conflicts
line to indicated that DistConv and ROCm don't work with +half
support.
* Fixed Flake8 and cleaned up ordering of variants.
* Flake8
* Backed out changes to not mark and cmake and ninja as build
dependencies, which was introduced to make sure that they appear in
a spack environment.
* Backed out changes to not mark doc related packages as build
dependencies, which was introduced to make sure that they appear
in a spack environment.
* Fixed how recipe communicates the intent to build and run tests to the
package CMake.
* Added guard for setting CUB_DIR to only when cuda variant is true
* Added support for OpenMP on OSX platforms
* Updated the way that LBANN, Hydrogen, and DiHydrogen handle
apple-clang with OpenMP and Clang installed on OS X via brew.
* Fixed bug in spec resolution
* Fixed merge conflict
* Fixed typo
* Fixed flake8
* Added code to help DiHydrogen find cuDNN and CUB
* Cleaning up dependencies on CUB and adding guards for when newer
versions of CUDA include CUB and it should be excluded.
* Changed Hydrogen to disable half support by default.
* Have LBANN force Hydrogen and DiHydrogen to build without half when the variant is disabled.
* Added explicit variants to enusre that if LBANN is build without Cuda,
Aluminum, or Half support, it enforces those constraints for Hydrogen
and DiHydrogen. Cleaned up the use of Python extend versus append in
LBANN and DiHydrogen recipes.
* Fixed Flake8
* Updates in LBANN an Aluminum code now allow working with versions
HWLOC 1.11.x and 2.x and up.
* Updating the minimum CMake version to address a pending PR in LBANN
that will require C++17 support and needs CMake to properly separate
the compiler flags from nvcc.
* Clarified the support for different versions of HWLOC in LBANN
* Added hash values for LBANN v0.101 and Hydrogen v1.5.0. Updated the
LBANN package to be more successful in resolving a legal configuration
of MPI and HWLOC packages. This required the removal of the MPI
virtual package since it is unable to resolve dependencies with
minimum version requirements. As a result to enable a reasonable
install line for LBANN this requires explicit forwarding of MPI
variants to Hydrogen and Aluminum. Due to the lack of variant
forwarding, there are many explicitly replicated dependencies for both
LBANN and Hydrogen. Fixed the error in LBANN where gpu variant was
replaced by the cuda variant, but not all dependencies were fixed.
* Fixed the minumum cuDNN version for newer versions of LBANN.
* Added explicit versioning of the MPI libraries for DiHydrogen to avoid
all of the conflicts with minimum required versions of the OpenMPI library.
* Removed explicit MPI versions and went back to using the MPI virtual
dependency. Updated construction of variant forwarding to use
iterative construction of constraints and variants. This exacerbates
the challenges with backtracking in the current concretizer, but
should be fixed in the new concretizer.
* Added support for including the DiHydrogen library in LBANN as well as
support for the distributed convolution (DistConv) parallel
algorithms. Also include support for building with half precision.
* Moving dependencies around
* Added conflict statement to ensure that the variant dihydrogen is
required for distconv.
* Removed the preferred field
* Fixed Flake8 and cuDNN version bounds
* Update LBANN, Hydrogen, Aluminum to inherit CudaPackage
* Update CMake constraints: LBANN, Hydrogen, and Aluminum now require
cmake@3.16.0: (better support for pthreads with nvcc)
* Aluminum: add variants for host-enabled MPI and RMA features in a
MPI-GPU RDMA-enabled library
* NCCL: add versions 2.7.5-1, 2.7.6-1, and 2.7.8-1
* Hydrogen: add version 1.4.0
* LBANN: add versions 0.99 and 0.100
* Aluminum: add versions 0.4.0 and 0.5.0
