* [mfem] A few updates: add 'strumpack' variant; add 'zlib'
variant (same as 'gzstream'); fix optmization flag
for v4.0.
* [mfem] flake8 fix
* [mfem] Add version 4.1
* [mfem] Add/tweak some 'conflicts' directives.
* [gslib] Add new release versions + 'develop' version.
* [petsc] Restrict hdf5 version to <= 1.10.99 since 1.12.0 fails
* [metis] Use the original metis url for v4.0.3.
* [conduit] Remove restrictions to the used hdf5 variant to allow
building with other packages that use hdf5, e.g. petsc.
* [mfem] Few updates:
* Replace the 'gzstream' variant with 'zlib' variant.
* Do not add system library paths with -L flags.
* Allow '+pumi+shared' variant.
* Update the 'test_builds.sh' script.
* [occa] Add version 1.0.9.
* [mfem] Some OCCA and RAJA updates.
* [gslib] Fix the build for new versions of the library.
* [mfem] Add 'gslib' variant for GSLIB.
* [mfem] Add 'cuda' variant.
* [mfem] Add 'libceed' variant + a few more tweaks.
* [mfem] Add 'umpire' variant.
* [ceed] Add a draft for v3.0. Not tested. Just made sure that
concretization works for 'ceed' and 'ceed+cuda'.
* [nek] Fix Nek5000/NekCEM
* [nek] Add Nek5000-v19 & polishing Nek packages
* [flake8] Fix flake8 failure
* petsc: use of HDF5 does not care about +hl+fortran
* [petsc] Temporarily allow any hypre version with petsc@develop.
[ceed] Remove the requirement for hypre@develop.
* [libceed] Do not explicitly set NVCCFLAGS for v0.5 and later.
* [laghos] Add version 3.0, pointing to dev branch for now.
Do not set CXX at the make command line.
Simplify the dependecy directives a little.
[ceed] Use laghos v3.0 for ceed v3.0.0.
* [laghos] Keep the injection of CXX in the makefile for laghos
versions <= 2.0.
* [nekcem] Recovert hash-versions used by older versions of the
'ceed' package.
* [occa] Disable hip autodetection because it fails on some machines.
* [laghos] Update v3.0 with the actual release source.
* [suite-sparse] Explicitly add the c11 flag to CFLAGS.
* Update package.py (#15749)
* [magma] Add forgotten specification of the 'cuda_arch' variant.
* [ceed] Use magma v2.5.3 for ceed v3.0.
* libceed-0.6
* mfem: depend on libceed 0.6:, not 0.6.0:
* [libceed] Add 'magma' variant -- enable MAGMA backend.
* [ceed] In v3.0, use '+magma' variant of libceed when cuda is enabled.
* Initial package for Remhos (needs to be updated with actual sha256
* Adding Remhos to CEED-3.0, for now @develop
* petsc: add 3.13.0 (using petsc-lite) and 3.12.5
* ceed: update to petsc@3.13.0:3.13.99
* Temporary fix
* [nekcem] Add hash-version for ceed v3.0.
* [nek5000] Simplify source urls.
* [nektools] Use the same sources and versions as in nek5000.
* [ceed] Update Nek-related package versions.
* libceed: add v0.6 portabilty fix
* libceed: better v0.6 portabilty fix
* Adding Remhos 1.0 release in CEED-3.0
* Updating hash for Remhos-1.0
* [petsc] Add cuda variant.
* [libceed] Flake8 fix.
* [petsc] Add cuda variant.
* [ceed] Fix the OCCA version to 1.0.9. Enable petsc+cuda when
compiling ceed@3.0.0+cuda.
* nek5000: fix python 2.7+ syntax
* [laghos] Fix testing.
* [remhos] Fix testing.
* [remhos] For testing use the 'tests' target instead of 'test'.
* Add/update the maintainers for ceed, libceed, mfem, laghos, and remhos.
* [ceed] Remove unnecessary dependencies.
* libceed: activate AVX when supported
Co-authored-by: Thilina Rathnayake <thilinarmtb@gmail.com>
Co-authored-by: Jed Brown <jed@jedbrown.org>
Co-authored-by: Stan Tomov <tomov@eecs.utk.edu>
Co-authored-by: Tzanio <tzanio@llnl.gov>
* fix metis src dl url
* update ascent, vtk-h and vtk-m recipes
* update conduit package
* fix vtk-m shas
* mfem conduit fix
* use vtk-h develop
* fix issue with stripped include paths in mfem
* more metis fixes
* simpler fix for mfem conduit include issue
* finish mfem changes
* pin to cmake 3.14, since we hit cuda issues with 3.15
* add rtd theme as dep for ascent
* add vtk-h 0.5.0 release, update ascent to use it
* add ascent 0.5.0 release
* fix cmake pin to allow all vers of 3.14
* fix format string error in mfem pkg
* review fixes for mfem pkg
* review fixes for vtk-h and vtk-m packages
* address review comments for ascent pkg
* changing default off of develop broke downstream use
* revert prefed
* add support for static (via ~shared) and use vtk-m 1.2
* updating vtkh package to output cmake configure file and pinning it to vtkm 1.2
* trying a different cmake for vtkh
* removing problematic b
* making conduit respect ~python
* fixing ascent python logic
* update ascent package
* consistant cmake usage
* conditionally add tbb in ascent if vtkh
* applying becker fix
* adding vtkh tag
* fixing vtkh tagged version
* updating ascent and conduit for static builds
* enabling openmp
* reverting files that should not have been changed
* ascent updates
* more robust handling of variants
* fixing ascent package typo
* ascent: add optional support for mfem
* enable mfem conduit support for ascent
* add optional adios dep to conduit
* remove ver req from conduit
* ascent: remove confusing comment
* tweaks to conduit and ascent pkg recipes
* fix typo in conduit package
* pref conduit master
* fixing mfem typo for ascent
* reverting files
* adding mirror for bzip
* changing mfem to depend on conduit@master to get updated relay
* restore use of conduit 0.3.1 or greater for mfem
* set master as prefered conduit version
* allow mfem to use conduit master
* adding rover package and editing ascent
* updating vtkm cmake dep
* updates from axom
* guard ascent python support on +shared
* removing rover to simply ascent package
* add fortran variant to conduit, to allow us to turn off conduit support even when a fortran compiler is specified
* fix fortran compiler check so it can work on cray systems
* working towards cuda fix for vtkm lagrange filter
* update ascent package with more variants, and patch to avoid nvcc issue
* hdf5 flags fix for BGQ
* flake8
* extra guards for cuda patch
* conduit and ascent fortran fix
* fix patch for non cuda case
* add test variant to conduit, tweak ascent pkg
* change min ver of cmake used for ascent
* h5z-zfp package: unset FC when ~fortran
* conform to expected upstream solution
* pinning vtkm
* going back to vtkm master
* add back vtk-m variant for shared libs
* update ascent and vtkh packages
* wire up option to run tests during install
* add post install test
* add testing to ascent
* tweak for blueos xl
* add ctest output on error for run_tests
* enable ctest output on error for run_tests
* add testing of the using-with-make example
* update using-with-make examples
* typo in ascent using-with-cmake test
* fix ascent using test exe names
* more fixes, less sleep
* more fixes, less sleep ...
* remove unwired up version
* improvments suggested on review
* adding new cmake
* Update package.py
* Update package.py
* changes post cori os update
* fix cray hack
* Update package.py
Fixing 'fix'. Inconsistent variable names in conduit package
* type in spack recipes
* add zfp support to conduit
* fix indent error in conduit pkg
* move to use build phases, add sphinx rtd as dep, fix ex names in tests
* add conduit 0.5.0 release
* flake8
* remove old cray hack
* incorp feedback from review
* fix to use proper build env sig
We'd like to use a consistent checksum scheme everywhere so that we can:
a) incorporate archive checksums into our specs and have a
consistent hashing algorithm across all specs.
b) index mirrors with a consistent type of checksum, and not one that
is dependent on how spack packages are written.
- [x] convert existing md5, sha224, sha512, sha1 checksums to sha256
* LBANN: add python dependency and required python library dependencies
* LBANN: add version 0.99 as alias for develop branch
* LBANN: build Conduit library by default
* LBANN: CUB is included with Hydrogen for later versions
* New package: py-graphviz (dependency for LBANN)
* New package: py-texttable (dependency for LBANN)
* New package: py-onnx (dependency for LBANN)
* New package: py-typing-extensions (dependency for py-onnx)
* Conduit: add version 0.4.0
* NCCL: Add versions 2.4.6-1 and 2.4.2-1
* Protobuf: Add versions 3.7.1 and 3.7.0
* py-protobuf: add version 3.7.1
* py-typing: add version 3.6.4
* cuDNN: add versions. The source differs depending on the
architecture, so the versions include a suffix to encode this.
For example this adds 7.5.1-10.1-x86_64 as well as
7.5.1-10.1-ppc64le
* cuDNN: Newer versions require newer versions of CUDA
* Hydrogen: add configure options for CUB and Aluminum
This adds a lot of fixes and updates for alpine ascent and its dependencies:
* add support for static (via ~shared) and use vtk-m 1.2
* update vtkh package to output cmake configure file and pinning it to vtkm 1.2
* make conduit respect ~python
* fix ascent python logic
* consistant cmake usage
* conditionally add tbb in ascent if vtkh
* enable openmp
* more robust handling of variants
* update ascent and conduit for static builds
* add optional support for mfem in ascent
* enable mfem conduit support for ascent
* add optional adios dep to conduit
* remove ver req from conduit
* ascent: remove confusing comment
* tweaks to conduit and ascent pkg recipes
* fix typo in conduit package
* pref conduit master
* changing mfem to depend on conduit@master to get updated relay
* restore use of conduit 0.3.1 or greater for mfem
* set master as preferred conduit version
* allow mfem to use conduit master
* adding rover package and editing ascent
* updating vtkm cmake dep
* guard ascent python support on +shared
* removing rover to simply ascent package
* add fortran variant to conduit, to allow us to turn off conduit support even when a fortran compiler is specified
* fix fortran compiler check so it can work on cray systems
* working towards cuda fix for vtkm lagrange filter
* update ascent package with more variants, and patch to avoid nvcc issue
* hdf5 flags fix for BGQ
* add post install test
* add testing to ascent
* add testing of the using-with-make example
* add ctest output on error for run_tests
* Added support for the Aluminum library to LBANN and Hydrogen. Also
fixed several bugs in the grouping of dependencies of both packages.
* Updated the conduit package to have the proper dependency on the
python variant.
* Added new versions for NCCL
* Fixed a bug in how Hydrogen set the path for OpenBLAS.
* Added support for conduit in LBANN.
- remove the old LGPL license headers from all files in Spack
- add SPDX headers to all files
- core and most packages are (Apache-2.0 OR MIT)
- a very small number of remaining packages are LGPL-2.1-only
* Remove variants disabling blas and lapack for py-numpy, issues
building these have been resolved
* For CMake greater than 3.10, FindMPI changed, so use
MPIEXE_EXECUTABLE instead of MPIEXE for 3.10 and newer
* optional path to use spacks py-site-pkgs, install host-cfg for provenance
* add more comments and address review requests
* only use st-pkgs-dir when +python
* ascent does not need doxygen
* capture mpiexec if it exists
* added missing import
* flake8 fix
* default adios variant to False
* ascent develop now reqs conduit master
* optional path to use spacks py-site-pkgs, install host-cfg for provenance
* add more comments and address review requests
* only use st-pkgs-dir when +python
* ascent does not need doxygen
* capture mpiexec if it exists
* added missing import
* flake8 fix
* default adios variant to False
* ascent develop now reqs conduit master
* conduit: use hdf5 1.8 for wider output compat
* add ascent package and and deps
* proper use of site_packages_dir prop
* flake8
* add maitain, small updates
* flake8
* flake8
* fixs for docstrings for sphinx
## Motivation
Python installations are both important and unfortunately inconsistent. Depending on the Python version, OS, and the strength of the Earth's magnetic field when it was installed, the name of the Python executable, directory containing its libraries, library names, and the directory containing its headers can vary drastically.
I originally got into this mess with #3274, where I discovered that Boost could not be built with Python 3 because the executable is called `python3` and we were telling it to use `python`. I got deeper into this mess when I started hacking on #3140, where I discovered just how difficult it is to find the location and name of the Python libraries and headers.
Currently, half of the packages that depend on Python and need to know this information jump through hoops to determine the correct information. The other half are hard-coded to use `python`, `spec['python'].prefix.lib`, and `spec['python'].prefix.include`. Obviously, none of these packages would work for Python 3, and there's no reason to duplicate the effort. The Python package itself should contain all of the information necessary to use it properly. This is in line with the recent work by @alalazo and @davydden with respect to `spec['blas'].libs` and friends.
## Prefix
For most packages in Spack, we assume that the installation directory is `spec['python'].prefix`. This generally works for anything installed with Spack, but gets complicated when we include external packages. Python is a commonly used external package (it needs to be installed just to run Spack). If it was installed with Homebrew, `which python` would return `/usr/local/bin/python`, and most users would erroneously assume that `/usr/local` is the installation directory. If you peruse through #2173, you'll immediately see why this is not the case. Homebrew actually installs Python in `/usr/local/Cellar/python/2.7.12_2` and symlinks the executable to `/usr/local/bin/python`. `PYTHONHOME` (and presumably most things that need to know where Python is installed) needs to be set to the actual installation directory, not `/usr/local`.
Normally I would say, "sounds like user error, make sure to use the real installation directory in your `packages.yaml`". But I think we can make a special case for Python. That's what we decided in #2173 anyway. If we change our minds, I would be more than happy to simplify things.
To solve this problem, I created a `spec['python'].home` attribute that works the same way as `spec['python'].prefix` but queries Python to figure out where it was actually installed. @tgamblin Is there any way to overwrite `spec['python'].prefix`? I think it's currently immutable.
## Command
In general, Python 2 comes with both `python` and `python2` commands, while Python 3 only comes with a `python3` command. But this is up to the OS developers. For example, `/usr/bin/python` on Gentoo is actually Python 3. Worse yet, if someone is using an externally installed Python, all 3 commands may exist in the same directory! Here's what I'm thinking:
If the spec is for Python 3, try searching for the `python3` command.
If the spec is for Python 2, try searching for the `python2` command.
If neither are found, try searching for the `python` command.
## Libraries
Spack installs Python libraries in `spec['python'].prefix.lib`. Except on openSUSE 13, where it installs to `spec['python'].prefix.lib64` (see #2295 and #2253). On my CentOS 6 machine, the Python libraries are installed in `/usr/lib64`. Both need to work.
The libraries themselves change name depending on OS and Python version. For Python 2.7 on macOS, I'm seeing:
```
lib/libpython2.7.dylib
```
For Python 3.6 on CentOS 6, I'm seeing:
```
lib/libpython3.so
lib/libpython3.6m.so.1.0
lib/libpython3.6m.so -> lib/libpython3.6m.so.1.0
```
Notice the `m` after the version number. Yeah, that's a thing.
## Headers
In Python 2.7, I'm seeing:
```
include/python2.7/pyconfig.h
```
In Python 3.6, I'm seeing:
```
include/python3.6m/pyconfig.h
```
It looks like all Python 3 installations have this `m`. Tested with Python 3.2 and 3.6 on macOS and CentOS 6
Spack has really nice support for libraries (`find_libraries` and `LibraryList`), but nothing for headers. Fixed.
* add package for conduit
* try to fix main conduit docstring
* use join_path instead of pjoin
* address a few requests in pr #2670
change name of 'github-master' to 'master'
change 'docs' variant to 'doc', set default to False
remove explicit +shared variant spec for silo and hdf5 deps
(in the conduit +shared case) cases since they default to True
add reference to static rpath issue
(https://github.com/LLNL/spack/issues/2658)
* address pr #2670 requests
add todos and more info on why variants for deps where selected
use python module install python to enable spack activate
use .format instead of %s
