Explicitly import package utilities in all packages, and corresponding fallout.
This includes:
* rename `spack.package` to `spack.package_base`
* rename `spack.pkgkit` to `spack.package`
* update all packages in builtin, builtin_mock and tutorials to include `from spack.package import *`
* update spack style
* ensure packages include the import
* automatically add the new import and remove any/all imports of `spack` and `spack.pkgkit`
from packages when using `--fix`
* add support for type-checking packages with mypy when SPACK_MYPY_CHECK_PACKAGES
is set in the environment
* fix all type checking errors in packages in spack upstream
* update spack create to include the new imports
* update spack repo to inject the new import, injection persists to allow for a deprecation period
Original message below:
As requested @adamjstewart, update all packages to use pkgkit. I ended up using isort to do this,
so repro is easy:
```console
$ isort -a 'from spack.pkgkit import *' --rm 'spack' ./var/spack/repos/builtin/packages/*/package.py
$ spack style --fix
```
There were several line spacing fixups caused either by space manipulation in isort or by packages
that haven't been touched since we added requirements, but there are no functional changes in here.
* [x] add config to isort to make sure this is maintained going forward
* Switch http to https where latter exists
* Hopefully restore original permissions
* Add URL updates after include the -L curl option
* Manual corrections to select URL format strings
This PR will add a new audit, specifically for spack package homepage urls (and eventually
other kinds I suspect) to see if there is an http address that can be changed to https.
Usage is as follows:
```bash
$ spack audit packages-https <package>
```
And in list view:
```bash
$ spack audit list
generic:
Generic checks relying on global variables
configs:
Sanity checks on compilers.yaml
Sanity checks on packages.yaml
packages:
Sanity checks on specs used in directives
packages-https:
Sanity checks on https checks of package urls, etc.
```
I think it would be unwise to include with packages, because when run for all, since we do requests it takes a long time. I also like the idea of more well scoped checks - likely there will be other addresses for http/https within a package that we eventually check. For now, there are two error cases - one is when an https url is tried but there is some SSL error (or other error that means we cannot update to https):
```bash
$ spack audit packages-https zoltan
PKG-HTTPS-DIRECTIVES: 1 issue found
1. Error with attempting https for "zoltan":
<urlopen error [SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed: Hostname mismatch, certificate is not valid for 'www.cs.sandia.gov'. (_ssl.c:1125)>
```
This is either not fixable, or could be fixed with a change to the url or (better) contacting the site owners to ask about some certificate or similar.
The second case is when there is an http that needs to be https, which is a huge issue now, but hopefully not after this spack PR.
```bash
$ spack audit packages-https xman
Package "xman" uses http but has a valid https endpoint.
```
And then when a package is fixed:
```bash
$ spack audit packages-https zlib
PKG-HTTPS-DIRECTIVES: 0 issues found.
```
And that's mostly it. :)
Signed-off-by: vsoch <vsoch@users.noreply.github.com>
Co-authored-by: vsoch <vsoch@users.noreply.github.com>
* fix remaining flake8 errors
* imports: sort imports everywhere in Spack
We enabled import order checking in #23947, but fixing things manually drives
people crazy. This used `spack style --fix --all` from #24071 to automatically
sort everything in Spack so PR submitters won't have to deal with it.
This should go in after #24071, as it assumes we're using `isort`, not
`flake8-import-order` to order things. `isort` seems to be more flexible and
allows `llnl` mports to be in their own group before `spack` ones, so this
seems like a good switch.
Add `manual_download = True` to packages that need to do manual
downloads but do not have the `manual_download attribute set. This
provides a message when installing these packages rather than a generic
fetch error.
- [x] add `concretize.lp`, `spack.yaml`, etc. to licensed files
- [x] update all licensed files to say 2013-2021 using
`spack license update-copyright-year`
- [x] appease mypy with some additions to package.py that needed
for oneapi.py
* Separate Apple Clang from LLVM Clang
Apple Clang is a compiler of its own. All places
referring to "-apple" suffix have been updated.
* Hack to use a dash in 'apple-clang'
To be able to use autodoc from Sphinx we need
a valid Python name for the module that contains
Apple's Clang code.
* Updated packages to account for the existence of apple-clang
Co-authored-by: Adam J. Stewart <ajstewart426@gmail.com>
* Added unit test for XCode related functions
Co-authored-by: Gregory Becker <becker33@llnl.gov>
Co-authored-by: Adam J. Stewart <ajstewart426@gmail.com>
* amber: Improved package.py and added version 18
- Added amber 18 with ambertools 19
- Added all available patches
- Added +update variant to use the self update
- Added +openmp variant to get openmp optomizations
- Added +x11 variant when possible
- Splitted amber 16 and 18 dependencies
- We now detect the copiler type and compile accordingly
- Added cray variant which is a bit special (untested)
- Improved detection of possible cuda versions
- All compilation optimizations +mpi +openmp +cuda are compatible
- Updated to use setup_build_environment(), setup_run_environment()
* dealii: Added 'threads' variant that controls the TBB dependency (#13931)
* dealii: Added 'threads' variant that controls the DEAL_II_WITH_THREADS cmake option and the dependency on Intel TBB
* Update var/spack/repos/builtin/packages/dealii/package.py
Co-Authored-By: Adam J. Stewart <ajstewart426@gmail.com>
* amber: Improved package.py and added version 18
- Added amber 18 with ambertools 19
- Added all available patches
- Added +update variant to use the self update
- Added +openmp variant to get openmp optomizations
- Added +x11 variant when possible
- Splitted amber 16 and 18 dependencies
- We now detect the copiler type and compile accordingly
- Added cray variant which is a bit special (untested)
- Improved detection of possible cuda versions
- All compilation optimizations +mpi +openmp +cuda are compatible
- Updated to use setup_build_environment(), setup_run_environment()
* amber: Adding missing flex and bison dependencies
* Removed cray variant; flex and bison now build only
* amber: Improved package.py and added version 18
- Added amber 18 with ambertools 19
- Added all available patches
- Added +update variant to use the self update
- Added +openmp variant to get openmp optomizations
- Added +x11 variant when possible
- Splitted amber 16 and 18 dependencies
- We now detect the copiler type and compile accordingly
- Added cray variant which is a bit special (untested)
- Improved detection of possible cuda versions
- All compilation optimizations +mpi +openmp +cuda are compatible
- Updated to use setup_build_environment(), setup_run_environment()
* amber: Adding missing flex and bison dependencies
* Removed cray variant; flex and bison now build only
* dealii: Fixed flake8 issues
* amber: corrected typo
* amber: Removed unused variant python
