Commit Graph

653 Commits

Author SHA1 Message Date
Adam J. Stewart
7ccb9992a6
Procedure to deprecate old versions of software (#20767)
* Procedure to deprecate old versions of software

* Add documentation

* Fix bug in logic

* Update tab completion

* Deprecate legacy packages

* Deprecate old mxnet as well

* More explicit docs
2021-02-09 13:51:18 -05:00
Adam J. Stewart
b597cbe1c0
PythonPackage: fewer phases (#20738) 2021-02-01 12:48:45 -06:00
Mark C. Miller
f766585882
Doc default behavior of install tests (#21309)
Co-authored-by: Tamara Dahlgren <35777542+tldahlgren@users.noreply.github.com>
2021-01-27 09:08:27 -08:00
Tamara Dahlgren
6b13909cc1
docs: Update the CudaPackage (build system) description (#20742)
Co-authored-by: Axel Huebl <axel.huebl@plasma.ninja>
2021-01-21 12:43:21 -08:00
Tamara Dahlgren
25bab31bc8
Added ROCmPackage (build system) documentation (#20743) 2021-01-21 10:46:39 -08:00
Adam J. Stewart
881bdd5b9e
Package Repositories docs: num packages has grown (#20735) 2021-01-11 11:00:14 +01:00
Tamara Dahlgren
31f26daf10
Update the docs footer copyright (#20741) 2021-01-11 08:00:08 +01:00
Vanessasaurus
67ce1939a3
spack python: allow use of IPython (#20329)
This adds a -i option to "spack python" which allows use of the
IPython interpreter; it can be used with "spack python -i ipython".
This assumes it is available in the Python instance used to run
Spack (i.e. that you can "import IPython").
2021-01-05 16:54:47 -08:00
Todd Gamblin
a8ccb8e116 copyrights: update all files with license headers for 2021
- [x] add `concretize.lp`, `spack.yaml`, etc. to licensed files
- [x] update all licensed files to say 2013-2021 using
      `spack license update-copyright-year`
- [x] appease mypy with some additions to package.py that needed
      for oneapi.py
2021-01-02 12:12:00 -08:00
Adam J. Stewart
0d95fd6d52
extends: add type kwarg (#20045)
* extends: add type kwarg

* Flake8 fix
2020-12-30 11:09:58 -06:00
Adam J. Stewart
05f8e08067
PythonPackage: add pypi attribute to infer homepage/url/list_url (#17587) 2020-12-29 09:03:08 +01:00
Tom Scogland
857749a9ba
add mypy to style checks; rename spack flake8 to spack style (#20384)
I lost my mind a bit after getting the completion stuff working and
decided to get Mypy working for spack as well. This adds a 
`.mypy.ini` that checks all of the spack and llnl modules, though
not yet packages, and fixes all of the identified missing types and
type issues for the spack library.

In addition to these changes, this includes:

* rename `spack flake8` to `spack style`

Aliases flake8 to style, and just runs flake8 as before, but with
a warning.  The style command runs both `flake8` and `mypy`,
in sequence. Added --no-<tool> options to turn off one or the
other, they are on by default.  Fixed two issues caught by the tools.

* stub typing module for python2.x

We don't support typing in Spack for python 2.x. To allow 2.x to
support `import typing` and `from typing import ...` without a
try/except dance to support old versions, this adds a stub module
*just* for python 2.x.  Doing it this way means we can only reliably
use all type hints in python3.7+, and mypi.ini has been updated to
reflect that.

* add non-default black check to spack style

This is a first step to requiring black.  It doesn't enforce it by
default, but it will check it if requested.  Currently enforcing the
line length of 79 since that's what flake8 requires, but it's a bit odd
for a black formatted project to be quite that narrow.  All settings are
in the style command since spack has no pyproject.toml and I don't
want to add one until more discussion happens. Also re-format
`style.py` since it no longer passed the black style check
with the new length.

* use style check in github action

Update the style and docs action to use `spack style`, adding in mypy
and black to the action even if it isn't running black right now.
2020-12-22 21:39:10 -08:00
Tamara Dahlgren
b7d9e269ef
docs: add single node concurrent build example (#20416) 2020-12-17 17:23:55 +01:00
Adam J. Stewart
826cd07cf7
PythonPackage: add import module smoke tests (#20023) 2020-12-16 15:15:03 -08:00
Greg Becker
3840c0ac45
docs: fix spack install debug arg order (#20428) 2020-12-16 13:57:08 -08:00
Adam J. Stewart
20752db103
Docs: add more Command Reference links to spack test (#20413) 2020-12-16 12:08:32 -08:00
Tamara Dahlgren
21f30e3074
Bugfix/docs: correct and expand smoke test documentation (#20278) 2020-12-15 08:38:00 -08:00
Adam J. Stewart
3f5f80956e
Command Reference: add link to spack test docs (#20054) 2020-12-08 09:26:03 -08:00
George Hartzell
bb9f5d613c
Typos: add missing closing parens (#20174) 2020-11-30 10:28:07 -06:00
Adam J. Stewart
868dbb24c1
libQGLViewer: add new package (#20164) 2020-11-30 10:25:40 -05:00
Michael Kuhn
20367e472d
cmd: add spack mark command (#16662)
This adds a new `mark` command that can be used to mark packages as either
explicitly or implicitly installed. Apart from fixing the package
database after installing a dependency manually, it can be used to
implement upgrade workflows as outlined in #13385.

The following commands demonstrate how the `mark` and `gc` commands can be
used to only keep the current version of a package installed:
```console
$ spack install pkgA
$ spack install pkgB
$ git pull # Imagine new versions for pkgA and/or pkgB are introduced
$ spack mark -i -a
$ spack install pkgA
$ spack install pkgB
$ spack gc
```

If there is no new version for a package, `install` will simply mark it as
explicitly installed and `gc` will not remove it.

Co-authored-by: Greg Becker <becker33@llnl.gov>
2020-11-18 03:20:56 -08:00
Greg Becker
77b2e578ec
spack test (#15702)
Users can add test() methods to their packages to run smoke tests on
installations with the new `spack test` command (the old `spack test` is
now `spack unit-test`). spack test is environment-aware, so you can
`spack install` an environment and then run `spack test run` to run smoke
tests on all of its packages. Historical test logs can be perused with
`spack test results`. Generic smoke tests for MPI implementations, C,
C++, and Fortran compilers as well as specific smoke tests for 18
packages.

Inside the test method, individual tests can be run separately (and
continue to run best-effort after a test failure) using the `run_test`
method. The `run_test` method encapsulates finding test executables,
running and checking return codes, checking output, and error handling.

This handles the following trickier aspects of testing with direct
support in Spack's package API:

- [x] Caching source or intermediate build files at build time for
      use at test time.
- [x] Test dependencies,
- [x] packages that require a compiler for testing (such as library only
      packages).

See the packaging guide for more details on using Spack testing support.
Included is support for package.py files for virtual packages. This does
not change the Spack interface, but is a major change in internals.

Co-authored-by: Tamara Dahlgren <dahlgren1@llnl.gov>
Co-authored-by: wspear <wjspear@gmail.com>
Co-authored-by: Adam J. Stewart <ajstewart426@gmail.com>
2020-11-18 02:39:02 -08:00
Peter Scheibel
c9ad2affcc
Documentation: spack load/environments prefix inspections (#19961)
As of #18260, `spack load` and `spack env activate` now use
`prefix_inspections` from the modules configuration to decide
how to modify environment variables.

This updates the modules configuration documentation to describe
how to update environment variables with the `prefix_inspections`
section. This also updates the `spack load` and environments
documentation to refer to the new `prefix_inspections` documentation.
2020-11-17 15:24:00 -08:00
Massimiliano Culpo
5f636fc317
spack containerize: allow users to customize the base image (#15028)
This PR reworks a few attributes in the container subsection of
spack.yaml to permit the injection of custom base images when
generating containers with Spack. In more detail, users can still
specify the base operating system and Spack version they want to use:

  spack:
    container:
      images:
        os: ubuntu:18.04
        spack: develop

in which case the generated recipe will use one of the Spack images
built on Docker Hub for the build stage and the base OS image in the
final stage. Alternatively, they can specify explicitly the two
base images:

  spack:
    container:
      images:
        build: spack/ubuntu-bionic:latest
        final: ubuntu:18.04

and it will be up to them to ensure their consistency.

Additional changes:

* This commit adds documentation on the two approaches.
* Users can now specify OS packages to install (e.g. with apt or yum)
  prior to the build (previously this was only available for the
  finalized image).
* Handles to avoid an update of the available system packages have been
  added to the configuration to facilitate the generation of recipes
  permitting deterministic builds.
2020-11-17 11:25:13 -08:00
Tamara Dahlgren
6fa6af1070
Support parallel environment builds (#18131)
As of #13100, Spack installs the dependencies of a _single_ spec in parallel.
Environments, when installed, can only get parallelism from each individual
spec, as they're installed in order.  This PR makes entire environments build
in parallel by extending Spack's package installer to accept multiple root
specs.  The install command and Environment class have been updated to use
the new parallel install method.

The specs and kwargs for each *uninstalled* package (when not force-replacing
installations) of an environment are collected, passed to the `PackageInstaller`,
and processed using a single build queue.

This introduces a `BuildRequest` class to track install arguments, and it
significantly cleans up the code used to track package ids during installation.
Package ids in the build queue are now just DAG hashes as you would expect,

Other tasks:

- [x] Finish updating the unit tests based on `PackageInstaller`'s use of
      `BuildRequest` and the associated changes
- [x] Change `environment.py`'s `install_all` to use the `PackageInstaller` directly
- [x] Change the `install` command to leverage the new installation process for multiple specs
- [x] Change install output messages for external packages, e.g.:
       `[+] /usr` -> `[+] /usr (external bzip2-1.0.8-<dag-hash>`
- [x] Fix incomplete environment install's view setup/update and not confirming all 
       packages are installed (?)
- [x] Ensure externally installed package dependencies are properly accounted for in 
       remaining build tasks
- [x] Add tests for coverage (if insufficient and can identity the appropriate, uncovered non-comment lines)
- [x] Add documentation
- [x] Resolve multi-compiler environment install issues
- [x] Fix issue with environment installation reporting (restore CDash/JUnit reports)
2020-11-17 02:41:07 -08:00
Adam J. Stewart
02281a891d
MavenPackage: allow additional build args (#19676) 2020-11-12 12:57:49 -08:00
Emir İşman
98e709da3b
[docs] getting_started.rst: fix typo (#19815) 2020-11-09 16:31:53 -06:00
Greg Becker
8b96e10ecc
Remove hardcoded version numbers from container logic (#19716)
Previously, we hardcoded a list of Spack versions which could be used by the containerize command.

This PR removes that list. It's a maintenance burden when cutting a release, and prevents older versions of Spack from creating containers to be used by newer versions.
2020-11-05 18:59:44 +01:00
Shahzeb Siddiqui
75e73d7fcc
documentation: fix formatting of code-block section (#19693) 2020-11-03 12:15:46 +01:00
Massimiliano Culpo
c4aa5cb5bc
Update documentation on containers (#19631)
fixes #15183

- Moved the container related content from
  workflows.rst into containers.rst
- Deleted the docker_for_developers.rst file,
  since it describes an outdated procedure

Co-authored-by: Axel Huebl <a.huebl@hzdr.de>
Co-authored-by: Omar Padron <omar.padron@kitware.com>
2020-10-30 21:17:15 +01:00
Todd Gamblin
2893c23e7c
docs: update docs on shell support and using packages (#19486)
Shell integration no longer requires setting `SPACK_ROOT`, so we can
simplify the documentation on it. The docs on shell support and using
packages are getting a bit old, and information on `spack load` (which
seems to be everyone's most common way of using packages) is hard to
find.

This PR simplifies the shell documentation to remove SPACK_ROOT, and also
moves some sections around for clearer organization.

- [x] make docs on sourcing setup scripts clearer and simpler

- [x] introduce `spack load` early in the basic usage guide instead of
      burying it in the module docs

- [x] clean up module docs so that spack module tcl loads comes later

- [x] be clear about the different ways to use packages so that the users
      can find the docs better.

Co-authored-by: Massimiliano Culpo <massimiliano.culpo@gmail.com>
2020-10-23 22:16:01 -07:00
Wouter Deconinck
9471e9cb03
[docs] Pkg list: current version, not latest release (#18213)
The package list at https://spack.readthedocs.io/en/latest/package_list.html claims "it is automatically generated based on the packages in the latest Spack release" but it is actually based on the develop branch. This leads to confusion when users find that e.g. herwigpp is included in the list, but it cannot be found when they install the latest release. That latest release has a package list at https://spack.readthedocs.io/en/stable/package_list.html which does indeed not include herwigpp.

Changing the language from "the latest Spack release" to "this Spack version" might make that clearer. Maybe.
2020-10-18 23:11:20 -05:00
Adam J. Stewart
372ac4a073
Add testing for Python 3.9 (#19261) 2020-10-11 21:16:00 -07:00
谭九鼎
5e9f4dc982
Use https for links (#19244) 2020-10-09 11:24:09 -05:00
Scott Wittenburg
031490f6aa Remove :<name> interpolation, add SPACK_VERSION variables
Also fix issues with documentation to reflect changes
2020-09-14 10:37:42 -06:00
Scott Wittenburg
bf90cdd6c7 Document pipeline keys which can be global but overridden
Update pipelines documentation to describe how 'tags', 'variables',
'image', 'before_script', 'script', and 'after_script' can be
supplied at the top level, to be used by any of the runner mappings,
and also overridden by any of the runner mappings.

Also show an example of capturing the custom spack SHA at pipeline
generation time, so all jobs are sure to run with the same version
of spack, as a means to illustrate the $env:VARIABLE_NAME syntax.
2020-09-14 10:37:42 -06:00
Scott Wittenburg
e686f1500e Update pipeline documentation to describe user-provided scripts 2020-09-14 10:37:42 -06:00
Adam J. Stewart
7d9f2bf4ed
depends_on cannot handle ^ sigil (#18220)
* depends_on cannot handle ^ sigil

* cardioid+mfem+cuda requires hypre+cuda

* Document this limitation

* Move warning message to Known Issues docs

* Better handling of parmetis dep
2020-09-03 17:31:00 -05:00
Adam J. Stewart
7728b0737b
Add new MavenPackage build system base class (#18185)
* Add new MavenPackage build system base class

* Fix flake8 and doc tests

* More specific regex

* Java 8 required for these packages
2020-09-03 17:30:39 -05:00
Adam J. Stewart
443407cda5
Add new RubyPackage build system base class (#18199)
* Add new RubyPackage build system base class

* Ruby: add spack external find support

* Add build tests for RubyPackage
2020-09-02 16:26:36 -07:00
Adam J. Stewart
e58db067c3
PythonPackage: update documentation (#18181) 2020-09-02 15:05:10 -07:00
Adam J. Stewart
1992fdf712
Document test dependency type (#18365) 2020-09-02 13:46:52 -07:00
Adam J. Stewart
17f7b23783
Deprecate spack setup (#18240) 2020-09-01 18:07:48 -07:00
Greg Becker
d10dff1b89
docs: add main version to docs for develop-like versions (#18328) 2020-08-27 12:35:11 -07:00
Richarda Butler
416afa0059
docs: fix bugs in contribution, getting started guides (#18216)
Co-authored-by: Greg Becker <becker33@llnl.gov>
Co-authored-by: Adam J. Stewart <ajstewart426@gmail.com>
2020-08-27 08:23:26 +02:00
Massimiliano Culpo
96ac5add9d
release procedure: add step to activate the documentation on readthedocs (#18288) 2020-08-26 12:47:47 -07:00
George Hartzell
b53b9fd17e
Fix redundant reset of terminal in prompt example (#17698)
I know that it's just an example, but I was trying to figure out what was going on and it wasn't making sense....

`tput sgr0` resets the terminal state (http://linuxcommand.org/lc3_adv_tput.php) and I can't see any reason to do it twice.  Deleting the second occurrence doesn't seem to break the fancy prompt effect.
2020-08-22 13:27:39 -05:00
Richarda Butler
d599e4d9d4
Added the file path for running a specific test (#18214) 2020-08-21 17:42:24 -05:00
Michael Kuhn
11b1ce84cf
docs: List tar and some compressors in prerequisites (#18169)
Fixes #18152
2020-08-20 17:13:47 -05:00
Adam J. Stewart
f0c0cd5c3f
Fix typo in spack external debug msg (#17982) 2020-08-11 09:55:44 +02:00
Massimiliano Culpo
1dba0ce81b Removed references to BlueGene/Q in docs and comments 2020-08-10 17:09:09 -07:00
Massimiliano Culpo
c0d490ffbe Simplify the detection protocol for packages
Packages can implement “detect_version” to support detection
of external instances of a package. This is generally easier
than implementing “determine_spec_details”. The API for
determine_version is similar: for example you can return
“None” to indicate that an executable is not an instance
of a package.

Users may implement a “determine_variants” method for a package.
When doing external detection, executables are grouped by version
and each group results in a single invocation of “determine_variants”
for the associated spec. The method returns a string specifying
the variants for the package. The method may additionally return
a dictionary representing extra attributes for the package.

These will be stored in the spec yaml and can be retrieved
from self.spec.extra_attributes

The Spack GCC package has been updated with an implementation
of “determine_variants” which adds the following extra
attributes to the package: c, cxx, fortran
2020-08-10 11:59:05 -07:00
Massimiliano Culpo
193e8333fa Update packages.yaml format and support configuration updates
The YAML config for paths and modules of external packages has
changed: the new format allows a single spec to load multiple
modules. Spack will automatically convert from the old format
when reading the configs (the updates do not add new essential
properties, so this change in Spack is backwards-compatible).

With this update, Spack cannot modify existing configs/environments
without updating them (e.g. “spack config add” will fail if the
configuration is in a format that predates this PR). The user is
prompted to do this explicitly and commands are provided. All
config scopes can be updated at once. Each environment must be
updated one at a time.
2020-08-10 11:59:05 -07:00
Harmen Stoppels
827ca72c26
Fix docs about containers on cray (#17431)
* For detecting Cray: CRAYPE_VERSION is not used, but MODULEPATH

* Fix typo and write Cray with a capital
2020-08-03 16:16:18 -07:00
Massimiliano Culpo
9dbad500bc
Move Python 2.6 unit tests to Github Actions (#17279)
* Run Python2.6 unit tests on Github Actions
* Skip url tests on Python 2.6 to reduce waiting times
* Skip foreground background tests on Python 2.6 to reduce waiting times
* Removed references to Travis in the documentation
* Deleted install_patchelf.sh (can be installed from repo on CentOS 6)
2020-07-31 15:01:12 -07:00
Chuck Atkins
547c71ad78
Revert "Add libglvnd packages/Add EGL support (#14572)" (#17682)
This reverts commit 573489db71.
2020-07-23 17:41:48 -04:00
Omar Padron
573489db71
Add libglvnd packages/Add EGL support (#14572)
* add new package: "libglvnd-frontend"

* add +glvnd variant to opengl package

* add +glvnd variant to mesa package

* add +egl variant to paraview package

* add libglvnd-frontend entries to default packages config

* fix style

* add default providers for glvnd virtuals

add default providers for glvnd-gl, glvnd-glx, and glvnd-egl

* WIP: rough start to external OpenGL documentation

* rename libglvnd-frontend package and backend virtual dependencies

* update documentation

* fix ligvnd-be-* typos

* fix libglvnd-fe package class name

* fix doc parse error
2020-07-13 11:32:36 -04:00
Greg Becker
406596af70
update docs on point releases (#17463) 2020-07-11 14:35:25 -07:00
Simon Byrne
f85da868ac
Improve Travis sample in the docs (#17420)
- printf is better than echo for multiline strings
- ** should be &&
- use line continuation
- Use multiline block
2020-07-08 07:25:37 +02:00
Todd Gamblin
b0506a722e
releases: document releases/latest tag (#17402)
We got rid of `master` after #17377, but users still want a way to get
the latest stable release without knowing its number.

We've added a `releases/latest` tag to replace what was once `master`.

Co-authored-by: Massimiliano Culpo <massimiliano.culpo@gmail.com>
2020-07-07 11:44:15 -07:00
Todd Gamblin
9ec9327f5a docs: document releases and branches in Spack
- [x] Remove references to `master` branch
- [x] Document how release branches are structured
- [x] Document how to make a major release
- [x] Document how to make a point release
- [x] Document how to do work in our release projects
2020-07-06 11:39:19 -07:00
Shahzeb Siddiqui
04f3000646
Pipelines doc: fixed two broken links (#17355) 2020-07-03 12:29:45 +02:00
Glenn Johnson
60283775b3
Documentation update for container example (#17321)
This updates the documentation to reflect #17316.
2020-07-01 08:40:36 +02:00
George Hartzell
e6bb8360d0
Fix indentation in containerize example (#17228)
[george.hartzell@172-16-193-97 spack-explore-docker]$ spack containerize
Running `spack containerize` with the example `spack.yaml` file fails
with an error that ends like so:

```
[...]
  File "/local_scratch/hartzell/tmp/spack-explore-docker/lib/spack/external/ruamel/yaml/scanner.py", line 165, in need_more_tokens
    self.stale_possible_simple_keys()
  File "/local_scratch/hartzell/tmp/spack-explore-docker/lib/spack/external/ruamel/yaml/scanner.py", line 309, in stale_possible_simple_keys
    "could not find expected ':'", self.get_mark())
ruamel.yaml.scanner.ScannerError: while scanning a simple key
  in "/local_scratch/hartzell/tmp/spack-explore-docker/spack.yaml", line 26, column 1
could not find expected ':'
  in "/local_scratch/hartzell/tmp/spack-explore-docker/spack.yaml", line 28, column 5
```

Indenting the block string fixes the problem for me.

CentOS 7,

```
$ spack --version
0.14.2-1529-ec58f28c2
```
2020-06-29 08:58:51 +02:00
Greg Becker
56b4abbe38
env: no automatic activation (#17258)
* env: no automatic activation

* Ensure ci rebuild jobs activate the environment (no longer automagic)

Co-authored-by: Scott Wittenburg <scott.wittenburg@kitware.com>
2020-06-26 17:20:15 -05:00
Greg Becker
f936e3a1db
environment spec lists: improve ability to query architecture in when clauses (#17056) 2020-06-25 19:13:26 +02:00
Massimiliano Culpo
14599f09be
Separate Apple Clang from LLVM Clang (#17110)
* Separate Apple Clang from LLVM Clang

Apple Clang is a compiler of its own. All places
referring to "-apple" suffix have been updated.

* Hack to use a dash in 'apple-clang'

To be able to use autodoc from Sphinx we need
a valid Python name for the module that contains
Apple's Clang code.

* Updated packages to account for the existence of apple-clang

Co-authored-by: Adam J. Stewart <ajstewart426@gmail.com>

* Added unit test for XCode related functions

Co-authored-by: Gregory Becker <becker33@llnl.gov>
Co-authored-by: Adam J. Stewart <ajstewart426@gmail.com>
2020-06-25 11:18:48 -05:00
Tamara Dahlgren
48d3e8d350
features: Add install failure tracking removal through spack clean (#15314)
* Add ability to force removal of install failure tracking data through spack clean

* Add clean failures option to packaging guide
2020-06-24 20:28:53 -05:00
Tamara Dahlgren
96932d65a8 Added support for --fail-fast install option to terminate on first failure 2020-06-23 10:22:41 -07:00
Adam J. Stewart
1f85d6eceb
Docs: fix rst link to spack ci (#17147) 2020-06-18 09:44:03 +02:00
Axel Huebl
896a14af82
Doc: E4S Build Cache Link (#16986)
Link to the E4S build cache in the according section of the docs.
2020-06-16 15:53:34 -07:00
Jonathon Anderson
feda2a3073
openmpi: add opa-psm2 dependency (#16873)
Also document with_or_without and enable_or_disable, (which are used
to configure the opa-psm2 dependency).
2020-06-01 15:33:24 -07:00
Greg Becker
32a9adcf60
modules: use projections format for naming schemes (#16629)
* update tcl naming_scheme to use projections

* add projections to lmod modules
2020-05-15 11:12:52 -07:00
Scott Wittenburg
e0572a7d96 Pipelines: Support DAG scheduling and dynamic child pipelines
This change also adds a code path through the spack ci pipelines
infrastructure which supports PR testing on the Spack repository.
Gitlab pipelines run as a result of a PR (either creation or pushing
to a PR branch) will only verify that the packages in the environment
build without error.  When the PR branch is merged to develop,
another pipeline will run which results in the generated binaries
getting pushed to the binary mirror.
2020-05-14 21:11:07 -07:00
Greg Becker
445cae5c81
Feature: Allow lmod configuration to set core specs (#16517)
Packages built with lmod core_compiler are placed in `Core`.

Other packages may belong in `Core`. For example, python may be built with a proprietary compiler for performance, but belong on the `Core` directory.

With this PR, lmod config can include a `core_specs` list. Any package that satisfies a spec in that list is placed in `Core`, regardless of its compiler or dependencies.
2020-05-14 11:27:37 -07:00
Peter Scheibel
701fc1fdb1
Update docs on "spack external find" (#16482)
This improves the documentation for `spack external find` in several ways:

* Provide a code example of implementing `determine_spec_details` for a package
* Explain how to define executables to look for (and also e.g. that they are treated as regular expressions and so can pull in unexpected files).
* Add the "why" for a couple of constraints (i.e. explain that this logic only works for build/run deps because it examines `PATH` for executables)
* Spread the docs between build customization and packaging sections
* Add cross-references
* Add a label so that `spack external find` is linked from the command reference.
2020-05-12 17:08:08 -07:00
Massimiliano Culpo
43c9ad3421
Remove 'spack bootstrap' and associated docs (#15179)
fixes #15145

This commit removes the outdated `spack bootstrap`
command and any reference to it in the documentation
and unit tests.
2020-05-11 10:55:18 -07:00
Peter Scheibel
b030a81a5f
Automatically find externals (#15158)
Add a `spack external find` command that tries to populate
`packages.yaml` with external packages from the user's `$PATH`. This
focuses on finding build dependencies. Currently, support has only been
added for `cmake`.

For a package to be discoverable with `spack external find`, it must define:
  * an `executables` class attribute containing a list of
    regular expressions that match executable names.
  * a `determine_spec_details(prefix, specs_in_prefix)` method

Spack will call `determine_spec_details()` once for each prefix where
executables are found, passing in the path to the prefix and the path to
all found executables. The package is responsible for invoking the
executables and figuring out what type of installation(s) are in the
prefix, and returning one or more specs (each with version, variants or
whatever else the user decides to include in the spec).

The found specs and prefixes will be added to the user's `packages.yaml`
file. Providing the `--not-buildable` option will mark all generated
entries in `packages.yaml` as `buildable: False`
2020-05-05 17:37:34 -07:00
Massimiliano Culpo
aa9c98b6a2
Make usage example consistent with current MV variants (#16468)
Since #9481 Python's None is not permitted as a value for
MV variants. The string 'none' is used instead.

Add the same fix for the amgx and lammps packages
2020-05-05 20:22:32 +02:00
Greg Becker
8e2f5ba861
environments: allow sigils to apply to entire reference (#15245)
* environments: allow sigils to apply to entire reference
2020-05-04 15:11:10 -07:00
Nathan
b6b44948b8
prefix gets removed when adding additional configure options (#16335)
* prefix gets removed when adding additional configure options

* Update waf documentation to reflect the new change
2020-04-29 10:52:23 -05:00
Axel Huebl
7670ae468f
Docs: Limit Microarch on Travis CI (#16200)
Update the travis CI docs: limiting the microarch is essential
for caching.
2020-04-28 11:54:02 -05:00
George Hartzell
63ff121980
spack environments documentation: -E -> -e (#16242)
To specify an environment for a comment, the user can specify
"spack -e <env>". The documentation incorrectly specified "-E" (which
is actually used to ignore any implicit use of environments).
2020-04-22 13:04:17 -07:00
Axel Huebl
ca4c0b97ec
Doc: Conflict Msg Pkg Guide (#16201)
Document the nice `msg=` argument in conflicts. Super useful
for less generic conflict guidance.
2020-04-21 16:22:30 -05:00
Todd Gamblin
de13ea2284
autotools: remove FORCE_UNSAFE_CONFIGURE from docs, commands, code (#16145)
Since #16132, we've consolidated the setting of FORCE_UNSAFE_CONFIGURE to
`autotools.py`, so we don't need to use it in packages like `coreutils`,
in our commands, or in our container recipes.

- [x] Remove FORCE_UNSAFE_CONFIGURE from packages
- [x] Remove FORCE_UNSAFE_CONFIGURE from container recipes
- [x] Remove FORCE_UNSAFE_CONFIGURE from `spack ci` command
2020-04-20 08:32:03 -07:00
George Hartzell
c62f18304d
Typo: vew -> view (#16155) 2020-04-19 20:13:44 -05:00
Massimiliano Culpo
a11de1d29d
Package extensions: fixed a link in docs (#16040)
* Moved link to the right place in the docs

* Fixed a few minor issues in extensions docs

Fixed a typo, added a subsubsection for better
navigation, reworded "modules in Python" as
"Python packages"
2020-04-14 12:54:53 -05:00
Scott Wittenburg
de95a9f5f3 pipelines: Document dynamic generation limitations 2020-04-07 15:20:01 -06:00
Adam J. Stewart
fab1f68399
Docs: fix Spack Environments command (#15891) 2020-04-06 11:54:22 -05:00
G-Ragghianti
bae4f91bfe
Add option "--first" for "spack load" (#15622)
* Implemented --first option for "spack load"

* added test for "spack load --first"

Co-authored-by: gragghia <gragghia@localhost.localdomain>
2020-04-03 13:33:20 -07:00
Greg Sjaardema
c3e8825f8e
Spelling fixes (#15805)
* Spelling fixes

* spelling fixes

* spelling fixes

* spelling fixes

* spelling fix

* spelling fix

* spelling fix

* spelling fixes

* spelling fix
2020-04-01 12:02:26 -05:00
Greg Becker
336191c251
packages.yaml: allow virtuals to specify buildable: false (#14934)
Currently, to force Spack to use an external MPI, you have to specify `buildable: False`
for every MPI provider in Spack in your packages.yaml file. This is both tedious and
fragile, as new MPI providers can be added and break your workflow when you do a
git pull.

This PR allows you to specify an entire virtual dependency as non-buildable, and
specify particular implementations to be built:

```
packages:
all:
    providers:
        mpi: [mpich]
mpi:
    buildable: false
    paths:
        mpich@3.2 %gcc@7.3.0: /usr/packages/mpich-3.2-gcc-7.3.0
```
will force all Spack builds to use the specified `mpich` install.
2020-03-31 16:09:08 -07:00
Massimiliano Culpo
7357b7023e
Update link for codecov's browser extensions (#15779) 2020-03-31 09:49:25 -05:00
Sinan
056fc99676
Use python extend_path as pyqt sip fix (#15297)
* try extend path to solve PyQt5.sip not found issue

* disable private sip installation in sippackage class

* undo manual PyQt5 dir creation in py-sip site-packages dir

* fix typo

* fix typo

* also apply fix to PyQt4

* tidy up

* flake8 and tidy up

* tidy and undo hardcoding of python_include_dir

* replace hardcoded python inc dir

* fix minor issues

* rethink include dir variable name

* improve style

* add new versions

* implement new sip setup to qsci installation

* set sip-incdir correctly for the new setup

* setup extend_path thing before qsci python bindings

* take care of conflict

* flake8

* also extend for PyQt4

* improve style

* improve style

* SipPackage build sys should depend on py-sip

* consolidate extend_path fixes into SipPackage

* fix typo

* fix bugs

* flake8

* revert sip doc to pre-resource setup

* import os module

* flake8

Co-authored-by: Sinan81 <sbulut@3vgeomatics.com>
2020-03-18 13:19:27 -05:00
Seth R. Johnson
a8706cc89b
CMakePackage: convert variants to CMake arguments (#14376)
Add a 'define_from_variant` helper function to CMake-based Spack
packages to convert package variants into CMake arguments. For
example:

  args.append('-DFOO=%s' % ('ON' if '+foo' in self.spec else 'OFF'))

can be replaced with:

  args.append(self.define_from_variant('foo'))

The following conversions are handled automatically:

* Flag variants will be converted to CMake booleans
* Multivalued variants will be converted to semicolon-separated strings
* Other variant values are converted to CMake string arguments

This also adds a 'define' helper method to convert any variable to
a CMake argument. It has the same conversion rules as
'define_from_variant' (but operates directly on values rather than
requiring the user to supply the name of a package variant).
2020-03-16 11:41:19 -07:00
TZ
d6d095a868
Docs: "spack env" is in the meantime called "spack build-env" (#15233) 2020-02-27 09:13:09 +01:00
Glenn Johnson
a568db00b8
A few edits for the Basic Usage Doc page (#15215)
This PR corrects a few minor things and adds a note about colorized
output.
2020-02-26 10:03:03 +01:00
Michael Kuhn
45306206c7
docs: Fix tiny typo (#15181) 2020-02-24 10:58:21 -06:00
Adam J. Stewart
7a60a04402
Docs: configure_args -> cmake_args (#15102) 2020-02-19 22:48:56 -06:00
Paul
e65aa7569e
docs: fix typo in arch in build settings examples (#14998) 2020-02-14 19:06:10 +01:00