Commit Graph

610 Commits

Author SHA1 Message Date
Massimiliano Culpo
c4521535e7 Multi-valued variants: better support for combinations (#9481)
This enforces conventions that allow for correct handling of
multi-valued variants where specifying no value is an option,
and adds convenience functionality for specifying multi-valued
variants with conflicting sets of values. This also adds a notion
of "feature values" for variants, which are those that are understood
by the build system (e.g. those that would appear as configure
options). In more detail:

* Add documentation on variants to the packaging guide
* Forbid usage of '' or None as a possible variant value, in
  particular as a default. To indicate choosing no value, the user
  must explicitly define an option like 'none'. Without this,
  multi-valued variants with default set to None were not parsable
  from the command line (Fixes #6314)
* Add "disjoint_sets" function to support the declaration of
  multi-valued variants with conflicting sets of options. For example
  a variant "foo" with possible values "a", "b", and "c" where "c"
  is exclusive of the other values ("foo=a,b" and "foo=c" are
  valid but "foo=a,c" is not).
* Add "any_combination_of" function to support the declaration of
  multi-valued variants where it is valid to choose none of the
  values. This automatically defines "none" as an option (exclusive
  with all other choices); this value does not appear when iterating
  over the variant's values, for example in "with_or_without" (which
  constructs autotools option strings from variant values).
* The "disjoint_sets" and "any_combination_of" methods return an
  object which tracks the possible values. It is also possible to
  indicate that some of these values do not correspond to options
  understood by the package's build system, such that methods like
  "with_or_without" will not define options for those values (this
  occurs automatically for "none")
* Add documentation for usage of new functions for specifying
  multi-valued variants
2019-01-04 19:02:34 -08:00
Todd Gamblin
6f50cd52ed copyright: update license headers for 2013-2019 copyright. 2019-01-01 00:44:28 -08:00
Todd Gamblin
d763e92141 commands: add spack resource command to inspect downloadable files
- currently just looks at patches
  - allows you to find out which package applied a patch to a spec

- intended to work with tarballs and resources in the future.

- add tab completion for `spack resource` and subcommands
2018-12-30 00:19:08 -08:00
Elsa Gonsiorowski, PhD
74a5d92f07 Remove /nfs/tmp2 from default build_stage locations (#10170)
* Remove /nfs/tmp2 from default configuration
* /nfs/tmp2 is going away from LC... and doesn’t exist for the rest of the world.
* update documentation to remove /nfs/tmp2 as well
2018-12-21 02:03:54 -08:00
Adam J. Stewart
c1a99bf8ec Add additional info to MesonPackage docs (#10133)
* Add additional info to MesonPackage docs

* No Pygments lexer for Meson code, default to none
2018-12-18 21:51:42 -06:00
Adam J. Stewart
b771b4015a Fix bugs in Modules Tutorial (#9821)
Found a couple bugs while trying to copy and paste the exact
tutorial content.
2018-11-13 09:31:09 -06:00
Adam J. Stewart
7bca5b9c61 Fix typos in Build Systems Tutorial (#9822) 2018-11-12 18:32:13 -06:00
Todd Gamblin
2efaf1b6f7 docs: update tutorial slide link for SC18 2018-11-12 09:44:47 -06:00
Todd Gamblin
f92349ff8c docs: add environments tutorial
- tutorial goes through three sections:
  - installing and uninstalling environments
  - dealing with many specs
  - spack.yaml and spack.lock and workflows
2018-11-12 05:57:08 -06:00
Adam J. Stewart
a1676ab012 Updates to Configuration Tutorial for SC18 (#9811)
* Updates to Configuration Tutorial for SC18

* Suggested rewording
2018-11-11 22:50:59 -08:00
Adam J. Stewart
71561f9d2b Consistent capitalization of Build Caches docs for SC18 (#9810) 2018-11-11 21:13:01 -08:00
Adam J. Stewart
49c66db2fa Minor changes to Build Settings docs (#9808) 2018-11-11 21:12:37 -08:00
Adam J. Stewart
c227e1f02e Minor changes to Basic Settings docs for SC18 (#9809)
Grammar/spelling issues
2018-11-11 23:10:05 -06:00
Greg Becker
dfd0e96508 Update basic usage and modules tutorial (#9807)
* "spack install" now uses cache by default, update examples accordingly
* Replace some example packages with others
* Packing tutorial reference to "spack env" replaced with "spack build-env"
* Command line prompts in examples are shortened
* Example output (including paths) are updated to be more relevant to training environment
2018-11-11 22:58:50 -06:00
Peter Scheibel
2a5d5fda26 Advanced packaging tutorial: reorganize for binary caches (#9804)
Update all examples that need an MPI provider to build with MPICH; reorganize so that fixing MPICH (as part of environment section) comes first in the tutorial (most examples in the tutorial use an MPI provider).
2018-11-11 21:31:51 -06:00
Mario Melara
7d98c73e40 Update buildsystem tut (#9795)
* Update Makefile to use property methods ("build_targets"/"install_targets")
  to demonstrate their usage
* Fix highlighting
* Change cbench example to ESMF:
  CBench package file was changed and no longer uses the example shown in
  the old docs
2018-11-11 14:21:54 -06:00
Adam J. Stewart
d366e642e4 Updates to Configuration Files docs for SC18 (#9801)
Scopes added with -C are now referred to as "custom scopes"
rather than "command line scopes". "command line scope" now refers
to specific config options that are set on the command line (like
"--insecure")
2018-11-11 10:57:46 -08:00
Todd Gamblin
0e60fcccfb utils: merge spack.environment into spack.util.environment
- `spack.util.environment` is the new home for routines that modify
  environment variables.

- This is to make room for `spack.environment` to contain new routines
  for dealing with spack environments
2018-11-09 00:31:24 -08:00
Peter Scheibel
7d97e8b067 New repo for advanced packaging tutorial (#9711)
* modified tutorial packages

* update hint in hdf5 tutorial file (typo for suggested argument)

* add repo.yaml to tutorial repository

* update tutorial docs to refer user to tutorial package repository

* flake edits

* recommend site scope vs. defaults

* you don't specify the repo's name when adding a repo, just the path
2018-11-06 19:57:32 -08:00
Omar Padron
aa1c814c75 docker: unite Dockerfiles; auto-deploy images to DockerHub (#9329)
* Unite Dockerfiles - add build/run/push scripts
* update docker documentation
* update .travis.yml
* switch to using a preprocessor on Dockerfiles
* skip building docker images on pull requests
* update files with copyright info
* tweak when travis builds for docker files are done
2018-10-26 10:15:05 -07:00
Todd Gamblin
eea786f4e8 relicense: replace LGPL headers with Apache-2.0/MIT SPDX headers
- remove the old LGPL license headers from all files in Spack
- add SPDX headers to all files
  - core and most packages are (Apache-2.0 OR MIT)
  - a very small number of remaining packages are LGPL-2.1-only
2018-10-17 14:42:06 -07:00
Mark W. Krentel
e5e8d89ad9 docs: add note about compilers with modules (#9520)
compilers.yaml can track a module that is needed for a compiler, but
Spack does not fill this in automatically. This adds a note to the
documentation informing the user how to do this.
2018-10-16 15:34:22 -07:00
Greg Becker
d1a5113cfe permissions: add permission configuration to packages.yaml (#8773)
Spack can now be configured to assign permissions to the files installed by a package.

In the `packages.yaml` file under `permissions`, the attributes `read`, `write`, and `group` control the package permissions. These attributes can be set per-package, or for all packages under `all`. If permissions are set under `all` and for a specific package, the package-specific settings take precedence.  The `read` and `write` attributes take one of `user`, `group`, and `world`.

   packages:
    all:
      permissions:
        write: group
        group: spack
    my_app:
      permissions:
        read: group
        group: my_team
2018-10-11 14:29:07 -07:00
Greg Becker
ca0d9ae7f0 Make builtin flag handlers available in package scope (#8668)
* Push default flag handlers into module scope

* Preserve backwards compatibility of builtin flag handler names

Ensure Spack continues to work for packages using the `Package.env_flags` idiom and equivalent.

* update docs and tests to match

* Update packages to match new syntax
2018-10-09 15:18:31 -07:00
Greg Becker
97d0dd2390 Update Spack on Cray docs for singularity platform=linux workaround (#9398) 2018-10-02 14:58:40 -07:00
Sergey Kosukhin
f9617b2ad8 Extended set of environment modification commands. (#8996) 2018-09-05 10:56:45 -07:00
Michael Sternberg
a86f22d755 Intel prefixes (#7469)
Consolidate prefix calculation logic for intel packages into the
IntelPackage class.

Add documentation on installing Intel packages with Spack an
(alternatively) adding them as external packages in Spack.
2018-08-29 21:09:34 -07:00
Todd Gamblin
82820efe37 Remove Python 3.3 from testing.
- Support for Python 3.3 isn't really needed, as nothing uses it as the
  default system Python, and nearly everyone will have a newer Python 3
  version installed.
2018-08-24 16:22:37 -07:00
Tom Merrick
b1182741d4 Branch with the meson build-system (#8467)
* Branch with the meson build-system

* Fix build_environment for dual loads and add create code

* Add documentation

* Fixed option list

* Update build_system_guess for meson

* Fixed documentation errors

* Added meson to build and configure and updated documentation

* fix typos
2018-08-10 12:52:09 -05:00
Adam J. Stewart
5f10d9239b Remove default variant bug from list of known issues (#8900) 2018-08-06 16:56:45 -04:00
Holly
38062a8abc docs: minor text changes (#8858) 2018-08-01 13:59:51 -07:00
Stephen Herbein
cdbc545793 docs: update spack view argument synatx (#8839)
The `--dependencies` and `--exclude` flags are now arguments to the `spack view`
comand rather than the `spack view symlink` command.
2018-07-31 10:41:41 -07:00
Massimiliano Culpo
1fd9574353 Fixed a typo in the docs (the docker image to pull was wrong) (#8814) 2018-07-27 09:56:01 -05:00
Adam J. Stewart
7902f3eb49 Add docs on new top-level VCS attributes 2018-07-25 23:10:10 -07:00
Massimiliano Culpo
35cef16974 Grouped all the module commands under spack module
As requested in the review all the commands meant to manage module
files have been grouped under the `spack module` command.

Unit tests have been refactored to match the new command structure.
2018-07-24 11:27:17 -07:00
Massimiliano Culpo
7cbe1aedb7 module file tutorial: rebuilt docker image and updated the script 2018-07-24 11:27:17 -07:00
Massimiliano Culpo
7a49ba56b6 Added blacklisting of implicit modules in docs + regression tests
fixes #4400

The feature requested in #4400 was already part of the module file
configuration, but it was neither tested nor documented. This
commit takes care of adding a few lines in the documentation and a
regression test.
2018-07-24 11:27:17 -07:00
Massimiliano Culpo
e81c0c3e2c Updated references to spack module in the documentation. 2018-07-24 11:27:17 -07:00
Massimiliano Culpo
0457c1fbee Split 'spack module' into multiple commands
'spack module' has been split into multiple commands, each one tied to a
specific module type. This permits the specialization of the new
commands with features that are module type specific (e.g. set the
default module file in lmod when multiple versions of the same package
are installed at the same time).
2018-07-24 11:27:17 -07:00
Todd Gamblin
b2f3b07ec0 docs: doucment spack config blame 2018-07-21 14:04:18 -07:00
Todd Gamblin
331da779f8 docs: updates to spack config get docs for command-line scopes 2018-07-21 14:04:18 -07:00
Elizabeth Fischer
46f87cb9f2 docs: command-line scopes
Co-authored-by: Todd Gamblin <tgamblin@llnl.gov>
2018-07-21 14:04:18 -07:00
Todd Gamblin
264761ccb8 add py-pep8-naming package 2018-07-19 00:41:36 -07:00
Adam J. Stewart
8ce62ba513 Add documentation on build systems (#5015)
Spack provides a number of classes based on commonly-used build systems
that users can extend when writing packages; the classes provide functionality
to perform the actions relevant to the build system (e.g. running "configure" for
an Autotools-based package). This adds documentation for classes supporting the
following build systems:

* Makefile
* Autotools
* CMake
* QMake
* SCons
* Waf

This includes build systems for managing extensions of the following packages:

* Perl
* Python
* R
* Octave

This also adds documentation on implementing packages that use a custom build
system (e.g. Perl/CMake).

Spack also provides extendable classes which aggregate functionality for related
sets of packages, e.g. those using CUDA. Documentation is added for
CudaPackage.
2018-07-17 11:28:38 -07:00
Matthias Diener
90042749e8 fetch: remove 'trying' when cloning repos (#8651)
* fetch: remove 'trying' when cloning repos

This makes it similar to what the URL strategy prints.

* change svn/hg names
2018-07-10 10:22:51 -05:00
Christoph Junghans
8bc3f7d726 Add ccache support (#3761)
If the user sets "ccache: true" in spack's config.yaml, Spack will use an available
ccache executable when compiling c/c++ code. This feature is disabled by default
(i.e. "ccache: false") and the documentation is updated with how to enable
ccache support
2018-07-09 14:06:10 -07:00
George Hartzell
7f814971a5 Clear up a bit of modules/dotkit confusion (#8650)
`use` is an overloaded word between dotkit, modules and spack.  Add additional words to make the distinction clear in the docs.
2018-07-06 12:06:53 -05:00
Paul Chelarescu
719f324423 Removed duplicate word (#8609) 2018-06-29 07:44:23 -05:00
scheibelp
3560f6dbe9 views: packages can customize how they're added to views (#7152)
Functional updates:

- `python` now creates a copy of the `python` binaries when it is added
  to a view

- Python extensions (packages which subclass `PythonPackage`) rewrite
  their shebang lines to refer to python in the view

- Python packages in the same namespace will not generate conflicts if
  both have `...lib/site-packages/namespace-example/__init__.py`

  - These `__init__` files will also remain when removing any package in
    the namespace until the last package in the namespace is removed


Generally (Updated 2/16):

- Any package can define `add_files_to_view` to customize how it is added
  to a view (and at the moment custom definitions are included for
  `python` and `PythonPackage`)

  - Likewise any package can define `remove_files_from_view` to customize
    which files are removed (e.g. you don't always want to remove the
    namespace `__init__`)

- Any package can define `view_file_conflicts` to customize what it
  considers a merge conflict

- Global activations are handled like views (where the view root is the
  spec prefix of the extendee)

  - Benefit: filesystem-management aspects of activating extensions are
    now placed in views (e.g. now one can hardlink a global activation)

  - Benefit: overriding `Package.activate` is more straightforward (see
    `Python.activate`)

  - Complication: extension packages which have special-purpose logic
    *only* when activated outside of the extendee prefix must check for
    this in their `add_files_to_view` method (see `PythonPackage`)

- `LinkTree` is refactored to have separate methods for copying a
  directory structure and for copying files (since it was found that
  generally packages may want to alter how files are copied but still
  wanted to copy directories in the same way)


TODOs (updated 2/20):

- [x] additional testing (there is some unit testing added at this point
  but more would be useful)

- [x] refactor or reorganize `LinkTree` methods: currently there is a
  separate set of methods for replicating just the directory structure
  without the files, and a set for replicating everything

- [x] Right now external views (i.e. those not used for global
  activations) call `view.add_extension`, but global activations do not
  to avoid some extra work that goes into maintaining external views. I'm
  not sure if addressing that needs to be done here but I'd like to
  clarify it in the comments (UPDATE: for now I have added a TODO and in
  my opinion this can be merged now and the refactor handled later)

- [x] Several method descriptions (e.g. for `Package.activate`) are out
  of date and reference a distinction between global activations and
  views, they need to be updated

- [x] Update aspell package activations
2018-06-26 16:14:05 -07:00
Todd Gamblin
c4c1d37fcd refactor: move templates from root to share/spack
- This complies with the unix directory hierarchy standard (which Spack
  attempts to follow)
- Also unclutters the repo root directory.
2018-06-24 16:38:36 -07:00