Commit Graph

3850 Commits

Author SHA1 Message Date
Massimiliano Culpo
2e8aa6cb24 Configuration schema are checked against jsonschema draft 4 meta-schema (#10247)
fixes #10246
2019-08-27 09:31:18 -06:00
Adam J. Stewart
f31e87935e RPackage docs: use cloud.r-project.org (#12471) 2019-08-25 12:34:58 -05:00
Ben Boeckel
c22a145344 Add implicit rpaths to compiler detection (#7153)
Uses code from CMake to detect implicit link paths from compilers
System paths are filtered out of implicit link paths
Implicit link paths added to compiler config and object under `implicit_rpaths`
Implicit link paths added as rpaths to compile line through env/cc wrapper

Authored by: "Ben Boeckel <ben.boeckel@kitware.com>"
Co-authored by: "Peter Scheibel <scheibel1@llnl.gov>"
Co-authored by: "Gregory Becker <becker33@llnl.gov>"
2019-08-24 06:21:45 -07:00
Tamara Dahlgren
b42324c23b bugfix: template create constructor args (#12536)
c9e214f updated template creation by passing **kwargs to package
template classes but the template classes were not updated to accept
them; this adds **kwargs to package template initializers where they
are needed.
2019-08-23 10:45:17 -07:00
Seth R. Johnson
c86006e948 Skip invisible non-packages in package directory. (#12467)
Having a non-directory invisible file causes `spack find` to die. This
fixes the logic to ignore invalid module names but only warn if they're
visible.
```
NotADirectoryError: [Errno 20] Not a directory: '/spack/var/spack/repos/builtin/packages/.DS_Store/package.py'
```
2019-08-23 06:42:17 +02:00
Tamara Dahlgren
c9e214f6d3 Spack BundlePackage: a group of other packages (#11981)
This adds a special package type to Spack which is used to aggregate
a set of packages that a user might commonly install together; it
does not include any source code itself and does not require a
download URL like other Spack packages. It may include an 'install'
method to generate scripts, and Spack will run post-install hooks
(including module generation).

* Add new BundlePackage type
* Update the Xsdk package to be a BundlePackage and remove the
  'install' method (previously it had a noop install method)
* "spack create --template" now takes "bundle" as an option
* Rename cmd_create_repo fixture to "mock_test_repo" and relocate it
  to shared pytest fixtures
* Add unit tests for BundlePackage behavior
2019-08-22 11:08:23 -07:00
Greg Becker
5af2378ece compiler install: fix bug in construction of compiler spec (#12507) 2019-08-22 09:57:53 -07:00
Peter Scheibel
661b1da5d0 'spack spec --yaml' command now writes build deps to yaml (#12317)
This allows "spack spec --yaml" to generate a spec YAML file that can
be used with "spack install -f". Before, this would fail in cases
where the spec had build dependencies.
2019-08-21 18:32:31 -07:00
Chris Green
2499c047f0 Emacs directives for fill-column setting. 2019-08-21 18:21:05 -07:00
Chris Green
ad4309b782 Fetch strategies: new global option no_cache, new git option get_full_repo.
* All fetch strategies now accept the Boolean version keyword option `no_cache` in order to allow per-version control of cache-ability.
* New git-specific version keyword option `get_full_repo` (Boolean). When true, disables the default `--depth 1` and `--single-branch` optimizations that are applied if supported by the git version and (in the former case) transport protocol.
* The try / catch blog attempting `--depth 1` and retrying on failure has been removed in favor of more accurately ascertaining when the `--depth` option should work based on git version and protocol choice. Any failure is now treated as a real problem, and the clone is only attempted once.
* Test improvements:
  * `mock_git_repository.checks[type_of_test].args['git']` is now specified as the URL (with leading `file://`) in order to avoid complaints when using `--depth`.
  * New type_of_test `tag-branch`.
  * mock_git_repository now provides `git_exe`.
  * Improved the action of the `git_version` fixture, which was previously hard-wired.
  * New tests of `--single-branch` and `--depth 1` behavior.
* Add documentation of new options to the packaging guide.
2019-08-21 18:21:05 -07:00
Chris Green
ccb688f575 Fix to option text for help command. (#12444) 2019-08-21 17:11:24 -07:00
Peter Scheibel
5127d95b2e properly format message with all paths that could not be used. also report canonicalized paths vs. Spack config variables like 'tempdir' (#12492) 2019-08-21 15:46:12 -07:00
Chris Green
2441c0bad7 pkg command uses a working directory for git invocations rather than -C. (#12491) 2019-08-21 09:31:27 -07:00
Greg Becker
3b115fffb1 permissions: fix file permissions on intermediate install directories (#12399)
- mkdirp now takes arguments to allow it to properly set permissions on created directories.
- Two arguments (group and mode) set permissions for the leaf directory.
- Intermediate directories can inherit permissions from either the topmost existing directory (the parent) or the leaf.
2019-08-20 23:08:02 -07:00
Todd Gamblin
368bf2d69c stage: avoid using group name in _first_accessible_path()
On machines where $TMP is owned by a gid with no name, this avoids the
following error when the default spack stage does not exist:

    (spackbook):spack$ spack clean
    ==> Removing all temporary build stages
    ==> Error: 'getgrgid(): gid not found: 57095'

Spack needs to deal with gids directly unless users pass them in.
2019-08-20 14:52:53 -07:00
Tamara Dahlgren
aed204e409 cleanup: remove extraneous prints (#12474)
Removes messages accidentally introduced in #12072 .
2019-08-19 14:49:09 -07:00
Peter Scheibel
31ff791180 features: Update compiler caching (#7675)
Compiler caching was using the `id()` function to refer to configuration dictionary objects. If these objects are garbage-collected, this can produce incorrect results (false positive cache hits). This change replaces `id()` with an object that keeps a reference to the config dictionary so that it is not garbage-collected.
2019-08-19 11:24:05 -07:00
Tamara Dahlgren
0ea6e0f817 features: Remove stage symlinks (#12072)
Fixes #11163

The goal of this work is to simplify stage directory structures by eliminating use of symbolic links. This means, among other things, that` $spack/var/spack/stage` will no longer be the core staging directory. Instead, the first accessible `config:build_stage` path will be used.

Spack will no longer automatically append `spack-stage` (or the like) to configured build stage directories so the onus of distinguishing the directory from other work -- so the other work is not automatically removed with a `spack clean` operation -- falls on the user.
2019-08-19 10:31:24 -07:00
Zack Galbreath
cef1e4e0b4 Do not report on packages installed from the cache (#12336)
Skip generating reports for any packages that were found in the binary cache.
2019-08-16 10:19:11 -07:00
Greg Becker
0307b5a3dc mirrors: make spack mirror environment aware (#12345)
Allow "spack mirror" with no spec arguments to create a mirror for the
active environment (download all concretized specs in the environment)
2019-08-15 18:45:17 -07:00
Peter Scheibel
1f6befabb1 Fix "spack loads" when retrieving upstream modules (#12203)
Fixes #12062

406c791 addressed "spack module load" for upstream modules but not
the "spack module loads" command. This applies the same fixes from
406c791 to "spack module loads".
2019-08-13 11:09:40 -07:00
Massimiliano Culpo
ab4b5deb97 bugfix: Python 2.6 parsing error (#11867)
Apparently shlex.split can't deal with unicode encoded characters in
Python2.6. The solution is to convert to str before calling the
function.
2019-08-11 13:00:36 -07:00
Greg Becker
be6ac6ce6d bugfix: nested matrices in spec lists (#12320)
* stack concretization: fix handling of variant names with dashes
* spec_list: bugfix for handling nested matrices
2019-08-08 20:53:31 -07:00
Adam J. Stewart
44f4ad7bbf Allow Spack to uninstall external extensions without permissions (#11991)
* Allow Spack to uninstall external extensions without permissions

Raise when self.projections is not empty
2019-08-08 17:46:14 -07:00
Pariksheet Nanda
72bb7646fe docs: fix compiler config (#12330)
It's no longer possible to set compiler flags under as an entry under
"paths" in compilers.yaml; instead the user must list these under the
"flags" section. This updates the docs accordingly.
2019-08-08 16:47:24 -07:00
Adam J. Stewart
b4862be875 Fix typo in yaml config docs (#12312) 2019-08-08 10:56:11 +02:00
Greg Becker
36fbd6cf4c bugfix: spack stacks mixed dependency and non-dep constraints (#12315)
Spack stacks drop invalid dependencies applied to packages by a
spec_list matrix operation

Without this fix, Spack would raise an error if orthogonal dependency
constraints and non-dependency constraints were applied to the same
package by a matrix and the dependency constraint was invalid for
that package. This is an error, fixed by this PR.

An example failing configuration:

spack:
  definitions:
  - packages: [libelf, hdf5+mpi]
  - compilers: ['%gcc']
  - mpis: [^openmpi]

  specs:
  - matrix:
    - $packages
    - $compilers
    - $mpis
2019-08-07 18:34:41 -07:00
Carson Woods
5b11f7aa4c Fix bug where environments would ignore global path 2019-08-07 18:32:28 -06:00
Carson Woods
97e46981b9 Remove old doc from doc index 2019-08-07 18:32:28 -06:00
Carson Woods
873ac5e890 Remove old documentation for shared mode 2019-08-07 18:32:28 -06:00
Carson Woods
4d7dae5716 Remove old share command from tests 2019-08-07 18:32:28 -06:00
Carson Woods
b19f0fafcc Remove outdate share command 2019-08-07 18:32:28 -06:00
Carson Woods
11b1bdd119 Pep8 Compliance Fix 2019-08-07 18:32:28 -06:00
Carson Woods
f749821dc2 Pep8 Compliance 2019-08-07 18:32:28 -06:00
Carson Woods
5abb20dcab Rename test 2019-08-07 18:32:28 -06:00
Carson Woods
0c233bdd0f Add test for validating upstream database initialization 2019-08-07 18:32:28 -06:00
Carson Woods
0f171c7ded Replace space with = in command parameter 2019-08-07 18:32:28 -06:00
Carson Woods
b4c7520dd8 Flake8 Test Compliance 2019-08-07 18:32:28 -06:00
Carson Woods
9ab7d8f01d Add config parameter for active upstream to set install location for modules 2019-08-07 18:32:28 -06:00
Carson Woods
a7ad344c2a Add upstreams.yaml testing file so calls to upstreams['global] doesn't cause tests to fail 2019-08-07 18:32:28 -06:00
Carson Woods
deb2d3745c Fix .spack-db/index.json not being created in global upstream if previously uninitialized 2019-08-07 18:32:28 -06:00
Carson Woods
ff96ec430b Can now specify upstream of anyname through -u/--upstream flag 2019-08-07 18:32:28 -06:00
Carson Woods
d4a959736a Flake8 Compliance Changes 2019-08-07 18:32:28 -06:00
Carson Woods
5ba51a0be0 --global option now works for both install and uninstall 2019-08-07 18:32:28 -06:00
Carson Woods
27e1140df7 Reset active directory after each global install 2019-08-07 18:32:28 -06:00
Carson Woods
7ab6af8a3b Add scope to setting active tree to ensure that it is set at user level 2019-08-07 18:32:28 -06:00
Carson Woods
0e6e93eaac Fix unit test config.yaml 2019-08-07 18:32:28 -06:00
Carson Woods
38f8bdd2bb Home expansion was removed because it was no longer being used 2019-08-07 18:32:27 -06:00
Carson Woods
8e45a3fc2f Fix flake8 compliance 2019-08-07 18:32:27 -06:00
Carson Woods
c22af99b04 Fix how upstream db paths are canonicalized 2019-08-07 18:32:27 -06:00