Commit Graph

4155 Commits

Author SHA1 Message Date
Todd Gamblin
64af0a9874 command: add spack find --format
- spack find --format allows you to supply a format string and have specs
  output in a more machine-readable way, without dedcoration

e.g.:

    spack find --format "{name}-{version}-{hash}"
    autoconf-2.69-icynozk7ti6h4ezzgonqe6jgw5f3ulx4
    automake-1.16.1-o5v3tc77kesgonxjbmeqlwfmb5qzj7zy
    bzip2-1.0.6-syohzw57v2jfag5du2x4bowziw3m5p67
    ...

or:

    spack find --format "{hash}"
    icynozk7ti6h4ezzgonqe6jgw5f3ulx4
    o5v3tc77kesgonxjbmeqlwfmb5qzj7zy
    syohzw57v2jfag5du2x4bowziw3m5p67
    ...

This is intended to make it much easier to script with `spack find`
2019-09-02 19:24:48 -07:00
Peter Scheibel
9432d4c7ee Archive all patches in a package after install (#12653)
When Spack installs a package it writes the package.py file and
patches to a separate repository (which reflects the state of the
package at the time it was installed). Previously, Spack only wrote
patches that were used at installation time. This updates the
archiving step to include all patch files that are relevant to the
package (in case that repository is used in another context).
2019-08-30 10:33:18 -07:00
t-karatsu
0c421589e6 Fujitsu compiler: Add flags for c11/c99 (#12622)
* Fujitsu compiler: Add flags for c11/c99

* Add unit tests about supported flags
2019-08-30 11:25:34 -05:00
Greg Becker
c8b613d261 fix packaging_guide variant table (#12296) 2019-08-29 09:35:48 -06:00
Geoffrey M Oxberry
56fd9bc7b6 AutotoolsPackage: remove redundant libtoolize call
This commit removes redundant calls to `libtoolize` and `aclocal`.

Some configurations, such as a Spack user using macOS with a
Homebrew-installed `libtool` added to their `packages.yaml`, have
`autoreconf` and GNU libtoolize installed as `glibtoolize`, but not
`libtoolize`. While Spack installations of `libtool` built from source
would install `glibtoolize` and symlink `libtoolize` to `glibtoolize`,
an external installation of GNU libtoolize as `glibtoolize` will not
have such a symlink, and thus the call `m.libtoolize()` will throw an
error because `libtoolize` does not exist at the path referenced by
`m.libtoolize()` (i.e.,
`self.spec['libtool'].prefix.bin.join('libtoolize')).

However, on these same systems, `autoreconf` runs correctly, and calls
`glibtoolize` instead of `libtoolize`, when appropriate. Thus,
removing the call to `libtoolize` should resolve the error mentioned
above.

The redundant call to `aclocal` is also removed in this commit because
the maintainers of GNU Automake state that "`aclocal` is expected to
disappear" and suggest that downstream users never call `aclocal`
directly -- rather, they suggest calling `autoreconf` instead.
2019-08-27 18:03:56 -07:00
Massimiliano Culpo
2e8aa6cb24 Configuration schema are checked against jsonschema draft 4 meta-schema (#10247)
fixes #10246
2019-08-27 09:31:18 -06:00
Adam J. Stewart
f31e87935e RPackage docs: use cloud.r-project.org (#12471) 2019-08-25 12:34:58 -05:00
Ben Boeckel
c22a145344 Add implicit rpaths to compiler detection (#7153)
Uses code from CMake to detect implicit link paths from compilers
System paths are filtered out of implicit link paths
Implicit link paths added to compiler config and object under `implicit_rpaths`
Implicit link paths added as rpaths to compile line through env/cc wrapper

Authored by: "Ben Boeckel <ben.boeckel@kitware.com>"
Co-authored by: "Peter Scheibel <scheibel1@llnl.gov>"
Co-authored by: "Gregory Becker <becker33@llnl.gov>"
2019-08-24 06:21:45 -07:00
Tamara Dahlgren
b42324c23b bugfix: template create constructor args (#12536)
c9e214f updated template creation by passing **kwargs to package
template classes but the template classes were not updated to accept
them; this adds **kwargs to package template initializers where they
are needed.
2019-08-23 10:45:17 -07:00
Seth R. Johnson
c86006e948 Skip invisible non-packages in package directory. (#12467)
Having a non-directory invisible file causes `spack find` to die. This
fixes the logic to ignore invalid module names but only warn if they're
visible.
```
NotADirectoryError: [Errno 20] Not a directory: '/spack/var/spack/repos/builtin/packages/.DS_Store/package.py'
```
2019-08-23 06:42:17 +02:00
Tamara Dahlgren
c9e214f6d3 Spack BundlePackage: a group of other packages (#11981)
This adds a special package type to Spack which is used to aggregate
a set of packages that a user might commonly install together; it
does not include any source code itself and does not require a
download URL like other Spack packages. It may include an 'install'
method to generate scripts, and Spack will run post-install hooks
(including module generation).

* Add new BundlePackage type
* Update the Xsdk package to be a BundlePackage and remove the
  'install' method (previously it had a noop install method)
* "spack create --template" now takes "bundle" as an option
* Rename cmd_create_repo fixture to "mock_test_repo" and relocate it
  to shared pytest fixtures
* Add unit tests for BundlePackage behavior
2019-08-22 11:08:23 -07:00
Greg Becker
5af2378ece compiler install: fix bug in construction of compiler spec (#12507) 2019-08-22 09:57:53 -07:00
Peter Scheibel
661b1da5d0 'spack spec --yaml' command now writes build deps to yaml (#12317)
This allows "spack spec --yaml" to generate a spec YAML file that can
be used with "spack install -f". Before, this would fail in cases
where the spec had build dependencies.
2019-08-21 18:32:31 -07:00
Chris Green
2499c047f0 Emacs directives for fill-column setting. 2019-08-21 18:21:05 -07:00
Chris Green
ad4309b782 Fetch strategies: new global option no_cache, new git option get_full_repo.
* All fetch strategies now accept the Boolean version keyword option `no_cache` in order to allow per-version control of cache-ability.
* New git-specific version keyword option `get_full_repo` (Boolean). When true, disables the default `--depth 1` and `--single-branch` optimizations that are applied if supported by the git version and (in the former case) transport protocol.
* The try / catch blog attempting `--depth 1` and retrying on failure has been removed in favor of more accurately ascertaining when the `--depth` option should work based on git version and protocol choice. Any failure is now treated as a real problem, and the clone is only attempted once.
* Test improvements:
  * `mock_git_repository.checks[type_of_test].args['git']` is now specified as the URL (with leading `file://`) in order to avoid complaints when using `--depth`.
  * New type_of_test `tag-branch`.
  * mock_git_repository now provides `git_exe`.
  * Improved the action of the `git_version` fixture, which was previously hard-wired.
  * New tests of `--single-branch` and `--depth 1` behavior.
* Add documentation of new options to the packaging guide.
2019-08-21 18:21:05 -07:00
Chris Green
ccb688f575 Fix to option text for help command. (#12444) 2019-08-21 17:11:24 -07:00
Peter Scheibel
5127d95b2e properly format message with all paths that could not be used. also report canonicalized paths vs. Spack config variables like 'tempdir' (#12492) 2019-08-21 15:46:12 -07:00
Chris Green
2441c0bad7 pkg command uses a working directory for git invocations rather than -C. (#12491) 2019-08-21 09:31:27 -07:00
Greg Becker
3b115fffb1 permissions: fix file permissions on intermediate install directories (#12399)
- mkdirp now takes arguments to allow it to properly set permissions on created directories.
- Two arguments (group and mode) set permissions for the leaf directory.
- Intermediate directories can inherit permissions from either the topmost existing directory (the parent) or the leaf.
2019-08-20 23:08:02 -07:00
Todd Gamblin
368bf2d69c stage: avoid using group name in _first_accessible_path()
On machines where $TMP is owned by a gid with no name, this avoids the
following error when the default spack stage does not exist:

    (spackbook):spack$ spack clean
    ==> Removing all temporary build stages
    ==> Error: 'getgrgid(): gid not found: 57095'

Spack needs to deal with gids directly unless users pass them in.
2019-08-20 14:52:53 -07:00
Tamara Dahlgren
aed204e409 cleanup: remove extraneous prints (#12474)
Removes messages accidentally introduced in #12072 .
2019-08-19 14:49:09 -07:00
Peter Scheibel
31ff791180 features: Update compiler caching (#7675)
Compiler caching was using the `id()` function to refer to configuration dictionary objects. If these objects are garbage-collected, this can produce incorrect results (false positive cache hits). This change replaces `id()` with an object that keeps a reference to the config dictionary so that it is not garbage-collected.
2019-08-19 11:24:05 -07:00
Tamara Dahlgren
0ea6e0f817 features: Remove stage symlinks (#12072)
Fixes #11163

The goal of this work is to simplify stage directory structures by eliminating use of symbolic links. This means, among other things, that` $spack/var/spack/stage` will no longer be the core staging directory. Instead, the first accessible `config:build_stage` path will be used.

Spack will no longer automatically append `spack-stage` (or the like) to configured build stage directories so the onus of distinguishing the directory from other work -- so the other work is not automatically removed with a `spack clean` operation -- falls on the user.
2019-08-19 10:31:24 -07:00
Zack Galbreath
cef1e4e0b4 Do not report on packages installed from the cache (#12336)
Skip generating reports for any packages that were found in the binary cache.
2019-08-16 10:19:11 -07:00
Greg Becker
0307b5a3dc mirrors: make spack mirror environment aware (#12345)
Allow "spack mirror" with no spec arguments to create a mirror for the
active environment (download all concretized specs in the environment)
2019-08-15 18:45:17 -07:00
Peter Scheibel
1f6befabb1 Fix "spack loads" when retrieving upstream modules (#12203)
Fixes #12062

406c791 addressed "spack module load" for upstream modules but not
the "spack module loads" command. This applies the same fixes from
406c791 to "spack module loads".
2019-08-13 11:09:40 -07:00
Massimiliano Culpo
ab4b5deb97 bugfix: Python 2.6 parsing error (#11867)
Apparently shlex.split can't deal with unicode encoded characters in
Python2.6. The solution is to convert to str before calling the
function.
2019-08-11 13:00:36 -07:00
Greg Becker
be6ac6ce6d bugfix: nested matrices in spec lists (#12320)
* stack concretization: fix handling of variant names with dashes
* spec_list: bugfix for handling nested matrices
2019-08-08 20:53:31 -07:00
Adam J. Stewart
44f4ad7bbf Allow Spack to uninstall external extensions without permissions (#11991)
* Allow Spack to uninstall external extensions without permissions

Raise when self.projections is not empty
2019-08-08 17:46:14 -07:00
Pariksheet Nanda
72bb7646fe docs: fix compiler config (#12330)
It's no longer possible to set compiler flags under as an entry under
"paths" in compilers.yaml; instead the user must list these under the
"flags" section. This updates the docs accordingly.
2019-08-08 16:47:24 -07:00
Adam J. Stewart
b4862be875 Fix typo in yaml config docs (#12312) 2019-08-08 10:56:11 +02:00
Greg Becker
36fbd6cf4c bugfix: spack stacks mixed dependency and non-dep constraints (#12315)
Spack stacks drop invalid dependencies applied to packages by a
spec_list matrix operation

Without this fix, Spack would raise an error if orthogonal dependency
constraints and non-dependency constraints were applied to the same
package by a matrix and the dependency constraint was invalid for
that package. This is an error, fixed by this PR.

An example failing configuration:

spack:
  definitions:
  - packages: [libelf, hdf5+mpi]
  - compilers: ['%gcc']
  - mpis: [^openmpi]

  specs:
  - matrix:
    - $packages
    - $compilers
    - $mpis
2019-08-07 18:34:41 -07:00
Carson Woods
5b11f7aa4c Fix bug where environments would ignore global path 2019-08-07 18:32:28 -06:00
Carson Woods
97e46981b9 Remove old doc from doc index 2019-08-07 18:32:28 -06:00
Carson Woods
873ac5e890 Remove old documentation for shared mode 2019-08-07 18:32:28 -06:00
Carson Woods
4d7dae5716 Remove old share command from tests 2019-08-07 18:32:28 -06:00
Carson Woods
b19f0fafcc Remove outdate share command 2019-08-07 18:32:28 -06:00
Carson Woods
11b1bdd119 Pep8 Compliance Fix 2019-08-07 18:32:28 -06:00
Carson Woods
f749821dc2 Pep8 Compliance 2019-08-07 18:32:28 -06:00
Carson Woods
5abb20dcab Rename test 2019-08-07 18:32:28 -06:00
Carson Woods
0c233bdd0f Add test for validating upstream database initialization 2019-08-07 18:32:28 -06:00
Carson Woods
0f171c7ded Replace space with = in command parameter 2019-08-07 18:32:28 -06:00
Carson Woods
b4c7520dd8 Flake8 Test Compliance 2019-08-07 18:32:28 -06:00
Carson Woods
9ab7d8f01d Add config parameter for active upstream to set install location for modules 2019-08-07 18:32:28 -06:00
Carson Woods
a7ad344c2a Add upstreams.yaml testing file so calls to upstreams['global] doesn't cause tests to fail 2019-08-07 18:32:28 -06:00
Carson Woods
deb2d3745c Fix .spack-db/index.json not being created in global upstream if previously uninitialized 2019-08-07 18:32:28 -06:00
Carson Woods
ff96ec430b Can now specify upstream of anyname through -u/--upstream flag 2019-08-07 18:32:28 -06:00
Carson Woods
d4a959736a Flake8 Compliance Changes 2019-08-07 18:32:28 -06:00
Carson Woods
5ba51a0be0 --global option now works for both install and uninstall 2019-08-07 18:32:28 -06:00
Carson Woods
27e1140df7 Reset active directory after each global install 2019-08-07 18:32:28 -06:00