Commit Graph

894 Commits

Author SHA1 Message Date
Tom Scogland
7f1659786b Add a Lua build-system (#28854)
Reworking lua to allow easier substitution of the base lua implementation.

Also adding in a maintained version of luajit and re-factoring the entire stack 
to use a custom build-system to centralize functionality like environment 
variable management and luarocks installation.

The `lua-lang` virtual is now versioned so that a package that requires 
Lua 5.1 semantics can get any lua, but one that requires 5.2 will only 
get upstream lua.

The luaposix package requires lua-bit32, but only when built with a 
lua conforming to version 5.1.  This adds the package, and the 
dependencies, but exposed a problem with luarocks dependency 
detection.  Since we're  installing each package in its own "tree" and 
there's no environment  variable to list extra trees, spack now 
generates a luarocks config  file that lists all the trees of all the 
dependencies, and references  it by setting `LUAROCKS_CONFIG` 
in the build environment of every LuaPackage.  This allows luarocks 
to find the spack installed  dependencies correctly rather than 
trying (and failing) to download them.

Co-authored-by: Adam J. Stewart <ajstewart426@gmail.com>
Co-authored-by: Tom Scogland <tscogland@llnl.gov>
Co-authored-by: Massimiliano Culpo <massimiliano.culpo@gmail.com>
2022-05-10 06:54:38 +02:00
Harmen Stoppels
2f14695882 docs: jobserver & generated makefiles (#30526) 2022-05-06 14:04:48 +02:00
Harmen Stoppels
2836648904 Makefile generator for parallel spack install of environments (#30254)
`make` solves a lot of headaches that would otherwise have to be implemented in Spack:

1. Parallelism over packages through multiple `spack install` processes
2. Orderly output of parallel package installs thanks to `make --sync-output=recurse` or `make -Orecurse` (works well in GNU Make 4.3; macOS is unfortunately on a 16 years old 3.x version, but it's one `spack install gmake` away...)
3. Shared jobserver across packages, which means a single `-j` to rule them all, instead of manually finding a balance between `#spack install processes` & `#jobs per package` (See #30302).

This pr adds the `spack env depfile` command that generates a Makefile with dag hashes as
targets, and dag hashes of dependencies as prerequisites, and a command
along the lines of `spack install --only=packages /hash` to just install
a single package.

It exposes two convenient phony targets: `all`, `fetch-all`. The former installs the environment, the latter just fetches all sources. So one can either use `make all -j16` directly or run `make fetch-all -j16` on a login node and `make all -j16` on a compute node. 

Example:

```yaml
spack:
  specs: [perl]
  view: false
```

running

```
$ spack -e . env depfile --make-target-prefix env | tee Makefile
```
generates

```Makefile
SPACK ?= spack

.PHONY: env/all env/fetch-all env/clean

env/all: env/env

env/fetch-all: env/fetch

env/env: env/.install/cdqldivylyxocqymwnfzmzc5sx2zwvww
	@touch $@

env/fetch: env/.fetch/cdqldivylyxocqymwnfzmzc5sx2zwvww env/.fetch/gv5kin2xnn33uxyfte6k4a3bynhmtxze env/.fetch/cuymc7e5gupwyu7vza5d4vrbuslk277p env/.fetch/7vangk4jvsdgw6u6oe6ob63pyjl5cbgk env/.fetch/hyb7ehxxyqqp2hiw56bzm5ampkw6cxws env/.fetch/yfz2agazed7ohevqvnrmm7jfkmsgwjao env/.fetch/73t7ndb5w72hrat5hsax4caox2sgumzu env/.fetch/trvdyncxzfozxofpm3cwgq4vecpxixzs env/.fetch/sbzszb7v557ohyd6c2ekirx2t3ctxfxp env/.fetch/c4go4gxlcznh5p5nklpjm644epuh3pzc
	@touch $@

env/dirs:
	@mkdir -p env/.fetch env/.install

env/.fetch/%: | env/dirs
	$(info Fetching $(SPEC))
	$(SPACK) -e '/tmp/tmp.7PHPSIRACv' fetch $(SPACK_FETCH_FLAGS) /$(notdir $@) && touch $@

env/.install/%: env/.fetch/%
	$(info Installing $(SPEC))
	+$(SPACK) -e '/tmp/tmp.7PHPSIRACv' install $(SPACK_INSTALL_FLAGS) --only-concrete --only=package --no-add /$(notdir $@) && touch $@

# Set the human-readable spec for each target
env/%/cdqldivylyxocqymwnfzmzc5sx2zwvww: SPEC = perl@5.34.1%gcc@10.3.0+cpanm+shared+threads arch=linux-ubuntu20.04-zen2
env/%/gv5kin2xnn33uxyfte6k4a3bynhmtxze: SPEC = berkeley-db@18.1.40%gcc@10.3.0+cxx~docs+stl patches=b231fcc arch=linux-ubuntu20.04-zen2
env/%/cuymc7e5gupwyu7vza5d4vrbuslk277p: SPEC = bzip2@1.0.8%gcc@10.3.0~debug~pic+shared arch=linux-ubuntu20.04-zen2
env/%/7vangk4jvsdgw6u6oe6ob63pyjl5cbgk: SPEC = diffutils@3.8%gcc@10.3.0 arch=linux-ubuntu20.04-zen2
env/%/hyb7ehxxyqqp2hiw56bzm5ampkw6cxws: SPEC = libiconv@1.16%gcc@10.3.0 libs=shared,static arch=linux-ubuntu20.04-zen2
env/%/yfz2agazed7ohevqvnrmm7jfkmsgwjao: SPEC = gdbm@1.19%gcc@10.3.0 arch=linux-ubuntu20.04-zen2
env/%/73t7ndb5w72hrat5hsax4caox2sgumzu: SPEC = readline@8.1%gcc@10.3.0 arch=linux-ubuntu20.04-zen2
env/%/trvdyncxzfozxofpm3cwgq4vecpxixzs: SPEC = ncurses@6.2%gcc@10.3.0~symlinks+termlib abi=none arch=linux-ubuntu20.04-zen2
env/%/sbzszb7v557ohyd6c2ekirx2t3ctxfxp: SPEC = pkgconf@1.8.0%gcc@10.3.0 arch=linux-ubuntu20.04-zen2
env/%/c4go4gxlcznh5p5nklpjm644epuh3pzc: SPEC = zlib@1.2.12%gcc@10.3.0+optimize+pic+shared patches=0d38234 arch=linux-ubuntu20.04-zen2

# Install dependencies
env/.install/cdqldivylyxocqymwnfzmzc5sx2zwvww: env/.install/gv5kin2xnn33uxyfte6k4a3bynhmtxze env/.install/cuymc7e5gupwyu7vza5d4vrbuslk277p env/.install/yfz2agazed7ohevqvnrmm7jfkmsgwjao env/.install/c4go4gxlcznh5p5nklpjm644epuh3pzc
env/.install/cuymc7e5gupwyu7vza5d4vrbuslk277p: env/.install/7vangk4jvsdgw6u6oe6ob63pyjl5cbgk
env/.install/7vangk4jvsdgw6u6oe6ob63pyjl5cbgk: env/.install/hyb7ehxxyqqp2hiw56bzm5ampkw6cxws
env/.install/yfz2agazed7ohevqvnrmm7jfkmsgwjao: env/.install/73t7ndb5w72hrat5hsax4caox2sgumzu
env/.install/73t7ndb5w72hrat5hsax4caox2sgumzu: env/.install/trvdyncxzfozxofpm3cwgq4vecpxixzs
env/.install/trvdyncxzfozxofpm3cwgq4vecpxixzs: env/.install/sbzszb7v557ohyd6c2ekirx2t3ctxfxp

env/clean:
	rm -f -- env/env env/fetch env/.fetch/cdqldivylyxocqymwnfzmzc5sx2zwvww env/.fetch/gv5kin2xnn33uxyfte6k4a3bynhmtxze env/.fetch/cuymc7e5gupwyu7vza5d4vrbuslk277p env/.fetch/7vangk4jvsdgw6u6oe6ob63pyjl5cbgk env/.fetch/hyb7ehxxyqqp2hiw56bzm5ampkw6cxws env/.fetch/yfz2agazed7ohevqvnrmm7jfkmsgwjao env/.fetch/73t7ndb5w72hrat5hsax4caox2sgumzu env/.fetch/trvdyncxzfozxofpm3cwgq4vecpxixzs env/.fetch/sbzszb7v557ohyd6c2ekirx2t3ctxfxp env/.fetch/c4go4gxlcznh5p5nklpjm644epuh3pzc env/.install/cdqldivylyxocqymwnfzmzc5sx2zwvww env/.install/gv5kin2xnn33uxyfte6k4a3bynhmtxze env/.install/cuymc7e5gupwyu7vza5d4vrbuslk277p env/.install/7vangk4jvsdgw6u6oe6ob63pyjl5cbgk env/.install/hyb7ehxxyqqp2hiw56bzm5ampkw6cxws env/.install/yfz2agazed7ohevqvnrmm7jfkmsgwjao env/.install/73t7ndb5w72hrat5hsax4caox2sgumzu env/.install/trvdyncxzfozxofpm3cwgq4vecpxixzs env/.install/sbzszb7v557ohyd6c2ekirx2t3ctxfxp env/.install/c4go4gxlcznh5p5nklpjm644epuh3pzc
```

Then with `make -O` you get very nice orderly output when packages are built in parallel:
```console
$ make -Orecurse -j16
spack -e . install --only-concrete --only=package /c4go4gxlcznh5p5nklpjm644epuh3pzc && touch c4go4gxlcznh5p5nklpjm644epuh3pzc
==> Installing zlib-1.2.12-c4go4gxlcznh5p5nklpjm644epuh3pzc
...
  Fetch: 0.00s.  Build: 0.88s.  Total: 0.88s.
[+] /tmp/tmp.b1eTyAOe85/store/linux-ubuntu20.04-zen2/gcc-10.3.0/zlib-1.2.12-c4go4gxlcznh5p5nklpjm644epuh3pzc
spack -e . install --only-concrete --only=package /sbzszb7v557ohyd6c2ekirx2t3ctxfxp && touch sbzszb7v557ohyd6c2ekirx2t3ctxfxp
==> Installing pkgconf-1.8.0-sbzszb7v557ohyd6c2ekirx2t3ctxfxp
...
  Fetch: 0.00s.  Build: 3.96s.  Total: 3.96s.
[+] /tmp/tmp.b1eTyAOe85/store/linux-ubuntu20.04-zen2/gcc-10.3.0/pkgconf-1.8.0-sbzszb7v557ohyd6c2ekirx2t3ctxfxp
```

For Perl, at least for me, using `make -j16` versus `spack -e . install -j16` speeds up the builds from 3m32.623s to 2m22.775s, as some configure scripts run in parallel.

Another nice feature is you can do Makefile "metaprogramming" and depend on packages built by Spack. This example fetches all sources (in parallel) first, print a message, and only then build packages (in parallel).

```Makefile
SPACK ?= spack

.PHONY: env

all: env

spack.lock: spack.yaml
	$(SPACK) -e . concretize -f

env.mk: spack.lock
	$(SPACK) -e . env depfile -o $@ --make-target-prefix spack

fetch: spack/fetch
	@echo Fetched all packages && touch $@

env: fetch spack/env
	@echo This executes after the environment has been installed

clean:
	rm -rf spack/ env.mk spack.lock

ifeq (,$(filter clean,$(MAKECMDGOALS)))
include env.mk
endif
```
2022-05-05 10:45:21 -07:00
Ken Raffenetti
a24070d532 docs: Fix ROCmPackage example syntax (#30168) 2022-04-26 16:00:40 -07:00
Massimiliano Culpo
b9d6a5103d ASP-based solver: allow configuring target selection (#29835)
* ASP-based solver: allow configuring target selection

This commit adds a new "concretizer:targets" configuration
section, and two options under it.

 - "concretizer:targets:granularity" allows switching from
considering only generic targets to consider all possible
microarchitectures.

 - "concretizer:targets:host_compatible" instead controls
whether we can concretize for microarchitectures that
are incompatible with the current host.

* Add documentation

* Add unit-tests
2022-04-25 17:19:51 -07:00
Harmen Stoppels
b7f8899d45 docs: add (config.yaml) to sections for faster lookup by config file (#30157) 2022-04-20 15:00:43 +02:00
Adam J. Stewart
83533fc31f PythonPackage docs: Spack now supports Windows (#30177) 2022-04-20 09:11:19 +02:00
Massimiliano Culpo
c846b5149d Add support for Python 3.10 (#29581)
* Add support for Python 3.10

* Update unit-tests to use 3.10

* Update Getting started section of the docs

* Update bootstrap action
2022-04-13 14:32:23 -07:00
Harmen Stoppels
99083f1706 Deprecate top-level module config (#28659)
* Ignore top-level module config; add auto-update

In Spack 0.17 we got module sets (modules:[name]:[prop]), and for
backwards compat modules:[prop] was short for modules:default:[prop].

But this makes it awkward to define default config for the "default"
module set.

Since 0.17 is branched off, we can now deprecate top-level module config
(that is, just ignore it with a warning).

This PR does that, and it implements `spack config update modules` to
make upgrading easy (we should have added that to 0.17 already...)

It also removes references to  `dotkit` stuff which was already
deprecated in 0.13 and could have been removed in 0.14.

Prefix inspections are the only exception, since the top-level prefix inspections
used for `spack load` and `spack env activate`.
2022-04-08 19:00:35 +00:00
Massimiliano Culpo
ff04d1bfc1 Use the non-deprecated MetaPathFinder interface (#29745)
* Extract the MetaPathFinder and Loaders for packages in their own classes

https://peps.python.org/pep-0451/

Currently, RepoPath and Repo implement the (deprecated) interface of
MetaPathFinder (find_module) and of Loader (load_module). This commit
extracts both of them and places the code in their own classes.

The MetaPathFinder interface is updated to contain both the deprecated
"find_module" (for Python 2.7 support) and the recommended "find_spec".
Update of the Loader interface is deferred at a subsequent commit.

* Move the lines to be prepended inside "RepoLoader"

Also adjust the naming of a few variables too

* Remove spack.util.imp, since code is only used in spack.repo

* Remove support from loading Python modules Python > 3 but < 3.5

* Remove `Repo._create_namespace`

This function was interacting badly with the MetaPathFinder
and causing issues with "normal" imports. Removing the
function allows to do things like:
```python
import spack.pkg.builtin.mpich
cls = spack.pkg.builtin.mpich.Mpich
```

* Remove code needed to trigger the Singleton evaluation

The finder is coded in a way to trigger the Singleton,
so we don't need external code now that we register it
at module level into `sys.meta_path`.

* Add unit tests
2022-04-07 15:58:20 -07:00
Massimiliano Culpo
f2fc4ee9af Allow conditional possible values in variants (#29530)
Allow declaring possible values for variants with an associated condition. If the variant takes one of those values, the condition is imposed as a further constraint.

The idea of this PR is to implement part of the mechanisms needed for modeling [packages with multiple build-systems]( https://github.com/spack/seps/pull/3). After this PR the build-system directive can be implemented as:
```python
variant(
    'build-system',
    default='cmake',
    values=(
        'autotools',
        conditional('cmake', when='@X.Y:')
    ), 
    description='...',
)
```

Modifications:
- [x] Allow conditional possible values in variants
- [x] Add a unit-test for the feature
- [x] Add documentation
2022-04-04 17:37:57 -07:00
Thomas Dickerson
ee505e6c69 Add support for racket packages (#27564)
- Add variants for various common build flags, including support for both versions of the Racket VM environment.
- Prevent `-j` flags to `make`, which has been known to cause problems with Racket builds.
- Prefer the minimal release to improve install times. Bells and whistles carry their own runtime dependencies and should be installed via `raco`. An enterprising user may even create a `RacketPackage` class to make spack aware of `raco` installed packages.
- Match the official version numbering scheme.
2022-04-04 10:32:25 +02:00
Harmen Stoppels
9516fa9447 cmake: use CMAKE_INSTALL_RPATH_USE_LINK_PATH (#29703)
* cmake: use CMAKE_INSTALL_RPATH_USE_LINK_PATH

Spack has a heuristic to add rpaths for packages it knows are required,
but it's really a heuristic, and it does not work when the dependencies
put their libraries in a different folder than `<prefix>/lib{64,}`.

CMake patches binaries after install with the "install rpaths", which by
default are provided by Spack and its heuristic through
`CMAKE_INSTALL_RPATH`.

CMake however knows better what libraries are effectively being linked
to, and has an option to include those in the install rpath too, through
`CMAKE_INSTALL_RPATH_USE_LINK_PATH`.

These two CMake options are complementary, repeated rpaths seem to be
filtered, and the "use link path" paths are appended to Spack's
heuristic "install rpath".

So, it seems like a good idea to enable "use link path" by default, so
that:
- `dlopen` by library name uses Spack's heuristic search paths
- linked libraries in non-standard locations within a prefix get an
rpath thanks to CMake.

* docs
2022-03-29 12:24:10 -04:00
百地 希留耶
cd00eba9d6 autotools doc: add missing right quote (#29734) 2022-03-27 22:42:29 +00:00
Massimiliano Culpo
8f89932aad Remove known issues from documentation (#29664)
Known issues reports only 2 issues, among the bugs reported on GitHub.
One of the two is also outdated, since the issue has been solved
with the new concretizer. Thus, this commit removes the section.
2022-03-23 08:06:49 +01:00
Harmen Stoppels
8f5b9a89fb major.minor.micro.dev0 Spack version (#25267)
When you install Spack from a tarball, it will always show an exact
version for Spack itself, even when you don't download a tagged commit:

```
$ wget -q https://github.com/spack/spack/archive/refs/heads/develop.tar.gz
$ tar -xf develop.tar.gz
$ ./spack-develop/bin/spack --version
0.16.2
```

This PR sets the Spack version to `0.18.0.dev0` on develop, following [PEP440](https://github.com/spack/spack/pull/25267#issuecomment-896340234) as
suggested by Adam Stewart.

```
spack (fix/set-dev-version)$ spack --version
0.18.0.dev0 (git 0.17.1-1526-e270464ae0)
spack (fix/set-dev-version)$ mv .git .git_
spack $ spack --version
0.18.0.dev0
```

- [x] Update the release guide
- [x] Add __version__ to spack's __init__.py
- [x] Use PEP 440 canonical version strings
- [x] Make spack --version output [actual version] (git version)

Co-authored-by: Todd Gamblin <tgamblin@llnl.gov>
2022-03-20 22:37:55 -06:00
Massimiliano Culpo
2fa495154e Split the workflow section and remove outdated advices (#29344)
This PR removes a few outdated sections from the "Basics" part of the 
documentation. It also makes a few topic under the environment section
more prominent by removing an unneeded spack.yaml subsection and 
promoting everything under it.
2022-03-18 10:41:27 +01:00
Zack Galbreath
5a72204d38 Remove references to features/windows-support branch (#29565) 2022-03-17 10:29:29 -07:00
John Parent
4aee27816e Windows Support: Testing Suite integration
Broaden support for execution of the test suite
on Windows.
General bug and review fixups
2022-03-17 09:01:01 -07:00
John Parent
cf1349ba35 "spack commands --update-completion" 2022-03-17 09:01:01 -07:00
John Parent
df4129d395 Expand external find for Windows (#27588)
* Incorporate new search location

* Add external user option

* proper doc string

* Explicit commands in getting started

* raise during chgrp on Win

recover installer changes

Notate admin privleges

Windows phase install hooks

Find external python and install ninja (#23496)

Allow external find python to find windows python and spack install ninja

Co-authored-by: Adam J. Stewart <ajstewart426@gmail.com>
Co-authored-by: Betsy McPhail <betsy.mcphail@kitware.com>
2022-03-17 09:01:01 -07:00
John Parent
3a994032f8 Spack on Windows package ports
CMake - Windows Bootstrap (#25825)

Remove hardcoded cmake compiler (#26410)

Revert breaking cmake changes
Ensure no autotools on Windows

Perl on Windows (#26612)

Python source build windows (#26313)

Reconfigure sysconf for Windows

Python2.6 compatibility

Fxixup new sbang tests for windows

Ruby support (#28287)

Add NASM support (#28319)

Add mock Ninja package for testing
2022-03-17 09:01:01 -07:00
John W. Parent
e65d3d14b4 Relocate spack_cmd and scripts from installer to root bin (#24651)
Relocate spack_cmd and scripts from installer
to root bin

Refactor documentation, installer, and launcher
to facilitate that change
2022-03-17 09:01:01 -07:00
Jared Popelar
15ef85e161 Packaging of netlib-lapack for windows (#24993)
MSVC's internal CMake and Ninja now detected by spack external find and added to packages.yaml

Saving progress on packaging zlib for Windows

Fixing the shared CMake flag

* Loading Intel's ifx Fortran compiler into MSVC; if there are multiple
versions of MSVC installed and detected, ifx will only be placed into
the first block written in compilers.yaml. The version number of ifx can
be detected using MSVC's version flag (instead of /QV) by using
ignore_version_errors. This commit also provides support for detection
of Intel compilers in their own compiler block by adding ifx.exe to the
fc/f77_name blocks inside intel.py

* Giving CMake a Fortran compiler argument

* Adding patch file for removing duplicated mangling header for versions 3.9.1 and older; static and shared now successfully building on Windows

* Have netlib-lapack depend  on ninja@1.10

Co-authored-by: John R. Cary <cary@txcorp.com>
Co-authored-by: Jared Popelar <jpopelar@txcorp.com>

Making a default config.yaml for Windows

Small path length for build_stage

Provide more prerequisite details, mention default config.yaml

Killing an unnecessary setvars call

Replacing some lost changes, proofreading, updating windows-supported package list

Co-authored-by: John Parent <john.parent@kitware.com>
2022-03-17 09:01:01 -07:00
lou.lawrence@kitware.com
012758c179 Windows: Create installer and environment
* Add 'make-installer' command for Windows

* Add '--bat' arg to env activate, env deactivate and unload commands

* An equivalent script to setup-env on linux: spack_cmd.bat. This script
has a wrapper to evaluate cd, load/unload, env activate/deactivate.(#21734)

* Add spacktivate and config editor (#22049)

* spack_cmd: will find python and spack on its own. It preferentially
tries to use python on your PATH (#22414)

* Ignore Windows python installer if found (#23134)

* Bundle git in windows installer (#23597)

* Add Windows section to Getting Started document
(#23131), (#23295), (#24240)

Co-authored-by: Stephen Crowell <stephen.crowell@kitware.com>
Co-authored-by: lou.lawrence@kitware.com <lou.lawrence@kitware.com>
Co-authored-by: Betsy McPhail <betsy.mcphail@kitware.com>
Co-authored-by: Jared Popelar <jpopelar@txcorp.com>
Co-authored-by: Ben Cowan <benc@txcorp.com>

Update Installer CI

Co-authored-by: John Parent <john.parent@kitware.com>
2022-03-17 09:01:01 -07:00
Harmen Stoppels
dc78f4c58a environment.py: allow link:run (#29336)
* environment.py: allow link:run

Some users want minimal views, excluding run-type dependencies, since
those type of dependencies are covered by rpaths and the symlinked
libraries in the view aren't used anyways.

With this change, an environment like this:

```
spack:
  specs: ['py-flake8']
  view:
    default:
      root: view
      link: run
```

includes python packages and python, but no link type deps of python.
2022-03-09 12:35:26 -08:00
Tom Scogland
8f5fcc6e95 extensions: allow multiple "extends" directives (#28853)
* extensions: allow multiple "extends" directives

This will allow multiple extends directives in a package as long as only one of
them is selected as a dependency in the concrete spec.

* document the option to have multiple extends
2022-02-16 21:23:12 +00:00
Todd Gamblin
d33973df6c docs: add section on concretizer configuration
* Document `concretizer.yaml`, `--reuse`, and `--fresh`.
2022-02-16 10:17:18 -08:00
Mark W. Krentel
87a3b72ef0 Add 'stable' to the list of infinity version names. (#28772)
* Add 'stable' to the list of infinity version names.
Rename libunwind 1.5-head to 1.5-stable.

* Add stable to the infinite version list in packaging_guide.rst.
2022-02-16 09:08:51 -08:00
Jordan Galby
37ae4c0fdb Support config variables in config.yaml extensions paths (#17772) 2022-02-07 11:40:52 +01:00
Massimiliano Culpo
cd04109e17 Add a "sticky" property to variants (#28630)
* Add sticky variants

* Add unit tests for sticky variants

* Add documentation for sticky variants

* Revert "Revert 19736 because conflicts are avoided by clingo by default (#26721)"

This reverts commit 33ef7d57c1.

* Add stickiness to "allow-unsupported-compiler"
2022-02-02 10:05:24 -08:00
Harmen Stoppels
e72f87ec64 Switch lmod default all:autoload from none to direct (#28357)
* Switch lmod module all autoload default from none to direct

* Fix the docs
2022-01-18 09:06:41 -08:00
Todd Gamblin
93377942d1 Update copyright year to 2022 2022-01-14 22:50:21 -08:00
Adam J. Stewart
3540f8200a PythonPackage: install packages with pip (#27798)
* Use pip to bootstrap pip

* Bootstrap wheel from source

* Update PythonPackage to install using pip

* Update several packages

* Add wheel as base class dep

* Build phase no longer exists

* Add py-poetry package, fix py-flit-core bootstrapping

* Fix isort build

* Clean up many more packages

* Remove unused import

* Fix unit tests

* Don't directly run setup.py

* Typo fix

* Remove unused imports

* Fix issues caught by CI

* Remove custom setup.py file handling

* Use PythonPackage for installing wheels

* Remove custom phases in PythonPackages

* Remove <phase>_args methods

* Remove unused import

* Fix various packages

* Try to test Python packages directly in CI

* Actually run the pipeline

* Fix more packages

* Fix mappings, fix packages

* Fix dep version

* Work around bug in concretizer

* Various concretization fixes

* Fix gitlab yaml, packages

* Fix typo in gitlab yaml

* Skip more packages that fail to concretize

* Fix? jupyter ecosystem concretization issues

* Solve Jupyter concretization issues

* Prevent duplicate entries in PYTHONPATH

* Skip fenics-dolfinx

* Build fewer Python packages

* Fix missing npm dep

* Specify image

* More package fixes

* Add backends for every from-source package

* Fix version arg

* Remove GitLab CI stuff, add py-installer package

* Remove test deps, re-add install_options

* Function declaration syntax fix

* More build fixes

* Update spack create template

* Update PythonPackage documentation

* Fix documentation build

* Fix unit tests

* Remove pip flag added only in newer pip

* flux: add explicit dependency on jsonschema

* Update packages that have been added since this was branched off of develop

* Move Python 2 deprecation to a separate PR

* py-neurolab: add build dep on py-setuptools

* Use wheels for pip/wheel

* Allow use of pre-installed pip for external Python

* pip -> python -m pip

* Use python -m pip for all packages

* Fix py-wrapt

* Add both platlib and purelib to PYTHONPATH

* py-pyyaml: setuptools is needed for all versions

* py-pyyaml: link flags aren't needed

* Appease spack audit packages

* Some build backend is required for all versions, distutils -> setuptools

* Correctly handle different setup.py filename

* Use wheels for py-tomli to avoid circular dep on py-flit-core

* Fix busco installation procedure

* Clarify things in spack create template

* Test other Python build backends

* Undo changes to busco

* Various fixes

* Don't test other backends
2022-01-14 12:37:57 -06:00
Harmen Stoppels
d74396ad21 Do not initialize config on spack compiler list (#28042)
When `spack compiler list` is run without being restricted to a
particular scope, and no compilers are found, say that none are 
available, and hint that the use should run spack compiler find to 
auto detect compilers.

* Improve docs
* Check if stdin is a tty
* add a test
2022-01-12 16:26:28 +00:00
Vanessasaurus
a94b4eef79 Fixing spacing of libabigail to : (#28085) 2021-12-20 03:56:03 -07:00
Vanessasaurus
da9e152ed1 Fix bugs in spack monitor (#27511)
Updates to installer.py did not account for spack monitor, so as currently implemented
there are three cases of failure that spack monitor will not account for. To fix this we add additional
hooks, including an on cancel and also do a custom action on concretization fail.

Signed-off-by: vsoch <vsoch@users.noreply.github.com>

Co-authored-by: vsoch <vsoch@users.noreply.github.com>
2021-12-20 06:54:41 +01:00
Harmen Stoppels
6357de4e61 Fix table formatting (#28037) 2021-12-16 12:13:12 +00:00
Christian Goll
bd0ffa8a3c Added opensuse/leap:15 to spack containerize (#27837)
Co-authored-by: Massimiliano Culpo <massimiliano.culpo@gmail.com>
2021-12-16 09:47:15 +00:00
Massimiliano Culpo
f81d84dfc6 Release procedure: add a step to update docs (#27734) 2021-12-07 11:30:14 +00:00
Maxim Belkin
6e095a9741 module_file_support: update format for configuration (#27598) 2021-11-25 08:41:32 +01:00
Massimiliano Culpo
fa7189b480 Remove support for Python 2.6 (#27256)
Modifications:
- [x] Removed `centos:6` unit test, adjusted vermin checks
- [x] Removed backport of `collections.OrderedDict`
- [x] Removed backport of `functools.total_ordering`
- [x] Removed Python 2.6 specific skip markers in unit tests
- [x] Fixed a few minor Python 2.6 related TODOs in code

Updating the vendored dependencies will be done in separate PRs
2021-11-23 09:06:17 -08:00
Harmen Stoppels
0024e5cc9b Make _enable_or_disable(...) return an empty array for conditional variants whose condition is not met (#27504) 2021-11-22 10:47:09 +01:00
Harmen Stoppels
c5aee4d9b4 define_from_variant: return an empty string for non-existing variants (#27503)
This permits to use conditional variants without a lot of boilerplate.
2021-11-19 14:10:00 +01:00
Massimiliano Culpo
f981682bdc Allow recent pytest versions to be used with Spack (#25371)
Currently Spack vendors `pytest` at a version which is three major 
versions behind the latest (3.2.5 vs. 6.2.4). We do that since v3.2.5 
is the latest version supporting Python 2.6. Remaining so much 
behind the currently supported versions though might introduce 
some incompatibilities and is surely a technical debt.

This PR modifies Spack to:
- Use the vendored `pytest@3.2.5` only as a fallback solution, 
  if the Python interpreter used for Spack doesn't provide a newer one
- Be able to parse `pytest --collect-only` in all the different output 
  formats from v3.2.5 to v6.2.4 and use it consistently for `spack unit-test --list-*`
- Updating the unit tests in Github Actions to use a more recent `pytest` version
2021-11-18 15:08:59 +01:00
Seth R. Johnson
a04cc4470e Add PyPI docs and warning in auto-generated package (#27404)
* docs: Add cross-references for pypi setup

* create: add warning for missing pypi
2021-11-12 10:58:44 -06:00
Harmen Stoppels
8bb5ed8464 make version docs reflect reality (#27149)
* make version docs reflect reality

* typo and make things

* 2.6 -> 2.7 in example
2021-11-05 09:39:31 +00:00
Todd Gamblin
e13e697067 commands: spack load --list alias for spack find --loaded (#27184)
See #25249 and https://github.com/spack/spack/pull/27159#issuecomment-958163679.
This adds `spack load --list` as an alias for `spack find --loaded`.  The new command is
not as powerful as `spack find --loaded`, as you can't combine it with all the queries or
formats that `spack find` provides.  However, it is more intuitively located in the command
structure in that it appears in the output of `spack load --help`.

The idea here is that people can use `spack load --list`  for simple stuff but fall back to
`spack find --loaded` if they need more.

- add help to `spack load --list` that references `spack find`
- factor some parts of `spack find` out to be called from `spack load`
- add shell tests
- update docs

Co-authored-by: Peter Josef Scheibel <scheibel1@llnl.gov>
Co-authored-by: Richarda Butler <39577672+RikkiButler20@users.noreply.github.com>
2021-11-05 00:58:29 -07:00
Todd Gamblin
8e76244266 docs for experimental --reuse argument to spack install
Add docs for `--reuse`, along with a warning that it will likely be
removed and refactored.
2021-11-05 00:15:47 -07:00
Greg Becker
67cd92e6a3 Allow conditional variants (#24858)
A common question from users has been how to model variants 
that are new in new versions of a package, or variants that are 
dependent on other variants. Our stock answer so far has been
an unsatisfying combination of "just have it do nothing in the old
version" and "tell Spack it conflicts".

This PR enables conditional variants, on any spec condition. The 
syntax is straightforward, and matches that of previous features.
2021-11-03 08:11:31 +01:00