Commit Graph

11582 Commits

Author SHA1 Message Date
Massimiliano Culpo
d5ab52b95a
Enclose ThreadPool.map call in a try/finally block 2019-06-07 09:30:15 +02:00
Massimiliano Culpo
2d5aadc21d
Simplified the implementation of 'search_paths_for_executables'
The function doesn't use anymore 'map', 'filter' and 'os.path.realpath'
 + it's based on a single loop.
2019-06-07 09:30:15 +02:00
Massimiliano Culpo
f2824d64d9
OS can optionally provide search paths for compiler detection
This permits to deal more neatly with peculiarities of Cray systems when
detecting compilers.
2019-06-07 09:30:15 +02:00
Massimiliano Culpo
a75fc3c6dd
Skip the 'release_jobs' test in Python 2.6 2019-06-07 09:30:15 +02:00
Massimiliano Culpo
d010cdb373
Fixed namespace issue occurred during rebase 2019-06-07 09:30:15 +02:00
Massimiliano Culpo
f05d5b82f1
Added test for the conversion of an OperatingSystem instance to dict 2019-06-07 09:30:15 +02:00
Massimiliano Culpo
2c2d0e2c44
Added back unit tests to detect compiler versions 2019-06-07 09:30:15 +02:00
Massimiliano Culpo
8edf6b7c81
Added compiler detection for CNL 2019-06-07 09:30:15 +02:00
Massimiliano Culpo
36cb494c8c
Added a few points of customization during compiler detection 2019-06-07 09:30:15 +02:00
Massimiliano Culpo
54313bbcc6
Added compiler detection for Cray front-end 2019-06-07 09:30:15 +02:00
Massimiliano Culpo
fdeb9e43fa
Cleaned code that performs compiler detection
* Simplified _CompilerID

* Extracted search_compiler_commands from Compiler

* Added search_regexps to Compiler

* A few functions manipulating paths that could be useful in other
  parts of the code have been moved to llnl.util.filesystem

* Removed deferred functions in favor of mapping arguments to
  functions as required in the review

* Moved most of the code involved in compiler detection in
  spack.compilers
2019-06-07 09:30:15 +02:00
Massimiliano Culpo
477ce206c2
Inlined llnl.util.multiproc.execute (abstraction was not very useful) 2019-06-07 09:30:15 +02:00
Massimiliano Culpo
675f944a3a
Renamed 'deferred' to 'defer' + don't use it as a decorator 2019-06-07 09:30:15 +02:00
Massimiliano Culpo
dce6851d37
Renamed 'is_accessible_dir' to 'can_access_dir'
This is to be consistent with the 'can_access' function already present
in the module.
2019-06-07 09:30:15 +02:00
Massimiliano Culpo
c2a8b0c373
Revert "tests: run 'test_compiler_add' serially in Python 2 (#10127)"
This reverts commit 28fd1baa86.
2019-06-07 09:30:15 +02:00
Massimiliano Culpo
b7619a33c1
Don't print debug info for python 2.6 2019-06-07 09:30:15 +02:00
Massimiliano Culpo
16f402a6c0
Handle unicode correctly when detecting compiler version 2019-06-07 09:30:15 +02:00
Massimiliano Culpo
a061c4a7ad
flake8 fix 2019-06-07 09:30:15 +02:00
Massimiliano Culpo
f8bbfe683c
ThreadPool doesn't implement the context manager protocol in Python 2 2019-06-07 09:30:15 +02:00
Massimiliano Culpo
d89aa69f96
Optimized filesystem access for spack compiler add 2019-06-07 09:30:15 +02:00
Massimiliano Culpo
56131f445f
Reduced the amount of data sent to workers
Compiler detection has been reorganized to send to workers only
the function call that detects the compiler's version, without further
information.
2019-06-07 09:30:15 +02:00
Massimiliano Culpo
5ccacb8733
CompilerKey has been split into _CompilerID + _NameVariation + language
This reflects better the use of information in the code that performs
compiler detection. Each 'compiler id' (os + compiler type + version)
value is a possibly unique compiler that gets detected. When searching
for a 'compiler id' we might find many different 'name variations'.
In the end we select the first one where most languages are supported.
2019-06-07 09:30:15 +02:00
Massimiliano Culpo
5330509bfe
find_compilers has been parallelized using ThreadPool
There should be no issues with the GIL, as the functions that are
invoked in parallel spawn a subprocess to start a system call and
then wait for I/O.
2019-06-07 09:30:15 +02:00
Massimiliano Culpo
db6ef96e87
Finding compilers executes all queries first and then process them
The function calls to find compilers have been reorganized to:
1. Collect all the queries that are needed to detect compiler's version
2. Execute them on demand
3. Process the results and register compilers
2019-06-07 09:30:15 +02:00
Massimiliano Culpo
74a13e665f
Reworked _find_matches_in_path to return lazy commands
These commands, when invoked, give back a tuple (CompilerKey,path) or
None if the compiler was not found. CompilerKey is a namedtuple
containing all the information needed to identify a compiler.

find_compiler has been changed accordingly.
2019-06-07 09:30:15 +02:00
Massimiliano Culpo
cae7e075a6
Refactored _find_matches_in_path
Now the function takes a compiler language and computes the compiler
names and detection function.

Skipping values while iterating has been changed to filter input values
+ iterate without jumps.
2019-06-07 09:30:15 +02:00
Massimiliano Culpo
0a8f4ad42e
Extracted a function transforming path hints to search paths
Given a list of possible paths it returns a list of the existing real
paths and of their 'bin' subdirectories, if they exist.
2019-06-07 09:30:15 +02:00
Todd Gamblin
06cc799fd3
commands: remove unused spack list --format=rst (#11651)
- We use `spack list --foramt=html` now, as it is much faster and doesn't
  make the docs build take forever.

- Remove `spack list --format=rst` as it is no longer used.
2019-06-06 18:26:43 -07:00
Gregory Lee
7c8e9ab406 added tix package and python variant for tix (#11599)
* added tix package and python variant for tix

* addressing tix review requests

* python +tix requires +tkinter
2019-06-06 17:19:24 -04:00
Todd Gamblin
0c13c3f91f
bugfix: StageComposite must include the expanded property` (#11647) 2019-06-06 14:09:19 -07:00
Todd Gamblin
89b891d264
bugfix: lua-luafilesystem package should use stage.source_path (#11648) 2019-06-06 14:07:46 -07:00
Jennifer Herting
541578a456 tests: package_sanity now checks packages in 3rd-party repos (#11637) 2019-06-06 09:18:43 -07:00
Todd Gamblin
3bd854f991 travis: disable spack spec for python 2.7 as well as 2.6
- keep this disabled until the `compiler find` fork bomb is fixed.
2019-06-05 22:41:28 -07:00
Tamara Dahlgren
b76fc827ec tests: add extra coverage for fetch strategy tests 2019-06-05 22:41:28 -07:00
Tamara Dahlgren
1842873f85 stage: make source_path available before stage is built
- `stage.source_path` was previously overloaded; it returned `None` if it
  didn't exist and this was used by client code
  - we want to be able to know the `source_path` before it's created

- make stage.source_path available before it exists.
  - use a well-known stage source path name, `$stage_path/src` that is
    available when `Stage` is instantiated but does not exist until it's
    "expanded"
  - client code can now use the variable before the stage is created.
  - client code can test whether the tarball is expanded by using the new
    `stage.expanded` property instead of testing whether `source_path` is
    `None`

- add tests for the new source_path semantics
2019-06-05 22:41:28 -07:00
Tamara Dahlgren
eb584d895b refactor: remove unused spack.stage._get_mirrors() function 2019-06-05 22:41:28 -07:00
Tamara Dahlgren
8e3fd3f7c2 tty: make tty.* print exception types
- make tty.msg, tty.info, etc. print the exception type and stringified
  message if the message argument is an exception.

- simplify parts of the code that call tty.debug(str(e))

- add extra tty.debug statements in places where exceptions were
  previously ignored
2019-06-05 22:41:28 -07:00
Justin S
8c173da4b7 nettle: fix broken build (#11611)
* nettle: force C99 build

* nettle: update to use c99_flag

from @alalazo

Co-Authored-By: Massimiliano Culpo <massimiliano.culpo@gmail.com>
2019-06-05 19:09:01 -05:00
Christoph Junghans
8fd64dfa08 lammps: added v20190605 (#11638) 2019-06-05 19:07:33 -05:00
Justin S
f4e7786786 clang: fix error messages in c11_flag, cxx17_flag (#11632) 2019-06-05 23:51:38 +02:00
Patrick Gartung
3c0b746f44
libxpm: revert change made in unrelated PR (#11634) 2019-06-05 15:13:19 -05:00
Gregory Lee
cfcf3996d8
set GUILE env var when building gnutls, fixes issue #11601 (#11607) 2019-06-05 10:14:06 -07:00
Todd Gamblin
dc8af3023e graph: refactor static graphs
- `spack graph --static` (and `spack.graph.dot_graph`) now do the "right
  thing" and print the possible dependency graph of provided packages.

- `spack graph --static` no longer concretizes specs, as it only relies
  on class level metadata

- Previously the behavior was not consistent -- `spack graph --static`
  would graph possible dependencies of concrete specs, but would only
  include some of them.  The new code properly pursues all possible
  dependencies, and allows traversing by different dependency types.
2019-06-05 06:11:18 -07:00
Todd Gamblin
2e22fc1090 spack dependencies: support --deptype argument
- `spack dependencies` can now take a --deptype argument to only traverse
  particular deptypes

- add a new "common" argument for deptype in spack.cmd.common.arguments

- Database.installed_relatives() can now also take a deptype argument
  - this is used by `spack dependencies --installed`
2019-06-05 06:11:18 -07:00
Todd Gamblin
3dac78fc19 package: make possible_dependencies consider deptypes
- `PackageBase.possible_dependencies` now:
  - accepts a deptype param that controls dependency types traversed
  - returns a dict mapping possible depnames to their immediate possible
    dependencies (this lets you build a graph easily)

- Add tests for PackageBaes
2019-06-05 06:11:18 -07:00
Todd Gamblin
087a511da7 refactor: remove vestigial ALLOWED_URL_SCHEMES from package.py 2019-06-05 06:11:18 -07:00
Todd Gamblin
87e6cb9f72 refactor: make Package.name consistent with other class attributes
- The 'name' attribute for packages was being set in DirectiveMeta, which
  wasn't consistent with other class properties (like fullname, etc.)

- Move it to be a class property of `PackageMeta`, and add the
  corresponding property method wrapper on `PackageBase`
2019-06-05 06:11:18 -07:00
Todd Gamblin
3f5141d629 refactor: make PackageBase.possible_dependencies() a class method. 2019-06-05 06:11:18 -07:00
Justin S
4ac64e6cd8 add C standard flags to compiler classes (#11618)
* add c99_flag, c11_flag to compiler class

* implement c99_flag, c11_flag for gcc

* implement c99_flag, c11_flag for arm

* implement c99_flag for cce

* implement c99_flag, c11_flag for clang

* implement c99_flag, c11_flag for intel

* implement c99_flag, c11_flag for xl
2019-06-05 11:43:11 +02:00
Andrew Gaspar
b3379a8890 Add Google Benchmark version 1.5.0 (#11626) 2019-06-04 22:53:07 -05:00