Commit Graph

135 Commits

Author SHA1 Message Date
Massimiliano Culpo
ed582cef68 New interface for passing build information among specs (#1875)
- Added a new interface for Specs to pass build information
  - Calls forwarded from Spec to Package are now explicit
  - Added descriptor within Spec to manage forwarding
  - Added state in Spec to maintain query information
  - Modified a few packages (the one involved in spack install pexsi) to showcase changes

- This uses an object wrapper to `spec` to implement the `libs` sub-calls.
  - wrapper is returned from `__getitem__` only if spec is concrete
  - allows packagers to access build information easily
2017-03-02 10:01:29 -08:00
Adam J. Stewart
e492aff4f7 More consistent yes/no prompts (#3174)
* More consistent yes/no prompts
* Add ==> prefix to yes/no and number prompts
2017-02-17 13:45:02 -08:00
Erik Schnetter
cade0181fd Spack: Correct fix_darwin_install_name (#2886)
Previously, fix_darwin_install_name would only handle dependencies that have no path set, and it ignore dependencies that have the build directory as path baked in. Catch this, and replace it by the install directory.
2017-01-20 10:22:59 -08:00
Massimiliano Culpo
7ea10e768e unit tests: replace nose with pytest (#2502)
* Porting: substitute nose with ytest

This huge commit substitutes nose with pytest as a testing system. Things done here:

* deleted external/nose as it is no longer used
* moved mock resources in their own directory 'test/mock/'
* ported two tests (cmd/find, build_system) to pytest native syntax as an example
* build_environment, log: used monkeypatch instead of try/catch
* moved global mocking of fetch_cache to an auto-used fixture
* moved global mocking from test/__init__.py to conftest.py
* made `spack test` a wrapper around pytest
* run-unit-tests: avoid running python 2.6 tests under coverage to speed them up
* use `pytest --cov` instead of coverage run to cut down testing time

* mock/packages_test: moved mock yaml configuration to files instead of leaving it in the code as string literals

* concretize.py: ported tests to native pytest, reverted multiprocessing in pytest.ini as it was creating the wrong report for coveralls

* conftest.py, fixtures: added docstrings

* concretize_preferences.py: uses fixtures instead of subclassing MockPackagesTest

* directory_layout.py: uses fixtures instead of subclassing MockPackagesTest

* install.py: uses fixtures instead of subclassing MockPackagesTest

* optional_deps.py: uses fixtures instead of subclassing MockPackagesTest

optional_deps.py: uses fixtures instead of subclassing MockPackagesTest

* packages.py: uses fixtures instead of subclassing MockPackagesTest

* provider_index.py: uses fixtures instead of subclassing MockPackagesTest

* spec_yaml.py: uses fixtures instead of subclassing MockPackagesTest

* multimethod.py: uses fixtures instead of subclassing MockPackagesTest

* install.py: now uses mock_archive_url

* git_fetch.py: uses fixtures instead of subclassing MockPackagesTest

* hg_fetch.py: uses fixtures instead of subclassing MockPackagesTest

* svn_fetch.py, mirror.py: uses fixtures instead of subclassing MockPackagesTest
repo.py: deleted

* test_compiler_cmd.py: uses fixtures instead of subclassing MockPackagesTest

* cmd/module.py, cmd/uninstall.py: uses fixtures instead of subclassing MockDatabase

* database.py: uses fixtures instead of subclassing MockDatabase, removed mock/database

* pytest: uncluttering fixture implementations

* database: changing the scope to 'module'

* config.py: uses fixtures instead of subclassing MockPackagesTest

* spec_dag.py, spec_semantics.py: uses fixtures instead of subclassing MockPackagesTest

* stage.py: uses fixtures instead of subclassing MockPackagesTest. Removed mock directory

* pytest: added docstrings to all the fixtures

* pytest: final cleanup

* build_system_guess.py: fixed naming and docstrings as suggested by @scheibelp

* spec_syntax.py: added expected failure on parsing multiple specs closes #1976

* Add pytest and pytest-cov to Spack externals.

* Make `spack flake8` ignore externals.

* run-unit-tests runs spack test and not pytest.

* Remove all the special stuff for `spack test`

- Remove `conftest.py` magic and all the special case stuff in `bin/spack`

- Spack commands can optionally take unknown arguments, if they want to
  handle them.

- `spack test` is now a command like the others.

- `spack test` now just delegates its arguments to `pytest`, but it does
  it by receiving unknown arguments and NOT taking an explicit
  help argument.

* Fix error in fixtures.

* Improve `spack test` command a bit.

- Now supports an approximation of the old simple interface
- Also supports full pytest options if you want them.

* Use external coverage instead of pytest-cov

* Make coverage use parallel-mode.

* change __init__.py docs to include pytest
2016-12-29 07:48:48 -08:00
Massimiliano Culpo
17b13b161b Directive inheritance: laziness for the win (#2623)
* inheritance of directives: using meta-classes to inject attributes coming from directives into packages + lazy directives

* _dep_types -> dependency_types
* using a meta-class to inject directives into packages
* directives are lazy

fixes #2466

* directives.py: allows for multiple inheritance. Added blank lines as suggested by @tgamblin

* directives.py: added a test for simple inheritance of directives

* Minor improvement requested by @tgamblin

CMakePackage: importing names from spack.directives
directives: wrap __new__ to respect pep8

* Refactoring requested by @tgamblin

directives: removed global variables in favor of class variables. Simplified the interface for directives (they return a callable on a package or a list of them).
2016-12-28 12:37:02 -08:00
Matthew Krafczyk
f1b26cb72e Improve stacktrace printing
Sometimes files in the stacktrace are not from spack. Remove these
files before finding the spack root.
2016-11-17 12:51:41 -05:00
Matthew Krafczyk
de7171db8a Use better scheme for getting root directory 2016-11-16 12:14:25 -05:00
Matthew Krafczyk
6c854246aa Fix some formatting errors caught by flake8. 2016-11-16 11:58:36 -05:00
Matthew Krafczyk
8e6d890a19 -s now includes the file and line number with info
The option -s now causes file and line number information to be printed
along with any invocation of msg, info, etc...

This will greatly ease debugging.
2016-11-16 11:58:36 -05:00
Massimiliano Culpo
36a4ca8b11 spack install: forward sys.stdin to child processes (#2158)
* spack install: forward sys.stdin to child processes fixes #2140

 - [ ] redirection process is spawned in __enter__ instead of __init__
 - [ ] sys.stdin is forwarded to child processes

* log: wrapped __init__ definition
2016-11-03 08:03:10 -07:00
Todd Gamblin
0da639298c Move temp directory configuration to config.yaml
- Moved temp finding logic to spack.stage
- Updated stage tests
- Added tests for new path substaitution of $user, $spack, $tempdir
2016-10-30 23:55:00 -07:00
alalazo
ab995df777 Merge branch 'develop' of https://github.com/LLNL/spack into features/install_with_phases_rebase
Conflicts:
	lib/spack/spack/build_environment.py
	lib/spack/spack/cmd/install.py
	lib/spack/spack/cmd/setup.py
	lib/spack/spack/package.py
	var/spack/repos/builtin/packages/gmp/package.py
	var/spack/repos/builtin/packages/hdf5/package.py
2016-10-11 16:28:08 +02:00
Todd Gamblin
3d8d8d3644 Fix bug with lock upgrades.
- Closing and re-opening to upgrade to write will lose all existing read
  locks on this process.
  - If we didn't allow ranges, sleeping until no reads would work.
  - With ranges, we may never be able to take some legal write locks
    without invalidating all reads. e.g., if a write lock has distinct
    range from all reads, it should just work, but we'd have to close the
    file, reopen, and re-take reads.

- It's easier to just check whether the file is writable in the first
  place and open for writing from the start.

- Lock now only opens files read-only if we *can't* write them.
2016-10-11 01:55:33 -07:00
Todd Gamblin
da6bbfb2d4 Add byte-range parameters to llnl.util.lock 2016-10-11 01:55:32 -07:00
Todd Gamblin
ea10e3bab0 Remove need to touch lock files before using.
- Locks will now create enclosing directories and touch the lock file
  automatically.
2016-10-11 01:55:32 -07:00
Todd Gamblin
907fe912ef Make llnl.util.lock use file objects instead of low-level OS fds.
- Make sure we write, truncate, flush when setting PID and owning host in
  the file.
2016-10-11 01:55:32 -07:00
alalazo
f229290880 stage : try to remove dead links only of folder that you actually care about
A use case where the previous approach was failing is :

 - more than one spack process running on compute nodes
 - stage directory is a link to fast LOCAL storage

 In this case the processes may try to unlink something that is "dead" for them, but actually used by other processes on storage they cannot see.
2016-10-11 01:55:32 -07:00
alalazo
34fe51a4aa install : finer graned locking for install command 2016-10-11 01:38:27 -07:00
alalazo
7a26c60dbd Merge branch 'develop' of https://github.com/LLNL/spack into features/install_with_phases_rebase
Conflicts:
	lib/spack/spack/cmd/setup.py
	lib/spack/spack/package.py
	var/spack/repos/builtin/packages/gmp/package.py
2016-10-05 09:33:59 +02:00
Todd Gamblin
bff1656a1a Read-only locks should close fd before opening for write. (#1906)
- Fixes bad file descriptor error in lock acquire, #1904
- Fix bug introduced in previous PR #1857
- Backported fix from soon-to-be merged fine-grained DB locking branch.
2016-10-04 15:36:37 -07:00
Michael Kuhn
8d1ec0df3d Fix read locks on read-only file systems (#1857) 2016-09-30 09:45:08 -07:00
Massimiliano Culpo
d848559f70 Reworking of lapack_shared_libs and similar properties (#1682)
* Turned <provider>_libs into an iterable

Modifications :
- added class LibraryList + unit tests
- added convenience functions `find_libraries` and `dedupe`
- modifed non Intel blas/lapack providers
- modified packages using blas_shared_libs and similar functions

* atlas : added pthread variant

* intel packages : added lapack_libs and blas_libs

* find_library_path : removed unused function

* PR review : fixed last issues

* LibraryList : added test on __add__ return type

* LibraryList : added __radd__ fixed unit tests

fix : failing unit tests due to missing `self`

* cp2k and dependecies : fixed blas-lapack related statements in package.py
2016-09-21 12:27:59 -07:00
alalazo
f5433477b9 qa : flake8 issues 2016-08-11 09:08:00 +02:00
alalazo
b4b9ebe7d7 Merge branch 'develop' of https://github.com/LLNL/spack into features/install_with_phases
Conflicts:
	lib/spack/llnl/util/tty/log.py
	lib/spack/spack/__init__.py
	lib/spack/spack/cmd/install.py
	lib/spack/spack/cmd/setup.py
	lib/spack/spack/package.py
	var/spack/repos/builtin/packages/blitz/package.py
	var/spack/repos/builtin/packages/gmp/package.py
	var/spack/repos/builtin/packages/qhull/package.py
	var/spack/repos/builtin/packages/szip/package.py
2016-08-11 08:55:20 +02:00
Todd Gamblin
bf1072c902 Make Spack core PEP8 compliant. 2016-08-10 16:33:37 -07:00
Todd Gamblin
9d4a36a62f Properly re-raise exceptions from lock context handler. 2016-08-09 02:25:09 -07:00
Todd Gamblin
0c75c13cc0 Flake8 fixes 2016-08-09 02:25:07 -07:00
Todd Gamblin
102ac7bcf1 Move provider cache to home directory and refactor Transactions
Major stuff:

- Created a FileCache for managing user cache files in Spack.  Currently just
  handles virtuals.

- Moved virtual cache from the repository to the home directory so that users do
  not need write access to Spack repositories to use them.

- Refactored `Transaction` class in `database.py` -- moved it to
  `LockTransaction` in `lock.py` and made it reusable by other classes.

Other additions:

- Added tests for file cache and transactions.

- Added a few more tests for database

- Fixed bug in DB where writes could happen even if exceptions were raised
  during a transaction.

- `spack uninstall` now attempts to repair the database when it discovers that a
  prefix doesn't exist but a DB record does.
2016-08-09 00:24:54 -07:00
Adam J. Stewart
5d48c108a3 Only strip newline chars, not spaces 2016-08-08 15:02:05 -05:00
becker33
14d861a41c Merge pull request #1414 from adamjstewart/fixes/filter_file
Fix backup=True for filter_file
2016-08-01 14:10:59 -07:00
Adam J. Stewart
5b79f0d04a Fix backup=True for filter_file 2016-08-01 15:35:02 -05:00
George Hartzell
d684b17c06 Fix doc for install_tree (cut/paste error)
It looks like the docs for copy_tree were cut/paste from copy and still referred to installing a "file".

This fixes that.
2016-07-25 08:45:44 -07:00
alalazo
40cb314638 Merge branch 'develop' of https://github.com/LLNL/spack into features/install_with_phases
Conflicts:
	lib/spack/spack/package.py
2016-07-20 21:02:19 +02:00
Denis Davydov
9ea4f80f15 flake8 fixes 2016-07-16 07:58:51 +02:00
Denis Davydov
992250ddea add to_lib_name() to get library name from a path 2016-07-16 07:24:05 +02:00
alalazo
1ecea4c2f1 log : refactored acquire and release semantic to meet the context manager protocol 2016-07-14 09:34:01 +02:00
alalazo
90b131260b log : changed semantic for start / join (now it's explicit) 2016-07-13 22:50:39 +02:00
alalazo
ad16830f71 log : added timeout to avoid deadlocks on daemon join 2016-07-13 11:20:23 +02:00
alalazo
813cb032c4 package.py : updated logic to log.py rework
Conflicts:
	lib/spack/spack/package.py
2016-07-13 09:27:09 +02:00
alalazo
9af964a6d6 log_output : moved from os.fork to multiprocessing.Process 2016-07-13 09:27:09 +02:00
Adam J. Stewart
71a297f121 Fix backup logic in filter_file 2016-06-23 16:44:41 -05:00
Todd Gamblin
c09111bbef Merge pull request #1053 from davydden/pkg/dealii_suite_update
update Petsc, Slepc, Trilinos, Superlu-dist and deal.II
2016-06-20 10:32:52 -07:00
Todd Gamblin
4569df025e Merge pull request #1060 from mathstuf/no-break-long-words
tty: do not break long words
2016-06-17 14:51:26 -07:00
Todd Gamblin
17b868381f Fixes #460: Do not show variants by default in spack find.
This does two things:

1. By default `spack find` no longer shows variants.  You have to
   supply `-v` to get that

2. This improves the `colify` implementation so that it no longer pads
   the rightmost column.  This avoids the issue where if one spec was
   too long in the output, *all* specs would have space padding added
   to that width, and it would look like the output of `spack find`
   was double spaced.  This no longer happens -- the one bad line
   wraps around and the other lines are now single-spaced when you use
   `-v` with boost.
2016-06-16 02:55:33 -07:00
Denis Davydov
3d3a26cd9a flake8 fixes for filesystem.py 2016-06-15 18:31:10 +02:00
Ben Boeckel
36decbfa69 tty: do not break long words
These "words" tend to be file paths which are best left unbroken.

Fixes #1057.
2016-06-15 12:22:35 -04:00
Denis Davydov
a860a3d614 superlu-dist: avoid harcoding blas/lapack/mpi; remove preferred version 2016-06-15 18:21:13 +02:00
Todd Gamblin
e7ced54369 Correct LLNL LGPL license template for clarity. 2016-05-11 21:22:25 -07:00
Todd Gamblin
220c72d67f Merge branch 'features/blas-lapack-hardening' into develop 2016-03-31 11:37:09 -07:00
Todd Gamblin
8b45d6f84f Merge pull request #658 from davydden/metis_parmetis_darwin
metis/parmetis/oce/boost: correct install_name on Darwin via a global tool
2016-03-27 16:10:05 -07:00