Commit Graph

130 Commits

Author SHA1 Message Date
Matthew Krafczyk
f1b26cb72e Improve stacktrace printing
Sometimes files in the stacktrace are not from spack. Remove these
files before finding the spack root.
2016-11-17 12:51:41 -05:00
Matthew Krafczyk
de7171db8a Use better scheme for getting root directory 2016-11-16 12:14:25 -05:00
Matthew Krafczyk
6c854246aa Fix some formatting errors caught by flake8. 2016-11-16 11:58:36 -05:00
Matthew Krafczyk
8e6d890a19 -s now includes the file and line number with info
The option -s now causes file and line number information to be printed
along with any invocation of msg, info, etc...

This will greatly ease debugging.
2016-11-16 11:58:36 -05:00
Massimiliano Culpo
36a4ca8b11 spack install: forward sys.stdin to child processes (#2158)
* spack install: forward sys.stdin to child processes fixes #2140

 - [ ] redirection process is spawned in __enter__ instead of __init__
 - [ ] sys.stdin is forwarded to child processes

* log: wrapped __init__ definition
2016-11-03 08:03:10 -07:00
Todd Gamblin
0da639298c Move temp directory configuration to config.yaml
- Moved temp finding logic to spack.stage
- Updated stage tests
- Added tests for new path substaitution of $user, $spack, $tempdir
2016-10-30 23:55:00 -07:00
alalazo
ab995df777 Merge branch 'develop' of https://github.com/LLNL/spack into features/install_with_phases_rebase
Conflicts:
	lib/spack/spack/build_environment.py
	lib/spack/spack/cmd/install.py
	lib/spack/spack/cmd/setup.py
	lib/spack/spack/package.py
	var/spack/repos/builtin/packages/gmp/package.py
	var/spack/repos/builtin/packages/hdf5/package.py
2016-10-11 16:28:08 +02:00
Todd Gamblin
3d8d8d3644 Fix bug with lock upgrades.
- Closing and re-opening to upgrade to write will lose all existing read
  locks on this process.
  - If we didn't allow ranges, sleeping until no reads would work.
  - With ranges, we may never be able to take some legal write locks
    without invalidating all reads. e.g., if a write lock has distinct
    range from all reads, it should just work, but we'd have to close the
    file, reopen, and re-take reads.

- It's easier to just check whether the file is writable in the first
  place and open for writing from the start.

- Lock now only opens files read-only if we *can't* write them.
2016-10-11 01:55:33 -07:00
Todd Gamblin
da6bbfb2d4 Add byte-range parameters to llnl.util.lock 2016-10-11 01:55:32 -07:00
Todd Gamblin
ea10e3bab0 Remove need to touch lock files before using.
- Locks will now create enclosing directories and touch the lock file
  automatically.
2016-10-11 01:55:32 -07:00
Todd Gamblin
907fe912ef Make llnl.util.lock use file objects instead of low-level OS fds.
- Make sure we write, truncate, flush when setting PID and owning host in
  the file.
2016-10-11 01:55:32 -07:00
alalazo
f229290880 stage : try to remove dead links only of folder that you actually care about
A use case where the previous approach was failing is :

 - more than one spack process running on compute nodes
 - stage directory is a link to fast LOCAL storage

 In this case the processes may try to unlink something that is "dead" for them, but actually used by other processes on storage they cannot see.
2016-10-11 01:55:32 -07:00
alalazo
34fe51a4aa install : finer graned locking for install command 2016-10-11 01:38:27 -07:00
alalazo
7a26c60dbd Merge branch 'develop' of https://github.com/LLNL/spack into features/install_with_phases_rebase
Conflicts:
	lib/spack/spack/cmd/setup.py
	lib/spack/spack/package.py
	var/spack/repos/builtin/packages/gmp/package.py
2016-10-05 09:33:59 +02:00
Todd Gamblin
bff1656a1a Read-only locks should close fd before opening for write. (#1906)
- Fixes bad file descriptor error in lock acquire, #1904
- Fix bug introduced in previous PR #1857
- Backported fix from soon-to-be merged fine-grained DB locking branch.
2016-10-04 15:36:37 -07:00
Michael Kuhn
8d1ec0df3d Fix read locks on read-only file systems (#1857) 2016-09-30 09:45:08 -07:00
Massimiliano Culpo
d848559f70 Reworking of lapack_shared_libs and similar properties (#1682)
* Turned <provider>_libs into an iterable

Modifications :
- added class LibraryList + unit tests
- added convenience functions `find_libraries` and `dedupe`
- modifed non Intel blas/lapack providers
- modified packages using blas_shared_libs and similar functions

* atlas : added pthread variant

* intel packages : added lapack_libs and blas_libs

* find_library_path : removed unused function

* PR review : fixed last issues

* LibraryList : added test on __add__ return type

* LibraryList : added __radd__ fixed unit tests

fix : failing unit tests due to missing `self`

* cp2k and dependecies : fixed blas-lapack related statements in package.py
2016-09-21 12:27:59 -07:00
alalazo
f5433477b9 qa : flake8 issues 2016-08-11 09:08:00 +02:00
alalazo
b4b9ebe7d7 Merge branch 'develop' of https://github.com/LLNL/spack into features/install_with_phases
Conflicts:
	lib/spack/llnl/util/tty/log.py
	lib/spack/spack/__init__.py
	lib/spack/spack/cmd/install.py
	lib/spack/spack/cmd/setup.py
	lib/spack/spack/package.py
	var/spack/repos/builtin/packages/blitz/package.py
	var/spack/repos/builtin/packages/gmp/package.py
	var/spack/repos/builtin/packages/qhull/package.py
	var/spack/repos/builtin/packages/szip/package.py
2016-08-11 08:55:20 +02:00
Todd Gamblin
bf1072c902 Make Spack core PEP8 compliant. 2016-08-10 16:33:37 -07:00
Todd Gamblin
9d4a36a62f Properly re-raise exceptions from lock context handler. 2016-08-09 02:25:09 -07:00
Todd Gamblin
0c75c13cc0 Flake8 fixes 2016-08-09 02:25:07 -07:00
Todd Gamblin
102ac7bcf1 Move provider cache to home directory and refactor Transactions
Major stuff:

- Created a FileCache for managing user cache files in Spack.  Currently just
  handles virtuals.

- Moved virtual cache from the repository to the home directory so that users do
  not need write access to Spack repositories to use them.

- Refactored `Transaction` class in `database.py` -- moved it to
  `LockTransaction` in `lock.py` and made it reusable by other classes.

Other additions:

- Added tests for file cache and transactions.

- Added a few more tests for database

- Fixed bug in DB where writes could happen even if exceptions were raised
  during a transaction.

- `spack uninstall` now attempts to repair the database when it discovers that a
  prefix doesn't exist but a DB record does.
2016-08-09 00:24:54 -07:00
Adam J. Stewart
5d48c108a3 Only strip newline chars, not spaces 2016-08-08 15:02:05 -05:00
becker33
14d861a41c Merge pull request #1414 from adamjstewart/fixes/filter_file
Fix backup=True for filter_file
2016-08-01 14:10:59 -07:00
Adam J. Stewart
5b79f0d04a Fix backup=True for filter_file 2016-08-01 15:35:02 -05:00
George Hartzell
d684b17c06 Fix doc for install_tree (cut/paste error)
It looks like the docs for copy_tree were cut/paste from copy and still referred to installing a "file".

This fixes that.
2016-07-25 08:45:44 -07:00
alalazo
40cb314638 Merge branch 'develop' of https://github.com/LLNL/spack into features/install_with_phases
Conflicts:
	lib/spack/spack/package.py
2016-07-20 21:02:19 +02:00
Denis Davydov
9ea4f80f15 flake8 fixes 2016-07-16 07:58:51 +02:00
Denis Davydov
992250ddea add to_lib_name() to get library name from a path 2016-07-16 07:24:05 +02:00
alalazo
1ecea4c2f1 log : refactored acquire and release semantic to meet the context manager protocol 2016-07-14 09:34:01 +02:00
alalazo
90b131260b log : changed semantic for start / join (now it's explicit) 2016-07-13 22:50:39 +02:00
alalazo
ad16830f71 log : added timeout to avoid deadlocks on daemon join 2016-07-13 11:20:23 +02:00
alalazo
813cb032c4 package.py : updated logic to log.py rework
Conflicts:
	lib/spack/spack/package.py
2016-07-13 09:27:09 +02:00
alalazo
9af964a6d6 log_output : moved from os.fork to multiprocessing.Process 2016-07-13 09:27:09 +02:00
Adam J. Stewart
71a297f121 Fix backup logic in filter_file 2016-06-23 16:44:41 -05:00
Todd Gamblin
c09111bbef Merge pull request #1053 from davydden/pkg/dealii_suite_update
update Petsc, Slepc, Trilinos, Superlu-dist and deal.II
2016-06-20 10:32:52 -07:00
Todd Gamblin
4569df025e Merge pull request #1060 from mathstuf/no-break-long-words
tty: do not break long words
2016-06-17 14:51:26 -07:00
Todd Gamblin
17b868381f Fixes #460: Do not show variants by default in spack find.
This does two things:

1. By default `spack find` no longer shows variants.  You have to
   supply `-v` to get that

2. This improves the `colify` implementation so that it no longer pads
   the rightmost column.  This avoids the issue where if one spec was
   too long in the output, *all* specs would have space padding added
   to that width, and it would look like the output of `spack find`
   was double spaced.  This no longer happens -- the one bad line
   wraps around and the other lines are now single-spaced when you use
   `-v` with boost.
2016-06-16 02:55:33 -07:00
Denis Davydov
3d3a26cd9a flake8 fixes for filesystem.py 2016-06-15 18:31:10 +02:00
Ben Boeckel
36decbfa69 tty: do not break long words
These "words" tend to be file paths which are best left unbroken.

Fixes #1057.
2016-06-15 12:22:35 -04:00
Denis Davydov
a860a3d614 superlu-dist: avoid harcoding blas/lapack/mpi; remove preferred version 2016-06-15 18:21:13 +02:00
Todd Gamblin
e7ced54369 Correct LLNL LGPL license template for clarity. 2016-05-11 21:22:25 -07:00
Todd Gamblin
220c72d67f Merge branch 'features/blas-lapack-hardening' into develop 2016-03-31 11:37:09 -07:00
Todd Gamblin
8b45d6f84f Merge pull request #658 from davydden/metis_parmetis_darwin
metis/parmetis/oce/boost: correct install_name on Darwin via a global tool
2016-03-27 16:10:05 -07:00
Denis Davydov
7eca1284c8 metis/parmetis/boost/oce/scalapack: correct install_name on Darwin via a global function. 2016-03-28 00:22:25 +02:00
Todd Gamblin
cde3320582 Add a method to find the containing directory of a library. 2016-03-27 11:32:42 -07:00
Todd Gamblin
b0b882cbb3 Speed up directives by skipping debug info in stack traversal.
- `caller_locals()` doesn't need debug info, only frame locals.
- `get_calling_module()` doesn't either.

- Changed calls to `inspect.stack()` -> `inspect.stack(0)`
2016-03-27 11:30:09 -07:00
Todd Gamblin
003fd4d834 Optimize __eq__ and __ne__ in key_ordering
- use `is` when possible before calling `_cmp_key()`
2016-03-14 04:55:30 -07:00
Todd Gamblin
214deeda54 Merge branch 'features/noexpand' into develop
Conflicts:
	lib/spack/llnl/util/filesystem.py
2016-03-08 03:29:57 -08:00