locks: add configuration and command-line options to enable/disable locks (#7692)

- spack.util.lock behaves the same as llnl.util.lock, but Lock._lock and
  Lock._unlock do nothing.

- can be disabled with a control variable.

- configuration options can enable/disable locking:
  - `locks` option in spack configuration controls whether Spack will use filesystem locks or not.
  - `-l` and `-L` command-line options can force-disable or force-enable locking.

- Spack will check for group- and world-writability before disabling
  locks, and it will not allow a group- or world-writable instance to
  have locks disabled.

- update documentation
This commit is contained in:
Todd Gamblin
2018-05-18 14:41:03 -07:00
committed by GitHub
parent 780cc9d72d
commit 54201e3c02
11 changed files with 323 additions and 54 deletions

View File

@@ -1093,22 +1093,43 @@ several variants:
Filesystem requirements
-----------------------
Spack currently needs to be run from a filesystem that supports
By default, Spack needs to be run from a filesystem that supports
``flock`` locking semantics. Nearly all local filesystems and recent
versions of NFS support this, but parallel filesystems may be mounted
without ``flock`` support enabled. You can determine how your
filesystems are mounted with ``mount -p``. The output for a Lustre
versions of NFS support this, but parallel filesystems or NFS volumes may
be configured without ``flock`` support enabled. You can determine how
your filesystems are mounted with ``mount``. The output for a Lustre
filesystem might look like this:
.. code-block:: console
$ mount -l | grep lscratch
pilsner-mds1-lnet0@o2ib100:/lsd on /p/lscratchd type lustre (rw,nosuid,noauto,_netdev,lazystatfs,flock)
porter-mds1-lnet0@o2ib100:/lse on /p/lscratche type lustre (rw,nosuid,noauto,_netdev,lazystatfs,flock)
$ mount | grep lscratch
mds1-lnet0@o2ib100:/lsd on /p/lscratchd type lustre (rw,nosuid,lazystatfs,flock)
mds2-lnet0@o2ib100:/lse on /p/lscratche type lustre (rw,nosuid,lazystatfs,flock)
Note the ``flock`` option on both Lustre mounts. If you do not see
this or a similar option for your filesystem, you may need ot ask your
system administrator to enable ``flock``.
Note the ``flock`` option on both Lustre mounts.
If you do not see this or a similar option for your filesystem, you have
a few options. First, you can move your Spack installation to a
filesystem that supports locking. Second, you could ask your system
administrator to enable ``flock`` for your filesystem.
If none of those work, you can disable locking in one of two ways:
1. Run Spack with the ``-L`` or ``--disable-locks`` option to disable
locks on a call-by-call basis.
2. Edit :ref:`config.yaml <config-yaml>` and set the ``locks`` option
to ``false`` to always disable locking.
.. warning::
If you disable locking, concurrent instances of Spack will have no way
to avoid stepping on each other. You must ensure that there is only
**one** instance of Spack running at a time. Otherwise, Spack may end
up with a corrupted database file, or you may not be able to see all
installed packages in commands like ``spack find``.
If you are unfortunate enough to run into this situation, you may be
able to fix it by running ``spack reindex``.
This issue typically manifests with the error below:

View File

@@ -150,6 +150,17 @@ checksum, and will refuse to build packages that it cannot verify. Set
to ``false`` to disable these checks. Disabling this can expose you to
attacks. Use at your own risk.
--------------------
``locks``
--------------------
When set to ``true``, concurrent instances of Spack will use locks to
avoid modifying the install tree, database file, etc. If false, Spack
will disable all locking, but you must **not** run concurrent instances
of Spack. For filesystems that don't support locking, you should set
this to ``false`` and run one Spack at a time, but otherwise we recommend
enabling locks.
--------------------
``dirty``
--------------------