Add docs on new top-level VCS attributes
This commit is contained in:
parent
5d07042f23
commit
7902f3eb49
@ -413,27 +413,26 @@ The most straightforward way to add new versions to your package is to
|
||||
add a line like this in the package class:
|
||||
|
||||
.. code-block:: python
|
||||
:linenos:
|
||||
|
||||
class Foo(Package):
|
||||
url = 'http://example.com/foo-1.0.tar.gz'
|
||||
version('8.2.1', '4136d7b4c04df68b686570afa26988ac')
|
||||
...
|
||||
|
||||
Versions should be listed with the newest version first.
|
||||
url = "http://example.com/foo-1.0.tar.gz"
|
||||
|
||||
version('8.2.1', '4136d7b4c04df68b686570afa26988ac')
|
||||
version('8.2.0', '1c9f62f0778697a09d36121ead88e08e')
|
||||
version('8.1.2', 'd47dd09ed7ae6e7fd6f9a816d7f5fdf6')
|
||||
|
||||
Versions should be listed in descending order, from newest to oldest.
|
||||
|
||||
^^^^^^^^^^^^^
|
||||
Date Versions
|
||||
^^^^^^^^^^^^^
|
||||
|
||||
If you wish to use dates as versions, it is best to use the format
|
||||
``@date-yyyy-mm-dd``. This will ensure they sort in the correct
|
||||
order. If you want your date versions to be numeric (assuming they
|
||||
don't conflict with other numeric versions), you can use just
|
||||
``yyyy.mm.dd``.
|
||||
``@yyyy-mm-dd``. This will ensure they sort in the correct order.
|
||||
|
||||
Alternately, you might use a hybrid release-version / date scheme.
|
||||
For example, ``@1.3.2016.08.31`` would mean the version from the
|
||||
For example, ``@1.3_2016-08-31`` would mean the version from the
|
||||
``1.3`` branch, as of August 31, 2016.
|
||||
|
||||
^^^^^^^^^^^^
|
||||
@ -442,7 +441,7 @@ Version URLs
|
||||
|
||||
By default, each version's URL is extrapolated from the ``url`` field
|
||||
in the package. For example, Spack is smart enough to download
|
||||
version ``8.2.1.`` of the ``Foo`` package above from
|
||||
version ``8.2.1`` of the ``Foo`` package above from
|
||||
http://example.com/foo-8.2.1.tar.gz.
|
||||
|
||||
If the URL is particularly complicated or changes based on the release,
|
||||
@ -515,44 +514,28 @@ of its versions, you can add an explicit URL for a particular version:
|
||||
|
||||
|
||||
When you supply a custom URL for a version, Spack uses that URL
|
||||
*verbatim* and does not perform extrapolation.
|
||||
*verbatim* and does not perform extrapolation. The order of precedence
|
||||
of these methods is:
|
||||
|
||||
^^^^^^^^^^^^^^^^^^^^^
|
||||
PyPI and version URLs
|
||||
^^^^^^^^^^^^^^^^^^^^^
|
||||
#. package-level ``url``
|
||||
#. ``url_for_version()``
|
||||
#. version-specific ``url``
|
||||
|
||||
In addition to their developer websites, many python packages are hosted at the
|
||||
`Python Package Index (PyPi) <https://pypi.python.org/pypi>`_. Although links to
|
||||
these individual files are typically `generated using a hash
|
||||
<https://bitbucket.org/pypa/pypi/issues/438>`_ it is often possible to find a
|
||||
reliable link of the format
|
||||
|
||||
.. code-block:: sh
|
||||
|
||||
https://pypi.python.org/packages/source/<first letter of package>/<package>/<package>-<version>.<extension>
|
||||
|
||||
Packages hosted on GitHub and the like are often developer versions that do not
|
||||
contain all of the files (e.g. configuration scripts) necessary to support
|
||||
compilation. For this reason it is ideal to link to a repository such as PyPi
|
||||
if possible.
|
||||
|
||||
More recently, sources are being indexed at `pypi.io <https://pypi.io>`_ as
|
||||
well. Links obtained from this site follow a similar pattern, namely
|
||||
|
||||
.. code-block:: sh
|
||||
|
||||
https://pypi.io/packages/source/<first letter of package>/<package>/<package>-<version>.<extension>
|
||||
|
||||
These links currently redirect back to `pypi.python.org
|
||||
<https://pypi.python.org>`_, but this `may change in the future
|
||||
<https://bitbucket.org/pypa/pypi/issues/438#comment-27243225>`_.
|
||||
so if your package contains a ``url_for_version()``, it can be overridden
|
||||
by a version-specific ``url``.
|
||||
|
||||
If your package does not contain a package-level ``url`` or ``url_for_version()``,
|
||||
Spack can determine which URL to download from even if only some of the versions
|
||||
specify their own ``url``. Spack will use the nearest URL *before* the requested
|
||||
version. This is useful for packages that have an easy to extrapolate URL, but
|
||||
keep changing their URL format every few releases. With this method, you only
|
||||
need to specify the ``url`` when the URL changes.
|
||||
|
||||
^^^^^^^^^^^^^^^^^^^^^^^^
|
||||
Skipping the expand step
|
||||
^^^^^^^^^^^^^^^^^^^^^^^^
|
||||
|
||||
Spack normally expands archives (e.g. `*.tar.gz` and `*.zip`) automatically
|
||||
Spack normally expands archives (e.g. ``*.tar.gz`` and ``*.zip``) automatically
|
||||
after downloading them. If you want to skip this step (e.g., for
|
||||
self-extracting executables and other custom archive types), you can add
|
||||
``expand=False`` to a ``version`` directive.
|
||||
@ -560,7 +543,7 @@ self-extracting executables and other custom archive types), you can add
|
||||
.. code-block:: python
|
||||
|
||||
version('8.2.1', '4136d7b4c04df68b686570afa26988ac',
|
||||
url='http://example.com/foo-8.2.1-special-version.tar.gz', expand=False)
|
||||
url='http://example.com/foo-8.2.1-special-version.sh', expand=False)
|
||||
|
||||
When ``expand`` is set to ``False``, Spack sets the current working
|
||||
directory to the directory containing the downloaded archive before it
|
||||
@ -605,7 +588,7 @@ here will suffice; for example, ``@develop``, ``@master``, ``@local``.
|
||||
The following rules determine the sort order of numeric
|
||||
vs. non-numeric versions:
|
||||
|
||||
#. The non-numeric versions ``@develop`` is considered greatest (newest).
|
||||
#. The non-numeric version ``@develop`` is considered greatest (newest).
|
||||
|
||||
#. Numeric versions are all less than ``@develop`` version, and are
|
||||
sorted numerically.
|
||||
@ -617,7 +600,7 @@ The logic behind this sort order is two-fold:
|
||||
|
||||
#. Non-numeric versions are usually used for special cases while
|
||||
developing or debugging a piece of software. Keeping most of them
|
||||
less than numeric versions ensures that Spack choose numeric
|
||||
less than numeric versions ensures that Spack chooses numeric
|
||||
versions by default whenever possible.
|
||||
|
||||
#. The most-recent development version of a package will usually be
|
||||
@ -679,7 +662,7 @@ example for ``libelf``:
|
||||
...
|
||||
0.5.2 http://www.mr511.de/software/libelf-0.5.2.tar.gz
|
||||
|
||||
How many would you like to checksum? (default is 5, q to abort)
|
||||
How many would you like to checksum? (default is 1, q to abort)
|
||||
|
||||
This does the same thing that ``spack create`` does, but it allows you
|
||||
to go back and add new versions easily as you need them (e.g., as
|
||||
@ -805,12 +788,12 @@ But its downloads are in many different subdirectories of
|
||||
homepage = "http://www.mpich.org"
|
||||
url = "http://www.mpich.org/static/downloads/3.0.4/mpich-3.0.4.tar.gz"
|
||||
list_url = "http://www.mpich.org/static/downloads/"
|
||||
list_depth = 2
|
||||
list_depth = 1
|
||||
|
||||
By default, Spack only looks at the top-level page available at
|
||||
``list_url``. ``list_depth`` tells it to follow up to 2 levels of
|
||||
links from the top-level page. Note that here, this implies two
|
||||
levels of subdirectories, as the ``mpich`` website is structured much
|
||||
``list_url``. ``list_depth = 1`` tells it to follow up to 1 level of
|
||||
links from the top-level page. Note that here, this implies 1
|
||||
level of subdirectories, as the ``mpich`` website is structured much
|
||||
like a filesystem. But ``list_depth`` really refers to link depth
|
||||
when spidering the page.
|
||||
|
||||
@ -823,13 +806,57 @@ Fetching from code repositories
|
||||
For some packages, source code is provided in a Version Control System
|
||||
(VCS) repository rather than in a tarball. Spack can fetch packages
|
||||
from VCS repositories. Currently, Spack supports fetching with `Git
|
||||
<git-fetch_>`_, `Mercurial (hg) <hg-fetch_>`_, and `Subversion (SVN)
|
||||
<svn-fetch_>`_.
|
||||
<git-fetch_>`_, `Mercurial (hg) <hg-fetch_>`_, `Subversion (svn)
|
||||
<svn-fetch_>`_, and `Go <go-fetch_>`_.
|
||||
|
||||
To fetch a package from a source repository, Spack needs to know which
|
||||
VCS to use and where to download from. Much like with ``url``, package
|
||||
authors can specify a class-level ``git``, ``hg``, ``svn``, or ``go``
|
||||
attribute containing the correct download location.
|
||||
|
||||
Many packages developed with Git have both a Git repository as well as
|
||||
release tarballs available for download. Packages can define both a
|
||||
class-level tarball URL and VCS. For example:
|
||||
|
||||
.. code-block:: python
|
||||
|
||||
class Trilinos(CMakePackage):
|
||||
|
||||
homepage = "https://trilinos.org/"
|
||||
url = "https://github.com/trilinos/Trilinos/archive/trilinos-release-12-12-1.tar.gz"
|
||||
git = "https://github.com/trilinos/Trilinos.git"
|
||||
|
||||
version('develop', branch='develop')
|
||||
version('master', branch='master')
|
||||
version('12.12.1', 'ecd4606fa332212433c98bf950a69cc7')
|
||||
version('12.10.1', '667333dbd7c0f031d47d7c5511fd0810')
|
||||
version('12.8.1', '9f37f683ee2b427b5540db8a20ed6b15')
|
||||
|
||||
If a package contains both a ``url`` and ``git`` class-level attribute,
|
||||
Spack decides which to use based on the arguments to the ``version()``
|
||||
directive. Versions containing a specific branch, tag, or revision are
|
||||
assumed to be for VCS download methods, while versions containing a
|
||||
checksum are assumed to be for URL download methods.
|
||||
|
||||
Like ``url``, if a specific version downloads from a different repository
|
||||
than the default repo, it can be overridden with a version-specific argument.
|
||||
|
||||
.. note::
|
||||
|
||||
In order to reduce ambiguity, each package can only have a single VCS
|
||||
top-level attribute in addition to ``url``. In the rare case that a
|
||||
package uses multiple VCS, a fetch strategy can be specified for each
|
||||
version. For example, the ``rockstar`` package contains:
|
||||
|
||||
.. code-block:: python
|
||||
|
||||
class Rockstar(MakefilePackage):
|
||||
|
||||
homepage = "https://bitbucket.org/gfcstanford/rockstar"
|
||||
|
||||
version('develop', git='https://bitbucket.org/gfcstanford/rockstar.git')
|
||||
version('yt', hg='https://bitbucket.org/MatthewTurk/rockstar')
|
||||
|
||||
To fetch a package from a source repository, you add a ``version()``
|
||||
call to your package with parameters indicating the repository URL and
|
||||
any branch, tag, or revision to fetch. See below for the parameters
|
||||
you'll need for each VCS system.
|
||||
|
||||
.. _git-fetch:
|
||||
|
||||
@ -837,11 +864,11 @@ you'll need for each VCS system.
|
||||
Git
|
||||
^^^
|
||||
|
||||
Git fetching is enabled with the following parameters to ``version``:
|
||||
Git fetching supports the following parameters to ``version``:
|
||||
|
||||
* ``git``: URL of the git repository.
|
||||
* ``tag``: name of a tag to fetch.
|
||||
* ``branch``: name of a branch to fetch.
|
||||
* ``git``: URL of the git repository, if different than the class-level ``git``.
|
||||
* ``branch``: Name of a branch to fetch.
|
||||
* ``tag``: Name of a tag to fetch.
|
||||
* ``commit``: SHA hash (or prefix) of a commit to fetch.
|
||||
* ``submodules``: Also fetch submodules recursively when checking out this repository.
|
||||
|
||||
@ -853,66 +880,72 @@ Default branch
|
||||
.. code-block:: python
|
||||
|
||||
class Example(Package):
|
||||
...
|
||||
version('develop', git='https://github.com/example-project/example.git')
|
||||
|
||||
This download method is untrusted, and is not recommended.
|
||||
git = "https://github.com/example-project/example.git"
|
||||
|
||||
Tags
|
||||
To fetch from a particular tag, use the ``tag`` parameter along with
|
||||
``git``:
|
||||
version('develop')
|
||||
|
||||
.. code-block:: python
|
||||
|
||||
version('1.0.1', git='https://github.com/example-project/example.git',
|
||||
tag='v1.0.1')
|
||||
|
||||
This download method is untrusted, and is not recommended.
|
||||
This download method is untrusted, and is not recommended. Aside from HTTPS,
|
||||
there is no way to verify that the repository has not been compromised, and
|
||||
the commit you get when you install the package likely won't be the same
|
||||
commit that was used when the package was first written. Additionally, the
|
||||
default branch may change. It is best to at least specify a branch name.
|
||||
|
||||
Branches
|
||||
To fetch a particular branch, use ``branch`` instead:
|
||||
To fetch a particular branch, use the ``branch`` parameter:
|
||||
|
||||
.. code-block:: python
|
||||
|
||||
version('experimental', git='https://github.com/example-project/example.git',
|
||||
branch='experimental')
|
||||
version('experimental', branch='experimental')
|
||||
|
||||
This download method is untrusted, and is not recommended.
|
||||
This download method is untrusted, and is not recommended. Branches are
|
||||
moving targets, so the commit you get when you install the package likely
|
||||
won't be the same commit that was used when the package was first written.
|
||||
|
||||
Tags
|
||||
To fetch from a particular tag, use ``tag`` instead:
|
||||
|
||||
.. code-block:: python
|
||||
|
||||
version('1.0.1', tag='v1.0.1')
|
||||
|
||||
This download method is untrusted, and is not recommended. Although tags
|
||||
are generally more stable than branches, Git allows tags to be moved.
|
||||
Many developers use tags to denote rolling releases, and may move the
|
||||
tag when a bug is patched.
|
||||
|
||||
Commits
|
||||
Finally, to fetch a particular commit, use ``commit``:
|
||||
|
||||
.. code-block:: python
|
||||
|
||||
version('2014-10-08', git='https://github.com/example-project/example.git',
|
||||
commit='9d38cd4e2c94c3cea97d0e2924814acc')
|
||||
version('2014-10-08', commit='9d38cd4e2c94c3cea97d0e2924814acc')
|
||||
|
||||
This doesn't have to be a full hash; you can abbreviate it as you'd
|
||||
expect with git:
|
||||
|
||||
.. code-block:: python
|
||||
|
||||
version('2014-10-08', git='https://github.com/example-project/example.git',
|
||||
commit='9d38cd')
|
||||
version('2014-10-08', commit='9d38cd')
|
||||
|
||||
This download method *is trusted*. It is the recommended way to
|
||||
securely download from a Git repository.
|
||||
|
||||
It may be useful to provide a saner version for commits like this,
|
||||
e.g. you might use the date as the version, as done above. Or you
|
||||
could just use the abbreviated commit hash. It's up to the package
|
||||
author to decide what makes the most sense.
|
||||
e.g. you might use the date as the version, as done above. Or, if you
|
||||
know the commit at which a release was cut, you can use the release
|
||||
version. It's up to the package author to decide what makes the most
|
||||
sense. Although you can use the commit hash as the version number,
|
||||
this is not recommended, as it won't sort properly.
|
||||
|
||||
Submodules
|
||||
|
||||
You can supply ``submodules=True`` to cause Spack to fetch submodules
|
||||
recursively along with the repository at fetch time. For more information
|
||||
about git submodules see the manpage of git: ``man git-submodule``.
|
||||
|
||||
.. code-block:: python
|
||||
|
||||
version('1.0.1', git='https://github.com/example-project/example.git',
|
||||
tag='v1.0.1', submodules=True)
|
||||
version('1.0.1', tag='v1.0.1', submodules=True)
|
||||
|
||||
|
||||
.. _github-fetch:
|
||||
@ -921,7 +954,7 @@ Submodules
|
||||
GitHub
|
||||
^^^^^^
|
||||
|
||||
If a project is hosted on GitHub, *any* valid Git branch, tag or hash
|
||||
If a project is hosted on GitHub, *any* valid Git branch, tag, or hash
|
||||
may be downloaded as a tarball. This is accomplished simply by
|
||||
constructing an appropriate URL. Spack can checksum any package
|
||||
downloaded this way, thereby producing a trusted download. For
|
||||
@ -931,7 +964,7 @@ checksum.
|
||||
.. code-block:: python
|
||||
|
||||
version('1.9.5.1.1', 'd035e4bc704d136db79b43ab371b27d2',
|
||||
url='https://www.github.com/jswhit/pyproj/tarball/0be612cc9f972e38b50a90c946a9b353e2ab140f')
|
||||
url='https://www.github.com/jswhit/pyproj/tarball/0be612cc9f972e38b50a90c946a9b353e2ab140f')
|
||||
|
||||
.. _hg-fetch:
|
||||
|
||||
@ -939,34 +972,38 @@ checksum.
|
||||
Mercurial
|
||||
^^^^^^^^^
|
||||
|
||||
Fetching with mercurial works much like `git <git-fetch>`_, but you
|
||||
Fetching with Mercurial works much like `Git <git-fetch>`_, but you
|
||||
use the ``hg`` parameter.
|
||||
|
||||
Default
|
||||
Add the ``hg`` parameter with no ``revision``:
|
||||
Default branch
|
||||
Add the ``hg`` attribute with no ``revision`` passed to ``version``:
|
||||
|
||||
.. code-block:: python
|
||||
|
||||
version('develop', hg='https://jay.grs.rwth-aachen.de/hg/example')
|
||||
class Example(Package):
|
||||
|
||||
This download method is untrusted, and is not recommended.
|
||||
hg = "https://bitbucket.org/example-project/example"
|
||||
|
||||
version('develop')
|
||||
|
||||
This download method is untrusted, and is not recommended. As with
|
||||
Git's default fetching strategy, there is no way to verify the
|
||||
integrity of the download.
|
||||
|
||||
Revisions
|
||||
Add ``hg`` and ``revision`` parameters:
|
||||
To fetch a particular revision, use the ``revision`` parameter:
|
||||
|
||||
.. code-block:: python
|
||||
|
||||
version('1.0', hg='https://jay.grs.rwth-aachen.de/hg/example',
|
||||
revision='v1.0')
|
||||
|
||||
This download method is untrusted, and is not recommended.
|
||||
version('1.0', revision='v1.0')
|
||||
|
||||
Unlike ``git``, which has special parameters for different types of
|
||||
revisions, you can use ``revision`` for branches, tags, and commits
|
||||
when you fetch with Mercurial.
|
||||
when you fetch with Mercurial. Like Git, fetching specific branches
|
||||
or tags is an untrusted download method, and is not recommended.
|
||||
The recommended fetch strategy is to specify a particular commit
|
||||
hash as the revision.
|
||||
|
||||
As with git, you can fetch these versions using the ``spack install
|
||||
example@<version>`` command-line syntax.
|
||||
|
||||
.. _svn-fetch:
|
||||
|
||||
@ -974,34 +1011,70 @@ example@<version>`` command-line syntax.
|
||||
Subversion
|
||||
^^^^^^^^^^
|
||||
|
||||
To fetch with subversion, use the ``svn`` and ``revision`` parameters:
|
||||
To fetch with subversion, use the ``svn`` and ``revision`` parameters.
|
||||
|
||||
Fetching the head
|
||||
Simply add an ``svn`` parameter to ``version``:
|
||||
Simply add an ``svn`` parameter to the package:
|
||||
|
||||
.. code-block:: python
|
||||
|
||||
version('develop', svn='https://outreach.scidac.gov/svn/libmonitor/trunk')
|
||||
class Example(Package):
|
||||
|
||||
This download method is untrusted, and is not recommended.
|
||||
svn = "https://outreach.scidac.gov/svn/example/trunk"
|
||||
|
||||
version('develop')
|
||||
|
||||
This download method is untrusted, and is not recommended for the
|
||||
same reasons as mentioned above.
|
||||
|
||||
Fetching a revision
|
||||
To fetch a particular revision, add a ``revision`` to the
|
||||
version call:
|
||||
To fetch a particular revision, add a ``revision`` argument to the
|
||||
version directive:
|
||||
|
||||
.. code-block:: python
|
||||
|
||||
version('develop', svn='https://outreach.scidac.gov/svn/libmonitor/trunk',
|
||||
revision=128)
|
||||
version('develop', revision=128)
|
||||
|
||||
This download method is untrusted, and is not recommended.
|
||||
|
||||
Subversion branches are handled as part of the directory structure, so
|
||||
you can check out a branch or tag by changing the ``url``.
|
||||
Unfortunately, Subversion has no commit hashing scheme like Git and
|
||||
Mercurial do, so there is no way to guarantee that the download you
|
||||
get is the same as the download used when the package was created.
|
||||
Use at your own risk.
|
||||
|
||||
-------------------------------------------------
|
||||
Subversion branches are handled as part of the directory structure, so
|
||||
you can check out a branch or tag by changing the URL. If you want to
|
||||
package multiple branches, simply add a ``svn`` argument to each
|
||||
version directive.
|
||||
|
||||
.. _go-fetch:
|
||||
|
||||
^^
|
||||
Go
|
||||
^^
|
||||
|
||||
Go isn't a VCS, it is a programming language with a builtin command,
|
||||
`go get <https://golang.org/cmd/go/#hdr-Download_and_install_packages_and_dependencies>`_,
|
||||
that fetches packages and their dependencies automatically.
|
||||
It can clone a Git repository, or download from another source location.
|
||||
For example:
|
||||
|
||||
.. code-block:: python
|
||||
|
||||
class ThePlatinumSearcher(Package):
|
||||
|
||||
homepage = "https://github.com/monochromegane/the_platinum_searcher"
|
||||
go = "github.com/monochromegane/the_platinum_searcher/..."
|
||||
|
||||
version('head')
|
||||
|
||||
Go cannot be used to fetch a particular commit or branch, it always
|
||||
downloads the head of the repository. This download method is untrusted,
|
||||
and is not recommended. Use another fetch strategy whenever possible.
|
||||
|
||||
------------------------------------
|
||||
Resources (expanding extra tarballs)
|
||||
-------------------------------------------------
|
||||
------------------------------------
|
||||
|
||||
Some packages (most notably compilers) provide optional features if additional
|
||||
resources are expanded within their source tree before building. In Spack it is
|
||||
|
@ -29,12 +29,11 @@
|
||||
|
||||
class ThePlatinumSearcher(Package):
|
||||
"""Fast parallel recursive grep alternative"""
|
||||
|
||||
homepage = "https://github.com/monochromegane/the_platinum_searcher"
|
||||
url = "https://github.com/monochromegane/the_platinum_searcher"
|
||||
go = "github.com/monochromegane/the_platinum_searcher/..."
|
||||
|
||||
package = 'github.com/monochromegane/the_platinum_searcher/...'
|
||||
|
||||
version('head', go=package)
|
||||
version('head')
|
||||
|
||||
extends("go", deptypes='build')
|
||||
|
||||
|
Loading…
Reference in New Issue
Block a user