zhangyiss/spack
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

Add new RubyPackage build system base class (#18199)

Browse Source 
* Add new RubyPackage build system base class

* Ruby: add spack external find support

* Add build tests for RubyPackage
This commit is contained in:
Adam J. Stewart
2020-09-02 18:26:36 -05:00
committed by GitHub
parent e22a0ca5cf
commit 443407cda5
27 changed files with 513 additions and 112 deletions
Download Patch File Download Diff File Expand all files Collapse all files

172
lib/spack/docs/build_systems/rubypackage.rst
View File

@@ -12,5 +12,173 @@ RubyPackage
Like Perl, Python, and R, Ruby has its own build system for
installing Ruby gems.
This build system is a work-in-progress. See
https://github.com/spack/spack/pull/3127 for more information.
^^^^^^
Phases
^^^^^^
The ``RubyPackage`` base class provides the following phases that
can be overridden:
#. ``build`` - build everything needed to install
#. ``install`` - install everything from build directory
For packages that come with a ``*.gemspec`` file, these phases run:
.. code-block:: console
$ gem build *.gemspec
$ gem install *.gem
For packages that come with a ``Rakefile`` file, these phases run:
.. code-block:: console
$ rake package
$ gem install *.gem
For packages that come pre-packaged as a ``*.gem`` file, the build
phase is skipped and the install phase runs:
.. code-block:: console
$ gem install *.gem
These are all standard ``gem`` commands and can be found by running:
.. code-block:: console
$ gem help commands
For packages that only distribute ``*.gem`` files, these files can be
downloaded with the ``expand=False`` option in the ``version`` directive.
The build phase will be automatically skipped.
^^^^^^^^^^^^^^^
Important files
^^^^^^^^^^^^^^^
When building from source, Ruby packages can be identified by the
presence of any of the following files:
* ``*.gemspec``
* ``Rakefile``
* ``setup.rb`` (not yet supported)
However, not all Ruby packages are released as source code. Some are only
released as ``*.gem`` files. These files can be extracted using:
.. code-block:: console
$ gem unpack *.gem
^^^^^^^^^^^
Description
^^^^^^^^^^^
The ``*.gemspec`` file may contain something like:
.. code-block:: ruby
summary = 'An implementation of the AsciiDoc text processor and publishing toolchain'
description = 'A fast, open source text processor and publishing toolchain for converting AsciiDoc content to HTML 5, DocBook 5, and other formats.'
Either of these can be used for the description of the Spack package.
^^^^^^^^
Homepage
^^^^^^^^
The ``*.gemspec`` file may contain something like:
.. code-block:: ruby
homepage = 'https://asciidoctor.org'
This should be used as the official homepage of the Spack package.
^^^^^^^^^^^^^^^^^^^^^^^^^
Build system dependencies
^^^^^^^^^^^^^^^^^^^^^^^^^
All Ruby packages require Ruby at build and run-time. For this reason,
the base class contains:
.. code-block:: python
extends('ruby')
depends_on('ruby', type=('build', 'run'))
The ``*.gemspec`` file may contain something like:
.. code-block:: ruby
required_ruby_version = '>= 2.3.0'
This can be added to the Spack package using:
.. code-block:: python
depends_on('ruby@2.3.0:', type=('build', 'run'))
^^^^^^^^^^^^^^^^^
Ruby dependencies
^^^^^^^^^^^^^^^^^
When you install a package with ``gem``, it reads the ``*.gemspec``
file in order to determine the dependencies of the package.
If the dependencies are not yet installed, ``gem`` downloads them
and installs them for you. This may sound convenient, but Spack
cannot rely on this behavior for two reasons:
#. Spack needs to be able to install packages on air-gapped networks.
If there is no internet connection, ``gem`` can't download the
package dependencies. By explicitly listing every dependency in
the ``package.py``, Spack knows what to download ahead of time.
#. Duplicate installations of the same dependency may occur.
Spack supports *activation* of Ruby extensions, which involves
symlinking the package installation prefix to the Ruby installation
prefix. If your package is missing a dependency, that dependency
will be installed to the installation directory of the same package.
If you try to activate the package + dependency, it may cause a
problem if that package has already been activated.
For these reasons, you must always explicitly list all dependencies.
Although the documentation may list the package's dependencies,
often the developers assume people will use ``gem`` and won't have to
worry about it. Always check the ``*.gemspec`` file to find the true
dependencies.
Check for the following clues in the ``*.gemspec`` file:
* ``add_runtime_dependency``
These packages are required for installation.
* ``add_dependency``
This is an alias for ``add_runtime_dependency``
* ``add_development_dependency``
These packages are optional dependencies used for development.
They should not be added as dependencies of the package.
^^^^^^^^^^^^^^^^^^^^^^
External documentation
^^^^^^^^^^^^^^^^^^^^^^
For more information on Ruby packaging, see:
https://guides.rubygems.org/

59
lib/spack/spack/build_systems/ruby.py Normal file
View File

@@ -0,0 +1,59 @@
# Copyright 2013-2020 Lawrence Livermore National Security, LLC and other
# Spack Project Developers. See the top-level COPYRIGHT file for details.
#
# SPDX-License-Identifier: (Apache-2.0 OR MIT)
import glob
import inspect
from spack.directives import depends_on, extends
from spack.package import PackageBase, run_after
class RubyPackage(PackageBase):
"""Specialized class for building Ruby gems.
This class provides two phases that can be overridden if required:
#. :py:meth:`~.RubyPackage.build`
#. :py:meth:`~.RubyPackage.install`
"""
#: Phases of a Ruby package
phases = ['build', 'install']
#: This attribute is used in UI queries that need to know the build
#: system base class
build_system_class = 'RubyPackage'
extends('ruby')
depends_on('ruby', type=('build', 'run'))
def build(self, spec, prefix):
"""Build a Ruby gem."""
# ruby-rake provides both rake.gemspec and Rakefile, but only
# rake.gemspec can be built without an existing rake installation
gemspecs = glob.glob('*.gemspec')
rakefiles = glob.glob('Rakefile')
if gemspecs:
inspect.getmodule(self).gem('build', '--norc', gemspecs[0])
elif rakefiles:
jobs = inspect.getmodule(self).make_jobs
inspect.getmodule(self).rake('package', '-j{0}'.format(jobs))
else:
# Some Ruby packages only ship `*.gem` files, so nothing to build
pass
def install(self, spec, prefix):
"""Install a Ruby gem.
The ruby package sets ``GEM_HOME`` to tell gem where to install to."""
gems = glob.glob('*.gem')
if gems:
inspect.getmodule(self).gem(
'install', '--norc', '--ignore-dependencies', gems[0])
# Check that self.prefix is there after installation
run_after('install')(PackageBase.sanity_check_prefix)

48
lib/spack/spack/cmd/create.py
View File

@@ -352,6 +352,34 @@ def __init__(self, name, *args, **kwargs):
super(OctavePackageTemplate, self).__init__(name, *args, **kwargs)
class RubyPackageTemplate(PackageTemplate):
"""Provides appropriate overrides for Ruby packages"""
base_class_name = 'RubyPackage'
dependencies = """\
# FIXME: Add dependencies if required. Only add the ruby dependency
# if you need specific versions. A generic ruby dependency is
# added implicity by the RubyPackage class.
# depends_on('ruby@X.Y.Z:', type=('build', 'run'))
# depends_on('ruby-foo', type=('build', 'run'))"""
body_def = """\
def build(self, spec, prefix):
# FIXME: If not needed delete this function
pass"""
def __init__(self, name, *args, **kwargs):
# If the user provided `--name ruby-numpy`, don't rename it
# ruby-ruby-numpy
if not name.startswith('ruby-'):
# Make it more obvious that we are renaming the package
tty.msg("Changing package name from {0} to ruby-{0}".format(name))
name = 'ruby-{0}'.format(name)
super(RubyPackageTemplate, self).__init__(name, *args, **kwargs)
class MakefilePackageTemplate(PackageTemplate):
"""Provides appropriate overrides for Makefile packages"""
@@ -410,6 +438,7 @@ def __init__(self, name, *args, **kwargs):
'perlmake': PerlmakePackageTemplate,
'perlbuild': PerlbuildPackageTemplate,
'octave': OctavePackageTemplate,
'ruby': RubyPackageTemplate,
'makefile': MakefilePackageTemplate,
'intel': IntelPackageTemplate,
'meson': MesonPackageTemplate,
@@ -464,12 +493,16 @@ def __call__(self, stage, url):
"""Try to guess the type of build system used by a project based on
the contents of its archive or the URL it was downloaded from."""
# Most octave extensions are hosted on Octave-Forge:
# https://octave.sourceforge.net/index.html
# They all have the same base URL.
if url is not None and 'downloads.sourceforge.net/octave/' in url:
self.build_system = 'octave'
return
if url is not None:
# Most octave extensions are hosted on Octave-Forge:
# https://octave.sourceforge.net/index.html
# They all have the same base URL.
if 'downloads.sourceforge.net/octave/' in url:
self.build_system = 'octave'
return
if url.endswith('.gem'):
self.build_system = 'ruby'
return
# A list of clues that give us an idea of the build system a package
# uses. If the regular expression matches a file contained in the
@@ -488,6 +521,9 @@ def __call__(self, stage, url):
(r'/WORKSPACE$', 'bazel'),
(r'/Build\.PL$', 'perlbuild'),
(r'/Makefile\.PL$', 'perlmake'),
(r'/.*\.gemspec$', 'ruby'),
(r'/Rakefile$', 'ruby'),
(r'/setup\.rb$', 'ruby'),
(r'/.*\.pro$', 'qmake'),
(r'/(GNU)?[Mm]akefile$', 'makefile'),
(r'/DESCRIPTION$', 'octave'),

1
lib/spack/spack/pkgkit.py
View File

@@ -27,6 +27,7 @@
from spack.build_systems.python import PythonPackage
from spack.build_systems.r import RPackage
from spack.build_systems.perl import PerlPackage
from spack.build_systems.ruby import RubyPackage
from spack.build_systems.intel import IntelPackage
from spack.build_systems.meson import MesonPackage
from spack.build_systems.sip import SIPPackage

3
lib/spack/spack/test/build_system_guess.py
View File

@@ -23,6 +23,9 @@
('WORKSPACE', 'bazel'),
('Makefile.PL', 'perlmake'),
('Build.PL', 'perlbuild'),
('foo.gemspec', 'ruby'),
('Rakefile', 'ruby'),
('setup.rb', 'ruby'),
('GNUmakefile', 'makefile'),
('makefile', 'makefile'),
('Makefile', 'makefile'),