zhangyiss/spack
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

Remove the spack.architecture module (#25986)

Browse Source 
The `spack.architecture` module contains an `Arch` class that is very similar to `spack.spec.ArchSpec` but points to platform, operating system and target objects rather than "names". There's a TODO in the class since 2016:

abb0f6e27c/lib/spack/spack/architecture.py (L70-L75)

and this PR basically addresses that. Since there are just a few places where the `Arch` class was used, here we query the relevant platform objects where they are needed directly from `spack.platforms`. This permits to clean the code from vestigial logic.

Modifications:
- [x] Remove the `spack.architecture` module and replace its use by `spack.platforms`
- [x] Remove unneeded tests
This commit is contained in:
Massimiliano Culpo 2021-10-06 19:28:12 +02:00 committed by GitHub
parent 232086c6af
commit 319ae9254e
No known key found for this signature in database
GPG Key ID: 4AEE18F83AFDEB23
37 changed files with 282 additions and 441 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

9
lib/spack/docs/developer_guide.rst
View File

@ -210,15 +210,6 @@ Spec-related modules
but compilers aren't fully integrated with the build process
yet.
:mod:`spack.architecture`
:func:`architecture.default_arch <spack.architecture.default_arch>` is used
to determine the host architecture while building.
.. warning::
Not yet implemented. Should eventually have architecture
descriptions for cross-compiling.
^^^^^^^^^^^^^^^^^
Build environment
^^^^^^^^^^^^^^^^^

242
lib/spack/spack/architecture.py
View File

@ -1,242 +0,0 @@
# Copyright 2013-2021 Lawrence Livermore National Security, LLC and other
# Spack Project Developers. See the top-level COPYRIGHT file for details.
#
# SPDX-License-Identifier: (Apache-2.0 OR MIT)
"""Aggregate the target processor, the operating system and the target
platform into an architecture object.
On a multiple architecture machine, the architecture spec field can be set to
build a package against any target and operating system that is present on the
platform. On Cray platforms or any other architecture that has different front
and back end environments, the operating system will determine the method of
compiler detection.
There are two different types of compiler detection:
1. Through the $PATH env variable (front-end detection)
2. Through the module system. (back-end detection)
Depending on which operating system is specified, the compiler will be detected
using one of those methods.
For platforms such as linux and darwin, the operating system is autodetected.
The command line syntax for specifying an architecture is as follows:
target=<Target name> os=<OperatingSystem name>
If the user wishes to use the defaults, either target or os can be left out of
the command line and Spack will concretize using the default. These defaults
are set in the 'platforms/' directory which contains the different subclasses
for platforms. If the machine has multiple architectures, the user can
also enter frontend, or fe or backend or be. These settings will concretize
to their respective frontend and backend targets and operating systems.
Platforms are an abstract class that are extended by subclasses. If the user
wants to add a new type of platform (such as cray_xe), they can create a
subclass and set all the class attributes such as priority, front_target,
back_target, front_os, back_os. Platforms also contain a priority class
attribute. A lower number signifies higher priority. These numbers are
arbitrarily set and can be changed though often there isn't much need unless a
new platform is added and the user wants that to be detected first.
Targets are created inside the platform subclasses. Most architecture
(like linux, and darwin) will have only one target family (x86_64) but in the case of
Cray machines, there is both a frontend and backend processor. The user can
specify which targets are present on front-end and back-end architecture
Depending on the platform, operating systems are either autodetected or are
set. The user can set the frontend and backend operating setting by the class
attributes front_os and back_os. The operating system as described earlier,
will be responsible for compiler detection.
"""
import contextlib
import archspec.cpu
import llnl.util.lang as lang
import spack.compiler
import spack.compilers
import spack.config
import spack.operating_systems
import spack.platforms
import spack.spec
import spack.target
import spack.util.spack_yaml as syaml
import spack.version
@lang.lazy_lexicographic_ordering
class Arch(object):
"""Architecture is now a class to help with setting attributes.
TODO: refactor so that we don't need this class.
"""
def __init__(self, plat=None, os=None, target=None):
self.platform = plat
if plat and os:
os = self.platform.operating_system(os)
self.os = os
if plat and target:
target = self.platform.target(target)
self.target = target
# Hooks for parser to use when platform is set after target or os
self.target_string = None
self.os_string = None
@property
def concrete(self):
return all(
(self.platform is not None,
isinstance(self.platform, spack.platforms.Platform),
self.os is not None,
isinstance(self.os, spack.operating_systems.OperatingSystem),
self.target is not None, isinstance(self.target, spack.target.Target))
)
def __str__(self):
if self.platform or self.os or self.target:
if self.platform.name == 'darwin':
os_name = self.os.name if self.os else "None"
else:
os_name = str(self.os)
return (str(self.platform) + "-" +
os_name + "-" + str(self.target))
else:
return ''
def __contains__(self, string):
return string in str(self)
# TODO: make this unnecessary: don't include an empty arch on *every* spec.
def __nonzero__(self):
return (self.platform is not None or
self.os is not None or
self.target is not None)
__bool__ = __nonzero__
def _cmp_iter(self):
if isinstance(self.platform, spack.platforms.Platform):
yield self.platform.name
else:
yield self.platform
if isinstance(self.os, spack.operating_systems.OperatingSystem):
yield self.os.name
else:
yield self.os
if isinstance(self.target, spack.target.Target):
yield self.target.microarchitecture
else:
yield self.target
def to_dict(self):
str_or_none = lambda v: str(v) if v else None
d = syaml.syaml_dict([
('platform', str_or_none(self.platform)),
('platform_os', str_or_none(self.os)),
('target', self.target.to_dict_or_value())])
return syaml.syaml_dict([('arch', d)])
def to_spec(self):
"""Convert this Arch to an anonymous Spec with architecture defined."""
spec = spack.spec.Spec()
spec.architecture = spack.spec.ArchSpec(str(self))
return spec
@staticmethod
def from_dict(d):
spec = spack.spec.ArchSpec.from_dict(d)
return arch_for_spec(spec)
def arch_for_spec(arch_spec):
"""Transforms the given architecture spec into an architecture object."""
arch_spec = spack.spec.ArchSpec(arch_spec)
assert arch_spec.concrete
arch_plat = spack.platforms.by_name(arch_spec.platform)
if not (arch_plat.operating_system(arch_spec.os) and
arch_plat.target(arch_spec.target)):
sys_type = str(default_arch())
msg = ("Can't recreate arch for spec {0} on current arch {1}; "
"spec architecture is too different")
raise ValueError(msg.format(arch_spec, sys_type))
return Arch(arch_plat, arch_spec.os, arch_spec.target)
@lang.memoized
def _platform():
return spack.platforms.host()
#: The "real" platform of the host running Spack. This should not be changed
#: by any method and is here as a convenient way to refer to the host platform.
real_platform = _platform
#: The current platform used by Spack. May be swapped by the use_platform
#: context manager.
platform = _platform
@lang.memoized
def default_arch():
"""Default ``Arch`` object for this machine"""
return Arch(platform(), 'default_os', 'default_target')
@lang.memoized
def compatible_sys_types():
"""Return a list of all the platform-os-target tuples compatible
with the current host.
"""
current_host = archspec.cpu.host()
compatible_targets = [current_host] + current_host.ancestors
compatible_archs = [
str(Arch(platform(), 'default_os', target)) for target in compatible_targets
]
return compatible_archs
class _PickleableCallable(object):
"""Class used to pickle a callable that may substitute either
_platform or _all_platforms. Lambda or nested functions are
not pickleable.
"""
def __init__(self, return_value):
self.return_value = return_value
def __call__(self):
return self.return_value
@contextlib.contextmanager
def use_platform(new_platform):
global platform
msg = '"{0}" must be an instance of Platform'
assert isinstance(new_platform, spack.platforms.Platform), msg.format(new_platform)
original_platform_fn = platform
try:
platform = _PickleableCallable(new_platform)
# Clear configuration and compiler caches
spack.config.config.clear_caches()
spack.compilers._cache_config_files = []
yield new_platform
finally:
platform = original_platform_fn
# Clear configuration and compiler caches
spack.config.config.clear_caches()
spack.compilers._cache_config_files = []

25
lib/spack/spack/bootstrap.py
View File

@ -23,7 +23,6 @@
import llnl.util.filesystem as fs
import llnl.util.tty as tty
import spack.architecture
import spack.binary_distribution
import spack.config
import spack.detection
@ -65,10 +64,10 @@ def _try_import_from_store(module, abstract_spec_str):
module: Python module to be imported
abstract_spec_str: abstract spec that may provide the module
"""
bincache_platform = spack.architecture.real_platform()
bincache_platform = spack.platforms.real_host()
if str(bincache_platform) == 'cray':
bincache_platform = spack.platforms.linux.Linux()
with spack.architecture.use_platform(bincache_platform):
with spack.platforms.use_platform(bincache_platform):
abstract_spec_str = str(spack.spec.Spec(abstract_spec_str))
# We have to run as part of this python interpreter
@ -191,10 +190,10 @@ def try_import(self, module, abstract_spec_str):
)
# On Cray we want to use Linux binaries if available from mirrors
bincache_platform = spack.architecture.real_platform()
bincache_platform = spack.platforms.real_host()
if str(bincache_platform) == 'cray':
bincache_platform = spack.platforms.Linux()
with spack.architecture.use_platform(bincache_platform):
with spack.platforms.use_platform(bincache_platform):
abstract_spec = spack.spec.Spec(
abstract_spec_str + ' ^' + spec_for_current_python()
)
@ -249,7 +248,7 @@ def try_import(self, module, abstract_spec_str):
"spec": str(index_spec.compiler),
"target": str(index_spec.target.family)
}
with spack.architecture.use_platform(bincache_platform):
with spack.platforms.use_platform(bincache_platform):
with spack.config.override(
'compilers', [{'compiler': compiler_entry}]
):
@ -287,7 +286,7 @@ def try_import(module, abstract_spec_str):
# Try to build and install from sources
with spack_python_interpreter():
# Add hint to use frontend operating system on Cray
if str(spack.architecture.platform()) == 'cray':
if str(spack.platforms.host()) == 'cray':
abstract_spec_str += ' os=fe'
concrete_spec = spack.spec.Spec(
@ -502,7 +501,7 @@ def _bootstrap_config_scopes():
('bootstrap', _config_path())
)
for name, path in configuration_paths:
platform = spack.architecture.platform().name
platform = spack.platforms.host().name
platform_scope = spack.config.ConfigScope(
'/'.join([name, platform]), os.path.join(path, platform)
)
@ -515,11 +514,7 @@ def _bootstrap_config_scopes():
def _add_compilers_if_missing():
# Do not use spack.architecture.default_arch() since it memoize the result
arch = spack.architecture.Arch(
spack.architecture.real_platform(), 'default_os', 'default_target'
)
arch = spack.spec.ArchSpec(str(arch)) # The call below expects an ArchSpec object
arch = spack.spec.ArchSpec.frontend_arch()
if not spack.compilers.compilers_for_arch(arch):
new_compilers = spack.compilers.find_new_compilers()
if new_compilers:
@ -540,7 +535,7 @@ def ensure_bootstrap_configuration():
bootstrap_store_path = store_path()
user_configuration = _read_and_sanitize_configuration()
with spack.environment.no_active_environment():
with spack.architecture.use_platform(spack.architecture.real_platform()):
with spack.platforms.use_platform(spack.platforms.real_host()):
with spack.repo.use_repositories(spack.paths.packages_path):
with spack.store.use_store(bootstrap_store_path):
# Default configuration scopes excluding command line
@ -611,7 +606,7 @@ def clingo_root_spec():
# Add a proper compiler hint to the root spec. We use GCC for
# everything but MacOS.
if str(spack.architecture.platform()) == 'darwin':
if str(spack.platforms.host()) == 'darwin':
spec_str += ' %apple-clang'
else:
spec_str += ' %gcc'

17
lib/spack/spack/build_environment.py
View File

@ -49,7 +49,6 @@
from llnl.util.tty.color import cescape, colorize
from llnl.util.tty.log import MultiProcessFd
import spack.architecture as arch
import spack.build_systems.cmake
import spack.build_systems.meson
import spack.config
@ -57,6 +56,7 @@
import spack.main
import spack.package
import spack.paths
import spack.platforms
import spack.repo
import spack.schema.environment
import spack.store
@ -148,9 +148,10 @@ def __call__(self, *args, **kwargs):
def _on_cray():
hostarch = arch.Arch(arch.platform(), 'default_os', 'default_target')
on_cray = str(hostarch.platform) == 'cray'
using_cnl = re.match(r'cnl\d+', str(hostarch.os))
host_platform = spack.platforms.host()
host_os = host_platform.operating_system('default_os')
on_cray = str(host_platform) == 'cray'
using_cnl = re.match(r'cnl\d+', str(host_os))
return on_cray, using_cnl
@ -780,7 +781,9 @@ def setup_package(pkg, dirty, context='build'):
pkg.spec, context, custom_mods_only=False))
# architecture specific setup
pkg.architecture.platform.setup_platform_environment(pkg, env)
platform = spack.platforms.by_name(pkg.spec.architecture.platform)
target = platform.target(pkg.spec.architecture.target)
platform.setup_platform_environment(pkg, env)
if context == 'build':
pkg.setup_build_environment(env)
@ -813,8 +816,8 @@ def setup_package(pkg, dirty, context='build'):
if on_cray:
module('unload', 'cray-libsci')
if pkg.architecture.target.module_name:
load_module(pkg.architecture.target.module_name)
if target.module_name:
load_module(target.module_name)
load_external_modules(pkg)

27
lib/spack/spack/cmd/arch.py
View File

@ -12,7 +12,7 @@
import llnl.util.tty.colify as colify
import llnl.util.tty.color as color
import spack.architecture as architecture
import spack.platforms
description = "print architecture information about this machine"
section = "system"
@ -78,21 +78,24 @@ def arch(parser, args):
display_targets(archspec.cpu.TARGETS)
return
os_args, target_args = 'default_os', 'default_target'
if args.frontend:
arch = architecture.Arch(architecture.platform(),
'frontend', 'frontend')
os_args, target_args = 'frontend', 'frontend'
elif args.backend:
arch = architecture.Arch(architecture.platform(),
'backend', 'backend')
else:
arch = architecture.Arch(architecture.platform(),
'default_os', 'default_target')
os_args, target_args = 'backend', 'backend'
host_platform = spack.platforms.host()
host_os = host_platform.operating_system(os_args)
host_target = host_platform.target(target_args)
architecture = spack.spec.ArchSpec(
(str(host_platform), str(host_os), str(host_target))
)
if args.platform:
print(arch.platform)
print(architecture.platform)
elif args.operating_system:
print(arch.os)
print(architecture.os)
elif args.target:
print(arch.target)
print(architecture.target)
else:
print(arch)
print(architecture)

5
lib/spack/spack/cmd/buildcache.py
View File

@ -10,7 +10,6 @@
import llnl.util.tty as tty
import spack.architecture
import spack.binary_distribution as bindist
import spack.cmd
import spack.cmd.common.arguments as arguments
@ -337,7 +336,7 @@ def match_downloaded_specs(pkgs, allow_multiple_matches=False, force=False,
specs = bindist.update_cache_and_get_specs()
if not other_arch:
arch = spack.architecture.default_arch().to_spec()
arch = spack.spec.Spec.default_arch()
specs = [s for s in specs if s.satisfies(arch)]
for pkg in pkgs:
@ -563,7 +562,7 @@ def listspecs(args):
"""list binary packages available from mirrors"""
specs = bindist.update_cache_and_get_specs()
if not args.allarch:
arch = spack.architecture.default_arch().to_spec()
arch = spack.spec.Spec.default_arch()
specs = [s for s in specs if s.satisfies(arch)]
if args.specs:

11
lib/spack/spack/cmd/debug.py
View File

@ -14,9 +14,9 @@
import llnl.util.tty as tty
from llnl.util.filesystem import working_dir
import spack.architecture as architecture
import spack.config
import spack.paths
import spack.platforms
from spack.main import get_version
from spack.util.executable import which
@ -87,10 +87,15 @@ def create_db_tarball(args):
def report(args):
host_platform = spack.platforms.host()
host_os = host_platform.operating_system('frontend')
host_target = host_platform.target('frontend')
architecture = spack.spec.ArchSpec(
(str(host_platform), str(host_os), str(host_target))
)
print('* **Spack:**', get_version())
print('* **Python:**', platform.python_version())
print('* **Platform:**', architecture.Arch(
architecture.platform(), 'frontend', 'frontend'))
print('* **Platform:**', architecture)
print('* **Concretizer:**', spack.config.get('config:concretizer'))

1
lib/spack/spack/compiler.py
View File

@ -16,7 +16,6 @@
import llnl.util.tty as tty
from llnl.util.filesystem import path_contains_subdirectory, paths_containing_libs
import spack.architecture
import spack.compilers
import spack.error
import spack.spec

4
lib/spack/spack/compilers/__init__.py
View File

@ -20,11 +20,11 @@
import llnl.util.lang
import llnl.util.tty as tty
import spack.architecture
import spack.compiler
import spack.config
import spack.error
import spack.paths
import spack.platforms
import spack.spec
from spack.util.environment import get_path
from spack.util.naming import mod_to_class
@ -519,7 +519,7 @@ def all_os_classes():
"""
classes = []
platform = spack.architecture.platform()
platform = spack.platforms.host()
for os_class in platform.operating_sys.values():
classes.append(os_class)

4
lib/spack/spack/concretize.py
View File

@ -31,10 +31,10 @@
import llnl.util.tty as tty
import spack.abi
import spack.architecture
import spack.compilers
import spack.environment
import spack.error
import spack.platforms
import spack.repo
import spack.spec
import spack.target
@ -269,7 +269,7 @@ def concretize_architecture(self, spec):
new_plat = spack.platforms.by_name(platform_spec.architecture.platform)
else:
# If no platform anywhere in this spec, grab the default
new_plat = spack.architecture.platform()
new_plat = spack.platforms.host()
# Get nearest spec with relevant platform and an os
# Generally, same algorithm as finding platform, except we only

4
lib/spack/spack/config.py
View File

@ -47,9 +47,9 @@
import llnl.util.tty as tty
from llnl.util.filesystem import mkdirp
import spack.architecture
import spack.compilers
import spack.paths
import spack.platforms
import spack.schema
import spack.schema.bootstrap
import spack.schema.compilers
@ -769,7 +769,7 @@ def override(path_or_scope, value=None):
def _add_platform_scope(cfg, scope_type, name, path):
"""Add a platform-specific subdirectory for the current platform."""
platform = spack.architecture.platform().name
platform = spack.platforms.host().name
plat_name = '%s/%s' % (name, platform)
plat_path = os.path.join(path, platform)
cfg.push_scope(scope_type(plat_name, plat_path))

27
lib/spack/spack/main.py
View File

@ -26,19 +26,21 @@
import archspec.cpu
import llnl.util.filesystem as fs
import llnl.util.lang
import llnl.util.tty as tty
import llnl.util.tty.colify
import llnl.util.tty.color as color
from llnl.util.tty.log import log_output
import spack
import spack.architecture
import spack.cmd
import spack.config
import spack.environment as ev
import spack.modules
import spack.paths
import spack.platforms
import spack.repo
import spack.spec
import spack.store
import spack.util.debug
import spack.util.executable as exe
@ -644,6 +646,23 @@ def _profile_wrapper(command, parser, args, unknown_args):
stats.print_stats(nlines)
@llnl.util.lang.memoized
def _compatible_sys_types():
"""Return a list of all the platform-os-target tuples compatible
with the current host.
"""
host_platform = spack.platforms.host()
host_os = str(host_platform.operating_system('default_os'))
host_target = archspec.cpu.host()
compatible_targets = [host_target] + host_target.ancestors
compatible_archs = [
str(spack.spec.ArchSpec((str(host_platform), host_os, str(target))))
for target in compatible_targets
]
return compatible_archs
def print_setup_info(*info):
"""Print basic information needed by setup-env.[c]sh.
@ -653,7 +672,6 @@ def print_setup_info(*info):
This is in ``main.py`` to make it fast; the setup scripts need to
invoke spack in login scripts, and it needs to be quick.
"""
shell = 'csh' if 'csh' in info else 'sh'
@ -666,9 +684,8 @@ def shell_set(var, value):
tty.die('shell must be sh or csh')
# print sys type
shell_set('_sp_sys_type', str(spack.architecture.default_arch()))
shell_set('_sp_compatible_sys_types',
':'.join(spack.architecture.compatible_sys_types()))
shell_set('_sp_sys_type', str(spack.spec.ArchSpec.default_arch()))
shell_set('_sp_compatible_sys_types', ':'.join(_compatible_sys_types()))
# print roots for all module systems
module_to_roots = {
'tcl': list(),

18
lib/spack/spack/operating_systems/_operating_system.py
View File

@ -11,10 +11,22 @@
class OperatingSystem(object):
"""Base class for all the Operating Systems.
Each Operating System contain its own compiler finding logic, that is used
to detect compilers.
"""
On a multiple architecture machine, the architecture spec field can be set to
build a package against any target and operating system that is present on the
platform. On Cray platforms or any other architecture that has different front
and back end environments, the operating system will determine the method of
compiler detection.
There are two different types of compiler detection:
1. Through the $PATH env variable (front-end detection)
2. Through the module system. (back-end detection)
Depending on which operating system is specified, the compiler will be detected
using one of those methods.
For platforms such as linux and darwin, the operating system is autodetected.
"""
def __init__(self, name, version):
self.name = name.replace('-', '_')
self.version = str(version).replace('-', '_')

8
lib/spack/spack/package.py
View File

@ -1266,14 +1266,6 @@ def prefix(self):
"""Get the prefix into which this package should be installed."""
return self.spec.prefix
@property
def architecture(self):
"""Get the spack.architecture.Arch object that represents the
environment in which this package will be built."""
if not self.spec.concrete:
raise ValueError("Can only get the arch for concrete package.")
return spack.architecture.arch_for_spec(self.spec.architecture)
@property # type: ignore
@memoized
def compiler(self):

79
lib/spack/spack/platforms/__init__.py
View File

@ -2,8 +2,9 @@
# Spack Project Developers. See the top-level COPYRIGHT file for details.
#
# SPDX-License-Identifier: (Apache-2.0 OR MIT)
import llnl.util.lang
import contextlib
from ._functions import _host, by_name, platforms
from ._platform import Platform
from .cray import Cray
from .darwin import Darwin
@ -15,41 +16,57 @@
'Cray',
'Darwin',
'Linux',
'Test'
'Test',
'platforms',
'host',
'by_name'
]
#: List of all the platform classes known to Spack
platforms = [Cray, Darwin, Linux, Test]
#: The "real" platform of the host running Spack. This should not be changed
#: by any method and is here as a convenient way to refer to the host platform.
real_host = _host
#: The current platform used by Spack. May be swapped by the use_platform
#: context manager.
host = _host
def host():
"""Detect and return the platform for this machine or None if detection fails."""
for platform_cls in sorted(platforms, key=lambda plt: plt.priority):
if platform_cls.detect():
return platform_cls()
return None
@llnl.util.lang.memoized
def cls_by_name(name):
"""Return a platform class that corresponds to the given name or None
if there is no match.
Args:
name (str): name of the platform
class _PickleableCallable(object):
"""Class used to pickle a callable that may substitute either
_platform or _all_platforms. Lambda or nested functions are
not pickleable.
"""
for platform_cls in sorted(platforms, key=lambda plt: plt.priority):
if name.replace("_", "").lower() == platform_cls.__name__.lower():
return platform_cls
return None
def __init__(self, return_value):
self.return_value = return_value
def __call__(self):
return self.return_value
def by_name(name):
"""Return a platform object that corresponds to the given name or None
if there is no match.
@contextlib.contextmanager
def use_platform(new_platform):
global host
Args:
name (str): name of the platform
"""
platform_cls = cls_by_name(name)
return platform_cls() if platform_cls else None
import spack.compilers
import spack.config
msg = '"{0}" must be an instance of Platform'
assert isinstance(new_platform, Platform), msg.format(new_platform)
original_host_fn = host
try:
host = _PickleableCallable(new_platform)
# Clear configuration and compiler caches
spack.config.config.clear_caches()
spack.compilers._cache_config_files = []
yield new_platform
finally:
host = original_host_fn
# Clear configuration and compiler caches
spack.config.config.clear_caches()
spack.compilers._cache_config_files = []

46
lib/spack/spack/platforms/_functions.py Normal file
View File

@ -0,0 +1,46 @@
# Copyright 2013-2021 Lawrence Livermore National Security, LLC and other
# Spack Project Developers. See the top-level COPYRIGHT file for details.
#
# SPDX-License-Identifier: (Apache-2.0 OR MIT)
import llnl.util.lang
from .cray import Cray
from .darwin import Darwin
from .linux import Linux
from .test import Test
#: List of all the platform classes known to Spack
platforms = [Cray, Darwin, Linux, Test]
def _host():
"""Detect and return the platform for this machine or None if detection fails."""
for platform_cls in sorted(platforms, key=lambda plt: plt.priority):
if platform_cls.detect():
return platform_cls()
return None
@llnl.util.lang.memoized
def cls_by_name(name):
"""Return a platform class that corresponds to the given name or None
if there is no match.
Args:
name (str): name of the platform
"""
for platform_cls in sorted(platforms, key=lambda plt: plt.priority):
if name.replace("_", "").lower() == platform_cls.__name__.lower():
return platform_cls
return None
def by_name(name):
"""Return a platform object that corresponds to the given name or None
if there is no match.
Args:
name (str): name of the platform
"""
platform_cls = cls_by_name(name)
return platform_cls() if platform_cls else None

21
lib/spack/spack/platforms/_platform.py
View File

@ -15,7 +15,26 @@ def __init__(self):
@llnl.util.lang.lazy_lexicographic_ordering
class Platform(object):
"""Base class for each type of Platform"""
"""Platform is an abstract class extended by subclasses.
To add a new type of platform (such as cray_xe), create a subclass and set all the
class attributes such as priority, front_target, back_target, front_os, back_os.
Platform also contain a priority class attribute. A lower number signifies higher
priority. These numbers are arbitrarily set and can be changed though often there
isn't much need unless a new platform is added and the user wants that to be
detected first.
Targets are created inside the platform subclasses. Most architecture (like linux,
and darwin) will have only one target family (x86_64) but in the case of Cray
machines, there is both a frontend and backend processor. The user can specify
which targets are present on front-end and back-end architecture.
Depending on the platform, operating systems are either autodetected or are
set. The user can set the frontend and backend operating setting by the class
attributes front_os and back_os. The operating system will be responsible for
compiler detection.
"""
# Subclass sets number. Controls detection order
priority = None # type: int

4
lib/spack/spack/relocate.py
View File

@ -15,7 +15,7 @@
import llnl.util.lang
import llnl.util.tty as tty
import spack.architecture
import spack.platforms
import spack.repo
import spack.spec
import spack.util.executable as executable
@ -94,7 +94,7 @@ def _patchelf():
return exe_path
# Skip darwin
if str(spack.architecture.platform()) == 'darwin':
if str(spack.platforms.host()) == 'darwin':
return None
# Install the spec and return its path

8
lib/spack/spack/solver/asp.py
View File

@ -30,7 +30,6 @@
import llnl.util.tty as tty
import spack
import spack.architecture
import spack.bootstrap
import spack.cmd
import spack.compilers
@ -41,6 +40,7 @@
import spack.error
import spack.package
import spack.package_prefs
import spack.platforms
import spack.repo
import spack.spec
import spack.util.timer
@ -1086,12 +1086,12 @@ def _supported_targets(self, compiler_name, compiler_version, targets):
def platform_defaults(self):
self.gen.h2('Default platform')
platform = spack.architecture.platform()
platform = spack.platforms.host()
self.gen.fact(fn.node_platform_default(platform))
def os_defaults(self, specs):
self.gen.h2('Possible operating systems')
platform = spack.architecture.platform()
platform = spack.platforms.host()
# create set of OS's to consider
possible = set([
@ -1111,7 +1111,7 @@ def target_defaults(self, specs):
"""Add facts about targets and target compatibility."""
self.gen.h2('Default target')
platform = spack.architecture.platform()
platform = spack.platforms.host()
uarch = archspec.cpu.TARGETS.get(platform.default)
self.gen.h2('Target compatibility')

35
lib/spack/spack/spec.py
View File

@ -92,7 +92,6 @@
import llnl.util.tty as tty
import llnl.util.tty.color as clr
import spack.architecture
import spack.compiler
import spack.compilers
import spack.config
@ -213,6 +212,27 @@ def __call__(self, match):
@lang.lazy_lexicographic_ordering
class ArchSpec(object):
"""Aggregate the target platform, the operating system and the target
microarchitecture into an architecture spec..
"""
@staticmethod
def _return_arch(os_tag, target_tag):
platform = spack.platforms.host()
default_os = platform.operating_system(os_tag)
default_target = platform.target(target_tag)
arch_tuple = str(platform), str(default_os), str(default_target)
return ArchSpec(arch_tuple)
@staticmethod
def default_arch():
"""Return the default architecture"""
return ArchSpec._return_arch('default_os', 'default_target')
@staticmethod
def frontend_arch():
"""Return the frontend architecture"""
return ArchSpec._return_arch('frontend', 'frontend')
def __init__(self, spec_or_platform_tuple=(None, None, None)):
""" Architecture specification a package should be built with.
@ -294,7 +314,7 @@ def os(self, value):
value = str(value) if value is not None else None
if value in spack.platforms.Platform.reserved_oss:
curr_platform = str(spack.architecture.platform())
curr_platform = str(spack.platforms.host())
self.platform = self.platform or curr_platform
if self.platform != curr_platform:
@ -331,7 +351,7 @@ def target_or_none(t):
value = target_or_none(value)
if str(value) in spack.platforms.Platform.reserved_targets:
curr_platform = str(spack.architecture.platform())
curr_platform = str(spack.platforms.host())
self.platform = self.platform or curr_platform
if self.platform != curr_platform:
@ -1020,6 +1040,13 @@ class Spec(object):
#: Cache for spec's prefix, computed lazily in the corresponding property
_prefix = None
@staticmethod
def default_arch():
"""Return an anonymous spec for the default architecture"""
s = Spec()
s.architecture = ArchSpec.default_arch()
return s
def __init__(self, spec_like=None, normal=False,
concrete=False, external_path=None, external_modules=None):
"""Create a new Spec.
@ -1242,7 +1269,7 @@ def _add_default_platform(self):
"""
arch = self.architecture
if arch and not arch.platform and (arch.os or arch.target):
self._set_architecture(platform=spack.architecture.platform().name)
self._set_architecture(platform=spack.platforms.host().name)
#
# Public interface

6
lib/spack/spack/subprocess_context.py
View File

@ -20,10 +20,10 @@
import sys
from types import ModuleType
import spack.architecture
import spack.config
import spack.environment
import spack.main
import spack.platforms
import spack.repo
import spack.store
@ -96,7 +96,7 @@ def __init__(self):
if _serialize:
self.repo_dirs = list(r.root for r in spack.repo.path.repos)
self.config = spack.config.config
self.platform = spack.architecture.platform
self.platform = spack.platforms.host
self.test_patches = store_patches()
self.store_token = spack.store.store.serialize()
@ -104,7 +104,7 @@ def restore(self):
if _serialize:
spack.repo.path = spack.repo._path(self.repo_dirs)
spack.config.config = self.config
spack.architecture.platform = self.platform
spack.platforms.host = self.platform
new_store = spack.store.Store.deserialize(self.store_token)
spack.store.store = new_store

42
lib/spack/spack/test/architecture.py
View File

@ -7,7 +7,6 @@
import pytest
import spack.architecture
import spack.concretize
import spack.operating_systems
import spack.platforms
@ -15,16 +14,6 @@
import spack.target
@pytest.fixture
def sample_arch():
"""Sample test architecture"""
arch = spack.architecture.Arch()
arch.platform = spack.architecture.platform()
arch.os = arch.platform.operating_system('default_os')
arch.target = arch.platform.target('default_target')
return arch
@pytest.fixture
def current_host_platform():
"""Return the platform of the current host as detected by the
@ -61,41 +50,12 @@ def os_str(request):
return str(request.param)
def test_dict_round_trip(sample_arch):
"""Check that a round trip through dict return an equivalent architecture"""
sample_arch_copy = spack.architecture.Arch.from_dict(sample_arch.to_dict())
assert sample_arch == sample_arch_copy
for current_arch in (sample_arch, sample_arch_copy):
assert isinstance(current_arch, spack.architecture.Arch)
assert isinstance(current_arch.platform, spack.platforms.Platform)
assert isinstance(current_arch.os, spack.operating_systems.OperatingSystem)
assert isinstance(current_arch.target, spack.target.Target)
def test_platform(current_host_platform):
"""Check that current host detection return the correct platform"""
detected_platform = spack.architecture.real_platform()
detected_platform = spack.platforms.real_host()
assert str(detected_platform) == str(current_host_platform)
@pytest.mark.parametrize('attribute_name,expected', [
# Make sure architecture reports that it's False when nothing's set.
(None, False),
# ... and it reports True otherwise
('platform', True),
('os', True),
('target', True)
])
def test_boolness(sample_arch, attribute_name, expected):
"""Boolean checks on an architecture object"""
arch = spack.architecture.Arch()
if attribute_name:
setattr(arch, attribute_name, getattr(sample_arch, attribute_name))
assert bool(arch) is expected
def test_user_input_combination(config, target_str, os_str):
"""Test for all the valid user input combinations that both the target and
the operating system match.

1
lib/spack/spack/test/build_systems.py
View File

@ -10,7 +10,6 @@
import llnl.util.filesystem as fs
import spack.architecture
import spack.environment
import spack.repo
from spack.build_environment import ChildError, get_std_cmake_args, setup_package

11
lib/spack/spack/test/cmd/debug.py
View File

@ -9,8 +9,8 @@
import pytest
import spack.architecture as architecture
import spack.config
import spack.platforms
from spack.main import SpackCommand, get_version
from spack.util.executable import which
@ -48,9 +48,14 @@ def test_create_db_tarball(tmpdir, database):
def test_report():
out = debug('report')
arch = architecture.Arch(architecture.platform(), 'frontend', 'frontend')
host_platform = spack.platforms.host()
host_os = host_platform.operating_system('frontend')
host_target = host_platform.target('frontend')
architecture = spack.spec.ArchSpec(
(str(host_platform), str(host_os), str(host_target))
)
assert get_version() in out
assert platform.python_version() in out
assert str(arch) in out
assert str(architecture) in out
assert spack.config.get('config:concretizer') in out

3
lib/spack/spack/test/concretize.py
View File

@ -11,7 +11,6 @@
import llnl.util.lang
import spack.architecture
import spack.compilers
import spack.concretize
import spack.error
@ -428,7 +427,7 @@ def test_external_package(self):
def test_external_package_module(self):
# No tcl modules on darwin/linux machines
# TODO: improved way to check for this.
platform = spack.architecture.real_platform().name
platform = spack.platforms.real_host().name
if platform == 'darwin' or platform == 'linux':
return

6
lib/spack/spack/test/conftest.py
View File

@ -24,7 +24,6 @@
from llnl.util.filesystem import mkdirp, remove_linked_tree, working_dir
import spack.architecture
import spack.binary_distribution
import spack.caches
import spack.compilers
@ -450,7 +449,7 @@ def _use_test_platform(test_platform):
# This is the only context manager used at session scope (see note
# below for more insight) since we want to use the test platform as
# a default during tests.
with spack.architecture.use_platform(test_platform):
with spack.platforms.use_platform(test_platform):
yield
#
@ -515,10 +514,9 @@ def linux_os():
"""Returns a named tuple with attributes 'name' and 'version'
representing the OS.
"""
platform = spack.architecture.platform()
platform = spack.platforms.host()
name, version = 'debian', '6'
if platform.name == 'linux':
platform = spack.architecture.platform()
current_os = platform.operating_system('default_os')
name, version = current_os.name, current_os.version
LinuxOS = collections.namedtuple('LinuxOS', ['name', 'version'])

3
lib/spack/spack/test/multimethod.py
View File

@ -6,6 +6,7 @@
"""Test for multi_method dispatch."""
import pytest
import spack.platforms
import spack.repo
from spack.multimethod import NoSuchMethodError
@ -79,7 +80,7 @@ def test_default_works(pkg_name):
def test_target_match(pkg_name):
platform = spack.architecture.platform()
platform = spack.platforms.host()
targets = list(platform.targets.values())
for target in targets[:-1]:
pkg = spack.repo.get(pkg_name + ' target=' + target.name)

1
lib/spack/spack/test/relocate.py
View File

@ -12,7 +12,6 @@
import llnl.util.filesystem
import spack.architecture
import spack.concretize
import spack.paths
import spack.relocate

1
lib/spack/spack/test/spec_dag.py
View File

@ -7,7 +7,6 @@
"""
import pytest
import spack.architecture
import spack.error
import spack.package
import spack.util.hash as hashutil

9
lib/spack/spack/test/spec_semantics.py
View File

@ -7,7 +7,6 @@
import pytest
import spack.architecture
import spack.directives
import spack.error
from spack.error import SpecError, UnsatisfiableSpecError
@ -690,7 +689,7 @@ def test_constrain_changed(self):
check_constrain_changed('libelf', 'debug=2')
check_constrain_changed('libelf', 'cppflags="-O3"')
platform = spack.architecture.platform()
platform = spack.platforms.host()
check_constrain_changed(
'libelf', 'target=' + platform.target('default_target').name)
check_constrain_changed(
@ -709,7 +708,7 @@ def test_constrain_not_changed(self):
check_constrain_not_changed(
'libelf cppflags="-O3"', 'cppflags="-O3"')
platform = spack.architecture.platform()
platform = spack.platforms.host()
default_target = platform.target('default_target').name
check_constrain_not_changed(
'libelf target=' + default_target, 'target=' + default_target)
@ -723,7 +722,7 @@ def test_constrain_dependency_changed(self):
check_constrain_changed('libelf^foo', 'libelf^foo~debug')
check_constrain_changed('libelf', '^foo')
platform = spack.architecture.platform()
platform = spack.platforms.host()
default_target = platform.target('default_target').name
check_constrain_changed(
'libelf^foo', 'libelf^foo target=' + default_target)
@ -742,7 +741,7 @@ def test_constrain_dependency_not_changed(self):
check_constrain_not_changed(
'libelf^foo cppflags="-O3"', 'libelf^foo cppflags="-O3"')
platform = spack.architecture.platform()
platform = spack.platforms.host()
default_target = platform.target('default_target').name
check_constrain_not_changed(
'libelf^foo target=' + default_target,

2
lib/spack/spack/test/spec_yaml.py
View File

@ -15,7 +15,6 @@
import pytest
import spack.architecture
import spack.hash_types as ht
import spack.spec
import spack.util.spack_json as sjson
@ -226,7 +225,6 @@ def test_ordered_read_not_required_for_consistent_dag_hash(
@pytest.mark.parametrize("module", [
spack.spec,
spack.architecture,
spack.version,
])
def test_hashes_use_no_python_dicts(module):

23
lib/spack/spack/util/environment.py
View File

@ -2,7 +2,6 @@
# Spack Project Developers. See the top-level COPYRIGHT file for details.
#
# SPDX-License-Identifier: (Apache-2.0 OR MIT)
"""Utilities for setting and modifying environment variables."""
import collections
import contextlib
@ -22,6 +21,8 @@
import llnl.util.tty as tty
from llnl.util.lang import dedupe
import spack.platforms
import spack.spec
import spack.util.executable as executable
system_paths = ['/', '/usr', '/usr/local']
@ -158,18 +159,20 @@ def get_host_environment():
"""Return a dictionary (lookup) with host information (not including the
os.environ).
"""
import spack.architecture as architecture
import spack.spec
arch = architecture.Arch(
architecture.platform(), 'default_os', 'default_target')
arch_spec = spack.spec.Spec('arch=%s' % arch)
host_platform = spack.platforms.host()
host_target = host_platform.target('default_target')
host_os = host_platform.operating_system('default_os')
arch_fmt = 'platform={0} os={1} target={2}'
arch_spec = spack.spec.Spec(
arch_fmt.format(host_platform, host_os, host_target)
)
return {
'target': str(arch.target),
'os': str(arch.os),
'platform': str(arch.platform),
'target': str(host_target),
'os': str(host_os),
'platform': str(host_platform),
'arch': arch_spec,
'architecture': arch_spec,
'arch_str': str(arch),
'arch_str': str(arch_spec),
'hostname': socket.gethostname()
}

4
var/spack/repos/builtin.mock/packages/multimethod/package.py
View File

@ -5,7 +5,7 @@
from six import string_types
import spack.architecture
import spack.platforms
from spack import *
from spack.pkg.builtin.mock.multimethod_base import MultimethodBase
@ -83,7 +83,7 @@ def has_a_default(self):
#
# Make sure we can switch methods on different target
#
platform = spack.architecture.platform()
platform = spack.platforms.host()
targets = list(platform.targets.values())
if len(targets) > 1:
targets = targets[:-1]

2
var/spack/repos/builtin/packages/arm/package.py
View File

@ -43,7 +43,7 @@
def get_os():
spack_os = spack.architecture.platform().default_os
spack_os = spack.platforms.host().default_os
return _os_map.get(spack_os, 'RHEL-7')

4
var/spack/repos/builtin/packages/gcc/package.py
View File

@ -10,7 +10,7 @@
import llnl.util.tty as tty
import spack.architecture
import spack.platforms
import spack.util.executable
from spack.operating_systems.mac_os import macos_sdk_path, macos_version
@ -324,7 +324,7 @@ def filter_detected_exes(cls, prefix, exes_in_prefix):
continue
# Filter out links in favor of real executables on
# all systems but Cray
host_platform = str(spack.architecture.platform())
host_platform = str(spack.platforms.host())
if os.path.islink(exe) and host_platform != 'cray':
continue

8
var/spack/repos/builtin/packages/sqlite/package.py
View File

@ -2,12 +2,11 @@
# Spack Project Developers. See the top-level COPYRIGHT file for details.
#
# SPDX-License-Identifier: (Apache-2.0 OR MIT)
import os
import re
from tempfile import NamedTemporaryFile
from spack import architecture
import spack.platforms
class Sqlite(AutotoolsPackage):
@ -168,9 +167,8 @@ def libs(self):
return find_libraries('libsqlite3', root=self.prefix.lib)
def get_arch(self):
arch = architecture.Arch()
arch.platform = architecture.platform()
return str(arch.platform.target('default_target'))
host_platform = spack.platforms.host()
return str(host_platform.target('default_target'))
def configure_args(self):
args = []

2
var/spack/repos/builtin/packages/upcxx/package.py
View File

@ -7,7 +7,7 @@
def cross_detect():
if spack.architecture.platform().name == 'cray':
if spack.platforms.host().name == 'cray':
if which('srun'):
return 'cray-aries-slurm'
if which('aprun'):