This PR provides complementary 2 features:
1. An augmentation to the package language to express ABI compatibility relationships among packages.
2. An extension to the concretizer that can synthesize splices between ABI compatible packages.
1. The `can_splice` directive and ABI compatibility
We augment the package language with a single directive: `can_splice`. Here is an example of a package `Foo` exercising the `can_splice` directive:
class Foo(Package):
version("1.0")
version("1.1")
variant("compat", default=True)
variant("json", default=False)
variant("pic", default=False)
can_splice("foo@1.0", when="@1.1")
can_splice("bar@1.0", when="@1.0+compat")
can_splice("baz@1.0+compat", when="@1.0+compat", match_variants="*")
can_splice("quux@1.0", when=@1.1~compat", match_variants="json")
Explanations of the uses of each directive:
- `can_splice("foo@1.0", when="@1.1")`: If `foo@1.0` is the dependency of an already installed spec and `foo@1.1` could be a valid dependency for the parent spec, then `foo@1.1` can be spliced in for `foo@1.0` in the parent spec.
- `can_splice("bar@1.0", when="@1.0+compat")`: If `bar@1.0` is the dependency of an already installed spec and `foo@1.0+compat` could be a valid dependency for the parent spec, then `foo@1.0+compat` can be spliced in for `bar@1.0+compat` in the parent spec
- `can_splice("baz@1.0", when="@1.0+compat", match_variants="*")`: If `baz@1.0+compat` is the dependency of an already installed spec and `foo@1.0+compat` could be a valid dependency for the parent spec, then `foo@1.0+compat` can be spliced in for `baz@1.0+compat` in the parent spec, provided that they have the same value for all other variants (regardless of what those values are).
- `can_splice("quux@1.0", when=@1.1~compat", match_variants="json")`:If `quux@1.0` is the dependency of an already installed spec and `foo@1.1~compat` could be a valid dependency for the parent spec, then `foo@1.0~compat` can be spliced in for `quux@1.0` in the parent spec, provided that they have the same value for their `json` variant.
2. Augmenting the solver to synthesize splices
### Changes to the hash encoding in `asp.py`
Previously, when including concrete specs in the solve, they would have the following form:
installed_hash("foo", "xxxyyy")
imposed_constraint("xxxyyy", "foo", "attr1", ...)
imposed_constraint("xxxyyy", "foo", "attr2", ...)
% etc.
Concrete specs now have the following form:
installed_hash("foo", "xxxyyy")
hash_attr("xxxyyy", "foo", "attr1", ...)
hash_attr("xxxyyy", "foo", "attr2", ...)
This transformation allows us to control which constraints are imposed when we select a hash, to facilitate the splicing of dependencies.
2.1 Compiling `can_splice` directives in `asp.py`
Consider the concrete spec:
foo@2.72%gcc@11.4 arch=linux-ubuntu22.04-icelake build_system=autotools ^bar ...
It will emit the following facts for reuse (below is a subset)
installed_hash("foo", "xxxyyy")
hash_attr("xxxyyy", "hash", "foo", "xxxyyy")
hash_attr("xxxyyy", "version", "foo", "2.72")
hash_attr("xxxyyy", "node_os", "ubuntu22.04")
hash_attr("xxxyyy", "hash", "bar", "zzzqqq")
hash_attr("xxxyyy", "depends_on", "foo", "bar", "link")
Rules that derive abi_splice_conditions_hold will be generated from
use of the `can_splice` directive. They will have the following form:
can_splice("foo@1.0.0+a", when="@1.0.1+a", match_variants=["b"]) --->
abi_splice_conditions_hold(0, node(SID, "foo"), "foo", BaseHash) :-
installed_hash("foo", BaseHash),
attr("node", node(SID, SpliceName)),
attr("node_version_satisfies", node(SID, "foo"), "1.0.1"),
hash_attr("hash", "node_version_satisfies", "foo", "1.0.1"),
attr("variant_value", node(SID, "foo"), "a", "True"),
hash_attr("hash", "variant_value", "foo", "a", "True"),
attr("variant_value", node(SID, "foo"), "b", VariVar0),
hash_attr("hash", "variant_value", "foo", "b", VariVar0).
2.2 Synthesizing splices in `concretize.lp` and `splices.lp`
The ASP solver generates "splice_at_hash" attrs to indicate that a particular node has a splice in one of its immediate dependencies.
Splices can be introduced in the dependencies of concrete specs when `splices.lp` is conditionally loaded (based on the config option `concretizer:splice:True`.
2.3 Constructing spliced specs in `asp.py`
The method `SpecBuilder._resolve_splices` implements a top-down memoized implementation of hybrid splicing. This is an optimization over the more general `Spec.splice`, since the solver gives a global view of exactly which specs can be shared, to ensure the minimal number of splicing operations.
Misc changes to facilitate configuration and benchmarking
- Added the method `Solver.solve_with_stats` to expose timers from the public interface for easier benchmarking
- Added the boolean config option `concretizer:splice` to conditionally load splicing behavior
Co-authored-by: Greg Becker <becker33@llnl.gov>
We added unification semantics for parsing specs from the CLI, but there are a couple
of special cases in which we can avoid calls to the concretizer for speed when the
specs can all be resolved by lookups.
- [x] special case 1: solving a single spec
- [x] special case 2: all specs are either concrete (come from a file) or have an abstract
hash. In this case if concretizer:unify:true we need an additional check to confirm
the specs are compatible.
- [x] add a parameterized test for unifying on the CI
---------
Signed-off-by: Todd Gamblin <tgamblin@llnl.gov>
Co-authored-by: Todd Gamblin <tgamblin@llnl.gov>
* libunwind: Add 1.7.2 and 1.8.1
* libunwind: Remove deprecated 1.1 version
* libunwind: Add newer *-stable branches: Remove 1.5-stable branch as well as cleanup.
* libunwind: Use GitHub url for all versions
* libunwind: Add conflict for PPC and 1.8.*
* libunwind: Add conflict for aarch64 and 1.8:
Build fails with
aarch64/Gos-linux.c: In function '_ULaarch64_local_resume':
aarch64/Gos-linux.c:147:1: error: x29 cannot be used in asm here
}
^
aarch64/Gos-linux.c:147:1: error: x29 cannot be used in asm here
make[2]: *** [Makefile:4795: aarch64/Los-linux.lo] Error 1
* added updated versions
* added pyhmmer
* updated infernal
* fix blast-plus for apple-clang
* fix py-biopython build on apple-clang
* remove erroneous biopython dep: build issue is with python 3.8, not biopython
* deepsig python 3.9: expanding unnecessary python restrictions
* add pyrodigal
* fix unnecessarily strict diamond version
* builds and updates: blast-plus indexing broken, still need to test db download and bakta pipeline
* builds and runs
* revert blast-plus changes: remove my personal hacks to get blast-plus to build
* fix the build error during compilation of rocdecode.was dependent on libva-devel packag
* address review comment
* address review changes.commit the changes
* Add two_level_namespace variant (default is disabled) for MacOS to enable building
executables and libraries with two level namespace enabled.
* Addressed reviewer comments.
* Moved two_level_namespace variant ahead of the patch that uses that variant to
get concretize to work properly.
* Removed extra print statements
* soqt: Add SoQt package
The geomodel package needs this if visualization is turned on.
* make qt versions explicit
* use virtual dependency for qt
* pr feedback
Remove myself as maintainer
Remove v1.6.0
Remove unused qt variant
This addresses part [1] of #46345#44713 introduced a bug where all non-spec query parameters like date
ranges, -x, etc. were ignored when an env was active.
This fixes that issue and adds tests for it.
---------
Co-authored-by: Harmen Stoppels <me@harmenstoppels.nl>
