CI boilerplate reduction (#34272)

* CI configuration boilerplate reduction and refactor

Configuration:
- New notation for list concatenation (prepend/append)
- New notation for string concatenation (prepend/append)
- Break out configuration files for: ci.yaml, cdash.yaml, view.yaml
- Spack CI section refactored to improve self-consistency and
composability
  - Scripts are now lists of lists and/or lists of strings
  - Job attributes are now listed under precedence ordered list that are
  composed/merged using Spack config merge rules.
  - "service-jobs" are identified explicitly rather than as a batch

CI:
- Consolidate common, platform, and architecture configurations for all CI stacks into composable configuration files
- Make padding consistent across all stacks (256)
- Merge all package -> runner mappings to be consistent across all
stacks

Unit Test:
- Refactor CI module unit-tests for refactor configuration

Docs:
- Add docs for new notations in configuration.rst
- Rewrite docs on CI pipelines to be consistent with refactored CI
workflow

* Script verbose environ, dev bootstrap

* Port #35409

share/spack/gitlab/cloud_pipelines/configs/ci.yaml

@@ -0,0 +1,47 @@
+ci:
+  target: gitlab
+
+  broken-tests-packages:
+    - gptune
+
+  broken-specs-url: "https://dummy.io" # s3://spack-binaries/broken-specs"
+
+  pipeline-gen:
+  - build-job:
+      script::
+      - - spack compiler find
+        - cd ${SPACK_CONCRETE_ENV_DIR}
+        - spack env activate --without-view .
+        - spack config add "config:install_tree:projections:${SPACK_JOB_SPEC_PKG_NAME}:'morepadding/{architecture}/{compiler.name}-{compiler.version}/{name}-{version}-{hash}'"
+        - mkdir -p ${SPACK_ARTIFACTS_ROOT}/user_data
+        # AWS runners mount E4S public key (verification), UO runners mount public/private (signing/verification)
+        - if [[ -r /mnt/key/e4s.gpg ]]; then spack gpg trust /mnt/key/e4s.gpg; fi
+        # UO runners mount intermediate ci public key (verification), AWS runners mount public/private (signing/verification)
+        - if [[ -r /mnt/key/intermediate_ci_signing_key.gpg ]]; then spack gpg trust /mnt/key/intermediate_ci_signing_key.gpg; fi
+        - if [[ -r /mnt/key/spack_public_key.gpg ]]; then spack gpg trust /mnt/key/spack_public_key.gpg; fi
+        - spack --color=always --backtrace ci rebuild --tests > >(tee ${SPACK_ARTIFACTS_ROOT}/user_data/pipeline_out.txt) 2> >(tee ${SPACK_ARTIFACTS_ROOT}/user_data/pipeline_err.txt >&2)
+      after_script:
+      - - cat /proc/loadavg || true
+      variables:
+        CI_JOB_SIZE: "default"
+        SPACK_VERBOSE_SCRIPT: "1"
+
+  - signing-job:
+      image: { "name": "ghcr.io/spack/notary:latest", "entrypoint": [""] }
+      tags: ["aws"]
+      script:
+      - - aws s3 sync --exclude "*" --include "*spec.json*" ${SPACK_REMOTE_MIRROR_OVERRIDE}/build_cache /tmp
+        - /sign.sh
+        - aws s3 sync --exclude "*" --include "*spec.json.sig*" /tmp ${SPACK_REMOTE_MIRROR_OVERRIDE}/build_cache
+        - aws s3 cp /tmp/public_keys ${SPACK_REMOTE_MIRROR_OVERRIDE}/build_cache/_pgp --recursive --exclude "*"" --include "*.pub"
+
+  - any-job:
+      image: "ghcr.io/spack/e4s-ubuntu-18.04:v2021-10-18"
+      tags: ["spack"]
+      before_script:
+      - - uname -a || true
+        - grep -E "vendor|model name" /proc/cpuinfo 2>/dev/null | sort -u || head -n10 /proc/cpuinfo 2>/dev/null || true
+        - nproc
+      - - . "./share/spack/setup-env.sh"
+        - spack --version
+        - spack arch