Make GHA tests parallel by using xdist (#32361)

* Add two no-op jobs named "all-prechecks" and "all"

These are a suggestion from @tgamblin, they are stable named markers we
can use from gitlab and possibly for required checks to make CI more
resilient to refactors changing the names of specific checks.

* Enable parallel testing using xdist for unit testing in CI

* Normalize tmp paths to deal with macos

* add -u flag compatibility to spack python

As of now, it is accepted and ignored.  The usage with xdist, where it
is invoked specifically by `python -u spack python` which is then passed
`-u` by xdist is the entire reason for doing this.  It should never be
used without explicitly passing -u to the executing python interpreter.

* use spack python in xdist to support python 2

When running on python2, spack has many import cycles unless started
through main.  To allow that, this uses `spack python` as the
interpreter, leveraging the `-u` support so xdist doesn't error out when
it unconditionally requests unbuffered binary IO.

* Use shutil.move to account for tmpdir being in a separate filesystem sometimes
This commit is contained in:
Tom Scogland
2022-09-07 11:12:57 -07:00
committed by GitHub
parent 8e5ccddc13
commit 762ba27036
25 changed files with 221 additions and 123 deletions

View File

@@ -7,7 +7,6 @@ export TMPDIR="${XDG_RUNTIME_DIR}"
export TMP_DIR="$(mktemp -d -t spack-test-XXXXX)"
clean_up() {
[[ -n "$TMPCONFIG_DEBUG" ]] && printf "cleaning up: $TMP_DIR\n"
[[ -n "$TMPCONFIG_DEBUG" ]] && tree "$TMP_DIR"
rm -rf "$TMP_DIR"
}
trap clean_up EXIT