Make GHA tests parallel by using xdist (#32361)

* Add two no-op jobs named "all-prechecks" and "all"

These are a suggestion from @tgamblin, they are stable named markers we
can use from gitlab and possibly for required checks to make CI more
resilient to refactors changing the names of specific checks.

* Enable parallel testing using xdist for unit testing in CI

* Normalize tmp paths to deal with macos

* add -u flag compatibility to spack python

As of now, it is accepted and ignored.  The usage with xdist, where it
is invoked specifically by `python -u spack python` which is then passed
`-u` by xdist is the entire reason for doing this.  It should never be
used without explicitly passing -u to the executing python interpreter.

* use spack python in xdist to support python 2

When running on python2, spack has many import cycles unless started
through main.  To allow that, this uses `spack python` as the
interpreter, leveraging the `-u` support so xdist doesn't error out when
it unconditionally requests unbuffered binary IO.

* Use shutil.move to account for tmpdir being in a separate filesystem sometimes

share/spack/qa/run-unit-tests

@@ -54,6 +54,11 @@ elif [[ "$SPACK_TEST_SOLVER" == "original" ]]; then
   export PYTEST_ADDOPTS='-m "not maybeslow"'
 fi
 
+# Check if xdist is available
+if python -m pytest --trace-config 2>&1 | grep xdist; then
+  export PYTEST_ADDOPTS="$PYTEST_ADDOPTS --dist loadfile --tx '${SPACK_TEST_PARALLEL:=3}*popen//python=./bin/spack-tmpconfig python -u ./bin/spack python'"
+fi
+
 $coverage_run $(which spack) unit-test -x --verbose
 
 bash "$QA_DIR/test-env-cfg.sh"

share/spack/spack-completion.bash

@@ -1568,7 +1568,7 @@ _spack_pydoc() {
 _spack_python() {
     if $list_options
     then
-        SPACK_COMPREPLY="-h --help -V --version -c -i -m --path"
+        SPACK_COMPREPLY="-h --help -V --version -c -u -i -m --path"
     else
         SPACK_COMPREPLY=""
     fi