Use JSON for the database instead of YAML. (#2189)

* Use JSON for the database instead of YAML.

- JSON is much faster than YAML *and* can preserve ordered keys.
  - 170x+ faster than Python YAML when using unordered dicts
  - 55x faster than Python YAML (both using OrderedDicts)
  - 8x faster than C YAML (with OrderedDicts)

- JSON is built into Python, unlike C YAML, so doesn't add a dependency.
- Don't need human readability for the package database.
- JSON requires no major changes to the code -- same object model as YAML.
- add to_json, from_json methods to spec.

* Add tests to ensure JSON and YAML don't need to be ordered in DB.

* Write index.json first time it's not found instead of requiring reindex.

* flake8 bug.
This commit is contained in:
Todd Gamblin
2016-12-05 10:03:58 -08:00
committed by becker33
parent 161d6205b2
commit 41b8f31bcd
8 changed files with 282 additions and 90 deletions

View File

@@ -41,14 +41,6 @@ SPACK_PREFIX = os.path.dirname(os.path.dirname(SPACK_FILE))
SPACK_LIB_PATH = os.path.join(SPACK_PREFIX, "lib", "spack")
sys.path.insert(0, SPACK_LIB_PATH)
# Try to use system YAML if it is available, as it might have libyaml
# support (for faster loading via C). Load it before anything in
# lib/spack/external so it will take precedence over Spack's PyYAML.
try:
import yaml
except ImportError:
pass # ignore and use slow yaml
# Add external libs
SPACK_EXTERNAL_LIBS = os.path.join(SPACK_LIB_PATH, "external")
sys.path.insert(0, SPACK_EXTERNAL_LIBS)