diff --git a/docs/contributing/code-review.rst b/docs/contributing/code-review.rst index 1c2f406..28feed9 100644 --- a/docs/contributing/code-review.rst +++ b/docs/contributing/code-review.rst @@ -46,4 +46,6 @@ feels exhaustively unit-testable, write unit tests too. When in doubt, add more tests. If you are unsure what kind of tests to add for your pull request, other -contributors to the repo will be happy to help guide you! \ No newline at end of file +contributors to the repo will be happy to help guide you! + +See :ref:`contributing/tests` for guidelines on writing tests. \ No newline at end of file diff --git a/docs/contributing/tests.rst b/docs/contributing/tests.rst new file mode 100644 index 0000000..f4e554e --- /dev/null +++ b/docs/contributing/tests.rst @@ -0,0 +1,59 @@ +.. _contributing/tests: + +============ +Testing TLJH +============ + +Unit and integration tests are a core part of TLJH, as important as +the code & documentation. They help validate that the code works as +we think it does, and continues to do so when changes occur. They +also help communicate in precise terms what we expect our code +to do. + +Integration tests +================= + +TLJH is a *distribution* where the primary value is the many +opinionated choices we have made on components to use and how +they fit together. Integration tests are perfect for testing +that the various components fit together and work as they should. +So we write a lot of integration tests, and put in more effort +towards them than unit tests. + +All integration tests are run on `CircleCI `_ +for each PR and merge, making sure we don't have broken tests +for too long. + +The integration tests are in the ``integration-tests`` directory +in the git repository. ``py.test`` is used to write the integration +tests. Each file should contain tests that can be run in any order +against the same installation of TLJH. + +Running integration tests locally +--------------------------------- + +You need ``docker`` installed and callable by the user running +the integration tests without needing sudo. + +You can then run the tests with: + +.. code-block:: bash + + .circleci/integration-test run-test + + +```` is an identifier for the tests - you can choose +anything you want. ``>`` is list of test files +(under ``integration-tests``) that should be run in one go. + +For example, to run all the basic tests, you would write: + +.. code-block:: bash + + .circleci/integration-test.py run-test basic-tests \ + test_hub.py \ + test_install.py \ + test_extensions.py + +This will run the tests in the three files against the same installation +of TLJH and report errors. \ No newline at end of file diff --git a/docs/index.rst b/docs/index.rst index 52bf9aa..9fa65fd 100644 --- a/docs/index.rst +++ b/docs/index.rst @@ -129,3 +129,4 @@ to people contributing in various ways. contributing/docs contributing/code-review contributing/dev-setup + contributing/tests