Merge pull request #932 from consideRatio/pr/add-upgrade-docs

docs: add docs about environments and upgrades

docs/conf.py

@@ -20,6 +20,7 @@ author = "Project Jupyter Contributors"
 #
 extensions = [
     "sphinx_copybutton",
+    "sphinx.ext.intersphinx",
     "sphinxext.opengraph",
     "sphinxext.rediraffe",
     "myst_parser",

docs/howto/admin/upgrade-tljh.md

@@ -0,0 +1,64 @@
+(howto-admin-upgrade-tljh)=
+
+# Upgrade TLJH
+
+A TLJH installation is supposed to be upgradable to get updates to JupyterHub
+itself and its dependencies in the [hub environment](hub-environment). For
+details on what is done during an upgrade, see
+[](topic-installer-upgrade-actions).
+
+## Step 1: Read the changelog
+
+Before making an upgrade, please read the [](changelog) to become aware about
+breaking changes. If there are breaking changes, you may need to update your
+configuration files or take other actions as well as part of the upgrade.
+
+Adjusting to the breaking changes isn't part of this documentation, please rely
+on the TLJH changelog and the changelogs of related projects linked to from the
+TLJH changelog.
+
+## Step 2: Consider making a backup
+
+Before making an upgrade, consider if you want to first make a backup in some
+way. While upgrades between TLJH versions are tested with automation, there are
+no guarantees.
+
+This project does't yet provide documentation on how to make backups, but if
+TLJH is installed on a virtual machine in a cloud, a good option is to try
+create a snapshot of the associated disk. If this isn't an option, you could
+consider making a backup of the files in `/opt/tljh` that contain most but not
+all things during an upgrade, or perhaps only the JupyterHub database with
+information about its users in `/opt/tljh/state` together with some other
+details.
+
+## Step 3: Make the upgrade
+
+To initialize the upgrade, do the following from a terminal on the machine where
+TLJH is installed.
+
+```shell
+# IMPORTANT: This should NOT be run from a JupyterHub started user server, but
+#            should only run from a standalone terminal session in the machine
+#            where TLJH has been installed.
+#
+curl -L https://tljh.jupyter.org/bootstrap.py \
+  | sudo python3 - \
+    --version=latest
+```
+
+You can also upgrade to specific version by changing `--version=latest` to
+`--version=1.0.0` or similar. There is no need to specify admin users etc again.
+
+## Step 4: Verify function
+
+After having made an upgrade its always good to verify that the JupyterHub
+installation still works as expected. You may want to try logging out, logging
+in, and starting a new server for example.
+
+If you have issues consider the [](troubleshooting) documentation. If you need
+help you can ask questions in [Jupyter forum], and if you think there is a bug
+or documentation improvement that should be made you can open an issue or pull
+request in the [TLJH GitHub project].
+
+[jupyter forum]: https://discourse.jupyter.org/c/jupyterhub/tljh
+[tljh github project]: https://github.com/jupyterhub/the-littlest-jupyterhub

docs/howto/index.md

@@ -56,6 +56,7 @@ admin/nbresuse
 admin/https
 admin/enable-extensions
 admin/systemd
+admin/upgrade-tljh
 ```
 
 ## Cloud provider configuration

docs/howto/user-env/notebook-interfaces.md

@@ -5,7 +5,7 @@
 By default a user starting a server will see the JupyterLab interface. This can
 be changed with TLJH config `user_environment.default_app` or with the
 JupyterHub config
-{external:py:attribute}`jupyterhub.spawner.Spawner.default_url` directly.
+{external:py:attr}`jupyterhub.spawner.Spawner.default_url` directly.
 
 The TLJH config supports the options `jupyterlab` and `classic`, which
 translates to a `Spawner.default_url` config of `/lab` and `/tree`.

docs/reference/changelog.md

@@ -75,7 +75,7 @@ The changes in the respective environments between TLJH version 0.2.0 and
 
 | Dependency changes in the _hub environment_                                    | Version in 0.2.0 | Version in 1.0.0b1 | Changelog link                                                                           | Note                                                 |
 | ------------------------------------------------------------------------------ | ---------------- | ------------------ | ---------------------------------------------------------------------------------------- | ---------------------------------------------------- |
-| [jupyterhub](https://github.com/jupyterhub/jupyterhub)                         | 1.\*             | >=4.0.1,<5         | [Changelog](https://jupyterhub.readthedocs.io/en/stable/reference/changelog.html)        | Running in the `jupyterhub` systemd service          |
+| [jupyterhub](https://github.com/jupyterhub/jupyterhub)                         | 1.\*             | >=4.0.2,<5         | [Changelog](https://jupyterhub.readthedocs.io/en/stable/reference/changelog.html)        | Running in the `jupyterhub` systemd service          |
 | [traefik](https://github.com/traefik/traefik)                                  | 1.7.33           | 2.10.1             | [Changelog](https://github.com/traefik/traefik/blob/master/CHANGELOG.md)                 | Running in the `traefik` systemd service             |
 | [traefik-proxy](https://github.com/jupyterhub/traefik-proxy)                   | 0.3.\*           | >=1.1.0,<2         | [Changelog](https://jupyterhub-traefik-proxy.readthedocs.io/en/latest/changelog.html)    | Run by jupyterhub, controls `traefik`                |
 | [systemdspawner](https://github.com/jupyterhub/systemdspawner)                 | 0.16.\*          | >=1.0.1,<2         | [Changelog](https://github.com/jupyterhub/systemdspawner/blob/master/CHANGELOG.md)       | Run by jupyterhub, controls user servers via systemd |
@@ -89,7 +89,7 @@ The changes in the respective environments between TLJH version 0.2.0 and
 
 | Dependency changes in the _user environment_             | Version in 0.2.0 | Version in 1.0.0 | Changelog link                                                                    | Note                     |
 | -------------------------------------------------------- | ---------------- | ---------------- | --------------------------------------------------------------------------------- | ------------------------ |
-| [jupyterhub](https://github.com/jupyterhub/jupyterhub)   | 1.\*             | >=4.0.1,<5       | [Changelog](https://jupyterhub.readthedocs.io/en/stable/reference/changelog.html) | Always upgraded.         |
+| [jupyterhub](https://github.com/jupyterhub/jupyterhub)   | 1.\*             | >=4.0.2,<5       | [Changelog](https://jupyterhub.readthedocs.io/en/stable/reference/changelog.html) | Always upgraded.         |
 | [pip](https://github.com/pypa/pip)                       | \*               | >=23.1.2         | [Changelog](https://pip.pypa.io/en/stable/news/)                                  | Only upgraded if needed. |
 | [conda](https://docs.conda.io/projects/conda/en/stable/) | 0.16.0           | >=0.16.0         | [Changelog](https://docs.conda.io/projects/conda/en/stable/release-notes.html)    | Only upgraded if needed. |
 | [mamba](https://mamba.readthedocs.io/en/latest/)         | 4.10.3           | >=4.10.0         | [Changelog](https://github.com/mamba-org/mamba/blob/main/CHANGELOG.md)            | Only upgraded if needed. |

docs/topic/index.md

@@ -8,9 +8,11 @@ Topic guides provide in-depth explanations of specific topics.
 
 whentouse
 requirements
+three-environments
 security
 customizing-installer
 installer-actions
+installer-upgrade-actions
 tljh-config
 authenticator-configuration
 escape-hatch

docs/topic/installer-upgrade-actions.md

@@ -0,0 +1,30 @@
+(topic-installer-upgrade-actions)=
+
+# What is done during an upgrade of TLJH?
+
+Once TLJH has been installed, it should be possible to upgrade the installation.
+This documentation is meant to capture the changes made during an upgrade.
+
+```{versionchanged} 1.0.0
+Ensuring upgrades work has only been done since 1.0.0 upgrading from version
+0.2.0.
+```
+
+## Changes to the system environment
+
+The [system environment](system-environment) is not meant to be influenced
+unless explicitly mentioned in the changelog, typically only during major
+version upgrades.
+
+## Changes to the hub environment
+
+The [hub environment](hub-environment) gets several packages upgraded based on
+version ranges specified in [tljh/requirements-hub-env.txt].
+
+## Changes to the user environment
+
+The [user environment](user-environment) gets is `jupyterhub` package upgraded,
+but no other packages gets upgraded unless explicitly mentioned in the
+changelog, typically only during major version upgrades.
+
+[tljh/requirements-hub-env.txt]: https://github.com/jupyterhub/the-littlest-jupyterhub/blob/HEAD/tljh/requirements-hub-env.txt

docs/topic/three-environments.md

@@ -0,0 +1,76 @@
+(topic-three-environments)=
+
+# The system, hub, and user environments
+
+TLJH's documentation mentions the _system environment_, the _hub environment_,
+and the _user environment_. This section will introduce what is meant with that
+and clarify the distinctions between the environments.
+
+(system-environment)=
+
+## The system environment
+
+When this documentation mentions the _system environment_, it refers to the
+Linux environment with its installed `apt` packages, users in `/etc/passwd`,
+etc.
+
+A part of the system environment is a Python environment setup via the `apt`
+package `python` installed by default in Linux distributions supported by TLJH.
+To be specific, we can refer to this as the _system's Python environment_.
+
+If you would do `sudo python3 -m pip install <something>` you would end up
+installing something in the system's Python environment, and that would not be
+available in the hub environment or the user environment.
+
+The system's Python environment is only used by TLJH to run the `bootstrap.py`
+script downloaded as part of installing or upgrading TLJH. This script is also
+responsible for setting up the hub environment.
+
+(hub-environment)=
+
+## The hub environment
+
+The _hub environment_ is a [virtual Python environment] setup in `/opt/tljh/hub`
+by the `bootstrap.py` script using the system's Python environment during TLJH
+installation.
+
+The hub environment has Python packages installed in it related to running
+JupyterHub itself such as an JupyterHub authenticator package, but it doesn't
+include packages to start user servers like JupyterLab.
+
+When TLJH is installed/upgraded, the packages listed in
+[tljh/requirements-hub-env.txt] are installed/upgraded in this environment.
+
+If you would do `sudo /opt/tljh/hub/bin/python3 -m pip install <something>` you
+would end up installing something in the hub environment, and that would not be
+available in the system's Python environment or the user environment.
+
+[virtual Python environment]: https://docs.python.org/3/library/venv.html
+
+[tljh/requirements-hub-env.txt]: https://github.com/jupyterhub/the-littlest-jupyterhub/blob/HEAD/tljh/requirements-hub-env.txt
+
+(user-environment)=
+
+## The user environment
+
+The _user environment_ is a Python environment setup in `/opt/tljh/user` by the
+TLJH installer during TLJH installation. The user environment is not a virtual
+environment because an entirely separate installation of Python has been made
+for it.
+
+The user environment has packages installed in it related to running individual
+jupyter servers, such as `jupyterlab`.
+
+When TLJH is _installed_, the packages listed in
+[tljh/requirements-user-env.txt] are installed in this environment. When TLJH is
+_upgraded_ though, as little as possible is done to this environment. Typically
+only `jupyterhub` is upgraded to match the version in the hub environment. If
+upgrading to a new major version of TLJH, then something small may be done
+besides this, and then it should be described the changelog.
+
+If you would do `sudo /opt/tljh/user/bin/python3 -m pip install <something>`, or
+from a user server's terminal do `sudo -E pip install <something>` you would end
+up installing something in the user environment, and that would not be available
+in the system's Python environment or the hub environment.
+
+[tljh/requirements-user-env-extras.txt]: https://github.com/jupyterhub/the-littlest-jupyterhub/blob/HEAD/tljh/requirements-user-env-extras.txt

docs/troubleshooting/index.md

@@ -1,3 +1,5 @@
+(troubleshooting)=
+
 # Troubleshooting
 
 In time, all systems have issues that need to be debugged. Troubleshooting

tljh/requirements-hub-env.txt

@@ -8,7 +8,7 @@
 # If a dependency is bumped to a new major version, we should make a major
 # version release of tljh.
 #
-jupyterhub>=4.0.1,<5
+jupyterhub>=4.0.2,<5
 jupyterhub-systemdspawner>=1.0.1,<2
 jupyterhub-firstuseauthenticator>=1.0.0,<2
 jupyterhub-nativeauthenticator>=1.2.0,<2