From 4cd5da40c38fb01bb5ac5a6df538ac1b13d5a1fb Mon Sep 17 00:00:00 2001 From: Erik Sundell Date: Thu, 10 Aug 2023 15:14:09 +0200 Subject: [PATCH 1/7] Update to jupyterhub >=4.0.2,<5 --- docs/reference/changelog.md | 4 ++-- tljh/requirements-hub-env.txt | 2 +- 2 files changed, 3 insertions(+), 3 deletions(-) diff --git a/docs/reference/changelog.md b/docs/reference/changelog.md index b0913c6..10ebbec 100644 --- a/docs/reference/changelog.md +++ b/docs/reference/changelog.md @@ -75,7 +75,7 @@ The changes in the respective environments between TLJH version 0.2.0 and | Dependency changes in the _hub environment_ | Version in 0.2.0 | Version in 1.0.0b1 | Changelog link | Note | | ------------------------------------------------------------------------------ | ---------------- | ------------------ | ---------------------------------------------------------------------------------------- | ---------------------------------------------------- | -| [jupyterhub](https://github.com/jupyterhub/jupyterhub) | 1.\* | >=4.0.1,<5 | [Changelog](https://jupyterhub.readthedocs.io/en/stable/reference/changelog.html) | Running in the `jupyterhub` systemd service | +| [jupyterhub](https://github.com/jupyterhub/jupyterhub) | 1.\* | >=4.0.2,<5 | [Changelog](https://jupyterhub.readthedocs.io/en/stable/reference/changelog.html) | Running in the `jupyterhub` systemd service | | [traefik](https://github.com/traefik/traefik) | 1.7.33 | 2.10.1 | [Changelog](https://github.com/traefik/traefik/blob/master/CHANGELOG.md) | Running in the `traefik` systemd service | | [traefik-proxy](https://github.com/jupyterhub/traefik-proxy) | 0.3.\* | >=1.1.0,<2 | [Changelog](https://jupyterhub-traefik-proxy.readthedocs.io/en/latest/changelog.html) | Run by jupyterhub, controls `traefik` | | [systemdspawner](https://github.com/jupyterhub/systemdspawner) | 0.16.\* | >=1.0.1,<2 | [Changelog](https://github.com/jupyterhub/systemdspawner/blob/master/CHANGELOG.md) | Run by jupyterhub, controls user servers via systemd | @@ -89,7 +89,7 @@ The changes in the respective environments between TLJH version 0.2.0 and | Dependency changes in the _user environment_ | Version in 0.2.0 | Version in 1.0.0 | Changelog link | Note | | -------------------------------------------------------- | ---------------- | ---------------- | --------------------------------------------------------------------------------- | ------------------------ | -| [jupyterhub](https://github.com/jupyterhub/jupyterhub) | 1.\* | >=4.0.1,<5 | [Changelog](https://jupyterhub.readthedocs.io/en/stable/reference/changelog.html) | Always upgraded. | +| [jupyterhub](https://github.com/jupyterhub/jupyterhub) | 1.\* | >=4.0.2,<5 | [Changelog](https://jupyterhub.readthedocs.io/en/stable/reference/changelog.html) | Always upgraded. | | [pip](https://github.com/pypa/pip) | \* | >=23.1.2 | [Changelog](https://pip.pypa.io/en/stable/news/) | Only upgraded if needed. | | [conda](https://docs.conda.io/projects/conda/en/stable/) | 0.16.0 | >=0.16.0 | [Changelog](https://docs.conda.io/projects/conda/en/stable/release-notes.html) | Only upgraded if needed. | | [mamba](https://mamba.readthedocs.io/en/latest/) | 4.10.3 | >=4.10.0 | [Changelog](https://github.com/mamba-org/mamba/blob/main/CHANGELOG.md) | Only upgraded if needed. | diff --git a/tljh/requirements-hub-env.txt b/tljh/requirements-hub-env.txt index e1c9273..f8e7efd 100644 --- a/tljh/requirements-hub-env.txt +++ b/tljh/requirements-hub-env.txt @@ -8,7 +8,7 @@ # If a dependency is bumped to a new major version, we should make a major # version release of tljh. # -jupyterhub>=4.0.1,<5 +jupyterhub>=4.0.2,<5 jupyterhub-systemdspawner>=1.0.1,<2 jupyterhub-firstuseauthenticator>=1.0.0,<2 jupyterhub-nativeauthenticator>=1.2.0,<2 From c6d86d059fb6006641b350184a25fc2c182d572d Mon Sep 17 00:00:00 2001 From: Erik Sundell Date: Thu, 10 Aug 2023 15:14:37 +0200 Subject: [PATCH 2/7] docs: ensure intersphinx extension is enabled --- docs/conf.py | 1 + 1 file changed, 1 insertion(+) diff --git a/docs/conf.py b/docs/conf.py index dcb64fa..f3e2e15 100644 --- a/docs/conf.py +++ b/docs/conf.py @@ -20,6 +20,7 @@ author = "Project Jupyter Contributors" # extensions = [ "sphinx_copybutton", + "sphinx.ext.intersphinx", "sphinxext.opengraph", "sphinxext.rediraffe", "myst_parser", From cc00bf3a1ac000ffd7aa8f21a13c18f910061787 Mon Sep 17 00:00:00 2001 From: Erik Sundell Date: Thu, 10 Aug 2023 15:16:11 +0200 Subject: [PATCH 3/7] docs: add docs on the system, hub, user environments --- docs/topic/index.md | 1 + docs/topic/three-environments.md | 76 ++++++++++++++++++++++++++++++++ 2 files changed, 77 insertions(+) create mode 100644 docs/topic/three-environments.md diff --git a/docs/topic/index.md b/docs/topic/index.md index 7ec4b88..90e7e9c 100644 --- a/docs/topic/index.md +++ b/docs/topic/index.md @@ -8,6 +8,7 @@ Topic guides provide in-depth explanations of specific topics. whentouse requirements +three-environments security customizing-installer installer-actions diff --git a/docs/topic/three-environments.md b/docs/topic/three-environments.md new file mode 100644 index 0000000..46718b7 --- /dev/null +++ b/docs/topic/three-environments.md @@ -0,0 +1,76 @@ +(topic-three-environments)= + +# The system, hub, and user environments + +TLJH's documentation mentions the _system environment_, the _hub environment_, +and the _user environment_. This section will introduce what is meant with that +and clarify the distinctions between the environments. + +(system-environment)= + +## The system environment + +When this documentation mentions the _system environment_, it refers to the +Linux environment with its installed `apt` packages, users in `/etc/passwd`, +etc. + +A part of the system environment is a Python environment setup via the `apt` +package `python` installed by default in Linux distributions supported by TLJH. +To be specific, we can refer to this as the _system's Python environment_. + +If you would do `sudo python3 -m pip install ` you would end up +installing something in the system's Python environment, and that would not be +available in the hub environment or the user environment. + +The system's Python environment is only used by TLJH to run the `bootstrap.py` +script downloaded as part of installing or upgrading TLJH. This script is also +responsible for setting up the hub environment. + +(hub-environment)= + +## The hub environment + +The _hub environment_ is a [virtual Python environment] setup in `/opt/tljh/hub` +by the `bootstrap.py` script using the system's Python environment during TLJH +installation. + +The hub environment has Python packages installed in it related to running +JupyterHub itself such as an JupyterHub authenticator package, but it doesn't +include packages to start user servers like JupyterLab. + +When TLJH is installed/upgraded, the packages listed in +[tljh/requirements-hub-env.txt] are installed/upgraded in this environment. + +If you would do `sudo /opt/tljh/hub/bin/python3 -m pip install ` you +would end up installing something in the hub environment, and that would not be +available in the system's Python environment or the user environment. + +[virtual Python environment]: https://docs.python.org/3/library/venv.html + +[tljh/requirements-hub-env.txt]: https://github.com/jupyterhub/the-littlest-jupyterhub/blob/HEAD/tljh/requirements-hub-env.txt + +(user-environment)= + +## The user environment + +The _user environment_ is a Python environment setup in `/opt/tljh/user` by the +TLJH installer during TLJH installation. The user environment is not a virtual +environment because an entirely separate installation of Python has been made +for it. + +The user environment has packages installed in it related to running individual +jupyter servers, such as `jupyterlab`. + +When TLJH is _installed_, the packages listed in +[tljh/requirements-user-env.txt] are installed in this environment. When TLJH is +_upgraded_ though, as little as possible is done to this environment. Typically +only `jupyterhub` is upgraded to match the version in the hub environment. If +upgrading to a new major version of TLJH, then something small may be done +besides this, and then it should be described the changelog. + +If you would do `sudo /opt/tljh/user/bin/python3 -m pip install `, or +from a user server's terminal do `sudo -E pip install ` you would end +up installing something in the user environment, and that would not be available +in the system's Python environment or the hub environment. + +[tljh/requirements-user-env-extras.txt]: https://github.com/jupyterhub/the-littlest-jupyterhub/blob/HEAD/tljh/requirements-user-env-extras.txt From e14b8d8b775d1cd410e23896afa1e3232592ea86 Mon Sep 17 00:00:00 2001 From: Erik Sundell Date: Thu, 10 Aug 2023 15:17:03 +0200 Subject: [PATCH 4/7] docs: add section on whats done during upgrades --- docs/topic/index.md | 1 + docs/topic/installer-upgrade-actions.md | 30 +++++++++++++++++++++++++ 2 files changed, 31 insertions(+) create mode 100644 docs/topic/installer-upgrade-actions.md diff --git a/docs/topic/index.md b/docs/topic/index.md index 90e7e9c..dc38061 100644 --- a/docs/topic/index.md +++ b/docs/topic/index.md @@ -12,6 +12,7 @@ three-environments security customizing-installer installer-actions +installer-upgrade-actions tljh-config authenticator-configuration escape-hatch diff --git a/docs/topic/installer-upgrade-actions.md b/docs/topic/installer-upgrade-actions.md new file mode 100644 index 0000000..f367434 --- /dev/null +++ b/docs/topic/installer-upgrade-actions.md @@ -0,0 +1,30 @@ +(topic-installer-upgrade-actions)= + +# What is done during an upgrade of TLJH? + +Once TLJH has been installed, it should be possible to upgrade the installation. +This documentation is meant to capture the changes made during an upgrade. + +```{versionchanged} 1.0.0 +Ensuring upgrades work has only been done since 1.0.0 upgrading from version +0.2.0. +``` + +## Changes to the system environment + +The [system environment](system-environment) is not meant to be influenced +unless explicitly mentioned in the changelog, typically only during major +version upgrades. + +## Changes to the hub environment + +The [hub environment](hub-environment) gets several packages upgraded based on +version ranges specified in [tljh/requirements-hub-env.txt]. + +## Changes to the user environment + +The [user environment](user-environment) gets is `jupyterhub` package upgraded, +but no other packages gets upgraded unless explicitly mentioned in the +changelog, typically only during major version upgrades. + +[tljh/requirements-hub-env.txt]: https://github.com/jupyterhub/the-littlest-jupyterhub/blob/HEAD/tljh/requirements-hub-env.txt From 9efe59f4f74f5c346c9ff687491a1151ae86c9f9 Mon Sep 17 00:00:00 2001 From: Erik Sundell Date: Thu, 10 Aug 2023 15:17:50 +0200 Subject: [PATCH 5/7] docs: add how-to section on making upgrades --- docs/howto/admin/upgrade-tljh.md | 64 ++++++++++++++++++++++++++++++++ docs/howto/index.md | 1 + docs/troubleshooting/index.md | 2 + 3 files changed, 67 insertions(+) create mode 100644 docs/howto/admin/upgrade-tljh.md diff --git a/docs/howto/admin/upgrade-tljh.md b/docs/howto/admin/upgrade-tljh.md new file mode 100644 index 0000000..1fcfe10 --- /dev/null +++ b/docs/howto/admin/upgrade-tljh.md @@ -0,0 +1,64 @@ +(howto-admin-upgrade-tljh)= + +# Upgrade TLJH + +An TLJH installation is supposed to be upgradable to get updates to JupyterHub +itself and its dependencies in the [hub environment](hub-environment). For +details on what is done during an upgrade, see +[](topic-installer-upgrade-actions). + +## Step 1: Read the changelog + +Before making an upgrade, please read the [](changelog) to become aware about +breaking changes. If there are breaking changes, you may need to update your +configuration files or take other actions as well as part of the upgrade. + +Adjusting to the breaking changes isn't part of this documentation, please rely +on the TLJH changelog and the changelogs of related projects linked to from the +TLJH changelog. + +## Step 2: Consider making a backup + +Before making an upgrade, consider if you want to first make a backup in some +way. While upgrades between TLJH versions are tested with automation, there are +no guarantees. + +This project does't yet provide documentation on how to make backups, but if +TLJH is installed on a virtual machine in a cloud, a good option is to try +create a snapshot of the associated disk. If this isn't an option, you could +consider making a backup of the files in `/opt/tljh` that contain most but not +all things during an upgrade, or perhaps only the JupyterHub database with +information about its users in `/opt/tljh/state` together with some other +details. + +## Step 3: Make the upgrade + +To initialize the upgrade, do the following from a terminal on the machine where +TLJH is installed. + +```shell +# IMPORTANT: This should NOT be run from a JupyterHub started user server, but +# should only run from a standalone terminal session in the machine +# where TLJH has been installed. +# +curl -L https://tljh.jupyter.org/bootstrap.py \ + | sudo python3 - \ + --version=latest +``` + +You can also upgrade to specific version by changing `--version=latest` to +`--version=1.0.0` or similar. There is no need to specify admin users etc again. + +## Step 4: Verify function + +After having made an upgrade its always good to verify that the JupyterHub +installation still works as expected. You may want to try logging out, logging +in, and starting a new server for example. + +If you have issues consider the [](troubleshooting) documentation. If you need +help you can ask questions in [Jupyter forum], and if you think there is a bug +or documentation improvement that should be made you can open an issue or pull +request in the [TLJH GitHub project]. + +[jupyter forum]: https://discourse.jupyter.org/c/jupyterhub/tljh +[tljh github project]: https://github.com/jupyterhub/the-littlest-jupyterhub diff --git a/docs/howto/index.md b/docs/howto/index.md index ea4135d..7b28885 100644 --- a/docs/howto/index.md +++ b/docs/howto/index.md @@ -56,6 +56,7 @@ admin/nbresuse admin/https admin/enable-extensions admin/systemd +admin/upgrade-tljh ``` ## Cloud provider configuration diff --git a/docs/troubleshooting/index.md b/docs/troubleshooting/index.md index 178a64b..9c899fd 100644 --- a/docs/troubleshooting/index.md +++ b/docs/troubleshooting/index.md @@ -1,3 +1,5 @@ +(troubleshooting)= + # Troubleshooting In time, all systems have issues that need to be debugged. Troubleshooting From fb78464dec8c7091c3338fb60b7ad9ac5b7552d4 Mon Sep 17 00:00:00 2001 From: Erik Sundell Date: Thu, 10 Aug 2023 15:19:26 +0200 Subject: [PATCH 6/7] docs: fix broken link --- docs/howto/user-env/notebook-interfaces.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/docs/howto/user-env/notebook-interfaces.md b/docs/howto/user-env/notebook-interfaces.md index 063c004..8560b17 100644 --- a/docs/howto/user-env/notebook-interfaces.md +++ b/docs/howto/user-env/notebook-interfaces.md @@ -5,7 +5,7 @@ By default a user starting a server will see the JupyterLab interface. This can be changed with TLJH config `user_environment.default_app` or with the JupyterHub config -{external:py:attribute}`jupyterhub.spawner.Spawner.default_url` directly. +{external:py:attr}`jupyterhub.spawner.Spawner.default_url` directly. The TLJH config supports the options `jupyterlab` and `classic`, which translates to a `Spawner.default_url` config of `/lab` and `/tree`. From c35a4cbe658fa2a676623fa591ab5b3f90ef7ec7 Mon Sep 17 00:00:00 2001 From: Min RK Date: Fri, 11 Aug 2023 11:46:37 +0200 Subject: [PATCH 7/7] typo --- docs/howto/admin/upgrade-tljh.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/docs/howto/admin/upgrade-tljh.md b/docs/howto/admin/upgrade-tljh.md index 1fcfe10..1cb417a 100644 --- a/docs/howto/admin/upgrade-tljh.md +++ b/docs/howto/admin/upgrade-tljh.md @@ -2,7 +2,7 @@ # Upgrade TLJH -An TLJH installation is supposed to be upgradable to get updates to JupyterHub +A TLJH installation is supposed to be upgradable to get updates to JupyterHub itself and its dependencies in the [hub environment](hub-environment). For details on what is done during an upgrade, see [](topic-installer-upgrade-actions).