GHSA-X3PX-J3QH-FP5W
Redirects to no_proxy hosts sent through the environment proxy
Texto original en inglés · Ver en español
CVSS:4.0/AV:N/AC:L/AT:P/PR:N/UI:N/VC:L/VI:N/VA:N/SC:N/SI:N/SA:N
Fuente: GitHub
Ciclo de vida de la vulnerabilidad
| Fecha | Fase | Evento | Detalle |
|---|---|---|---|
| Divulgación | Publicada | GHSA-X3PX-J3QH-FP5W divulgada · CVSS 6.3 (GitHub) | |
| Remediación | Parche 2.20.0 | Scrapy · 0 d desde divulgación | |
| Monitoreo | Detectada | Ingresó al índice | |
| Monitoreo | Verificada | Última verificación contra las fuentes |
Impact
When using a proxy from the `http_proxy` or `https_proxy` environment variables, `HttpProxyMiddleware` https://docs.scrapy.org/en/latest/topics/downloader-middleware.html#scrapy.downloadermiddlewares.httpproxy.HttpProxyMiddleware does not use it for hosts that match the `no_proxy` environment variable.
However, the built-in redirect middlewares (`RedirectMiddleware` https://docs.scrapy.org/en/latest/topics/downloader-middleware.html#scrapy.downloadermiddlewares.redirect.RedirectMiddleware and `MetaRefreshMiddleware` https://docs.scrapy.org/en/latest/topics/downloader-middleware.html#scrapy.downloadermiddlewares.redirect.MetaRefreshMiddleware) kept the proxy of the original request on redirects to a URL with the same scheme, even if the redirect host matched `no_proxy`.
A website could redirect Scrapy to an internal host, and that request would be sent through the proxy. For `http://` URLs, this exposed the request to the proxy, including any cookies or credentials for that host, and allowed the proxy to forge the response. For `https://` URLs, it exposed the host and port.
Patches
Upgrade to Scrapy 2.20.0+.
Workarounds
If you cannot upgrade to Scrapy 2.20.0, consider the following mitigations.
- Disable redirects: If you do not need Scrapy to follow redirects, set `REDIRECT_ENABLED`
https://docs.scrapy.org/en/latest/topics/downloader-middleware.html#redirect-enabledand `METAREFRESH_ENABLED`https://docs.scrapy.org/en/latest/topics/downloader-middleware.html#metarefresh-enabledto `False`, or set the `dont_redirect`https://docs.scrapy.org/en/latest/topics/downloader-middleware.html#std-reqmeta-dont_redirectrequest meta key to `True` on requests that could be redirected to internal hosts. - Drop the proxy from redirect requests: On Scrapy 2.11.2+, add a downloader middleware
https://docs.scrapy.org/en/latest/topics/downloader-middleware.htmlthat runs before `HttpProxyMiddleware` on requests, i.e. with a lower order than its 750, and removes the proxy that `HttpProxyMiddleware` set from the environment, so that it sets the proxy again for the redirect URL. For example:
class NoProxyRedirectMiddleware:
def process_request(self, request, spider):
if request.meta.pop("_scheme_proxy", False):
request.meta.pop("proxy", None)
request.meta.pop("_auth_proxy", None)
request.headers.pop(b"Proxy-Authorization", None)DOWNLOADER_MIDDLEWARES = {
"myproject.middlewares.NoProxyRedirectMiddleware": 749,
}Desglose de métricas
Rangos afectados
Estado del parche por paquete
| Paquete / producto | Estado | Corregida en | Desde divulgación | Última versión |
|---|---|---|---|---|
pypiScrapy |
Corregida | 2.20.0 | 0 d | 2.20.0 no afectada |
2.20.0Arsenal público
Sin exploits, módulos ni PoC públicos conocidos a la fecha de verificación.
https://github.com/scrapy/scrapy/security/advisories/GHSA-x3px-j3qh-fp5wLas referencias externas se muestran para copiar: cve.llama.ec no lo redirige a otros sitios.
