Scrapy allows redirect following in protocols other than HTTP
Impact
Scrapy was following redirects regardless of the URL protocol, so redirects were working for data://, file://, ftp://, s3://, and any other scheme defined in the DOWNLOAD_HANDLERS setting.
However, HTTP redirects should only work between URLs that use the http:// or https:// schemes.
A malicious actor, given write access to the start requests (e.g. ability to define start_urls) of a spider and read access to the spider output, could exploit this vulnerability to:
- Redirect to any local file using the
file://scheme to read its contents. - Redirect to an
ftp://URL of a malicious FTP server to obtain the FTP username and password configured in the spider or project. - Redirect to any
s3://URL to read its content using the S3 credentials configured in the spider or project.
For file:// and s3://, how the spider implements its parsing of input data into an output item determines what data would be vulnerable. A spider that always outputs the entire contents of a response would be completely vulnerable, while a spider that extracted only fragments from the response could significantly limit vulnerable data.
Patches
Upgrade to Scrapy 2.11.2.
Workarounds
Replace the built-in retry middlewares (RedirectMiddleware and MetaRefreshMiddleware) with custom ones that implement the fix from Scrapy 2.11.2, and verify that they work as intended.
References
This security issue was reported by @mvsantos at https://github.com/scrapy/scrapy/issues/457.
Software afectado
Haz clic en una tecnología para ver todas sus vulnerabilidades.
| Plataforma | Paquete | Versiones vulnerables | Corregida en |
|---|---|---|---|
| Python (PyPI) | scrapy | < 2.11.2 | 2.11.2 |
Referencias
- Vector CVSS
CVSS:3.1/AV:N/AC:L/PR:L/UI:N/S:U/C:H/I:N/A:N- Debilidad (CWE)
CWE-552- Plataforma
- Python (PyPI)
- Paquete
scrapy- Publicada
- 14.05.2024
- Actualizado
- 11.10.2026 12:18
Fuentes y licencias
OSV / GitHub Advisory Database - GHSA-23j4-mw76-5v7h - CC BY 4.0
Historial de vulnerabilidades: scrapy
Todas las vulnerabilidades publicadas de este paquete (Python (PyPI)), de la más reciente a la más antigua. Muestra si es un fallo aislado o un paquete que hay que vigilar.
- Entradas en total
- 14
- Primera → última
- 09.2017 09.2026
- Sin corrección
- 1
- Explotada en ataques
- 0
Distribución por gravedad
- Alta 6
- Media 8
Divulgaciones por año
- 2017: 1
- 2018: 0
- 2019: 0
- 2020: 0
- 2021: 1
- 2022: 3
- 2023: 0
- 2024: 6
- 2025: 1
- 2026: 2
-
Scrapy allows redirect following in protocols other than HTTP esta entrada
Identifica las vulnerabilidades de tu aplicación
Con la integración de DockRay y el inventario de software activado, DockRay compara las versiones notificadas de los componentes con las vulnerabilidades conocidas y avisa de las coincidencias confirmadas. La cobertura depende de la tecnología compatible y de los avisos disponibles.