Skip to content

.xlsx extraction returns one unstructured blob with no row/sheet boundaries #9

Description

@dvejsada

Split out of the bilingual-contract table bloat work (branch claude/contract-table-bloat-v5wikg).

Problem

.xls/.xlsx are loaded with UnstructuredExcelLoader(filepath) (app/utils/document_loader.py:220) at its default mode. On the /text path the result is flattened into a single string, so a workbook arrives as one undifferentiated blob: no sheet names, no row boundaries, no column headers attached to the values beneath them.

Two consequences for legal use:

  1. Size. A large sheet is emitted in full with no structure to compress or chunk along — the same class of problem as the .docx table padding, from a different loader.
  2. Usefulness. A damages schedule or a payment table loses the header-to-value association that makes it readable, so downstream extraction has to guess which number belongs to which column.

Suggested fix

  • Consider mode="elements" (as .rst already uses) or emitting one Document per sheet with the sheet name in metadata.
  • Render each sheet as a compact pipe table with its header row, rather than free text — the same shape the .docx path now produces.
  • Bound the output: a 50,000-row sheet is not usefully extractable as text, and should say so rather than return 5 MB.

Needs a real-world sample workbook to size the work; the fix should be measured before/after the way the .docx change was.

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions