Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
10 changes: 10 additions & 0 deletions AGENTS.md
Original file line number Diff line number Diff line change
Expand Up @@ -61,6 +61,7 @@ bytes ─▢ magic/open_strategy ─▢ DecodedFile ─▢ Document ─▢ Eleme
| `src/odr/internal/odf/` | OpenDocument (odt/ods/odp/odg); see [`odf/AGENTS.md`](src/odr/internal/odf/AGENTS.md). |
| `src/odr/internal/ooxml/` | OOXML (docx/pptx/xlsx); see [`ooxml/AGENTS.md`](src/odr/internal/ooxml/AGENTS.md) + per-format docs. |
| `src/odr/internal/oldms/` | **Legacy MS binary** (.doc/.ppt/.xls). |
| `src/odr/internal/iwork/` | Apple iWork (`.pages` today); see [`iwork/AGENTS.md`](src/odr/internal/iwork/AGENTS.md) + [`iwork/PLAN.md`](src/odr/internal/iwork/PLAN.md). |
| `src/odr/internal/pdf/` | PDF (own parser). |
| `src/odr/internal/xml/` | XML, rendered as a source view; see [`xml/AGENTS.md`](src/odr/internal/xml/AGENTS.md). |
| `src/odr/internal/svg/` | SVG, detected by reading it as xml; see [`svg/AGENTS.md`](src/odr/internal/svg/AGENTS.md). |
Expand Down Expand Up @@ -218,6 +219,15 @@ Dispatch `release.yml` against main, publish the draft that appears β€”
4. Register the factory (e.g. `oldms_file.cpp::document()` switches on
`file_type()`), add sources to `CMakeLists.txt`, add a GoogleTest.

## Apple iWork (`iwork`)

`.pages` opens as a text document and renders its body text; `.numbers` and
`.key` are named but not decoded. There is no spec β€” the module cites fixtures
instead, keeps its own Snappy and protobuf readers, and fails soft on archive
types it has not mapped. Read [`iwork/AGENTS.md`](src/odr/internal/iwork/AGENTS.md)
before touching it, and [`iwork/PLAN.md`](src/odr/internal/iwork/PLAN.md) for
what comes next.

## Legacy Microsoft binary formats (`oldms`)

CFB container handling exists; each format is a small module under `oldms/`
Expand Down
6 changes: 6 additions & 0 deletions CHANGELOG.md
Original file line number Diff line number Diff line change
Expand Up @@ -16,6 +16,12 @@ The release run heads these entries with the version and opens a fresh

## Unreleased

- Apple iWork: a `.pages` file opens as a text document and renders its body
text, instead of coming back as the zip it is made of. Styles, page geometry,
images and tables are not read yet. `.numbers` and `.key` are named β€”
`FileType::iwork_numbers`, `FileType::iwork_keynote`, their extensions and
MIME types β€” but there is no decoder behind either.

## v6.10.1 - 2026-08-21

- A linked image in a docx or xlsx (`embed_images = false`) is named relative
Expand Down
8 changes: 8 additions & 0 deletions CMakeLists.txt
Original file line number Diff line number Diff line change
Expand Up @@ -151,6 +151,14 @@ set(ODR_SOURCE_FILES
"src/odr/internal/html/text_file.cpp"
"src/odr/internal/html/xml_file.cpp"

"src/odr/internal/iwork/iwork_archive.cpp"
"src/odr/internal/iwork/iwork_document.cpp"
"src/odr/internal/iwork/iwork_element_registry.cpp"
"src/odr/internal/iwork/iwork_file.cpp"
"src/odr/internal/iwork/iwork_parser.cpp"
"src/odr/internal/iwork/iwork_protobuf.cpp"
"src/odr/internal/iwork/iwork_snappy.cpp"

"src/odr/internal/json/json_file.cpp"
"src/odr/internal/json/json_util.cpp"

Expand Down
4 changes: 4 additions & 0 deletions apple/include/OdrCoreObjC/ODRFile.h
Original file line number Diff line number Diff line change
Expand Up @@ -83,6 +83,10 @@ typedef NS_ENUM(NSInteger, ODRFileType) {
ODRFileTypeEnhancedMetafile,

ODRFileTypeXml,

ODRFileTypeIworkPages,
ODRFileTypeIworkNumbers,
ODRFileTypeIworkKeynote,
} NS_SWIFT_NAME(FileType);

typedef NS_ENUM(NSInteger, ODRFileCategory) {
Expand Down
4 changes: 4 additions & 0 deletions apple/src/ODRFile.mm
Original file line number Diff line number Diff line change
Expand Up @@ -92,6 +92,10 @@
ODR_SAME_ENUM(ODRFileTypeEnhancedMetafile, odr::FileType::enhanced_metafile);
ODR_SAME_ENUM(ODRFileTypeXml, odr::FileType::xml);

ODR_SAME_ENUM(ODRFileTypeIworkPages, odr::FileType::iwork_pages);
ODR_SAME_ENUM(ODRFileTypeIworkNumbers, odr::FileType::iwork_numbers);
ODR_SAME_ENUM(ODRFileTypeIworkKeynote, odr::FileType::iwork_keynote);

ODR_SAME_ENUM(ODRFileCategoryUnknown, odr::FileCategory::unknown);
ODR_SAME_ENUM(ODRFileCategoryText, odr::FileCategory::text);
ODR_SAME_ENUM(ODRFileCategoryImage, odr::FileCategory::image);
Expand Down
3 changes: 2 additions & 1 deletion jni/java/app/opendocument/core/FileType.java
Original file line number Diff line number Diff line change
Expand Up @@ -16,7 +16,8 @@ public enum FileType {
OGG_AUDIO, WAVEFORM_AUDIO, FREE_LOSSLESS_AUDIO_CODEC, MPEG4_VIDEO,
QUICKTIME_VIDEO, THIRD_GENERATION_PARTNERSHIP_VIDEO, MATROSKA_VIDEO,
AUDIO_VIDEO_INTERLEAVE, SCALABLE_VECTOR_GRAPHICS, WINDOWS_ICON, JPEG_XL,
JPEG_2000, PHOTOSHOP_DOCUMENT, WINDOWS_METAFILE, ENHANCED_METAFILE, XML;
JPEG_2000, PHOTOSHOP_DOCUMENT, WINDOWS_METAFILE, ENHANCED_METAFILE, XML,
IWORK_PAGES, IWORK_NUMBERS, IWORK_KEYNOTE;

static FileType fromNative(int code) {
return code < 0 ? null : values()[code];
Expand Down
5 changes: 4 additions & 1 deletion python/src/bind_file.cpp
Original file line number Diff line number Diff line change
Expand Up @@ -83,7 +83,10 @@ void odr_python::bind_file(py::module_ &m) {
.value("photoshop_document", odr::FileType::photoshop_document)
.value("windows_metafile", odr::FileType::windows_metafile)
.value("enhanced_metafile", odr::FileType::enhanced_metafile)
.value("xml", odr::FileType::xml);
.value("xml", odr::FileType::xml)
.value("iwork_pages", odr::FileType::iwork_pages)
.value("iwork_numbers", odr::FileType::iwork_numbers)
.value("iwork_keynote", odr::FileType::iwork_keynote);

py::enum_<odr::FileCategory>(m, "FileCategory")
.value("unknown", odr::FileCategory::unknown)
Expand Down
2 changes: 2 additions & 0 deletions src/odr/exceptions.cpp
Original file line number Diff line number Diff line change
Expand Up @@ -68,6 +68,8 @@ NoFontFile::NoFontFile() : Exception("not a font file") {}
NoLegacyMicrosoftFile::NoLegacyMicrosoftFile()
: Exception("not a legacy microsoft office file") {}

NoIworkFile::NoIworkFile() : Exception("not an iwork file") {}

NoXmlFile::NoXmlFile() : Exception("not an xml file") {}

NoSvgFile::NoSvgFile() : Exception("not an svg file") {}
Expand Down
5 changes: 5 additions & 0 deletions src/odr/exceptions.hpp
Original file line number Diff line number Diff line change
Expand Up @@ -142,6 +142,11 @@ struct NoLegacyMicrosoftFile final : Exception {
NoLegacyMicrosoftFile();
};

/// @brief No iWork file exception
struct NoIworkFile final : Exception {
NoIworkFile();
};

/// @brief No XML file exception
struct NoXmlFile final : Exception {
NoXmlFile();
Expand Down
8 changes: 8 additions & 0 deletions src/odr/file.hpp
Original file line number Diff line number Diff line change
Expand Up @@ -149,6 +149,14 @@ enum class FileType {
// `[text_file, xml, scalable_vector_graphics]`.
// https://en.wikipedia.org/wiki/XML
xml,

// https://en.wikipedia.org/wiki/IWork
iwork_pages,
// Classification only - `.numbers` and `.key` sit in the same package the
// pages engine reads, but which app wrote one is read off its root archive
// and no fixture pins those two, so nothing detects or decodes them yet.
iwork_numbers,
iwork_keynote,
};

/// @brief Collection of file categories.
Expand Down
45 changes: 45 additions & 0 deletions src/odr/internal/file_type_table.cpp
Original file line number Diff line number Diff line change
Expand Up @@ -250,6 +250,24 @@ constexpr std::array avi_mimetypes{"video/x-msvideo"sv, "video/avi"sv,
// `decrypt` on an OOXML document type means a password-protected package,
// detected as `office_open_xml_encrypted` and decrypting into the type named
// here. ODF files decrypt in place and keep their type.
constexpr std::array pages_extensions{"pages"sv};
constexpr std::array pages_mimetypes{
"application/vnd.apple.pages"sv,
"application/x-iwork-pages-sffpages"sv,
};

constexpr std::array numbers_extensions{"numbers"sv};
constexpr std::array numbers_mimetypes{
"application/vnd.apple.numbers"sv,
"application/x-iwork-numbers-sffnumbers"sv,
};

constexpr std::array keynote_extensions{"key"sv};
constexpr std::array keynote_mimetypes{
"application/vnd.apple.keynote"sv,
"application/x-iwork-keynote-sffkey"sv,
};

constexpr std::array table{
Row{FileType::unknown,
"unknown"sv,
Expand Down Expand Up @@ -755,6 +773,33 @@ constexpr std::array table{
.open = true,
.translate_html = true,
.color_scheme = true}},

Row{FileType::iwork_pages,
"pages"sv,
pages_extensions,
pages_mimetypes,
FileCategory::document,
DocumentType::text,
{.detect_by_content = true,
.open = true,
.translate_html = true,
.color_scheme = true}},
// Classified so a caller can name these two and hand their MIME types to a
// file picker; no engine reads either yet.
Row{FileType::iwork_numbers,
"numbers"sv,
numbers_extensions,
numbers_mimetypes,
FileCategory::document,
DocumentType::spreadsheet,
{}},
Row{FileType::iwork_keynote,
"key"sv,
keynote_extensions,
keynote_mimetypes,
FileCategory::document,
DocumentType::presentation,
{}},
};

/// Finds the row whose list, selected by @p list, contains @p needle.
Expand Down
109 changes: 109 additions & 0 deletions src/odr/internal/iwork/AGENTS.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,109 @@
# AGENTS.md β€” `internal/iwork`

Read the root [`AGENTS.md`](../../../../AGENTS.md) first, then
[`PLAN.md`](PLAN.md), which is where this module is going and in what order.
This file is what the landed stages decided, and why.

Landed: **stage 1** (detection and the container) and **stage 2** (Pages body
text). A `.pages` opens as a text document and renders its paragraphs.
Everything else in `PLAN.md` is still ahead.

## There is no spec, so a fixture is the citation

Apple has never published the `.proto` schemas and nothing is vendored under
`offline/documentation/`. Where `oldms/` writes `[MS-XLS] Β§2.4.1`, this module
writes `empty.pages Index/Document.iwa +0` β€” the byte layout verified against a
file in the repo is the only claim treated as fact.

Everything the engine reads by number lives in `iwork_types.hpp`, each constant
cited to the fixture it was read off. Read `numbers-parser`, `keynote-parser`,
`obriensp/iWorkFileFormat` and `libetonyek` for facts; **copy code from none of
them**.

**Fail soft on a type id we have not mapped, fail fast on broken framing.** The
root `AGENTS.md` says to throw where the spec dictates what to expect. Here
there is no spec, and an unknown type id means Apple shipped a version we have
not seen β€” a reader that throws on one cannot open next year's files. What does
throw: framing that overruns the file, a Snappy block that does not fill its
declared length, a varint that does not terminate, an identifier the package
does not hold, and text that is not UTF-8.

## No new dependencies

Two pieces would normally be a conan line each, and both would be wrong.

- **Snappy** β€” the `.iwa` framing is Apple's own (`0x00`, a little-endian
24-bit compressed length, repeated to EOF), not Snappy's stream framing, so
only the *block* decoder applies. `iwork_snappy.cpp` is that, in about a
hundred lines.
- **Protobuf** β€” only the wire format is needed, and with no schemas a code
generator has nothing to generate. Linking conan `protobuf` would drag it
into the wasm, android and apple builds to replace `iwork_protobuf.cpp`.

Both stay inside `iwork/` until something else wants them; a wire reader with
one user has not earned a package.

## `Message` views the buffer it was read from

`iwork::Message` parses one level eagerly and leaves nested messages, strings
and packed fields as `std::string_view`s into the bytes it was handed. So the
buffer has to outlive it β€” `Component` owns its decompressed data behind a
`unique_ptr` for exactly that reason, and a `Message(some_temporary())` is a
dangling read rather than a compile error.

## An `.iwa` is an object graph, not a tree

A component file is a flat sequence of `(varint length, TSP.ArchiveInfo,
payload)`, and objects reference each other by identifier β€” across components.
So `Package` reads the component list from `Index/Metadata.iwa` first and
decompresses a component when something in it is asked for; `object(id)` loads
further components until the identifier turns up. Walking files in directory
order and hoping a tree falls out is the mistake to avoid.

**Component names are not file names.** `Index/Metadata.iwa` maps a component's
name to its locator, and the locator carries an identifier suffix often enough
that globbing for `CalculationEngine.iwa` finds it in one fixture and not in
the other.

The one place that skips the component list is detection: `IworkFile` reads
`/Index/Document.iwa` directly, because it runs on every zip a caller opens and
`Document` is the component whose file name never carries a suffix.

## Which app wrote the package comes off the root archive

`TP.DocumentArchive` is type 10000, verified on both `.pages` fixtures. The
extension is not consulted β€” a caller may have lost it β€” and neither is
`Metadata/Properties.plist`, which names an app version but not the app.

That is also why only `.pages` is detected. `iwork_numbers` and `iwork_keynote`
have `file_type_table.cpp` rows so a caller can name them and hand a file
picker their MIME types, but no capabilities: reading their root archive types
off a guess is exactly what this module does not do, and neither has a fixture
in the test data yet.

## Paragraphs come from the run table

A `TSWP.StorageArchive` holds its text as a few large strings plus run tables
parallel to it β€” index/value pairs for paragraph styles, character styles and
attachments. Paragraph boundaries are the **paragraph style table's**, not
every `\n` in the text. The two agree on both fixtures, but the table is what
says so, and `U+2028` is a line break *inside* a paragraph rather than a
paragraph boundary.

Run tables count in **UTF-16 code units** while the text is UTF-8, so
`iwork_parser.cpp` translates the indices in one pass over the text. An index
that lands mid-character is an error, not a rounding.

`U+FFFC` is where a drawable is anchored. Nothing reads drawables yet, so the
anchor is dropped rather than rendered as a glyph β€” see stage 4.

`empty.pages` is the regression that matters at this level: a body storage that
carries no text at all must produce an empty body, not an exception.

## Not read yet

`Index/DocumentStylesheet.iwa` (so `text_root_page_layout` is empty and every
style is the default), drawables and images, `Index/Tables/`, and everything
`PLAN.md` lists as deferred. `password_encrypted()` is not answered either: an
encrypted package is one whose `Index/Document.iwa` does not decompress, which
falls back to reporting the file as a zip.
Loading
Loading