feat(rtf): read an rtf as a text document

AGENTS.md

@@ -62,6 +62,7 @@ bytes ─▶ magic/open_strategy ─▶ DecodedFile ─▶ Document ─▶ Eleme
 | `src/odr/internal/ooxml/` | OOXML (docx/pptx/xlsx); see [`ooxml/AGENTS.md`](src/odr/internal/ooxml/AGENTS.md) + per-format docs. |
 | `src/odr/internal/oldms/` | **Legacy MS binary** (.doc/.ppt/.xls). |
 | `src/odr/internal/pdf/` | PDF (own parser). |
+| `src/odr/internal/rtf/` | RTF, read as a text document; see [`rtf/AGENTS.md`](src/odr/internal/rtf/AGENTS.md) + [`rtf/PLAN.md`](src/odr/internal/rtf/PLAN.md). |
 | `src/odr/internal/xml/` | XML, rendered as a source view; see [`xml/AGENTS.md`](src/odr/internal/xml/AGENTS.md). |
 | `src/odr/internal/svg/` | SVG, detected by reading it as xml; see [`svg/AGENTS.md`](src/odr/internal/svg/AGENTS.md). |
 | `src/odr/internal/{csv,json,text,svm}/` | Smaller formats. |

CHANGELOG.md

@@ -16,6 +16,12 @@ The release run heads these entries with the version and opens a fresh
 
 ## Unreleased
 
+- An rtf opens and renders instead of throwing `UnknownFileType`. Text, its
+  encoding (`\ansicpgN`, `\'hh`, `\uN` including emoji), paragraphs, line
+  breaks and tabs; character and paragraph formatting, tables and pictures are
+  not read yet. `FileType::rich_text_format` now reports `DocumentType::text`
+  and the `open` / `translate_html` / `color_scheme` capabilities.
+
 ## v6.10.1 - 2026-08-21
 
 - A linked image in a docx or xlsx (`embed_images = false`) is named relative

CMakeLists.txt

@@ -239,6 +239,13 @@ set(ODR_SOURCE_FILES
         "src/odr/internal/font/sfnt_transform.cpp"
         "src/odr/internal/font/font_file.cpp"
 
+        "src/odr/internal/rtf/rtf_document.cpp"
+        "src/odr/internal/rtf/rtf_element_registry.cpp"
+        "src/odr/internal/rtf/rtf_file.cpp"
+        "src/odr/internal/rtf/rtf_parser.cpp"
+        "src/odr/internal/rtf/rtf_state.cpp"
+        "src/odr/internal/rtf/rtf_tokenizer.cpp"
+
         "src/odr/internal/svg/svg_file.cpp"
 
         "src/odr/internal/svm/svm_file.cpp"

src/odr/exceptions.cpp

@@ -72,6 +72,8 @@ NoXmlFile::NoXmlFile() : Exception("not an xml file") {}
 
 NoSvgFile::NoSvgFile() : Exception("not an svg file") {}
 
+NoRtfFile::NoRtfFile() : Exception("not an rtf file") {}
+
 UnsupportedCryptoAlgorithm::UnsupportedCryptoAlgorithm()
     : Exception("unsupported crypto algorithm") {}
 

src/odr/exceptions.hpp

@@ -152,6 +152,11 @@ struct NoSvgFile final : Exception {
   NoSvgFile();
 };
 
+/// @brief No RTF file exception
+struct NoRtfFile final : Exception {
+  NoRtfFile();
+};
+
 /// @brief Unsupported crypto algorithm exception
 struct UnsupportedCryptoAlgorithm final : Exception {
   UnsupportedCryptoAlgorithm();

src/odr/internal/file_type_table.cpp

@@ -395,21 +395,25 @@ constexpr std::array table{
          .color_scheme = true}},
 
     // Recognised by magic so a caller can name the type, but there is no
-    // decoder behind either of these.
+    // decoder behind it.
     Row{FileType::word_perfect,
         "wpd"sv,
         wpd_extensions,
         wpd_mimetypes,
         FileCategory::document,
         DocumentType::unknown,
         {.detect_by_content = true}},
+
     Row{FileType::rich_text_format,
         "rtf"sv,
         rtf_extensions,
         rtf_mimetypes,
         FileCategory::document,
-        DocumentType::unknown,
-        {.detect_by_content = true}},
+        DocumentType::text,
+        {.detect_by_content = true,
+         .open = true,
+         .translate_html = true,
+         .color_scheme = true}},
 
     Row{FileType::portable_document_format,
         "pdf"sv,

src/odr/internal/open_strategy.cpp

@@ -18,6 +18,7 @@
 #include <odr/internal/oldms/oldms_file.hpp>
 #include <odr/internal/ooxml/ooxml_file.hpp>
 #include <odr/internal/pdf/pdf_file.hpp>
+#include <odr/internal/rtf/rtf_file.hpp>
 #include <odr/internal/svg/svg_file.hpp>
 #include <odr/internal/svm/svm_file.hpp>
 #include <odr/internal/xml/xml_file.hpp>
@@ -112,6 +113,16 @@ open_file_as(const std::shared_ptr<abstract::File> &file, const FileType as,
     throw NoPdfFile();
   }
 
+  if (as == FileType::rich_text_format) {
+    ODR_VERBOSE(logger, "open as rtf");
+    try {
+      return std::make_unique<rtf::RtfFile>(file);
+    } catch (...) {
+      ODR_VERBOSE(logger, "failed to open as rtf");
+    }
+    throw NoRtfFile();
+  }
+
   if (as == FileType::starview_metafile) {
     ODR_VERBOSE(logger, "open as svm");
     try {
@@ -396,6 +407,10 @@ open_strategy::open_file(const std::shared_ptr<abstract::File> &file,
     ODR_VERBOSE(logger, "open as pdf");
     return std::make_unique<pdf::PdfFile>(file);
   }
+  if (file_type == FileType::rich_text_format) {
+    ODR_VERBOSE(logger, "open as rtf");
+    return std::make_unique<rtf::RtfFile>(file);
+  }
   if (file_type == FileType::starview_metafile) {
     ODR_VERBOSE(logger, "open as svm");
     return std::make_unique<svm::SvmFile>(file);
@@ -565,6 +580,9 @@ open_strategy::open_document_file(const std::shared_ptr<abstract::File> &file,
     } catch (...) {
       ODR_VERBOSE(logger, "failed to open as ooxml");
     }
+  } else if (file_type == FileType::rich_text_format) {
+    ODR_VERBOSE(logger, "open as rtf");
+    return std::make_unique<rtf::RtfFile>(file);
   }
 
   ODR_ERROR(logger, "unsupported file type for document file "

src/odr/internal/rtf/AGENTS.md

@@ -0,0 +1,97 @@
+# AGENTS.md — `internal/rtf`
+
+Rich Text Format, read as a text document. Read [`PLAN.md`](PLAN.md) first: it
+carries the staging, the decisions taken up front and what each later stage
+owes. This file records what stage 1 actually built and where it deviates.
+
+Spec references are the RTF Specification 1.9.1 (March 2008). It has no section
+numbers, so cite the heading and the control word — *Conventions of an RTF
+Reader*, *Control Word*, *Table Definitions*. Not `[MS-OXRTFEX]` /
+`[MS-OXRTFCP]`, which are different documents and out of scope.
+
+## Shape
+
+```
+bytes ─▶ Tokenizer ─▶ TreeBuilder ─▶ ElementRegistry ─▶ Document ─▶ RtfFile
+         (rtf_tokenizer)  (rtf_parser)                  (rtf_document)  (rtf_file)
+```
+
+| File | What |
+|------|------|
+| `rtf_token.hpp` | The `Token` variant the tokenizer yields. |
+| `rtf_tokenizer.*` | Bytes → tokens. Knows nothing about groups or destinations. |
+| `rtf_state.*` | The group stack: `{` saves, `}` restores. |
+| `rtf_parser.*` | `parse_tree` — tokens → `root → paragraph → (text \| line break)`. |
+| `rtf_element_registry.*` | The flat registry, copied from `oldms/text` minus its style index. |
+| `rtf_document.*` | `internal::Document` + the element adapter. |
+| `rtf_file.*` | `abstract::DocumentFile`; validates the magic, hands out the document. |
+
+There is no filesystem: an rtf is one byte stream, so `internal::Document` gets
+a null `ReadableFilesystem` the way `csv` does.
+
+## What stage 1 decodes
+
+Paragraph structure and text: `\par`, `\line`, `\tab`, `\page`, `\sect`, the
+literal-character control words (`\emdash`, `\bullet`, the quotes, …), the
+escapes `\\` `\{` `\}` `\~` `\_` `\-`, `\'hh` in the run's encoding, and `\uN`
+including surrogate pairs. The encoding comes from `\ansi` / `\mac` / `\pc` /
+`\pca` / `\ansicpgN` through `internal/encoding`.
+
+Everything else is ignored, which is the spec's own rule for a reader meeting a
+control word it does not know. Character and paragraph formatting, tables,
+pictures and lists are stages 2–5 in `PLAN.md`.
+
+## Rules worth knowing before touching this
+
+- **Leniency is the spec here, and does not violate the root `AGENTS.md`
+  fail-fast rule.** Unknown control words are ignored, `{\*` groups whose
+  destination we do not implement are discarded, an unmatched `}` is ignored.
+  What *does* throw: a group left open at EOF, an invalid hex digit after `\'`,
+  a `\binN` running past EOF, a trailing `\`, and nesting past `State`'s depth
+  bound.
+- **`\binN` is read by the tokenizer, not the parser.** Its payload is raw
+  bytes that may contain braces and backslashes, so a brace-counting scan over
+  them would desync the group nesting — including inside a group being
+  discarded. That is why skipping an ignorable destination still tokenizes.
+- **Text is bytes until the run ends.** `\'hh` yields one *byte*: in a
+  double-byte run two consecutive escapes are one character. The accumulator is
+  `m_bytes` plus the run's `TextEncoding`, decoded through `encoding::to_utf8`
+  only at a flush. Decoding per escape corrupts every multibyte run.
+- **`\uN` is a UTF-16 code unit, signed.** `U+F020` arrives as `\u-4064` (fold
+  with `+ 65536` *before* the surrogate test), and anything above the BMP
+  arrives as a surrogate pair across two `\uN`. A high surrogate is held pending
+  and combined with the low one; unpaired, it becomes U+FFFD.
+- **`\ucN` counts control words and symbols as one character each**, strictly,
+  a `\binN` and its payload included. Real writers always emit the fallback, so
+  the strict reading costs nothing and is what the spec says. A group boundary
+  cancels a pending skip.
+- **A `{\*`-marked destination needs no entry in the discard table**; the `\*`
+  control symbol discards whatever follows it. The table in `rtf_parser.cpp` is
+  only for destinations that are *not* marked — `\fonttbl`, `\colortbl`,
+  `\info`, `\pict`, and notably `\nonshppict`, the unmarked twin of
+  `{\*\shppict}` that would otherwise emit every image a second time.
+
+## Deviations from `PLAN.md`
+
+- **`HexEscape` is its own token**, not folded into `Text`. The plan's variant
+  had no place to put the decoded byte of a `\'hh`, and `\ucN` counts an escape
+  as one character where a text run counts bytes.
+- **`\page` emits `ElementType::page_break`** (a child of root, as `oldms/text`
+  does) even though the html renderer ignores that type today.
+- **`\cell` renders as a tab and `\row` as a paragraph end.** The plan defers
+  tables to stage 4; until then this keeps table text readable rather than
+  running it together.
+- **An undecodable run degrades per byte**, not per run: ascii passes through
+  and only bytes ≥ 0x80 become U+FFFD. The plan said the whole run degrades,
+  which loses the ascii skeleton for no reason.
+
+## Testing
+
+Everything is inline string literals — an rtf fragment is readable in a raw
+string, and there is no `.rtf` anywhere under `test/data`. `rtf_tokenizer_test`
+covers the delimiter rules token by token; `rtf_document_test` runs
+`parse_tree` and flattens the tree to one line (`P(…)`, `|`, `PB`).
+
+A render test against a real fixture needs a file committed to
+`test/data/input` plus the reference-output regen, and is owed once stage 5
+lands a picture that cannot be written inline.

src/odr/internal/rtf/PLAN.md

@@ -5,12 +5,15 @@ it honest as stages land.
 
 ## Today
 
-Nothing decodes. `FileType::rich_text_format` exists (`file.hpp:62`, under the
-"Detection only" comment), `magic.cpp:123` matches `7B 5C 72 74 66 31`
-(`{\rtf1`), and `file_type_table.cpp:390` carries a row with `rtf` extensions,
-three mime types, `FileCategory::document`, `DocumentType::unknown` and
-`{.detect_by_content = true}`. `open_strategy::open_file` has no branch for it,
-so `odr::open` on an rtf reaches the fallthrough and throws `UnknownFileType`.
+**Stage 1 has landed.** An rtf opens as a text document and renders: the
+tokenizer, the group/destination machinery and plain text with paragraphs, line
+breaks, tabs and the full `\'hh` / `\uN` decoding. The table row now declares
+`DocumentType::text` and `{.open, .translate_html, .color_scheme}`, and
+`open_strategy` has its three branches. See [`AGENTS.md`](AGENTS.md) for what
+was built and where it deviates from what follows.
+
+Stages 2–5 below are untouched: no character or paragraph formatting, no page
+layout, no tables, no pictures.
 
 The public enum is already mirrored by the bindings (`bind_file.cpp:39`,
 `ODRFile.mm:43`), so **no binding work is needed for any stage below** — the
@@ -132,7 +135,7 @@ it rather than inventing a shared one.
 
 ---
 
-## Stage 1 — plumbing, tokenizer, plain text
+## Stage 1 — plumbing, tokenizer, plain text — **done**
 
 The narrowest thing that renders. No formatting at all beyond paragraph
 structure, so the tokenizer and the group machinery can be proven before