Add the output_eol setting, and the \e escape sequence of string literals

[gdisirio]

Drafted by Claude (an AI coding assistant) working on this branch with me. The design decisions below — the three-state semantics with "unset" preserving the current behavior, keeping \e deterministic rather than platform-dependent by default, and normalizing in TextBlock rather than by wrapping the Writer — were discussed with me and are mine; the implementation and this write-up are its work.

---

Backward compatibility first

Nothing changes unless you ask for it. The new output_eol setting defaults to not being set, and in that state FreeMarker behaves exactly as it does today: the static text of the template is written out with whatever line breaks the template file happens to contain. There is no incompatible_improvements gating needed, because there is no behavior to gate — the default is the old behavior.

The \e escape is additive in the same way hex literals were: "\e" is currently a parse error (the lexer rejects unknown escapes in string literals), so no currently valid template can change meaning.

What it's for

There's currently no way to tell FreeMarker what line breaks the output should use. Whether the output ends up with LF or CRLF depends on the editor and operating system of whoever last saved the template file. That's invisible in the template source, it survives copying between machines, and when the generated files are stored in a version control system it shows up as line ending churn in diffs.

Setting output_eol prescribes the line break, and then it doesn't matter what the template file itself uses.

Semantics

output_eol	Static text of the template	\e gives
unset (default)	written as-is	line feed
"\n" / "\r\n" / "\r"	all line breaks replaced with it	that value
"JVM default"	replaced with the JVM's line separator	same

Two things are deliberately not affected:

• Values inserted by ${...}. The setting is about the template, not about the data. A string in the data model that contains CRLF is written out unchanged.
• \n. It keeps giving a line feed even when output_eol is set. Having both only makes sense if they differ: \n when you need that exact character, \e when you need "whatever the output should use".

"JVM default" follows the convention locale, time_zone and default_encoding already use. I'd suggest treating it as a documented escape hatch rather than something to reach for: it makes the output depend on the machine that produced it, which is the opposite of what you want if the output is committed to a repository. The manual entry says as much.

On the implementation

The static text is normalized in TextBlock, not by wrapping the output Writer. Wrapping the Writer was the obvious approach and it's wrong twice over:

1. It would cost every template. Every character of all output would have to be inspected, including for the templates that never set the setting. TextBlock currently stores char[] rather than String specifically to avoid one array copy per write — adding a per-character scan on that path would undo considerably more than that comment was protecting.
2. It can't tell \n from \e. Both are ordinary line feeds by the time output reaches the Writer, so normalizing there would silently rewrite deliberate \ns and destroy the distinction above.

Instead TextBlock caches the normalized text, keyed on the setting value it was computed for; that setting practically never changes while a template is executed, so it's computed once per node and reused. When the setting is unset, the original array is written directly, so the existing path is untouched. The cache is a single immutable holder object rather than two fields, so that a thread which doesn't see another thread's write just recomputes, and can never observe a half-initialized state.

The \e escape reuses the existing placeholder mechanism: it's decoded to a placeholder character when the template is parsed, and resolved when the string literal is evaluated, since output_eol is a Configurable and so its value is only known at that point.

Testing & docs

• OutputEolTest: 27 tests. The static text ones are parameterized over LF/CRLF/CR template files × the four setting states. The rest cover \e, that \n and interpolated values are unaffected, that unknown escapes are still rejected, <#setting> in both naming conventions, setSetting including "JVM default" and "null", and inheritance from the Configuration.
• Manual: the escape sequence table, the setting directive's list of settings, and a 2.3.36 version history section.
• ./gradlew check and ./gradlew manualOffline pass.

One thing you'll want to know about, unrelated to this PR

DefaultObjectWrapperTest.testIncompatibleImprovementsVersionBreakPoints currently fails on 2.3-gae, and it fails the same way with this branch's changes stashed, so it isn't caused by this PR. It iterates versions up to Configuration.getVersion() while its expected list stops at 2.3.35, so bumping the version to 2.3.36-SNAPSHOT in bea5c3577 left it one entry short. I left it alone rather than mixing an unrelated fix into this branch, but it will fail for you too until someone adds the entry.

[ddekany]

I think it's enough to add this on ParserConfiguration level, and then we don't need to add overhead to TextBlock.

DefaultObjectWrapperTest.testIncompatibleImprovementsVersionBreakPoints failing: Fixed it... sorry about that.

[ddekany]

(I meant that I have already fixed it in the last comment. I updated that comment, but not sure how notifications go out.)

[gdisirio]

Drafted by Claude (an AI coding assistant) working on this branch with me. The decision to take your ParserConfiguration suggestion, and to drop the setting directive support that follows from it, was discussed with me and is mine; the implementation and this write-up are its work.

---

You're right, and it's a better design than mine — thanks. Reworked and force-pushed.

Making it a ParserConfiguration setting removes more than the TextBlock overhead. Since the parser now knows the value, the \e escape can be decoded to the actual line break while the string literal is being parsed, rather than to a placeholder that gets resolved on each evaluation. So the placeholder and its resolver are gone too, and with them the only reason StringLiteral had to do anything at runtime.

The result is that no class involved in executing a template is touched at all. TextBlock, StringLiteral, Environment and Configurable are now byte-identical to 2.3-gae; the static text is normalized once, when parsed. My previous version had a cache on TextBlock to keep the cost down — that whole problem simply doesn't arise now.

What this changes for users

<#setting output_eol=...> is gone, since a parser setting can't be changed while the template runs. I think that's the right trade: switching line endings midway through a file isn't a sensible thing to want, and it can still differ per template via TemplateConfiguration, which is the case that matters. Say if you'd rather keep it settable and pay the runtime cost.

Configuration.setOutputEol(String) is unchanged as an API, so that part reads the same as before.

Also

Thanks for fixing DefaultObjectWrapperTest — I've rebased onto it, and ./gradlew check is fully green now, with no pre-existing failures to work around.

./gradlew manualOffline passes too. The manual no longer lists output_eol among the setting directive's settings, and the version history entry says it's parser-level.

[ddekany]

The setting name output_eol is somewhat misleading, as it sounds as if you filter the output of the templates (so before the output Writer). I think template_eol expresses this better. But I'm open for any better name.

For some people, \e is already associated with the ASCII Escape character. It's sometimes used that way in Bash, etc. I was considering \R which means "any line-break" in regular expressions. Or maybe \N?

[ddekany]

Or maybe slightly better template_normalized_eol? Because template_eol sounds as if we were only considering that specific type of EOL as a EOL.

[gdisirio]

Drafted by Claude (an AI coding assistant) working on this branch with me; the choices below were discussed with me and are mine.

---

Agreed on both counts, and template_normalized_eol is the better of your two — it says the value is what the template's line breaks are normalized to, which template_eol leaves open. I'll take it. (normalized_eol would fit the existing parser settings, none of which carry a template_ prefix, but I'd rather be unambiguous than short here — your call if you prefer the shorter one.)

On the escape: you're right that \e is taken in people's heads by ASCII ESC. But both alternatives collide in case with an existing escape whose meaning is adjacent:

• \R vs \r — carriage return
• \N vs \n — line feed, and here the two would be near-synonyms, so a typo changes behaviour silently

FTL has no uppercase escapes at all today, so this would introduce case-significance at exactly the spot where the meanings are easiest to confuse.

What about \eol? No ESC association, nothing to remember, and it can't be mistyped into something else that works. \xHHHH is already multi-character, so the lexer isn't limited to single letters.

Happy to go with \R if you'd rather keep it to one letter — just wanted the collision on the record first. Let me know which, and I'll push the rename.

[ddekany]

True, normalized_eol fits more. People has to look into the docs to know when/were we normalize anyway.

Escape: FTL is case-sensitivem, so if I see \R, I would not blindly think that it's just some alias to \r. I guess \eol can become annyoing if you relly on this feature (because then I guess you will use it often).

BTW, officially, these should be decided on the dev@ mailing list. I will point to there over there, just in case someone care and yet doesn't notice these comments.