Render and input text in languages other than English on macOS - #2
Open
hard25670559 wants to merge 3 commits into
Open
Render and input text in languages other than English on macOS#2hard25670559 wants to merge 3 commits into
hard25670559 wants to merge 3 commits into
Conversation
added 3 commits
August 10, 2026 20:49
The bundled bitmap fonts only carry the first 128 codepoints, so every other character was drawn as a "[U+XXXX]" placeholder. Build names, notes, account names and item text pasted from a localised client are effectively restricted to ASCII, whatever language the user works in. Add a parallel glyph path instead of replacing the existing one: - r_dynfont (macOS/CoreText) rasterises a single codepoint on demand. Going through CTFontCreateForString gives us the system fallback chain, so Han, kana, Cyrillic and emoji all resolve without shipping or hand-picking a font. - r_font caches those glyphs per (codepoint, size) as small textures and consults them in the three places that previously produced tofu: width measurement, cursor hit testing, and drawing. Latin text still goes through the bitmap fonts untouched, so existing layout is unchanged, and tofu remains the fallback when no glyph exists. Glyphs are rasterised at the display's pixel density and scaled back to layout units so they stay sharp on HiDPI, and are anchored to a fixed fraction of the line height: the font's own ascent includes leading and would sit noticeably below where the bitmap fonts place their baseline. Also fix r_tex_c's image constructor, which assigned the mip set to its parameter rather than the member that PerformUpload reads, so building a texture from memory always dereferenced a null image.
Typing with an IME did nothing visible: the composition never reached the application, keys the IME was about to consume also hit the UI, and the candidate window opened in a fixed screen corner. Committed text was turned into "?" before it arrived. GLFW's content view conforms to NSTextInputClient but leaves several parts inert, so replace those at runtime and chain to the originals rather than forking the library: - setMarkedText: forwards the composition up to the UI so it can be drawn at the caret; unmarkText clears it. - firstRectForCharacterRange: answers with the caret rectangle the UI reports, converted to screen space, so the candidate window follows the text instead of sitting in a corner. - markedRange was one unit short of the text it holds and selectedRange was missing entirely; both are consulted while positioning the candidate window and a bad answer makes the system give up on it. - keyDown: withholds keys from the application while a composition is in progress. GLFW reports the raw key first, which let Return confirm a dialog before the composed text ever arrived. - insertText: clears the marked text GLFW leaves behind on commit, which otherwise kept every later keystroke away from the application. On the UI side, character events now carry the codepoint encoded as UTF-8 instead of being looked up in the key name table, which only knows named keys and answered "?" for anything above ASCII. Composition state is delivered through a new OnPreedit callback, and SetIMECaretRect lets the focused control report where its caret is.
The system caches where it believes the caret is and only re-reads it when told the coordinates are stale, so a candidate window could open at a position the caret had already left. Invalidate those coordinates whenever the reported rectangle changes. Also answer attributedSubstringForProposedRange with the text being composed rather than nil, which is what an input method consults for context around the composition.
Author
|
Also submitted upstream as PathOfBuildingCommunity#114. Same reasoning as #1: the change belongs to the engine, but it was developed and verified on this branch because |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Problem
The bundled bitmap fonts carry only the first 128 codepoints, so every other character is drawn as a
[U+XXXX]placeholder. Typing through an input method does nothing visible at all: the composition never reaches the application, keys the IME is about to consume also hit the UI, the candidate window opens in a screen corner, and committed text is turned into?on the way in.Together this means build names, notes, account names and item text pasted from a localised client are effectively limited to ASCII, whatever language the user works in. PathOfBuildingCommunity/PathOfBuilding#5632 has been open on this since early 2023; users there resort to third-party rebuilt engine binaries.
Changes
Glyph rendering — a parallel path rather than a replacement. A new macOS module rasterises a codepoint on demand through CoreText; going via
CTFontCreateForStringuses the system fallback chain, so Han, kana, Cyrillic and emoji all resolve with no font to bundle or curate.r_fontcaches these per (codepoint, size) as small textures and consults them in the three places that previously produced tofu — width measurement, cursor hit testing and drawing. Latin text still goes through the bitmap fonts untouched, so existing layout is unchanged, and tofu remains the fallback when no glyph exists anywhere.Glyphs are rasterised at the display's pixel density and scaled back to layout units so they stay sharp on HiDPI, and are anchored to a fixed fraction of the line height: a font's own ascent includes leading and sits noticeably below where the bitmap fonts place their baseline.
Input method support — GLFW's content view conforms to
NSTextInputClientbut leaves several parts inert, so those are replaced at runtime and chained to the originals rather than forking the library:setMarkedText:forwards the composition to the UI so it can be drawn at the caret;unmarkTextclears it.firstRectForCharacterRange:answers with the caret rectangle the UI reports, so the candidate window follows the text.markedRangewas one unit short of the text it holds andselectedRangewas missing entirely — both are consulted while positioning the candidate window, and a bad answer makes the system give up on it.keyDown:withholds keys from the application while composing. GLFW reports the raw key first, which let Return confirm a dialog before the composed text arrived.insertText:clears the marked text GLFW leaves behind on commit, which otherwise kept every later keystroke away from the application.UI plumbing — character events carry the codepoint encoded as UTF-8 instead of being looked up in the key name table, which only knows named keys and answered
?for anything above ASCII. Composition state is delivered through a newOnPreeditcallback, andSetIMECaretRectlets the focused control report where its caret is.Dependencies
SetIMECaretRectis absent.Verification
Built for
arm64-osxand run on a three-display setup.Known limitation
With the system Bopomofo input method the candidate window appears at a stale position for the first character of a session and corrects itself once you move through the candidates. It picks that first position from a cached caret location without querying the application; other applications on the same machine behave identically. Third-party input methods are unaffected.