Skip to content

[Runtime] Fix wrong file offsets on shared file access on .NET 6 and later - #2082

Merged
RobertvanderHulst merged 3 commits into
X-Sharp:devfrom
hpetriffer:fix/xswin32filestream-position
Sep 15, 2026
Merged

RobertvanderHulst merged 3 commits into
X-Sharp:devfrom
hpetriffer:fix/xswin32filestream-position

Conversation

@hpetriffer

@hpetriffer hpetriffer commented Sep 15, 2026

Copy link
Copy Markdown
Contributor

Fixes #2081

The problem

XsWin32FileStream never overrides Position. Since the .NET 6 FileStream rewrite that makes FTell(), FEof(), FReadLine() and the whole RDD record positioning path (SafeReadAt() / SafeWriteAt() / SafeSetPos()) read and write at the wrong file offset, silently and without an exception, for every file opened with shared access.

The class does all of its I/O with the Win32 API on the OS file pointer, but Position falls through to the base class. Until .NET 5 that was fine: FileStream re-verified its cached position against the OS handle on every access once SafeFileHandle had been exposed (VerifyOSHandlePosition). The .NET 6 strategy rewrite removed that, so FileStream.Position is now a purely in-memory counter that the overrides never touch — it stays at 0 while the real file pointer moves.

Runtime Result
.NET Framework 4.8 OK
.NET Core 3.1 / .NET 5 OK
.NET 6 / 8 / 10 broken

Not a .NET 10 regression — it starts with .NET 6.

Reproduction

VAR h := FOpen(cFile, FO_READWRITE + FO_SHARED)   // -> XsWin32FileStream
FSeek3(h, 6, FS_SET)
? FTell(h)              // expected 6                      .NET 6+: 0
? FReadStr(h, 5)        // expected "World"                 ok
? FTell(h)              // expected 11                      .NET 6+: 0
FSeek3(h, 0, FS_SET)
? FReadLine(h, 100)     // expected "Hello World line1"     ok
? FReadLine(h, 100)     // expected "second line here"      .NET 6+: ""
? FEof(h)               // expected TRUE at end             .NET 6+: FALSE

The serious part is the RDD layer, which positions with oStream:Position := pos and then calls Read() / Write(). With a broken setter those land at the wrong offset with no exception:

.NET Framework 4.8                      .NET 10.0.11
OK   SafeReadAt(6,5) = "World"          FAIL SafeReadAt(6,5) = "Hello"
OK   SafeSetPos(0)+SafeRead = "Hello"   FAIL SafeSetPos(0)+SafeRead = " Worl"

The change

From .NET 6 on, FileStream reads and writes at an explicit offset and does not cache the length of a file that others may write to, so it is correct for shared access by itself. The shared stream therefore becomes a plain unbuffered FileStream that commits on Flush(), selected with the standard define:

#ifdef NET6_0_OR_GREATER   -> managed implementation
#else                      -> the existing Win32 implementation, unchanged
#endif

Below .NET 6 nothing changes. The #else branch is byte-for-byte the shipped class, CreateFileStream() keeps its original body including the platform check and the buffered non-Windows fallback, and XsFileStream is untouched. Verified by diff and by reflection over the built assemblies:

build P/Invokes declared members
net46 9 Flush, get_Length, Lock, Read, Seek, SetLength, Unlock, Write, WriteByte
net8.0 0 Flush, Lock, Unlock

Lock() / Unlock() are overridden on the managed shared stream to set NetErr(TRUE), because SetErrorState() only sets NetErr for a sharing violation (32) while a lock violation is 33 — the Win32 implementation does that itself. Deliberately on the shared stream only: putting it on XsFileStream also changed exclusive streams, which then reported NetErr on .NET 6+ but not on .NET Framework.

Reviewer notes

Two differences remain between the two paths. Both are pre-existing defects of the Win32 implementation, fixed only on the .NET 6+ side because the pre-.NET 6 path is intentionally left untouched:

  • Read() with a non-zero offset copies count bytes instead of the number actually read, zeroing the caller's buffer past the end of the file.
  • SetLength() seeks to the new length unconditionally instead of clamping only when the position is past the new end.

Every SetLength / SafeSetLength caller in the RDDs was checked; none depends on either behaviour.

The first two commits are intermediate steps (a Win32-based fix, then the fully managed one) and are superseded by the third — squash if preferred.

Testing

A purpose-built harness (~65 assertions: FTell/FEof/FReadLine, the RDD positioning path, EOF and partial reads, all Seek origins, truncate, lock error state, SafeFileHandle drift) run against the real XSharp.Core:

net48 net8.0 net10
harness 0 failures 0 failures 0 failures

XSharp.Core.Tests File IO 3/3 and XSharp.RDD.Tests 29/29 across repeated runs. Both target frameworks build with 0 warnings.

Lock error state verified identical on both targets — shared: FError=33, NetErr=True; exclusive: FError=33, NetErr=False.

Performance on net472 (BenchmarkDotNet, RDD record pattern) was measured before deciding to gate the change: the Win32 implementation is roughly 25 % faster there than the managed one, mostly because FileStream.Position does a SeekCore syscall on .NET Framework. That is why the old path is kept below .NET 6 rather than replaced everywhere.

hpetriffer and others added 3 commits September 11, 2026 16:05
XsWin32FileStream does all its IO with the Win32 API on the OS file pointer
but never overrode Position, so Position came from the base class. Until
.Net 5 that worked, because FileStream resynced its cached position from the
OS whenever the SafeFileHandle had been exposed. The .Net 6 FileStream
rewrite removed that resync, so Position stayed at 0 forever while the real
file pointer moved. FTell(), FEof(), FReadLine() and the SafeReadAt() /
SafeWriteAt() / SafeSetPos() calls that the RDDs use for every record all
read and wrote at the wrong offset, silently and without an exception, for
every file opened with shared access.

- Track the position in a field and pass it explicitly in an OVERLAPPED on
  every ReadFile() / WriteFile(), the way the .Net FileStream does it. The OS
  file pointer is no longer relevant for correctness and Seek() is arithmetic.
- Translate ERROR_HANDLE_EOF back into "0 bytes read": with an explicit offset
  Windows reports reading at or past the end of the file as a failure, where a
  read from the file pointer simply returned 0 bytes.
- Override ReadByte() and the Span overloads. FileStream overrides those on
  .Net Core and they bypassed the Read() / Write() overrides.
- Override SafeFileHandle: the base class moves the OS file pointer when the
  handle is exposed. SetLength() is the only operation that still moves it.
- Read() with offset != 0 copied count bytes instead of the number of bytes
  really read, overwriting the caller buffer past the end of the file.
- Read() returned -1 on failure, which violates the Stream.Read contract.
- Construct the base stream unbuffered. Its buffer was filled from the base
  class position and was never used by the overrides anyway.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
The hand written Win32 layer in XsWin32FileStream existed because the
FileStream of the time cached the file position and the file length, which is
wrong for a file that other processes may change. That is no longer true: the
.Net FileStream reads and writes at an explicit offset and does not cache the
length of a file that others may write to, so it reports the truth for shared
access by itself, on every platform.

XsWin32FileStream is now a plain XsFileStream that is unbuffered and commits
on Flush(). That removes 8 DllImports and every hand rolled Seek / Read /
Write / Lock / Length implementation, and with them the whole class of bugs
where our position and the position of the base class disagree.

Carried over, because these lived only in the Win32 class:
- Lock() and Unlock() set NetErr(TRUE) and FError(33) when a lock fails.
  SetErrorState() only sets NetErr for a sharing violation (32), so a lock
  violation had to be handled explicitly. Moved to XsFileStream, so the
  non-Windows path gets it too.
- A shared stream commits to disk on Flush().
- A shared stream is unbuffered.

Behaviour changes, both of them towards what the rest of the runtime already
did:
- SetLength() no longer moves the position to the new length unconditionally.
  It now follows the Stream contract and only clamps the position when it is
  past the new end of the file, which is what XsFileStream and the non-Windows
  path always did. The old behaviour was flagged in the source as a wart.
- Shared access is no longer buffered on non-Windows platforms. That branch
  passed a buffer size of 0xFFFF, so a second process could not see writes and
  reads could return stale data.

The class name is kept because it is public and is handed to user code through
DBI_FILESTREAM. CreateWin32FileStream() is kept as an obsolete forwarder to
the new CreateSharedFileStream().

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
The managed shared file stream is now selected with the standard .Net define
NET6_0_OR_GREATER. Below .Net 6 nothing changes at all: both the class and
CreateFileStream() are byte for byte what ships today, so .Net Framework is
untouched.

The Position bug only exists from .Net 6 on. Until .Net 5, FileStream
re-verified the OS file position on every access once the SafeFileHandle had
been exposed, which kept the Win32 implementation correct even though it never
overrode Position. The .Net 6 FileStream rewrite removed that re-verification.
From .Net 6 on, FileStream also reads and writes at an explicit offset and does
not cache the length of a file that others may write to, so the hand written
Win32 layer buys nothing there.

- SharedFileStream.prg carries both implementations, selected by
  #ifdef NET6_0_OR_GREATER. The #else branch is identical to the shipped code.
- CreateFileStream() keeps its original body for the pre .Net 6 path, including
  the platform check and the buffered non Windows fallback. Only the .Net 6 path
  is new, and it needs no platform check because it is plain managed code.
- The managed shared stream overrides Lock() and Unlock() to set NetErr(TRUE),
  because SetErrorState() only sets NetErr for a sharing violation (32) and a
  lock violation is 33. The Win32 implementation does that itself. Deliberately
  on the shared stream only: putting it on XsFileStream also changed exclusive
  streams, which then reported NetErr on .Net 6+ but not on .Net Framework.

Two differences between the two paths, both pre existing defects of the Win32
implementation that are now fixed on .Net 6 and later only:
- Read() with a non zero offset copies count bytes instead of the number of
  bytes really read, so the caller buffer is zeroed past the end of the file.
- SetLength() seeks to the new length unconditionally instead of only clamping
  the position when it is past the new end.
No caller in the RDDs depends on either.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
@hpetriffer hpetriffer closed this Sep 15, 2026
@hpetriffer hpetriffer reopened this Sep 15, 2026
@hpetriffer
hpetriffer marked this pull request as ready for review September 15, 2026 06:36
@RobertvanderHulst
RobertvanderHulst merged commit 47b6695 into X-Sharp:dev Sep 15, 2026
2 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants