Skip to content

Specialize UTF8 encoding in StringTextConverter writing#6297

Draft
bbowyersmyth wants to merge 1 commit intonpgsql:mainfrom
bbowyersmyth:users/bruceb/utf8textconverter
Draft

Specialize UTF8 encoding in StringTextConverter writing#6297
bbowyersmyth wants to merge 1 commit intonpgsql:mainfrom
bbowyersmyth:users/bruceb/utf8textconverter

Conversation

@bbowyersmyth
Copy link
Contributor

@bbowyersmyth bbowyersmyth commented Nov 8, 2025

To avoid the double counting of UTF8 bytes in writing UTF16 strings, create a specialized StringBasedTextConverter that can use the newer System.Text.Unicode.Utf8 namespace.

When the encoding is the same object instance as NpgsqlWriteBuffer.UTF8Encoding, which is a throwing encoder, we can handle partial encodes to the write remaining buffer and iterate if needed.

Before

Method Value Mean Error StdDev Op/s Gen0 Allocated
Write xxxxxxxxxx 17.62 ns 0.052 ns 0.048 ns 56,761,267.5 0.0000 -
WriteAsync xxxxxxxxxx 18.96 ns 0.028 ns 0.024 ns 52,749,799.6 0.0000 -
Write xxxx(...)xxxx [100] 23.58 ns 0.043 ns 0.038 ns 42,413,674.2 0.0005 5 B
WriteAsync xxxx(...)xxxx [100] 24.22 ns 0.109 ns 0.102 ns 41,293,211.8 0.0005 5 B
Write xxxx(...)xxxx [1000] 66.98 ns 0.292 ns 0.274 ns 14,928,853.2 0.0051 54 B
WriteAsync xxxx(...)xxxx [1000] 76.86 ns 0.203 ns 0.180 ns 13,010,451.6 0.0051 54 B

After

Method Value Mean Error StdDev Op/s Gen0 Allocated
Write xxxxxxxxxx 13.22 ns 0.024 ns 0.021 ns 75,642,930.8 - -
WriteAsync xxxxxxxxxx 15.90 ns 0.036 ns 0.034 ns 62,878,778.0 - -
Write xxxx(...)xxxx [100] 15.75 ns 0.024 ns 0.021 ns 63,478,157.3 - -
WriteAsync xxxx(...)xxxx [100] 18.56 ns 0.034 ns 0.030 ns 53,876,045.3 - -
Write xxxx(...)xxxx [1000] 34.91 ns 0.148 ns 0.139 ns 28,646,020.9 - -
WriteAsync xxxx(...)xxxx [1000] 44.01 ns 0.179 ns 0.149 ns 22,723,470.8 - -

@bbowyersmyth
Copy link
Contributor Author

bbowyersmyth commented Nov 8, 2025

@NinoFloris Another alternative to #5985, what do you think about this approach?
#6016 could then be moved to the new class as well.

@NinoFloris
Copy link
Member

NinoFloris commented Nov 17, 2025

Thanks @bbowyersmyth, that might work. I will take a better look after 10.0 is released, currently busy getting the remaining bits ready for release.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@roji roji Awaiting requested review from roji roji will be requested when the pull request is marked ready for review roji is a code owner

@vonzshik vonzshik Awaiting requested review from vonzshik vonzshik will be requested when the pull request is marked ready for review vonzshik is a code owner

Assignees

No one assigned

Labels

None yet

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

2 participants

@bbowyersmyth @NinoFloris