Release 5.16.0

dotnet · Dec 5, 2024 · 754bc9f · 754bc9f
1 parent c282cf4
commit 754bc9f
Show file tree

Hide file tree

Showing 81 changed files with 1,580 additions and 463 deletions.
diff --git a/CHANGELOG.md b/CHANGELOG.md
@@ -1,7 +1,35 @@
 Release Notes
 ====
 
-# 10-16-2023
+# 12-05-2024
+<a href="https://www.nuget.org/packages/dotnext/5.16.0">DotNext 5.16.0</a>
+* Added [LEB128](https://en.wikipedia.org/wiki/LEB128) encoder and decoder as a public API. See `DotNext.Buffers.Binary.Leb128<T>` type for more information
+* Added `SlideToEnd` method to `SpanWriter<T>` type
+* Added `IsBitSet` and `SetBit` generic methods to `Number` type
+* Added `DetachOrCopyBuffer` to `BufferWriterSlim<T>` type
+
+<a href="https://www.nuget.org/packages/dotnext.metaprogramming/5.16.0">DotNext.Metaprogramming 5.16.0</a>
+* Updated dependencies
+
+<a href="https://www.nuget.org/packages/dotnext.unsafe/5.16.0">DotNext.Unsafe 5.16.0</a>
+* Updated dependencies
+
+<a href="https://www.nuget.org/packages/dotnext.threading/5.16.0">DotNext.Threading 5.16.0</a>
+* Async locks with synchronous acquisition methods now throw [LockRecursionException](https://learn.microsoft.com/en-us/dotnet/api/system.threading.lockrecursionexception) if the current thread tries to acquire the lock synchronously and recursively.
+* Added support of cancellation token to synchronous acquisition methods of `AsyncExclusiveLock` and `AsyncReaderWriterLock` classes
+* Introduced `LinkTo` method overload that supports multiple cancellation tokens
+
+<a href="https://www.nuget.org/packages/dotnext.io/5.16.0">DotNext.IO 5.16.0</a>
+* Introduced `RandomAccessStream` class that represents [Stream](https://learn.microsoft.com/en-us/dotnet/api/system.io.stream) wrapper over the underlying data storage that supports random access pattern
+* Added extension method for `SpanWriter<byte>` that provides length-prefixed string encoding
+
+<a href="https://www.nuget.org/packages/dotnext.net.cluster/5.16.0">DotNext.Net.Cluster 5.16.0</a>
+* Updated dependencies
+
+<a href="https://www.nuget.org/packages/dotnext.aspnetcore.cluster/5.16.0">DotNext.AspNetCore.Cluster 5.16.0</a>
+* Updated dependencies
+
+# 10-16-2024
 <a href="https://www.nuget.org/packages/dotnext.threading/5.15.0">DotNext.Threading 5.15.0</a>
 * Added support of synchronous lock acquisition to `AsyncExclusiveLock`, `AsyncReaderWriterLock`, `AsyncManualResetEvent`, `AsyncAutoResetEvent` so the users can easily migrate step-by-step from monitors and other synchronization primitives to async-friendly primitives
 * Fixed random `InvalidOperationException` caused by `RandomAccessCache<TKey, TValue>`

diff --git a/README.md b/README.md
@@ -44,12 +44,34 @@ All these things are implemented in 100% managed code on top of existing .NET AP
 * [NuGet Packages](https://www.nuget.org/profiles/rvsakno)
 
 # What's new
-Release Date: 10-16-2024
+Release Date: 12-05-2024
 
-<a href="https://www.nuget.org/packages/dotnext.threading/5.15.0">DotNext.Threading 5.15.0</a>
-* Added support of synchronous lock acquisition to `AsyncExclusiveLock`, `AsyncReaderWriterLock`, `AsyncManualResetEvent`, `AsyncAutoResetEvent` so the users can easily migrate step-by-step from monitors and other synchronization primitives to async-friendly primitives
-* Fixed random `InvalidOperationException` caused by `RandomAccessCache<TKey, TValue>`
-* Added synchronous methods to `RandomAccessCache<TKey, TValue>` to support [251](https://github.com/dotnet/dotNext/issues/251) feature request
+<a href="https://www.nuget.org/packages/dotnext/5.16.0">DotNext 5.16.0</a>
+* Added [LEB128](https://en.wikipedia.org/wiki/LEB128) encoder and decoder as a public API. See `DotNext.Buffers.Binary.Leb128<T>` type for more information
+* Added `SlideToEnd` method to `SpanWriter<T>` type
+* Added `IsBitSet` and `SetBit` generic methods to `Number` type
+* Added `DetachOrCopyBuffer` to `BufferWriterSlim<T>` type
+
+<a href="https://www.nuget.org/packages/dotnext.metaprogramming/5.16.0">DotNext.Metaprogramming 5.16.0</a>
+* Updated dependencies
+
+<a href="https://www.nuget.org/packages/dotnext.unsafe/5.16.0">DotNext.Unsafe 5.16.0</a>
+* Updated dependencies
+
+<a href="https://www.nuget.org/packages/dotnext.threading/5.16.0">DotNext.Threading 5.16.0</a>
+* Async locks with synchronous acquisition methods now throw [LockRecursionException](https://learn.microsoft.com/en-us/dotnet/api/system.threading.lockrecursionexception) if the current thread tries to acquire the lock synchronously and recursively.
+* Added support of cancellation token to synchronous acquisition methods of `AsyncExclusiveLock` and `AsyncReaderWriterLock` classes
+* Introduced `LinkTo` method overload that supports multiple cancellation tokens
+
+<a href="https://www.nuget.org/packages/dotnext.io/5.16.0">DotNext.IO 5.16.0</a>
+* Introduced `RandomAccessStream` class that represents [Stream](https://learn.microsoft.com/en-us/dotnet/api/system.io.stream) wrapper over the underlying data storage that supports random access pattern
+* Added extension method for `SpanWriter<byte>` that provides length-prefixed string encoding
+
+<a href="https://www.nuget.org/packages/dotnext.net.cluster/5.16.0">DotNext.Net.Cluster 5.16.0</a>
+* Updated dependencies
+
+<a href="https://www.nuget.org/packages/dotnext.aspnetcore.cluster/5.16.0">DotNext.AspNetCore.Cluster 5.16.0</a>
+* Updated dependencies
 
 Changelog for previous versions located [here](./CHANGELOG.md).
 
@@ -62,7 +84,7 @@ The libraries are versioned according to [Semantic Versioning 2.0](https://semve
 | 1.x | .NET Standard 2.0 | :x: |
 | 2.x | .NET Standard 2.1 | :x: |
 | 3.x | .NET Standard 2.1, .NET 5 | :x: |
-| 4.x | .NET 6 | :white_check_mark: |
+| 4.x | .NET 6 | :x: |
 | 5.x | .NET 8 | :heavy_check_mark: |
 
 :x: - unsupported, :white_check_mark: - bug and security fixes only, :heavy_check_mark: - active development

diff --git a/src/Directory.Packages.props b/src/Directory.Packages.props
@@ -19,22 +19,22 @@
   </ItemGroup>
   <ItemGroup>
     <!--Microsoft packages-->
-    <PackageVersion Include="Microsoft.AspNetCore.Connections.Abstractions" Version="8.0.10" />
+    <PackageVersion Include="Microsoft.AspNetCore.Connections.Abstractions" Version="8.0.11" />
     <PackageVersion Include="Microsoft.Extensions.Logging.Abstractions" Version="8.0.2" />
     <PackageVersion Include="Microsoft.Extensions.Hosting.Abstractions" Version="8.0.1" />
     <PackageVersion Include="Microsoft.Extensions.Hosting" Version="8.0.1" />
     <PackageVersion Include="Microsoft.Extensions.Http" Version="8.0.1" />
     <PackageVersion Include="Microsoft.Extensions.Options.ConfigurationExtensions" Version="8.0.0" />
     <PackageVersion Include="Microsoft.Extensions.Logging.Console" Version="8.0.1" />
     <PackageVersion Include="Microsoft.IO.RecyclableMemoryStream" Version="3.0.1" />
-    <PackageVersion Include="Microsoft.NET.Test.Sdk" Version="17.11.1" />
+    <PackageVersion Include="Microsoft.NET.Test.Sdk" Version="17.12.0" />
   </ItemGroup>
   <ItemGroup>
     <!--Misc packages-->
-    <PackageVersion Include="BenchmarkDotNet" Version="0.13.12" />
+    <PackageVersion Include="BenchmarkDotNet" Version="0.14.0" />
     <PackageVersion Include="FastMember.Signed" Version="1.5.0" />
-    <PackageVersion Include="xunit" Version="2.9.0" />
-    <PackageVersion Include="xunit.runner.visualstudio" Version="2.8.0"/>
+    <PackageVersion Include="xunit" Version="2.9.2" />
+    <PackageVersion Include="xunit.runner.visualstudio" Version="2.8.2"/>
     <PackageVersion Include="coverlet.collector" Version="6.0.2"/>
   </ItemGroup>
 </Project>
diff --git a/src/DotNext.Benchmarks/DotNext.Benchmarks.csproj b/src/DotNext.Benchmarks/DotNext.Benchmarks.csproj
@@ -8,7 +8,7 @@
     <RootNamespace>DotNext</RootNamespace>
     <StartupObject>DotNext.Program</StartupObject>
     <IsPackable>false</IsPackable>
-    <Version>5.11.0</Version>
+    <Version>5.16.0</Version>
     <Authors>.NET Foundation and Contributors</Authors>
     <Product>.NEXT Family of Libraries</Product>
     <Description>Various benchmarks demonstrating performance aspects of .NEXT extensions</Description>

diff --git a/src/DotNext.IO/Buffers/Binary/Leb128Reader.cs b/src/DotNext.IO/Buffers/Binary/Leb128Reader.cs
@@ -0,0 +1,20 @@
+using System.Numerics;
+using System.Runtime.CompilerServices;
+using System.Runtime.InteropServices;
+
+namespace DotNext.Buffers.Binary;
+
+[StructLayout(LayoutKind.Auto)]
+internal struct Leb128Reader<T>() : IBufferReader, ISupplier<T>
+    where T : struct, IBinaryInteger<T>
+{
+    private Leb128<T> decoder;
+    private bool incompleted = true;
+
+    readonly int IBufferReader.RemainingBytes => Unsafe.BitCast<bool, byte>(incompleted);
+
+    void IReadOnlySpanConsumer<byte>.Invoke(ReadOnlySpan<byte> source)
+        => incompleted = decoder.Append(MemoryMarshal.GetReference(source));
+
+    readonly T ISupplier<T>.Invoke() => decoder.Value;
+}
diff --git a/src/DotNext.IO/Buffers/BufferWriter.cs b/src/DotNext.IO/Buffers/BufferWriter.cs
@@ -7,6 +7,7 @@ namespace DotNext.Buffers;
 
 using EncodingContext = DotNext.Text.EncodingContext;
 using LengthFormat = IO.LengthFormat;
+using SevenBitEncodedInt = Binary.Leb128<uint>;
 
 /// <summary>
 /// Represents extension methods for writing typed data into buffer.
@@ -47,28 +48,18 @@ internal static unsafe int WriteLength(this ref SpanWriter<byte> destination, in
         {
             LengthFormat.LittleEndian => &ByteBuffer.WriteLittleEndian,
             LengthFormat.BigEndian => &ByteBuffer.WriteBigEndian,
-            LengthFormat.Compressed => &Write7BitEncodedInt,
+            LengthFormat.Compressed => &ByteBuffer.WriteLeb128<int>,
             _ => throw new ArgumentOutOfRangeException(nameof(lengthFormat)),
         };
 
         return writer(ref destination, value);
-
-        static int Write7BitEncodedInt(ref SpanWriter<byte> writer, int value)
-        {
-            foreach (var b in new SevenBitEncodedInt(value))
-            {
-                writer.Add() = b;
-            }
-
-            return writer.WrittenCount;
-        }
     }
 
     internal static int WriteLength(this IBufferWriter<byte> buffer, int length, LengthFormat lengthFormat)
     {
-        var writer = new SpanWriter<byte>(buffer.GetSpan(SevenBitEncodedInt.MaxSize));
-        buffer.Advance(writer.WriteLength(length, lengthFormat));
-        return writer.WrittenCount;
+        var bytesWritten = WriteLength(buffer.GetSpan(SevenBitEncodedInt.MaxSizeInBytes), length, lengthFormat);
+        buffer.Advance(bytesWritten);
+        return bytesWritten;
     }
 
     internal static int WriteLength(Span<byte> buffer, int length, LengthFormat lengthFormat)
@@ -77,26 +68,131 @@ internal static int WriteLength(Span<byte> buffer, int length, LengthFormat leng
         return writer.WriteLength(length, lengthFormat);
     }
 
+    private static int WriteLength(this ref BufferWriterSlim<byte> buffer, int length, LengthFormat lengthFormat)
+    {
+        var bytesWritten = WriteLength(buffer.GetSpan(SevenBitEncodedInt.MaxSizeInBytes), length, lengthFormat);
+        buffer.Advance(bytesWritten);
+        return bytesWritten;
+    }
+
     /// <summary>
     /// Encodes string using the specified encoding.
     /// </summary>
     /// <param name="writer">The buffer writer.</param>
-    /// <param name="value">The sequence of characters.</param>
+    /// <param name="chars">The sequence of characters.</param>
     /// <param name="context">The encoding context.</param>
     /// <param name="lengthFormat">String length encoding format; or <see langword="null"/> to prevent encoding of string length.</param>
     /// <returns>The number of written bytes.</returns>
-    public static long Encode(this IBufferWriter<byte> writer, ReadOnlySpan<char> value, in EncodingContext context, LengthFormat? lengthFormat = null)
+    public static long Encode(this IBufferWriter<byte> writer, ReadOnlySpan<char> chars, in EncodingContext context, LengthFormat? lengthFormat = null)
     {
-        long result = lengthFormat.HasValue
-            ? writer.WriteLength(context.Encoding.GetByteCount(value), lengthFormat.GetValueOrDefault())
+        var result = lengthFormat.HasValue
+            ? writer.WriteLength(context.Encoding.GetByteCount(chars), lengthFormat.GetValueOrDefault())
             : 0L;
 
-        context.GetEncoder().Convert(value, writer, true, out var bytesWritten, out _);
+        context.GetEncoder().Convert(chars, writer, true, out var bytesWritten, out _);
+        result += bytesWritten;
+
+        return result;
+    }
+
+    /// <summary>
+    /// Encodes string using the specified encoding.
+    /// </summary>
+    /// <param name="writer">The buffer writer.</param>
+    /// <param name="chars">The sequence of characters.</param>
+    /// <param name="context">The encoding context.</param>
+    /// <param name="lengthFormat">String length encoding format; or <see langword="null"/> to prevent encoding of string length.</param>
+    /// <returns>The number of written bytes.</returns>
+    public static int Encode(this ref SpanWriter<byte> writer, scoped ReadOnlySpan<char> chars, in EncodingContext context, LengthFormat? lengthFormat = null)
+    {
+        var result = lengthFormat.HasValue
+            ? writer.WriteLength(context.Encoding.GetByteCount(chars), lengthFormat.GetValueOrDefault())
+            : 0;
+
+        var bytesWritten = context.TryGetEncoder() is { } encoder
+            ? encoder.GetBytes(chars, writer.RemainingSpan, flush: true)
+            : context.Encoding.GetBytes(chars, writer.RemainingSpan);
         result += bytesWritten;
+        writer.Advance(bytesWritten);
 
         return result;
     }
 
+    /// <summary>
+    /// Writes a sequence of bytes prefixed with the length.
+    /// </summary>
+    /// <param name="writer">The buffer writer.</param>
+    /// <param name="value">A sequence of bytes to be written.</param>
+    /// <param name="lengthFormat">A format of the buffer length to be written.</param>
+    /// <returns>A number of bytes written.</returns>
+    public static int Write(this ref SpanWriter<byte> writer, scoped ReadOnlySpan<byte> value, LengthFormat lengthFormat)
+    {
+        var result = writer.WriteLength(value.Length, lengthFormat);
+        result += writer.Write(value);
+
+        return result;
+    }
+
+    /// <summary>
+    /// Encodes string using the specified encoding.
+    /// </summary>
+    /// <param name="writer">The buffer writer.</param>
+    /// <param name="chars">The sequence of characters.</param>
+    /// <param name="context">The encoding context.</param>
+    /// <param name="lengthFormat">String length encoding format; or <see langword="null"/> to prevent encoding of string length.</param>
+    /// <returns>The number of written bytes.</returns>
+    public static int Encode(this ref BufferWriterSlim<byte> writer, scoped ReadOnlySpan<char> chars, in EncodingContext context,
+        LengthFormat? lengthFormat = null)
+    {
+        Span<byte> buffer;
+        int byteCount, result;
+        if (lengthFormat.HasValue)
+        {
+            byteCount = context.Encoding.GetByteCount(chars);
+            result = writer.WriteLength(byteCount, lengthFormat.GetValueOrDefault());
+
+            buffer = writer.GetSpan(byteCount);
+            byteCount = context.TryGetEncoder() is { } encoder
+                ? encoder.GetBytes(chars, buffer, flush: true)
+                : context.Encoding.GetBytes(chars, buffer);
+
+            result += byteCount;
+            writer.Advance(byteCount);
+        }
+        else
+        {
+            result = 0;
+            var encoder = context.GetEncoder();
+            byteCount = context.Encoding.GetMaxByteCount(1);
+            for (int charsUsed, bytesWritten; !chars.IsEmpty; chars = chars.Slice(charsUsed), result += bytesWritten)
+            {
+                buffer = writer.GetSpan(byteCount);
+                var maxChars = buffer.Length / byteCount;
+
+                encoder.Convert(chars, buffer, chars.Length <= maxChars, out charsUsed, out bytesWritten, out _);
+                writer.Advance(bytesWritten);
+            }
+        }
+
+        return result;
+    }
+
+    /// <summary>
+    /// Writes a sequence of bytes prefixed with the length.
+    /// </summary>
+    /// <param name="writer">The buffer writer.</param>
+    /// <param name="value">A sequence of bytes to be written.</param>
+    /// <param name="lengthFormat">A format of the buffer length to be written.</param>
+    /// <returns>A number of bytes written.</returns>
+    public static int Write(this ref BufferWriterSlim<byte> writer, scoped ReadOnlySpan<byte> value, LengthFormat lengthFormat)
+    {
+        var result = writer.WriteLength(value.Length, lengthFormat);
+        writer.Write(value);
+        result += value.Length;
+
+        return result;
+    }
+
     private static bool TryFormat<T>(IBufferWriter<byte> writer, T value, Span<char> buffer, in EncodingContext context, LengthFormat? lengthFormat, ReadOnlySpan<char> format, IFormatProvider? provider, out long bytesWritten)
         where T : notnull, ISpanFormattable
     {
@@ -167,12 +263,12 @@ public static int Format<T>(this IBufferWriter<byte> writer, T value, LengthForm
         {
             null => 0,
             LengthFormat.BigEndian or LengthFormat.LittleEndian => sizeof(int),
-            LengthFormat.Compressed => SevenBitEncodedInt.MaxSize,
+            LengthFormat.Compressed => SevenBitEncodedInt.MaxSizeInBytes,
             _ => throw new ArgumentOutOfRangeException(nameof(lengthFormat)),
         };
 
         int bytesWritten;
-        for (int bufferSize = 0, actualLengthSize; ; bufferSize = bufferSize <= MaxBufferSize ? bufferSize << 1 : throw new InsufficientMemoryException())
+        for (int bufferSize = 0; ; bufferSize = bufferSize <= MaxBufferSize ? bufferSize << 1 : throw new InsufficientMemoryException())
         {
             var buffer = writer.GetSpan(bufferSize);
 
@@ -182,7 +278,7 @@ public static int Format<T>(this IBufferWriter<byte> writer, T value, LengthForm
                 continue;
             }
 
-            actualLengthSize = lengthFormat.HasValue
+            var actualLengthSize = lengthFormat.HasValue
                 ? WriteLength(buffer.Slice(0, expectedLengthSize), bytesWritten, lengthFormat.GetValueOrDefault())
                 : 0;
 

diff --git a/src/DotNext.IO/Buffers/IBufferReader.cs b/src/DotNext.IO/Buffers/IBufferReader.cs
@@ -252,7 +252,6 @@ TResult ISupplier<TResult>.Invoke()
 }
 
 [StructLayout(LayoutKind.Auto)]
-
 internal struct SkippingReader(long length) : IBufferReader
 {
     readonly int IBufferReader.RemainingBytes => int.CreateSaturating(length);
@@ -261,7 +260,7 @@ void IReadOnlySpanConsumer<byte>.Invoke(ReadOnlySpan<byte> source)
         => length -= source.Length;
 }
 
-internal struct BufferReader<TReader>(TReader reader) : IBufferReader, ISupplier<TReader>
+internal struct ProxyReader<TReader>(TReader reader) : IBufferReader, ISupplier<TReader>
     where TReader : struct, IBufferReader
 {
     int IBufferReader.RemainingBytes => reader.RemainingBytes;
@@ -273,5 +272,5 @@ void IReadOnlySpanConsumer<byte>.Invoke(ReadOnlySpan<byte> source)
 
     readonly TReader ISupplier<TReader>.Invoke() => reader;
 
-    public static implicit operator BufferReader<TReader>(TReader reader) => new(reader);
+    public static implicit operator ProxyReader<TReader>(TReader reader) => new(reader);
 }
diff --git a/src/DotNext.IO/Buffers/ReadOnlySpanFunc.cs b/src/DotNext.IO/Buffers/ReadOnlySpanFunc.cs
@@ -10,4 +10,4 @@ namespace DotNext.Buffers;
 /// <param name="span">A read-only span of objects.</param>
 /// <param name="arg">A state object.</param>
 /// <returns>The value returned by the delegate.</returns>
-public delegate TResult ReadOnlySpanFunc<T, TArg, TResult>(ReadOnlySpan<T> span, TArg arg);
+public delegate TResult ReadOnlySpanFunc<T, in TArg, out TResult>(ReadOnlySpan<T> span, TArg arg);