Anixe.IO 3.0.0

Anixe.IO

Package includes

• Base64ReverseStream
• BatchesEnumerator
• CustomMemoryPool
• DeflateReverseStream
• DelegatingTextWriter
• DelimitedEachSpanEnumerator
• DelimitedSpanEnumerator
• EnchancedTextWriter
• EscapedXmlStreamReader
• FastStreamReader
• JsonTextWriter
• ListSegment
• Log
• LoggingHandler
• MemoryOwnedStream
• PooledStream
• RefineReverseStream
• Stream2Stream
• Stream2TextWriter
• StreamCopy
• StreamReader
• StreamWriter
• TextFormatter
• TextWriter2StreamDelegate
• UnescapeXmlStreamReader
• UrlEncodededWriter
• UrlWriter
• ValueStringBuilder
• ValueUrlWriter

Base64ReverseStream

The reverse stream is a kind of stream which allows to copy from source to destination with passing source stream in constructor and destination steam while copy. Regular forward streams requires destination stream in constructor, which is not always possible to archive.

the Base64ReverseStream allows you to encode source stream into base64 while copying data into destination stream

  var source = new MemoryStream(Encoding.UTF8.GetBytes("test sample"));
  var target = new MemoryStream();
  using (var subject = new Base64ReverseStream(source, 2048, leaveOpen: true))
  {
    subject.CopyTo(target);
  }

BatchesEnumerator

The struct allows to split IList<T> into batches without allocations using ListSegment class. Check BatchesEnumeratorTest for details

CustomMemoryPool

The class implements an MemoryPool<T> abstraction. The class holds only ArrayPool instance inside and does not require disposing. Creating multiple instances of this class will spawn multiple ArrayPool instances so use it as a singleton in the application. The class has been added here because of limitation inside MemoryPool<T> default implementation which causes memory leaks for > 1 Mb arrays. For explanation refer to: https://adamsitnik.com/Array-Pool/

var oneMb  = 1024 * 1024;
var pool = new CustomMemoryPool<byte>(50 * oneMb);
using (var memoryOwner = pool.Rent(30 * oneMb))
{
  DoSomething(memoryOwner);
}

var oneMb  = 1024 * 1024;
var pool = new CustomMemoryPool<byte>(50 * oneMb);
using (var stream = new MemoryOwnedStream(pool)) // pass custom pool to the stream for renting
{
  SerializeObject(stream);
  return (stream.GetBuffer(), stream.Length); // pass the ownership of IMemoryOwner<byte> to next executor and length to have information about bytes written into the memory buffer.
}

DeflateReverseStream

The reverse stream is a kind of stream which allows to copy from source to destination with passing source stream in constructor and destination steam while copy. Regular forward streams requires destination stream in constructor, which is not always possible to archive.

the DeflateReverseStream allows you to compress source stream into gzip format while copying data into destination stream

  var source = new MemoryStream(Encoding.UTF8.GetBytes("test sample"));
  var target = new MemoryStream();
  using (var subject = new DeflateReverseStream(source, 2048, leaveOpen: true))
  {
    subject.CopyTo(target);
  }

DelegatingTextWriter

The class delegates TextWriter signals to provided inner TextWriters, it is useful to feed multiple TextWriters at the same time.

Examples

  var tw1 = new StreamWriter(someStream);
  var tw2 = Console.Out;
  using (var subject = new DelegatingTextWriter(tw1, tw2))
  {
    subject.WriteLine("some text");
  }

DelimitedEachSpanEnumerator

The struct allows to enumerate through tokens delimited by single char from ReadOnlySpan<char> It has the same output as string.Split with StringSplitOptions.None

Examples

  var iterator = new DelimitedEachSpanEnumerator(text.AsSpan(), delimiter);
  while (iterator.MoveNext())
  {
    ReadOnlySpan<char> segment = iterator.Current;
    // do something
  }

DelimitedSpanEnumerator

The struct allows to enumerate through tokens delimited by single char from ReadOnlySpan<char> It has the same output as string.Split with StringSplitOptions.RemoveEmptyEntries Use case:

  var iterator = new DelimitedSpanEnumerator(text.AsSpan(), delimiter);
  while (iterator.MoveNext())
  {
    ReadOnlySpan<char> segment = iterator.Current;
    // do something
  }

EnchancedTextWriter

The class implements TextWriter + some Stream class methods:

• TotalBytesWritten - returns total bytes written to target stream. It is useful to track if target stream receives data even for forward-only streams (e.g. NetworkStream)
• WriteBase64(byte[] data, int offset, int count) - writes data as base64 encoded string into underlying stream
• CanRead - calls the same method from underlying stream
• CanSeek - calls the same method from underlying stream
• CanWrite - calls the same method from underlying stream
• Length - calls the same method from underlying stream
• Position - calls the same method from underlying stream
• Seek(long offset, SeekOrigin origin) - calls the same method from underlying stream
• SetLength(long value) - calls the same method from underlying stream

EscapedXmlStreamReader

Reads XML-entities encoded stream

Example usage

  using (var ms = new MemoryStream(Encoding.UTF8.GetBytes(escapedXml)))
  {
    var reader = XmlReader.Create(new EscapedXmlStreamReader(ms));
    return ReadResponse(reader);
  }

FastStreamReader

The class reads content from file line by line exposing it as ArraySegment<char>. It has minimal memory footprint Use case:

using(var reader = new FastStreamReader(File.OpenRead(path)
{
  while(reader.Read())
  {
    ArraySegment<char> line = reader.CurrentRawLine;
    // do something
  }
}

JsonTextWriter

The class derives from Newtonsoft.Json.JsonTextWriter providing additions WriteValue routines (check JsonTextWriterTest for details)

ListSegment

The class (similar to ArraySegment<T>) provides a allocation free view of the IList<T>.

Log

Added in order to ease integration with Resfinity Logging Platform

LoggingHandler

Added allocationless http payloads logger which passes pooled array filled with data to the logging system. Use LoggingHandler class as http client delegating handler.

MemoryOwnedStream

This is a memory stream wrapping ReadOnlyMemory<byte> (read-only mode) or wrapping IMemoryOwner<byte> taken from MemoryPool<byte> (read or write mode). It allows you to encapsulate Memory<byte> inside managed stream. Be aware that default constructor requires to dispose IMemoryOwner<byte> manually.

using (var stream = new MemoryOwnedStream())
{
  SerializeObject(stream);
  return (stream.GetBuffer(), stream.Length); // pass the ownership of IMemoryOwner<byte> to next executor and length to have information about bytes written into the memory buffer.
}

ReadOnlyMemory<byte> mem = ReadSomething();
using (var stream = new MemoryOwnedStream(mem)) // only for reading
{
  var obj = DeserializeObject(stream);
  return obj;
}

PooledStream

This memory stream with internal byte buffer pooled from ArrayPool<byte>. Use it as regular stream. Useful hints:

• The stream resizes its internal array by rent an array with bigger size if the bytes written exceeds the actual buffer size.
• The stream must be disposed to avoid memory leaks.
• The stream exposes it's internal buffer similar to MemoryStream with GetBuffer() method.
• SetLength method is important to limit the content of the stream it it's real length. Internal buffer is always bigger than written content.

  var str = "test string";
  var stream = new PooledStream(str.Length);
  var bytesConverted = Encoding.UTF8.GetBytes(str, 0, str.Length, stream.GetBuffer(), 0);
  stream.SetLength(bytesConverted);

RefineReverseStream

the RefineReverseStream allows you to remove certain bytes from source stream while copying data into destination stream. It is useful to remove new line character, but it can be any byte.

  var source = new MemoryStream(Encoding.UTF8.GetBytes("test sample\ntest sample2"));
  var target = new MemoryStream();
  using (var subject = new RefineReverseStream(source, byteToRefine: (byte) '\n', leaveOpen: true))
  {
    subject.CopyTo(target);
  }

Stream2Stream

The class delegates source Stream reads and writes to destination Stream writes

Stream2TextWriter

The class writes input bytes into both: Stream and TextWriter.

Example usage

var subject = new Stream2TextWriter(outputStream, outputTextWriter, writeBuffer: new char[1024]);
subject.Write(inputByteArray, offset: 0, count: inputByteArray.Length);

StreamCopy

Example usage

using(var tw = new StreamWriter(outputStream, Encoding.UTF8))
{
  var buffer = new char[buffSize];
  StreamCopy.Copy(inputStream, tw, 0, 50, buffer);
}

StreamReader

The class uses array pool for internal buffers allocations (.NET Core only)

StreamWriter

The class uses array pool for internal buffers allocations (.NET Core only)

TextFormatter

Stream implementation that allows to write input as text in provided encoding and exposes counter of written bytes.

Example usage

using(var tf = new TextFormatter(outputStream, Encoding.UTF8, 1024))
{
    tf.Write(123);
    tf.Write("text");
    tf.WriteBase64(bytes, 0, bytes.Length);
}

TextWriter2StreamDelegate

The class delegates source Stream reads and writes to destination Stream writes

UnescapeXmlStreamReader

UrlEncodededWriter

UrlWriter

Example usage

  var urlWriter = new UrlWriter();
  urlWriter.WriteBaseUrl("http://site.com");
  urlWriter.WriteUrlSegment("page");
  urlWriter.WriteUrlSegment("subpage");
  urlWriter.WriteStartArguments();
  urlWriter.WriteArgument("id", "1234");
  urlWriter.WriteArgument("device", "mobile");
  var url = UrlWriter.toString();
  // produces: http://site.com/page/subpage?id=1234&device=mobile

ValueStringBuilder

Anixe implementation of internal .NET library type for building strings without heap allocations. It uses a ReadOnlySpan<char> provided in constructor as an internal buffer, or an array from ArrayPool.Shared if the buffer was not provided or was too small.

Use case:

var sb = new ValueStringBuilder(stackalloc char[50]);
var names = new[] { "first", "second" };
for (int i = 0; i < names.Length; i++)
{
    if (i > 0)
    {
        sb.Append(',');
    }
    sb.Append(i.ToString());
    sb.Append(": ");
    sb.Append(name);
}
return sb.ToString();

ValueUrlWriter

Allocationless implementation of UrlWriter. It is implemented as ref struct and operates on Span<char> buffer. Rents an array from ArrayPool.Shared if provided buffer was too small to write.

Example usage

  var urlWriter = new ValueUrlWriter(stackallock char[256]);
  urlWriter.WriteBaseUrl("http://site.com");
  urlWriter.WriteUrlSegment("page");
  urlWriter.WriteUrlSegment("subpage");
  urlWriter.WriteStartArguments();
  urlWriter.WriteArgument("id", "1234");
  urlWriter.WriteArgument("device", "mobile");
  var url = UrlWriter.toString();
  // produces: http://site.com/page/subpage?id=1234&device=mobile

Remarks

ListSegment<T> is a wrapper around an IList<T> that delimits a range of elements in that list. Multiple ListSegment<T> instances can refer to the same original list and can overlap. The original list must have zero-based indexing and cannot change size in the meantime.

Each message requires logging level to be assigned. For majority of Anixe apps it's easy mapping

• transactions with status OK → MessageLevel.INFORMATIONAL
• transactions with status ERR → MessageLevel.ERROR

If you don't need any customization register:

• MessageAsSyslogWriter as IMessageWriter for syslog messages
• MessageAsGelfWriter as IMessageWriter for GELF messages]
  • MessageAsGelfWriter writer has dependency to IArrayPool<char> - it must be passed directly to the constructor or registered via dependency injection framework

If you don't need any custom message object use Anixe.IO.Log.Message

• if you need to store data in Graylog and model does not define key for it you can use
  • keyDoublePairs, keyLongPairs, keyIntPairs for numbers
  • keyStringPairs for strings

If customization is required

• define custom writer that inherits from MessageAsSyslogWriter and override:

  • WriteCustom(TextWriter writer, IMessage message) method for SeriLog
  • AppendCustom(StringBuilder builder, IMessage message) method for NLog

• define custom writer that inherits from MessageAsGelfWriter and override:

  • WriteCustom(Anixe.IO.JsonTextWriter, IMessage message) method for both SeriLog/NLog

• define custom IMessage model that either inherits from Message or implements IMessage directly

public class CustomMessage : Message
{
    public string foo;
    public string bar;
}

public class CustomWriter : MessageAsSyslogTextWriterWriter
{
    protected override void WriteCustom(TextWriter writer, IMessage message)
    {
        var m = message as CustomMessage;

        WriteProperty(writer, "foo", m.foo);
        WriteProperty(writer, "bar", m.bar);
    }
}