RunningStatistics 1.0.2

RunningStatistics

Online (single pass) algorithms for statistical measures based on the Julia package OnlineStats.jl by Josh Day. Useful for streaming and big data.

List of Statistics

List of Distributions

Common Interface

All running statistics inherit from the following abstract class:

public abstract class AbstractRunningStatistic<TObs, TSelf> where TSelf : AbstractRunningStatistic<TObs, TSelf>
{
    private long _nobs;

    public long Nobs
    {
        get => GetNobs();
        protected set => _nobs = value;
    }

    protected virtual long GetNobs() => _nobs;

    public abstract void Fit(TObs value);

    public virtual void Fit(IEnumerable<TObs> values)
    {
        foreach (var value in values)
        {
            Fit(value);
        }
    }

    public abstract void Reset();

    public abstract TSelf CloneEmpty();

    public abstract TSelf Clone();

    public abstract void Merge(TSelf other);

    public static TSelf Merge(TSelf sourceStatistic, params TSelf[] stats)
    {
        var newStat = sourceStatistic.Clone();
        
        foreach (var stat in stats)
        {
            newStat.Merge(stat);
        }

        return newStat;
    }
}

Examples

using RunningStatistics;

var mean1 = new Mean();
var mean2 = new Mean();
var ecdf = new EmpiricalCdf();

var rng = new Random();
for (var i = 0; i < 1000; i++)
{
    var x = rng.NextDouble();
    
    mean1.Fit(x);
    mean2.Fit(2*x);
    
    ecdf.Fit(x);
}

mean1.Merge(mean2);
var q1 = ecdf.Quantile(0.25);

Combining Statistics

Different statistics are intended to be used on their own. If a collection of statistics is appropriate, then we recommend that you write your own class that inherits from AbstractRunningStatistic<TObs, YourClass> and use the desired statistics as private members. Example:

public class MyClass : AbstractRunningStatistic<double, MyClass>
{
    private EmpiricalCdf Ecdf { get; init; } = new();
    
    private CountMap<double> CountMap { get; init; } = new();


    protected override long GetNobs() => _ecdf.Nobs; 

    public override void Fit(double value)
    {
        Ecdf.Fit(value);
        CountMap.Fit(value);
    }

    public override void Reset()
    {
        Ecdf.Reset();
        CountMap.Reset();
    }

    public override MyClass CloneEmpty() => return new();

    public override MyClass Clone()
    {
        return new MyClass
        {
            Ecdf = Ecdf.Clone(),
            CountMap = CountMap.Clone()
        };
    }

    public override void Merge(MyClass other)
    {
        Ecdf.Merge(other.Ecdf);
        CountMap.Merge(other.CountMap);
    }
}

Notice that in this class, the number of observations is retrieved from _ecdf.Nobs by overriding GetNobs, and therefore Nobs never needs to be updated in the Fit method.