readme.md

loghisto

A metric system for high performance counters and histograms. Unlike popular metric systems today, this does not destroy the accuracy of histograms by sampling. Instead, a logarithmic bucketing function compresses values, generally within 1% of their true value (although between 0 and 1 the precision loss may not be within this boundary). This allows for extreme compression, which allows us to calculate arbitrarily high percentiles with no loss of accuracy - just a small amount of precision. This is particularly useful for highly-clustered events that are tolerant of a small precision loss, but for which you REALLY care about what the tail looks like, such as measuring latency across a distributed system.

Copied out of my work for the CockroachDB metrics system. Based on an algorithm created by Keith Frost.

adding an embedded metric system to your code

import (
"time""fmt""github.com/spacejam/loghisto"
)
funcExampleMetricSystem() {
// Create metric system that reports once a minute, and includes stats// about goroutines, memory usage and GC.includeGoProcessStats:=truems:= loghisto.NewMetricSystem(time.Minute, includeGoProcessStats)
ms.Start()
// create a channel that subscribes to metrics as they are produced once// per minute.// NOTE: if you allow this channel to fill up, the metric system will NOT// block, and will FORGET about your channel if you fail to unblock the// channel after 3 configured intervals (in this case 3 minutes) rather// than causing a memory leak.myMetricStream:=make(chan *loghisto.ProcessedMetricSet, 2)
ms.SubscribeToProcessedMetrics(myMetricStream)
// create some metricstimeToken:= ms.StartTimer("time for creating a counter and histo")
ms.Counter("some event", 1)
ms.Histogram("some measured thing", 123)
timeToken.Stop()
form:=range myMetricStream {
fmt.Printf("number of goroutines: %f\n", m.Metrics["sys.NumGoroutine"])
}
// if you want to manually unsubscribe from the metric stream
ms.UnsubscribeFromProcessedMetrics(myMetricStream)
// to stop and clean up your metric system
ms.Stop()
}