Skip to content

Export OS stats as Sensision Metrics

License

Notifications You must be signed in to change notification settings

clever-telemetry/noderig

 
 

Repository files navigation

Noderig - Export OS stats as Sensision or Prometheus Metrics

Build Status

Noderig collect OS metrics and expose them through a Sensision HTTP endpoint. Each collector is easily configurable, thanks to a simple level cursor

Noderig metrics:

  • CPU
  • Memory
  • Load
  • Disk
  • Net
  • External collectors

Status

Noderig is currently under development. Feel free to comment or contribute!

Building

Noderig is pretty easy to build.

  • Clone the repository
  • Install glide, follow instructions here https://glide.sh/
  • Download dependencies glide install
  • Build and run go run noderig.go

Usage

noderig [flags]

Flags:
      --config string       config file to use
  -l  --listen string       listen address (default "127.0.0.1:9100")
  -v  --verbose             verbose output
      --period uint         default collection period (default 1000)
      --cpu uint8           cpu metrics level (default 1)
      --disk uint8          disk metrics level (default 1)
      --mem uint8           memory metrics level (default 1)
      --net uint8           network metrics level (default 1)
      --load uint8          load metrics level (default 1)
  -c  --collectors string   external collectors directory (default "./collectors")
  -k  --keep-for uint       keep collectors data for the given number of fetch (default 3)
      --net-opts.interfaces give a filtering list of network interfaces to collect metrics on
      --disk-opts.names     give a filtering list of disks names to collect metrics on

Collectors

Noderig have some built-in collectors.

CPU

LevelMetricDescriptionModule
0disabled metrics
1os.cpu{}combined percentage of cpu usage
2os.cpu.iowait{}combined percentage of cpu iowait
os.cpu.user{}combined percentage of cpu user
os.cpu.systems{}combined percentage of cpu systems
os.cpu.nice{}combined percentage of cpu nice
os.cpu.irq{}combined percentage of cpu irq
os.cpu.steal{}combined percentage of cpu stolen
os.cpu.idlel{}combined percentage of cpu idle
os.cpu.temperature{id=n}temperature of cpu ntemperature
3os.cpu.iowait{chore=n}chore percentage of cpu iowait
os.cpu.user{chore=n}chore percentage of cpu user
os.cpu.systems{chore=n}chore percentage of cpu systems
os.cpu.nice{chore=n}chore percentage of cpu nice
os.cpu.irq{chore=n}chore percentage of cpu irq
os.cpu.steal{chore=n}chore percentage of cpu stolen
os.cpu.idle{chore=n}chore percentage of cpu idle
os.cpu.temperature{core=n}temperature of cpu core ntemperature

Memory

0disabled metrics
1os.mem{}percentage of memory used
os.swap{}percentage of swap used
2os.mem.used{}used memory (bytes)
os.mem.total{}total memory (bytes)
os.swap.used{}used swap (bytes)
os.swap.total{}total swap (bytes)
3os.mem.free{}free memory (bytes)
os.mem.buffers{}buffers memory (bytes)
os.mem.cached{}cached memory (bytes)

Load

0disabled metrics
1os.load1{}load 1
2os.load5{}load 5
os.load15{}load 15

Disk

0disabled metrics
1os.disk.fs{disk=/dev/sda1}disk used percent
2os.disk.fs.used{disk=/dev/sda1, mount=/}disk used capacity (bytes)
os.disk.fs.total{disk=/dev/sda1,mount=/}disk total capacity (bytes)
os.disk.fs.inodes.used{disk=/dev/sda1,mount=/}disk used inodes
os.disk.fs.inodes.total{disk=/dev/sda1,mount=/}disk total inodes
3os.disk.fs.bytes.read{name=sda1}disk read count (bytes)
os.disk.fs.bytes.write{name=sda1}disk write count (bytes)
4os.disk.fs.io.read{name=sda1}disk io read count (bytes)
os.disk.fs.io.write{disk=/sda1}disk io write count (bytes)
5os.disk.fs.io.read.ms{name=sda1}disk io read time (ms)
os.disk.fs.io.write.ms{name=sda1}disk io write time (ms)
os.disk.fs.io{name=sda1}disk io in progress (count)
os.disk.fs.io.ms{name=sda1}disk io time (ms)
os.disk.fs.io.weighted.ms{name=sda1}disk io weighted time (ms)

Net

0disabled metrics
1os.net.bytes{direction=in}in bytes count (bytes)
os.net.bytes{direction=out}out bytes count (bytes)
2os.net.bytes{direction=in,iface=eth0}iface in bytes count (bytes)
os.net.bytes{direction=out,iface=eth0}iface out bytes count (bytes)
3os.net.packets{direction=in,iface=eth0}iface in packet count (packets)
os.net.packets{direction=out,iface=eth0}iface out packet count (packets)
os.net.errs{direction=in,iface=eth0}iface in error count (errors)
os.net.errs{direction=out,iface=eth0}iface out error count (errors)
os.net.dropped{direction=in,iface=eth0}iface in drop count (drops)
os.net.dropped{direction=out,iface=eth0}iface out drop count (drops)

Custom

With Noderig you can define set-up custom collectors as defined in http://bosun.org/scollector/external-collectors. To be enable you need to define a collectors folder using the noderig parameter "collectors". This fold need to have a strict arborescence: a number folder and then the exectutable collectors.

For example to define a script shell collectors reach the noderig collectors file:

cd ~/collectors
mkdir 10

Then inside the 10 folder write the following executable test.sh shell script.

#!/bin/sh

now=$(date +%s)

echo my.metric $now 42

And execute noderig:

./build/noderig --collectors ~/collectors

To conclude you can tun noderig custom collectors with the following configuration parameters:

keep-metrics: true # To always keep in Noderig the last metrics values
keep-for: 3 # Keep-for returned the number values to keep

The keep-for parameter with keep-metrics at true keep the last N values otherwise it keep each values for n calls to the noderig metrics endpoint.

Configuration

Noderig can read a simple default config file.

Configuration is load and override in the following order:

  • /etc/noderig/config.yaml
  • ~/noderig/config.yaml
  • ./config.yaml
  • config filepath from command line

Definitions

Config is composed of three main parts and some config fields:

Collectors

Noderig have some built-in collectors. They could be configured by a log level. You can also defined custom collectors, in an scollector way. (see: http://bosun.org/scollector/external-collectors) To configure a custom collectors in noderig reach custom collectors.

cpu: 1  # CPU collector level     (Optional, default: 1)
mem: 1  # Memory collector level  (Optional, default: 1)
load: 1 # Load collector level    (Optional, default: 1)
disk: 1 # Disk collector level    (Optional, default: 1)
net: 1  # Network collector level (Optional, default: 1)

Collectors Modules

Some collectors have additionals modules. Add module to <collector>-mods list to enable them.

cpu-mods:
  - temperature

Collectors Options

Some collectors can accept optional parameters.

net-opts:
  interfaces:            # Give a filtering list of interfaces for which you want metrics
    - eth0
    - eth1

Net-opts, interfaces field support now regular expression to white-list interface based on golang MatchString implementation. However to use a regular expression you need to prefix the string value by a ~. To whitelist all eth interfaces, you can set:

net-opts:
  interfaces:            # Give a filtering list of interfaces for which you want metrics
    - ~eth*
disk-opts:
  names:            # Give a filtering list of disks for which you want metrics
    - sda1
    - sda3

Disk-opts, names field support now regular expression to white-list disks names based on golang MatchString implementation. However to use a regular expression you need to prefix the string value by a ~. To whitelist all disk names, you can set:

disk-opts:
  names:            # Give a filtering list of disks names for which you want metrics
    - ~disk*

Parameters

Noderig can be customized through some parameters.

period: 1000             # Duration within all the sources should be scraped in ms (Optional, default: 1000)
listen: none             # Listen address, set to none to disable http endpoint    (Optional, default: 127.0.0.1:9100)
collectors: /opt/noderig # Custom collectors directory                             (Optional, default: none)

To force default labels to each metrics in Noderig, you can set up a configuration key called labels. It expects a label string map as defined below:

labels: { 
  host: "srv001", 
  dc: "uk1", 
  type: "web_server", 
}

Sample metrics

1484828198557102// os.cpu{} 2.5202020226869237
1484828198560976// os.mem{} 24.328345730457112
1484828198560976// os.swap{} 0
1484828198557435// os.load1{} 0.63
1484828198561366// os.net.bytes{direction=in} 858
1484828198561366// os.net.bytes{direction=out} 778
1484828197570759// os.disk.fs{disk=/dev/sda1} 4.967614357908193

Prometheus output format

To use Noderig and expose a Prometheus native format, just set the following two configuration lines in the config file:

format: "prometheus"  # Expose a Prometheus format in Noderig as: https://prometheus.io/docs/instrumenting/exposition_formats/
separator: "_"        # Metrics classnames separator, '_' is the default one for Prom, but you can use any other supported by your storage backend

Contributing

Instructions on how to contribute to Noderig are available on the Contributing page.

Get in touch

About

Export OS stats as Sensision Metrics

Resources

License

Stars

Watchers

Forks

Packages

No packages published

Languages

  • Go 93.8%
  • Makefile 4.5%
  • Dockerfile 1.7%