Super simple Golang DB to be used when iterating over []float64 slices of fixed length, when they do not fit in RAM anymore.
I use this for machine learning, when some times you have a lot of float slices you want to iterate over.
go get github.com/jurschreuder/ffsdb
create at path test.db
save []float64 slices of length 256
overwrite old database
fdb, err := NewFfsdb("test.db", 256, true, false) // (filepath, []float64 length, overwrite old file, save as float32 to save space)
if err != nil {
panic(err)
}
defer fdb.Close()where foo is a []float64
foo := make([]float64, 256)
err := fdb.Add(foo)where foo is a []float64
fdb.Rewind()
ok := true
for ok {
foo, ok = fdb.ReadNext()
}id = int64(100)
vals, err := fdb.ReadId(id)id = int64(100)
foo = make([]float64, 256)
err := fdb.Update(id, foo)for []float64 with length 256
on 2018 mac book
ffsdb_test.go:27: added 100000 in time: 1.62976014s
ffsdb_test.go:41: read 100000 in time: 580.782646ms