Skip to content

A simple and fast tool to parse the AOF file of redis / 一个简单快速的解析Redis AOF文件的工具

License

Notifications You must be signed in to change notification settings

WGrape/parseAOF

Folders and files

NameName
Last commit message
Last commit date

Latest commit

 

History

27 Commits
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 

Repository files navigation

img

A simple and fast tool to parse the AOF file of redis

Document :中文 / English

Content

1、Introduction

A simple and fast tool to parse the AOF file of redis

(1) Features

  • Code is clean, simple and easy to customize
  • Speed up parsing through multiple goroutines
  • A list of commands will be generated after parsing for log querying

(2) Architecture

2、Build

git clone https://github.com/WGrape/parseAOF
cd parseAOF
go mod download
make build

3、Usage

Run the binary under bin dir parseAOF_<os>_<arch> with the path of the aof file

./bin/parseAOF_macos_arm64 -i ~/Download/appendonly.aof -r 8
./bin/parseAOF_macos_arm64 -h
parse redis aof to readable

Usage:
  parseAOF [flags]

Flags:
  -h, --help            help for parseAOF
  -i, --input string    input AOF file path
  -o, --output string   output dir path
  -r, --routines int    max goroutines (default 8)

(1) The input file

Here's an example input file ./data/appendonly.aof for you to test

Before running, pass the path of the aof file to the start.sh script, the content is as follows

*2
$6
SELECT
$1
0
... ...

(2) The output file

Here's an example output file ./data/aof.merged for you to test

After the parsing is complete, the file aof.merged will be generated in the directory of data, the content is as follows

--------------------parseAOF | version=0.5.0--------------------
SELECT 0 
set key1 1 
set key2 2 
set key3 3 
sadd key4 1 2 3 4 
lpush key5 1 2 3 4 5 
zadd key6 1 2 3 4 5 6 

(3) Example

example

4、Performance

  • The average speed to parse is 50000 lines/s
  • The maximum size of the aof is 1GB

(1) Testing

Id Lines Size Cost CPU
1 1,2301,117 39MB 3m50s <=65%
2 3,435,263 13MB 1m12s <=65%
3 357,850 8.6MB 3.47s <=113%

About

A simple and fast tool to parse the AOF file of redis / 一个简单快速的解析Redis AOF文件的工具

Topics

Resources

License

Stars

Watchers

Forks

Releases

No releases published

Packages

No packages published