AmILeaked is a simple leak/email data registry service with open REST API and gRPC service. This is given as a coding challenge by Binaryedge.
The project consists of two main parts, these are the RPC server and the REST gateway. The gateway is a HTTP reverse proxy which translates the binary gRPC messages to JSON messages.
The project is implemented using gRPC framework with Golang. You need to install some packages first to compile the .proto files. Then you can compile the server microservices.
First get the required packages to use gRPC environment to compile the RPC server.
go get google.golang.org/grpc
go get google.golang.org/grpc/reflection
go get github.com/grpc-ecosystem/go-grpc-middleware
go get github.com/grpc-ecosystem/go-grpc-middleware/recoveryNow get the packages required to compile the REST gateway.
go install \
github.com/grpc-ecosystem/grpc-gateway/protoc-gen-grpc-gateway \
github.com/grpc-ecosystem/grpc-gateway/protoc-gen-swagger \
github.com/golang/protobuf/protoc-gen-goThe database connection URI and database name are both hardcoded into the rpc server code (I know I shoud use environment variables or flags for this but I didn't :/ )
So before compiling the src/rpc-server codes, you first need to go to src/rpc-server/main.go and change the values on fields DBConnURI and DatabaseName of the struct LeakServiceServerHandler on the lines 39 and 40.
Let's start with compiling the .proto files and generating the corresponding server/client codes.
Note
The protocompile.sh file is written to work in *Nix environments. In Win environments, the -I/usr/local/include flag should be changed to a proper one.
./protocompile.shThis command generates Golang codes for gRPC server/client and serializer/deserializer codes in rpc-server/ and rest-server/ directories.
Now go to rpc-server/ directory and compile the RPC server.
cd rpc-server/
go buildThis command compiles the RPC server into the file rpc-server.
Now go to rest-server/ directory and compile the REST gateway.
cd ../rest-server/
go buildThis command compiles the RPC server into the file rest-server.
Now both the RPC service and API service are ready to be used.
To use the system, you just need to run the rpc-server binary in the background, and then the rest-server after it.
./rpc-server &
./rest-serverNote
The RPC server is hardcoded to run on port 50051 on localhost.
The REST gateway is hardcoded to run on port 8081 on localhost.
- Endpoint:
/v1-beta/leaks - Method:
GET - Example response:
{
"result": {
"id": "5efdb25c0479ba527b1788f6",
"name": "fancy leak",
"emails": [
{
"email": "[email protected]",
"domain": "ieee.metu.edu.tr",
"first_occurance_ts": "1593684730000",
"last_occurance_ts": "1593684730000"
},
{
"email": "-----%%%@metu.edu.tr",
"domain": "metu.edu.tr",
"first_occurance_ts": "1593684730000",
"last_occurance_ts": "1593684730000"
}
],
"email_count": "2"
}
}
{
"result": {
"id": "5efdb2660479ba527b1788f7",
"name": "moore fancy leak",
"emails": [
{
"email": "[email protected]",
"domain": "ieee.metu.edu.tr",
"first_occurance_ts": "1593684730000",
"last_occurance_ts": "1593684730000"
}
],
"email_count": "1"
}
}
{
"result": {
"id": "5efdb27c0479ba527b1788f8",
"name": "the big turkey data leak",
"emails": [
{
"email": "[email protected]",
"domain": "ieee.metu.edu.tr",
"first_occurance_ts": "1593684730000",
"last_occurance_ts": "1593684730000"
},
{
"email": "[email protected]",
"domain": "ieee.metu.edu.tr",
"first_occurance_ts": "1593684730000",
"last_occurance_ts": "1593684730000"
},
{
"email": "-----%%%@metu.edu.tr",
"domain": "metu.edu.tr",
"first_occurance_ts": "1593684730000",
"last_occurance_ts": "1593684730000"
},
{
"email": "AdM----iiiiNNN%%%[email protected]",
"domain": "metu.edu.tr",
"first_occurance_ts": "1593684730000",
"last_occurance_ts": "1593684730000"
}
],
"email_count": "4"
}
}Note
As this endpoint is the gateway-ed version of a stream RPC function, the response is actually not a JSON message, it is the concatenation of seperate JSONs each for a different Leak object.
- Endpoint:
/v1-beta/leaks-by-domain - Method:
POST - Example request:
{
"domain": "ieee.metu.edu.tr"
}- Example response:
{
"leaks": [
{
"id": "5efdb25c0479ba527b1788f6",
"name": "fancy leak",
"emails": [
{
"email": "[email protected]",
"domain": "ieee.metu.edu.tr",
"first_occurance_ts": "1593684730000",
"last_occurance_ts": "1593684730000"
}
],
"email_count": "1"
},
{
"id": "5efdb2660479ba527b1788f7",
"name": "moore fancy leak",
"emails": [
{
"email": "[email protected]",
"domain": "ieee.metu.edu.tr",
"first_occurance_ts": "1593684730000",
"last_occurance_ts": "1593684730000"
}
],
"email_count": "1"
}
]
}- Endpoint:
/v1-beta/leaks-by-email - Method:
POST - Example request:
{
"domain": "ieee.metu.edu.tr"
}- Example response:
{
"leaks": [
{
"id": "5efdb25c0479ba527b1788f6",
"name": "fancy leak"
},
{
"id": "5efdb2660479ba527b1788f7",
"name": "moore fancy leak"
},
{
"id": "5efdb27c0479ba527b1788f8",
"name": "the big turkey data leak"
},
{
"id": "5efdb27c0479ba527b1788f8",
"name": "the big turkey data leak"
},
{
"id": "5efdb2940479ba527b1788f9",
"name": "n00bZ l€@k"
}
]
}- Example error response:
{
"error": "Unable to get Email object ID",
"code": 13,
"message": "Unable to get Email object ID"
}Further information about the system -such as API details and design decisions- can be found on the system documentations in docs/.