This was brought in the @kubernetes/sig-node meeting previously. I think having a CRI validation test is necessary eventually to qualify a new CRI shim and help CRI shim developers. It would be good to have such a test sooner than later.

Basically, we want a test framework and a set of tests to validate a CRI (grpc) server alone. For starters, we can just run tests locally on a machine where the grpc server runs. The test only needs to know the socket to talk to, and can perform the basic operations such as creating/deleting sandboxes/containers. Ideally, no runtime-specific bits should be included in the tests.

The CRI validation test should replace/subsume the existing runtime validation test (formerly known as automatic docker validation test: kubernetes/enhancements#57).

@kubernetes/sig-node, since we have multiple ongoing projects implementing CRI shims, this should be a sig-node community effort. If someone already has testing bits in their repository, you're welcome to donate such tests and/or harness. Volunteers to work on the CRI validation test is also welcome.

/cc @dchen1107 @Random-Liu @euank @yifan-gu @mrunalp