We have some APIs that expect byte[], some that expect ByteBuffer and some that can handle both. I think we can switch all of them to ByteBuffer since it's apparently cheap.

Furthermore, we copy a lot. That should be reduced.