Hi Maciej, we really appreciate that you let us know your thoughts.

There are 2 ways to handle key expiration.

1. TTLs are persisted along with the key/value pairs in the server’s dataset. Servers ignore and delete expired keys in RAM during serving.
2. TTLs are stored outside the dataset. Servers are not aware of TTLs in any way. The server only supports deleting keys.

In both cases, there should be a separate offline GC job that manages TTL bookkeeping, and issues mutations (for # 1 TTL updates, for # 2 deletes) to servers.

In our opinion # 2 has quite a few advantages, major ones being:

• # 1 introduces a separate source of truth of what should be served that makes debugging hard and serving logic more complex. (source of truth 1: regular data updates; source of truth 2: garbage collection. the server needs complex logic to correctly handle updates for the same key from data updates and from the garbage collection if the key's TTL is up).
• # 1 Extra resources need to be used for garbage collection and we prefer to use it for other indispensable purposes.
• # 1 requires a dependency on absolute time clock that is risky inside TEE.
• # 1 still needs to support direct delete for adhoc deletion use cases. So overall # 1 adds more complexity.

That explains why there is no TTL mentioned in the current data format. Regarding manual updating:

The original API was a HTTP API so the idea was that there will be an update server accepting update requests and through some medium (e.g., files) the updates get persisted and propagated to the real bidding/scoring server. The current implementation is still part of that vision.

However, the offline garbage collection job (and the update server) is currently a low priority thing for us because that is something Adtech users can implement on their own. The current file-based server updates accept direct deletes, so it still follows the same direction. Right now our main focus is the critical serving path.