tensorflow::serving::ServableHandle

#include <servable_handle.h>

A smart pointer to the underlying servable object T retrieved from the Loader.

Summary

Frontend code gets these handles from the ServableManager. The handle keeps the underlying object alive as long as the handle is alive. The frontend should not hold onto it for a long time, because holding it can delay servable reloading.

The T returned from the handle is generally shared among multiple requests, which means any mutating changes made to T must preserve correctness vis-a-vis the application logic. Moreover, in the presence of multiple request threads, thread-safe usage of T must be ensured.

T is expected to be a value type, and is internally stored as a pointer. Using a pointer type for T will fail to compile, since it would be a mistake to do so in most situations.

Example use:

// Define or use an existing servable:
class MyServable {
public:
  void MyMethod();
};

// Get your handle from a manager.
ServableHandle handle;
TF_RETURN_IF_ERROR(manager->GetServableHandle(id, &handle));

// Use your handle as a smart-pointer:
handle->MyMethod();