tensorflow::serving::CachingManager

#include <caching_manager.h>

A manager that manages and loads servables on-demand.

Summary

Upon receiving the request for a servable name and optional version, the manager checks if it already has the requested servable loaded. If not, it initiates the load operation and then serves the request.

The manager blocks on the load operation and returns the handle when the servable has been loaded, or upon error.

Inheritance

Inherits from: tensorflow::serving::Manager

Constructors and Destructors
~CachingManager()

Friend classes
test_util::CachingManagerTestAccess
friend class

Public static functions
Create(Options options, std::unique_ptr< LoaderFactory > loader_factory, std::unique_ptr< CachingManager > *caching_manager)
Status

Public functions
GetAvailableUntypedServableHandles() const override
virtual std::map< ServableId, std::unique_ptr< UntypedServableHandle > >
ListAvailableServableIds() const override
virtual std::vector< ServableId >
Gets a list of all available servable ids, i.e.

Classes
tensorflow::serving::CachingManager::LoaderFactory

An abstraction for a loader-factory to map from a servable request to the corresponding loader.

Structs
tensorflow::serving::CachingManager::Options

Config options and pluggable objects that will be used by the CachingManager.

Friend classes

test_util::CachingManagerTestAccess

friend class test_util::CachingManagerTestAccess

Public static functions

Create

Status Create(
  Options options,
  std::unique_ptr< LoaderFactory > loader_factory,
  std::unique_ptr< CachingManager > *caching_manager
)

Public functions

GetAvailableUntypedServableHandles

virtual std::map< ServableId, std::unique_ptr< UntypedServableHandle > > GetAvailableUntypedServableHandles() const override

ListAvailableServableIds

virtual std::vector< ServableId > ListAvailableServableIds() const override

Gets a list of all available servable ids, i.e.

each of these can be retrieved using GetServableHandle.

~CachingManager

 ~CachingManager() override