tensorflow::serving::ResourceUnsafeLoader

#include <loader.h>

A Loader that is oblivious to resources.

Summary

Its EstimateResources() method returns 0, thus effectively disabling resource-based safety checks in the serving system.

Loaders that are experimental, or run in environments that do not need the resource safety checks, can subclass ResourceUnsafeLoader instead of Loader.

Inheritance

Inherits from: tensorflow::serving::Loader

Public functions
EstimateResources(ResourceAllocation *estimate) const final
virtual Status
Estimates the resources a servable will use.

Public functions

EstimateResources

virtual Status EstimateResources(
  ResourceAllocation *estimate
) const final

Estimates the resources a servable will use.

IMPORTANT: This method's implementation must obey following requirements, which enable the serving system to reason correctly about which servables can be loaded safely:

  1. The estimate must represent an upper bound on the actual value.
  2. Prior to load, the estimate may include resources that are not bound to any specific device instance, e.g. RAM on one of the two GPUs.
  3. While loaded, for any devices with multiple instances (e.g. two GPUs), the estimate must specify the instance to which each resource is bound.
  4. The estimate must be monotonically non-increasing, i.e. it cannot increase over time. Reasons to have it potentially decrease over time Returns
    an estimate of the resources the servable will consume once loaded. If the servable has already been loaded, returns an estimate of the actual resource usage.