Hello, we want to solicit feedback from the ecosystem on a new proposal to add functionality to B&A Monitoring Services. Specifically, the ability for adtech to define custom metrics and error conditions in a way that allows adtech to get enough signal to confidently look at dashboards and alerts based on near real-time aggregations. We also seek to allow adtechs to migrate their implementation of this use case from the Chrome-client based implementation of the Real-Time Monitoring API to B&A Monitoring Services with minimal disruption.

Custom Metrics

We propose for adtech to be able to collect custom defined metrics within their UDF code. This is in addition to system collected metrics as defined in the monitoring explainer. Ad tech UDFs will be able to collect custom metrics and return these as a part of their function calls. These metrics will be treated similar to metrics with noise and Laplace noise will be added to them.

The privacy budget for monitoring metrics is fixed at epsilon=10 and users already have a way of allocating portions of the privacy budget to different metrics. Custom metrics will follow suite and users will be able to set the proportion of the privacy budget allocated to each custom metric.

Custom metrics will also support partitions and the noising will be done similar to system metrics as described in the explainer. The metrics will be aggregated inside the TEE for a period of time. Then, Laplace noise will be added to the aggregate before releasing it from the TEE. Once released from the TEE, collection of these metrics will be exactly the same as system metrics and. Dashboard etc. can be set up similar to those described in the monitoring explainer.

The monitoring explainer will be updated with details of custom metrics in due course.

Migration from RTM to Custom metrics

This section helps understand the parallels to the Real-Time Monitoring API and how you can migrate to custom metrics with minimal disruption.

Custom metrics can be used as an array partitioned into 1024 bits in which each bit can be independently set based on some custom [error] condition detectable within the UDF. Then, after all/any error conditions have been detected and the appropriate bits have been set, adtech would be able to emit this array as a single histogram contribution from within the UDF (either generateBid() for buyers or scoreAd() for sellers) to a destination server outside B&A.

In order to preserve privacy we will follow the same approach of the Chrome-client based solution to use differential privacy. However a key difference is that since the individual contributions are safe within the B&A environment, the noising doesn’t have to be local. This means that we are not following the RAPPOR algorithm of the Chrome-client based solution. Instead we will apply the noising on egress at aggregation time, using the same Laplace noising algorithm that we use for the B&A Services-defined System metrics we determined need to be noised.

It is important to note that we don’t want the introduction of these Custom Metrics to require additional privacy budget from the currently configured Epsilon = 10 for noised system metrics in B&A Monitoring Services. And because we want to make migration from the Client -based RTM solution easy, we propose for this custom metric’s privacy budget weight to be set by adtech,such that if “x” is the chosen privacy budget for the custom metrics, then the remaining budget for the noised system metrics is [10 - x].

Another key difference between the Chrome-client only and B&A implementations is that there is no need for adtech to debias the report. Instead, in a future explainer we aim to illustrate to adtech how to interpret the Laplace-noised metrics based on the sought confidence level interval for a particular alert (based on additional variables beyond available privacy budget like QPS) so that adtech can determine how to correctly set alerting thresholds for their use case.

We welcome the ecosystem to comment on this proposal!