Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Reducing interest group payload by compressing renderURLs #1076

Open
ardianp-google opened this issue Mar 6, 2024 · 8 comments
Open

Reducing interest group payload by compressing renderURLs #1076

ardianp-google opened this issue Mar 6, 2024 · 8 comments

Comments

@ardianp-google
Copy link
Contributor

There’s a hard limit on interest group metadata size, both per IG (1MB) and in total size (10MB). In Google Ads, a significant chunk of the metadata is used to store renderURLs.

As renderURLs have a specific format (boiler plate with some identifiers), we wonder if the API allows some compressions, e.g., by having a template and only providing the identifiers for each renderURL.

For Bidding and Auction service, there’s a concept of adRenderId to reduce the payload size to be sent to the server. However, we still need to store the full renderURL on the device, and hence, this mechanism doesn’t help with the metadata size issue.

@dmdabbs
Copy link
Contributor

dmdabbs commented Mar 6, 2024

@ardianp-google could you sketch a bit to illustrate what you are thinking?

Are you imagining some named template(s), not unlike how size groups are defined, and then a renderURL could be specified as some combination of a) template key and b) a macro k/v map? Chrome would need to canonicalize this prior to calculating k-anon. Do you envision Chrome merging the identifiers into the template so the seller can score it?

"ads": [
    {
      "renderURL": "https://tdsf.doubleclick.net/td/adfetch/gda?adg_id=140544969557&cr_id=683341607979&cv_id=0&format=${AD_WIDTH}x${AD_HEIGHT}&rds=${RENDER_DATA}",
      "metadata": ...
    }
]

Are you envisioning something like:

"renderURLTemplates": {
    "template1":"https://tdsf.doubleclick.net/td/adfetch/gda?adg_id=${A}&cr_id=${B}&cv_id=${C}&format=${AD_WIDTH}x${AD_HEIGHT}&rds=${RENDER_DATA}"
},

"ads": [
    /* "legacy" renderURL */
    {
        "renderURL": "https://tdsf.doubleclick.net/td/adfetch/gda?adg_id=140544969557&cr_id=683341607979&cv_id=0&format=${AD_WIDTH}x${AD_HEIGHT}&rds=${RENDER_DATA}",
        "metadata": ...
    },
    /* "compressed" renderURL */
    {
        "renderURL": {"template1": {"A":140544969557, "B":683341607979,"C":0}},
        "metadata": ...
    }
]

@patmmccann
Copy link
Contributor

patmmccann commented Mar 6, 2024

This seems like an obvious win, look at all this repeating text; it happens not just in renderUrl but also in metadata

image

@patmmccann
Copy link
Contributor

patmmccann commented Mar 6, 2024

this could be adComponents: [], adTokens: { a: 'https://..../gda?', b: '150058563090'}, ads: [{ metadata:"[\"{$b}\"...], renderURL: '{$a}adg_id={$b}&cr_id=.....' etc, or some more valid form of macro definition and expansion

@JensenPaul
Copy link
Collaborator

This is something I've been thinking about for a while to reduce interest group sizes. I imagined instead of:

joinAdInterestGroup({ads: [{renderURL: 'https://www.example-dsp.com/ads?ad_id=123456&ad2_id=987654', ...},
                           {renderURL: 'https://www.example-dsp.com/ads?ad_id=234567&ad2_id=876543', ...},
                           {renderURL: 'https://www.example-dsp.com/ads?ad_id=345678&ad2_id=765432', ...}],

I imagined a templated version like:

joinAdInterestGroup({ad_template: 'https://www.example-dsp.com/ads?ad_id=${1}&ad2_id=${2}',
                     ads: [{templatedRenderURL: ['123456', '987654'], ...},
                           {templatedRenderURL: ['234567', '876543'], ...},
                           {templatedRenderURL: ['345678', '765432'], ...}],

Are you imagining some named template(s), not unlike how size groups are defined, and then a renderURL could be specified as some combination of a) template key and b) a macro k/v map?

Yes, I think that's what I'm thinking.

Chrome would need to canonicalize this prior to calculating k-anon. Do you envision Chrome merging the identifiers into the template so the seller can score it?

Yes, I imagine the browser would only use the unexpanded/compresssed form when stored.

@dmdabbs
Copy link
Contributor

dmdabbs commented Mar 6, 2024

Yes, positional (tokens in an array) is slimmer. And it may be the case that a DSP would only need a single template, as in your sketch, though I'm sure there will be some that need flexibility due to various systems or whatever.

Is this what you are also thinking @ardianp-google?

@ardianp-google
Copy link
Contributor Author

I imagine something closer to what @JensenPaul's proposed, and that's sufficient for our needs. Though it's also a valid point that a more expressive language could be needed to cater more complicated systems.

@dmdabbs
Copy link
Contributor

dmdabbs commented Mar 11, 2024

I imagine something closer to what @JensenPaul's proposed, and that's sufficient for our needs. Though it's also a valid point that a more expressive language could be needed to cater more complicated systems.

This may suffice in today's PA that only does banners. A single renderURL template may be restrictive once PS handles newfangled media like video or native.

@JensenPaul , assuming a single URL template per IG, here's another take (changed to camel case).

"adTemplate": "https://www.example-dsp.com/ads?ad_id=${1}&ad2_id=${2}&cv_id=${3}&format=${AD_WIDTH}x${AD_HEIGHT}&rds=${RENDER_DATA}",

"ads": [
    /* "legacy" renderURL */
    {
        "renderURL": "https://tdsf.doubleclick.net/td/adfetch/gda?adg_id=140544969557&cr_id=683341607979&cv_id=0&format=${AD_WIDTH}x${AD_HEIGHT}&rds=${RENDER_DATA}",
        "sizeGroup": "mySizeGroup",
        "metadata": {...},
    },
    /* "compressed" renderURL */
    {
        "templatedRenderURL": ['140544969557', '683341607979', '0'],
        "sizeGroup": "mySizeGroup",
        "metadata": {...},
    }
]

When bidding with a templated creative, does the bidder submit the tokens array as render.url?

      return {         
            bid: 1.25,
            
            render: {
               'url': renderURL,   /* this should be a reference to the ad's templatedRenderURL array? */
               'width': '300', 
               'height': '250'
            },
            
            allowComponentAuction: true
        };

@MattMenke2
Copy link
Contributor

MattMenke2 commented Mar 12, 2024

If we do this, we may need some sort of abuse protection, since simple substitution into a shared string potentially allows O(n^2) blowup, for strings that don't even need to be received from a server, but could instead be generated by Javascript.

Edit: I'd think the main effect of abuse would be to bring the browser to a crawl or crash it as it extracts URLs, if it keeps them all in memory. It wouldn't swamp the k-anon server (since we only send hashes there).

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

5 participants