interpret Mastodon ActivityPub HTTP 202 response as failure #16

snarfed · 2017-10-17T20:18:31Z

mastodon's AP code returns HTTP 202 to inbox POSTs that fail signature verification:

TODO:

interpret this as a failure instead of success, and return non-2xx ourselves
figure out why production instances like https://mastodon.technology/ and https://mastodon.social/ are currently failing to verify our signatures

snarfed · 2017-10-17T20:19:16Z

snarfed · 2017-10-18T14:30:39Z

aha, looks like mastodon changed this 202 to a 401 in head (2.0.0): mastodon/mastodon@dfaa219#diff-b71581bf00ae84fbac757c5373533ebb

snarfed · 2018-01-26T23:36:39Z

tentatively reopening. i'm seeing 202 from mastodon again on Create but the reply isn't showing up, e.g. https://snarfed.org/2018-01-26_chris-aldrich-%f0%9f%8d%8d-anarchivistchaos-social-edsudigipres-club-sn-mastodon to https://mastodon.social/@chrisaldrich/99418039176997784 .

https://github.com/tootsuite/mastodon/blob/master/app/controllers/activitypub/inboxes_controller.rb at master shows that mastodon evidently returns 202 for success, so probably something else is going on that prevents it from appearing.

{
  "@context": "https://www.w3.org/ns/activitystreams",
  "object": {
    "inReplyTo": "https://mastodon.social/users/chrisaldrich/statuses/99418039176997784",
    "cc": [
      "https://www.w3.org/ns/activitystreams#Public",
      "https://mastodon.social/users/chrisaldrich",
      "https://mastodon.social/users/chrisaldrich/followers",
      "https://mastodon.technology/users/snarfed",
      "https://digipres.club/users/edsu",
      "https://chaos.social/users/anarchivist"
    ],
    "attributedTo": [
      {
        "preferredUsername": "snarfed.org",
        "name": "Ryan Barrett",
        "url": "https://snarfed.org/",
        "image": [
          {
            "url": "https://secure.gravatar.com/avatar/947b5f3f323da0ef785b6f02d9c265d6?s=96&d=blank&r=g",
            "type": "Image"
          }
        ],
        "type": "Person",
        "id": "https://fed.brid.gy/snarfed.org",
        "icon": [
          {
            "url": "https://secure.gravatar.com/avatar/947b5f3f323da0ef785b6f02d9c265d6?s=96&d=blank&r=g",
            "type": "Image"
          }
        ]
      }
    ],
    "tag": [
      {
        "name": "reply"
      }
    ],
    "id": "https://snarfed.org/2018-01-26_chris-aldrich-%f0%9f%8d%8d-anarchivistchaos-social-edsudigipres-club-sn-mastodon",
    "name": "reply \n\nJanuary 26, 2018 Ryan Barrett\nLeave a comment\n\n\nthanks chris! bridgy fed is definitely open and ready for (more) users. it\u2019s not perfect yet, but it is usable. i\u2019m using it to post this reply right now! \n\nStandard",
    "url": "https://snarfed.org/2018-01-26_chris-aldrich-%f0%9f%8d%8d-anarchivistchaos-social-edsudigipres-club-sn-mastodon",
    "content": " <p>thanks chris! bridgy fed is definitely open and ready for (more) users. it\u2019s not perfect yet, but it is usable. i\u2019m using it to post this reply right now! <a class=\"u-in-reply-to\" href=\"https://mastodon.social/@chrisaldrich/99418039176997784\"></a> <cite class=\"via\"><a href=\"https://fed.brid.gy/\"></a></cite></p> ",
    "published": "2018-01-26T12:50:25-08:00",
    "@context": "https://www.w3.org/ns/activitystreams",
    "type": "Note"
  },
  "type": "Create"
}

snarfed · 2018-01-27T00:24:42Z

i suspect this is due to settings on mastodon.social specifically (currently 2.2.0rc2), since replies and likes still work on lgbt.io (2.1.2), chaos.social (2.1.2), and others.

swentel · 2018-05-25T20:25:30Z

Some feedback re: 202 responses.

Been playing around with integrating Drupal. I've done 3 tests, of which 2 work fine. Test url is https://mastodon.technology/@snarfed/3194674

What works is like (https://realize.be/like/1468) and repost (https://realize.be/repost/1469)
Interestingly enough, I got "202 response! If this is Mastodon 1.x, their signature verification probably failed. :(" back for both responses.

When trying to send a reply, I get back the same response, but nothing shows up on the post on mastodon itself.

(It might be that I'm missing something, currently, I have the .htaccess and atom feed, but nothing done yet with WebSub. But AFAICT, that shouldn't matter for sending a webmention.)

valpackett · 2018-07-06T12:55:02Z

mastodon 2.4.2, replies still don't show up (log), and likes do still seem to work

swentel · 2018-09-08T16:01:52Z

So out of curiosity, I tried a reply again today :) (to https://jawns.club/@timplunkett/100606109220664799)

The latest reply I tested now gives something back like 'Public key not found for key acct:[email protected]'. I haven't checked yet what that means and/or if it's relevant, but just wanted to leave it here.

snarfed · 2018-09-08T18:48:37Z

@swentel thanks for the update! that looks like #29, but the conclusion there was that they needed a representative h-card (#29 (comment)), which isn't your problem. https://fed.brid.gy/.well-known/[email protected] works fine.

...ah, but your site isn't actually set up for bridgy fed. https://realize.be/.well-known/host-meta and https://realize.be/.well-known/webfinger currently 404. they should redirect to fed.brid.gy instead. https://fed.brid.gy/#setup

swentel · 2018-09-08T19:03:25Z

Right - my bad, I commented out the lines in my .htaccess .. still the same outcome now.

snarfed · 2018-09-08T19:10:17Z

thanks! it actually gets a 202 from mastodon now. so we're back at the original issue here. whee!

https://fed.brid.gy/log?start_time=1536433273&key=https%3A%2F%2Frealize.be%2Freply%2Fcontent%2F1541+https%3A%2F%2Fjawns.club%2F%40timplunkett%2F100606109220664799

swentel · 2018-09-24T11:54:25Z

So, I checked mastodon, in app/controllers/activitypub/inboxes_controller.rb it will return 202 when the request is signed valid. But process_payload can still well basically do nothing. Kind of weird behavior if you tell me ..

swentel · 2018-09-26T17:19:32Z

So I finally found out why replies don't show up. The problem is that mastodon compares the id on the object with the actor url.

See https://github.com/tootsuite/mastodon/blob/master/app/lib/activitypub/activity/create.rb#L8
See https://github.com/tootsuite/mastodon/blob/master/app/lib/activitypub/activity/create.rb#L269

Since fed.brid.gy sends the url of the post on the website as id, and the actor url is something like https://fed.brid.gy/{sitename} this will now fail. Also, the 'url' property in an object gets the same check, but is ignored when it fails, the reply will still show up if the id check is fine.

When sending an id which looks like 'https://fed.brid.gy/realize.be?unique_id_here', the reply will show up. This has an impact on the canonical URL of the post being fed.brid.gy and not the website url. As a consequence, I guess round trips can't work anymore. One workaround could be to create the id in the form of https://fed.brid.gy/realize.be?url={original_url} - that works (see the 'Welcome to the club! AGAIN' canonical url). Of course, if at some point mastodon decides to start ignore get params, then it's back to the drawing table.

(I guess this also has impact on discovery of posts as they basically will have the same problem)

I often wonder whether it would be possible to set the id of https://fed.brid.gy/realize.be' to realize.be, but I'm afraid that ain't possible at all ?

Working replies:
https://mastodon.social/users/swentel/statuses/100776214865184909
https://mastodon.technology/@snarfed/3194674

swentel · 2018-09-26T17:31:36Z

this is the minimal object that I send:

stdClass Object
(
    [@context] => https://www.w3.org/ns/activitystreams
    [type] => Create
    [actor] => https://fed.brid.gy/realize.be
    [to] => Array
        (
            [0] => https://www.w3.org/ns/activitystreams#Public
            [1] => https://mastodon.social/users/swentel
        )

    [object] => stdClass Object
        (
            [id] => https://fed.brid.gy/realize.be?unique-id-needed-7
            [url] => https://realize.be/reply/content/1567
            [type] => Note
            [published] => 2018-09-26T17:27:21Z
            [attributedTo] => https://fed.brid.gy/realize.be
            [inReplyTo] => https://mastodon.social/users/swentel/statuses/100776214865184909
            [content] => <a href="https://mastodon.social/users/swentel">@swentel</a> A minimal object
            [to] => Array
                (
                    [0] => https://www.w3.org/ns/activitystreams#Public
                    [1] => https://mastodon.social/users/swentel
                )

        )

)

This is another one which also includes the 'mention' tag. This makes it so that a notification is also send to the user. The reply above did show up on the post, so it isn't that much of a deal.

So, I guess the when posting, bridgy should prepend @{username}.
Or, documentation could also say that we we find @{username} urls in the post, we add the mention tag. (this could be another issue of course, but wanted to document this too)

stdClass Object
(
    [@context] => https://www.w3.org/ns/activitystreams
    [type] => Create
    [actor] => https://fed.brid.gy/realize.be
    [to] => Array
        (
            [0] => https://www.w3.org/ns/activitystreams#Public
            [1] => https://mastodon.social/users/swentel
        )

    [object] => stdClass Object
        (
            [id] => https://fed.brid.gy/realize.be?unique-id-needed-6
            [url] => https://realize.be/reply/content/1567
            [type] => Note
            [published] => 2018-09-26T17:28:38Z
            [attributedTo] => https://fed.brid.gy/realize.be
            [inReplyTo] => https://mastodon.social/users/swentel/statuses/100776214865184909
            [content] => <a href="https://mastodon.social/users/swentel">@swentel</a> with url
            [to] => Array
                (
                    [0] => https://www.w3.org/ns/activitystreams#Public
                    [1] => https://mastodon.social/users/swentel
                )

            [tag] => Array
                (
                    [0] => stdClass Object
                        (
                            [type] => Mention
                            [href] => https://mastodon.social/users/swentel
                            [name] => @swentel
                        )

                )

        )

)

snarfed · 2018-09-26T17:45:27Z

this is really really great progress. thank you so much @swentel!

swentel · 2018-09-26T19:34:04Z

So I've been reading the comparison code. It compares the id of the object with @account.uri which is fetched here:
https://github.com/tootsuite/mastodon/blob/master/app/services/resolve_account_service.rb#L123
and then goes to
https://github.com/tootsuite/mastodon/blob/master/app/services/resolve_account_service.rb#L161

There's a lot going on here, it goes to fetch the atom feed, and then tries to find the author uri there, if I'm reading that right. Now, my atom feed uses granary, but it seems fine afaics.
When nothing is found, it either fills in nothing or falls back to author_uri.content - which I have no idea atm what this does. Haven't done much ruby, so clueless.

Anyway, I feel like there might be a way out here, but I might be naive. It's also late now, so might not be fresh anymore :)

swentel · 2018-09-26T20:31:07Z

So when running that code on the body of my atom feed, it returns http://realize.be - out of ideas again :)

snarfed · 2018-10-11T00:28:54Z

@swentel i think you're right that the best fix here is probably to wrap our translated post object ids under https://fed.brid.gy/... somewhere, so that they have the same prefix as the already wrapped actor ids.

(i doubt they'd drop query parameters when checking uniqueness, but if they do, we can always put the original post URL in the path instead of query, eg https://fed.brid.gy/realize.be/reply/content/1567.)

one key question: i only see one reply of yours on the original mastodon post you were testing against, https://mastodon.social/users/swentel/statuses/100776214865184909 . mastodon shows its URL as https://fed.brid.gy/realize.be#create . do you know if you put that in both id and url? or just id? i'm wondering which one mastodon uses. if url, then maybe we wouldn't have to actual redirect or serve from our wrapped id URLs at all.

swentel · 2018-10-11T07:11:44Z

Right, I've deleted some replies. I added a new one. So mastodon either gets it from id or url, see https://github.com/tootsuite/mastodon/blob/master/app/lib/activitypub/activity/create.rb#L52

I've added another reply on https://mastodon.social/@swentel/100776214865184909 with "This is pretty cool!"

This is the payload from the request

stdClass Object
(
    [type] => Create
    [@context] => https://www.w3.org/ns/activitystreams
    [actor] => https://fed.brid.gy/realize.be
    [object] => Array
        (
            [type] => Note
            [to] => Array
                (
                    [0] => https://www.w3.org/ns/activitystreams#Public
                    [1] => https://mastodon.social/users/swentel/inbox
                )

            [id] => https://fed.brid.gy/realize.be?post=https://realize.be/notes/1573
            [inReplyTo] => https://mastodon.social/@swentel/100776214865184909
            [content] => This is pretty cool!
            [published] => 2018-10-11T07:08:01Z
        )

)

swentel · 2018-10-11T13:40:01Z

So I've looked where the rewrite could happen, but I'm not 100% sure exactly what I'm doing :/
I've attached a diff I have locally, but not sure if the place I'm changing it makes sense .. :)

rewrite.txt

snarfed · 2018-10-11T18:28:41Z

thanks!

i'm still curious, if we point id to fed.brid.gy but keep url pointing to the user's own site, whether mastodon will render links from url or id. i noticed you omitted url in #16 (comment) . mind trying again with url pointing somewhere other than id to see which one mastodon renders?

and your patch looks good! i'd probably put it in common.postprocess_as2() instead, and i'm inclined to put the full URL in the path like the existing/wm/... URL route, eg https://fed.brid.gy/redir/https://realize.be/ and https://fed.brid.gy/redir/https://realize.be/reply/content/1567. but you're definitely on the right path!

swentel · 2018-10-11T21:58:34Z

Url pointing to the post will be discarded, see https://github.com/tootsuite/mastodon/blob/master/app/lib/activitypub/activity/create.rb#L243 - it uses the same validation function (which still annoys me, but oh well)

Just to be sure that I'm not fooling myself, send this payload:

stdClass Object
(
    [type] => Create
    [@context] => https://www.w3.org/ns/activitystreams
    [actor] => https://fed.brid.gy/realize.be
    [object] => Array
        (
            [type] => Note
            [to] => Array
                (
                    [0] => https://www.w3.org/ns/activitystreams#Public
                    [1] => https://mastodon.social/users/swentel/inbox
                )

            [id] => https://fed.brid.gy/realize.be?post=https://realize.be/notes/1550
            [url] => https://realize.be/notes/1550
            [inReplyTo] => https://mastodon.social/@swentel/100776214865184909
            [content] => Sending different host in url. Will be discarded normally.
            [published] => 2018-10-11T21:55:10Z
        )

)

The url has been ignored here. Payload underneath has bridgy host in the url, and then it takes that one

stdClass Object
(
    [type] => Create
    [@context] => https://www.w3.org/ns/activitystreams
    [actor] => https://fed.brid.gy/realize.be
    [object] => Array
        (
            [type] => Note
            [to] => Array
                (
                    [0] => https://www.w3.org/ns/activitystreams#Public
                    [1] => https://mastodon.social/users/swentel/inbox
                )

            [id] => https://fed.brid.gy/realize.be?post=https://realize.be/notes/1530
            [url] => https://fed.brid.gy/realize.be?post=https://realize.be/notes/1545
            [inReplyTo] => https://mastodon.social/@swentel/100776214865184909
            [content] => Sending url with host same as actor, should take one now
            [published] => 2018-10-11T21:57:03Z
        )

)

You can see them both on https://mastodon.social/@swentel/100776214865184909

@swentel

for #16, #32. cc @swentel. more to come.

@swentel

for #16, #32. cc @swentel.

swentel · 2018-10-14T15:50:07Z

Tested, awesome, awesome :)

I guess we can close this one!

Or maybe remove the log message in bridgy, we know that 202 is ok it terms of signature.

for #16

snarfed · 2018-10-17T14:42:35Z

agreed! i've removed the log message. thanks for the nudge.

snarfed closed this as completed Oct 23, 2017

snarfed reopened this Jan 26, 2018

swentel mentioned this issue May 25, 2018

fediverse: investigate https://fed.brid.gy/ swentel/indieweb#67

Closed

swentel mentioned this issue Sep 26, 2018

My toot doesn't show up :( mastodon/mastodon#8328

Closed

2 tasks

swentel mentioned this issue Oct 10, 2018

When fetching fed.brid.gy/{domain} as get, redirect to user site #32

Closed

snarfed mentioned this issue Oct 11, 2018

Fix bad URL schemes being accepted mastodon/mastodon#6219

Merged

snarfed added a commit that referenced this issue Oct 14, 2018

add /r/ redirect endpoint

e720e54

for #16, #32. cc @swentel. more to come.

snarfed added a commit that referenced this issue Oct 14, 2018

wrap activity ids and urls and actor ids in our /r/ endpoint

b25960c

for #16, #32. cc @swentel.

swentel mentioned this issue Oct 15, 2018

Add mention tags in replies #34

Closed

snarfed mentioned this issue Oct 15, 2018

fix AP => webmention #35

Closed

snarfed added a commit that referenced this issue Oct 17, 2018

wm => AP: drop warning on 202 response, now that mastodon is fixed

eea1904

for #16

snarfed closed this as completed Oct 17, 2018

snarfed mentioned this issue Aug 19, 2019

Website posts not federating to Mastodon #56

Closed

This was referenced Dec 14, 2022

For articles, link to original post directly instead of using BF redirect? #345

Closed

Validated Content with Fed.Brid.gy Link Won't Publish #343

Closed

snarfed mentioned this issue Oct 15, 2023

Authorization checks on incoming activities #566

Closed

8 tasks

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

interpret Mastodon ActivityPub HTTP 202 response as failure #16

interpret Mastodon ActivityPub HTTP 202 response as failure #16

snarfed commented Oct 17, 2017 •

edited

Loading

snarfed commented Oct 17, 2017

snarfed commented Oct 18, 2017 •

edited

Loading

snarfed commented Jan 26, 2018

snarfed commented Jan 27, 2018

swentel commented May 25, 2018 •

edited

Loading

valpackett commented Jul 6, 2018

swentel commented Sep 8, 2018

snarfed commented Sep 8, 2018

swentel commented Sep 8, 2018

snarfed commented Sep 8, 2018

swentel commented Sep 24, 2018 •

edited

Loading

swentel commented Sep 26, 2018

swentel commented Sep 26, 2018 •

edited

Loading

snarfed commented Sep 26, 2018

swentel commented Sep 26, 2018

swentel commented Sep 26, 2018

snarfed commented Oct 11, 2018

swentel commented Oct 11, 2018

swentel commented Oct 11, 2018

snarfed commented Oct 11, 2018

swentel commented Oct 11, 2018

swentel commented Oct 14, 2018 •

edited

Loading

snarfed commented Oct 17, 2018

interpret Mastodon ActivityPub HTTP 202 response as failure #16

interpret Mastodon ActivityPub HTTP 202 response as failure #16

Comments

snarfed commented Oct 17, 2017 • edited Loading

snarfed commented Oct 17, 2017

snarfed commented Oct 18, 2017 • edited Loading

snarfed commented Jan 26, 2018

snarfed commented Jan 27, 2018

swentel commented May 25, 2018 • edited Loading

valpackett commented Jul 6, 2018

swentel commented Sep 8, 2018

snarfed commented Sep 8, 2018

swentel commented Sep 8, 2018

snarfed commented Sep 8, 2018

swentel commented Sep 24, 2018 • edited Loading

swentel commented Sep 26, 2018

swentel commented Sep 26, 2018 • edited Loading

snarfed commented Sep 26, 2018

swentel commented Sep 26, 2018

swentel commented Sep 26, 2018

snarfed commented Oct 11, 2018

swentel commented Oct 11, 2018

swentel commented Oct 11, 2018

snarfed commented Oct 11, 2018

swentel commented Oct 11, 2018

swentel commented Oct 14, 2018 • edited Loading

snarfed commented Oct 17, 2018

snarfed commented Oct 17, 2017 •

edited

Loading

snarfed commented Oct 18, 2017 •

edited

Loading

swentel commented May 25, 2018 •

edited

Loading

swentel commented Sep 24, 2018 •

edited

Loading

swentel commented Sep 26, 2018 •

edited

Loading

swentel commented Oct 14, 2018 •

edited

Loading