Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Fix performance of account timelines #17709

Merged
merged 4 commits into from
Mar 8, 2022

Conversation

Gargron
Copy link
Member

@Gargron Gargron commented Mar 7, 2022

When exclude_reblogs, tagged or only_media filters are requested, there is no way that a reblog would turn up in the results; nevertheless, previously a join was performed even on those queries to filter out potential reblogs from excluded accounts. This made those queries unnecessarily slower.

Comparison for a "Posts" tab query as a logged-in follower (same profile):

Server Before After
mastodon.online (93 GB) 32.9ms 3.9ms
mastodon.social (401 GB) 829.9ms 6.5ms

Comparison for the "Media" tab (same profile):

Server Before After
mastodon.online (93 GB) 128ms 4ms
mastodon.social (401 GB) 18s 11ms

@Gargron Gargron added the performance Runtime performance label Mar 7, 2022
@Gargron Gargron force-pushed the fix-account-statuses-query-performance branch from 9a750f4 to 79f7354 Compare March 7, 2022 06:36
Copy link
Contributor

@ClearlyClaire ClearlyClaire left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The refactor is welcome, but I think I found a bug (inline comment).

Otherwise I'm on the fence about not listing DMs (or private messages, if you don't follow the person) that mention you, as there are little other places where you can check them (especially with Mastodon's DM conversation view not handling broken or forking threads very well). On the other hand, the speedup you're claiming is more than tempting.

I wonder if there's a way to improve the existing query, but I'm not sure we can efficiently handle the case where someone who often gets mentioned looks at the profile of someone they don't follow (or who sends a lot of DMs).

@Gargron Gargron force-pushed the fix-account-statuses-query-performance branch from e0606c8 to fa9a38f Compare March 7, 2022 22:35
@Gargron
Copy link
Member Author

Gargron commented Mar 7, 2022

No more behaviour change regarding mentions, join is faster than subquery. Fixed typo.

Copy link
Contributor

@ClearlyClaire ClearlyClaire left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The queries look good, they returned the same results than the old permitted_for in the few real-world data tests I tried, and they seem to please the query planner! Thanks!

@Gargron Gargron merged commit 8f6c67b into main Mar 8, 2022
@Gargron Gargron deleted the fix-account-statuses-query-performance branch March 8, 2022 08:14
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
performance Runtime performance
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants