How do we want to run this? For all wikis? By shards?

Taking as example what @Legoktm did for T299244, I made a draft patch at https://gerrit.wikimedia.org/r/c/operations/puppet/+/756069

If you think we can go directly with foreachwiki, please let me know. It'll make the file substantially shorter too.

SUPPORT, all wikis, once per month.

In T299823#7642475, @MarcoAurelio wrote:

How do we want to run this? For all wikis? By shards?

Taking as example what @Legoktm did for T299244, I made a draft patch at https://gerrit.wikimedia.org/r/c/operations/puppet/+/756069

If you think we can go directly with foreachwiki, please let me know. It'll make the file substantially shorter too.

I think we can do this with a plain foreachwiki; in T299244 there was some urgency to get the counts fixed so I split it by shard but here if it's a bit slower that's fine. Each wiki will still get counts refreshed once a month anyways.

Change 756069 had a related patch set uploaded (by Legoktm; author: MarcoAurelio):

[operations/puppet@production] [WIP] p::mediawiki::maintenance: Run recountCategories.php regularly

https://gerrit.wikimedia.org/r/756069

In T299823#7642742, @Legoktm wrote:

I think we can do this with a plain foreachwiki; in T299244 there was some urgency to get the counts fixed so I split it by shard but here if it's a bit slower that's fine. Each wiki will still get counts refreshed once a month anyways.

Thank you. Patch amended to use foreachwiki.

I see that the script suggests to run rMW maintenance/cleanupEmptyCategories.php in --mode remove if the recounting category script is run in the pages mode (we're configuring it to use all which includes that mode). Shall we configure a periodic job for cleanupEmptyCategories.php too?

In T299823#7643093, @MarcoAurelio wrote:

In T299823#7642742, @Legoktm wrote:

I think we can do this with a plain foreachwiki; in T299244 there was some urgency to get the counts fixed so I split it by shard but here if it's a bit slower that's fine. Each wiki will still get counts refreshed once a month anyways.

Thank you. Patch amended to use foreachwiki.

LGTM!

I see that the script suggests to run rMW maintenance/cleanupEmptyCategories.php in --mode remove if the recounting category script is run in the pages mode (we're configuring it to use all which includes that mode). Shall we configure a periodic job for cleanupEmptyCategories.php too?

Good point. See https://gerrit.wikimedia.org/r/c/mediawiki/core/+/756415

@Majavah Could you please run the improved recountCategories script on the Beta Cluster and save the output for review in a Paste?

foreachwiki maintenance/recountCategories --mode all >path/to/logs

@Legoktm and myself would be interested in knowing how it does before merging the Puppet patch.

Thanks!

Mentioned in SAL (#wikimedia-releng) [2022-01-28T21:45:05Z] <taavi> running recountCategories.php on all beta wikis per T299823#7652496

In T299823#7652496, @MarcoAurelio wrote:

@Majavah Could you please run the improved recountCategories script on the Beta Cluster and save the output for review in a Paste?

P19566

Change 756069 merged by RLazarus:

[operations/puppet@production] mediawiki::maintenance: Run recountCategories.php monthly on all wikis

https://gerrit.wikimedia.org/r/756069

Mentioned in SAL (#wikimedia-operations) [2022-02-03T20:43:10Z] <rzl> rzl@mwmaint1002:~$ sudo systemctl start mediawiki_job_recount_categories.service # T299823

As requested on IRC, here's the log from today's run:

In T299823#7677609, @Legoktm wrote:

Maybe not, https://en.wikipedia.org/wiki/Wikipedia:Village_pump_(technical)#Empty,_but_not_empty?_categories is a bit weird.

I've checked the example categories mentioned by Liz in that thread and they show as empty on Category:Empty categories awaiting deletion and they have no articles inside either (so they're indeed empty categories correctly displaying as empty right now). Unless I am not reading the thread right, looks like there are no issues now?

As for the cause, could it be some caching issue? or DB master/replica lag?

enwiki had quite a bit of rows to update https://phabricator.wikimedia.org/F34941981$6776 only next to commonswiki https://phabricator.wikimedia.org/F34941981$2934

Re: Tech News - let me know if any changes are needed, ASAP. I'll be freezing it for translations within ~2 hours.

In T299823#7684603, @Quiddity wrote:

Re: Tech News - let me know if any changes are needed, ASAP. I'll be freezing it for translations within ~2 hours.

I think what we have now is fine. On IRC MA and I discussed that while it is possible things are temporarily wrong because of this script (which is still undetermined), overall things are less wrong.

Shall we ensure: absent the script while the investigation on the reported issues is ongoing?