What change would you like to see?

I would like more insight into how the crawl queue has grown over time, and how close it might be to finishing.

Context

We came up with this graph idea while crawling Vice but it never made it into an issue.

@edsu Also came up with this in parallel and prototyped it!! (Above image is also his)

From Ed on Mastodon:

I think it ought to be possible to use the data to project when the crawl might complete, by fitting a line to each and then projecting where they (hopefully) intersect...