(Cross-post from https://amistrongeryet.substack.com/p/are-we-on-the-brink-of-agi, lightly edited for LessWrong. The original has a lengthier introduction and a bit more explanation of jargon.)

No one seems to know whether transformational AGI is coming within a few short years. Or rather, everyone seems to know, but they all have conflicting opinions. Have we entered into what will in hindsight be not even the early stages, but actually the middle stage, of the mad tumbling rush into singularity? Or are we just witnessing the exciting early period of a new technology, full of discovery and opportunity, akin to the boom years of the personal computer and the web?

AI is approaching elite skill at programming, possibly barreling into superhuman status at advanced mathematics, and only picking up speed. Or so the framing goes. And...

This is a follow-up to last week's D&D.Sci scenario: if you intend to play that, and haven't done so yet, you should do so now before spoiling yourself.

There is a web interactive here you can use to test your answer, and generation code available here if you're interested, or you can read on for the ruleset and scores.

RULESET

Encounters

The following encounters existed:

Each encounter had a Threat that determined how dangerous it was to adventurers.  When adventurers encountered that, they would roll [Threat]d2 to determine how challenging they found it.

However, many encounters had two different Threat levels, depending on whether they were alerted to the adventurers or not.  (A dragon that's woken up from...

Summary

In my first piece, I settled on conceiving of radical empathy as an object view. I highlight some other potential moral implications of object views:

1. More of what someone finds pleasant or is (knowingly or unknowingly) disposed to find pleasant is better for them, all else equal. Death can be bad for them if and because it deprives them of the objects of those preferences. Death can also be bad for them if and because they explicitly disprefer it or its consequences (more).
2. We should care about what animals care about, e.g. preventing their pain and keeping them close to those to which they’re attached (more).
3. Someone’s suffering might not necessarily always count against their life, in case it isn't about their life or aspects of it, e.g. grieving the loss of a loved one (more).
4. If

Summary

I illustrate and defend actualist object views as my conception of radical empathy, of being concerned exactly with what we would actually care about. As a kind of asymmetric person-affecting view, the most important implication for cause prioritization is probably lower priority for extinction risk reduction relative to total utilitarianism.

1. I illustrate with an example where I find actualist object views handle changing preferences better than other views (more).
2. An important implication for cause prioritization is the (Procreation) Asymmetry: we have reasons to prevent miserable lives, but not to create happy or fulfilling lives, for the sake of what those lives would actually care about (more).
   1. This is not (necessarily) antinatalist or pro-extinction, but it would probably lead to less priority for extinction risk reduction, compared to total utilitarianism (more).
3. I highlight some problems for actualism, addressing

Fatebook is a website that makes it extremely low friction to make and track predictions.

It's designed to be very fast - just open a new tab, go to fatebook.io, type your prediction, and hit enter. Later, you'll get an email reminding you to resolve your question as YES, NO, or AMBIGUOUS.

It's private by default, so you can track personal questions and give forecasts that you don't want to share publicly. You can also share questions with specific people, or publicly.

Fatebook syncs with Fatebook for Slack - if you log in with the email you use for Slack, you’ll see all of your questions on the website.

As you resolve your forecasts, you'll build a track record - Brier score, Relative Brier score, and see your calibration chart. You can...

Epistemic Status: This post is an attempt to condense some ideas I've been thinking about for quite some time. I took some care grounding the main body of the text, but some parts (particularly the appendix) are pretty off the cuff, and should be treated as such. 

The magnitude and scope of the problems related to AI safety have led to an increasingly public discussion about how to address them. Risks of sufficiently advanced AI systems involve unknown unknowns that could impact the global economy, national and personal security, and the way we investigate, innovate, and learn. Clearly, the response from the AI safety community should be as multi-faceted and expansive as the problems it aims to address. In a previous post, we framed fruitful collaborations between applied...