While I doubt anyone will disagree with the aim, the means (mass surveillance) are not only violating privacy rights but also ineffective, for various reasons. In this thead I'll explain some of these reasons and what YOU can do to help stop the regulation. Let's start with the most important thing: Chatcontrol completely misses the point, as criminals basically rarely use messengers to share material - they're too slow to share large collections of CSAM.

Instead, they encrypt the files and upload them to a completely normal filesharing service. Since the files are encrypted, the service is unable to scan the contents, even if it wants to detect CSAM. Criminals can then simply share links to the content whereever they want [C1,4-6] - the scanning can't/won't hinder them! This alone should be enough to scrap the regulation! If that's enough for you, scroll far down to the sign (you'll see it) to learn what you can do.

But maybe we can catch criminals which DO share CSAM via messengers - there are some out there, and the proposal is directed at them. The regulation differentiates between 3 categories to detect [L2a]:
- Known CSAM
- New CSAM
- Grooming
While there has been lots of research, only detection of known CSAM is really feasible, and even that comes with issues. In the next posts, I'll explain some technical details why, skip 13 posts ahead if that's not your thing.

Known CSAM images are the easiest to detect - still, it's not an easy task. A computer cannot simply compare an image like a human can and say if they're the same or not. It could compare all pixels, but that would be slow AND you'd need to have CSAM stored wherever the code which is doing the comparison run. Such a comparison would also only work as long as the image is not modified at all. If the image is resized, brightened or blurred this detection won't work.

To fix this, so-called perceptual hashing is used. The idea is simple: Split an image into different areas and calculate a value for each area. This list of values for an image is called a perceptual hash. To compare images, calculate the difference between the values of the two hashes. If the difference is below a value (the detection threshold), the images are likely the same. Now the problem is: How to choose the threshold where you say the images are the same?

Two approaches:
Set threshold high. More modified CSAM will be detected (high recall). But set the threshold too high, and two unrelated images would be considered the same, in practice meaning a harmless image would be detected as CSAM.

Or: Set threshold low → less false positives (high precision). Too low, and you'll run into the same issue as when comparing pixels - images which have been slightly modified are not considered the same, not detecting known CSAM.

You can learn more about precision/recall here: https://mlu-explain.github.io/precision-recall/ For the detection of CSAM, there is no choice but to optimize for high precision: A low precision would mean many false positives. As there far are more normal messages than messges than messages containing CSAM, the amount of false postives would quickly drown out the true positives. Luckily, perceptual hashing is *just* precise enough. How precise exactly?

A well-known perceptual hashing algorithm is Microsofts PhotoDNA algorithm [T1a]. It's creator claims the chance of misidentification on a big analysis is about 1 in 50 billion [T1b]. While this is very impressive, such low error probabilities are an absolute must for reliably detecting CSAM. Take (as an example) WhatsApp: Assuming 100 billion messages per day [T2], even at these odds, there'll still be about 2 false positives PER DAY, ONLY for WhatsApp.

So yes, image comparison is practical, AS LONG AS YOU INCLUDE HUMAN REVIEW. There are other issues, but to detect known CSAM, the accuracy is not really in question. An important question is: Where does the image get analyzed? On a central server? You'd have to throw out end-to-end encryption (E2EE). Analyzing on the users devices is the only solution to *technically* keep E2EE intact, but why would you trust criminals to use unmodified devices which scan for CSAM?

There are also concerns that PhotoDNA may be reversible, so distributing hashes for detection would be equivalent to distributing CSAM [T3]. If reversible, it'd also be possible to intentionally create images which are falsely detected as CSAM, then flood the detection system with it, making detection of real CSAM a lot more difficult. If the above issues are resolved, we MAY be able to reliably detect known CSAM. However, finding new CSAM is far less precise (3 posts):

When detecting unknown CSAM, the previous issues apply AND there are two connected addtional issues: AI training and accuracy. To detect new material, an AI is trained by "looking at" lots of CSAM so it can learn what CSAM looks like - note that hashes are not enough, it needs to be actual CSAM. This means that if only the new EU center is allowed access to CSAM, only the EU center will be able to train an AI which detects new CSAM! Will they get it right?

As a rule of thumb: The better the AI works, the worse we can currently explain it's behavior [T4]. This means we will not be able to be sure the AI learned "the right way" to detect CSAM. And there's many possibilities to make false decisions: As an example, an AI cannot simply rely on detecting images of naked children, as that could probably lead to false positives when parents send photos of their children at the beach or in the bathtub to the grandparents.

Even if the AI can detect *actual porn* with people that *look* underage, this can still lead to false positives - there have been HUMAN mistakes about this [C2]. So how could an AI correctly classify CSAM vs. no CSAM if not even humans are able to? It's a difficult problem, and there WILL be many false positives. Even if the error rate was just 1% (in practice it's higher), this would mean many 100s of millions of messages are falsely reported EVERY DAY.

For grooming detection: I haven't found many details, but it seems even less advanced. The european comission says that Microsofts claims to have a technology with an accuracy of 88% [L2b], meaning a 12% error rate (it's unclear if the commission confused it with precision, but that would indicate low recall, which makes sense). In 2020, Microsoft presented their tool. Watch the presentation [T1c]: It's clear that it's not ready for automated large-scale detection.

The tool consisted (consists?) of simple text matching rules (for programmers: Regexps).
It
- is intended to help human moderators
- is not intended to run in real-time
- not for law enforcement
- only supports English! [T1c]
Adapting the detection techniques to other languages will, of course, take time. It's looking for specific phrases, so it'll probably have a high precision, but low recall. Still, sexting adults are probably going to trigger false positives.

The tool is also not intended for end-to-end encrypted messages [T1c]. Once the rules for matching text are known (they HAVE to be on-device if you want to keep end-to-end encryption), criminals could just switch to different phrases (or even write bots to flood the report system)! Even if detection works well, as it only works on text, it'll be trivial to bypass: Ask the kid to join a discord call, send voice messages, send images containing text etc. etc.

To summarize the technical aspects: It's possible to detect known CSAM, but that's about all you can reliably detect. When scanning for unknown CSAM even a low error rate of a few percent will lead to billions of messages being falsely reported EVERY DAY. Grooming detection is at best a protoype and relies on human moderation. Not only can detection be bypassed, it'll also be possible for criminials to deliberately cause false positives to disrupt investigations.

The swiss police receives automated reports and states that in about 87% of cases, they are useless [C3] - what will the police do if this number increases to 99%?

Also, again, scanning messages won't help if criminals encrypt their files, upload them to a hosting service and share links, as they are already doing [C1,4-6]. But even assuming the police had a magical technology that has a 100% accuracy for all reports, would they care about reducing distribution of CSAM?