|
|
Subscribe / Log in / New account

"Full disclosure" from the University of Minnesota

The researchers at the University of Minnesota have posted a description of the work they did [PDF] as part of their "hypocrite commits" project. It includes a list of the buggy commits they posted and how they were handled.
In the following we will show two parts: (1) the message log of our disclosure of the findings to the community, and (2) the patches we submitted. By showing the details of the patches and the exchange of messages, we wish to help the community to confirm that the buggy patches were "stopped" during message exchanges and not merged into the actual Linux code. No other interactions with the Linux Kernel team has involved intentional deception or intentionally misleading or bad patches. This misguided behavior on our part was limited to the patches described and clarified in this document.

Amusingly, one of their attempts to submit a buggy commit was, itself, buggy, yielding a valid change overall.


to post comments

"Full disclosure" from the University of Minnesota

Posted Apr 28, 2021 17:44 UTC (Wed) by wtarreau (subscriber, #51152) [Link] (6 responses)

It's nice they've finally published everything, but quite frankly, all that for that...

I mean, they draw conclusions such as "it's easy to inject bugs" when all their attempts failed, and they base their reasoning on "one patch that was accepted could have brought a bug if it had been different" (but did not) and "one patch was rejected but the reviewer didn't comment on a second issue that was hidden in it". Do they want that people comment on all their mistakes to justify a patch rejection ?

Finally "have you thought about wasting even more time using a myriad of tools and complex processes to improve your faulty security?" is laughable at best since it basically demonstrates they've never been involved in reviewing code for distributed projects, that they have no idea what they're speaking about, yet they have good advices to share!

Thank you very much for knocking on the door but we don't need your laundry detergent, we get the same result by washing by hand, and in less time, so we eliminate more crap at the end of the day.

"Full disclosure" from the University of Minnesota

Posted Apr 28, 2021 22:47 UTC (Wed) by amacater (subscriber, #790) [Link] (3 responses)

It's as if, for any given situation, there exists an xkcd that is apposite just as if Randall Monroe were monitoring - https://xkcd.com/2456/

XKCD 2456

Posted Apr 29, 2021 9:32 UTC (Thu) by pr1268 (subscriber, #24648) [Link] (2 responses)

And if I understand how Monroe set up his web page, that's the latest (as of 29 April 2021) XKCD. How convenient (or even prescient)...

And, just curious,

WE SCANNED SOME UNDERGRADUATES

I don't get that, but perhaps that's the point. 😕

XKCD 2456

Posted Apr 29, 2021 11:38 UTC (Thu) by rschroev (subscriber, #4164) [Link]

> I don't get that, but perhaps that's the point

An explanation can be found, as always, on the explain xkcd wiki: https://www.explainxkcd.com/wiki/index.php/2456:_Types_of...

> We scanned some undergraduates:

>> Some initial research, especially that on a low budget, may recruit students at the same institution as easily available test-subjects. Quite often these are psychological or sociological studies, but can involve more medical (but non-invasive) 'scans', from simple eyeball-tracking to full-body MRI. When misread as "scammed", this paper can also refer to numerous famous psychological studies done before the establishment of certain ethical rules, such as the Milgram experiment.

XKCD 2456

Posted May 5, 2021 10:09 UTC (Wed) by ceplm (subscriber, #41334) [Link]

>> WE SCANNED SOME UNDERGRADUATES
>
> I don't get that, but perhaps that's the point.

The most affordable testing human material for the university researchers are sophomores, so they are by far the most common test subjects of most human research studies. There are many researchers in social and experimental psychology (where I have some knowledge of the field) acutely aware of how many papers are distorted by most of the research done on not-completely-mature and in-many-other-ways special test subjects.

"Full disclosure" from the University of Minnesota

Posted Apr 29, 2021 17:28 UTC (Thu) by HenrikH (subscriber, #31152) [Link]

Quite clear that they come to, and wrote, their conclusion before they tested their hypothesis. And now they are trying to verify their original conclusions by changing the goal posts (they did not reject the patch due to the real problem and so on).

"Full disclosure" from the University of Minnesota

Posted Apr 30, 2021 10:28 UTC (Fri) by Uqbar (guest, #121169) [Link]

I hope the kernel team will keep the permanent ban. For two reasons.

1. Software development has rules, just like democracy.
They tried to break the rules. And they did that with one of the most "important" and valuable projects with clear intention to do harm.

2. The kernel team doesn't work for you, the submitter. And not even the other way around (I think). You work for the community with code and documentation contributions. If there's no code and documentation contribution there's little room for you.

"Full disclosure" from the University of Minnesota

Posted Apr 28, 2021 20:59 UTC (Wed) by gtb (guest, #3978) [Link] (2 responses)

I am curious: How does their ineffective nonsense-patch from early April 2021, which caused Greg-Kroah Hartman to ban all UMN patches, fit into the picture? It is not mentioned in this allegedly-full disclosure.

April patches

Posted Apr 28, 2021 21:14 UTC (Wed) by corbet (editor, #1) [Link]

That patch is not a part of this picture; it was an independent event. Stay tuned for an update tomorrow...:)

"Full disclosure" from the University of Minnesota

Posted Apr 28, 2021 21:16 UTC (Wed) by mokki (subscriber, #33200) [Link]

Patches from the original research paper have been confused with the new tool that the authors used to generate different set of bad patches. The new tool is not mentioned in any research paper. It might be part of some new research, but there is no information about it anywhere.
The current practice of clearly marking automatically generated patches as such was not properly followed, nor were the patches first properly reviewed by the authors before bothering others with them.

"Full disclosure" from the University of Minnesota

Posted Apr 29, 2021 8:45 UTC (Thu) by error27 (subscriber, #8346) [Link] (6 responses)

Ugh... This is hilarious for how badly it's done. They tried to introduce four bugs and failed all four times. Two times they failed because they didn't understand the code and made harmless changes #1, and #3. #2 and #4 were caught during review.

The accidentally submitted a "valid" patch #5, but that was ignored and it was buggy. It is a double free.

I was the reviewer for #3. They next printk is fine. It's not a use after free as they claim. Their patch did fix a resource leak but I asked them to fix a more serious use after free and clean up the code.cause it is a double free.

"Full disclosure" from the University of Minnesota

Posted Apr 29, 2021 11:23 UTC (Thu) by ledow (guest, #11753) [Link] (5 responses)

My entire objection would focus not on how successful they were, but on the fact that they were deliberately trying to put bugs into production code (security or not).

They could have done anything from crashing systems to trashing filesystems to corrupting data to damaging hardware, and they were TRYING to put those bugs in, it's not like it was an unfortunate accident.

I have to agree with GregKH that this can't be tolerated.

If they want to do this, co-operate with higher-level maintainers to make sure they know exactly what patches are trying to do this, so they can make sure they don't hit production kernels, and they are then just a test of how far things can get through the process while having such bugs.

"Full disclosure" from the University of Minnesota

Posted Apr 29, 2021 12:47 UTC (Thu) by mpg (subscriber, #70797) [Link] (4 responses)

> they were deliberately trying to put bugs into production code (security or not).

I'm surprised to read that. My understanding is that they only tried to see if the bugs would be accepted, but then withdrew each bug before it was merged. That is, they never had any intention of putting bugs in production code. If you have any evidence to the contrary, could you please quote a precise source for that?

"Full disclosure" from the University of Minnesota

Posted Apr 29, 2021 14:04 UTC (Thu) by nedu (guest, #50951) [Link] (2 responses)

A precise source for "any evidence to the contrary". Emphasis on "any":

It is my understanding that on or about Nov 21, 2020, Kangjie Lu posted a tweet announcing acceptance of the paper for the upcoming IEEE S&P symposium. In that tweet, he published a screenshot of the first page of that paper.[1][2]

It is also my understanding that on or before Nov 22, 2020, Kangjie Lu deleted that tweet, with a statement that, "it created many confusions".[3]

I have seen what has been represented to me as a copy of that screenshot, in which an abstract of the paper is reproduced.[4]

In that version of the abstract, I read the sentence, "As a proof of concept, we successfully introduce multiple exploitable use-after-free in the latest linux kernel (in a safe way)."[2]

Do also note, that besides the withdrawal of the tweet, that original sentence in the abstract was revised in a currently available version of the now-withdrawn paper.[5][6]

[1] https://pbs.twimg.com/media/End-XVOUUAA7gZ-?format=png&...
[2] https://pbs.twimg.com/media/End-YlwVoAADi1r?format=png&...
[2(a)] https://web.archive.org/web/20210429134044/https://pbs.tw...
[3] https://archive.is/SBpHn
[4] https://archive.is/iUdX0
[5] https://web.archive.org/web/20210427031705/https://raw.gi...
[6] https://web.archive.org/web/20210427062206/https://www-us...

In discussion: again, you asked for "any evidence to the contrary".

And, I would concede that the original statement in the accepted abstract may have been puffery. It may have been confusion. It may have required further explanation.

But I also have to think "the latest linux kernel" may easily and naturally be read as referring to software running on other people's computers. And I can read that sentence as a cold, factual claim of an accomplished deed.

"Full disclosure" from the University of Minnesota

Posted Apr 29, 2021 22:28 UTC (Thu) by mpg (subscriber, #70797) [Link] (1 responses)

Thank you for providing precise references. Looking at it in chronological context, I think it helps me understand some the reactions better. The first things I read about the topic were the paper and the LWN article, and from the paper I quickly learned that they tried to make sure the commits didn't get into the kernel. (Also, a few days later they disclosed the full details, so I didn't have to go for months wondering about what really happened.) So I probably got a very different experience than people reading scary/unclear/contraditory* fragments on twitter and then having to wait a long time for the details. Thank you for helping me see that perspective.

*I'll note that if we scroll down a bit on [3] they already tried to clarify that the malicious commits never got into the kernel, and in [4] the commenter already notes the contradiction.

Still, considering the full set of data we have today, I don't think it's justified to state that "they were deliberately trying to put bugs into production code", as the recent comment I was replying to did.

They did several things they shouldn't have done, such as experimenting on human subjects without their consent, making a paper with IMO weak methodology and relatively poor execution (I think we can hardly draw useful conclusions from their experiment) and communicating poorly about it (which again, would have been less of a problem if they had sought informed consent beforehand), but I don't think "deliberately trying to put bugs into production code" was one of them.

"Full disclosure" from the University of Minnesota

Posted Apr 30, 2021 21:56 UTC (Fri) by rgmoore (✭ supporter ✭, #75) [Link]

I quickly learned that they tried to make sure the commits didn't get into the kernel.

Not exactly. They say they never intended for their bugs to make it into a released kernel, but they said that only after the whole thing blew up and they were in damage control mode. We don't know what their true intent was. Maybe they intended to stop the bugs from ever being released. Maybe they thought it would be good to get one into a released version and then stealthily patch it in the next version, and that would be safe because it would be fixed by the time they told the world what they had done. We simply can't know what they would have done in the hypothetical world in which one of their patches had made it into the mainline kernel. We have only their word for it, and honestly their word isn't very good with a lot of people right now.

"Full disclosure" from the University of Minnesota

Posted Apr 29, 2021 17:35 UTC (Thu) by HenrikH (subscriber, #31152) [Link]

Well that is impossible to know since all their tries where caught and rejected so we will never know how they would have reacted if one of their patches would have been accepted. Withdrawing the patch would mean that they would have exposed their intentions with the first bug and where they really prepared to do that?


Copyright © 2021, Eklektix, Inc.
Comments and public postings are copyrighted by their creators.
Linux is a registered trademark of Linus Torvalds