kernelmethod/notes/Recent content on kernelmethoden-usWed, 27 Nov 2024 00:00:00 +0000Mapping all 13.8 gigapixels of the Caves of Qud world map/notes/qud_worldmap/Wed, 27 Nov 2024 00:00:00 +0000/notes/qud_worldmap/<p><a href="https://www.cavesofqud.com">Caves of Qud</a> is a science-fantasy roguelike coming out on December 5th, 2024. In roguelike tradition, the majority of its maps are procedurally generated.</p> <p>The world of Qud is 240 zones (&ldquo;screens&rdquo;) wide and 75 zones tall. Each zone is 25 tiles tall and 80 tiles wide, and each tile is 16 pixels wide and 24 pixels tall. That means that the surface map from a single run of Qud comes in at a cozy <strong>13.8 gigapixels</strong>.</p> <p>And here&rsquo;s what that map looks like in practice! Zoom in to see any part of the surface map from a game of Qud. (If you&rsquo;re on mobile you may have to zoom in quite a bit first. Sorry, I&rsquo;m awful at frontend dev!)</p> <div id="map"></div> <p>The game also has <code>Int32.MaxValue</code> z-levels. For fun I&rsquo;ve included a toggle to show the first underground level, although for the sake of brevity (and cloud storage costs) I must omit the other 2,147,483,645 z-levels.</p> <p>If you want to follow along and generate a map for your own world, I&rsquo;ve made all of my scripts (and a little proof-of-concept) available at <a href="https://github.com/kernelmethod/Qud-WorldMap-Viewer">Qud-Worldmap-Viewer</a>.</p> <p>Finally, if any of this looks interesting to you, check out Caves of Qud when it comes out next week! You can find it on Itch and Steam:</p> <ul> <li><a href="https://freeholdgames.itch.io/cavesofqud">Itch</a></li> <li><a href="https://store.steampowered.com/app/333640/Caves_of_Qud/">Steam</a></li> </ul> <link rel="stylesheet" href="/vendor/leaflet/leaflet.min.a11fc0238f51ad2f4c8c2628f5c1150976eb07abe9a48a4ae2d54dda1d4c14bc.css" integrity="sha256-oR/AI49RrS9MjCYo9cEVCXbrB6vppIpK4tVN2h1MFLw="> <script src="/vendor/leaflet/leaflet.min.ca2d2cc237347254967d77034a4fafa14601c983c13c272eab54512f1d70f6cd.js" integrity="sha256-yi0swjc0clSWfXcDSk&#43;voUYByYPBPCcuq1RRLx1w9s0="> </script> <link rel="stylesheet" href="/css/qud_worldmap.min.8ddb956c4fc7c455b6c6cb16427caa5f0be56cc211c517218767174d7f35fd6d.css" integrity="sha256-jduVbE/HxFW2xssWQnyqXwvlbMIRxRchh2cXTX81/W0="> <script type="application/javascript" src="/js/qud_worldmap.js" integrity=""> </script> Short story: "Paquete Drop"/notes/paquete_drop/Sun, 16 Jun 2024 15:13:36 -0400/notes/paquete_drop/<p>The following is a short story I submitted for DEF CON 32&rsquo;s Creative Writing Short Story Contest.</p> <p><strong>Update 2024-07-10:</strong> this story <a href="https://forum.defcon.org/node/250286">won first place</a> in the Creative Writing Contest! Major thanks to the contest organizers for running it this year, and for the judges who took the time to read all of the stories that were submitted.</p> <hr> <div class="short-story"> <h1>Paquete Drop</h1> <p>DEF CON 323 had gone well so far. Modulo one incident in which a young pod jockey&rsquo;s &ldquo;prank&rdquo; nearly asphyxiated everyone in the Ethical Cloning Village, the conference was running smoothly.</p> <p>Smoothly for everyone except Norodi. She had a problem: as a representative for the asteroid 65 Cybele, and as an operator for PaqNet, she was expected to pursue her home&rsquo;s best interests at the conference. She was expected to connect with other operators to help put the precarious future of the Cybelian internet on the right path. So far, she had chosen to honor this responsibility by running around the villages and getting trashed every night.</p> <p>But it was the last day of DEF CON, and Norodi was determined to make something out of her trip. So she headed over to the hall hosting the PaqNet Operators&rsquo; Meetup. On the way to the meetup, Norodi bumped into Kacy, a friend and fellow operator hailing from Enceladus. She nudged Norodi. &ldquo;Hey you. Where have you been for the last few days?&rdquo;</p> <p>Norodi was too hung over to figure out a convincing cover story, so she simply responded with &ldquo;I&rsquo;ve been wasting my time doing everything except what I was supposed to do.&rdquo;</p> <p>Kacy laughed. &ldquo;Then let&rsquo;s fix that, shall we?&rdquo;</p> <p>The year&rsquo;s meetup began, as it always did, with a summary of the network&rsquo;s accomplishments over the previous year. Thirty-two new nodes had joined the network, bringing the total number of nodes to one hundred and sixteen. Most nodes were averaging fifty petabits of data exchanged with their peers per day, and the longest path in the network &ndash; the largest amount of time it could take an update from any space platform&rsquo;s, lunar colony&rsquo;s, or asteroid outpost&rsquo;s intranet to propagate across PaqNet &ndash; had been reduced to five days. This last point drew a polite round of applause from the audience.</p> <p>Kacy and Norodi spent the day bouncing between talks. One talk discussed operations research for paquete optimization. Another covered paquete security, basic opsec for node technicians, and secure tape encryption. The day ended with a keynote from Kelli Adachi, one of the original founders of PaqNet. After a long introduction listing Kelli&rsquo;s achievements and contributions to the PaqNet community, she stepped up to the podium and began speaking.</p> <p>&ldquo;It&rsquo;s always been difficult to get internet half a billion miles from Earth. Before PaqNet, the internet that you <em>did</em> get was also shit.</p> <p>&ldquo;We started PaqNet as an experiment between Europa and Callisto, before we gained our independence. In those days, the only internet you got was out of a VOCorp uplink, and it was awful. The standard uplink costed hundreds of thousands of credits, it could only be fixed up by an approved technician, and the corporation had a view into every single packet sent into the ether. And at first, there was nothing we could do. The only group with the infrastructure to stand up an interplanetary internet was VOCorp.</p> <p>&ldquo;Thanks to all of your contributions, PaqNet has become the de facto means of attaining internet for billions of people across the Solar System. More than that, it has liberated &ndash;&rdquo;</p> <p>A conference goon sped into the room and pulled Kelli aside, followed by some of the meetups&rsquo; organizers. They spoke in hushed tones to one another, and Kelli&rsquo;s expression morphed from her usual confidence to shock, followed by dismay. Norodi glanced over at Kacy, who gave her an uncertain shrug in response.</p> <p>A few minutes passed before Kelli finally took the podium again.</p> <p>&ldquo;I&hellip; I&rsquo;m sorry for the interruption, everyone. This is a terrible time to deliver this news, but you should know as well: we just found out that VOCorp has purchased Earth&rsquo;s PaqNet node.&rdquo;</p> <hr> <p>The Earth node had faced financial difficulties for a while. Compared to other nodes, paquetes sent from Earth face more atmospheric drag on the ascent and higher gravity, which meant greater fuel expenses. The node served one billion people on the surface of the planet, and &ndash; crucially &ndash; processed more traffic than every other node combined.</p> <p>All of these factors made the node a vulnerable target, and a valuable one.</p> <p><em>We should have seen this coming,</em> Kacy thought. <em>We should have; perhaps we did. Either way, we did nothing about it.</em></p> <p>At the end of DEF CON, Kacy said her goodbyes to Norodi. &ldquo;Before you go,&rdquo; Kacy said, &ldquo;take this. It&rsquo;s got my long-term public key on it.&rdquo;</p> <p>She lifted a thumb drive out of her pocket and pressed it into Norodi&rsquo;s palm. The two friends hugged, and parted ways.</p> <p>If VOCorp had control over the Earth node, then it could see whatever traffic was passing through the node. Even if that traffic was encrypted, they would have means of telling where data was being sent to and from. They could identify the Enceladus node. They could identify Kacy. She had to get back before that happened.</p> <p>She searched for the fastest ride she could get back to Enceladus. There weren&rsquo;t any civilian shuttles making their way over to Saturn any time soon, and even if there were, it was too dangerous. Kacy couldn&rsquo;t risk going through customs on Enceladus; she wasn&rsquo;t supposed to be off-planet in the first place, especially for an underground hacker conference. Instead, Kacy hitched a ride from Europa, and from there she rode a paquete the rest of the way to Enceladus.</p> <p>Most paquetes are purpose-built space vessels, designed to carry crates with dozens of petabytes&rsquo; worth of magnetic tapes from Point A to Point B. The paquete that Kacy had boarded had been expanded to include a tiny bathroom, a cupboard with a day&rsquo;s worth of food, and bunks to accommodate four passengers; spartan accommodations for those who had no other means of travel. After a seemingly interminable flight cramped into a small box, the paquete started to shudder, and klaxons blared above Kacy&rsquo;s head. They were beginning to decelerate as they closed in on Enceladus.</p> <p>Kacy held tight to the railings of her bunk, preparing just in case the paquete exploded on its descent. PaqNet was, nominally, completely legal &ndash; and indeed, it operated freely and openly on the inner planets. But the charter that granted VOCorp its mandate over the outer Solar System allowed it to control all communications coming into or out of its stations. Whereas a VOCorp uplink was allowed to scan client traffic and report people for &ldquo;subversion of VOCorp or VOCorp-affiliated entities&rdquo; (in the words of the standard worker&rsquo;s contract), PaqNet traffic was encrypted between nodes and allowed to carry whatever content users felt like. So VOCorp considered it a threat, and on the occasion that they were able to identify a paquete entering of their stations, they shot it down.</p> <p>Kacy hoped that this wasn&rsquo;t one of those occasions.</p> <hr> <p>Norodi arrived at 65 Cybele and rode the tram back into the inner city. The tram had a good view overlooking the asteroid, and as it headed into town Norodi could still pick out the signs saying &ldquo;WELCOME TO VOCORP - 65 CYBELE STATION&rdquo;. Most of these signs had been torn down by the workers&rsquo; union years ago, when VOCorp was forced to abandon its administrative hold over the asteroid. 65 Cybele was lucky; many stations were still under corporate rule.</p> <p>Like Enceladus Station. Her mind flitted towards Kacy; Norodi hoped that she had returned home safely.</p> <p>Norodi hopped off the tram at the second-to-last stop. She crossed two rows of taco stands and walked through a small arts and crafts market. At the last block before her apartment, she passed by Cleo&rsquo;s Scavenge and Repair. The titular Cleo was sitting in his workshop, typing furiously at his terminal. In spite of his wild hair and disheveled clothing, Cleo kept a tidy space; all surfaces were cleaned regularly, all chips and wires and bits and bobs stored in carefully-labeled drawers. On this particular occasion, Cleo had an additional visitor in his pristine workshop: a large, black tube, scratched from top to bottom. Paint was peeling off to reveal discolored metals underneath, and two fins protruded from each side, each hoisting a shattered solar panel.</p> <p>Cleo perked up and flashed a big, goofy grin at Norodi as she passed by. &ldquo;Heya Norodi, how was DEF CON?&rdquo;</p> <p>She winced. &ldquo;It was alright, until the end. You heard about what happened?&rdquo;</p> <p>&ldquo;Yeah.&rdquo; He sighed. &ldquo;I still remember when this place was a corporate stronghold. We fought like hell, like hell. I gotta give it to those bastards, they still gotta lotta moxie to be keeping up with their schemes after so many years. I&rsquo;m glad there&rsquo;s kids like you to carry the torch and continue our side of the fight.&rdquo;</p> <p>Norodi shrugged. &ldquo;I don&rsquo;t think that running an underground sneakernet takes half as much courage as you think, but thanks.&rdquo; She decided to change the topic. &ldquo;What do you have back there?&rdquo;</p> <p>&ldquo;Ah, this old hunka scrap?&rdquo; he asked, jabbing a thumb towards the tube. &ldquo;That&rsquo;s an old Earth satellite. Used ta be that folks would shoot these up into orbit an use them as relays to send packets ta one another. Still a bunch of these floating &lsquo;round Earth today. Pretty ancient tech, eh?&rdquo;</p> <p>&ldquo;How&rsquo;d you get it?&rdquo;</p> <p>&ldquo;Fell off the back of a ship,&rdquo; Cleo said. He gave Norodi a sly wink.</p> <p>Norodi smirked. &ldquo;Looks like fun. You pulled anything valuable off it yet?&rdquo;</p> <p>&ldquo;Nah, it&rsquo;s a huge time-waste. Spent a week tryna crack it and when I did, I found out that most of the storage degraded years ago. Shame. Antennas still work though, as do mosta the internal components, I just gotta find someone ta pawn it off on.&rdquo; Cleo&rsquo;s eyes narrowed. &ldquo;You in the market for an antique satellite?&rdquo;</p> <p>Norodi laughed. &ldquo;I don&rsquo;t think so, but thanks. See you around, Cleo.&rdquo;</p> <p>He waved Norodi off and set back to work at his terminal. When Norodi returned to her apartment, the first thing she did was hop on her computer to see whether any messages had arrived from Kacy. She was relieved to see the message at the top of her inbox:</p> <p>&ldquo;&lt;kacy (Enceladus)&gt; Hey. I got back alright. Miss ya already -K&rdquo;</p> <hr> <p>Several weeks passed after DEF CON, and a problem was starting to form before Kacy. She didn&rsquo;t know quite what the problem was yet, although she had a general sense of its size and shape.</p> <p>Her problem started with the disappearance of a node technician.</p> <p>In a PaqNet router, each node technician is tasked with two responsibilities: pulling tapes off of paquetes, and putting them back on. Each paquete carries hundreds of tapes, and each tape carries hundreds of terabytes of PaqNet updates &ndash; large media files, encrypted chat messages, forum threads, news, books, posts &ndash; anything that a PaqNet user might feasibly wish to upload to the internet. When a paquete arrives at the Enceladus router, each tape is carried off of the paquete and loaded into a reader, where its contents are used to synchronize the state of the Enceladus intranet with the PaqNet at large. Once a node receives enough updates from its peers &ndash; and enough new content uploaded from its own intranet &ndash; it is responsible for passing along updates to other nodes. And so it goes, with the daily rhythm of a node tech converging towards a cycle of unloading tapes from paquetes, reading them in, loading new tapes onto paquetes, and sending them off.</p> <p>One evening, a tech left work and didn&rsquo;t return the following morning. Or the morning after.</p> <p>This alone was cause enough for concern, but there were other signs that something was amiss. Scheduled paquetes were failing to arrive, and the ones that did contained more corrupted data. Each tape that went through the Enceladus router was scanned for malicious content, but none of them ever raised any flags.</p> <p>Kacy was not yet aware that in a storage closet just a hundred meters outside of the router, locked in the darkness, a device had turned on. She was not yet aware that for the last several days it had been issuing its silent report to a VOCorp carrier ship that was now in transit to Enceladus. She was not yet aware of the danger that she was in.</p> <hr> <p>Probably the most irritating part of VOCorp&rsquo;s incursion into PaqNet, in Norodi&rsquo;s opinion, was their introduction of VOCorp Integrated Advertising (TM) into updates sent from Earth paquetes. Norodi spent an hour or two each day scouring instances of</p> <pre><code>BUY MARTIAN FOREIGN FILMS &gt;&gt;&gt;HERE&lt;&lt;&lt; </code></pre> <p>and</p> <pre><code>paq://hottiez.prn/xzBt8g1f FIND HOTTIES IN YOUR SPACEPORT </code></pre> <p>that had been injected into various websites. This was primarily a nuisance, but occasionally something more insidious would be slipped into an update. Embedded scripts designed to phone home to some shady VOCorp affiliate, or implants precision-targeted for specific individuals&rsquo; PaqNet terminals.</p> <p>If Norodi felt unfazed by these challenges, it was because she had experienced them before. VOCorp had spent many years engaging in subterfuge against the network, intercepting paquetes, and flipping operators. The only difference now was how much more brazen it had become.</p> <p>Norodi received a text from a tech in the 65 Cybele router that one of the tape readers was malfunctioning; the team wanted her expertise to diagnose the issue. As she left her apartment she passed Cleo once again &ndash; hard at work disassembling some inscrutable device, too busy even for a passing &ldquo;hello&rdquo; &ndash; and made her way down to the tram. Once she arrived at the router, she found the lead technician.</p> <p>&ldquo;Hey, I got a message that the reader isn&rsquo;t working. What happened?&rdquo;</p> <p>The tech gave her a shrug. &ldquo;We received a paquete this morning from 41 Daphne. The reader started making a funny noise while it was running through one of the tape spools, and then it stopped. I&rsquo;ve been busy all day and this seemed more like your wheelhouse, so we gave you a ring.&rdquo;</p> <p>Norodi jacked into the reader&rsquo;s serial port to inspect its logs. The firewall logs indicated a large number of packets blocked against several hundred devices connected to the router&rsquo;s local area network. A forward proxy contained records of several failures to authenticate against different devices on the network, and ended with one successful HTTP response. The device was named <code>vocup-27.rtinternal.cybele.pqn</code>.</p> <p>&ldquo;Hey, do you have any VOCorp communications equipment laying around?&rdquo; Norodi asked.</p> <p>The tech looked puzzled. &ldquo;We have an old uplink lying in storage. We used to use it for emergency broadcasts, but it&rsquo;s been broken for almost as long as I&rsquo;ve been here.&rdquo;</p> <p>Norodi felt her stomach starting to turn. She ejected the tape from the reader and saved off a dump of the system&rsquo;s memory to analyze in her homelab.</p> <p>But before she could go home, she had to ask Cleo about his satellites.</p> <hr> <p>The message from Norodi arrived late at night, encrypted and sealed with the key that Kacy had given her at DEF CON. Kacy was taking a big risk by checking her inbox, but this would be her last chance before she left Enceladus.</p> <p>&ldquo;&lt;norodi (65 Cybele)&gt; Do not respond. Do NOT respond.</p> <p>&ldquo;I&rsquo;ve identified a malware campaign that VOCorp appears to be running against PaqNet crews. VOCorp is exploiting a zero-day in the tape deserialization software to reach out to old equipment that operators still have installed in their routers and ping their command-and-control servers. The bottom line is this: they&rsquo;re targeting operators now, and you&rsquo;re in danger.</p> <p>&ldquo;65 Cybele got lucky &ndash; all of our VOCorp equipment is broken beyond repair, and the implant crashed while trying to inject itself into a defective VOCorp uplink. You probably won&rsquo;t be so fortunate. When you finish reading this, find and destroy any VOCorp equipment that may still be attached to your network. Do not wait for them to knock on your door.</p> <p>&ldquo;The current situation is untenable, but I have a plan. Meet me on Luna in one week.&rdquo;</p> <p>Oh Norodi. If only her message had come just a bit sooner.</p> <p>Kacy hadn&rsquo;t known about the zero-day, but she&rsquo;d intuited the rest of the details after VOCorp had raided Enceladus&rsquo;s router three days earlier. All of the technicians who were present at the time were arrested, and then forced to keep the router operational. By now VOCorp would already have started injecting a beacon into every message sent via PaqNet into the Enceladan intranet, and in Kacy&rsquo;s estimation, they would roll up every remaining PaqNet operator who&rsquo;d fled by the end of the week.</p> <p>Including her. And now that Kacy had pulled data from her inbox into her local PaqNet terminal, they would know where to find her. Kacy threw the terminal into the nearest garbage disposer and broke into a sprint.</p> <p>Kacy couldn&rsquo;t ride a paquete out of Enceladus with the router compromised. She couldn&rsquo;t board a passenger shuttle either; even in better times, they were under strict watch by corporate authorities. Almost all cargo shipments out of the moon had been shut down. That left Kacy with exactly one option.</p> <p>She would need to steal a VOCorp ship.</p> <p>Luckily, VOCorp had disavowed the classic analog interface preferred by most pilots for a purely digital flying experience. This made VOCorp ships easy to fly &ndash; and easy to hack. Kacy had phished the employee credentials from several corporate security officers over the last couple of days. She found one that matched the profile that she sought: a mid-career pilot with permissions to enter and exit the Enceladan space port at will, and to fly any unreserved ship with a Type B or lower rating.</p> <p>Kacy stopped running as she closed in on the spaceport. She walked over to a door leading in to the ship lot. The door was equipped with a badge scanner, but Kacy had come prepared. She took a blank radio-frequency card out of her backpack, wrote the pilot&rsquo;s credentials to it, and swiped the card against the scanner.</p> <p>The scanner emitted a sharp <em>bzzzt</em> and flashed a red light at her. Bad identification. Kacy threw away the card and tried a different one. If the scanner didn&rsquo;t accept her credentials, then her escape would be very short-lived. She pressed the second card against the scanner.</p> <p>This time, it gave her a happy chirp and flashed green. Kacy passed through the door and headed towards the closest ship that she could find, a small passenger shuttle typically used to carry ambassadors, dignitaries, and other people of note. She entered the pilot&rsquo;s username and password, and the ship&rsquo;s dashboard came to life. Kacy searched the ship&rsquo;s directory for its list of prerecorded destinations, and found the one she was looking for: Luna, Earth&rsquo;s moon. She set the ship on autopilot and buckled into her seat.</p> <p>She felt the acceleration push down against her when the ship&rsquo;s thrusters ignited. As the ship escaped Enceladus&rsquo;s gravity she felt the pressure against her lighten. She would make it.</p> <hr> <p>Norodi refitted a paquete with seat, a dashboard, and a pilot&rsquo;s yoke to take her from 65 Cybele to Luna, where she reconnected with Kacy. After three stuffy paquete rides, four repair stations visited in search of parts, one count of sneaking past lunar customs officials, and six trips from the moon to Earth&rsquo;s orbit and back, she was on her seventh and final trip. Norodi prayed in full view of the heavens that she wouldn&rsquo;t be picked up by an wandering VOCorp scout before finishing her job.</p> <p>&ldquo;Norodi, I just got my last satellite working over India,&rdquo; said Kacy over her microphone. &ldquo;I&rsquo;m finishing some repairs over here, and then I&rsquo;m heading back. How are things looking on your end?&rdquo;</p> <p>&ldquo;Not great,&rdquo; Norodi said, being honest. &ldquo;The last two satellites I tried were totally busted.&rdquo;</p> <p>&ldquo;Keep at it. We&rsquo;re at ninety-nine percent coverage.&rdquo;</p> <p>Cleo had given them a set of instructions to follow. First, they should run basic diagnostics to determine whether the satellite was functional, or whether the circuity had degraded too far from radiation. If it was still operational, then they would connect to its serial line and run Cleo&rsquo;s exploit. Assuming the exploit ran successfully, the satellite would connect to their makeshift satellite network. Norodi was waiting for the second step to complete, idly watching the spinner on her display as the exploit tried to gain remote code execution on the satellite that she was connected to.</p> <p>For his part, Cleo was finishing setting up a router on the far side of the moon. He&rsquo;d spent the previous days getting in touch with PaqNet operators on the ground to prepare them for the final stage of the plan. A plan that, so far, had run smoothly. Kacy had succeeded in evading the authorities on Enceladus to reach Norodi and Cleo. They had succeeded in completing six trips in and out of Earth&rsquo;s orbit. Everything was fine. Everything apart from these damned old, broken satellites.</p> <p>Cleo had told Norodi and Kacy that because the satellites would primarily be relaying data back and forth from the moon, only a subset of them would actually need to work. Which was fortunate, as Norodi was already at the limit of her patience. If the exploit failed on the satellite that she was working on now, she thought she might start hitting it with a wrench until it worked.</p> <p>Kacy&rsquo;s voice crackled over the mic: &ldquo;I&rsquo;m back on my ship, and heading back towards the spaceport.&rdquo;</p> <p>Norodi said nothing. She was too focused now. <em>This is it. It&rsquo;ll work this time,</em> she thought to herself.</p> <p>Finally, text splashed across her display:</p> <pre><code>EXPLOIT SUCCESSFUL CONNECTING TO NETWORK... ACQUIRED ID SAT-0277 CONNECTING TO ROUTER... CONNECTED TO RT-DARK_SIDE_OF_THE_MOON AWAITING UPLINK </code></pre> <p>Norodi disconnected and headed back towards her ship.</p> <hr> <p>Before Kacy could drive her moon rover back to the central spaceport, she would have to scuttle the ship she&rsquo;d stolen. She couldn&rsquo;t let anyone find out that she had been here, much less what she was up to. She identified a quiet spot in the Pacific Ocean where the ship could crash unnoticed and set the ship on autopilot.</p> <p>In a few minutes their work would be done. In a few minutes, several hundred satellites would wake up from a century-long slumber and start relaying data to Cleo&rsquo;s surreptitious router. And the router would send its first paquete out to the rest of PaqNet. The planet &ndash; and everyone else &ndash; would be free from VOCorp&rsquo;s stranglehold.</p> <p>First, Kacy wanted to get a drink.</p> <p>She parked the rover outside of a service airlock. Anybody who was monitoring entry into the station would see her spacesuit &ndash; stolen when she and Norodi and Cleo had first landed on the moon &ndash; and assume that she had just gotten back from a maintenance run. That was her hope, in any case. Once she got through the airlock, she slipped into the spaceport and headed over to one of the bars.</p> <p>Kacy sat down and ordered a couple of martinis for herself and for Norodi and started to reflect on the past couple of days. Where would she go next? She couldn&rsquo;t return to Enceladus; by now she was a wanted fugitive. Norodi had told her that she would always have a home on Cybele 65. That seemed like a nice idea.</p> <p>The martinis arrived, and Kacy started to take a sip, to relax in her seat, when all of a sudden some instinct in the corner of her brain noticed <em>it</em>. The uncomfortable stillness in the air. The empty chairs and tables that surrounded her. The nervous bartender, peering in her direction. And then she felt a polite tap on her shoulder.</p> <p>&ldquo;Hello miss. Do you have a few minutes to chat with us?&rdquo;</p> <p>She turned around to find a couple of well-groomed men in expensive suits standing behind her. One of them folded his arms, and Kacy saw the briefest flash of a pistol from inside his sports coat.</p> <p>She pushed past both of them and ran. She ran back towards the airlock. She&rsquo;d warn Norodi. She&rsquo;d find another way off the moon. Another way to 65 Cybele. She just had to escape those men.</p> <p>The last thing Kacy remembered was the butt of a rifle slamming into her chin.</p> <hr> <p>Back on her ship, Norodi tapped her mic. &ldquo;Cleo, how are you doing?&rdquo;</p> <p>&ldquo;Mighty well, seems like,&rdquo; Cleo said. &ldquo;Me an&rsquo; my team just got the connection from yer last satellite. We &lsquo;ave one hundred percent coverage over the planet now.&rdquo;</p> <p>&ldquo;Excellent,&rdquo; Norodi said, relieved. She was sick of the spacewalks. &ldquo;I&rsquo;ll see you soon.&rdquo; Norodi switched channels over to Kacy. &ldquo;I&rsquo;m finished over here, I&rsquo;m headed back to Luna. Let me know when you&rsquo;ve landed.&rdquo;</p> <p>She aligned the ship&rsquo;s trajectory with the moon and engaged her thrusters. As she approached the moon, two dots appeared on the horizon. Ships emblazoned with the VOCorp logo. One of the ships automatically connected itself to the ship&rsquo;s intercom and its mechanical voice came in over the ship&rsquo;s speakers. &ldquo;NORODI OLERUD OF 65 CYBELE &ndash; YOU ARE UNDER ARREST. STOP IMMEDIATELY AND PREPARE TO BE BOARDED.&rdquo;</p> <p><em>This can&rsquo;t be how it ends. This can&rsquo;t. It can&rsquo;t.</em> Norodi spiraled into a panic. She had everything on the ship&rsquo;s computers &ndash; Cleo&rsquo;s exploit, an inventory of all of the satellites they had gained access to. Everything. She couldn&rsquo;t let them take the ship.</p> <p>She floored it.</p> <p>The moon came into full view. Klaxons blared over Norodi&rsquo;s head. <em>You&rsquo;re going to die</em>, they told her. She kept accelerating anyways. A missile sailed past and rocked the ship. &ldquo;CEASE IMMEDIATELY&rdquo;, the voice boomed. She pushed the pedal even harder. She started to see Luna&rsquo;s texture. Its ridges. Individual rocks.</p> <p>She ejected.</p> <p>The world spun around Norodi. She didn&rsquo;t hear the ship as it slammed into Luna, only saw a bright light, followed by a million million fragments of metal and plastic and rock scattering themselves into space. The world continued to spin; Earth, moon, and stars locked in some frenetic dance as her vision blurred. And she fell, fell for what seemed like a lifetime, pulled into the moon&rsquo;s gentle embrace until she was a thousand feet, a hundred feet, ten feet above the surface.</p> <p>Her chair engaged mini-thrusters and stopped the descent at the last moment. Norodi couldn&rsquo;t stand up, couldn&rsquo;t run. Two gruff pairs of hands pulled her out of her chair and hauled her into the back of a rover. They zip tied her and locked the doors, and then the rover started to move. The homogeneous landscape of the moon rolled underneath her, while the sparkling cloud of debris left by her ship receded into the horizon. Hours passed.</p> <p>The rover stopped, and the hands pulled Norodi out of her cage in the back of the rover and back onto the surface. She could see now that they had arrived at a nondescript building tucked into the wall of a crater. The people who had seized her dragged her inside the building and ordered her to take off her suit. She did. Then they walked her down an aisle, past windows peering into dark rooms. At one point she passed a room with a bloody mess of a woman, slumped across the floor. It was Kacy.</p> <p>&ldquo;Kacy! Kacy!&rdquo; Norodi screamed. She screamed and screamed until all she could hear was ringing in her ears. Kacy didn&rsquo;t respond.</p> <hr> <p>Kacy awoke with the taste of copper in her mouth. She was in a concrete room, dimly lit, her hands and feet bound together. The two men that had approached Kacy at the bar entered the room, and the questions began.</p> <p>&ldquo;How did you find out about the implant?&rdquo;</p> <p>&ldquo;Who have you told?&rdquo;</p> <p>&ldquo;Why are you out here?&rdquo;</p> <p>&ldquo;Where were you running to?&rdquo;</p> <p>Neither of the men asked Kacy about Cleo or paquetes or satellites or a mysterious PaqNet router that had suddenly appeared on the other side of the moon. <em>They don&rsquo;t know,</em> Kacy thought to herself. She repeated it to herself over and over again, a mantra, or perhaps a prayer. <em>They don&rsquo;t know. They don&rsquo;t know. They don&rsquo;t know.</em></p> <p>The men beat Kacy for hours. She said nothing.</p> <p>Eventually they left her alone. Kacy&rsquo;s body was wracked with pain from head to toe. She watched idly as her own blood dripped onto the ground and dried. Eventually one of the men reentered the room, and knelt in front of her.</p> <p>&ldquo;We saw the message between you and your friend before you left Enceladus. We know you know about the implant.&rdquo;</p> <p>Kacy didn&rsquo;t respond. He still wasn&rsquo;t asking about satellites.</p> <p>&ldquo;It doesn&rsquo;t matter now,&rdquo; he said, a thin smile across his lips. &ldquo;Your friend notified everyone else before she left 65 Cybele. She sent them a sample of the implant, and details about the zero-day. Now some jackass has written some detections for the implant, and another jackass has written a patch for the tape deserialization software, and everybody&rsquo;s getting their detections set up and their devices patched. She did a good job. We&rsquo;re burnt.&rdquo;</p> <p>&ldquo;Serves you right,&rdquo; Kacy said.</p> <p>The man shrugged. &ldquo;That cost us a few million credits, but this company is worth tens of trillions. We&rsquo;re worth more than most governments.&rdquo; He leaned in. &ldquo;We&rsquo;ll manage.&rdquo;</p> <p>&ldquo;We&rsquo;ll stop you again. It doesn&rsquo;t matter what you throw at us. For every one of you there&rsquo;s five of us, working in the dark.&rdquo;</p> <p>&ldquo;You know, if it wasn&rsquo;t us, it&rsquo;d be someone else. Someone a lot nastier, a lot meaner. Somebody who wouldn&rsquo;t think twice about throwing you and your pal out of the airlock.&rdquo; The man stood up. &ldquo;But we&rsquo;re nice, and truthfully, there really isn&rsquo;t much you can do for us anymore. So you&rsquo;re coming back with us to Enceladus to face trial for unapproved abandonment of your assigned station, for the theft of a VOCorp ship, and for subversion of the state in the service an illegal interplanetary communications platform. And your friend is going back to her asteroid, where she will be barred from transiting through any VOCorp-owned spaceport for the rest of her life.&rdquo;</p> <p>He cut the zip ties around Kacy&rsquo;s hands and feet and opened the door to exit the room, gesturing towards her. Kacy picked herself up off the ground, and limped through the exit.</p> <p>It was a long ride back to the spaceport. Kacy sat in the back of the vehicle, unbound and alone. The guards that were driving hadn&rsquo;t even bothered to lock the back door. It&rsquo;s not like she could escape; without a suit, she would die in seconds if she exited the vehicle. And there was nowhere left for her to escape in any case.</p> <p>Kacy caught one last glimpse of Norodi at the spaceport. She was being escorted onto the boarding ramp of a shuttle going back to Enceladus, when a group of armed guards entered the loading bay. Norodi &ndash; as battered and bruised as she was &ndash; stood in between them. As Kacy ascended the ramp, she shot one last look at Norodi. Though beaten and pained, Norodi could make out a clear expression in her face, in the burning intensity with which Kacy was staring. Kacy knew that they&rsquo;d won.</p> <hr> <p>Norodi never spoke a word about the satellite network. When she was dropped off at 65 Cybele, she was informed that she would never again be allowed to step onto VOCorp property, but was otherwise free to go.</p> <p>Norodi rode the tram back to the inner city of 65 Cybele once more. She gazed out the window, across the rock, to the Cybelian PaqNet router. She saw a paquete fly out of the router, shimmering against the void, its thrusters faint lights as it accelerated towards its next stop. Norodi wondered at what the paquete was carrying: messages to loved ones separated by billions of miles of vacuum; missives from the lives of tens of thousands people living on a rock as it completed another journey around the sun.</p> <p>On this trip, the old VOCorp signs looked just a little more worn than before.</p> <p>Norodi got off at the second-to-last stop and passed the rows of taco stands and the market and turned at the last block before her apartment and hoped, hoped she would see the big, goofy grin that she had been waiting for. The garage door at Cleo&rsquo;s Scavenge and Repair was raised.</p> <p>Cleo was at his workbench, wearing his signature grin across his face. &ldquo;Heya, kid.&rdquo;</p> <p>Norodi felt tears start to well up. Thoughts like <em>did it work</em> and <em>I didn&rsquo;t tell them anything about you</em> and <em>I don&rsquo;t know if she&rsquo;s okay, and I miss her so much</em> all passed through her head. The only words that she could think to say, though, were &ldquo;I&rsquo;m so glad you made it out alright.&rdquo;</p> <p>&ldquo;Of course!&rdquo; Cleo laughed. &ldquo;I&rsquo;ve been runnin&rsquo; from those bastards almost as long as you&rsquo;ve been alive.&rdquo;</p> <p>Norodi wiped away the tears and smiled.</p> <p>Cleo pulled his terminal over to Norodi. The interface displayed a table with columns labeled &ldquo;DEVICE&rdquo;, &ldquo;STATUS&rdquo;, &ldquo;LAST PING&rdquo;, &ldquo;COORDINATES&rdquo;. &ldquo;I ran some tests, all of yer satellites are hooked up now. The two of you did well.&rdquo; He gave Norodi a bittersweet smile. &ldquo;My crew on Luna has their paquetes set up, and the operators on the ground are ready to start sendin&rsquo;.&rdquo;</p> <p>Norodi turned to Cleo, eyes bright. &ldquo;Then let&rsquo;s start receiving.&rdquo;</p> </div> <hr> <p><em><strong>Author&rsquo;s note:</strong> this is the longest &ndash; and for that matter, almost the only &ndash; work of fiction that I&rsquo;ve written in the last decade. Thanks to the folks at DEF CON for hosting this writing contest.</em></p> <p><em>The story here is very loosely inspired by <a href="https://en.wikipedia.org/wiki/El_Paquete_Semanal">El Paquete Semanal</a>, a sneakernet used to deliver internet in Cuba. If you&rsquo;re interested in learning more about El Paquete, I highly recommend <a href="https://dl.acm.org/doi/abs/10.1145/3173574.3174213">&ldquo;El Paquete Semanal: The Week’s Internet in Havana&rdquo; from ACM CHI 2018</a>.</em></p>A short note on setting up a Linux kernel debugging environment/notes/linux_kernel_debugging/Fri, 09 Feb 2024 00:00:00 +0000/notes/linux_kernel_debugging/<h2 id="a-short-note-on-setting-up-a-linux-kernel-debugging-environment">A short note on setting up a Linux kernel debugging environment</h2> <p>This is a relatively short post on how to set up a Linux environment for kernel debugging using <a href="https://buildroot.org/">Buildroot</a>.</p> <p>This note overlaps a lot with <a href="https://nickdesaulniers.github.io/blog/2018/10/24/booting-a-custom-linux-kernel-in-qemu-and-debugging-it-with-gdb/">this blog post by Nick Desaulniers</a>, and another <a href="https://www.josehu.com/technical/2021/01/02/linux-kernel-build-debug.html">post from Guanzhou Hu</a>. Relative to both of them I&rsquo;ve added a little more information around Buildroot configuration, but they are nonetheless good resources.</p> <h2 id="setup">Setup</h2> <p>To start off, you&rsquo;ll want to clone the Buildroot sources:</p> <div class="highlight"><pre tabindex="0" style="color:#f8f8f2;background-color:#272822;-moz-tab-size:4;-o-tab-size:4;tab-size:4;"><code class="language-bash" data-lang="bash"><span style="display:flex;"><span>git clone https://git.buildroot.net/buildroot </span></span></code></pre></div><p>You&rsquo;ll need to make an initial <code>.config</code> file for Buildroot. For the rest of this note, I&rsquo;m assuming that we&rsquo;ll want to build our kernel as x86-64 and run it under QEMU, so we&rsquo;ll use the following command:</p> <div class="highlight"><pre tabindex="0" style="color:#f8f8f2;background-color:#272822;-moz-tab-size:4;-o-tab-size:4;tab-size:4;"><code class="language-bash" data-lang="bash"><span style="display:flex;"><span>make qemu_x86_64_defconfig </span></span></code></pre></div><p>Now start the Buildroot TUI with</p> <div class="highlight"><pre tabindex="0" style="color:#f8f8f2;background-color:#272822;-moz-tab-size:4;-o-tab-size:4;tab-size:4;"><code class="language-bash" data-lang="bash"><span style="display:flex;"><span>make menuconfig </span></span></code></pre></div><p>If you&rsquo;ve built the Linux kernel from source before, the Buildroot UI should feel familiar. Instead of being presented with kernel build parameters to tweak, you&rsquo;ll have various options for how you want to configure the operating system, such as</p> <ul> <li>what version of the Linux kernel you want to use;</li> <li>what packages should be installed by default;</li> <li>whether login with the root user is permitted (and the root user&rsquo;s default password);</li> </ul> <p>and so on.</p> <p>Once you&rsquo;ve finished selecting the options that you want for Buildroot, you should save your config file (by default, to <code>.config</code>) and exit. For the purposes of this note, I used the following configuration options:</p> <div class="highlight"><pre tabindex="0" style="color:#f8f8f2;background-color:#272822;-moz-tab-size:4;-o-tab-size:4;tab-size:4;"><code class="language-bash" data-lang="bash"><span style="display:flex;"><span><span style="color:#75715e"># Some packages require C++ support in order to be installed</span> </span></span><span style="display:flex;"><span><span style="color:#75715e"># -&gt; Toolchain -&gt; Enable C++ support</span> </span></span><span style="display:flex;"><span>BR2_TOOLCHAIN_BUILDROOT_CXX<span style="color:#f92672">=</span>y </span></span><span style="display:flex;"><span> </span></span><span style="display:flex;"><span><span style="color:#75715e"># I like having OpenSSH installed by default as another means of getting access</span> </span></span><span style="display:flex;"><span><span style="color:#75715e"># to the environment, but it isn&#39;t strictly necessary.</span> </span></span><span style="display:flex;"><span><span style="color:#75715e"># -&gt; Target packages -&gt; Networking applications -&gt; openssh</span> </span></span><span style="display:flex;"><span>BR2_PACKAGE_OPENSSH<span style="color:#f92672">=</span>y </span></span><span style="display:flex;"><span>BR2_PACKAGE_OPENSSH_CLIENT<span style="color:#f92672">=</span>y </span></span><span style="display:flex;"><span>BR2_PACKAGE_OPENSSH_SERVER<span style="color:#f92672">=</span>y </span></span><span style="display:flex;"><span>BR2_PACKAGE_OPENSSH_KEY_UTILS<span style="color:#f92672">=</span>y </span></span><span style="display:flex;"><span>BR2_PACKAGE_OPENSSH_SANDBOX<span style="color:#f92672">=</span>y </span></span><span style="display:flex;"><span> </span></span><span style="display:flex;"><span><span style="color:#75715e"># Add a post-build script for some minor SSH configuration</span> </span></span><span style="display:flex;"><span><span style="color:#75715e"># -&gt; System configuration -&gt; Custom scripts to run before creating filesystem images</span> </span></span><span style="display:flex;"><span>BR2_ROOTFS_POST_BUILD_SCRIPT<span style="color:#f92672">=</span><span style="color:#e6db74">&#34;board/qemu/x86_64/post-build.sh ./scripts/config.sh&#34;</span> </span></span><span style="display:flex;"><span> </span></span><span style="display:flex;"><span><span style="color:#75715e"># Use Ext4 for the filesystem image</span> </span></span><span style="display:flex;"><span><span style="color:#75715e"># -&gt; Filesystem images -&gt; ext2/3/4 root filesystem</span> </span></span><span style="display:flex;"><span>BR2_TARGET_ROOTFS_EXT2_4<span style="color:#f92672">=</span>y </span></span><span style="display:flex;"><span>BR2_TARGET_ROOTFS_EXT2_GEN<span style="color:#f92672">=</span><span style="color:#ae81ff">4</span> </span></span></code></pre></div><p>The <code>./scripts/config.sh</code> shell script just contains the following:</p> <div class="highlight"><pre tabindex="0" style="color:#f8f8f2;background-color:#272822;-moz-tab-size:4;-o-tab-size:4;tab-size:4;"><code class="language-sh" data-lang="sh"><span style="display:flex;"><span><span style="color:#75715e">#!/bin/sh </span></span></span><span style="display:flex;"><span><span style="color:#75715e"></span> </span></span><span style="display:flex;"><span>PUBKEY<span style="color:#f92672">=</span><span style="color:#e6db74">&#34;...&#34;</span> <span style="color:#75715e"># Replace with your public key</span> </span></span><span style="display:flex;"><span>OUTDIR<span style="color:#f92672">=</span>output/target </span></span><span style="display:flex;"><span> </span></span><span style="display:flex;"><span>addline<span style="color:#f92672">()</span> <span style="color:#f92672">{</span> </span></span><span style="display:flex;"><span> grep -qxF <span style="color:#e6db74">&#34;</span>$1<span style="color:#e6db74">&#34;</span> <span style="color:#e6db74">&#34;</span>$2<span style="color:#e6db74">&#34;</span> <span style="color:#f92672">||</span> echo <span style="color:#e6db74">&#34;</span>$1<span style="color:#e6db74">&#34;</span> &gt;&gt; <span style="color:#e6db74">&#34;</span>$2<span style="color:#e6db74">&#34;</span> </span></span><span style="display:flex;"><span><span style="color:#f92672">}</span> </span></span><span style="display:flex;"><span> </span></span><span style="display:flex;"><span>mkdir -p <span style="color:#e6db74">${</span>OUTDIR<span style="color:#e6db74">}</span>/root/.ssh </span></span><span style="display:flex;"><span>addline <span style="color:#e6db74">&#34;PermitRootLogin yes&#34;</span> <span style="color:#e6db74">${</span>OUTDIR<span style="color:#e6db74">}</span>/etc/ssh/sshd_config </span></span><span style="display:flex;"><span>addline <span style="color:#e6db74">&#34;</span><span style="color:#e6db74">${</span>PUBKEY<span style="color:#e6db74">}</span><span style="color:#e6db74">&#34;</span> <span style="color:#e6db74">${</span>OUTDIR<span style="color:#e6db74">}</span>/root/.ssh/authorized_keys </span></span><span style="display:flex;"><span>chmod <span style="color:#ae81ff">600</span> <span style="color:#e6db74">${</span>OUTDIR<span style="color:#e6db74">}</span>/root/.ssh/authorized_keys </span></span></code></pre></div><p>Once you&rsquo;ve set up your configuration, you can start building the Linux environment with</p> <div class="highlight"><pre tabindex="0" style="color:#f8f8f2;background-color:#272822;-moz-tab-size:4;-o-tab-size:4;tab-size:4;"><code class="language-bash" data-lang="bash"><span style="display:flex;"><span>make -j<span style="color:#66d9ef">$(</span>nproc<span style="color:#66d9ef">)</span> </span></span></code></pre></div><h3 id="using-a-custom-kernel">Using a custom kernel</h3> <p>If you just want to build the kernel with custom configuration parameters, you can run</p> <div class="highlight"><pre tabindex="0" style="color:#f8f8f2;background-color:#272822;-moz-tab-size:4;-o-tab-size:4;tab-size:4;"><code class="language-bash" data-lang="bash"><span style="display:flex;"><span>make linux-menuconfig </span></span></code></pre></div><p>to create a kernel build config file. Then you can add the following to your Buildroot config:</p> <div class="highlight"><pre tabindex="0" style="color:#f8f8f2;background-color:#272822;-moz-tab-size:4;-o-tab-size:4;tab-size:4;"><code class="language-text" data-lang="text"><span style="display:flex;"><span>BR2_LINUX_KERNEL_USE_CUSTOM_CONFIG=y </span></span><span style="display:flex;"><span>BR2_LINUX_KERNEL_CUSTOM_CONFIG_FILE=./path/to/config/file </span></span></code></pre></div><p>If you&rsquo;re experimenting with some changes to the kernel sources, however, you&rsquo;ll need to do one of the following:</p> <ul> <li>Configure some <a href="https://buildroot.org/downloads/manual/manual.html#_adding_project_specific_patches_and_hashes">custom kernel patches</a> via <code>BR2_GLOBAL_PATCH_DIR</code> or <code>BR2_LINUX_KERNEL_PATCH</code>.</li> <li>Point to a tarball with the modified kernel source using</li> </ul> <div class="highlight"><pre tabindex="0" style="color:#f8f8f2;background-color:#272822;-moz-tab-size:4;-o-tab-size:4;tab-size:4;"><code class="language-text" data-lang="text"><span style="display:flex;"><span>BR2_LINUX_KERNEL_CUSTOM_TARBALL_LOCATION=&#34;file:///path/to/sources.tar.gz&#34; </span></span></code></pre></div><h3 id="booting-the-kernel-with-qemu">Booting the kernel with QEMU</h3> <p>If everything has gone smoothly up to this point, your newly-built Linux system should now be in the <code>output/images</code> directory (relative to the root of the Buildroot repository). You can now run your environment under QEMU with (for instance)</p> <div class="highlight"><pre tabindex="0" style="color:#f8f8f2;background-color:#272822;-moz-tab-size:4;-o-tab-size:4;tab-size:4;"><code class="language-bash" data-lang="bash"><span style="display:flex;"><span>qemu-system-x86_64 <span style="color:#ae81ff">\ </span></span></span><span style="display:flex;"><span><span style="color:#ae81ff"></span> -kernel ./output/images/bzImage <span style="color:#ae81ff">\ </span></span></span><span style="display:flex;"><span><span style="color:#ae81ff"></span> -drive file<span style="color:#f92672">=</span>./output/images/rootfs.ext4,if<span style="color:#f92672">=</span>virtio,format<span style="color:#f92672">=</span>raw <span style="color:#ae81ff">\ </span></span></span><span style="display:flex;"><span><span style="color:#ae81ff"></span> -nographic <span style="color:#ae81ff">\ </span></span></span><span style="display:flex;"><span><span style="color:#ae81ff"></span> -enable-kvm <span style="color:#ae81ff">\ </span></span></span><span style="display:flex;"><span><span style="color:#ae81ff"></span> -cpu host <span style="color:#ae81ff">\ </span></span></span><span style="display:flex;"><span><span style="color:#ae81ff"></span> -m 512M <span style="color:#ae81ff">\ </span></span></span><span style="display:flex;"><span><span style="color:#ae81ff"></span> -append <span style="color:#e6db74">&#34;root=/dev/vda console=tty1 console=ttyS0&#34;</span> </span></span></code></pre></div><p>You should see your system booting and reach the Buildroot login prompt (by default the only user you can login as is <code>root</code>, without any password):</p> <div class="highlight"><pre tabindex="0" style="color:#f8f8f2;background-color:#272822;-moz-tab-size:4;-o-tab-size:4;tab-size:4;"><code class="language-text" data-lang="text"><span style="display:flex;"><span>Welcome to Buildroot </span></span><span style="display:flex;"><span>buildroot login: root </span></span><span style="display:flex;"><span># </span></span></code></pre></div><p>Press <code>Ctrl-a x</code> to exit.</p> <p>If you installed OpenSSH, you should also add</p> <div class="highlight"><pre tabindex="0" style="color:#f8f8f2;background-color:#272822;-moz-tab-size:4;-o-tab-size:4;tab-size:4;"><code class="language-bash" data-lang="bash"><span style="display:flex;"><span>-net nic,model<span style="color:#f92672">=</span>virtio -net user,hostfwd<span style="color:#f92672">=</span>tcp::2222-:22 </span></span></code></pre></div><p>to your command. This will forward port 22 from the virtual machine to port 2222 on your host, making the VM accessible over SSH.</p> <h2 id="debugging-the-kernel-with-gdb">Debugging the kernel with GDB</h2> <p>Now that we have our VM running, we want to start using GDB to debug it. First, we&rsquo;ll need to build the kernel to support debugging using a custom build config. You should run <code>make linux-menuconfig</code> and set the following options:</p> <ul> <li><code>Kernel hacking -&gt; Kernel debugging</code></li> <li><code>Kernel hacking -&gt; Compile-time checks and compiler options -&gt; Debug information</code></li> <li><code>Kernel hacking -&gt; Compile-time checks and compiler options -&gt; Provide GDB scripts for kernel debugging</code></li> </ul> <p>Alternatively, put the following in your kernel build config:</p> <div class="highlight"><pre tabindex="0" style="color:#f8f8f2;background-color:#272822;-moz-tab-size:4;-o-tab-size:4;tab-size:4;"><code class="language-text" data-lang="text"><span style="display:flex;"><span>CONFIG_DEBUG_KERNEL=y </span></span><span style="display:flex;"><span>CONFIG_DEBUG_INFO_DWARF_TOOLCHAIN_DEFAULT=y </span></span><span style="display:flex;"><span>CONFIG_GDB_SCRIPTS=y </span></span></code></pre></div><p>Run <code>make</code> again to build your environment. When we run QEMU now, we&rsquo;ll add the following to our command:</p> <ul> <li><code>-gdb tcp::1234</code> will open up a gdbserver on TCP port 1234.</li> <li>The <code>-S</code> flag will tell the kernel to wait before booting. This will allow us to attach GDB to the kernel before the boot process starts.</li> <li>We&rsquo;ll add <code>nokaslr</code> to the kernel command line parameters to disable <a href="https://en.wikipedia.org/wiki/Address_space_layout_randomization">address space layout randomization</a> in the kernel.</li> </ul> <p>Your full command should now look something like this:</p> <div class="highlight"><pre tabindex="0" style="color:#f8f8f2;background-color:#272822;-moz-tab-size:4;-o-tab-size:4;tab-size:4;"><code class="language-bash" data-lang="bash"><span style="display:flex;"><span>qemu-system-x86_64 <span style="color:#ae81ff">\ </span></span></span><span style="display:flex;"><span><span style="color:#ae81ff"></span> -kernel ./output/images/bzImage <span style="color:#ae81ff">\ </span></span></span><span style="display:flex;"><span><span style="color:#ae81ff"></span> -drive file<span style="color:#f92672">=</span>./output/images/rootfs.ext4,if<span style="color:#f92672">=</span>virtio,format<span style="color:#f92672">=</span>raw <span style="color:#ae81ff">\ </span></span></span><span style="display:flex;"><span><span style="color:#ae81ff"></span> -nographic <span style="color:#ae81ff">\ </span></span></span><span style="display:flex;"><span><span style="color:#ae81ff"></span> -append <span style="color:#e6db74">&#34;nokaslr root=/dev/vda console=tty1 console=ttyS0&#34;</span> <span style="color:#ae81ff">\ </span></span></span><span style="display:flex;"><span><span style="color:#ae81ff"></span> -enable-kvm <span style="color:#ae81ff">\ </span></span></span><span style="display:flex;"><span><span style="color:#ae81ff"></span> -cpu host <span style="color:#ae81ff">\ </span></span></span><span style="display:flex;"><span><span style="color:#ae81ff"></span> -m 512M <span style="color:#ae81ff">\ </span></span></span><span style="display:flex;"><span><span style="color:#ae81ff"></span> -gdb tcp::1234 <span style="color:#ae81ff">\ </span></span></span><span style="display:flex;"><span><span style="color:#ae81ff"></span> -S </span></span></code></pre></div><p>Before starting GDB, you&rsquo;ll want to ensure that you&rsquo;ll be able to load the <code>vmlinux-gdb.py</code> script by adding it to your auto-load safe-path:</p> <div class="highlight"><pre tabindex="0" style="color:#f8f8f2;background-color:#272822;-moz-tab-size:4;-o-tab-size:4;tab-size:4;"><code class="language-bash" data-lang="bash"><span style="display:flex;"><span>echo <span style="color:#e6db74">&#34;add-auto-load-safe-path </span><span style="color:#66d9ef">$(</span>realpath output/images/linux/vmlinux-gdb.py<span style="color:#66d9ef">)</span><span style="color:#e6db74">&#34;</span> <span style="color:#ae81ff">\ </span></span></span><span style="display:flex;"><span><span style="color:#ae81ff"></span> &gt;&gt; ~/.gdbinit </span></span></code></pre></div><p>Then run</p> <div class="highlight"><pre tabindex="0" style="color:#f8f8f2;background-color:#272822;-moz-tab-size:4;-o-tab-size:4;tab-size:4;"><code class="language-bash" data-lang="bash"><span style="display:flex;"><span>gdb -x ./kerneldebug.gdb ./output/images/linux/vmlinux </span></span></code></pre></div><p>where <code>kerneldebug.gdb</code> contains the following:</p> <div class="highlight"><pre tabindex="0" style="color:#f8f8f2;background-color:#272822;-moz-tab-size:4;-o-tab-size:4;tab-size:4;"><code class="language-text" data-lang="text"><span style="display:flex;"><span># Attach to QEMU </span></span><span style="display:flex;"><span>target remote :1234 </span></span><span style="display:flex;"><span> </span></span><span style="display:flex;"><span># Add any breakpoints you already know you want to set here, e.g. </span></span><span style="display:flex;"><span># </span></span><span style="display:flex;"><span># hbreak function_1 </span></span><span style="display:flex;"><span># hbreak function_2 </span></span><span style="display:flex;"><span># ... </span></span><span style="display:flex;"><span> </span></span><span style="display:flex;"><span>hbreak start_kernel </span></span></code></pre></div><p>If all went well, you should be able to run</p> <div class="highlight"><pre tabindex="0" style="color:#f8f8f2;background-color:#272822;-moz-tab-size:4;-o-tab-size:4;tab-size:4;"><code class="language-text" data-lang="text"><span style="display:flex;"><span>(gdb) c </span></span></code></pre></div><p>in GDB, and then see Linux boot up in the terminal where you ran QEMU.</p> <h2 id="additional-references">Additional references</h2> <p>As mentioned earlier, <a href="https://nickdesaulniers.github.io/blog/2018/10/24/booting-a-custom-linux-kernel-in-qemu-and-debugging-it-with-gdb/">Nick Desaulnier&rsquo;s post</a> and <a href="https://www.josehu.com/technical/2021/01/02/linux-kernel-build-debug.html">Guanzhou Hu&rsquo;s post</a> were both helpful references for me.</p> <p>The <a href="https://buildroot.org/downloads/manual/manual.html">Buildroot user manual</a> is the most complete source of information on how to use Buildroot.</p> <p>The kernel sources have <a href="https://docs.kernel.org/dev-tools/gdb-kernel-debugging.html">some documentation</a> on how to debug the kernel with GDB.</p>Similarity search is better than most people give it credit for/notes/similarity_search_with_gzip/Thu, 13 Jul 2023 16:09:27 -0400/notes/similarity_search_with_gzip/<h2 id="on-k-nearest-neighbors">On k-nearest neighbors</h2> <p>If you ever read an introductory machine learning textbook or take a course on the subject, one of the first classification algorithms that you are likely to learn about is <a href="https://en.wikipedia.org/wiki/K-nearest_neighbors_algorithm">k-nearest neighbors (kNN)</a>. The idea behind it is pretty straightforward: suppose that you have a dataset split into two different classes, and you are given a new point that you want to classify. To do so, you would find the \(k\) points that are closest to it, and classify the new point as belonging the most common class among those points.</p> <p>kNN is a completely respectable algorithm, and still an active area of research. But in most ML education and discussions, kNN gets written off pretty quickly, because although training a kNN classifier is extremely cheap, classification takes (naively) \(O(Nd)\) time, where \(N\) is the size of the dataset and \(d\) is the dimensionality<sup id="fnref:1"><a href="#fn:1" class="footnote-ref" role="doc-noteref">1</a></sup>. So it pleased me to see <a href="https://aclanthology.org/2023.findings-acl.426/">this paper</a>, &ldquo;&lsquo;Low-Resource&rsquo; Text Classification: A Parameter-Free Classification Method with Compressors&rdquo;, making the rounds recently. The authors of this paper construct a new string metric, defined as</p> <p>\[ \begin{aligned} d(x,y) = \frac{C(xy) - \min{\{C(x), C(y)\}}}{\max{\{C(x), C(y)\}}} \end{aligned} \]</p> <p>where \(C(\cdot)\) is the length of its input after compression with gzip. The idea, roughly, is to figure out the difference in the <a href="https://en.wikipedia.org/wiki/Kolmogorov_complexity">Kolmogorov complexity</a> of the concatenated string \(xy\) from the strings \(x\) or \(y\). Since Kolmogorov complexity isn&rsquo;t actually computable, we use a lossless compression algorithm as a stand-in.</p> <p>It turns out that kNN, with this metric, is an extremely solid classifier on a range of tasks, even compared to many state-of-the-art <a href="https://en.wikipedia.org/wiki/BERT_(language_model)">BERT</a>-based models. The authors demonstrate that it is quite competitive on the test set to most of the baselines it compares against, and in fact beats all of them on out-of-domain datasets<sup id="fnref:2"><a href="#fn:2" class="footnote-ref" role="doc-noteref">2</a></sup>.</p> <h2 id="accelerating-similarity-search">Accelerating similarity search</h2> <p>One downside of the paper is that it never really addresses its original promise, which is specifically <em>low-resource</em> classification. The methods that are presented still take \(O(Nd)\) time to execute, which for a reasonably-sized dataset is still much more expensive than a neural network. There are, however, a few different tricks that can be used to accelerate similarity search. <a href="https://en.wikipedia.org/wiki/K-d_tree">k-d trees</a> and <a href="https://en.wikipedia.org/wiki/Ball_tree">ball trees</a> reduce the impact of \(N\), while dimensionality reduction through the likes of <a href="https://en.wikipedia.org/wiki/Principal_component_analysis">PCA</a> and <a href="https://en.wikipedia.org/wiki/Random_projection">randomized projections</a> reduces the impact of \(d\).</p> <p>My personal favorite trick in this genre is <a href="https://en.wikipedia.org/wiki/Locality-sensitive_hashing">locality-sensitive hashing</a>, or LSH. An LSH family for a given similarity function is a family of randomized hash functions with the property that, for two inputs and a randomly-sampled hash function, the probability of a hash collision between those inputs increases the more similar they are to one another. Families of locality-sensitive hash functions are known to exist for a ton of different similarity functions, including:</p> <ul> <li>Cosine similarity<sup id="fnref:3"><a href="#fn:3" class="footnote-ref" role="doc-noteref">3</a></sup></li> <li>\(\ell^p\) distance (in particular, Manhattan and Euclidean distance)<sup id="fnref:4"><a href="#fn:4" class="footnote-ref" role="doc-noteref">4</a></sup></li> <li>Jaccard similarity<sup id="fnref:5"><a href="#fn:5" class="footnote-ref" role="doc-noteref">5</a></sup></li> <li>Inner product similarity<sup id="fnref:6"><a href="#fn:6" class="footnote-ref" role="doc-noteref">6</a></sup></li> <li>Hamming distance<sup id="fnref:7"><a href="#fn:7" class="footnote-ref" role="doc-noteref">7</a></sup></li> </ul> <p>It&rsquo;s also possible to use a hybrid approach to construct hash functions for other notions of similarity. For instance, you can use contrastive learning to train an embedding that maps inputs to some vector space, and then use SimHash to hash those vectors on their cosine similarity. You can also apply LSH even when your metric doesn&rsquo;t have its own hash family, as long as you have another metric that&rsquo;s correlated to the original. In that case, you&rsquo;d use LSH with the second metric to reduce your original search space, and then run kNN with the original metric over the remaining points. A classic example of this is using Jaccard similarity with shingling as an initial, coarse-grained string similarity metric, and then Levenshtein distance as a fine(r)-grained metric for kNN.</p> <p>The way that LSH works in practice is that you randomly generate a bunch of hash functions, construct a few hash tables (say, 8 tables with 16 hash functions each), and insert your database into each of those tables<sup id="fnref:8"><a href="#fn:8" class="footnote-ref" role="doc-noteref">8</a></sup>. Then, to use kNN to classify a point, you would hash it with all of the hash functions you&rsquo;ve constructed. At the end, your search space is reduced to just those points with which you got a hash collision in each of the tables you queried, and you just search against those remaining candidates.</p> <h2 id="comparing-knn-with-other-learning-algorithms">Comparing kNN with other learning algorithms</h2> <p>The resulting classifier that you get with kNN can be much faster and require less specialized hardware than what you&rsquo;d get with e.g. a neural network. Moreover, the outputs of kNN are a lot more easily interpretable than what you&rsquo;d get with other models; it&rsquo;s easier to understand why your classifier came to a particular conclusion, as well as to characterize the failure conditions of your classifier.</p> <p>The main downside, really, is that choosing a good similarity metric can be challenging for some datasets. It requires a lot more domain expertise than the more exotic classifiers, which are in large part plug-and-play. Choosing metrics that are easily compatible with LSH can be even more of a challenge, and if you&rsquo;re at the point where you&rsquo;re reaching for something like contrastive learning to generate an embedding where LSH would be feasible, you might (reasonably) want to try learning a neural net anyways.</p> <p>But at a fundamental level, you aren&rsquo;t losing anything by using kNN; kNN is, at least, theoretically equivalent (contingent on the choice of similarity function) to any other classifier in the machine learning grab-bag.</p> <div class="footnotes" role="doc-endnotes"> <hr> <ol> <li id="fn:1"> <p>In fact it can be much worse than that, depending on your choice of metric.&#160;<a href="#fnref:1" class="footnote-backref" role="doc-backlink">&#x21a9;&#xfe0e;</a></p> </li> <li id="fn:2"> <p>See Tables 3 and 5 of the paper.&#160;<a href="#fnref:2" class="footnote-backref" role="doc-backlink">&#x21a9;&#xfe0e;</a></p> </li> <li id="fn:3"> <p>Moses S. Charikar. Similarity estimation techniques from rounding algorithms. In Proceedings of the Thiry-Fourth Annual ACM Symposium on Theory of Computing, STOC &lsquo;02, page 380–388, New York, NY, USA, 2002. Association for Computing Machinery. 10.1145/509907.509965.&#160;<a href="#fnref:3" class="footnote-backref" role="doc-backlink">&#x21a9;&#xfe0e;</a></p> </li> <li id="fn:4"> <p>Datar, Mayur and Indyk, Piotr and Immorlica, Nicole and Mirrokni, Vahab. (2004). Locality-sensitive hashing scheme based on p-stable distributions. Proceedings of the Annual Symposium on Computational Geometry. 10.1145/997817.997857.&#160;<a href="#fnref:4" class="footnote-backref" role="doc-backlink">&#x21a9;&#xfe0e;</a></p> </li> <li id="fn:5"> <p>Broder, A. On the resemblance and containment of documents. Compression and Complexity of Sequences: Proceedings, Positano, Amalfitan Coast, Salerno, Italy, June 11-13, 1997. doi:10.1109/SEQUEN.1997.666900.&#160;<a href="#fnref:5" class="footnote-backref" role="doc-backlink">&#x21a9;&#xfe0e;</a></p> </li> <li id="fn:6"> <p>Shrivastava, Anshumali, and Ping Li. &ldquo;Improved asymmetric locality sensitive hashing (ALSH) for maximum inner product search (MIPS).&rdquo; arXiv preprint arXiv:1410.5410 (2014). <a href="https://arxiv.org/abs/1410.5410">https://arxiv.org/abs/1410.5410</a>&#160;<a href="#fnref:6" class="footnote-backref" role="doc-backlink">&#x21a9;&#xfe0e;</a></p> </li> <li id="fn:7"> <p>Indyk, Piotr.; Motwani, Rajeev. (1998). &ldquo;Approximate Nearest Neighbors: Towards Removing the Curse of Dimensionality.&rdquo;. Proceedings of 30th Symposium on Theory of Computing.&#160;<a href="#fnref:7" class="footnote-backref" role="doc-backlink">&#x21a9;&#xfe0e;</a></p> </li> <li id="fn:8"> <p>The number of hash functions/tables you construct will depend on what you want the cutoff probability to look like for which points you filter in. Each LSH family has some characterization of collision probability versus similarity which you can use to make this determination. For instance, for SimHash (the standard LSH family for cosine similarity), that probability for a single randomly-sampled hash function is \(1 - \theta/\pi\), where \(\theta = \arccos{(\text{cossim}(x,y))}\) is the angle between \(x\) and \(y\).&#160;<a href="#fnref:8" class="footnote-backref" role="doc-backlink">&#x21a9;&#xfe0e;</a></p> </li> </ol> </div>Connect to UCLA's wifi so that strangers can send you malware for free/notes/ucla_networks/Mon, 13 Feb 2023 00:00:00 +0000/notes/ucla_networks/<p>Here&rsquo;s the TL;DR (you can also check out the <a href="#faq">FAQ</a>):</p> <blockquote> <p>If you&rsquo;re a UCLA student, faculty, or staff member, or are visiting UCLA, <strong>you are exposing yourself to a tremendous level of risk by connecting a device to eduroam or ethernet<sup id="fnref:1"><a href="#fn:1" class="footnote-ref" role="doc-noteref">1</a></sup>.</strong> If possible, you should connect to UCLA_WIFI or UCLA_WEB instead.</p> </blockquote> <h2 id="vulnerability-scanners-on-my-ip-range-its-more-likely-than-you-think">&ldquo;Vulnerability scanners, on <em>my</em> IP range?&rdquo; It&rsquo;s more likely than you think.</h2> <p>I transferred to UCLA&rsquo;s PhD program in Electrical and Computer Engineering about a month ago. My first week on campus, I noticed that UCLA&rsquo;s <a href="https://en.wikipedia.org/wiki/Eduroam">eduroam</a> network was assigning my laptop a rather unusual IP address:</p> <pre tabindex="0"><code>@kernelmethod ➜ ~ ip addr show dev wlp0s20f3 | grep inet inet 131.179.58.220/23 brd 131.179.58.255 scope global dynamic wlp0s20f3 inet 131.179.59.35/23 brd 131.179.58.255 scope global secondary dynamic noprefixroute wlp0s20f3 </code></pre><p>Dust off your <a href="https://en.wikipedia.org/wiki/Reserved_IP_addresses">table of reserved IP addresses</a> and you might notice that <code>131.179.58.220</code> and <code>131.179.59.35</code> aren&rsquo;t IP addresses reserved for LANs. In fact, <code>131.179.0.0/16</code> is actually an IP range belonging to UCLA.</p> <p>I also noticed that my firewall was suddenly dropping a <em>lot</em> of traffic, from a bunch of random IP addresses. Lots of machines trying to hit lots of ports on my laptop, most of which <em>definitely were not</em> on my LAN.</p> <h3 id="under-the-hood">Under the hood</h3> <p>It seems<sup id="fnref:2"><a href="#fn:2" class="footnote-ref" role="doc-noteref">2</a></sup> that UCLA&rsquo;s eduroam and ethernet assign IP addresses to people&rsquo;s devices from the following IPv4 subnets:</p> <ul> <li><code>128.97.0.0/16</code></li> <li><code>131.179.0.0/16</code></li> <li><code>149.142.0.0/16</code></li> <li><code>164.67.0.0/16</code></li> <li><code>169.232.0.0/16 </code></li> </ul> <p>The normal way to assign IP addresses to devices would be to use a reserved range like <code>10.0.0.0/8</code> or <code>192.168.0.0/16</code> and then use <a href="https://en.wikipedia.org/wiki/Network_address_translation">NAT</a> to translate between internal IP addresses and the outside world. In contrast, UCLA gives each device that connects to its network a unique address from the ranges that it owns.</p> <p>This feels like a somewhat strange way to set up a network, but what do I know? I&rsquo;m not an network admin for an organization with tens of thousands of devices that also happens to own several Class B networks. What UCLA is doing is a bit odd, but not <em>necessarily</em> problematic.</p> <p>The bigger issue is that there&rsquo;s absolutely no firewall between devices on UCLA&rsquo;s network and the outside world.</p> <figure><img src="../../img/ucla_networks/virus_firewall.png" alt="A little pixel art drawing I made of a virus standing in front of a firewall."/> </figure> <h2 id="lets-pay-shodan-a-visit">Let&rsquo;s pay Shodan a visit</h2> <p>Type in <code>org:&quot;University of California Los Angeles&quot;</code> into <a href="https://www.shodan.io/">Shodan</a> and you&rsquo;ll immediately pull somewhere around 15,000 - 20,000 IP addresses from devices at UCLA, most of which are on the five aforementioned IP ranges:</p> <figure><img src="../../img/ucla_networks/shodan_list.webp" alt="A list of the first few devices that appear on Shodan for the query org:&#34;University of California Los Angeles&#34;. The first few devices listed here are somebody&#39;s Macbook, a Wordpress server, and a MySQL server. For this particular query, Shodan returned 18,739 results."/> </figure> <p>Not! Great!</p> <p>There&rsquo;s plenty to dig into here. As one might expect (if you&rsquo;ve ever browsed <a href="https://github.com/jakejarvis/awesome-shodan-queries">Shodan&rsquo;s horror stories</a> before), there are dozens of internet-connected cameras, smart lightbulbs, and similar devices that are exposed. Among those results there are also thousands of personal devices. For instance, as I write this, a search for <code>org:&quot;University of California Los Angeles&quot; macbook pro</code> reveals ~1,000 Macbooks on campus (usually through an open <a href="https://en.wikipedia.org/wiki/AirPlay">AirPlay</a> port). The query <code>org:&quot;University of California Los Angeles&quot; port:62078</code> turns up over 6,000 iPhones and iPads<sup id="fnref:3"><a href="#fn:3" class="footnote-ref" role="doc-noteref">3</a></sup>.</p> <p>Between Shodan queries and some of my own tests, it appears that all traffic from the outside world is permitted into UCLA&rsquo;s networks. <strong>This is a significant security risk to students, faculty, and staff connecting to eduroam or ethernet</strong>. A lot of devices that have a security vulnerability in some shape or fashion aren&rsquo;t easily exploitable only because they&rsquo;re protected from the outside world by some combination of firewalls and NAT. UCLA completely removes this layer of protection.</p> <p>It&rsquo;s hard to measure the downstream impacts of this exposure. In most cases, it&rsquo;s going to be really difficult to figure out whether a device was hacked by being exploited remotely while exposed on UCLA&rsquo;s network, or if it was hacked for a different reason. To give one data point, though, when I started this investigation I found two exposed and unauthenticated Elasticsearch instances on UCLA&rsquo;s networks. Both had been ransomed, with their indices apparently backed up to a third-party server, deleted, and then replaced with the following ransom note.</p> <figure><img src="../../img/ucla_networks/elasticsearch_ransom.webp" alt="A screenshot of a note left on a public Elasticsearch instance on the UCLA eduroam IP range. The note starts &#39;All indexs has been dropped. But we backup all indexs. The only method of recoveribing database is to pay 0.021 BTC.&#39; It then gives details on how to pay the ransom to recover the lost data."/><figcaption> <p>This does not spark joy.</p> </figcaption> </figure> <h3 id="privacy-risks">Privacy risks</h3> <p>Beyond the security implications I&rsquo;ve listed, there are also privacy issues at stake. Every single device that connects to eduroam is given a publicly routable IP address with no traffic filtering, which makes it trivial to see what kinds of devices people are connecting at UCLA.</p> <p>In a lot of cases, these are devices that are most likely hooked up to the wifi in UCLA housing. For instance, try the following Shodan queries:</p> <ul> <li><code>org:&quot;University of California Los Angeles&quot; os:playstation</code></li> <li><code>org:&quot;University of California Los Angeles&quot; &quot;bedroom&quot;</code></li> <li><code>org:&quot;University of California Los Angeles&quot; &quot;apple tv&quot;</code></li> </ul> <p>It&rsquo;s also pretty easy to find the IP addresses of lots of students&rsquo; personal machines. Depending on what data sources you have access to, it&rsquo;d be pretty trivial to do things like figure out what sites students are browsing to and even where they physically are throughout the day.</p> <h2 id="some-speculation">Some speculation</h2> <p>UCLA should be filtering traffic coming from the outside world.</p> <p>That&rsquo;s easier said than done, though, and I can guess at a few different reasons why a firewall hasn&rsquo;t been set up yet. For one, UCLA&rsquo;s network is <em>really old</em> &ndash; to give you a sense, the first message sent over ARPANET was between Stanford and UCLA. From a technical perspective, there are probably some significant challenges in performing any major architectural changes.</p> <p>However, I think there are more compelling social reasons that explain why eduroam is such a mess. There are probably quite a few people (mostly faculty and staff) who benefit from the network being wide open. It is <em>incredibly easy</em> to set up a webserver with a reserved and mostly-static IP address on eduroam. Browsing through Shodan, I found a few dozen labs and research projects clearly benefitting from this by running servers off of UCLA&rsquo;s network (although a good portion of these haven&rsquo;t been updated in several years), and there are likely many more servers providing access to computing resources, shared datasets, and so on. Some of these machines are undoubtedly running critical services &ndash; for all I know, a firewall would cut off access to a website run by a nearly-retired physics professor that somehow underpins the entire UCLA course registration system (<em>*glares at Lou&rsquo;s List*</em>)<sup id="fnref:4"><a href="#fn:4" class="footnote-ref" role="doc-noteref">4</a></sup>.</p> <figure><img src="../../img/ucla_networks/louslist.png" alt="A screenshot of Lou&#39;s list (https://louslist.org/). The shown page is titled &#39;UVa Class Schedules&#39;."/><figcaption> <p>This professor&rsquo;s side project is the only thing standing between the University of Virginia and chaos.</p> </figcaption> </figure> <p>And I can sympathize. I&rsquo;m hoping to help set up a compute server for my lab sometime this year, and UCLA&rsquo;s network architecture makes it much easier to expose those resources to members who are off-campus and even to people at other institutions. On the other hand, a public network used by tens of thousands of students, faculty, and staff probably <em>isn&rsquo;t</em> the best place to run an experiment in figuring out how much malware people would receive in a world where NAT wasn&rsquo;t invented. And many of the issues I&rsquo;ve mentioned in this post would be at least partially alleviated by a firewall with a relatively generous allowlist.</p> <p>While we&rsquo;re on the topic of social mechanisms hampering UCLA&rsquo;s security, there&rsquo;s one other cultural issue worth mentioning here.</p> <h3 id="stop-blaming-users-for-your-bad-security-challenge-2023-difficulty-impossible">Stop Blaming Users for Your Bad Security Challenge 2023 (Difficulty: Impossible)</h3> <p>UCLA&rsquo;s policy directory has several IT security policies on file. Here&rsquo;s a direct quote from one of them, <a href="https://web.archive.org/web/20220814214943/https://www.adminpolicies.ucla.edu/APP/Number/401">UCLA Policy 401 : Minimum Security Standards for Network Devices</a>:</p> <blockquote> <p>B. Responsibilities for Compliance and Enforcement</p> <p>System Administrators</p> <p>System Administrators shall ensure that every Device for which they are responsible is in compliance with the Minimum Security Standards.</p> <p>A System Administrator may be an IT staff member whose responsibilities include ongoing maintenance for all Devices in a department or computer lab. <strong>A faculty member functions as a System Administrator when his or her personally owned computer at home connects to the Campus Network (e.g., via the UCLA wireless or through the UCLA Virtual Private Network (VPN)).</strong></p> </blockquote> <p>&ldquo;What do the Minimum Security Standards entail?&rdquo;, you might ask. Well, <a href="https://www.adminpolicies.ucla.edu/APP/Attachment?fileName=401-A">among other things</a>:</p> <blockquote> <ol start="3"> <li>Host-based firewall software</li> </ol> <p><strong>System Administrators are responsible for ensuring that computers with native host-based firewall software included in the operating system have the firewall activated and properly configured.</strong></p> <p>Exceptions may be made for firewall software that compromises the usability of critical applications.</p> </blockquote> <p>Hmmm. HMMMMM. Why <em>the</em> <em><strong>hell</strong></em> would you expect users to implement security measures that even your own security team won&rsquo;t deploy?</p> <p>Looking at this and other policies, it&rsquo;s pretty clear that the University of California&rsquo;s administration has chosen to deploy victim-blaming as a core tenet of its security posture. None of these policies, for instance, mandate the deployment of defensive measures for the protection of students or faculty, except to the extent that university assets contain confidential information (e.g. student medical and financial info) that has to be protected in compliance with federal and California law. <a href="https://web.archive.org/web/20220707224118/https://security.ucop.edu/files/documents/policies/minimum-security-standard.pdf">The policies do</a>, however, require that every &ldquo;workforce member&rdquo; (broadly defined to include anybody volunteering for UC or on UC&rsquo;s payroll in any capacity<sup id="fnref:5"><a href="#fn:5" class="footnote-ref" role="doc-noteref">5</a></sup>) is personally responsible for installing anti-malware, administrating a host-based firewall, backup and recovery, encryption, and eight other security requirements for <em>any</em> device they connect to a UC network<sup id="fnref:6"><a href="#fn:6" class="footnote-ref" role="doc-noteref">6</a></sup>.</p> <p>So the implication seems to be that if your device gets pwned on UCLA&rsquo;s network, then it&rsquo;s <em>your</em> fault, rather than an organizational failure. In their defense, UC is hardly the only organization to treat security like an individual burden rather than a communal responsibility; on the other hand, most other orgs don&rsquo;t have hundreds of thousands of IP addresses on hand to run a network like it&rsquo;s the Wild West.</p> <h3 id="the-ucla-security-teams-response">The UCLA security team&rsquo;s response</h3> <p>I reached out to UCLA&rsquo;s security team several weeks ago to get their perspective on the problems I&rsquo;ve mentioned here and to alert them to the ransomed servers. I was initially hesitant to discuss their response in this post as</p> <ul> <li>I don&rsquo;t think they&rsquo;re really at fault here; rather, they&rsquo;re stuck between several competing interests.</li> <li>I don&rsquo;t want to antagonize the security team of the institution that I work for. 🙃</li> </ul> <p>I think that omitting this piece would leave out a valuable perspective, however. So for the sake of completeness (and in the spirit of transparency) I&rsquo;ll briefly discuss the questions I sent and the response I received.</p> <p>On January 23rd I sent a message to the Office of the CISO with the following questions. My general hope was to get a better understanding of UC&rsquo;s (perceived) threat profile, its assessment of the risk that devices connected to eduroam face, and what preventative measures were being taken.</p> <blockquote> <p>(a) Is the Office for the CISO aware of the level of exposure that devices on the eduroam network are facing to the rest of the internet?</p> <p>(b) Is the Office for the CISO aware that there are devices currently on these networks that are actively being exploited?</p> <p>(c) Is the Office and/or UCLA IT actively tracking this issue, and planning on deploying a firewall, NAT, and/or other mitigations for this issue for devices connecting to the eduroam network in the near future?</p> <p>(d) If so, when is a fix for this issue likely to be deployed?</p> </blockquote> <p>The Office of the CISO responded:</p> <blockquote> <p>(a) - We are aware of the &ldquo;exposure&rdquo; that devices of any type sitting on publicly-routable IP address space present. Please do keep in mind that there are many different types of usage of the campus wireless networks by a variety of populations on campus. The ability to host an accessible resource is viewed as a benefit and convenience by many.</p> <p>(b) - All Internet attached devices are at risk of exploitation. Our office has extensive threat detection and identification tools built to detect adversary behavior and take the appropriate steps to contain and remediate threats to campus. This includes campus-wide network threat detection and monitoring. You can read a little more about our efforts at <a href="https://ociso.ucla.edu/service/threat-detection-and-identification">https://ociso.ucla.edu/service/threat-detection-and-identification</a>.</p> <p>(c) - We constantly strive to improve the campus security posture through enhancements in not only technological baselines, but improved user awareness and training. We are on the precipice of launching a few key initiatives throughout the UCLA community, including our Bruin Secure program (<a href="https://it.ucla.edu/it-ucla/key-initiatives)">https://it.ucla.edu/it-ucla/key-initiatives)</a>, which will focus on tailored training and awareness for all members of the UCLA community. Additional mitigations such as those you mentioned including firewalls and NATs will be reviewed as part of a larger network review discussion attached to the IT Transformation effort currently underway for campus (<a href="https://it.ucla.edu/it-ucla/about-it-services/key-initiatives/it-assessment)">https://it.ucla.edu/it-ucla/about-it-services/key-initiatives/it-assessment)</a>.</p> <p>(d) - Timelines will be developed as the projects kick-off later in 2023.</p> </blockquote> <p>This response doesn&rsquo;t actually deny the fact that the &ldquo;accessible&rdquo; nature of UCLA&rsquo;s network is a major security risk that is vastly more dangerous to the median user than it is helpful. In general, it seems like the threat is being written off because (a) there are various stakeholders who would complain if the campus network was closed off, and (b) any networked device can, hypothetically, be exploited. I&rsquo;m sympathetic to (if unconvinced by) the first excuse, but not the second. It suggests a very binary view of security that is largely ignorant to attackers&rsquo; points of friction.</p> <p>As for the various initiatives that are described, I can&rsquo;t speak to many of them as there simply isn&rsquo;t a lot of public information about what they look like in practice. <a href="https://web.archive.org/web/20230214030612/https://security.ucop.edu/services/threat-detection-and-identification/fire-eye/faq.html">This FAQ</a> (which reads more like a press release) was the most information I could find on the Trellix<sup id="fnref:7"><a href="#fn:7" class="footnote-ref" role="doc-noteref">7</a></sup> TDI appliances. There isn&rsquo;t any information, for example, about whether these devices scan the public IP ranges (in any case, I&rsquo;m skeptical that they provide any value to devices not owned by the university). There&rsquo;s even less information about Bruin Secure, although the program seems to be in line with UC&rsquo;s stance that <em>you</em> are exclusively responsible for your own security. UCLA wants to teach its community how to avoid stepping on mines when it could just as easily remove the minefield.</p> <h2 id="i-pretend-you-ask-me-questions-and-i-answer-them">I pretend you ask me questions and I answer them</h2> <a id="faq"></a> <span class="text-underline"> <strong>Q: I connected to eduroam last week! Have I been hacked???</strong> </span> <p>Uh, I don&rsquo;t know. Probably not though.</p> <span class="text-underline"> <strong>Q: Okay, but how worried should I be?</strong> </span> <p>That&rsquo;ll depend on your <a href="https://en.wikipedia.org/wiki/Threat_model">threat model</a>, but on balance I&rsquo;d say that you probably shouldn&rsquo;t be in full-blown panic mode. That being said, if you visit UCLA on a regular basis (and especially if you live on university property), you may want to consider taking some basic protective measures.</p> <span class="text-underline"> <strong>Q: What can I do to protect myself?</strong> </span> <p>Unfortunately, UCLA&rsquo;s current stance on network security is to leave every student and staff member to fend for themselves. Until that changes, you&rsquo;re the only person who&rsquo;ll be able to protect your devices.</p> <p>The most practical recommendation I can give is this:</p> <blockquote> <p><strong>Don&rsquo;t connect your devices to eduroam or ethernet at UCLA.</strong></p> </blockquote> <p>I would be especially cautious about connecting random IoT (<a href="https://en.wikipedia.org/wiki/Internet_of_things">&ldquo;Internet of Things&rdquo;</a>) devices like cameras and printers to eduroam. There are two other networks on campus, UCLA_WEB and UCLA_WIFI, that you should connect to instead.</p> <p>If you&rsquo;re confident in your technical skills, you can also try setting up a host-based firewall on your laptop or PC. This is essentially a firewall that runs on your computer, rather than on a router or dedicated hardware. Ironically, it seems that UCLA doesn&rsquo;t offer help on setting up firewalls (despite the fact that, in theory, faculty and staff are expected to set them up for devices they attach to UCLA&rsquo;s network). Firewall guidance is beyond the scope of this post, but here are some resources you can use to get started:</p> <ul> <li><a href="https://learn.microsoft.com/en-us/windows/security/threat-protection/windows-firewall/best-practices-configuring">Windows</a></li> <li><a href="https://support.apple.com/guide/mac-help/block-connections-to-your-mac-with-a-firewall-mh34041/13.0/mac/13.0">macOS</a></li> <li><a href="https://wiki.archlinux.org/title/Simple_stateful_firewall">Linux</a><sup id="fnref:8"><a href="#fn:8" class="footnote-ref" role="doc-noteref">8</a></sup></li> </ul> <span class="text-underline"> <strong>Q: What&rsquo;s the risk if I don&rsquo;t do anything?</strong> </span> <p>In the worst case, somebody might be able to exploit a vulnerable program running on your machine, getting full remote access to your device and everything on it. While this is certainly something that could happen even if UCLA had a firewall in place, the lack of traffic filtering makes this relatively trivial for someone to exploit (if such a vulnerability exists on your machine).</p> <p>The good news is that if you install software updates regularly, the risk of this happening (for <em>most</em> people, on <em>most</em> of their devices) is pretty low. Regardless of whether or not you install patches, however, it&rsquo;s pretty much impossible to avoid the privacy issues I mentioned earlier.</p> <span class="text-underline"> <strong>Q: That&rsquo;s helpful, thanks!</strong> </span> <p>No problem!</p> <span class="text-underline"> <strong>Q: None of this has been helpful. Go to hell.</strong> </span> <p>Sorry.</p> <h2 id="stuff-that-i-didnt-think-was-worth-putting-in-the-main-body-of-this-post">Stuff that I didn&rsquo;t think was worth putting in the main body of this post</h2> <h3 id="whats-a-firewall">What&rsquo;s a firewall?</h3> <p>(You can skip this part if you&rsquo;re familiar with how firewalls work.)</p> <p>There&rsquo;s a good chance that you&rsquo;ve heard the term &ldquo;firewall&rdquo; somewhere before (as in &ldquo;the hackers have breached the firewall!&rdquo;). But if you don&rsquo;t come from a networking or security background, you might not be familiar with what a firewall actually does.</p> <p>For some background: at any given time, your phone or computer is running a bunch of different programs on it, and some of those may need to wait for a connection from another device on your network. A good example of this is a printer. Printers spend most of their time sitting idle, and while they&rsquo;re waiting they listen for computers trying to connect to them.</p> <p>Most of the time, attackers can&rsquo;t access random printers on the internet for one of two reasons. The first is that your printer may not be publicly accessible. Most of the time, when you connect a device to a network, your router will use NAT (<a href="https://en.wikipedia.org/wiki/Network_address_translation">Network Address Translation</a>) to assign your device an IP address. The end result is that your printer doesn&rsquo;t have a public IP address that the outside world can use to communicate with it.</p> <p>The second reason is that those printers are behind a firewall. In this context, a firewall is a system that limits what traffic can enter or exit a network. You could have a firewall rule, for instance, that blocks any connections to your printer from machines outside of the network.</p> <p>Despite the existence of these mechanisms, there are still a ton of printers and such that are publicly exposed on the internet. If you&rsquo;ve ever tried standing up a server, you&rsquo;ll know just how much crap a publicly accessible machine receives from various botnets and scanners (malicious or not). It&rsquo;s pretty easy for a system to get auto-pwned by Joe Ransomware Operator in a matter of minutes if it isn&rsquo;t properly secured.</p> <p>In UCLA&rsquo;s case, it doesn&rsquo;t use either NAT or a firewall. The former it can likely do without &ndash; NAT isn&rsquo;t really intended to be a security mechanism, and for orgs like UCLA that own a large swath of IP space it might not be particularly desirable. On the other hand, firewalls are a pretty common and useful security measure. But for one reason or another, UCLA doesn&rsquo;t seem to have any firewall between students&rsquo; devices and the outside world.</p> <figure><img src="../../img/ucla_networks/wall-of-fire.jpg" alt="&#39;Wall of Fire&#39; card from Magic The Gathering."/><figcaption> <p>Source: Magic the Gathering / Wizards of the Coast</p> </figcaption> </figure> <hr> <p> <span class="text-underline"> <em>Edited 2023-02-13:</em> </span> <em>I added a section on the UCLA security team&rsquo;s response to my questions about eduroam.</em></p> <div class="footnotes" role="doc-endnotes"> <hr> <ol> <li id="fn:1"> <p>This post is primarily about the eduroam wifi, since that&rsquo;s the primary way most people are going to encounter the problems I describe. But any device that hooks up to UCLA&rsquo;s ethernet is equally exposed to the rest of the world.&#160;<a href="#fnref:1" class="footnote-backref" role="doc-backlink">&#x21a9;&#xfe0e;</a></p> </li> <li id="fn:2"> <p>Curiously, there appear to be some locations on campus where eduroam will assign a <code>10.0.0.0/8</code> IP address to your machine. I&rsquo;m neither familiar enough with UCLA&rsquo;s network topology nor wifi configuration in general to speak to exactly where or why this is the case, but I can say that in <em>most</em> locations that I tested, my machine was assigned an IP address from one of the listed ranges.&#160;<a href="#fnref:2" class="footnote-backref" role="doc-backlink">&#x21a9;&#xfe0e;</a></p> </li> <li id="fn:3"> <p>Apple uses TCP port 62078 on iPhone and iPad devices for reasons I don&rsquo;t fully understand. To my knowledge, though, it&rsquo;s a fairly easy way of identifying iOS / iPadOS devices from a port scan.&#160;<a href="#fnref:3" class="footnote-backref" role="doc-backlink">&#x21a9;&#xfe0e;</a></p> </li> <li id="fn:4"> <p><a href="https://louslist.org">Lou&rsquo;s List</a>, run by physics professor Lou Bloomfield, is used by almost all of the students and faculty over at the University of Virginia to find and sign up for courses. Apparently the <a href="https://devhub.virginia.edu/API">official API</a> provided by UVA for the university schedule is (or at least was) just a frontend to Lou&rsquo;s List.&#160;<a href="#fnref:4" class="footnote-backref" role="doc-backlink">&#x21a9;&#xfe0e;</a></p> </li> <li id="fn:5"> <p>The Minimum Security Standards specifically defines a &ldquo;workforce member&rdquo; as &ldquo;an employee, faculty, staff, volunteer, contractor, researcher, student worker, student supporting/performing research, medical center staff/personnel, clinician, student intern, student volunteer or person working for UC in any capacity or through any other augmentation to UC staffing levels.&rdquo;&#160;<a href="#fnref:5" class="footnote-backref" role="doc-backlink">&#x21a9;&#xfe0e;</a></p> </li> <li id="fn:6"> <p>Notably, most of these &ldquo;workforce members&rdquo; (which include grad students and TAs) don&rsquo;t receive a workplace machine from UC, much less an external HDD/SSD for backups, security keys, or similar.&#160;<a href="#fnref:6" class="footnote-backref" role="doc-backlink">&#x21a9;&#xfe0e;</a></p> </li> <li id="fn:7"> <p>(The company formerly known as FireEye.)&#160;<a href="#fnref:7" class="footnote-backref" role="doc-backlink">&#x21a9;&#xfe0e;</a></p> </li> <li id="fn:8"> <p>Nowadays I would recommend using <a href="https://www.nftables.org/">nftables</a> instead of iptables (which is used by the linked article to the Arch Linux wiki).&#160;<a href="#fnref:8" class="footnote-backref" role="doc-backlink">&#x21a9;&#xfe0e;</a></p> </li> </ol> </div>Linux Security Modules/notes/lsm/Thu, 05 Mar 2020 00:00:00 -0700/notes/lsm/<p><em>Disclaimer: I threw these notes together for CU Boulder&rsquo;s Collegiate Cyber Defense Competition team for spring 2020. There may be some inaccuracies, and parts of the notes are still incomplete.</em></p> <h2 id="intro-to-lsm">Introduction</h2> <p>The Linux kernel has a built-in <em>discretionary access control</em> (DAC) system for controlling system resources. DACs are characterized by the fact that the owner of a resource may grant somebody else access to that resource; for instance, in Linux, the owner of a file could run <code>chmod +rwx myfile</code> to give every user on the system read, write, and execute permissions to the file. As a result, a process can privesc if it&rsquo;s able to run commands as a user with higher permissions.</p> <p><a href="https://en.wikipedia.org/wiki/Linux%5FSecurity%5FModules">Linux Security Modules</a> (LSM) is a framework for implementating <em>mandatory access control</em> (MAC) in Linux. Under a MAC policy, users cannot modify the permissions of resources; the permissions are set in stone by the system administrator. This makes privilege escalation significantly more difficult. For instance, say an attacker manages to get a reverse shell into an Apache webserver, and is eventually able to login as root. Even as the root user, the attacker is confined by whatever permissions are granted to the webserver process, which may be extremely restrictive.</p> <p>There are a few other general characteristics of LSMs that differentiate them from traditional Linux DAC:</p> <ul> <li>LSMs focus on assigning permissions to <em>processes</em>, whereas the Linux DAC policy assigns permissions to users and groups.</li> <li>LSMs are generally &ldquo;deny by default&rdquo;: if access to a resource isn&rsquo;t explicitly granted, then it&rsquo;s denied.</li> <li>LSMs can have much more fine-grained permissions than Linux&rsquo;s DAC. For instance, SELinux doesn&rsquo;t just control access to files; you can define permissions over ports, syscalls, regions of memory, and more.</li> </ul> <hr> <h2 id="logging-and-auditd">Logging and auditd</h2> <h3 id="lsm-logging-intro">Introduction</h3> <p><code>auditd</code> is an auditing daemon that logs system events and provides various utilities for searching logs. It is extremely useful for SELinux as it can be used to record denial to various resources.</p> <p><code>auditd</code> logs generally look something like this:</p> <pre tabindex="0"><code class="language--n" data-lang="-n">time-&gt;Sun Mar 1 00:11:43 2020 type=PROCTITLE msg=audit(1583046703.755:2547): proctitle=636174002F6574632F736861646F77 type=SYSCALL msg=audit(1583046703.755:2547): arch=c000003e syscall=257 success=yes exit=3 a0=ffffff9c a1=7ffeef4fae7b a2=0 a3=0 items=0 ppid=11335 pid=11336 auid=4294967295 uid=33 gid=33 euid=33 suid=33 fsuid=33 egid=33 sgid=33 fsgid=33 tty=(none) ses=4294967295 comm=&#34;cat&#34; exe=&#34;/usr/bin/cat&#34; subj=system_u:system_r:httpd_t:s0 key=(null) type=AVC msg=audit(1583046703.755:2547): avc: denied { open } for pid=11336 comm=&#34;cat&#34; path=&#34;/etc/shadow&#34; dev=&#34;sda1&#34; ino=277524 scontext=system_u:system_r:httpd_t:s0 tcontext=system_u:object_r:shadow_t:s0 tclass=file permissive=1 type=AVC msg=audit(1583046703.755:2547): avc: denied { read } for pid=11336 comm=&#34;cat&#34; name=&#34;shadow&#34; dev=&#34;sda1&#34; ino=277524 scontext=system_u:system_r:httpd_t:s0 tcontext=system_u:object_r:shadow_t:s0 tclass=file permissive=1 </code></pre><p>These logs are fairly verbose, but there are a few pieces of information that are worth looking at:</p> <ul> <li><code>success</code> (under <code>type=SYSCALL</code>): tells us whether or not a given syscall was successful.</li> <li><code>comm</code> and <code>exe</code>: the command/executable that activated the rule.</li> <li><code>scontext</code>: the SELinux context of the resource that triggered the rule. Here, it&rsquo;s the context in which <code>/usr/bin/cat</code> is run.</li> <li><code>tcontext</code>: the SELinux context of the resource that the source was trying to access. Here, it&rsquo;s the context of <code>/etc/shadow</code>.</li> </ul> <h3 id="auditctl-create-auditd-rules"><code>auditctl</code>: create <code>auditd</code> rules</h3> <p><code>auditctl -w /etc/shadow -p rwa -k shadow_file</code></p> <ul> <li>Checks for reads, writes, and attribute changes to <code>/etc/shadow</code></li> <li>The <code>-k</code> flag assigns the key <code>shadow_file</code> to this rule</li> </ul> <p><code>auditctl -w /usr/local/bin -p x -k cron</code></p> <ul> <li>Checks for execution of binaries in the <code>/usr/local/bin</code> directory</li> </ul> <p><code>auditctl -a always,exit -F arch=b64 -F &quot;auid&gt;=1000&quot; -S rename -S renameat -k rename</code></p> <ul> <li>Log attempts to rename a file using the <code>rename</code> or <code>renameat</code> syscalls</li> <li>Only logs attempts for users with ID greater than or equal to 1000</li> </ul> <p>Note that rules created with <code>auditctl</code> are lost after rebooting. If you want to make them permanent, you need to save them in <code>/etc/audit/rules.d/audit.rules</code>.</p> <p>To delete a rule, use <code>-W</code> instead of <code>-w</code>.</p> <h3 id="ausearch-search-audit-logs"><code>ausearch</code>: search audit logs</h3> <p><code>ausearch -k shadow_file</code></p> <ul> <li>Search for logs with the <code>shadow_file</code> key</li> </ul> <p><code>ausearch --start recent -k cron</code></p> <ul> <li>Search for logs in the last ten minutes that are tagged with <code>cron</code></li> <li>You can also choose <code>today</code>, <code>yesterday</code>, etc. instead of <code>recent</code>.</li> <li>You can specify a specific time or date too, e.g. <code>--start 03/01/2020</code> or <code>--start 03/01/2020 16:00:00</code></li> <li>There is an <code>--end</code> flag that, in conjunction with <code>--start</code>, allows you to specify the end of a time range.</li> </ul> <p><code>ausearch -se system_u:system_r:httpd_t:s0</code></p> <ul> <li>Search for all logs where either the source SELinux context or target SELinux context is <code>system_u:system_r:httpd_t:s0</code></li> </ul> <p><code>ausearch -c apache2 -f /etc/shadow</code></p> <ul> <li>Search for all logs triggered by the <code>apache2</code> command related to the <code>/etc/shadow</code> file.</li> <li>You can use <code>-x</code> instead of <code>-x</code> to specify the path to the executable instead of its name.</li> </ul> <h3 id="ausyscall-view-available-syscalls"><code>ausyscall</code>: view available syscalls</h3> <p><code>ausyscall --dump</code></p> <ul> <li>List all linux syscalls and their numeric codes</li> </ul> <p><code>ausyscall 4</code></p> <ul> <li>Find the name of the syscall corresponding to the ID <code>4</code></li> <li>Useful when reviewing <code>auditd</code> logs</li> </ul> <hr> <h2 id="lsm-selinux">SELinux</h2> <h3 id="selinux-intro">Introduction</h3> <p>SELinux is the default LSM enhancement for RHEL, Fedora, and CentOS. It is also installed in Debian, although as of Debian 10 (Buster) it is deactivated by default, with AppArmor enabled in its place. SELinux for Ubuntu has been largely unmaintained since Ubuntu 9.10 and broken since Ubuntu 12.04. Although it is still possible to install and use SELinux via the Debian repositories (where SELinux is still maintained), it is generally recommended that you use AppArmor for Ubuntu instead.</p> <h3 id="selinux-installation">Installation</h3> <p>To get SELinux up and running, you probably want to install the following packages: <code>selinux-basics selinux-policy-default auditd</code>. In addition, depending on your system you may need to explicitly enable SELinux. You can check whether or not SELinux has been enabled by running <code>getenforce</code>; if you get the response <code>Disabled</code>, then you will need to explicitly activate it. Generally, this can be done by running <code>selinux-activate</code> and rebooting.</p> <h3 id="selinux-utilities">SELinux utilities</h3> <h4 id="getenforce-sestatus-and-seinfo-get-information-about-selinux"><code>getenforce</code>, <code>sestatus</code>, and <code>seinfo</code>: get information about SELinux</h4> <p>These three utilities each provide information about the current status and configuration of SELinux on your system:</p> <ul> <li><code>getenforce</code>: gets the current SELinux state, which can be &ldquo;Disabled&rdquo;, &ldquo;Permissive&rdquo;, or &ldquo;Enforcing&rdquo;.</li> <li><code>sestatus</code>: shows the current SELinux state like <code>getenforce</code>, but also provides a little more information (e.g. the name of the loaded policy).</li> <li><code>seinfo</code>: get information about the current SELinux policy. Here are some example queries you can run with <code>seinfo</code>: <ul> <li><code>seinfo -b</code>: show all of the available SELinux booleans, and whether they are on or off.</li> <li><code>seinfo -x -u staff_u</code>: show the full definition of the <code>staff_u</code> SELinux user. This will show you what roles are assigned to the <code>staff_u</code> user.</li> <li><code>seinfo -x -r user_r</code>: show the full definition of the <code>user_r</code> SELinux role. This will show you what types are assigned to the <code>user_r</code> user.</li> </ul> </li> </ul> <h4 id="selinux-activate-enable-selinux"><code>selinux-activate</code>: enable SELinux</h4> <p>Sets SELinux to activate after you reboot. Your machine will reboot into whatever mode has been assigned to the <code>SELINUX</code> variable in <code>/etc/selinux/config</code>.</p> <h4 id="setenforce-set-selinux-state"><code>setenforce</code>: set SELinux state</h4> <p>When SELinux is activated, you can use <code>setenforce</code> to change the current SELinux mode. <code>setenforce 0</code> will put SELinux in permissive mode, and <code>setenforce 1</code> will put it in enforcing mode.</p> <h4 id="sesearch-search-selinux-policies"><code>sesearch</code>: search SELinux policies</h4> <p><code>sesearch</code> allows you to search for specific rules in your SELinux policy. Here are some examples of how to use it:</p> <ul> <li><code>sesearch --allow -s auditd_t -t auditd_log_t</code>: search for all <code>allow</code> rules with <code>auditd_t</code> as their source type and <code>auditd_log_t</code> as their target type.</li> <li><code>sesearch --allow -s httpd_t -p read,write</code>: find all rules with source type <code>httpd_t</code> that grant the <code>read</code> and <code>write</code> permissions.</li> <li><code>sesearch --allow -s httpd_t -t auditd_log_t -ds -dt</code>: find all rules that <em>exactly</em> have <code>httpd_t</code> as their source type, and <code>auditd_log_t</code> as their destination type. Without the <code>-ds</code> and <code>-dt</code> flags, <code>sesearch</code> typically just matches by attribute contents.</li> </ul> <h4 id="chcon-change-context-of-a-file"><code>chcon</code>: change context of a file</h4> <p>Suppose that we want to change the SELinux context of <code>/var/www/html/index.html</code> to match that of the directory <code>/var/www/html/</code>. Running <code>ls -dZ /var/www/html</code>, we see that this directory has context <code>system_u:object_r:httpd_sys_content_t</code>. To change the context of <code>/var/www/html/index.html</code>, we use <code>chcon</code> as follows:</p> <pre tabindex="0"><code class="language-nil" data-lang="nil"># chcon -u system_u -r object_r -t httpd_sys_content_t /var/www/html/index.html </code></pre><p>But if you just want to change the type, you only have to use the <code>-t</code> flag, e.g. <code>chcon -t $selinux_type $target</code></p> <p>You can make things even easier by using the <code>--reference</code> flag:</p> <pre tabindex="0"><code class="language-nil" data-lang="nil"># chcon --reference /var/www/html/ /var/www/html/index.html </code></pre><p>This gives <code>/var/www/html/index.html</code> the same SELinux context as the <code>/var/www/html/</code> directory.</p> <h4 id="restorecon-restore-context-of-a-file"><code>restorecon</code>: restore context of a file</h4> <p>Recursively restore context of all files in <code>/var/www/html</code> to have the same context as the <code>/var/www/html/</code> directory:</p> <pre tabindex="0"><code class="language-nil" data-lang="nil"># restorecon -R /var/www/html </code></pre><p>You can use the <code>-v</code> flag for verbose output.</p> <h4 id="semanage-main-selinux-configuration-tool"><code>semanage</code>: main SELinux configuration tool</h4> <p>Can be used to manage SELinux settings for</p> <ul> <li>node</li> <li>file context</li> <li>booleans</li> <li>permissive state</li> <li>dontaudit</li> </ul> <p>Unlike <code>chcon</code> or <code>restorecon</code>, rules set via <code>semanage</code> are saved so that they become reapplied after reboot.</p> <p>Examples:</p> <ul> <li><code>semanage fcontext -l</code>: List all of the SELinux policy rules around file contexts.</li> <li><code>semanage fcontext -a -t httpd_sys_content_t &quot;/foo(/.*)?&quot;</code>: add the type <code>httpd_sys_content_t</code> to the <code>/foo</code> directory and all of its contents.</li> <li><code>semanage fcontext -a -e /var/www/html /foo</code>: add a rule to make the context of <code>/foo</code> match that of <code>/var/www/html</code>. After this, you can run <code>restorecon -vR /foo</code> to apply the new label to <code>/foo</code>.</li> </ul> <h4 id="audit2why"><code>audit2why</code></h4> <h4 id="audit2allow-generate-selinux-policies-from-logs-of-denied-operations"><code>audit2allow</code>: generate SELinux policies from logs of denied operations</h4> <p><code>audit2allow</code> uses log files (<code>/var/log/audit/audit.log</code> if <code>auditd</code> is installed, and <code>/var/log/messages</code> otherwise) to generate SELinux rules. It&rsquo;s an extremely useful tool for fixing broken policies.</p> <p>For example, suppose you run <code>ip addr</code>, and it appears to be denied by SELinux (a situation I encountered on Debian 10). To figure out where the issue is, let&rsquo;s start by reviewing the audit logs:</p> <pre tabindex="0"><code class="language-nil" data-lang="nil"># ausearch --start recent .. time-&gt;Wed Mar 4 21:20:21 2020 type=AVC msg=audit(1583360421.558:1732): avc: denied { signull } for pid=305 comm=&#34;systemd-journal&#34; scontext=system_u:system_r:syslogd_t:s0 tcontext=system_u:system_r:NetworkManager_t:s0 tclass=process permissive=0 </code></pre><p>We can immediately make a few notes about the log:</p> <ul> <li>The source context is <code>system_u:system_r:syslogd_t:s0</code></li> <li>The target context is <code>system_u:system_r:NetworkManager_t:s0</code></li> <li>The command that triggered the denial was <code>systemd-journal</code></li> </ul> <p>Overall, it looks like the command was denied because <code>systemd-journal</code> sent <code>SIGNULL</code> to a process with a different context, and SELinux blocked it. We can use <code>sesearch</code> to try and see what rules are defined for <code>syslogd_t</code> and <code>NetworkManager_t</code>:</p> <pre tabindex="0"><code class="language-nil" data-lang="nil"># sesearch -s syslogd_t -t NetworkManager_t -A allow syslogd_t domain:dir { getattr ioctl lock open read search }; allow syslogd_t domain:file { getattr ioctl lock open read }; allow syslogd_t domain:lnk_file { getattr read }; allow syslogd_t domain:process getattr; </code></pre><p>It looks like there aren&rsquo;t any rules directly related to both <code>syslogd_t</code> and <code>NetworkManager_t</code>. We&rsquo;re going to need to add a new rule that allows processes of type <code>syslogd_t</code> to send <code>SIGNULL</code> to processes of type <code>NetworkManager_t</code>.</p> <p>To do this, we use the <code>audit2allow</code> command. First, we use <code>audit2allow</code> to give us more information about how to fix this issue:</p> <pre tabindex="0"><code class="language-nil" data-lang="nil"># sudo ausearch -se system_u:system_r:NetworkManager_t | tail -n 1 | audit2allow -w type=AVC msg=audit(1583407237.188:26): avc: denied { search } for pid=478 comm=&#34;NetworkManager&#34; name=&#34;.cache&#34; dev=&#34;vda1&#34; ino=410827 scontext=system_u:system_r:NetworkManager_t:s0 tcontext=system_u:object_r:xdg_cache_t:s0 tclass=dir permissive=1 Was caused by: Missing type enforcement (TE) allow rule. You can use audit2allow to generate a loadable module to allow this access. </code></pre><p>With the <code>-a</code> flag, <code>audit2allow</code> will tell us what rule needs to be added in order to fix the error:</p> <pre tabindex="0"><code class="language-nil" data-lang="nil"># sudo ausearch -se system_u:system_r:NetworkManager_t | tail -n 1 | audit2allow -a #============= NetworkManager_t ============== allow NetworkManager_t xdg_cache_t:dir search; </code></pre><p>(<strong>Note</strong>: you can pipe in multiple logs at once in order to generate multiple SELinux rules.)</p> <p>Finally, we create a new policy module with the rule:</p> <p>After running <code>semodule -i syslogd.pp</code> and rebooting, a new SELinux rule will be added to your policy that permits the previously forbidden action.</p> <pre tabindex="0"><code class="language-nil" data-lang="nil"># sudo ausearch -se system_u:system_r:NetworkManager_t | tail -n 1 | audit2allow -a -M syslogd ************************** IMPORTANT ****************************** To make this policy package active, execute: semodule -i syslogd.pp </code></pre><h3 id="selinux-addtl-refs">Additional references</h3> <ul> <li><a href="https://wiki.gentoo.org/wiki/SELinux">SELinux on the Gentoo Linux wiki</a></li> <li><a href="https://wiki.gentoo.org/wiki/SELinux/Quick%5Fintroduction">Quick intro to SELinux</a></li> <li><em>The SELinux Notebook, 4th edition</em>.</li> <li><a href="https://access.redhat.com/documentation/en-us/red%5Fhat%5Fenterprise%5Flinux/6/html/security-enhanced%5Flinux/index">The RHEL SELinux documentation</a></li> </ul> <hr> <h2 id="apparmor">AppArmor</h2> <h3 id="apparmor-intro">Introduction</h3> <p>AppArmor is another Linux Security Module for implementing MAC that is enabled by default on Ubuntu and recent Debian distributions. Its chief goal is to be more user-friendly than SELinux.</p> <p>Whereas SELinux is label-based, AppArmor is path-based: access to files and executables is based on their path in the filesystem. In addition, AppArmor rules are defined in profiles (e.g. <code>/etc/apparmor.d/usr.sbin.apache2</code>), in contrast to SELinux&rsquo;s concept of a system-wide policy. In order to restrict a binary, we must define a new profile for it, and then put that profile in enforcing mode.</p> <h3 id="apparmor-installation">Installation</h3> <p>AppArmor is enabled by default on Ubuntu since version 7.10, and on Debian since Debian 10 (Buster). However, like SELinux, AppArmor has a few packages with convenient files and utilities that you should install if you want to use it:</p> <ul> <li><code>apt install apparmor-profiles apparmor-easyprof apparmor-utils</code></li> </ul> <h3 id="using-apparmor">Using AppArmor</h3> <p>In this section, we&rsquo;ll create an AppArmor profile for the <code>apache2</code> binary using the following steps:</p> <ol> <li>Create a template profile for the binary using <code>aa-easyprof</code>;</li> <li>Load that profile using <code>apparmor_parser</code>;</li> <li>Put the profile in complain mode using <code>aa-complain</code>;</li> <li>Add new AppArmor rules using <code>aa-logprof</code>; and</li> <li>Enforce the profile with <code>aa-enforce</code>.</li> </ol> <p>Note that you may not necessarily need to go through all of these steps, e.g. if you already have a default profile for the binary. There are also some tools that wrap a few of these steps together, such as <code>aa-genprof</code>.</p> <h4 id="1-dot-create-a-template-apache2-profile">1. Create a template <code>apache2</code> profile</h4> <p>Call <code>aa-easyprof /usr/sbin/apache2 &gt; /etc/apparmor.d/usr.sbin.apache2</code>. The newly created profile will be mostly empty; it may look something like this:</p> <pre tabindex="0"><code class="language-nil" data-lang="nil">/etc/apparmor.d/usr.sbin.apache2: #include &lt;tunables/global&gt; # No template variables specified &#34;/usr/sbin/apache2&#34; { #include &lt;abstractions/base&gt; # No abstractions specified # No policy groups specified # No read paths specified # No write paths specified } </code></pre><h4 id="2-dot-load-the-profile">2. Load the profile</h4> <p>Now call <code>apparmor_parser /etc/apparmor.d/usr.sbin.apache2</code> to read the new profile. Note that if you&rsquo;ve previously loaded in <code>/etc/apparmor.d/usr.sbin.apache</code>, you&rsquo;ll need to reload it by adding the <code>-r</code> flag.</p> <h4 id="3-dot-put-the-profile-in-complain-mode">3. Put the profile in complain mode</h4> <p>Put the binary in complain mode by calling <code>aa-complain apache2</code>. In complain mode, AppArmor will scan log files to try to identify what capabilities the <code>apache2</code> binary needs to have.</p> <p>Visit your Apache webserver, and start trying to simulate how it would be used in real life. As you test the webserver more and more, AppArmor will get an increasingly good idea of what capabilities the server is using.</p> <h4 id="4-dot-add-new-apparmor-rules-using-aa-logprof">4. Add new AppArmor rules using <code>aa-logprof</code></h4> <p>After you&rsquo;ve tested your webserver sufficiently, run <code>aa-logprof</code>. This will read various logfiles in <code>/var/log</code> to see what files the Apache webserver accessed, and what capabilities it used. <code>aa-logprof</code> will then present you with a list of capabilities, which you can choose to accept, deny, ignore, and so on. Choose whichever capabilities your webserver needs; any capabilities that are not explicitly granted to the webserver will be denied.</p> <h4 id="5-dot-enforce-the-profile">5. Enforce the profile</h4> <p>Call <code>aa-enforce apache2</code> to put the profile in enforcing mode. AppArmor is now running, and will deny capabilities not granted in <code>/etc/apparmor.d/usr.sbin.apache2</code>.</p> <h3 id="customizing-profiles">Customizing profiles</h3> <p>It is often desirable to hand-edit your AppArmor profiles to make them a little more specific. Here are some general syntax rules for AppArmor profiles:</p> <p><strong>Deny rules</strong>: To write a deny rule, you just write <code>deny /path/to/file perms</code>.</p> <ul> <li><code>deny /etc/shadow rw</code>: deny read and write access to <code>/etc/shadow</code>.</li> <li><code>deny /etc/sudo x</code>: deny execute permission to <code>/etc/sudo</code>.</li> </ul> <p>Note that AppArmor is deny-by-default, so you don&rsquo;t generally need to explicitly state every resource that you want to deny access to.</p> <p><strong>Accept rules</strong>:</p> <p><strong>Tunables</strong>: where possible, you should avoid using explicit paths like <code>/home/</code>, and instead use tunables, which are configurable variables defined in <code>/etc/apparmor.d/tunables</code>. For instance, instead of writing a rule like <code>/home/*/ r</code>, you should use <code>@{HOME}/ r</code>.</p> <h3 id="debugging-apparmor">Debugging AppArmor</h3> <h3 id="apparmor-addtl-refs">Additional references</h3> <ul> <li><a href="https://gitlab.com/apparmor/apparmor/-/wikis/Documentation">AppArmor documentation</a></li> <li><a href="https://ubuntu.com/tutorials/beginning-apparmor-profile-development">A tutorial on creating a new AppArmor profile</a></li> <li>You can download a high-level <a href="https://ubuntu.com/engage/apparmor-intro">whitepaper</a> on AppArmor from Canonical&rsquo;s site.</li> </ul> <h2 id="miscellaneous-references">Miscellaneous references</h2> <ul> <li>The <a href="https://www.kernel.org/doc/ols/2002/ols2002-pages-604-617.pdf"><em>Linux Security Module Framework</em> paper</a> that first introduced LSM.</li> </ul>