System Fingerprinting With Nmap By Rik Farrow, Network Magazine Nov 6, 2000 (8:58 AM) URL: http://www.networkmagazine.com:80/article/NMG20001102S0005 When someone with half a clue decides to attack your system, he or she will first try to identify the operating system. Not every attack proceeds this way: Script kiddies probe huge address spaces looking for any system with a particular port open, which indicates that just maybe that system will be vulnerable. But for the professional penetration tester or hacker, identifying the operating system is an essential step in probing. My September 2000 column ("ICMP Stands for Trouble") discussed the use of Internet Control Message Protocol (ICMP) packets for probing and for detecting operating system types. Since that time, an interesting paper on the subject has been updated by its author, Ofir Arkin. The update includes new tricks for operating system identification (see Resources). However, using ICMP will not always work, as most firewalls will block some (or all) incoming ICMP packets. That leaves the classical methods as well as a newer technique known as stack fingerprinting. The king of stack fingerprinting programs today is Network Map, or nmap. The author of the nmap program, who goes by the name Fyodor, has written a paper on the subject, in which he discusses TCP options and their usefulness in identifying operating systems via stack fingerprinting (see Resources). This column goes deeper into TCP options and how they are used by TCP/IP and nmap. BANNER GRABBING The easiest way to identify an operating system is to contact it by using an open port. While this takes a little finessing (not much), most Telnet, e-mail, FTP, and Web servers will identify not only themselves but also the operating system on which they are running. This is almost too easy, although it can sometimes be unreliable-cagey administrators may change the login banners to mislead an attacker. Other classic strategies for identifying operating systems include port sweeps and grabbing e-mail headers. Port sweeps identify open ports, and, on a system that has not been hardened, this list of open ports is often enough to identify most operating systems. E-mail headers can identify mail user agents, the user's operating system, the mail server, and sometimes even the firewall that the mail passed through. (The amount of information that can be gleaned from an e-mail message is amazing.) ID PLEASE However, the easiest way to identify operating systems is to run nmap. Nmap started off as a very functional network and port scanner, but in 1998 Fyodor added operating system fingerprinting techniques, and nmap has grown into the most feature-rich, free, stack-scanning tool in existence. Like many tools available on the Internet, this one is just as useful whether you are wearing a white hat or a black hat. Looking in the fingerprints file of a recent version of nmap (2.53), I counted fingerprints for 465 different stacks, including operating systems, routers, printers, and other networked devices. The fingerprint file nmap-os-fingerprints provides some clues in the comments at the head of the file. There are nine tests, each with subparts, for identifying an operating system, although the order presented is a bit misleading. For a deeper look, you can try reading the source for the operating system fingerprinting module yourself, osscan.c, as well as running nmap --O and monitoring your network, so that you can see what it is doing. Besides getting a better understanding of nmap, this exercise will help you recognize it on your network and potentially give you ideas about how to block its actions. PEEKING ON NMAP For example, suppose you build nmap and run it against a known target while you capture the packets it outputs, as in nmap -p20-23 -O target. The -p20-23 limits the number of ports scanned (be sure you include an open port in this range), and the -O enables the operating system identification feature. Nmap (after a DNS lookup, if required) goes through four phases: Ping, port scan, characteristics collection, and sequence number analysis. Nmap begins with two tests: It first sends an ICMP Echo Request, or Ping, and then probes port 80 (the Web server port), just to see if the target is alive. These initial probes can be disabled or modified with nmap options, so they are not very reliable indicators of an active nmap scan. Next, nmap will scan for open and closed ports in the port 20-to-port 23 range. The target system in this example had port 23 (Telnet) open, which is important for the next phase of scanning. Once nmap has identified one open and one closed port, it can begin the actual operating system identification portion of its agenda. Nmap will send seven carefully crafted TCP packets to the target-four to the open port and three to a closed port-and then sniff the responses as they come back. The responses are parsed and stored in a format that matches the entries in the fingerprint file. Each of the packets has a different combination of TCP flags sent to tickle out an operating system-identifying response from the target. For example, the first packet to an open port includes only the SYN flag, typically used when opening a TCP connection. The second packet includes no flags (a null scan), and the third probe to the open port includes SYN, FIN, PUSH, and URGENT (SFPU) flags, an unusual, but not illegal, set of TCP flags. RFC 1025 calls this set of flags with 1 byte of data a Kamikaze packet. It was used in testing TCP/IP stacks. An open-source intrusion detection system called Snort uses this peculiar set of flags to detect nmap operating system scans. Another clue for identifying these probes is that the same TCP sequence number is used for all seven of these probes. Besides just sending packets with different TCP flag settings, these packets also include several TCP options, something that I had noticed previously in network monitor output but not really understood. Digging into the RFCs, I found several references to TCP options, but the most important for understanding what nmap does is RFC 1323 (see Resources). Also, Richard Stevens' book TCP/IP Illustrated, Volume I made interpreting the options easy. First, you can tell that options are present in a TCP header by examining the length field. This field normally has a value of 5, with a maximum value of 15. If it's greater than 5, some options are present at the end of the header. Options begin with a kind field that identifies the type of options. Only the No-Operation (NOP, an option used for padding) and the End-of-list options consist of a single byte. All other options consist of the kind, a length in bytes that includes the kind byte, and the actual data. The NOP aligns the data within options into 32-bit words, or to fill out a 32-bit word. The End of list always ends the options. Fyodor uses three options, plus a NOP for padding and the final end of list. The options are Window Scale, Maximum Segment Size, and Timestamp. Of these, Window Scale and Timestamp were added to TCP/IP in RFC 1323 to support high-speed networks with relatively long latency (or, to quote Van Jacobsen, "long, fat networks"). The Maximum Segment Size is normally used by the sender to avoid fragmentation. In nmap, the order of these options is significant, as the fingerprints record the order in which the target of the scan returns the options. These options should be sent first in an initial (SYN) packet, a rule that nmap ignores in several tests. Some TCP/IP stacks do not understand any options. Others will respond with some or all of the options, or even include additional ones. Nmap records these responses, along with other characteristics, such as where padding (NOP) is used; whether the Don't Fragment flag is set; what the size of the Window value is in the TCP header; and if the acknowledgement number is set to zero, incremented, or left unchanged from the sequence number sent. The Window is a flow control mechanism used in TCP connections. By sending a Window value, the sender tells the receiver how many bytes of data can be sent without receiving an acknowledgement. This keeps one system from sending data to an overloaded and slowly responding machine. Alternatively, an overloaded machine can also stop the flow of data by setting the Window value to zero. Although it may seem logical that all systems use the same Window value, the initial Window value varies enormously, from zero for some devices to 32,767 for Windows NT (Service Pack 3) and just one Linux kernel (2.1.76). In cases where several systems do have the same initial Window value, nmap uses other characteristics to select the OS type. Nmap also collects characteristics from a response sent to a closed UDP port (the response will be an ICMP Destination Unreachable, type Port Unreachable), as the target is supposed to include part of the packet that caused the ICMP message. The accuracy of the quoting of the packet in the ICMP response varies depending on the target stack, and this, too, can be used to identify an operating system. The final stage uses SYN packets to probe for initial sequence numbers. Nmap sends a couple of resets first to the open port, then sends six packets with just SYN set (the normal method for opening a TCP connection), followed each time with a reset (a TCP header with reset and ACK flags set, which aborts the connection). The sequence numbers in packets sent increase incrementally by one each time; this is abnormal behavior but is characteristic of sequence number collectors, such as rbone and the unpublished tool used to take down security specialist Tsutomo Shimomura's site on Christmas Day, 1994 (see "Source Address Spoofing," May 2000). Nmap collects the initial sequence numbers received from the target and looks for a pattern in the way they are incremented. Really old Unix systems still use a constant increment, while newer and more secure systems use a random increment. Newer Microsoft stacks use a time-dependent increment, which might make them vulnerable if they ran a Unix service such as rlogind (which is not the case). NMAP COUNTERMEASURES Nmap's accuracy relies on collecting a lot of information from the target's stack. Anything that tampers with this information will affect that accuracy. What might tamper with this? Firewalls do to a greater or lesser extent. Most stacks permit tuning some of the tested parameters, but only a small portion of them. Packet-filtering firewalls are ineffective. Firewalls that perform Network Address Translation (NAT) improve effectiveness by rewriting sequence numbers (not all of them do). Application gateways running on a firewall prevent nmap from identifying the target, as nmap will be probing the firewall's stack instead. Of course, the best defense is to prevent any scanning from penetrating your network in the first place, and promptly detecting unauthorized scanning when it takes place within the confines of your network. N Rik Farrow is an independent security consultant. His Web site, www.spirit.com, contains security links and information about network and computer security courses. He can be reached at [email protected]. Resources The home site for Network Map (nmap) is www.insecure.org/nmap/, at which nmap author Fyodor has a paper describing the techniques he uses for operating system detection. Go to www.insecure.org/nmap/nmap-fingerprinting-article.html. Also available is the source code for nmap. A new version of Ofir Arkin's paper on Internet Control Message Protocol (ICMP) used in scanning can be found at www.sys-security.com/archive/papers/ICMP_Scanning_v2.0.pdf. RFC 1323, which describes TCP extensions for high performance with two of the three TCP options used in operating system detection, is available at www.faqs.org/rfcs/rfc1323.html. You can find Snort, an intrusion detection scanner for low-volume networks, at www.snort.org. The Windows NT version of nmap is available at www.eEye.com/html/Databases/Software/nmapnt.html. Richard Stevens is author of TCP/IP Illustrated, Volume 1 from Addison-Wesley; of particular interest are pages 253 to 254. Source: Network Magazine