fuzzing udhcpd: a hacky approach

• Introduction
• Code Review
• Creating the Harness
• Generating Testcases
• Resolving Issues
• Fuzzing!
• Summary
• References

Introduction

I’ve been looking at a router on and off for a little over a year now and recently began looking at fuzzing some of the code for exposed services, one of them being DHCP. As DHCP is reachable pre-auth, I was interested in seeing whether there may be any bugs that could be triggered via DHCP messages. Specifically, I wanted to use AFL to fuzz the functions that parsed/handled DHCP client messages. For the first run, I focused on DHCPDISCOVER messages.

This post will go over an idea I had for how to create a custom fuzzing harness and the steps I took to get everything working. This includes:

• Create a fuzzing harness from the udhcpd server code using AFL’s LLVM persistent mode fuzzing feature
• Generate a fuzzing corpus using the udhcpc client code to produce dhcpMessage data and write it to files

Disclaimer: this post contains 0 0days. Sorry!

A Hacky(?) Approach

Before I could get started I needed to figure out how to create a harness that I could use to fuzz the target functions with AFL, which isn’t designed to do network-based fuzzing. There are forks of AFL that do support network fuzzing, but I hadn’t looked into them much and I had an idea for how I might be able to do it that would allow me to remove the socket binding entirely.

The idea was basically this:

• Create a modified version of the udhcp client code (udhcpc) to generate valid DHCP message packets and write the raw bytes of the data structures to files
• Create a modified version of the udhcp server code (udhcpd) that reads packet data from the output files instead of from the network and then enter a fuzzing loop where that seed data is continuously mutated and resubmitted for fuzzing

I chose to use AFL’s persistent mode feature since it would allow me to target only the code used to handle DHCPDISCOVER messages and would improve overall performance. This also fit perfectly for my idea since persistent mode does almost exactly what I wanted (enter a fuzzing loop to target specific functions). You can learn more about persistent mode here.

Dev Setup

Below are the details for the working environment and tools I used throughout the development of the project:

• OS: Ubuntu Server 20.04 VM
• Specs: 8 cores, 16Gb RAM (you definitely don’t need this to just build the code but it helps for the actual fuzzing part!)
• Tools: Docker, vim, AFL++

For simplicity’s sake, I chose to use the AFL++ docker container rather than go through the setup needed to install the necessary LLVM/Clang versions required for AFL’s persistent mode. This offers a fully functional working environment where you can easily mount your target source code and compile with the desired instrumentation.

docker run -ti -v $PWD:/src aflplusplus/aflplusplus

About udhcp

This application is now built directly into Busybox, but used to be distributed as a standalone application that provides both server and client functionality. The router in question is running a modified version of udhcpd version 0.9.8. I was able to get the source code for the modified version but couldn’t get it to build so I just went with the vanilla source downloaded from here.

Code Review

I began by reviewing the code in dhcpd.c to identify where handling of DHCPDISCOVER messages was being done and to get a feel for how the code worked. dhcpd.c is a relatively small file (less than 300 LOC) and the main function contains entry points for handling every type of DHCP client message.

The code begins by declaring and initializing some variables and then loading the server configuration from a config file:

memset(&server_config, 0, sizeof(struct server_config_t));
	
	if (argc < 2)
		read_config(DHCPD_CONF_FILE);
	else read_config(argv[1]);

After this the code does some more initialization of vars and server config values from the parsed config file and then enters the main server loop that handles receiving packets for processing. The capture of packets is done by get_packet() in the following code, which parses the packet into a dhcpMessage struct called packet:

if ((bytes = get_packet(&packet, server_socket)) < 0) { /* this waits for a packet - idle */
			if (bytes == -1 && errno != EINTR) {
				DEBUG(LOG_INFO, "error on read, %s, reopening socket", strerror(errno));
				close(server_socket);
				server_socket = -1;
			}
			continue;
		}

Immediately after this, the code parses the DHCP message type from packet and saves it to state:

if ((state = get_option(&packet, DHCP_MESSAGE_TYPE)) == NULL) {
			DEBUG(LOG_ERR, "couldn't get option from packet, ignoring");
			continue;
		}

The value saved to state is then used in a switch statement with cases for handling each type of DHCP message. This is where the handling for DHCPDISCOVER is defined, which calls a single function, sendOffer(), passing in packet as an argument:

switch (state[0]) {
		case DHCPDISCOVER:
			DEBUG(LOG_INFO,"received DISCOVER");
			
			if (sendOffer(&packet) < 0) {
				LOG(LOG_ERR, "send OFFER failed");
			}
			break;
		<--- snip --->
}

With all of this in mind, I made the following determinations:

• The testcase data would need to be read/parsed into a dhcpMessage struct so that it can be passed to the target function
• The target function would be sendOffer()
• The best place to insert the fuzzing loop would be right before entering the switch block that checks the DHCP message type

Creating the Harness

To get started, I created a copy of the original dhcpd.c file that I would use to create the harness.

Server Configuration and Initialization

Most of the original code that handles the reading of the config file, initialization of the server_config global structure, memory allocation for leases, and reading of the network interface for more server_config initialization was left as-is except for some minor changes (shown here):

  // read config file into server_config global
	memset(&server_config, 0, sizeof(struct server_config_t));
	read_config("./test-udhcpd.conf"); // edit: hardcode the config file name

	pid_fd = pidfile_acquire(server_config.pidfile);
	pidfile_write_release(pid_fd);
  server_config.lease = LEASE_TIME; // edit: always use default lease time
	
	// allocate mem for leases and read from lease file
	leases = malloc(sizeof(struct dhcpOfferedAddr) * server_config.max_leases);
	memset(leases, 0, sizeof(struct dhcpOfferedAddr) * server_config.max_leases);
	read_leases(server_config.lease_file);

	if (read_interface(server_config.interface, &server_config.ifindex,
			   &server_config.server, server_config.arp) < 0)
		return 1;

As shown above, the harness is hardcoded to search for a config file named test-udhcpd.conf in the local working directory. Below is a minimal configuration file used for the harness (the start/end values and the interface must match the actual network configuration of the host where the harness will run):

start 		172.0.2.10	  #default: 192.168.0.20
end		    172.0.2.20	  #default: 192.168.0.254
interface	eth0		      #default: eth0
opt	dns	  192.168.10.2 192.168.10.10
option subnet	255.255.255.0
opt	router	172.0.2.1
option	  domain	local
option	  lease	864000		# 10 days of seconds

Inserting the Persistent Mode Fuzzing Loop

I then replaced the infinite server loop (while (1)) with the main AFL fuzzing loop (_while (__AFL_LOOP())). Since testcase data was going to be passed in via input files, there was no need to have the server bind to a socket and actually listen for packets, so I removed the majority of the server networking code from the start of the main loop.

I replaced this code with the code needed to reset state, read the testcase data into a dhcpMessage structure, and call the target function. Specifically, this code does the following:

• Zeroing out the leases buffer and reinitializing it by calling read_leases() to ensure each loop starts with the same state
• After a sanity check of the testcase buffer size, the dhcpMessage struct that will hold the testcase data is zeroed out and the data is copied to the fuzz_packet struct using memcpy.
• The DHCP message type is then read from the fuzz_packet struct and saved to state . If no valid message type is found we continue to next testcase; otherwise the original switch block from the server code is entered with a single case to match DHCPDISCOVER messages.
• The fuzz data in fuzz_packet is passed to sendOffer()

The final code within the fuzzing loop is shown below:

int main(int argc, char *argv[])
{	

  [... SNIP ...]

	// START AFL LOOP
		__AFL_INIT();

		unsigned char *aflbuf = __AFL_FUZZ_TESTCASE_BUF; // afl will automatically read the file data into this buf
	
		while (__AFL_LOOP(10000000)) {
			// reset leases at the start of each run
			memset(leases, 0, sizeof(struct dhcpOfferedAddr) * server_config.max_leases);
			read_leases(server_config.lease_file);
			
			// check len is good
			ssize_t afllen = __AFL_FUZZ_TESTCASE_LEN;
			if (afllen > sizeof(struct dhcpMessage)) continue;
	
			struct dhcpMessage fuzz_packet;
			memset(&fuzz_packet, 0, sizeof(struct dhcpMessage));
			memcpy(&fuzz_packet, aflbuf, sizeof(struct dhcpMessage));
	
			if ((state = get_option(&fuzz_packet, DHCP_MESSAGE_TYPE)) == NULL) {
				continue;
			}

			lease = find_lease_by_chaddr(fuzz_packet.chaddr);
			
			// Pass the testcase data to the target functions based on message type
			switch (state[0]) {
				case DHCPDISCOVER:
					if (sendOffer(&fuzz_packet) < 0) {
						printf("send OFFER failed");
					}
					break;
			default:
				continue;
		}
	}
	return 0;
}

Generating Testcases

Next, I had to figure out a way to generate testcases and write them out to files to make them readable by AFL, since this is how it’s designed to read inputs. To accomplish this I used the DHCP client code included with udhcp to generate valid dhcpMessage structures and write the raw bytes out to disk.

Creating Packets

The majority of the client components are located in dhcpc.c and clientpacket.c. Upon reviewing the code, I found that dhcpMessage objects are initialized with the function init_packet(); this function is called by other higher-level functions used to create specific types of DHCP messages (send_discover(), send_renew(), etc). Below is an example of one of these functions, used to send DHCP Discover messages:

/* Broadcast a DHCP discover packet to the network, with an optionally requested IP */
int send_discover(unsigned long xid, unsigned long requested)
{
	struct dhcpMessage packet;

	init_packet(&packet, DHCPDISCOVER);
	packet.xid = xid;
	if (requested)
		add_simple_option(packet.options, DHCP_REQUESTED_IP, requested);

	add_requests(&packet);
	LOG(LOG_DEBUG, "Sending discover...");
	return raw_packet(&packet, INADDR_ANY, CLIENT_PORT, INADDR_BROADCAST, 
				SERVER_PORT, MAC_BCAST_ADDR, client_config.ifindex);
}

This actually made things incredibly easy — all I had to do was make modified copies of these higher-level functions that I could use to generate mutated packets and write them to files. I made a new file in the source directory for the testcase generator and copied over the relevant code from dhcpc.c and clientpacket.c . I won’t post the code for every modified function to keep things readable but below is an example of the modified version of the send_discover() function showed in the code block above.

/* Broadcast a DHCP discover packet to the network, with an optionally requested IP */
static void make_discover(struct dhcpMessage *packet, unsigned long xid, unsigned long requested)
{
	printf("debug - discover start\n");
	init_packet_fuzz(packet, DHCPDISCOVER);
	packet->xid = xid;
	if (requested)
		printf("requested ip\n");
		add_simple_option(packet->options, DHCP_REQUESTED_IP, requested);

	add_requests_fz(packet);
	printf("debug - add_req finished\n");

	// EDIT: Removed code the sends the packet over the network
  // return raw_packet(&packet, INADDR_ANY, CLIENT_PORT, INADDR_BROADCAST, 
	//			SERVER_PORT, MAC_BCAST_ADDR, client_config.ifindex);
}

For each of these functions, I removed the calls to raw_packet() that were present in the originals; this function takes care of adding TCP/UDP headers to the packet and actually sending the packet over the network. Apart from the fact that we don’t actually need to send any packets since we intend to write the out to files, we also don’t need to to add TCP/UDP headers since the target function sendOffer() expects to receive a parsed dhcpMessage struct which has already been read from the socket and stripped of the encapsulating layer. Instead, each of these functions just returns after initializing the packet arg (this is passed by reference, so the calling function has the pointer to packet).

Additionally, I also created a couple of customized versions of the init_packet() function that would allow me to easily pass in custom values for fields I might want to focus on. For example, I created a version that would allow me to pass in a custom length value for the vendor_id that’s added to the packet using add_option_string() :

static void init_packet_long_vendor_str(struct dhcpMessage *packet, char type, unsigned int fuzz_length)
{
	struct vendor  {
		char vendor, length;
		char str[sizeof("udhcp")];
	} vendor_id = { DHCP_VENDOR,  (sizeof("A") * fuzz_length) - 1, "udhcp"};
	
	init_header(packet, type);
	memcpy(packet->chaddr, client_config.arp, 6);
	add_option_string(packet->options, client_config.clientid);
	if (client_config.hostname) add_option_string(packet->options, client_config.hostname);
	add_option_string(packet->options, (unsigned char *) &vendor_id);
}

Handling the Client Configuration

With the modified functions in place, I then moved onto main(), which is where I would call these customized functions to create the testcase files. The original main() begins by parsing command-line arguments for configuration options and initializing a global client_config struct. As is probably obvious, this contains the various configuration values used by the DHCP client, and a lot of the code reads from this data structure.

I wanted to be sure this got properly initialized but didn’t want to have to pass in commandline arguments, so I created the following function using the same code from the original main() but modified to only initialize the important config values needed for my purposes.

static void create_fuzz_client_config(char *hostname, char *interface, char *client_id)
{
		// set the client ID
    int len = strlen(client_id) > 255 ? 255 : strlen(client_id);
    if (client_config.clientid) free(client_config.clientid);
    client_config.clientid = xmalloc(len + 2);
    client_config.clientid[OPT_CODE] = DHCP_CLIENT_ID;
    client_config.clientid[OPT_LEN] = len;
    client_config.clientid[OPT_DATA] = '\0';
    strncpy(client_config.clientid + OPT_DATA, client_id, len);

		// set the hostname
    len = strlen(hostname) > 255 ? 255 : strlen(hostname);
    if (client_config.hostname) free(client_config.hostname);
    client_config.hostname = xmalloc(len + 2);
    client_config.hostname[OPT_CODE] = DHCP_HOST_NAME;
    client_config.hostname[OPT_LEN] = len;
    strncpy(client_config.hostname + 2, hostname, len);
		
		// set the network interface
    client_config.interface = interface;
}

Finally, I created a small function to handle the actual writing of the packet data to disk:

int write_packet_to_testcase_file(struct dhcpMessage *packet, char *type_prefix) {
    FILE *fd;
    char outname[64];
    sprintf(outname, "./%s_%ld", type_prefix, time(0));
    fd = fopen(outname, "wb");
    int res = fwrite(&packet, sizeof(struct dhcpMessage), 1, fd);
    printf("wrote packet data to file: '%s'\n", outname);
    fclose(fd);
    return res;
}

With all of these pieces in place, I was able to replace almost all of the code from the original main() with calls to the custom packet functions to generate packets and then writing the packet data to output files. Below is a snippet of some of the testcases I created:

    // normal discover packet
	  struct dhcpMessage discover_1;
    printf("[+] creating testcase: discover 1\n");
    make_discover(&discover_1, random_xid(), requested_ip);
    write_packet_to_testcase_file(&discover_1, "discover_1");

	  // discover with long vendor_id
	  struct dhcpMessage discover_long_vendor;
    printf("[+] creating testcase: discover long\n");
    init_packet_long_vendor_str(&discover_long_vendor, DHCPDISCOVER, 255);
    write_packet_to_testcase_file(&discover_long_vendor, "discover_long");

	  // discover with tweaked vendor_id
	  struct dhcpMessage discover_mut;
    printf("[+] creating testcase: discover custom vendor\n");
    make_discover(&discover_long_vendor, random_xid(), requested_ip);
    add_custom_vendor_id(&discover_mut, "AAAAAAAAAAAAAAAA", 40);
    write_packet_to_testcase_file(&discover_mut, "discover_cvendor");

I created a modified Makefile from the original to only build the client components I needed and was able to successfully compile the application. I ran this was and produced valid testcase files that I could feed to the harness.

Note: Testcases Should Be Generated on the Fuzzing Node

One consequence of this approach to generating the testcases was the testcase files needed to be generated on the same host where the fuzz job would run. This is due to the fact that the application reads from the configured network interface and uses it to determine which IPs and MAC addresses it uses in various places when generating packets. While this likely doesn’t matter for the harness (since it won’t be sending packets or checking leases, doing arp pings, etc), it is likely that any crashes or interesting outputs will only be reproduceable against a vanilla version of udhcpd running on the same host where the testcases were generated running a compatible configuration, since that code will perform those checks and attempt to send packets. So, to keep things as consistent as possible, it’s best to do everything on the same machine.

This means that, ideally, the testcase generator should be configured to use one network interface while the harness/vanilla server uses another on the same host. This should ensure that when trying to reproduce issues the packets will appear to be coming from a MAC address that the server can know about/reach and that values will be compatible with the network configuration.

Resolving Issues

ARP Ping Timeouts

I was initially able to run afl-fuzz and get a job actually running but eventually noticed that I had made a typo in the config file where I defined the starting address range for the server. The typo resulted in an invalid IP address (1011.13.13.50) which I had missed since I couldn’t see output from the bin when running with AFL.

After fixing the typo and adding a valid address, I immediately ran into timeout issues when trying to run afl-fuzz again. A bit of troubleshooting later, I was able to isolate the timeouts as being caused by the following call chain:

• [sendOffer() → find_address() → check_ip() → arpping()]

This code path is taken when sending a DHCP offer to the client in order to find an IP within the lease range and check whether it is on the network. It does so by sending an ARP ping to the potential address — this was causing the timeouts as there were no hosts that would respond to this ARP message and the server was waiting for a response. Since there was no need to actually have the server send the packet, I modified find_address to return the IP it picks before it calls check_ip().

Preventing Response Packets

Another bit of code I thought was likely to slow things down and that wasn’t really needed was the networking code responsible for sending responses to clients. Since there would be no real network clients to receive the packets and I wasn’t interested in fuzzing the underlying network stack, I made the following changes to preserve as much code as possible while eliminating the actual network access:

• remove the socket binding code from packet.c:raw_packet() and have it return before sending the packet
• remove the socket binding code from packet.c:kernel_packet() have it return before sending the packet

This is a snippet of the code from raw_packet() showing where I removed the call to bind() and return early before the call to sendto() is made near the end of the function:

if ((fd = socket(PF_PACKET, SOCK_DGRAM, htons(ETH_P_IP))) < 0) {
		DEBUG(LOG_ERR, "socket call failed: %s", strerror(errno));
		return -1;
	}
	
	memset(&dest, 0, sizeof(dest));
	memset(&packet, 0, sizeof(packet));
	
	dest.sll_family = AF_PACKET;
	dest.sll_protocol = htons(ETH_P_IP);
	dest.sll_ifindex = ifindex;
	dest.sll_halen = 6;
	memcpy(dest.sll_addr, dest_arp, 6);
	
	// < -- removed call to bind() -- > 

	packet.ip.protocol = IPPROTO_UDP;
	packet.ip.saddr = source_ip;
	packet.ip.daddr = dest_ip;
	packet.udp.source = htons(source_port);
	packet.udp.dest = htons(dest_port);
	packet.udp.len = htons(sizeof(packet.udp) + sizeof(struct dhcpMessage)); /* cheat on the psuedo-header */
	packet.ip.tot_len = packet.udp.len;
	memcpy(&(packet.data), payload, sizeof(struct dhcpMessage));
	packet.udp.check = checksum(&packet, sizeof(struct udp_dhcp_packet));
	
	packet.ip.tot_len = htons(sizeof(struct udp_dhcp_packet));
	packet.ip.ihl = sizeof(packet.ip) >> 2;
	packet.ip.version = IPVERSION;
	packet.ip.ttl = IPDEFTTL;
	packet.ip.check = checksum(&(packet.ip), sizeof(packet.ip));

	// RETURN BEFORE CALLING sendto()
	close(fd);
	return 1;

	result = sendto(fd, &packet, sizeof(struct udp_dhcp_packet), 0, (struct sockaddr *) &dest, sizeof(dest));
	if (result <= 0) {
		DEBUG(LOG_ERR, "write on socket failed: %s", strerror(errno));
	}

Fuzzing!

With all of the important pieces complete, it was time to put it all together and start fuzzing! All of the commands below were run from within the AFL++ docker container.

Compiling the Testcase Generator & Creating the Corpus

After dropping to a shell in the container, I changed directories to the source directory for the testcase generator and ran make using the custom Makefile I created. This produced the binary mk-testcase which I then execute to generate the testcase files.

-> % sudo docker run --security-opt seccomp=unconfined -ti -v $PWD:/src aflplusplus/aflplusplus
[afl++ dda3efae1cdf] $ cd /src/corpus-gen/src/
[afl++ dda3efae1cdf] /src/corpus-gen/src/ $ make
gcc -c -DSYSLOG -W -Wall -Wstrict-prototypes -DVERSION='"0.9.8"' -g -DDEBUG dhcpc.c
dhcpc.c: In function 'main':
dhcpc.c:256:35: warning: pointer targets in passing argument 1 of 'strncpy' differ in signedness [-Wpointer-sign]
<--- SNIP --->
gcc  mk-testcases.o clientpacket.o script.o options.o socket.o packet.o pidfile.o -o mk-testcases

# Create the corpus using mk-testcase
[afl++ dda3efae1cdf] /src/corpus-gen/src/ $ mkdir corpus && cp mk-testcases corpus/
[afl++ dda3efae1cdf] /src/corpus-gen/src/ $ cd corpus/
[afl++ dda3efae1cdf] /src/corpus-gen/src/corpus $ ./mk-testcases && rm mk-testcase
adapter index 33
adapter hardware address 02:42:ac:11:00:02
[+] creating testcase: discover 1
<--- SNIP --->
[afl++ dda3efae1cdf] $ ls
discover_1_1654463092  discover_cvendor_1654463092  request_1654463092            request_cid_mod4_1654463092
discover_2_1654463092  discover_long_1654463092     request_cid_mod32_1654463092

Compiling the Harness

In the same shell session, I then changed over to the harness source directory and ran make there to produce the udhcpd-harness binary.

# Building the harness
[afl++ dda3efae1cdf] $ cd /src/harness1/src
[afl++ dda3efae1cdf] /src/harness1/src $ CC=afl-clang-fast make
afl-clang-fast -c -DSYSLOG -static -W -Wall -Wstrict-prototypes -DVERSION='"0.9.8"' -g -DDEBUG dhcpd.c
afl-cc++4.01a by Michal Zalewski, Laszlo Szekeres, Marc Heuse - mode: LLVM-PCGUARD
SanitizerCoveragePCGUARD++4.01a
[+] Instrumented 54 locations with no collisions (non-hardened mode) of which are 1 handled and 0 unhandled selects.
afl-clang-fast -c -DSYSLOG -static -W -Wall -Wstrict-prototypes -DVERSION='"0.9.8"' -g -DDEBUG arpping.c
<--- SNIP --->
afl-clang-fast -static dhcpd-harness.o arpping.o files.o leases.o serverpacket.o options.o socket.o packet.o pidfile.o -o udhcpd-harness

Fuzzing

With that done, I put everything together in a single directory and got everything ready for the run.

# Create a directory for the fuzzing run
[afl++ dda3efae1cdf] $ cd /src
[afl++ dda3efae1cdf] /src $ mkdir -p fuzz_run/outputs
[afl++ dda3efae1cdf] /src $ mkdir -p fuzz_run/inputs

# Copy over the harness, generated corpus, and server config file
[afl++ dda3efae1cdf] /src $ cp harness1/src/udhcpd-harness fuzz_run/
[afl++ dda3efae1cdf] /src $ cp corpus-gen/src/corpus/* fuzz_run/inputs/.
[afl++ dda3efae1cdf] /src $ cp harness1/test-udhcpd.conf fuzz_run/
[afl++ dda3efae1cdf] /src $ cd fuzz_run && ls
inputs  outputs  test-udhcpd.conf  udhcpd-harness

# start afl-fuzz
[afl++ dda3efae1cdf] /src/fuzz_run $ afl-fuzz -i inputs/ -o outputs/ -- ./udhcpd-harness

And it’s off!

Summary

Overall, this process was a great learning opportunity for concepts I’m hoping to be able to transfer over to larger/more complex codebases. Getting all of the pieces to work together was a challenge and porting over the necessary code while removing the unnecessary code was tedious and required spending a lot of time tracing execution paths, but all of this proved to be useful in the end. I’m glad I was able to validate my idea of using existing server/client code to create fuzzing harness and generate testcases but realize this approach probably becomes much more difficult with more complex targets.

I built a couple of copies of the harness using ASAN and some other features and spun of a few instances. As shown in the screenshot at the top of this post, eventually there were some crashes found in the ASAN versions. After some testing and creating another copy of the server code to use for reproducing crashes, I was actually only able to get an ASAN crash from one of those crash files. For now I haven’t dug into it much but may come back around with another post if anything interesting comes of it.

As you may be able to tell from this post, I’m far from being an expert at this stuff — in fact, quite the opposite. There may be some glaring issues with this approach and I haven’t spent enough time testing and validating everything to be completely sure I’m not doing something very wrong, so please don’t take anything I say here as coming from a source of expertise. But please do let me know about it :)

References

• Github repo for the harness+testcase generator
• AFL++ persistent mode
• udhcpd Source