[tcpdump-workers] TPACKET_V3 support added big packet delivery delay and/or affected timestamps

Guy Harris guy at alum.mit.edu
Thu Dec 4 15:01:30 EST 2014


On Dec 4, 2014, at 10:13 AM, Thomas Habets <thomas at habets.se> wrote:

> Actual behaviour:
> $ sudo sh -c "LD_PRELOAD=$HOME/opt/buggypcap/lib/libpcap.so ./arping 10.0.0.1"
> ARPING 10.0.0.1
> 60 bytes from 00:xx:xx:xx:xx:xx (10.0.0.1): index=0 time=52.633 msec
> 60 bytes from 00:xx:xx:xx:xx:xx (10.0.0.1): index=1 time=90.928 msec
> 60 bytes from 00:xx:xx:xx:xx:xx (10.0.0.1): index=2 time=115.183 msec
> 60 bytes from 00:xx:xx:xx:xx:xx (10.0.0.1): index=3 time=285.153 msec
> ^C
> --- 10.0.0.1 statistics ---
> 4 packets transmitted, 4 packets received,   0% unanswered
> 
> Expected (and received with 1.4.0):
> $ sudo sh -c "LD_PRELOAD=$HOME/opt/goodpcap/lib/libpcap.so ./arping 10.0.0.1"
> ARPING 10.0.0.1
> 60 bytes from 00:xx:xx:xx:xx:xx (10.0.0.1): index=0 time=817.060 usec
> 60 bytes from 00:xx:xx:xx:xx:xx (10.0.0.1): index=1 time=895.977 usec
> 60 bytes from 00:xx:xx:xx:xx:xx (10.0.0.1): index=2 time=759.840 usec
> 60 bytes from 00:xx:xx:xx:xx:xx (10.0.0.1): index=3 time=827.074 usec
> ^C
> --- 10.0.0.1 statistics ---
> 4 packets transmitted, 4 packets received,   0% unanswered
> 
> Notice the unit difference. About a hundred milliseconds vs about 800
> microseconds.

TPACKET_V3 does the same style of buffer as does BPF, so packets are *not* guaranteed to be delivered as soon as they arrive; instead, they buffer packets so that multiple packets are delivered in a batch.  See, for example:

	https://www.kernel.org/doc/Documentation/networking/packet_mmap.txt

This is different from TPACKET_V1 and TPACKET_V2.

If your program needs to have packets delivered as soon as they arrive, it should, if the version of libpcap with which it's being built has the pcap_set_immediate_mode() API, open the capture device by doing

	p = pcap_create(device, errbuf);
	if (p == NULL) {
		report failure, using errbuf;
		quit;
	}
	pcap_set_immediate_mode(p, 1);

	{set timeout, etc. using pcap_set_timeout(), pcap_set_promisc(), pcap_set_snaplen(), etc.}

	status = pcap_activate(p);
	if (status != 0) {
		report warning or error, using status and pcap_geterr();
		if it's an error, quit;
	}

> Questions:
> Is TPACKET V3 (and V2?) much slower than V1?

No, TPACKET_V3 is just different, with the buffering.

TPACKET_V1 and TPACKET_V2 don't do the same buffering.

> Can I disable them?

You can disable buffering with pcap_set_immediate_mode().  *On Linux*, that *happens* to do so by falling back to TPACKET_V2 on systems that have TPACKET_V3; on systems using BPF, it does so with a BIOCIMMEDIATE ioctl.

> Is this actual delay, or just a different way of measuring?

Since it's the same version of arping, and since the time stamps are from the system clock, not from the packets as provided by libpcap:

	https://github.com/ThomasHabets/arping/blob/arping-2.x/src/arping.c

it's clearly not a different way of measuring.



More information about the tcpdump-workers mailing list