[tcpdump-workers] Huge latency increase libpcap-1.4.0 -> libpcap-1.5.3

[Steve Bourland]

On Fri, 14 Nov 2014, Guy Harris wrote:

>
> On Nov 14, 2014, at 2:08 PM, Steve Bourland <sbourland at swri.edu> wrote:
>
>> On Fri, 14 Nov 2014, Guy Harris wrote:
>>
>>> On Nov 14, 2014, at 1:17 PM, Steve Bourland <sbourland at swri.edu> wrote:
>>>
>>>> I have some programs that suffered terrible performance degradation 
>>>> when moving from Ubuntu 12.04 to Ubuntu 14.04.
>>>
>>> Are those programs capturing and processing network traffic, are they 
>>> sending packets and expecting to receive a response, or are they doing 
>>> something else?
>>
>> They are essentially servers, listening for specific packets and then 
>> building and sending the appropriate reply to the client.
>
> OK, that's the same case, from the point of view of using libpcap, as 
> the client case of sending packets and expecting to receive a response.
>
>>> Is the performance issue one of high latency or of dropped packets 
>>> when capturing?
>>
>> Sorry, the main concern I have is high latency,
>
> As I suspected.
>
>>> What timeout are you specifying with pcap_open_live() or 
>>> pcap_set_timeout()?
>>
>> In the test program I am using 1 msec as the timeout for 
>> pcap_open_live...but then have a selectable fd and am using a select 
>> call to let me know when a packet arrives.
>
> Well, 1 ms is the smallest that the timeout can get.
>
> For PF_PACKET sockets, which are what libpcap uses on Linux, the 
> TPACKET_V3 mechanism, which is the default in 1.5.0 and later and which 
> works better for the packet-capture case, doesn't work well for the 
> request-and-response application case.
>
> I would suggest that:
>
> 	if the version of libpcap with which you're building has the 
> pcap_set_immediate_mode() API, use pcap_create(), 
> pcap_set_immediate_mode(), and pcap_activate() rather than 
> pcap_open_live(), as libpcap falls back to TPACKET_V2 in immediate mode;
>
> 	otherwise, libpcap also doesn't support TPACKET_V3, so that's not 
> an issue - keep using pcap_open_live() (unless you're already using 
> pcap_create()/pcap_activate(), which first appeared in 1.0.0).
>
> This may require a configure script, if you're OK with building from 
> source different versions of the program for different machines, or some 
> dlopen()/dlsym() trickery if you want to make a binary that'll work with 
> both libpcap 1.5-and-later and earlier versions.

Initial testing on an Ubuntu 14.04 system showed that the pcap_create, 
pcap_set_immediate_mode, pcap_activate fixed the problems beautifully for 
me.  The simplified icmp "server" went from horrible latencies and losses 
to a mean round trip time of 298 usec (reported by ping on a 14.04 server 
system) and zero losses.  Thanks so much for you quick help, and in terms 
of the full program, I am very fortunate that it will be compiled and 
deployed on a per system basis so I don't need to worry about a generic 
binary and can sort out a configure script or Makefile trickery.  Thanks 
for answering within a few hours a question that has been in the back of 
my mind for months.