Linux Archive

Linux Archive (http://www.linux-archive.org/)
-   CentOS (http://www.linux-archive.org/centos/)
-   -   Slow DNS lookups on CentOS6 (http://www.linux-archive.org/centos/580702-slow-dns-lookups-centos6.html)

Alfred von Campe 09-27-2011 03:04 PM

Slow DNS lookups on CentOS6
 
Most of my desktops are still running CentOS5, but I have installed CentOS6 on a few of them. The users on those desktops are reporting that DNS lookups are slow, and from my brief tests, that does appear to be the case. After some googling, I found a suggestion to disable IPv6, but that didn't help. So I tried to figure out where the delay occurs using strace and ltrace, but that didn't help much. When running ltrace, the process I'm trying to trace (host, nslookup) just dies. And the output of strace does show a 3+ second delay, but I can't figure out what it's telling me:

10:46:25.317517 futex(0xd1ba94, FUTEX_WAKE_PRIVATE, 2147483647) = 0
10:46:25.317586 open("/usr/lib/locale/locale-archive", O_RDONLY|O_LARGEFILE) = 6
10:46:25.317635 fstat64(6, {st_mode=S_IFREG|0644, st_size=99158720, ...}) = 0
10:46:25.317702 mmap2(NULL, 2097152, PROT_READ, MAP_PRIVATE, 6, 0) = 0xb569d000
10:46:25.317754 mmap2(NULL, 1171456, PROT_READ, MAP_PRIVATE, 6, 0x19c) = 0xb557f000
10:46:25.317792 mmap2(NULL, 4096, PROT_READ, MAP_PRIVATE, 6, 0x1084) = 0xb557e000
10:46:25.317827 close(6) = 0
10:46:25.317904 futex(0xb76a5050, FUTEX_WAKE_OP_PRIVATE, 1, 1, 0xb76a504c, {FUTEX_OP_SET, 0, FUTEX_OP_CMP_GT,
10:46:25.317990 futex(0xb76a5018, FUTEX_WAKE_PRIVATE, 1) = 1
10:46:25.318037 rt_sigaction(SIGHUP, {0x141390, ~[RTMIN RT_1], 0}, NULL, 8) = 0
10:46:25.318145 rt_sigsuspend([]) = ? ERESTARTNOHAND (To be restarted)
10:46:28.510794 --- SIGTERM (Terminated) @ 0 (0) ---
10:46:28.510842 sigreturn() = ? (mask now [HUP INT TERM])
10:46:28.510926 futex(0xb76a5050, FUTEX_WAKE_OP_PRIVATE, 1, 1, 0xb76a504c, {FUTEX_OP_SET, 0, FUTEX_OP_CMP_GT,
10:46:28.511015 futex(0xb76a5018, FUTEX_WAKE_PRIVATE, 1) = 1
10:46:28.511070 futex(0xb76a5050, FUTEX_CMP_REQUEUE_PRIVATE, 1, 2147483647, 0xb76a5018, 18) = 1
10:46:28.511144 futex(0xb76a5018, FUTEX_WAKE_PRIVATE, 1) = 1
10:46:28.511256 futex(0xb769fbd8, FUTEX_WAIT, 2371, NULL) = -1 EAGAIN (Resource temporarily unavailable)
10:46:28.511316 brk(0x920a000) = 0x920a000

The CentOS5 systems on the same network with the same /etc/resolv.conf file do not have these delays. Has anyone else seen this or have some suggestions as to how to debug this?

Alfred

_______________________________________________
CentOS mailing list
CentOS@centos.org
http://lists.centos.org/mailman/listinfo/centos

John Hodrien 09-27-2011 03:29 PM

Slow DNS lookups on CentOS6
 
On Tue, 27 Sep 2011, Alfred von Campe wrote:

> Most of my desktops are still running CentOS5, but I have installed CentOS6
> on a few of them. The users on those desktops are reporting that DNS
> lookups are slow, and from my brief tests, that does appear to be the case.
> After some googling, I found a suggestion to disable IPv6, but that didn't
> help. So I tried to figure out where the delay occurs using strace and
> ltrace, but that didn't help much. When running ltrace, the process I'm
> trying to trace (host, nslookup) just dies. And the output of strace does
> show a 3+ second delay, but I can't figure out what it's telling me:
>
> The CentOS5 systems on the same network with the same /etc/resolv.conf file
> do not have these delays. Has anyone else seen this or have some
> suggestions as to how to debug this?

You probably want to do an strace -f host blah rather than a basic strace, or
I think you'll lose what's going on.

Are you running either sssd or nscd on your box?

I've done nothing special, and DNS seems fast enough.

jh
_______________________________________________
CentOS mailing list
CentOS@centos.org
http://lists.centos.org/mailman/listinfo/centos

Frank Cox 09-27-2011 05:05 PM

Slow DNS lookups on CentOS6
 
On Tue, 27 Sep 2011 11:04:42 -0400
Alfred von Campe wrote:

> Most of my desktops are still running CentOS5, but I have installed CentOS6
> on a few of them. The users on those desktops are reporting that DNS lookups
> are slow, and from my brief tests, that does appear to be the case.

Have you considered installing dnsmasq on those machines?

--
MELVILLE THEATRE ~ Real D 3D Digital Cinema ~ www.melvilletheatre.com
www.creekfm.com - FIFTY THOUSAND WATTS of POW WOW POWER!
_______________________________________________
CentOS mailing list
CentOS@centos.org
http://lists.centos.org/mailman/listinfo/centos

Alfred von Campe 09-27-2011 05:28 PM

Slow DNS lookups on CentOS6
 
On Sep 27, 2011, at 11:29, John Hodrien wrote:

> You probably want to do an strace -f host blah rather than a basic strace, or
> I think you'll lose what's going on.

Good point. Using -f doesn't show a 3+ second gap, but I still have no idea
why it's slow (3-5 seconds) compared to CentOS5, or why using ltrace causes
host or nslookup to crash. Can anybody else get ltrace to work with host?

> Are you running either sssd or nscd on your box?

Yes, I am running nscd, but I didn't think it cached DNS queries.

> I've done nothing special, and DNS seems fast enough.

And I have done nothing different than what I do in CentOS5, yet it is much
slower (by at least an order of magnitude).

On Sep 27, 2011, at 13:05, Frank Cox wrote:

> Have you considered installing dnsmasq on those machines?

No, this is in a corporate environment, and the queries that are slow are
for names outside of our domain (i.e., the Internet), so I don't think it
would help.

Any other ideas on how to diagnose the root cause?

Alfred

_______________________________________________
CentOS mailing list
CentOS@centos.org
http://lists.centos.org/mailman/listinfo/centos

Frank Cox 09-27-2011 05:53 PM

Slow DNS lookups on CentOS6
 
On Tue, 27 Sep 2011 13:28:19 -0400
Alfred von Campe wrote:

> > Have you considered installing dnsmasq on those machines?
>
> No, this is in a corporate environment, and the queries that are slow are
> for names outside of our domain (i.e., the Internet), so I don't think it
> would help.

Why do you think dnscache won't help? Caching is not restricted to your local
domain.

--
MELVILLE THEATRE ~ Real D 3D Digital Cinema ~ www.melvilletheatre.com
www.creekfm.com - FIFTY THOUSAND WATTS of POW WOW POWER!
_______________________________________________
CentOS mailing list
CentOS@centos.org
http://lists.centos.org/mailman/listinfo/centos

Alfred von Campe 09-27-2011 06:02 PM

Slow DNS lookups on CentOS6
 
On Sep 27, 2011, at 13:53, Frank Cox wrote:

> Why do you think dnscache won't help? Caching is not restricted to your local
> domain.

I guess I forgot to mention that only the first query is slow. If you repeat
the query, the response is fast, so it's already being cached somewhere. I
always assumed that's how DNS worked. So in order to test this "slowness", I
have to keep thinking of domains to look up.

Alfred

_______________________________________________
CentOS mailing list
CentOS@centos.org
http://lists.centos.org/mailman/listinfo/centos

Les Mikesell 09-27-2011 06:02 PM

Slow DNS lookups on CentOS6
 
On Tue, Sep 27, 2011 at 12:28 PM, Alfred von Campe <alfred@von-campe.com> wrote:
> On Sep 27, 2011, at 11:29, John Hodrien wrote:
>
>> You probably want to do an strace -f host blah rather than a basic strace, or
>> I think you'll lose what's going on.
>
> Good point. *Using -f doesn't show a 3+ second gap, but I still have no idea
> why it's slow (3-5 seconds) compared to CentOS5, or why using ltrace causes
> host or nslookup to crash. *Can anybody else get ltrace to work with host?

The usual reason for a delay is that you have more than one nameserver
specified in resolv.conf and the first one tried is down or
unreachable so you time out and retry. Try "dig @namserver hostname"
with each of the nameserver addresses to see if they are working or
just slow.

--
Les Mikesell
lesmikesell@gmail.com
_______________________________________________
CentOS mailing list
CentOS@centos.org
http://lists.centos.org/mailman/listinfo/centos

Les Mikesell 09-27-2011 06:08 PM

Slow DNS lookups on CentOS6
 
On Tue, Sep 27, 2011 at 1:02 PM, Alfred von Campe <alfred@von-campe.com> wrote:
>
>> Why do you think dnscache won't help? *Caching is not restricted to your local
>> domain.
>
> I guess I forgot to mention that only the first query is slow. *If you repeat
> the query, the response is fast, so it's already being cached somewhere. *I
> always assumed that's how DNS worked. *So in order to test this "slowness", I
> have to keep thinking of domains to look up.

DNS servers normally do cache, but clients don't. Are you running
named locally on each machine and pointing resolv.conf to localhost?
It wouldn't make much sense for a central DNS server to act
differently depending on whether a c5 or c6 client was first to ask
for a name lookup.

--
Les Mikesell
lesmikesell@gmail.com
_______________________________________________
CentOS mailing list
CentOS@centos.org
http://lists.centos.org/mailman/listinfo/centos

Michael Gliwinski 09-28-2011 01:36 PM

Slow DNS lookups on CentOS6
 
On Tuesday 27 Sep 2011 19:08:53 Les Mikesell wrote:
> On Tue, Sep 27, 2011 at 1:02 PM, Alfred von Campe <alfred@von-campe.com>
wrote:
> >> Why do you think dnscache won't help? Caching is not restricted to your
> >> local domain.
> >
> > I guess I forgot to mention that only the first query is slow. If you
> > repeat the query, the response is fast, so it's already being cached
> > somewhere. I always assumed that's how DNS worked. So in order to test
> > this "slowness", I have to keep thinking of domains to look up.
>
> DNS servers normally do cache, but clients don't. Are you running
> named locally on each machine and pointing resolv.conf to localhost?

He's running nscd which caches DNS (group 'hosts' in /etc/nscd.conf).


--
Michael Gliwinski
Henderson Group Information Services
9-11 Hightown Avenue, Newtownabby, BT36 4RT
Phone: 028 9034 3319

************************************************** ********************************************
The information in this email is confidential and may be legally privileged. It is intended solely for the addressee and access to the email by anyone else is unauthorised.
If you are not the intended recipient, any disclosure, copying, distribution or any action taken or omitted to be taken in reliance on it, is prohibited and may be unlawful.
When addressed to our clients, any opinions or advice contained in this e-mail are subject to the terms and conditions expressed in the governing client engagement leter or contract.
If you have received this email in error please notify support@henderson-group.com

John Henderson (Holdings) Ltd
Registered office: 9 Hightown Avenue, Mallusk, County Antrim, Northern Ireland, BT36 4RT.
Registered in Northern Ireland
Registration Number NI010588
Vat No.: 814 6399 12
************************************************** *******************************

_______________________________________________
CentOS mailing list
CentOS@centos.org
http://lists.centos.org/mailman/listinfo/centos


All times are GMT. The time now is 05:25 PM.

VBulletin, Copyright ©2000 - 2014, Jelsoft Enterprises Ltd.
Content Relevant URLs by vBSEO ©2007, Crawlability, Inc.