FAQ Search Today's Posts Mark Forums Read
» Video Reviews

» Linux Archive

Linux-archive is a website aiming to archive linux email lists and to make them easily accessible for linux users/developers.


» Sponsor

» Partners

» Sponsor

Go Back   Linux Archive > Redhat > Fedora Directory

 
 
LinkBack Thread Tools
 
Old 11-11-2010, 12:27 PM
Gerrard Geldenhuis
 
Default Slow response from server

Hi
We are getting a slow responses from one of our LDAP servers and I am not sure what is causing the problem I have run a logconv.pl -j and the following is interesting:
*
Connections Reset By Peer:*** 0
Resource Unavailable:******** 136
**** -* 136* (T1) Idle Timeout Exceeded***
*
We have a cache hit ratio of 99% and only using about 10% of the available cache size. I am leaning towards the problem being network related and maybe needing to set timeout values less aggresively on our clients for this particular host but was hoping that there might be some more checks and analysis that I can run to determine the health of the LDAP server.
*
I am working on an internal trouble shooting guide and could potentially move some of the generic information into the 389 wiki.
*
Regards


__________________________________________________ ______________________

In order to protect our email recipients, Betfair Group use SkyScan from

MessageLabs to scan all Incoming and Outgoing mail for viruses.



__________________________________________________ ______________________

--
389 users mailing list
389-users@lists.fedoraproject.org
https://admin.fedoraproject.org/mailman/listinfo/389-users
 
Old 11-12-2010, 03:32 PM
Rich Megginson
 
Default Slow response from server

Gerrard Geldenhuis wrote:
>
> Hi
>
> We are getting a slow responses from one of our LDAP servers and I am
> not sure what is causing the problem I have run a logconv.pl -j and
> the following is interesting:
>
>
>
> Connections Reset By Peer: 0
>
> Resource Unavailable: 136
>
> - 136 (T1) Idle Timeout Exceeded
>
>
>
> We have a cache hit ratio of 99% and only using about 10% of the
> available cache size. I am leaning towards the problem being network
> related and maybe needing to set timeout values less aggresively on
> our clients for this particular host but was hoping that there might
> be some more checks and analysis that I can run to determine the
> health of the LDAP server.
>
does logconv.pl -V show anything like unindexed searches, admin limit
exceeded, long operation times?
>
>
>
> I am working on an internal trouble shooting guide and could
> potentially move some of the generic information into the 389 wiki.
>
>
>
> Regards
>
>
> __________________________________________________ ______________________
> In order to protect our email recipients, Betfair Group use SkyScan from
> MessageLabs to scan all Incoming and Outgoing mail for viruses.
>
> __________________________________________________ ______________________
> ------------------------------------------------------------------------
>
> --
> 389 users mailing list
> 389-users@lists.fedoraproject.org
> https://admin.fedoraproject.org/mailman/listinfo/389-users

--
389 users mailing list
389-users@lists.fedoraproject.org
https://admin.fedoraproject.org/mailman/listinfo/389-users
 
Old 11-12-2010, 05:21 PM
Gerrard Geldenhuis
 
Default Slow response from server

> -----Original Message-----
> From: 389-users-bounces@lists.fedoraproject.org [mailto:389-users-
> bounces@lists.fedoraproject.org] On Behalf Of Rich Megginson
> Sent: 12 November 2010 16:32
> To: General discussion list for the 389 Directory server project.
> Subject: Re: [389-users] Slow response from server
>
> Gerrard Geldenhuis wrote:
> >
> > Hi
> >
> > We are getting a slow responses from one of our LDAP servers and I am
> > not sure what is causing the problem I have run a logconv.pl -j and
> > the following is interesting:
> >
>
> > Connections Reset By Peer: 0
> >
> > Resource Unavailable: 136
> >
> > - 136 (T1) Idle Timeout Exceeded
> >
> does logconv.pl -V show anything like unindexed searches, admin limit
> exceeded, long operation times?
> >
> >
> >

No admin limit exceeded or long operation times.

Stricly spoken we don't have unindexed searches but my test bash script caused quite a number of. We are seeing random timeouts to this specific server when doing a search like the following:
while true ; do ldapsearch -h ldapserver.company -ZZ -x -W -D "uid=johndoe,ou=people,dc=company" -b "dc=company" -L -y pwd ; sleep 0.5; done

Watching the performance figures in the console does not highlight any specific problems.

I am fairly certain that it is an internal network issue but I need to have proof which is why we are currently doing tcpdumps.

We have only seen the problem on one of our consumer servers. The only difference being the order of servers in the nsfarmserverurl. Changing the order seems to fix the problem but as it is intermittent we can't really say that conclusively.

Our chaining settings is quite aggressive though and might need further tuning.
nsbindconnectionslimit: 5
nsconcurrentoperationslimit: 5
nsconnectionlife: 130
nsbindtimeout: 3
nsbindretrylimit: 3
nsmaxresponsedelay: 3
nsmaxtestresponsedelay: 5

Any thoughts on those values? Should/Could some values be increased?

Best Regards

__________________________________________________ ______________________
In order to protect our email recipients, Betfair Group use SkyScan from
MessageLabs to scan all Incoming and Outgoing mail for viruses.

__________________________________________________ ______________________
--
389 users mailing list
389-users@lists.fedoraproject.org
https://admin.fedoraproject.org/mailman/listinfo/389-users
 
Old 11-12-2010, 05:45 PM
Rich Megginson
 
Default Slow response from server

Gerrard Geldenhuis wrote:
>> -----Original Message-----
>> From: 389-users-bounces@lists.fedoraproject.org [mailto:389-users-
>> bounces@lists.fedoraproject.org] On Behalf Of Rich Megginson
>> Sent: 12 November 2010 16:32
>> To: General discussion list for the 389 Directory server project.
>> Subject: Re: [389-users] Slow response from server
>>
>> Gerrard Geldenhuis wrote:
>>
>>> Hi
>>>
>>> We are getting a slow responses from one of our LDAP servers and I am
>>> not sure what is causing the problem I have run a logconv.pl -j and
>>> the following is interesting:
>>>
>>>
> >
>
>>> Connections Reset By Peer: 0
>>>
>>> Resource Unavailable: 136
>>>
>>> - 136 (T1) Idle Timeout Exceeded
>>>
>>>
>> does logconv.pl -V show anything like unindexed searches, admin limit
>> exceeded, long operation times?
>>
>>>
>>>
>
> No admin limit exceeded or long operation times.
>
> Stricly spoken we don't have unindexed searches but my test bash script caused quite a number of. We are seeing random timeouts to this specific server when doing a search like the following:
> while true ; do ldapsearch -h ldapserver.company -ZZ -x -W -D "uid=johndoe,ou=people,dc=company" -b "dc=company" -L -y pwd ; sleep 0.5; done
>
> Watching the performance figures in the console does not highlight any specific problems.
>
> I am fairly certain that it is an internal network issue but I need to have proof which is why we are currently doing tcpdumps.
>
You're using TLS - if you remove the -ZZ, do you still have the same
problem?
> We have only seen the problem on one of our consumer servers. The only difference being the order of servers in the nsfarmserverurl. Changing the order seems to fix the problem but as it is intermittent we can't really say that conclusively.
>
> Our chaining settings is quite aggressive though and might need further tuning.
> nsbindconnectionslimit: 5
> nsconcurrentoperationslimit: 5
> nsconnectionlife: 130
> nsbindtimeout: 3
> nsbindretrylimit: 3
> nsmaxresponsedelay: 3
> nsmaxtestresponsedelay: 5
>
> Any thoughts on those values? Should/Could some values be increased?
>
Not sure. Sounds like you're going about collecting data correctly.
> Best Regards
>
> __________________________________________________ ______________________
> In order to protect our email recipients, Betfair Group use SkyScan from
> MessageLabs to scan all Incoming and Outgoing mail for viruses.
>
> __________________________________________________ ______________________
> --
> 389 users mailing list
> 389-users@lists.fedoraproject.org
> https://admin.fedoraproject.org/mailman/listinfo/389-users
>

--
389 users mailing list
389-users@lists.fedoraproject.org
https://admin.fedoraproject.org/mailman/listinfo/389-users
 
Old 11-24-2010, 01:08 PM
Gerrard Geldenhuis
 
Default Slow response from server

> -----Original Message-----
> From: 389-users-bounces@lists.fedoraproject.org [mailto:389-users-
> bounces@lists.fedoraproject.org] On Behalf Of Rich Megginson
> Sent: 12 November 2010 18:45
> To: General discussion list for the 389 Directory server project.
> Subject: Re: [389-users] Slow response from server
>
> Gerrard Geldenhuis wrote:
> >> -----Original Message-----
> >> From: 389-users-bounces@lists.fedoraproject.org [mailto:389-users-
> >> bounces@lists.fedoraproject.org] On Behalf Of Rich Megginson
> >> Sent: 12 November 2010 16:32
> >> To: General discussion list for the 389 Directory server project.
> >> Subject: Re: [389-users] Slow response from server
> >>
> >> Gerrard Geldenhuis wrote:
> >>
> >>> Hi
> >>>
> >>> We are getting a slow responses from one of our LDAP servers and I
> >>> am not sure what is causing the problem I have run a logconv.pl -j
> >>> and the following is interesting:
> >>>
> >>>
> > >
> >
> >>> Connections Reset By Peer: 0
> >>>
> >>> Resource Unavailable: 136
> >>>
> >>> - 136 (T1) Idle Timeout Exceeded
> >>>
> >>>
> >> does logconv.pl -V show anything like unindexed searches, admin limit
> >> exceeded, long operation times?
> >>
> >>>
> >>>
> >
> > No admin limit exceeded or long operation times.
> >
> > Stricly spoken we don't have unindexed searches but my test bash script
> caused quite a number of. We are seeing random timeouts to this specific
> server when doing a search like the following:
> > while true ; do ldapsearch -h ldapserver.company -ZZ -x -W -D
> > "uid=johndoe,ou=people,dc=company" -b "dc=company" -L -y pwd ; sleep
> > 0.5; done
> >
> > Watching the performance figures in the console does not highlight any
> specific problems.
> >
> > I am fairly certain that it is an internal network issue but I need to have
> proof which is why we are currently doing tcpdumps.
> >
> You're using TLS - if you remove the -ZZ, do you still have the same
> problem?

We are seeing the same problem with or without TLS based searches. At the moment I am not seeing the error...
There is no retransmits or packets errors so most likely not a networking issue. I have compared the config between this non-working server and another working server and there is very little difference between them. The normal timestamp change differences and order of some entries but nothing else.

Any suggestions on where to look next...?

To refresh what the thread was about. We are seeing timeouts against a 389 server on occasion when doing a very simple bind. The servers is a provider server with chaining configured. Password policy is configured to be global.

Regards



__________________________________________________ ______________________
In order to protect our email recipients, Betfair Group use SkyScan from
MessageLabs to scan all Incoming and Outgoing mail for viruses.

__________________________________________________ ______________________
--
389 users mailing list
389-users@lists.fedoraproject.org
https://admin.fedoraproject.org/mailman/listinfo/389-users
 
Old 11-24-2010, 01:36 PM
Gerrard Geldenhuis
 
Default Slow response from server

Interesting statistics when running ldclt...

On the problematic server I see:
ldclt[16390]: Average rate: 7.10/thr ( 7.10/sec), total: 71
ldclt[16390]: Average rate: 6.80/thr ( 6.80/sec), total: 68
ldclt[16390]: Average rate: 6.90/thr ( 6.90/sec), total: 69
ldclt[16390]: Average rate: 7.00/thr ( 7.00/sec), total: 70

and on the functioning server I see:
ldclt[16420]: Average rate: 1397.00/thr (1397.00/sec), total: 13970
ldclt[16420]: Average rate: 1336.70/thr (1336.70/sec), total: 13367
ldclt[16420]: Average rate: 1387.20/thr (1387.20/sec), total: 13872
ldclt[16420]: Average rate: 1387.80/thr (1387.80/sec), total: 13878
ldclt[16420]: Average rate: 1332.70/thr (1332.70/sec), total: 13327


clearly one server is much more capable than the other.... what could be the cause of that?

They both have identical hardware and were configured identical as far as I am aware.

Regards

> -----Original Message-----
> From: 389-users-bounces@lists.fedoraproject.org [mailto:389-users-
> bounces@lists.fedoraproject.org] On Behalf Of Gerrard Geldenhuis
> Sent: 24 November 2010 14:09
> To: 'General discussion list for the 389 Directory server project.'
> Subject: Re: [389-users] Slow response from server
>
> > -----Original Message-----
> > From: 389-users-bounces@lists.fedoraproject.org [mailto:389-users-
> > bounces@lists.fedoraproject.org] On Behalf Of Rich Megginson
> > Sent: 12 November 2010 18:45
> > To: General discussion list for the 389 Directory server project.
> > Subject: Re: [389-users] Slow response from server
> >
> > Gerrard Geldenhuis wrote:
> > >> -----Original Message-----
> > >> From: 389-users-bounces@lists.fedoraproject.org [mailto:389-users-
> > >> bounces@lists.fedoraproject.org] On Behalf Of Rich Megginson
> > >> Sent: 12 November 2010 16:32
> > >> To: General discussion list for the 389 Directory server project.
> > >> Subject: Re: [389-users] Slow response from server
> > >>
> > >> Gerrard Geldenhuis wrote:
> > >>
> > >>> Hi
> > >>>
> > >>> We are getting a slow responses from one of our LDAP servers and I
> > >>> am not sure what is causing the problem I have run a logconv.pl -j
> > >>> and the following is interesting:
> > >>>
> > >>>
> > > >
> > >
> > >>> Connections Reset By Peer: 0
> > >>>
> > >>> Resource Unavailable: 136
> > >>>
> > >>> - 136 (T1) Idle Timeout Exceeded
> > >>>
> > >>>
> > >> does logconv.pl -V show anything like unindexed searches, admin
> > >> limit exceeded, long operation times?
> > >>
> > >>>
> > >>>
> > >
> > > No admin limit exceeded or long operation times.
> > >
> > > Stricly spoken we don't have unindexed searches but my test bash
> > > script
> > caused quite a number of. We are seeing random timeouts to this
> > specific server when doing a search like the following:
> > > while true ; do ldapsearch -h ldapserver.company -ZZ -x -W -D
> > > "uid=johndoe,ou=people,dc=company" -b "dc=company" -L -y pwd ;
> sleep
> > > 0.5; done
> > >
> > > Watching the performance figures in the console does not highlight
> > > any
> > specific problems.
> > >
> > > I am fairly certain that it is an internal network issue but I need
> > > to have
> > proof which is why we are currently doing tcpdumps.
> > >
> > You're using TLS - if you remove the -ZZ, do you still have the same
> > problem?
>
> We are seeing the same problem with or without TLS based searches. At the
> moment I am not seeing the error...
> There is no retransmits or packets errors so most likely not a networking
> issue. I have compared the config between this non-working server and
> another working server and there is very little difference between them.
> The normal timestamp change differences and order of some entries but
> nothing else.
>
> Any suggestions on where to look next...?
>
> To refresh what the thread was about. We are seeing timeouts against a 389
> server on occasion when doing a very simple bind. The servers is a provider
> server with chaining configured. Password policy is configured to be global.
>
> Regards
>
>
>
> __________________________________________________ _________________
> _____
> In order to protect our email recipients, Betfair Group use SkyScan from
> MessageLabs to scan all Incoming and Outgoing mail for viruses.
>
> __________________________________________________ _________________
> _____
> --
> 389 users mailing list
> 389-users@lists.fedoraproject.org
> https://admin.fedoraproject.org/mailman/listinfo/389-users

__________________________________________________ ______________________
In order to protect our email recipients, Betfair Group use SkyScan from
MessageLabs to scan all Incoming and Outgoing mail for viruses.

__________________________________________________ ______________________
--
389 users mailing list
389-users@lists.fedoraproject.org
https://admin.fedoraproject.org/mailman/listinfo/389-users
 
Old 11-29-2010, 05:15 PM
Rich Megginson
 
Default Slow response from server

On 11/24/2010 07:36 AM, Gerrard Geldenhuis wrote:
> Interesting statistics when running ldclt...
>
> On the problematic server I see:
> ldclt[16390]: Average rate: 7.10/thr ( 7.10/sec), total: 71
> ldclt[16390]: Average rate: 6.80/thr ( 6.80/sec), total: 68
> ldclt[16390]: Average rate: 6.90/thr ( 6.90/sec), total: 69
> ldclt[16390]: Average rate: 7.00/thr ( 7.00/sec), total: 70
>
> and on the functioning server I see:
> ldclt[16420]: Average rate: 1397.00/thr (1397.00/sec), total: 13970
> ldclt[16420]: Average rate: 1336.70/thr (1336.70/sec), total: 13367
> ldclt[16420]: Average rate: 1387.20/thr (1387.20/sec), total: 13872
> ldclt[16420]: Average rate: 1387.80/thr (1387.80/sec), total: 13878
> ldclt[16420]: Average rate: 1332.70/thr (1332.70/sec), total: 13327
>
>
> clearly one server is much more capable than the other.... what could be the cause of that?
I don't know.
> They both have identical hardware and were configured identical as far as I am aware.
Then there must be some difference you are not aware of.
> Regards
>
>> -----Original Message-----
>> From: 389-users-bounces@lists.fedoraproject.org [mailto:389-users-
>> bounces@lists.fedoraproject.org] On Behalf Of Gerrard Geldenhuis
>> Sent: 24 November 2010 14:09
>> To: 'General discussion list for the 389 Directory server project.'
>> Subject: Re: [389-users] Slow response from server
>>
>>> -----Original Message-----
>>> From: 389-users-bounces@lists.fedoraproject.org [mailto:389-users-
>>> bounces@lists.fedoraproject.org] On Behalf Of Rich Megginson
>>> Sent: 12 November 2010 18:45
>>> To: General discussion list for the 389 Directory server project.
>>> Subject: Re: [389-users] Slow response from server
>>>
>>> Gerrard Geldenhuis wrote:
>>>>> -----Original Message-----
>>>>> From: 389-users-bounces@lists.fedoraproject.org [mailto:389-users-
>>>>> bounces@lists.fedoraproject.org] On Behalf Of Rich Megginson
>>>>> Sent: 12 November 2010 16:32
>>>>> To: General discussion list for the 389 Directory server project.
>>>>> Subject: Re: [389-users] Slow response from server
>>>>>
>>>>> Gerrard Geldenhuis wrote:
>>>>>
>>>>>> Hi
>>>>>>
>>>>>> We are getting a slow responses from one of our LDAP servers and I
>>>>>> am not sure what is causing the problem I have run a logconv.pl -j
>>>>>> and the following is interesting:
>>>>>>
>>>>>>
>>>> >
>>>>
>>>>>> Connections Reset By Peer: 0
>>>>>>
>>>>>> Resource Unavailable: 136
>>>>>>
>>>>>> - 136 (T1) Idle Timeout Exceeded
>>>>>>
>>>>>>
>>>>> does logconv.pl -V show anything like unindexed searches, admin
>>>>> limit exceeded, long operation times?
>>>>>
>>>>>>
>>>> No admin limit exceeded or long operation times.
>>>>
>>>> Stricly spoken we don't have unindexed searches but my test bash
>>>> script
>>> caused quite a number of. We are seeing random timeouts to this
>>> specific server when doing a search like the following:
>>>> while true ; do ldapsearch -h ldapserver.company -ZZ -x -W -D
>>>> "uid=johndoe,ou=people,dc=company" -b "dc=company" -L -y pwd ;
>> sleep
>>>> 0.5; done
>>>>
>>>> Watching the performance figures in the console does not highlight
>>>> any
>>> specific problems.
>>>> I am fairly certain that it is an internal network issue but I need
>>>> to have
>>> proof which is why we are currently doing tcpdumps.
>>> You're using TLS - if you remove the -ZZ, do you still have the same
>>> problem?
>> We are seeing the same problem with or without TLS based searches. At the
>> moment I am not seeing the error...
>> There is no retransmits or packets errors so most likely not a networking
>> issue. I have compared the config between this non-working server and
>> another working server and there is very little difference between them.
>> The normal timestamp change differences and order of some entries but
>> nothing else.
>>
>> Any suggestions on where to look next...?
>>
>> To refresh what the thread was about. We are seeing timeouts against a 389
>> server on occasion when doing a very simple bind. The servers is a provider
>> server with chaining configured. Password policy is configured to be global.
>>
>> Regards
>>
>>
>>
>> __________________________________________________ _________________
>> _____
>> In order to protect our email recipients, Betfair Group use SkyScan from
>> MessageLabs to scan all Incoming and Outgoing mail for viruses.
>>
>> __________________________________________________ _________________
>> _____
>> --
>> 389 users mailing list
>> 389-users@lists.fedoraproject.org
>> https://admin.fedoraproject.org/mailman/listinfo/389-users
> __________________________________________________ ______________________
> In order to protect our email recipients, Betfair Group use SkyScan from
> MessageLabs to scan all Incoming and Outgoing mail for viruses.
>
> __________________________________________________ ______________________
> --
> 389 users mailing list
> 389-users@lists.fedoraproject.org
> https://admin.fedoraproject.org/mailman/listinfo/389-users

--
389 users mailing list
389-users@lists.fedoraproject.org
https://admin.fedoraproject.org/mailman/listinfo/389-users
 

Thread Tools




All times are GMT. The time now is 01:14 PM.

VBulletin, Copyright ©2000 - 2014, Jelsoft Enterprises Ltd.
Content Relevant URLs by vBSEO ©2007, Crawlability, Inc.
Copyright 2007 - 2008, www.linux-archive.org