I've got an old problem with clamd, which creates a bunch of threads.
Every so often the logs will show e.g.,

Jul 31 06:01:41 mx1 clamd[24070]: pthread_create failed
Jul 31 06:01:41 mx1 clamd[24070]: pthread_create failed
Jul 31 06:01:41 mx1 clamd[24070]: pthread_create failed
Jul 31 06:01:41 mx1 clamd[24070]: pthread_create failed

It doesn't cause any noticeable problems, so I've sort of left it alone
but tonight I dug in a little. The problem seems (somehow) related to
that box's hardening.

I'm using a test program that creates a bunch of threads and then just
kills them. On the box in question,

# uname -a
Linux mx1 3.4.2-hardened #1 SMP Wed Jul 11 13:41:57 EDT 2012 i686
Intel(R) Xeon(TM) CPU 3.06GHz GenuineIntel GNU/Linux
# ./pthread_test 25
Creating 25 threads
Created thread #0...
Created thread #1...
Created thread #2...
Created thread #3...
pthread_create failed: Resource temporarily unavailable

Disabling all paxctl protections helps, but doesn't allow me to get all
the way to 25. I tried doing the protections one-at-a-time; it doesn't
really help:

# paxctl -pemrxs pthread_test
# ./pthread_test 25
Creating 25 threads
Created thread #0...
Created thread #1...
Created thread #2...
Created thread #3...
Created thread #4...
Created thread #5...
Created thread #6...
Created thread #7...
Created thread #8...
Created thread #9...
pthread_create failed: Resource temporarily unavailable

I get nothing in my dmesg, which otherwise records most limit-based denials.

Is there some way I can troubleshoot this? It works on amd64 with the
same kernel hardening options.

On 31 Jul 2012 at 22:12, Michael Orlitzky wrote:

> I get nothing in my dmesg, which otherwise records most limit-based denials.
>
> Is there some way I can troubleshoot this? It works on amd64 with the
> same kernel hardening options.

an strace -f may help to see what exactly fails.

On 08/01/2012 06:56 AM, PaX Team wrote:
> On 31 Jul 2012 at 22:12, Michael Orlitzky wrote:
>
>> I get nothing in my dmesg, which otherwise records most limit-based denials.
>>
>> Is there some way I can troubleshoot this? It works on amd64 with the
>> same kernel hardening options.
>
> an strace -f may help to see what exactly fails.
>
>

Thanks, here are strace -f logs from both the hardened box (where it
fails) and a vanilla gentoo x86 VM (where it works).

On 1 Aug 2012 at 8:41, Michael Orlitzky wrote:

> Thanks, here are strace -f logs from both the hardened box (where it
> fails) and a vanilla gentoo x86 VM (where it works).

mmap2(NULL, 307200000, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS|MAP_STACK, -1, 0) = -1 ENOMEM (Cannot allocate memory)

this can fail for several reasons, not enough RAM (depends on how overcommit is set),
not enough address space (hardened/PIE and ASLR together change how big the holes in
the address space end up, SEGMEXEC halves the address space), etc.

On 08/01/12 09:08, PaX Team wrote:
> On 1 Aug 2012 at 8:41, Michael Orlitzky wrote:
>
>> Thanks, here are strace -f logs from both the hardened box (where it
>> fails) and a vanilla gentoo x86 VM (where it works).
>
> mmap2(NULL, 307200000, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS|MAP_STACK, -1, 0) = -1 ENOMEM (Cannot allocate memory)
>
> this can fail for several reasons, not enough RAM (depends on how overcommit is set),
> not enough address space (hardened/PIE and ASLR together change how big the holes in
> the address space end up, SEGMEXEC halves the address space), etc.
>
>

Hmm.. I think this indirectly solves the problem. I've got,

# cat /etc/security/limits.d/50-clamd.conf
#<domain> <type> <item> <value>
clamav - stack 512000

But it isn't taking effect:

# cat /proc/25394/limits | grep stack
Max stack size 307200000 307200000 bytes

So, clamd is likely running out of stack just like the test program. I
can probably figure that one out.

But, I'd ruled out the stack size limitation because resource oversteps
are supposed to be reported:

# cat /proc/config.gz | gunzip | grep GRKERNSEC_RESLOG
CONFIG_GRKERNSEC_RESLOG=y

I've got nothing logged, even after the failures.


Wed Aug 1 16:30:01 2012
Return-Path: <bounce-debian-user=tom=linux-archive.org@lists.debian.org>
X-Spam-Checker-Version: SpamAssassin 3.3.2 (2011-06-06) on
eagle542.startdedicated.com
X-Spam-Level:
X-Spam-Status: No, score=-4.9 required=5.0 tests=DKIM_SIGNED,FSL_RCVD_USER,
RCVD_IN_DNSWL_HI,T_DKIM_INVALID,T_RP_MATCHES_RCVD autolearn=ham version=3.3.2
X-Original-To: tom@linux-archive.org
Delivered-To: tom-linux-archive.org@eagle542.startdedicated.com
Received: from bendel.debian.org (bendel.debian.org [82.195.75.100])
by eagle542.startdedicated.com (Postfix) with ESMTP id BA04420E0287
for <tom@linux-archive.org>; Wed, 1 Aug 2012 16:00:56 +0200 (CEST)
Received: from localhost (localhost [127.0.0.1])
by bendel.debian.org (Postfix) with QMQP
id 3BF02287; Wed, 1 Aug 2012 14:00:33 +0000 (UTC)
Old-Return-Path: <debian-user@list-post.mks-mail.de>
X-Original-To: lists-debian-user@bendel.debian.org
Delivered-To: lists-debian-user@bendel.debian.org
Received: from localhost (localhost [127.0.0.1])
by bendel.debian.org (Postfix) with ESMTP id 23C1E258
for <lists-debian-user@bendel.debian.org>; Wed, 1 Aug 2012 14:00:24 +0000 (UTC)
X-Virus-Scanned: at lists.debian.org with policy bank en-ht
X-Amavis-Spam-Status: No, score=-7 tagged_above=-10000 required=5.3
tests=[BAYES_00=-2, DKIM_SIGNED=0.1, DKIM_VALID=-0.1,
DKIM_VALID_AU=-0.1, FOURLA=0.1, LDO_WHITELIST=-5] autolearn=ham
Received: from bendel.debian.org ([127.0.0.1])
by localhost (lists.debian.org [127.0.0.1]) (amavisd-new, port 2525)
with ESMTP id aarasubY6oEi for <lists-debian-user@bendel.debian.org>;
Wed, 1 Aug 2012 14:00:14 +0000 (UTC)
X-policyd-weight: DYN_NJABL=ERR(0) NOT_IN_SBL_XBL_SPAMHAUS=-1.5 NOT_IN_BL_NJABL=-1.5 DSBL_ORG=ERR(0) CL_IP_EQ_FROM_MX=-3.1; rate: -6.1
Received: from mail.ddt-consult.de (mail.ddt-consult.de [176.9.143.18])
(using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits))
(Client did not present a certificate)
by bendel.debian.org (Postfix) with ESMTPS id 8B5DD274
for <debian-user@lists.debian.org>; Wed, 1 Aug 2012 14:00:02 +0000 (UTC)
Received: from ddt-filter.ddt-consult.intern (ddt-filter.ddt-consult.intern [192.168.1.116])
by mail.ddt-consult.de (Postfix) with ESMTP id 14E5B2C9580
for <debian-user@lists.debian.org>; Wed, 1 Aug 2012 15:59:59 +0200 (CEST)
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=
list-post.mks-mail.de; h=content-transfer-encoding:content-type
:content-type:in-reply-to:references:subject:subject
:mime-version:user-agent:reply-to:from:from:date:date:message-id
:received:received; s=lpm; t=1343829641; bh=DBFexEd3ukWlPWAnUq4o
VajmUQQqFCD4uIqU7y6ztj8=; b=ozHbExpyLajegeuz+FKOW6MnKwDApPQnk0lg
59GnJtF0x3bivAtinvZ8KEe1vdfShz7bWZ5swrnqe/IhAHvRxNSoWzIt0m4sDtmp
6B/U2W0/M8kJGv7SIdH5WgRW1YfBq22D40Hw7CqxQ1l0XDgs9rzEaUOZY2 7kSBZX
RxaZDbM=
X-Virus-Scanned: Debian amavisd-new at ns1
Received: from mail.ddt-consult.de ([192.168.1.101])
by ddt-filter.ddt-consult.intern (ddt-filter.ddt-consult.intern [192.168.1.116]) (amavisd-new, port 20024)
with LMTP id J4xUprtxbJ8J for <debian-user@lists.debian.org>;
Wed, 1 Aug 2012 16:00:41 +0200 (CEST)
Received: from legolas.home.ddt.intern (p5DC3799C.dip.t-dialin.net [93.195.121.156])
(Authenticated sender: mks@list-post.mks-mail.de)
by mail.ddt-consult.de (Postfix) with ESMTPSA id EBD2C2C957E
for <debian-user@lists.debian.org>; Wed, 1 Aug 2012 15:59:55 +0200 (CEST)
Message-ID: <5019365A.10002@list-post.mks-mail.de>
Date: Wed, 01 Aug 2012 15:59:54 +0200
From: =?ISO-8859-1?Q?Markus_Sch=F6nhaber?=
<debian-user@list-post.mks-mail.de>
Reply-To: debian-user@lists.debian.org
User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:14.0) Gecko/20120717 Thunderbird/14.0
MIME-Version: 1.0
To: debian-user@lists.debian.org
Subject: Re: SSL problem/help
References: <5019217E.90203@venda.com>
In-Reply-To: <5019217E.90203@venda.com>
Content-Type: text/plain; charset=ISO-8859-1
Content-Transfer-Encoding: 7bit
X-Rc-Virus: 2007-09-13_01
X-Rc-Spam: 2008-11-04_01
Resent-Message-ID: <lDnfNw8aDRL.A.ZxB.BaTGQB@bendel>
Resent-From: debian-user@lists.debian.org
X-Mailing-List: <debian-user@lists.debian.org> archive/latest/636721
X-Loop: debian-user@lists.debian.org
List-Id: <debian-user.lists.debian.org>
List-Post: <mailto:debian-user@lists.debian.org>
List-Help: <mailto:debian-user-request@lists.debian.org?subject=help>
List-Subscribe: <mailto:debian-user-request@lists.debian.org?subject=subscribe>
List-Unsubscribe: <mailto:debian-user-request@lists.debian.org?subject=unsubscribe>
Precedence: list
Resent-Sender: debian-user-request@lists.debian.org
Resent-Date: Wed, 1 Aug 2012 14:00:33 +0000 (UTC)

01.08.2012 14:30, Benjamin Martin:

> I am having trouble connecting to a https url from machine A but not
> from machine B.
>
> Both machines are on the same network, but machine A is debian-testing
> and machine B is ubuntu 10.04. (both 64bit)

The important difference between the two machines is probably the
different versions of OpenSSL. Wheezy has OpenSSL 1.0.1 which introduces
new TLS protocols (TLS v1.1, v1.2).
The server seems to be unable to cope with those new protocols.

> It is a perl script that is doing the connecting (same script on both
> machines), on machine A it was reporting:
> "500 Can't connect to api.channeladvisor.com:443 "
> ... but not failing at all on B
>
> After doing some investigating, it seems the error message is abit
> misleading, as I CAN connect to the host on port 443 .. some more
> investigation shows that when I run this:
>
> openssl s_client -host api.channeladvisor.com -port 443
>
> .. on machine B, I see nothing worrying and I can "GET /" the html page.
> (it's a forbidden page, but it returns none the less)
>
> .. but on machine A, I get the following error:
>
> write:errno=104
> ---
> no peer certificate available
>
> After some more investigation I found that if I add "-cipher 3DES" to
> the command so it becomes:
>
> openssl s_client -host api.channeladvisor.com -port 443 -cipher 3DES
>
> It works!

As would adding -tls1 which sets the protocol to TLSv1 (which means v1.x
is not offered).

> So this leaves me with a few questions/concerns.
>
> Why do I have to add the "-cipher" switch to get this to work?
>
> I am guessing there is slight problem with the cert at
> "api.channeladvisor.com" as not all https sites have this problem ...
> with that in mind I guess "testing" has been updated with stricter SSL
> processing.... or is this a bug?
>
> If this is a bug I would like to report it
> .. or ...
> Does anyone know how to the "loosen" the SSL processing rules so the
> cert at api.channeladvisor.com is deemed valid?

As said above, it's probably not a cert but a protocol issue.
I don't know how to tell Perl to not use specific TLS versions, sorry.

> I don't really know what I am doing but I can use google and the command
> line.. so sorry if I missed any important detail or broke a list rule
> somehow... i am just abit stuck

You have obviously tried to understand the problem, you have even come
up with a workaround, you have described pretty decently what exactly
you did - I don't see what else could be expected from you.

> PS. I have tried this on gentoo and centos and all seem to be ok .. just
> "testing" seems to display this problem

I'd bet the Gentoo and CentOS systems you tried that on, come with an
OpenSSL version < 1.0.1.

--
Regards
mks


--
To UNSUBSCRIBE, email to debian-user-REQUEST@lists.debian.org
with a subject of "unsubscribe". Trouble? Contact listmaster@lists.debian.org
Archive: http://lists.debian.org/5019365A.10002@list-post.mks-mail.de

On 1 Aug 2012 at 9:56, Michael Orlitzky wrote:

> But, I'd ruled out the stack size limitation because resource oversteps
> are supposed to be reported:

it's not a resource overstep but simply not enough virtual address space
(either because it's too fragmented to fit such a big allocation or the
free space is not enough itself).

On 08/01/2012 05:29 PM, PaX Team wrote:
> On 1 Aug 2012 at 9:56, Michael Orlitzky wrote:
>
>> But, I'd ruled out the stack size limitation because resource oversteps
>> are supposed to be reported:
>
> it's not a resource overstep but simply not enough virtual address space
> (either because it's too fragmented to fit such a big allocation or the
> free space is not enough itself).
>

I don't completely understand, but I believe you =)

Setting `ulimit -s unlimited` in my global rc.conf cleared up the
problem. Thanks again for the help.