FAQ Search Today's Posts Mark Forums Read
» Video Reviews

» Linux Archive

Linux-archive is a website aiming to archive linux email lists and to make them easily accessible for linux users/developers.


» Sponsor

» Partners

» Sponsor

Go Back   Linux Archive > Debian > Debian dpkg

 
 
LinkBack Thread Tools
 
Old 09-13-2011, 05:56 AM
Guillem Jover
 
Default Hardening patch

On Sun, 2011-09-11 at 08:19:42 +0200, Raphael Hertzog wrote:
> On Sun, 11 Sep 2011, Guillem Jover wrote:
> > > + "bindnow" => 1
> >
> > Any reason you seem to have ignored the concerns I rised about
> > defaulting to bindnow?
>
> Well, you mentioned potential performance problems and Kees said
> that his tests did not conclude that it resulted in significant
> performance loss. Kees has been doing the work, I trust him.

I specifically asked on which arches he performed the tests. If he had
said on armel too, then I'd not have any problem with that, but he
didn't reply to that, so I don't see how this is a matter of trust,
when there's just lack of information.

> You told that you would expect the loss to be more significant
> on slower architectures but you did not backup that claim with
> anything.

Well, the concerns were coming from first-hand experience from working
on ARM systems, otherwise I'd not have commented. Specifically on
Maemo the startup time was so bad for UI apps, we created maemo-launcher
just to improve it.

I installed iceweasel on an ARM system (Thecus N2100), w/o X forwarding,
and no user profile, so it just stops when it's not able to find the
DISPLAY, but that should be good enough to get timings close to just the
startup relocation times, which is what the ld.so stats show on amd64
for example. Caches flushed on each iteration, which were pretty
consistent, I've included two different ones for each:

,--- bind lazy ---
# echo 3 > /proc/sys/vm/drop_caches
$ time LD_DEBUG=statistics MOZ_NO_REMOTE=1 iceweasel -ProfileManagea
[...]
Error: no display specified
17887:
17887: runtime linker statistics:
17887: final number of relocations: 4233
17887: final number of relocations from cache: 29307

real 0m2.279s
user 0m0.310s
sys 0m0.440s

# echo 3 > /proc/sys/vm/drop_caches
$ time LD_DEBUG=statistics MOZ_NO_REMOTE=1 iceweasel -ProfileManager
[...]
Error: no display specified
17883:
17883: runtime linker statistics:
17883: final number of relocations: 4233
17883: final number of relocations from cache: 29307

real 0m2.187s
user 0m0.320s
sys 0m0.440s
`---

,--- bind now ---
# echo 3 > /proc/sys/vm/drop_caches
$ time LD_BIND_NOW=1 LD_DEBUG=statistics MOZ_NO_REMOTE=1 iceweasel -ProfileManager
[...]
Error: no display specified
17932:
17932: runtime linker statistics:
17932: final number of relocations: 19873
17932: final number of relocations from cache: 29307

real 0m3.188s
user 0m1.050s
sys 0m0.560s

# echo 3 > /proc/sys/vm/drop_caches
$ time LD_BIND_NOW=1 LD_DEBUG=statistics MOZ_NO_REMOTE=1 iceweasel -ProfileManager
[...]
Error: no display specified
17924:
17924: runtime linker statistics:
17924: final number of relocations: 19873
17924: final number of relocations from cache: 29307

real 0m3.255s
user 0m1.140s
sys 0m0.490s
`---

As it can bee seen the difference is pretty significant.

> Of course 5% more of 200ms is not the same than 5%
> of 2s but other than that, I do not see any technical reason
> why they would be more affected than i386/amd64.

If it was just a 5% (regardless of the total time) then of course
that'd be ok... And for the technical reasons, well the ELF format
and its ABIs are pretty tied to the architecture and its instruction
set, which influence things like the types of relocations, how the
PLT entries are handled, etc.

> > In any case I don't think enabling this w/o further data demonstrating
> > it's fine to do so is acceptable, as fixing any such regression would
> > imply needing to hunt down packages built with the new flags and trigger
> > binNMUs for them all. The default for it can always be changed later
> > on, I don't see the need to rush it?
>
> Feel free to change it if you think it's better that way. I'm not attached
> to it.

I'm changing it now on my local tree, will be included in my next
push.

> But I'm not convinced that anyone will do the research that you require on
> non-mainstream architectures and we will just end up with bindnow
> disabled for the indefinite future.

Eh, there's probably more ARM devices in the world than x86 and x86-64
put together. In any case that's how these things are supposed to work,
when someone proposes to change current defaults that might affect the
whole archive, it's on them to provide convincing arguments, and
mitigate raised concerns, not the other way around. If no one is
intersted enough to provide them, well too bad, I guess.

regards,
guillem


--
To UNSUBSCRIBE, email to debian-dpkg-REQUEST@lists.debian.org
with a subject of "unsubscribe". Trouble? Contact listmaster@lists.debian.org
Archive: 20110913055641.GC8690@gaara.hadrons.org">http://lists.debian.org/20110913055641.GC8690@gaara.hadrons.org
 
Old 09-13-2011, 06:08 AM
Kees Cook
 
Default Hardening patch

On Tue, Sep 13, 2011 at 07:56:41AM +0200, Guillem Jover wrote:
> On Sun, 2011-09-11 at 08:19:42 +0200, Raphael Hertzog wrote:
> > On Sun, 11 Sep 2011, Guillem Jover wrote:
> > > > + "bindnow" => 1
> > >
> > > Any reason you seem to have ignored the concerns I rised about
> > > defaulting to bindnow?
> >
> > Well, you mentioned potential performance problems and Kees said
> > that his tests did not conclude that it resulted in significant
> > performance loss. Kees has been doing the work, I trust him.
>
> I specifically asked on which arches he performed the tests. If he had
> said on armel too, then I'd not have any problem with that, but he
> didn't reply to that, so I don't see how this is a matter of trust,
> when there's just lack of information.

Ah, sorry about that; I didn't have access to hardware.

> I installed iceweasel on an ARM system (Thecus N2100), w/o X forwarding,
> and no user profile, so it just stops when it's not able to find the
> DISPLAY, but that should be good enough to get timings close to just the
> startup relocation times, which is what the ld.so stats show on amd64
> for example. Caches flushed on each iteration, which were pretty
> consistent, I've included two different ones for each:

Excellent, this is a good test. Thanks for doing this!

> real 0m2.279s
...
> real 0m3.255s
...
>
> As it can bee seen the difference is pretty significant.

Yeah, that's massive. I would totally agree -- remove bindnow from
defaults.

> I'm changing it now on my local tree, will be included in my next
> push.

Thanks! I'll include "+bindnow" in the documentation that was already going
to include "+pie" for maintainers that want to transition from
hardening-wrapper/-includes to dpkg-buildflags.

-Kees

--
Kees Cook @debian.org


--
To UNSUBSCRIBE, email to debian-dpkg-REQUEST@lists.debian.org
with a subject of "unsubscribe". Trouble? Contact listmaster@lists.debian.org
Archive: 20110913060845.GY31645@outflux.net">http://lists.debian.org/20110913060845.GY31645@outflux.net
 
Old 09-13-2011, 06:51 AM
Raphael Hertzog
 
Default Hardening patch

Hi Guillem,

On Tue, 13 Sep 2011, Guillem Jover wrote:
> Well, the concerns were coming from first-hand experience from working
> on ARM systems, otherwise I'd not have commented. Specifically on
> Maemo the startup time was so bad for UI apps, we created maemo-launcher
> just to improve it.

Ok, but I'm not yet able to read into minds :-)

> I installed iceweasel on an ARM system (Thecus N2100), w/o X forwarding,
> and no user profile, so it just stops when it's not able to find the
> DISPLAY, but that should be good enough to get timings close to just the
> startup relocation times, which is what the ld.so stats show on amd64
> for example. Caches flushed on each iteration, which were pretty
> consistent, I've included two different ones for each:

I did the same on my eSATA SheevaPlug (armel too) and I don't get the same
results than you.

rhertzog@nas:~$ echo 3 | sudo sponge /proc/sys/vm/drop_caches
rhertzog@nas:~$ time LD_DEBUG=statistics MOZ_NO_REMOTE=1 iceweasel -ProfileManager
[...]
Error: no display specified
10941:
10941: runtime linker statistics:
10941: final number of relocations: 3306
10941: final number of relocations from cache: 26441

real 0m1.771s
user 0m0.100s
sys 0m0.090s
rhertzog@nas:~$ echo 3 | sudo sponge /proc/sys/vm/drop_caches
rhertzog@nas:~$ time LD_DEBUG=statistics MOZ_NO_REMOTE=1 iceweasel -ProfileManager
[...]
Error: no display specified
10948:
10948: runtime linker statistics:
10948: final number of relocations: 3306
10948: final number of relocations from cache: 26441

real 0m1.905s
user 0m0.110s
sys 0m0.080s
rhertzog@nas:~$ echo 3 | sudo sponge /proc/sys/vm/drop_caches
rhertzog@nas:~$ time LD_BIND_NOW=1 LD_DEBUG=statistics MOZ_NO_REMOTE=1 iceweasel -ProfileManager
[...]
Error: no display specified
11034:
11034: runtime linker statistics:
11034: final number of relocations: 17420
11034: final number of relocations from cache: 26441

real 0m1.771s
user 0m0.230s
sys 0m0.140s
rhertzog@nas:~$ echo 3 | sudo sponge /proc/sys/vm/drop_caches
rhertzog@nas:~$ time LD_BIND_NOW=1 LD_DEBUG=statistics MOZ_NO_REMOTE=1 iceweasel -ProfileManager
[...]
Error: no display specified
11041:
11041: runtime linker statistics:
11041: final number of relocations: 17420
11041: final number of relocations from cache: 26441

real 0m2.056s
user 0m0.270s
sys 0m0.100s

> As it can bee seen the difference is pretty significant.

As it can be seen, the difference is not something that affects
all armel machines in the same way.

I did 10 run of each, dropped the biggest value and did the mean
value on the rest:
- bind lazy: 1.842s
- bind now: 1.910s

Difference: +3,6 %

I'm using the plain Debian kernel and all squeeze packages.

$ uname -a
Linux nas 2.6.32-5-kirkwood #1 Wed Jan 12 15:27:07 UTC 2011 armv5tel GNU/Linux

> > Feel free to change it if you think it's better that way. I'm not attached
> > to it.
>
> I'm changing it now on my local tree, will be included in my next
> push.

Given my test, I'm not convinced it's the right course of action.

I did the same test on my i386 setup and I got this:
- bind lazy: 0.320s
- bind now: 0.330s

Difference: +3,1 %

At the very least, we could keep it enabled for i386/amd64, no ?

Cheers,
--
Raphaël Hertzog ◈ Debian Developer

Follow my Debian News ▶ http://RaphaelHertzog.com (English)
▶ http://RaphaelHertzog.fr (Français)


--
To UNSUBSCRIBE, email to debian-dpkg-REQUEST@lists.debian.org
with a subject of "unsubscribe". Trouble? Contact listmaster@lists.debian.org
Archive: 20110913065117.GP13263@rivendell.home.ouaza.com">h ttp://lists.debian.org/20110913065117.GP13263@rivendell.home.ouaza.com
 
Old 09-20-2011, 02:37 AM
Guillem Jover
 
Default Hardening patch

On Tue, 2011-09-13 at 08:51:17 +0200, Raphael Hertzog wrote:
> On Tue, 13 Sep 2011, Guillem Jover wrote:
> > I installed iceweasel on an ARM system (Thecus N2100), w/o X forwarding,
> > and no user profile, so it just stops when it's not able to find the
> > DISPLAY, but that should be good enough to get timings close to just the
> > startup relocation times, which is what the ld.so stats show on amd64
> > for example. Caches flushed on each iteration, which were pretty
> > consistent, I've included two different ones for each:
>
> I did the same on my eSATA SheevaPlug (armel too) and I don't get the same
> results than you.

[SheevaPlug tests]

> > As it can bee seen the difference is pretty significant.
>
> As it can be seen, the difference is not something that affects
> all armel machines in the same way.
>
> I did 10 run of each, dropped the biggest value and did the mean
> value on the rest:
> - bind lazy: 1.842s
> - bind now: 1.910s
>
> Difference: +3,6 %

Well an eSATA SheevaPlug is a pretty fast ARM machine! In any case it
was not my intention to imply all ARM machines would be affected
equally. I said the impact would be more noticable on slow architectures,
which ARM tends to be. In this case I/O is most probably the culprit,
something I hinted at on my first mail on this sub-thread. And this
will affect any system with slow I/O.

> > > Feel free to change it if you think it's better that way. I'm not attached
> > > to it.
> >
> > I'm changing it now on my local tree, will be included in my next
> > push.

I took the commit out from my push because this was still under
discussion, that does not mean I've changed my mind though and I
still do not really feel comfortable uploading a dpkg defaulting
to bind now.

> Given my test, I'm not convinced it's the right course of action.
>
> I did the same test on my i386 setup and I got this:
> - bind lazy: 0.320s
> - bind now: 0.330s
>
> Difference: +3,1 %
>
> At the very least, we could keep it enabled for i386/amd64, no ?

I've written some of this in some previous mail, but I'll repeat. This
can have real impact on performance, it potentially affects the whole
archive (once it all switches to using dpkg-buildflags), and even on
overally fast archiectures it might still affect a range of its slow
systems, once bind now is set on an object (via DF_1_NOW, DF_BIND_NOW
or DT_BIND_NOW) it cannot be disabled by neither of dlopen(RTLD_LAZY)
nor environment variables, it's trading an optimization with a security
measure.

So given its obvious drawbacks (something the other options do not
seem to present), that we can always change it in the future, I don't
think the current default should be changed. In this kind of case our
stance should really be “if in doubt, do not change”. But if this was
to get changed, this is something the project at large should get
consensus on, not just the dpkg team.

Of course specific maintainers are free to choose compilation options
for their own packages, because that does (generally) have a global
impact, and they can assess what side of the trade off is more
important.

regards,
guillem


--
To UNSUBSCRIBE, email to debian-dpkg-REQUEST@lists.debian.org
with a subject of "unsubscribe". Trouble? Contact listmaster@lists.debian.org
Archive: 20110920023705.GA19369@gaara.hadrons.org">http://lists.debian.org/20110920023705.GA19369@gaara.hadrons.org
 
Old 09-21-2011, 07:08 AM
Raphael Hertzog
 
Default Hardening patch

On Tue, 20 Sep 2011, Guillem Jover wrote:
> I took the commit out from my push because this was still under
> discussion, that does not mean I've changed my mind though and I
> still do not really feel comfortable uploading a dpkg defaulting
> to bind now.
[...]
> I've written some of this in some previous mail, but I'll repeat. This
> can have real impact on performance, it potentially affects the whole
> archive (once it all switches to using dpkg-buildflags), and even on
> overally fast archiectures it might still affect a range of its slow
> systems, once bind now is set on an object (via DF_1_NOW, DF_BIND_NOW
> or DT_BIND_NOW) it cannot be disabled by neither of dlopen(RTLD_LAZY)
> nor environment variables, it's trading an optimization with a security
> measure.

Ok, you have convinced me. Please put your commit back and change the
default to disabled.

Cheers,
--
Raphaël Hertzog ◈ Debian Developer

Follow my Debian News ▶ http://RaphaelHertzog.com (English)
▶ http://RaphaelHertzog.fr (Français)


--
To UNSUBSCRIBE, email to debian-dpkg-REQUEST@lists.debian.org
with a subject of "unsubscribe". Trouble? Contact listmaster@lists.debian.org
Archive: 20110921070812.GO4183@rivendell.home.ouaza.com">ht tp://lists.debian.org/20110921070812.GO4183@rivendell.home.ouaza.com
 

Thread Tools




All times are GMT. The time now is 01:39 AM.

VBulletin, Copyright ©2000 - 2014, Jelsoft Enterprises Ltd.
Content Relevant URLs by vBSEO ©2007, Crawlability, Inc.
Copyright 2007 - 2008, www.linux-archive.org