Portage to die on sure-enough _FORTIFY_SOURCE overflows
 

Hi all,

since the last time I asked Zac about this it came back to bite me[1]
this time I'm going to send the announce to the list first, and if
nobody can actually come up with a good reason not to, I'm going to ask
Zac tomorrow to re-enable the feature.

What is this about? Portage already reports some of the overflow
warnings coming from the glibc fortified sources (-D_FORTIFY_SOURCE=2
-O2 — enabled since gcc 4.3.3-r1 and even stronger with gcc 4.5 and
glibc 2.12+, afaict), but they really are divided into two categories:

- might overflow (depends on combination of parameters and variables the
compiler can't completely untangle);
- _will_ overflow (whenever that code path is hit, an overflow will
happen).

The former we should highlight but not die upon; the latter, though...

As Mike and me expressed on the linked bug, code that is built with that
warning is code that is going to crash as surely as

char *foo = NULL;
foo[3] = 'a';

which could result in nasty surprises for users (see [2] for the whole
reasoning).

Now, we've not seen "proper" false positives (in the Portage sense I
mean — because even if the C library hits a false positive, it _will_
crash with an abort() from its own code!), but Kumba pointed me at a
case that wasn't entirely clear, and took a bit of detective work to
track down [3] so you could have users report issues you cannot easily
identify or reproduce. I cannot make promises, but if all else fail I'll
see to be around to help you with those cases.

So if you want to have your say, gentoo-qa is there for that.

Thank you,

[1] https://bugs.gentoo.org/show_bug.cgi?id=337031
[2]
http://blog.flameeyes.eu/2010/09/14/not-all-failures-are-caused-equal
[3]
http://blog.flameeyes.eu/2010/09/12/some-_fortify_source-far-fetched-warnings-are-funny

--
Diego Elio Pettenò — “Flameeyes�
http://blog.flameeyes.eu/

If you found a .asc file in this mail and know not what it is,
it's a GnuPG digital signature: http://www.gnupg.org/

Portage to die on sure-enough _FORTIFY_SOURCE overflows
 

On 09/28/2010 12:43 PM, Diego Elio Pettenò wrote:

>
> So if you want to have your say, gentoo-qa is there for that.
>

You should not cross post like this. Following the recent discussion the
only list allowing cross posting is gentoo-dev-announce.

Regards,
Petteri

Portage to die on sure-enough _FORTIFY_SOURCE overflows
 

On Tue, Sep 28, 2010 at 2:43 AM, Diego Elio Pettenò <flameeyes@gmail.com> wrote:
> Hi all,
>
> since the last time I asked Zac about this it came back to bite me[1]
> this time I'm going to send the announce to the list first, and if
> nobody can actually come up with a good reason not to, I'm going to ask
> Zac tomorrow to re-enable the feature.
>
> What is this about? Portage already reports some of the overflow
> warnings coming from the glibc fortified sources (-D_FORTIFY_SOURCE=2
> -O2 — enabled since gcc 4.3.3-r1 and even stronger with gcc 4.5 and
> glibc 2.12+, afaict), but they really are divided into two categories:
>
> - might overflow (depends on combination of parameters and variables the
> compiler can't completely untangle);
> - _will_ overflow (whenever that code path is hit, an overflow will
> happen).
>
> The former we should highlight but not die upon; the latter, though...
>
> As Mike and me expressed on the linked bug, code that is built with that
> warning is code that is going to crash as surely as
>
> char *foo = NULL;
> foo[3] = 'a';
>
> which could result in nasty surprises for users (see [2] for the whole
> reasoning).
>
> Now, we've not seen "proper" false positives (in the Portage sense I
> mean — because even if the C library hits a false positive, it _will_
> crash with an abort() from its own code!), but Kumba pointed me at a
> case that wasn't entirely clear, and took a bit of detective work to
> track down [3] so you could have users report issues you cannot easily
> identify or reproduce. I cannot make promises, but if all else fail I'll
> see to be around to help you with those cases.
>
> So if you want to have your say, gentoo-qa is there for that.

So do you expect:

1. Developers to fix these bugs?
2. Report them upstream?
3. Remove packages?

Its not clear to me what your purpose is. It is likely that many
developers will be unable to do 1. Does that concern you? Should
developers ask QA for help on packages?

-A

>
> Thank you,
>
> [1] https://bugs.gentoo.org/show_bug.cgi?id=337031
> [2]
> http://blog.flameeyes.eu/2010/09/14/not-all-failures-are-caused-equal
> [3]
> http://blog.flameeyes.eu/2010/09/12/some-_fortify_source-far-fetched-warnings-are-funny
>
> --
> Diego Elio Pettenò — “Flameeyes�
> http://blog.flameeyes.eu/
>
> If you found a .asc file in this mail and know not what it is,
> it's a GnuPG digital signature: http://www.gnupg.org/
>
>
>
>

Portage to die on sure-enough _FORTIFY_SOURCE overflows
 

On Tuesday, September 28, 2010 15:33:10 Alec Warner wrote:
> On Tue, Sep 28, 2010 at 2:43 AM, Diego Elio Pettenò wrote:
> > since the last time I asked Zac about this it came back to bite me[1]
> > this time I'm going to send the announce to the list first, and if
> > nobody can actually come up with a good reason not to, I'm going to ask
> > Zac tomorrow to re-enable the feature.
> >
> > What is this about? Portage already reports some of the overflow
> > warnings coming from the glibc fortified sources (-D_FORTIFY_SOURCE=2
> > -O2 — enabled since gcc 4.3.3-r1 and even stronger with gcc 4.5 and
> > glibc 2.12+, afaict), but they really are divided into two categories:
> >
> > - might overflow (depends on combination of parameters and variables the
> > compiler can't completely untangle);
> > - _will_ overflow (whenever that code path is hit, an overflow will
> > happen).
> >
> > The former we should highlight but not die upon; the latter, though...
> >
> > As Mike and me expressed on the linked bug, code that is built with that
> > warning is code that is going to crash as surely as
> >
> > char *foo = NULL;
> > foo[3] = 'a';
> >
> > which could result in nasty surprises for users (see [2] for the whole
> > reasoning).
> >
> > Now, we've not seen "proper" false positives (in the Portage sense I
> > mean — because even if the C library hits a false positive, it _will_
> > crash with an abort() from its own code!), but Kumba pointed me at a
> > case that wasn't entirely clear, and took a bit of detective work to
> > track down [3] so you could have users report issues you cannot easily
> > identify or reproduce. I cannot make promises, but if all else fail I'll
> > see to be around to help you with those cases.
> >
> > So if you want to have your say, gentoo-qa is there for that.
>
> So do you expect:
>
> 1. Developers to fix these bugs?
> 2. Report them upstream?
> 3. Remove packages?
>
> Its not clear to me what your purpose is. It is likely that many
> developers will be unable to do 1. Does that concern you? Should
> developers ask QA for help on packages?

developers are expected to get their package fixed. how they get that done is
up to them.

as Diego said, this isnt a matter of "i see a compile warning, so lets abort
the install". the code in question _will_ call abort() all by itself if you
attempt to execute it.
-mike

Portage to die on sure-enough _FORTIFY_SOURCE overflows
 

Il giorno mar, 28/09/2010 alle 12.33 -0700, Alec Warner ha scritto:
> Its not clear to me what your purpose is. It is likely that many
> developers will be unable to do 1. Does that concern you? Should
> developers ask QA for help on packages?

Fixing the package is the solution, it's usually quick and easy to
identify; if you can bring up _any_ example of unfixable or
difficult-to-fix code, feel free.

I don't think that was not explained by me, I even wrote a whole blog
post about identifying, tracking down and fixing _FORTIFY_SOURCE
warnings.

--
Diego Elio Pettenò — “Flameeyes�
http://blog.flameeyes.eu/

If you found a .asc file in this mail and know not what it is,
it's a GnuPG digital signature: http://www.gnupg.org/

Portage to die on sure-enough _FORTIFY_SOURCE overflows
 

On Tue, 28 Sep 2010 11:43:28 +0200
Diego Elio Pettenò <flameeyes@gmail.com> wrote:

> since the last time I asked Zac about this it came back to bite me[1]
> this time I'm going to send the announce to the list first, and if
> nobody can actually come up with a good reason not to, I'm going to ask
> Zac tomorrow to re-enable the feature.

"Tomorrow" isn't much of a warning. Can you please give people a chance to
fix the bugs you've filed?

Something I forgot to ask before: are the 'always overflow' warnings new w/
GCC 4.5 / glibc 2.12? If they're new w/ 4.5 then we don't have a problem.


--
fonts, gcc-porting, we hold our breath, we spin around the world
toolchain, wxwidgets you and me cling to the outside of the earth
@ gentoo.org EFFD 380E 047A 4B51 D2BD C64F 8AA8 8346 F9A4 0662

Portage to die on sure-enough _FORTIFY_SOURCE overflows
 

On Tuesday, September 28, 2010 20:33:52 Ryan Hill wrote:
> On Tue, 28 Sep 2010 11:43:28 +0200 Diego Elio Pettenò wrote:
> > since the last time I asked Zac about this it came back to bite me[1]
> > this time I'm going to send the announce to the list first, and if
> > nobody can actually come up with a good reason not to, I'm going to ask
> > Zac tomorrow to re-enable the feature.
>
> "Tomorrow" isn't much of a warning. Can you please give people a chance to
> fix the bugs you've filed?
>
> Something I forgot to ask before: are the 'always overflow' warnings new
> w/ GCC 4.5 / glibc 2.12? If they're new w/ 4.5 then we don't have a
> problem.

the fortify warnings typically come from glibc, not gcc. i dont believe many
of these warnings are new. the portage update i posted was because i was
reviewing a specific package, noticed a worrisome warning (and fixed it), and
then proceeded to data mine the last years worth of build logs on my system
for gcc warnings.
-mike

Portage to die on sure-enough _FORTIFY_SOURCE overflows
 

On Tue, 28 Sep 2010 22:25:38 -0400
Mike Frysinger <vapier@gentoo.org> wrote:

> > Something I forgot to ask before: are the 'always overflow' warnings new
> > w/ GCC 4.5 / glibc 2.12? If they're new w/ 4.5 then we don't have a
> > problem.
>
> the fortify warnings typically come from glibc, not gcc. i dont believe many
> of these warnings are new. the portage update i posted was because i was
> reviewing a specific package, noticed a worrisome warning (and fixed it), and
> then proceeded to data mine the last years worth of build logs on my system
> for gcc warnings.

Okay, I noticed that some of these bugs are only happening with 4.5 (eg.
337020) so I thought it might have been expanded to catch more cases or
something.


--
fonts, gcc-porting, we hold our breath, we spin around the world
toolchain, wxwidgets you and me cling to the outside of the earth
@ gentoo.org EFFD 380E 047A 4B51 D2BD C64F 8AA8 8346 F9A4 0662

Portage to die on sure-enough _FORTIFY_SOURCE overflows
 

On Wednesday, September 29, 2010 00:35:45 Ryan Hill wrote:
> On Tue, 28 Sep 2010 22:25:38 -0400 Mike Frysinger wrote:
> > > Something I forgot to ask before: are the 'always overflow' warnings
> > > new w/ GCC 4.5 / glibc 2.12? If they're new w/ 4.5 then we don't have
> > > a problem.
> >
> > the fortify warnings typically come from glibc, not gcc. i dont believe
> > many of these warnings are new. the portage update i posted was because
> > i was reviewing a specific package, noticed a worrisome warning (and
> > fixed it), and then proceeded to data mine the last years worth of build
> > logs on my system for gcc warnings.
>
> Okay, I noticed that some of these bugs are only happening with 4.5 (eg.
> 337020) so I thought it might have been expanded to catch more cases or
> something.

it might be a cumulative effect -- better constant propagation in gcc allows
updated glibc fortify checks to catch more naughty code. you can see in this
bug the warning is coming via checks in the glibc headers. but i'd have to
sit down with different gcc/glibc versions and do some fiddlin' to give a less
vague answer.
-mike