FAQ Search Today's Posts Mark Forums Read
» Video Reviews

» Linux Archive

Linux-archive is a website aiming to archive linux email lists and to make them easily accessible for linux users/developers.


» Sponsor

» Partners

» Sponsor

Go Back   Linux Archive > Redhat > Fedora Advisory Board

 
 
LinkBack Thread Tools
 
Old 05-13-2008, 06:45 PM
Greg DeKoenigsberg
 
Default The Debian/Ubuntu SSL bug

So I've been having a conversation with Mark Cox about the Debian/Ubuntu
SSL bug. This is basically a horror story of what can go wrong when
packagers don't maintain close relationships with upstream. I asked Mark,
"what security policies do we have in place to keep this from happening in
Fedora-land?" And his response was, "I don't know, what security policies
do we have in place to keep this from happening in Fedora-land?"


We know that RHEL is secure and stable, and we *do* have safeguards in
place to prevent this from happening in RHEL-land. But a mistake like
this in Fedora-land would be every bit as bad for the Red Hat and Fedora
brands.


Are there any steps we can take to protect ourselves from this kind of
mistake -- in which a packager does something dumb to the package and no
one notices it?


--g

--
Greg DeKoenigsberg
Community Development Manager
Red Hat, Inc. :: 1-919-754-4255
"To whomsoever much hath been given...
...from him much shall be asked"

_______________________________________________
fedora-advisory-board mailing list
fedora-advisory-board@redhat.com
http://www.redhat.com/mailman/listinfo/fedora-advisory-board
 
Old 05-13-2008, 06:52 PM
Rex Dieter
 
Default The Debian/Ubuntu SSL bug

Greg DeKoenigsberg wrote:

Are there any steps we can take to protect ourselves from this kind of
mistake -- in which a packager does something dumb to the package and no
one notices it?


Aside from the already-in-place strong recommendations and policies wrt
encouraging comaintainers and working closely with upstream projects,
excellent question.


-- Rex

_______________________________________________
fedora-advisory-board mailing list
fedora-advisory-board@redhat.com
http://www.redhat.com/mailman/listinfo/fedora-advisory-board
 
Old 05-13-2008, 06:56 PM
Dave Jones
 
Default The Debian/Ubuntu SSL bug

On Tue, May 13, 2008 at 02:45:29PM -0400, Greg DeKoenigsberg wrote:
>
> So I've been having a conversation with Mark Cox about the Debian/Ubuntu
> SSL bug. This is basically a horror story of what can go wrong when
> packagers don't maintain close relationships with upstream. I asked Mark,
> "what security policies do we have in place to keep this from happening in
> Fedora-land?" And his response was, "I don't know, what security policies
> do we have in place to keep this from happening in Fedora-land?"
>
> We know that RHEL is secure and stable, and we *do* have safeguards in
> place to prevent this from happening in RHEL-land. But a mistake like
> this in Fedora-land would be every bit as bad for the Red Hat and Fedora
> brands.
>
> Are there any steps we can take to protect ourselves from this kind of
> mistake -- in which a packager does something dumb to the package and no
> one notices it?

The thing I find amazing about this bug is that it took 2 years for someone
to notice it. I think in part this is due to the size of debian making it
pretty much impossible for someone to review every change that goes in.
The casual observer just has no chance to keep up with the rate of change.
(and there are parallels to kernel development here, git has increased
the merge rate to the point where one person really can't sanely review everything
that goes in).

I think the only real answer is to make sure that for critical packages,
there exists >1 person actually reading the changes that go in.
Though the possibility of groupthink taints even this approach.
Clearly a number of debian developers thought that change was a great idea,
without realising the consequences.

Other than review though, and making sure that any patches we carry
are either a) short-lived [ie, on their way upstream], or b) carried forever
for really good reasons.

Something the SuSE guys have done which I'm thinking we should adopt for our
patches (in the kernel at least), is a header at the top of each patch
detailing its upstream status, (and if not upstream, why not).

Additionally, something else I think we should be doing more of is
automated testing. We do a bunch of this for RHEL already. There was
a whole brouhaha at rh summit a few years ago about how we were going to
opensource our QA test harnesses. None of that happened afaict.
Perhaps it'd be faster to reinvent something new anyway.
(Seemed to work out for the better that way with koji).

Good questions though. This bug could easily have been us on the recieving end.

Dave

--
http://www.codemonkey.org.uk

_______________________________________________
fedora-advisory-board mailing list
fedora-advisory-board@redhat.com
http://www.redhat.com/mailman/listinfo/fedora-advisory-board
 
Old 05-13-2008, 06:57 PM
Mark J Cox
 
Default The Debian/Ubuntu SSL bug

The thing I find amazing about this bug is that it took 2 years for someone
to notice it. I think in part this is due to the size of debian making it
pretty much impossible for someone to review every change that goes in.


In this case it's also a little to do with the complexity of the issue, it
was in fact proposed by the vendor to the upstream project development
list and no one really noticed it would have a bad side-effect:

http://marc.info/?m=114651085826293&w=2


Something the SuSE guys have done which I'm thinking we should adopt for our
patches (in the kernel at least), is a header at the top of each patch
detailing its upstream status, (and if not upstream, why not).


Yeah, this should be enforced. We ought to be including signatures for
the pristine upstream tarballs in the srpms too (where upstream signs
their output). At least we then can know for certain what has been
touched outside of upstream.


Cheers, Mark

_______________________________________________
fedora-advisory-board mailing list
fedora-advisory-board@redhat.com
http://www.redhat.com/mailman/listinfo/fedora-advisory-board
 
Old 05-13-2008, 07:04 PM
"Stephen John Smoogen"
 
Default The Debian/Ubuntu SSL bug

On Tue, May 13, 2008 at 12:45 PM, Greg DeKoenigsberg <gdk@redhat.com> wrote:
>
> So I've been having a conversation with Mark Cox about the Debian/Ubuntu
> SSL bug. This is basically a horror story of what can go wrong when
> packagers don't maintain close relationships with upstream. I asked Mark,
> "what security policies do we have in place to keep this from happening in
> Fedora-land?" And his response was, "I don't know, what security policies
> do we have in place to keep this from happening in Fedora-land?"
>
> We know that RHEL is secure and stable, and we *do* have safeguards in
> place to prevent this from happening in RHEL-land. But a mistake like this
> in Fedora-land would be every bit as bad for the Red Hat and Fedora brands.
>
> Are there any steps we can take to protect ourselves from this kind of
> mistake -- in which a packager does something dumb to the package and no one
> notices it?
>

Well the biggest step would be to add additional code review steps for
packages... and probably trying to increase the number of
'code-monkeys' per package. However, I am not sure that is the best
step... especially to the crowd that believes Fedora is too
bureaucratic now.

Would having a review release, where instead of trying to put in
things as newer we worked on getting more eyes on the code make sense?
How could it be done?

Also, how many times does a patch get added because someone saw it in
the 'Debian' or 'SuSE' trees and it looked like it 'fixed' something?



--
Stephen J Smoogen. -- BSD/GNU/Linux
How far that little candle throws his beams! So shines a good deed
in a naughty world. = Shakespeare. "The Merchant of Venice"

_______________________________________________
fedora-advisory-board mailing list
fedora-advisory-board@redhat.com
http://www.redhat.com/mailman/listinfo/fedora-advisory-board
 
Old 05-13-2008, 07:10 PM
Dave Jones
 
Default The Debian/Ubuntu SSL bug

On Tue, May 13, 2008 at 01:04:11PM -0600, Stephen John Smoogen wrote:

> Also, how many times does a patch get added because someone saw it in
> the 'Debian' or 'SuSE' trees and it looked like it 'fixed' something?

I really hope cargo-culting of other distros patches doesn't happen
a lot, (especially for anything important) without at least one person
who maintains that package understanding the reason why that patch
is important.

"Because debian did" really should never be an excuse for committing something.

Dave

--
http://www.codemonkey.org.uk

_______________________________________________
fedora-advisory-board mailing list
fedora-advisory-board@redhat.com
http://www.redhat.com/mailman/listinfo/fedora-advisory-board
 
Old 05-13-2008, 07:15 PM
"Jeff Spaleta"
 
Default The Debian/Ubuntu SSL bug

On Tue, May 13, 2008 at 10:56 AM, Dave Jones <davej@redhat.com> wrote:
> Something the SuSE guys have done which I'm thinking we should adopt for our
> patches (in the kernel at least), is a header at the top of each patch
> detailing its upstream status, (and if not upstream, why not).

A status header for all patches might be a good thing, if....
we can do it in such a way that we can establish some sort of process
that periodically reviews the status headers for each patch and uses
manpower to do the follow-up for older patches or patches without a
status header.

I would imagine it could be run in a similar way to how the Feature
Process is run, with a Patch Wrangler (Team) who is(are) deputized to
seek out maintainers when updates concern patches status are needed.

Did you also intend to draw a line in the sand concerning the age of a
patch? If a patch is a certain age it automatically needs more
frequent status updates? Sort of like when you reach a certain age and
you need to go in for a colonoscopy on a regular basis?


-jef

_______________________________________________
fedora-advisory-board mailing list
fedora-advisory-board@redhat.com
http://www.redhat.com/mailman/listinfo/fedora-advisory-board
 
Old 05-13-2008, 07:23 PM
Jason L Tibbitts III
 
Default The Debian/Ubuntu SSL bug

>>>>> "GD" == Greg DeKoenigsberg <gdk@redhat.com> writes:

GD> Are there any steps we can take to protect ourselves from this
GD> kind of mistake -- in which a packager does something dumb to the
GD> package and no one notices it?

Well, we're starting with
http://fedoraproject.org/wiki/PackagingDrafts/PatchUpstreamStatus
which has been passed by the packaging committee and ratified by
FESCo. Of course, it's not mandatory, but it's a start. (And as much
as I hate to think about more bureaucracy, it's probably worth
considering whether it should be mandatory in light of the problem
under discussion.)

>From here we can both extend the information we keep about patches and
write some tools for tracking and displaying that information so that
folks can examine the patch status of a package without having to read
the specfile or pulling patches from CVS.

- J<

_______________________________________________
fedora-advisory-board mailing list
fedora-advisory-board@redhat.com
http://www.redhat.com/mailman/listinfo/fedora-advisory-board
 
Old 05-13-2008, 07:43 PM
Greg DeKoenigsberg
 
Default The Debian/Ubuntu SSL bug

On Tue, 13 May 2008, Dave Jones wrote:


On Tue, May 13, 2008 at 02:45:29PM -0400, Greg DeKoenigsberg wrote:
>
> So I've been having a conversation with Mark Cox about the
> Debian/Ubuntu SSL bug. This is basically a horror story of what can
> go wrong when packagers don't maintain close relationships with
> upstream. I asked Mark, "what security policies do we have in place
> to keep this from happening in Fedora-land?" And his response was, "I
> don't know, what security policies do we have in place to keep this
> from happening in Fedora-land?"

>
> We know that RHEL is secure and stable, and we *do* have safeguards in
> place to prevent this from happening in RHEL-land. But a mistake like
> this in Fedora-land would be every bit as bad for the Red Hat and
> Fedora brands.

>
> Are there any steps we can take to protect ourselves from this kind of
> mistake -- in which a packager does something dumb to the package and
> no one notices it?


The thing I find amazing about this bug is that it took 2 years for
someone to notice it. I think in part this is due to the size of debian
making it pretty much impossible for someone to review every change that
goes in. The casual observer just has no chance to keep up with the rate
of change. (and there are parallels to kernel development here, git has
increased the merge rate to the point where one person really can't
sanely review everything that goes in).


I think the only real answer is to make sure that for critical packages,
there exists >1 person actually reading the changes that go in. Though
the possibility of groupthink taints even this approach. Clearly a
number of debian developers thought that change was a great idea,
without realising the consequences.


Seems like a good idea -- not that we should go do it tomorrow or
anything, but it seems sensible to me. How do we identify which packages
are critical? Certainly anything that creates encrypted keys goes in this
bucket.


Other than review though, and making sure that any patches we carry are
either a) short-lived [ie, on their way upstream], or b) carried forever
for really good reasons.


Something the SuSE guys have done which I'm thinking we should adopt for
our patches (in the kernel at least), is a header at the top of each
patch detailing its upstream status, (and if not upstream, why not).


This sounds like a *really* good idea to me. It sounds sensible, highly
auditable, not too much of a headache, and something that packagers coudl
keep up with.



Additionally, something else I think we should be doing more of is
automated testing. We do a bunch of this for RHEL already. There was
a whole brouhaha at rh summit a few years ago about how we were going to
opensource our QA test harnesses. None of that happened afaict.
Perhaps it'd be faster to reinvent something new anyway.
(Seemed to work out for the better that way with koji).


This is a whole other kettle of worms, I'm afraid. And yes, maybe we
should reinvent from scratch, but I don't even know where we'd begin.


--g

--
Greg DeKoenigsberg
Community Development Manager
Red Hat, Inc. :: 1-919-754-4255
"To whomsoever much hath been given...
...from him much shall be asked"

_______________________________________________
fedora-advisory-board mailing list
fedora-advisory-board@redhat.com
http://www.redhat.com/mailman/listinfo/fedora-advisory-board
 
Old 05-13-2008, 07:44 PM
Dennis Gilmore
 
Default The Debian/Ubuntu SSL bug

On Tuesday 13 May 2008, Dave Jones wrote:

>
> Additionally, something else I think we should be doing more of is
> automated testing. We do a bunch of this for RHEL already. There was
> a whole brouhaha at rh summit a few years ago about how we were going to
> opensource our QA test harnesses. None of that happened afaict.
> Perhaps it'd be faster to reinvent something new anyway.
> (Seemed to work out for the better that way with koji).

koji is brew rebadged, it wasn't written from scratch for us. I do wonder
what ever happend to the QA framework we were promised? Will Woods please
speak up.

Dennis
_______________________________________________
fedora-advisory-board mailing list
fedora-advisory-board@redhat.com
http://www.redhat.com/mailman/listinfo/fedora-advisory-board
 

Thread Tools




All times are GMT. The time now is 01:12 AM.

VBulletin, Copyright ©2000 - 2014, Jelsoft Enterprises Ltd.
Content Relevant URLs by vBSEO ©2007, Crawlability, Inc.
Copyright 2007 - 2008, www.linux-archive.org