Linux Archive

Linux Archive (http://www.linux-archive.org/)
-   Ubuntu Development (http://www.linux-archive.org/ubuntu-development/)
-   -   #1 Complaint about Ubuntu: Updates break things (http://www.linux-archive.org/ubuntu-development/215030-1-complaint-about-ubuntu-updates-break-things.html)

"Cody A.W. Somerville" 12-19-2008 04:46 PM

#1 Complaint about Ubuntu: Updates break things
 
Hello Folks,

*After getting yet another phone call from one of the many average, everyday users I know using Ubuntu about how after downloading and installing yet another mammoth load of updates on her Ubuntu machine, running a stable, supported release, her computer crashed with the following scary message, I've decided to write this e-mail.


Aborted because of invalid compressed format (err=2)

Kernel Panic: SYNCING VFS, unable to mount root fs on unknown-block.
The #1 complaint about Ubuntu that I hear time and time again from both joe blow end users and business/enterprise users is that updates to stable releases *break* stable releases. However, the impression I gather of the developer mentality is that we want to increase our bandwidth and ability, via changes to policy and/or processes, to make SRUs quicker and fastar. Instead, I believe we should be focusing on how to improve the quality of these updates and reduce the number of regressions and breakages occurring in stable releases.


For example, "Oh, we'll fix that in -updates" is the wrong attitude IMHO. "The stable serie usually has fixes worth having and not too many
unstable changes (though the stable SVN code usually doesn't get lot of
testing)" is the wrong attitude IMHO. "I'll just rely on someone else to test my changes properly and thoroughly" is the wrong attitude IMHO!

I think its rather sad when I hear comments like the following:


dude, update breakage is why I jump to the devel release as soon as I can.

It really sucks when you can't stick with a release for more than a few months before it's kinda unusable.

Specially with LTS releases, 3 years? heck no
Updates are the most infuriating part about using Ubuntu. I'd disable them but even -security breaks my computer sometimes because they base off of the packages in -updates which means I get non-security updates I don't want!

*Oh look, time to download the usual barrage of 20-60 updates I get with Ubutu each week. I wonder if its going to break my computer this week.
*Ubuntu updates is like Russian roulette.

Updates breaking stable releases is a serious issue.* I think we need to re-evaluate stable release updates and the numerous exceptions/breaks that have been granted to packages (or groups of packages) such as gnome.


Cheers,

--
Cody A.W. Somerville
Software Systems Release Engineer
Custom Engineering Solutions Group
Canonical OEM Services
Cell: 506-449-5899
Email: cody.somerville@canonical.com


--
ubuntu-devel mailing list
ubuntu-devel@lists.ubuntu.com
Modify settings or unsubscribe at: https://lists.ubuntu.com/mailman/listinfo/ubuntu-devel

"Odysseus Flappington" 12-19-2008 04:56 PM

#1 Complaint about Ubuntu: Updates break things
 
2008/12/19 Cody A.W. Somerville <cody-somerville@ubuntu.com>

Hello Folks,

*After getting yet another phone call from one of the many average, everyday users I know using Ubuntu about how after downloading and installing yet another mammoth load of updates on her Ubuntu machine, running a stable, supported release, her computer crashed with the following scary message, I've decided to write this e-mail.



Aborted because of invalid compressed format (err=2)


Kernel Panic: SYNCING VFS, unable to mount root fs on unknown-block.
The #1 complaint about Ubuntu that I hear time and time again from both joe blow end users and business/enterprise users is that updates to stable releases *break* stable releases. However, the impression I gather of the developer mentality is that we want to increase our bandwidth and ability, via changes to policy and/or processes, to make SRUs quicker and fastar. Instead, I believe we should be focusing on how to improve the quality of these updates and reduce the number of regressions and breakages occurring in stable releases.



For example, "Oh, we'll fix that in -updates" is the wrong attitude IMHO. "The stable serie usually has fixes worth having and not too many
unstable changes (though the stable SVN code usually doesn't get lot of
testing)" is the wrong attitude IMHO. "I'll just rely on someone else to test my changes properly and thoroughly" is the wrong attitude IMHO!

I think its rather sad when I hear comments like the following:



dude, update breakage is why I jump to the devel release as soon as I can.


It really sucks when you can't stick with a release for more than a few months before it's kinda unusable.


Specially with LTS releases, 3 years? heck no
Updates are the most infuriating part about using Ubuntu. I'd disable them but even -security breaks my computer sometimes because they base off of the packages in -updates which means I get non-security updates I don't want!


*Oh look, time to download the usual barrage of 20-60 updates I get with Ubutu each week. I wonder if its going to break my computer this week.

*Ubuntu updates is like Russian roulette.

Updates breaking stable releases is a serious issue.* I think we need to re-evaluate stable release updates and the numerous exceptions/breaks that have been granted to packages (or groups of packages) such as gnome.



Cheers,

--
Cody A.W. Somerville
Software Systems Release Engineer
Custom Engineering Solutions Group
Canonical OEM Services
Cell: 506-449-5899
Email: cody.somerville@canonical.com




--

ubuntu-devel mailing list

ubuntu-devel@lists.ubuntu.com

Modify settings or unsubscribe at: https://lists.ubuntu.com/mailman/listinfo/ubuntu-devel



Funny you should say that actually.

This is very true. I stopped installing updates about a year and a half ago. Every time I install, I get things set-up and working, then I uncheck Update Manager from the session startup altogether.


Next time anything gets updated is when I do a reinstall (usually once a year), or if I'm trying to fix something specific, otherwise, if it ain't broke don't fix it.

Alex

--
ubuntu-devel mailing list
ubuntu-devel@lists.ubuntu.com
Modify settings or unsubscribe at: https://lists.ubuntu.com/mailman/listinfo/ubuntu-devel

Steve Langasek 12-19-2008 05:20 PM

#1 Complaint about Ubuntu: Updates break things
 
On Fri, Dec 19, 2008 at 12:46:28PM -0500, Cody A.W. Somerville wrote:

> After getting yet another phone call from one of the many average, everyday
> users I know using Ubuntu about how after downloading and installing yet
> another mammoth load of updates on her Ubuntu machine, running a stable,
> supported release, her computer crashed with the following scary message,
> I've decided to write this e-mail.

> Aborted because of invalid compressed format (err=2)
> >
> Kernel Panic: SYNCING VFS, unable to mount root fs on unknown-block.
> >

Please note the procedure documented in
<https://wiki.ubuntu.com/StableReleaseUpdates#Procedure> for handling
SRU regressions that reach -updates. Regressions *are* a serious business,
and will be handled seriously by the SRU team, but we have to be told about
them in order to act.

The error message listed above sounds like an initramfs loading error at
boot time. I'm not sure how that could come to pass as a consequence of an
SRU; do you have any more information about this?

Given that such a problem was not detected during the SRU verification
process, we definitely need more information here: what release the user is
running, what kernel update was most recently installed, probably a list of
all updates applied since the last reboot.

--
Steve Langasek Give me a lever long enough and a Free OS
Debian Developer to set it on, and I can move the world.
Ubuntu Developer http://www.debian.org/
slangasek@ubuntu.com vorlon@debian.org

--
ubuntu-devel mailing list
ubuntu-devel@lists.ubuntu.com
Modify settings or unsubscribe at: https://lists.ubuntu.com/mailman/listinfo/ubuntu-devel

Bryce Harrington 12-19-2008 06:57 PM

#1 Complaint about Ubuntu: Updates break things
 
On Fri, Dec 19, 2008 at 10:20:26AM -0800, Steve Langasek wrote:
> On Fri, Dec 19, 2008 at 12:46:28PM -0500, Cody A.W. Somerville wrote:
>
> > After getting yet another phone call from one of the many average, everyday
> > users I know using Ubuntu about how after downloading and installing yet
> > another mammoth load of updates on her Ubuntu machine, running a stable,
> > supported release, her computer crashed with the following scary message,
> > I've decided to write this e-mail.
>
> > Aborted because of invalid compressed format (err=2)
> > >
> > Kernel Panic: SYNCING VFS, unable to mount root fs on unknown-block.
> > >
>
> Please note the procedure documented in
> <https://wiki.ubuntu.com/StableReleaseUpdates#Procedure> for handling
> SRU regressions that reach -updates. Regressions *are* a serious business,
> and will be handled seriously by the SRU team, but we have to be told about
> them in order to act.
>
> The error message listed above sounds like an initramfs loading error at
> boot time. I'm not sure how that could come to pass as a consequence of an
> SRU; do you have any more information about this?

I think the update mechanism gets blamed far more than it deserves.
It's easy to understand why. Consider some scenario like this:

1. sudo cat /dev/random > /etc/X11/xorg.conf

2. Continue using your _stable_ _robust_ Ubuntu O/S for several weeks

3. Notice there's a security update to <foo> that requires a reboot

4. Reboot and OMFG! X broke! The update for <foo> broke X!!
You bastards!

Anyway, not to say that every SRU is perfect, and I don't know the
particulars of this VFS case, but based on the number of times I've seen
bugs incorrectly blamed on updates, I'd withhold judgements about the
updates system until you've definitively proved that an update caused
it. It could easily just be coincidental.

Bryce

--
ubuntu-devel mailing list
ubuntu-devel@lists.ubuntu.com
Modify settings or unsubscribe at: https://lists.ubuntu.com/mailman/listinfo/ubuntu-devel

"Cody A.W. Somerville" 12-19-2008 07:09 PM

#1 Complaint about Ubuntu: Updates break things
 
On Fri, Dec 19, 2008 at 2:57 PM, Bryce Harrington <bryce@canonical.com> wrote:

On Fri, Dec 19, 2008 at 10:20:26AM -0800, Steve Langasek wrote:

> On Fri, Dec 19, 2008 at 12:46:28PM -0500, Cody A.W. Somerville wrote:

>

> > *After getting yet another phone call from one of the many average, everyday

> > users I know using Ubuntu about how after downloading and installing yet

> > another mammoth load of updates on her Ubuntu machine, running a stable,

> > supported release, her computer crashed with the following scary message,

> > I've decided to write this e-mail.

>

> > Aborted because of invalid compressed format (err=2)

> > >

> > Kernel Panic: SYNCING VFS, unable to mount root fs on unknown-block.

> > >

>

> Please note the procedure documented in

> <https://wiki.ubuntu.com/StableReleaseUpdates#Procedure> for handling

> SRU regressions that reach -updates. *Regressions *are* a serious business,

> and will be handled seriously by the SRU team, but we have to be told about

> them in order to act.

>

> The error message listed above sounds like an initramfs loading error at

> boot time. *I'm not sure how that could come to pass as a consequence of an

> SRU; do you have any more information about this?



I think the update mechanism gets blamed far more than it deserves.

It's easy to understand why. *Consider some scenario like this:

<snip>


Anyway, not to say that every SRU is perfect, and I don't know the

particulars of this VFS case, but based on the number of times I've seen

bugs incorrectly blamed on updates, I'd withhold judgements about the

updates system until you've definitively proved that an update caused

it. *It could easily just be coincidental.



Bryce


Sure, I can buy that. I'm not saying that this error is caused by an update (although I can guarantee you she wasn't running anything in the commandline let alone under in the commandline under sudo). However, there has been enough times where it has been connected to an update that I feel we do need to discuss this.


For example, I remember one time an update to the nvidia binary blob drivers caused the xserver to fail to start in certain cases. It didn't affect the *right* people or *enough* people for it to get the attention it deserved. I asked someone about it because I did happen to hear about it from one of my friends who I converted to Ubuntu and was told that by whoever uploaded it that they had heard about cases where that was occurring but there were no plans for remedial action.


Pushing updates to -updates is so much riskier than people realize due to the massively different amount of QA resources invested in -updates compared to testing the development release. Often times, breakages only affect a subset of people using Ubuntu or it causes a problem and a lot of folks are savy enough to work around it or know someone who is. However, people get fed up and tired of having to fix things after updates and turn away from Ubuntu; an undesirable conclusion to say the least.


Cheers,
--
Cody A.W. Somerville
Software Systems Release Engineer
Custom Engineering Solutions Group
Canonical OEM Services
Cell: 506-449-5899
Email: cody.somerville@canonical.com


--
ubuntu-devel mailing list
ubuntu-devel@lists.ubuntu.com
Modify settings or unsubscribe at: https://lists.ubuntu.com/mailman/listinfo/ubuntu-devel

"Justin Dugger" 12-19-2008 07:23 PM

#1 Complaint about Ubuntu: Updates break things
 
> For example, I remember one time an update to the nvidia binary blob drivers
> caused the xserver to fail to start in certain cases. It didn't affect the
> *right* people or *enough* people for it to get the attention it deserved. I
> asked someone about it because I did happen to hear about it from one of my
> friends who I converted to Ubuntu and was told that by whoever uploaded it
> that they had heard about cases where that was occurring but there were no
> plans for remedial action.

Normally, I'd cite something about anecdotal evidence and whatnot, but
if I remember correctly, this is the case that led to the creation of
-proposed. That would be remedial action, no?

Actually, I think I'll harp on anecdotal evidence anyways. The first
step towards improving is putting in place tools to measure the status
quo, so we can track progress. How can we measure the impact and
frequency of regressions in -SRU and friends?

Justin Dugger

--
ubuntu-devel mailing list
ubuntu-devel@lists.ubuntu.com
Modify settings or unsubscribe at: https://lists.ubuntu.com/mailman/listinfo/ubuntu-devel

Bryce Harrington 12-19-2008 07:44 PM

#1 Complaint about Ubuntu: Updates break things
 
On Fri, Dec 19, 2008 at 03:09:01PM -0500, Cody A.W. Somerville wrote:
> On Fri, Dec 19, 2008 at 2:57 PM, Bryce Harrington <bryce@canonical.com>wrote:
> Sure, I can buy that. I'm not saying that this error is caused by an update
> (although I can guarantee you she wasn't running anything in the commandline
> let alone under in the commandline under sudo). However, there has been
> enough times where it has been connected to an update that I feel we do need
> to discuss this.
>
> For example, I remember one time an update to the nvidia binary blob drivers
> caused the xserver to fail to start in certain cases. It didn't affect the
> *right* people or *enough* people for it to get the attention it
> deserved.

When did this happen? I cannot recall ever seeing a binary X driver
update accepted by the SRU team.

(I don't even bother pushing for them anymore; instead I work with the
vendors to make them aware of our release dates and to do testing
earlier - which is a much better solution for a range of reasons).

Even if a binary X driver SRU did happen once, binary drivers are a
whole other kettle of fish since the lack of source code inhibits the
SRU team (or really any of us) from doing their job. So conclusions
built off experiences with bad binary driver updates aren't going to
give very good generalizations for SRU processes overall.


Anyway, I can sympathize with your concerns about the SRU process; a lot
of what you're saying is preaching to the choir, and I think if you have
ideas on specific ways things could be improved, they would be well
received. However, I have to say in my own experience, in updating
dozens of machines owned by me and family, I can't recall ever running
into a problem due to an update (at least, none that couldn't be
resolved by simply logging out and logging back in). I do run into
problems sometimes with _upgrades_, but not updates. I don't know if
anyone has statistical data on this, but from my anecdotal experience
the SRU process has been working quite well.

Bryce

--
ubuntu-devel mailing list
ubuntu-devel@lists.ubuntu.com
Modify settings or unsubscribe at: https://lists.ubuntu.com/mailman/listinfo/ubuntu-devel

"Douglass Clem" 12-19-2008 09:00 PM

#1 Complaint about Ubuntu: Updates break things
 
Suppose a new repository mirror was set up, which was updated 12 hours prior to the rest of the mirrors around the world. (I know that some security updates could not afford a 12 hour delay, but I think many updates could.) If properly promoted, then I could imagine a lot of developers, plus people who are not developers, but advanced users / system administrators, would get their updates from this mirror. This I think could catch many update related problems early, before they went out to the general populace, and in time for developers to fix the issue.


Anyone think this is a plausible/good idea?

Douglass Clem
crashsystems.net
Public Key: http://crashsystems.net/pubkey.asc

--
ubuntu-devel mailing list
ubuntu-devel@lists.ubuntu.com
Modify settings or unsubscribe at: https://lists.ubuntu.com/mailman/listinfo/ubuntu-devel

Sam Tygier 12-19-2008 10:27 PM

#1 Complaint about Ubuntu: Updates break things
 
Bryce Harrington wrote:
> On Fri, Dec 19, 2008 at 03:09:01PM -0500, Cody A.W. Somerville wrote:
>> For example, I remember one time an update to the nvidia binary blob drivers
>> caused the xserver to fail to start in certain cases. It didn't affect the
>> *right* people or *enough* people for it to get the attention it
>> deserved.
>
> When did this happen? I cannot recall ever seeing a binary X driver
> update accepted by the SRU team.

could that be the X update that cause X to fail for many intel users back in ~2006 (i think). that led to the introduction of the proposed repos (or at least the introduction of the proposed repo was closely correlated in time).

sam

--
ubuntu-devel mailing list
ubuntu-devel@lists.ubuntu.com
Modify settings or unsubscribe at: https://lists.ubuntu.com/mailman/listinfo/ubuntu-devel

"Cody A.W. Somerville" 12-20-2008 12:57 AM

#1 Complaint about Ubuntu: Updates break things
 
On Fri, Dec 19, 2008 at 6:27 PM, Sam Tygier <samtygier@yahoo.co.uk> wrote:

Bryce Harrington wrote:


On Fri, Dec 19, 2008 at 03:09:01PM -0500, Cody A.W. Somerville wrote:


For example, I remember one time an update to the nvidia binary blob drivers

caused the xserver to fail to start in certain cases. It didn't affect the

*right* people or *enough* people for it to get the attention it

deserved.




When did this happen? *I cannot recall ever seeing a binary X driver

update accepted by the SRU team.




could that be the X update that cause X to fail for many intel users back in ~2006 (i think). that led to the introduction of the proposed repos (or at least the introduction of the proposed repo was closely correlated in time).

No, this was not the infamous X update I was referring to. As to the exact update that caused the issue I discussed, I'm not entirely sure. However, it was an X related update and the little details I do remember would indicate to me that it would not have mattered if the source code was available or not.

*sam
If folks *are* looking for specific examples, just ask the Canonical OEM Services project engineers ;]


However, I think a more productive discussion would instead focus on how we can properly test tricky updates and updates that have a higher risk for regression and how we can get simpler updates out quick and effectively while not sacrificing quality testing.


To start the ball, I'll throw an idea out: the introduction of multi-tier system that would classify an update based on an agreed set of quantitative and qualitative criteria such as where the component falls in the stack (ie. distinction between the kernel, desktop environment, and an application), popcon score, etc. etc. Each tier would demand a different degree of testing, verification, time in -proposed, sign off from different parties, etc. That way we ensure appropirate people are looking at the SRUs, appropriate testing is occuring, and appropriate happiness is occuring! :)


Cheers,
--
Cody A.W. Somerville
Software Systems Release Engineer
Custom Engineering Solutions Group
Canonical OEM Services
Cell: 506-449-5899
Email: cody.somerville@canonical.com


--
ubuntu-devel mailing list
ubuntu-devel@lists.ubuntu.com
Modify settings or unsubscribe at: https://lists.ubuntu.com/mailman/listinfo/ubuntu-devel


All times are GMT. The time now is 11:35 PM.

VBulletin, Copyright ©2000 - 2014, Jelsoft Enterprises Ltd.
Content Relevant URLs by vBSEO ©2007, Crawlability, Inc.