FAQ Search Today's Posts Mark Forums Read
» Video Reviews

» Linux Archive

Linux-archive is a website aiming to archive linux email lists and to make them easily accessible for linux users/developers.


» Sponsor

» Partners

» Sponsor

Go Back   Linux Archive > Redhat > Fedora Development

 
 
LinkBack Thread Tools
 
Old 03-11-2010, 07:24 AM
yersinia
 
Default Adventurous yet Safety-Minded

On Wed, Mar 10, 2010 at 1:54 PM, Alexander Kahl <e-user@fsfe.org> wrote:


-----BEGIN PGP SIGNED MESSAGE-----

Hash: SHA1



On 03/10/2010 01:06 PM, Steven I Usdansky wrote:

There is no real recovery for traditional package systems as each change

on a package (install, update, remove etc.) changes the state of the

system as a whole, i.e. the system relies on side effects (mostly

writing files into shared/global locations) and provides no referential

transparency for such actions, same output for same input is never

guaranteed.
My
(silly perhaps) personal opinion that the rollback part of a package management system
depends very much on QA which are made the same packages in first place. The
way you write* seems that the problem is insoluble for a package
manager: if so it did seem strange that several rpm developers have lost so time to implement it in the first place and to extend it over time.* But I think this has already been discussed in the past, many time. For example http://lists.rpm.org/pipermail/rpm-maint/2008-February/001912.html


and

http://lists.rpm.org/pipermail/rpm-list/2009-April/000227.html

http://lists.rpm.org/pipermail/rpm-list/2009-April/000231.html

*

But the problem is always relevant
http://www.mancoosi.org/work/#wp3

Regards



--
devel mailing list
devel@lists.fedoraproject.org
https://admin.fedoraproject.org/mailman/listinfo/devel
 
Old 03-11-2010, 09:53 AM
Alexander Kahl
 
Default Adventurous yet Safety-Minded

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

On 03/11/2010 09:24 AM, yersinia wrote:
> On Wed, Mar 10, 2010 at 1:54 PM, Alexander Kahl <e-user@fsfe.org> wrote:
>
>> -----BEGIN PGP SIGNED MESSAGE-----
>> Hash: SHA1
>>
>> On 03/10/2010 01:06 PM, Steven I Usdansky wrote:
>> There is no real recovery for traditional package systems as each change
>> on a package (install, update, remove etc.) changes the state of the
>> system as a whole, i.e. the system relies on side effects (mostly
>> writing files into shared/global locations) and provides no referential
>> transparency for such actions, same output for same input is never
>> guaranteed.
>>
> My (silly perhaps) personal opinion that the rollback part of a package
> management system depends very much on QA which are made the same packages
> in first place. The way you write seems that the problem is insoluble for a
> package manager: if so it did seem strange that several rpm developers have
> lost so time to implement it in the first place and to extend it over time.
Because they've never seemed to question the foundations of a system
that relies on global state made up by interdependence between packages
during and after build and install time. What we have now is basically
an automaton that works deterministic in theory but is unpredictable in
practice. Using a mechanism like filesystem snapshots could enable real
rollbacks to a state that did actually exist but will make you lose
every other change happened in the meantime which might not be what you
want. Implementing and using RPM rollbacks, including filesystem-level
rollbacks or not, is like "guessing" a lost (full rollback of all
actions) or new (partly rollback, some actions) state by interpolating
stuff, like "rollback of B after installation of {A,B,C} at state `e'
could yield a state approximate to the one after installation of {A,C}
at state `e'". This can turn out as a lucky day in Las Vegas at best.
Running C after B has been installed could have modified files belonging
to A or C, installation of C might behave different if B was installed
during %post (currently we work around the latter case by always
requiring all otherwise optional dependencies being present).
Better but still not perfect would be "filesystem rollback to `e',
installation of {A,C}" to get the "real" desired state without B.

There are of course cases where package maintainer work and QA can
improve rollback quality:
Imagine mysql-server would run an unconditional database format upgrade
upon update in %post (which it luckily doesn't!) ruining your day after
an RPM rollback. Here, the maintainers could increase the atomicity of
the package by creating a snapshot of the currently running database(s),
upgrading the snapshot and storing the old data to an offside storage
that is brought back into place upon rollback.
To my knowledge right now though, we neither have a mechanism to help
with that nor any guideline encouraging such behavior. Same goes for
configuration files with something better than those lame .rpm{save,new}
backup files.

> But I think this has already been discussed in the past, many time. For
> example http://lists.rpm.org/pipermail/rpm-maint/2008-February/001912.html

Quoting from that post:
> - Most scriptlets in RedHat packages are fairly simple so their
> typically is nothing to really undo.
But if they're not simple (which is the case too often), the user is
screwed.

> - In house scriptlets were always written with the view that they
> were going to be in rolled back so we made them work.
In house scriptlets, yeah, but we're a community with many contributors.
And there *are* update scriptlets as noted above without proper offside
storage and rollbacks for that.

> - Often we could work around an upstream scriptlets issuses in
> rollback via some other rpm, or some code external to rpm.
Which will, with current technology in place, again give an
unpredictable state `e3' instead of install (e, {A,C}) -> e2.

> ...
> That said it will never be as reliable and simple as a rollback of a
> filesystem snapshot (however that is implemented). OTOH, that solution
> is outside the rpm problem space.
But it's in scope of the packaging and filesystem layout spaces, read on.

> and
>
> http://lists.rpm.org/pipermail/rpm-list/2009-April/000227.html

Quoting again:
> The problem with rollback is that scripts with in the rpm can execute
> arbitrary shell commands.
Correct.

> This means that RPM cannot foresee which files are
> affected by a package and therefore cannot backup all files.
Correct, we're doing changes without properly recording former state.

> http://lists.rpm.org/pipermail/rpm-list/2009-April/000231.html
> Is it a implementation problem - difficult, sure, no dubt - or a
> universal law ? I think the first but JMHO.
No, it's a logical problem of global state and referential transparency.
There are several scopes of this where rolling back is affected, most of
the actually covered and solved by Nix in elegant ways:
- - Non-atomic, global state of the runtime system configuration comprised
by all files residing in /etc at a given state. This is indeed to a
large degree outside of RPM's scope as users and tools executing during
runtime can change these files, making the problem one of filesystem
layout: Nix provides mechanisms to create snapshots of a set of
configuration files affecting one or more programs' behaviors both
during install and runtime so rollback to any former state is possible
without any low-level filesystem implementation magic.
- - Non-atomic global state of files not changing during runtime comprised
by all files residing in /bin, /sbin, /lib, /lib64, /boot, /usr and
usually only changed by RPM during transactions. While this is the
easiest to cover by implementing RPM rollbacks it's still not
referential transparent that way due to filesystems' hierarchical
natures. Nix solves this by having the physical files reside in a unique
place identified and referenced by a unique hash per package *and* per
package version allowing for an arbitrary number of versions installed
in parallel; a set of libraries and binaries each of a given version in
use for one user or system-wide as the default is, again, identified by
a hash and implemented by something as simple as symlinks giving the
admin and users the power of rollbacks by just tweaking a symlink
location and even enabling package installations, updates and rollbacks
of unprivileged users (!) without affecting real system state.
- - Non-atomic global state of files that are mutable at any time, usually
residing in /var and /home. This is the most difficult, unpredictable
area. One can, however, provide rollbacks by using a mechanism like
drafted above, creating snapshots during transactions, and, again, using
Nix' hash/symlink mechanisms to allow for rollbacks even without writing
legions of code files. /home however is - of course - permanently
outside all scope of packaging

So what you get is an omnipresent, healthy and minimal level of
redundancy that allows for >0.9 (just a guess) clean rollbacks without
using magic, predictable states through referential transparency /
atomicity of package installations, full support of *real* parallel
package version installations without any further packager addo, total
invulnerability from ABI breakage.. and the kitchen sink.

> But the problem is always relevant
> http://www.mancoosi.org/work/#wp3
Most of what is described here is already covered by Nix (leaving out
the hardware driver part).

This is why I endorse dumping yum, RPM *and* the stupid FHS in favor of
Nix, focusing all development on something that is clearly superior by
substantiating the synthesis of DOS (*cough*) and POSIX style
installations: Minimal redundancy, maximum compatibility, (mostly)
predictable rollbacks, atomicity, referential transparency: Purely
functional package management.

Oh, and by the way, we could leave behind all those discussions
regarding dynamic linking: RPATH for everything and everyone. If you've
linked against libfoo-4.2-2 during build time, libfoo-4.2-2 will be
present during runtime, same location, same file. Period.

Regards

- --
Alexander Kahl
GNU/Linux Software Developer
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.10 (GNU/Linux)
Comment: Using GnuPG with Fedora - http://enigmail.mozdev.org/

iEYEARECAAYFAkuYy54ACgkQVTRddCFHw121KQCgpXdw41ospr ZoRXjdB48sg6B9
Ww4AnAqSIkrK0uFeL0VeJUFpCkLgVWQD
=RTmW
-----END PGP SIGNATURE-----
--
devel mailing list
devel@lists.fedoraproject.org
https://admin.fedoraproject.org/mailman/listinfo/devel
 
Old 03-11-2010, 10:40 AM
yersinia
 
Default Adventurous yet Safety-Minded

On Thu, Mar 11, 2010 at 11:53 AM, Alexander Kahl <e-user@fsfe.org> wrote:

-----BEGIN PGP SIGNED MESSAGE-----

Hash: SHA1



On 03/11/2010 09:24 AM, yersinia wrote:

> On Wed, Mar 10, 2010 at 1:54 PM, Alexander Kahl <e-user@fsfe.org> wrote:

>

>> -----BEGIN PGP SIGNED MESSAGE-----

>> Hash: SHA1

>>

>> On 03/10/2010 01:06 PM, Steven I Usdansky wrote:

No reply. But i am sure this was a very interesting thread. BTW, It was for me.

Best Regards

--
devel mailing list
devel@lists.fedoraproject.org
https://admin.fedoraproject.org/mailman/listinfo/devel
 
Old 03-11-2010, 12:16 PM
Kevin Kofler
 
Default Adventurous yet Safety-Minded

Alexander Kahl wrote:

> On 03/10/2010 08:26 PM, Steven I Usdansky wrote:
>> If there's a magic solution that will satisfy the vast majority
>> of Fedora users, I have absolutely no clue as to what it might be.
>
> Please read: http://nixos.org/nix/
> Esp. "Atomic upgrades and rollbacks"

That's not a solution for Fedora at all. It's a completely different package
manager (competing with RPM). It also works by keeping all the old versions
of all packages stored on disk all the time, a massive waste of disk space,
and also not compliant with the FHS (Filesystem Hierarchy Standard).

Kevin Kofler

--
devel mailing list
devel@lists.fedoraproject.org
https://admin.fedoraproject.org/mailman/listinfo/devel
 
Old 03-11-2010, 12:25 PM
Alexander Kahl
 
Default Adventurous yet Safety-Minded

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

On 03/11/2010 02:16 PM, Kevin Kofler wrote:
> Alexander Kahl wrote:
>
>> On 03/10/2010 08:26 PM, Steven I Usdansky wrote:
>>> If there's a magic solution that will satisfy the vast majority
>>> of Fedora users, I have absolutely no clue as to what it might be.
>>
>> Please read: http://nixos.org/nix/
>> Esp. "Atomic upgrades and rollbacks"
>
> That's not a solution for Fedora at all. It's a completely different package
> manager (competing with RPM).
Well, now that you mention it..!

> It also works by keeping all the old versions
> of all packages stored on disk all the time, a massive waste of disk space,
Please define "massive" if you're keeping exactly what's needed to keep
everything running and prune anything else by using a sophisticated,
tunable garbage collection mechanism.

> and also not compliant with the FHS (Filesystem Hierarchy Standard).
Yep. It's much better, comparable to the GAC (global assembly cache) of
the CLR (Mono), just applied as a packaging / filesystem layout paradigm
for the whole system. Please read my other post in reply to yersinia -
sorry, this thread is cluttered, bad MUAs.

- --
Alexander Kahl
GNU/Linux Software Developer
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.10 (GNU/Linux)
Comment: Using GnuPG with Fedora - http://enigmail.mozdev.org/

iEYEARECAAYFAkuY70YACgkQVTRddCFHw12oiACgqZp+94Q6rU QM2jAaKzWfobP8
oOgAoKTV9OurZ8oBq1pwn2oniZFXPOeC
=GsWd
-----END PGP SIGNATURE-----
--
devel mailing list
devel@lists.fedoraproject.org
https://admin.fedoraproject.org/mailman/listinfo/devel
 
Old 03-11-2010, 01:05 PM
Tomas Mraz
 
Default Adventurous yet Safety-Minded

On Thu, 2010-03-11 at 11:53 +0100, Alexander Kahl wrote:

> Most of what is described here is already covered by Nix (leaving out
> the hardware driver part).
>
> This is why I endorse dumping yum, RPM *and* the stupid FHS in favor of
> Nix, focusing all development on something that is clearly superior by
> substantiating the synthesis of DOS (*cough*) and POSIX style
> installations: Minimal redundancy, maximum compatibility, (mostly)
> predictable rollbacks, atomicity, referential transparency: Purely
> functional package management.
>
> Oh, and by the way, we could leave behind all those discussions
> regarding dynamic linking: RPATH for everything and everyone. If you've
> linked against libfoo-4.2-2 during build time, libfoo-4.2-2 will be
> present during runtime, same location, same file. Period.

I'm sorry for not studying the Nix concepts in depth, but can you please
answer me just a simple question how security or other critical bugfixes
_in libraries_ are handled under this "RPATH for everything" paradigm?
--
Tomas Mraz
No matter how far down the wrong road you've gone, turn back.
Turkish proverb

--
devel mailing list
devel@lists.fedoraproject.org
https://admin.fedoraproject.org/mailman/listinfo/devel
 
Old 03-11-2010, 01:16 PM
Bruno Wolff III
 
Default Adventurous yet Safety-Minded

On Thu, Mar 11, 2010 at 11:53:18 +0100,
Alexander Kahl <e-user@fsfe.org> wrote:
>
> There are of course cases where package maintainer work and QA can
> improve rollback quality:

I got burned a while back by a Sunbird change (maybe in a library it
uses) that broke access to my local calendar data. I was not a happy camper.
I was even less happy when I asked about going backwards or working around
the problem to get access to the data and the maintainer was unable to help.
Fortunately I eventually figured out a work around, but not having access
to that data for a while caused me some grief.
The good news was that this happened in update-testing, so that lots of
people didn't get burned.
https://bugzilla.redhat.com/show_bug.cgi?id=542736
--
devel mailing list
devel@lists.fedoraproject.org
https://admin.fedoraproject.org/mailman/listinfo/devel
 
Old 03-11-2010, 01:57 PM
Alexander Kahl
 
Default Adventurous yet Safety-Minded

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

On 03/11/2010 03:05 PM, Tomas Mraz wrote:
> On Thu, 2010-03-11 at 11:53 +0100, Alexander Kahl wrote:
>> Oh, and by the way, we could leave behind all those discussions
>> regarding dynamic linking: RPATH for everything and everyone. If you've
>> linked against libfoo-4.2-2 during build time, libfoo-4.2-2 will be
>> present during runtime, same location, same file. Period.
>
> I'm sorry for not studying the Nix concepts in depth, but can you please
> answer me just a simple question how security or other critical bugfixes
> _in libraries_ are handled under this "RPATH for everything" paradigm?

Yep: Nix is a mixed source/binary distro, as far as I understand the
documentation, stuff gets (re)build locally when necessary; furthermore
Nix uses PatchELF: http://nixos.org/patchelf.html

I've been running NixOS in a VM for a while and created Nix RPMs for
Fedora (unreleased though) to investigate and understand the system better.
Furthermore the system heavily relies on automated testing and nightly
builds before pushing out anything anywhere.

- --
Alexander Kahl
GNU/Linux Software Developer
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.10 (GNU/Linux)
Comment: Using GnuPG with Fedora - http://enigmail.mozdev.org/

iEYEARECAAYFAkuZBMEACgkQVTRddCFHw12SoACePlZeFvrG/SVpgeFN/E2Uf1Be
BM0AoKHYEcAVrNch5MisXtK4LbeS7Vkb
=nSwo
-----END PGP SIGNATURE-----
--
devel mailing list
devel@lists.fedoraproject.org
https://admin.fedoraproject.org/mailman/listinfo/devel
 
Old 03-11-2010, 09:24 PM
Przemek Klosowski
 
Default Adventurous yet Safety-Minded

On 03/10/2010 07:54 AM, Alexander Kahl wrote:
> -----BEGIN PGP SIGNED MESSAGE-----
> Hash: SHA1
>
> On 03/10/2010 01:06 PM, Steven I Usdansky wrote:
>> Instead of worrying about the occasional brokenness caused by an update to a stable release, how about focusing on a mechanism to easily recover from it?
>
> There is no real recovery for traditional package systems as each change
> on a package (install, update, remove etc.) changes the state of the
> system as a whole,

But, but, the BTRFS-based rollback just rolls the entire disk back to
the state before an upgrade. It's a filesystem-based snapshot. What
could go wrong with that?

Of course BTRFS is still kind-of experimental...
--
devel mailing list
devel@lists.fedoraproject.org
https://admin.fedoraproject.org/mailman/listinfo/devel
 
Old 03-11-2010, 09:54 PM
Kevin Kofler
 
Default Adventurous yet Safety-Minded

Alexander Kahl wrote:
> Please define "massive" if you're keeping exactly what's needed to keep
> everything running and prune anything else by using a sophisticated,
> tunable garbage collection mechanism.

The whole point of the exercise was to do a lot of updates. But the more
updates we do, the more disk space the old versions you're keeping around
are going to eat up.

>> and also not compliant with the FHS (Filesystem Hierarchy Standard).
> Yep. It's much better, comparable to the GAC (global assembly cache) of
> the CLR (Mono),

Yuck! Do you seriously call that "better"?

Anyway, the FHS is not really negotiable.

Kevin Kofler

--
devel mailing list
devel@lists.fedoraproject.org
https://admin.fedoraproject.org/mailman/listinfo/devel
 

Thread Tools




All times are GMT. The time now is 07:20 AM.

VBulletin, Copyright ©2000 - 2014, Jelsoft Enterprises Ltd.
Content Relevant URLs by vBSEO ©2007, Crawlability, Inc.
Copyright 2007 - 2008, www.linux-archive.org