Multi-arch and dependencies on arch: all packages
 

On Thu, Feb 10, 2011 at 05:00:13PM +0100, Raphael Hertzog wrote:
> Steve reported me this problem concerning the current implementation of
> the multiarch spec (he uses my latest pu/multiarch/snapshot/* branch).

> Le mercredi 09 févr. 2011, Steve Langasek a écrit :
> > - I've just marked tzdata (an Architecture: all package) as Multi-Arch:
> > foreign in the Ubuntu archive because I noticed my test libc6 package
> > (libc6 depends on tzdata in Ubuntu but not in Debian) failed to install,
> > listing this dependency as one of the reasons. At first I was going to
> > report this as a bug, but then I remembered this:
> > https://wiki.ubuntu.com/MultiarchSpec#Dependencies%20involving%20Architect ure:%20all%20packages
> >
> > Now dpkg fails to install this package at all, with this error:
> >
> > parsing file '/var/lib/dpkg/tmp.ci/control' near line 18 package 'tzdata':
> > package has multiarch field but is architecture all
> >
> > And doing so complies with this requirement from the spec:
> >
> > "Setting the Multi-Arch field on a package which is Architecture: all is
> > considered an error. dpkg-deb must refuse to generate a .deb with this
> > combination of values. Behavior when trying to install such a package is
> > undefined."
> >
> > (https://wiki.ubuntu.com/MultiarchSpec#Binary%20package%20control%20fields)
> >
> > These two requirements are clearly in conflict. I've updated the wiki
> > page to make it clear that only Multi-Arch: same is disallowed for
> > Architecture: all packages. Please update the dpkg implementation to
> > match when you have a chance.

> I think it's wrong to (have to) add the Multi-Arch field to architecture
> all packages. I would rather suggest that we consider them as
> automatically satisfying any dependency (i.e. the Multi-Arch: foreign
> would be implicit).

> What do other people think?

> An architecture all package that provides something arch specific is
> very rare and it's often meant to be used in situation where you precisely
> want to make this available to other architectures (i.e. syslinux-common).

> So I don't think that we have to go through all the trouble to whitelist
> them one by one.

Sorry, I should have included in my mail the rationale for this requirement
- which I remembered from the drafting sessions but didn't add when updating
the wiki page.

The reason we determined while drafting that Architecture: all packages
should not automatically satisfy dependencies as if they were Multi-Arch:
foreign is that in the current scheme, an Architecture: all package can very
well be used as a metapackage to bridge a dependency between two
Architecture: any packages that need to be of the same architecture.

As a practical example: the python package is Architecture: all. Every
python C extension package is Architecture: any and depends on python as an
/interface/ to the python2.x Architecture: any packages which *must* be of
the same architecture.

I believe the consensus when drafting was that, if we didn't treat
Architecture: all packages as being equivalent to the native arch for
dependency solving, there would be cases when we would have insufficient
information to properly verify dependencies due to the masking effect.

With that rationale, does this requirement make more sense to you? Or do
you see some other way to make this work? (Bearing in mind that both dpkg
and higher-level package managers like apt need to be able to sort through
the deps correctly)

--
Steve Langasek Give me a lever long enough and a Free OS
Debian Developer to set it on, and I can move the world.
Ubuntu Developer http://www.debian.org/
slangasek@ubuntu.com vorlon@debian.org

Multi-arch and dependencies on arch: all packages
 

Steve Langasek <vorlon@debian.org> writes:

> On Thu, Feb 10, 2011 at 05:00:13PM +0100, Raphael Hertzog wrote:
>> Steve reported me this problem concerning the current implementation of
>> the multiarch spec (he uses my latest pu/multiarch/snapshot/* branch).
>
>> Le mercredi 09 févr. 2011, Steve Langasek a écrit :
>> > - I've just marked tzdata (an Architecture: all package) as Multi-Arch:
>> > foreign in the Ubuntu archive because I noticed my test libc6 package
>> > (libc6 depends on tzdata in Ubuntu but not in Debian) failed to install,
>> > listing this dependency as one of the reasons. At first I was going to
>> > report this as a bug, but then I remembered this:
>> > https://wiki.ubuntu.com/MultiarchSpec#Dependencies%20involving%20Architect ure:%20all%20packages
>> >
>> > Now dpkg fails to install this package at all, with this error:
>> >
>> > parsing file '/var/lib/dpkg/tmp.ci/control' near line 18 package 'tzdata':
>> > package has multiarch field but is architecture all
>> >
>> > And doing so complies with this requirement from the spec:
>> >
>> > "Setting the Multi-Arch field on a package which is Architecture: all is
>> > considered an error. dpkg-deb must refuse to generate a .deb with this
>> > combination of values. Behavior when trying to install such a package is
>> > undefined."
>> >
>> > (https://wiki.ubuntu.com/MultiarchSpec#Binary%20package%20control%20fields)
>> >
>> > These two requirements are clearly in conflict. I've updated the wiki
>> > page to make it clear that only Multi-Arch: same is disallowed for
>> > Architecture: all packages. Please update the dpkg implementation to
>> > match when you have a chance.
>
>> I think it's wrong to (have to) add the Multi-Arch field to architecture
>> all packages. I would rather suggest that we consider them as
>> automatically satisfying any dependency (i.e. the Multi-Arch: foreign
>> would be implicit).
>
>> What do other people think?
>
>> An architecture all package that provides something arch specific is
>> very rare and it's often meant to be used in situation where you precisely
>> want to make this available to other architectures (i.e. syslinux-common).
>
>> So I don't think that we have to go through all the trouble to whitelist
>> them one by one.
>
> Sorry, I should have included in my mail the rationale for this requirement
> - which I remembered from the drafting sessions but didn't add when updating
> the wiki page.
>
> The reason we determined while drafting that Architecture: all packages
> should not automatically satisfy dependencies as if they were Multi-Arch:
> foreign is that in the current scheme, an Architecture: all package can very
> well be used as a metapackage to bridge a dependency between two
> Architecture: any packages that need to be of the same architecture.
>
> As a practical example: the python package is Architecture: all. Every
> python C extension package is Architecture: any and depends on python as an
> /interface/ to the python2.x Architecture: any packages which *must* be of
> the same architecture.
>
> I believe the consensus when drafting was that, if we didn't treat
> Architecture: all packages as being equivalent to the native arch for
> dependency solving, there would be cases when we would have insufficient
> information to properly verify dependencies due to the masking effect.
>
> With that rationale, does this requirement make more sense to you? Or do
> you see some other way to make this work? (Bearing in mind that both dpkg
> and higher-level package managers like apt need to be able to sort through
> the deps correctly)

It actually makes less sense.

You have identified the 2 cases of arch:all apckages:

1) truely architecture independent and satisfies dependencies for any
arch
2) meta packages that are placeholder for an arch:any package

So why then forbid the use of the Multi-Arch field to let maintainers
choose between the 2 types?


The other solution is to let arch:all packages fullfill depends for any
arch but consider them as if they were arch:any when computing the
closure of dependencies. That means things the arch:all package depends
on must be the same arch as the package depending on the arch:all
package or Multi-Arch: foreign.

Dependency chains could then look like this:
foo:i386 -> bla-plugins-meta:all -> bla-plugin:i386
foo:amd64 -> bla-plugins-meta:all -> bla-plugin:amd64

This is how I read the specs:

"Pre-multiarch, architecture-dependent packages may depend on
Architecture: all packages and assume that the transitive dependencies
will be resolved using packages of the same architecture or other
packages that are Architecture: all."

Under multiarch the "same architecture" would mean the same architecture
as the package depending on the arch:all package.

Now you seem to say that the second chain would be illegal. Which
matches the next sentence in the specs:

"To avoid breaking this assumption, Architecture: all packages will, at
least initially, be treated as equivalent to packages of the native
architecture for all dependency resolution."

But that seems unneccessary. You only need to use the same, not the
native architecture.



Note that both methods can be combined for the best effect. Default
would be to use the same architecture for transitive dependencies unless
Multi-Arch: foreign is specified.

MfG
Goswin


--
To UNSUBSCRIBE, email to debian-dpkg-REQUEST@lists.debian.org
with a subject of "unsubscribe". Trouble? Contact listmaster@lists.debian.org
Archive: 87vd0kc624.fsf@frosties.localnet">http://lists.debian.org/87vd0kc624.fsf@frosties.localnet

Multi-arch and dependencies on arch: all packages
 

On Wed, Feb 16, 2011 at 05:12:03PM +0100, Goswin von Brederlow wrote:
> It actually makes less sense.

> You have identified the 2 cases of arch:all apckages:

> 1) truely architecture independent and satisfies dependencies for any
> arch
> 2) meta packages that are placeholder for an arch:any package

> So why then forbid the use of the Multi-Arch field to let maintainers
> choose between the 2 types?

What? We didn't do that at all. This thread is about making it clear that
the Multi-Arch field *can* be set on Architecture: all packages! Case 1 is
Multi-Arch: foreign; case 2 is Multi-Arch: none.

> The other solution is to let arch:all packages fullfill depends for any
> arch but consider them as if they were arch:any when computing the
> closure of dependencies. That means things the arch:all package depends
> on must be the same arch as the package depending on the arch:all
> package or Multi-Arch: foreign.

> Dependency chains could then look like this:
> foo:i386 -> bla-plugins-meta:all -> bla-plugin:i386
> foo:amd64 -> bla-plugins-meta:all -> bla-plugin:amd64

Which requires the package manager to keep track of state about *transitive*
dependencies. Which is what I think we should avoid, especially for dpkg.

Raphaël seems to agree with me, and has implemented my request in his
branch.

> "Pre-multiarch, architecture-dependent packages may depend on
> Architecture: all packages and assume that the transitive dependencies
> will be resolved using packages of the same architecture or other
> packages that are Architecture: all."

> Under multiarch the "same architecture" would mean the same architecture
> as the package depending on the arch:all package.

> Now you seem to say that the second chain would be illegal. Which
> matches the next sentence in the specs:

> "To avoid breaking this assumption, Architecture: all packages will, at
> least initially, be treated as equivalent to packages of the native
> architecture for all dependency resolution."

> But that seems unneccessary. You only need to use the same, not the
> native architecture.

There is no way to know that it's the same architecture without having to
fully traverse the dependency tree. That's the complexity that should be
avoided in dpkg. By smashing these to :native instead of :same, we have
enough information to decide *locally* whether a given package's
dependencies are satisfied. Robust, efficient, and avoids getting hung up
on hypothetical corner cases.

--
Steve Langasek Give me a lever long enough and a Free OS
Debian Developer to set it on, and I can move the world.
Ubuntu Developer http://www.debian.org/
slangasek@ubuntu.com vorlon@debian.org

Multi-arch and dependencies on arch: all packages
 

Steve Langasek <vorlon@debian.org> writes:

> On Wed, Feb 16, 2011 at 05:12:03PM +0100, Goswin von Brederlow wrote:
>> It actually makes less sense.
>
>> You have identified the 2 cases of arch:all apckages:
>
>> 1) truely architecture independent and satisfies dependencies for any
>> arch
>> 2) meta packages that are placeholder for an arch:any package
>
>> So why then forbid the use of the Multi-Arch field to let maintainers
>> choose between the 2 types?
>
> What? We didn't do that at all. This thread is about making it clear that
> the Multi-Arch field *can* be set on Architecture: all packages! Case 1 is
> Multi-Arch: foreign; case 2 is Multi-Arch: none.

Yet Multi-Arch: same is forbidden in the specs. Which you would need for
a meta package that depends on a library.

>> The other solution is to let arch:all packages fullfill depends for any
>> arch but consider them as if they were arch:any when computing the
>> closure of dependencies. That means things the arch:all package depends
>> on must be the same arch as the package depending on the arch:all
>> package or Multi-Arch: foreign.
>
>> Dependency chains could then look like this:
>> foo:i386 -> bla-plugins-meta:all -> bla-plugin:i386
>> foo:amd64 -> bla-plugins-meta:all -> bla-plugin:amd64
>
> Which requires the package manager to keep track of state about *transitive*
> dependencies. Which is what I think we should avoid, especially for dpkg.
>
> Raphaël seems to agree with me, and has implemented my request in his
> branch.
>
>> "Pre-multiarch, architecture-dependent packages may depend on
>> Architecture: all packages and assume that the transitive dependencies
>> will be resolved using packages of the same architecture or other
>> packages that are Architecture: all."
>
>> Under multiarch the "same architecture" would mean the same architecture
>> as the package depending on the arch:all package.
>
>> Now you seem to say that the second chain would be illegal. Which
>> matches the next sentence in the specs:
>
>> "To avoid breaking this assumption, Architecture: all packages will, at
>> least initially, be treated as equivalent to packages of the native
>> architecture for all dependency resolution."
>
>> But that seems unneccessary. You only need to use the same, not the
>> native architecture.
>
> There is no way to know that it's the same architecture without having to
> fully traverse the dependency tree. That's the complexity that should be
> avoided in dpkg. By smashing these to :native instead of :same, we have
> enough information to decide *locally* whether a given package's
> dependencies are satisfied. Robust, efficient, and avoids getting hung up
> on hypothetical corner cases.

You don't have to traverse the whole tree, only until you hit a package
that isn't arch:all. Also you can work around that by simply duplicating
the package internally with different Architecture strings as needed.
I totaly agree that this is a step up in complexity. It would be nice to
have though.

By smashing arch:all packages to :native you greatly reduce the
installability of packages. Lets take a simple game (freeciv) as example
and pretend this were a non-free package available for i386 only:

Package: freeciv-client-gtk
Architecture: i386
Depends: libatk1.0-0 (>= 1.29.3), ..., freeciv-data (>= 2.2.0), ggzcore-bin

Package: freeciv-data
Architecture: all
Version: 2.2.4-1

That means freeciv will be uninstallable on amd64 for no good reason.


Could we default arch:all leaf packages (no further Depends) to
Multi-Arch: foreign? That would cover data packages like the above
example. That should be local enough to check.

MfG
Goswin


--
To UNSUBSCRIBE, email to debian-dpkg-REQUEST@lists.debian.org
with a subject of "unsubscribe". Trouble? Contact listmaster@lists.debian.org
Archive: 8762sf3iw2.fsf@frosties.localnet">http://lists.debian.org/8762sf3iw2.fsf@frosties.localnet

Multi-arch and dependencies on arch: all packages
 

On Sat, Feb 19, 2011 at 08:48:45PM +0100, Goswin von Brederlow wrote:
> > On Wed, Feb 16, 2011 at 05:12:03PM +0100, Goswin von Brederlow wrote:
> >> It actually makes less sense.

> >> You have identified the 2 cases of arch:all apckages:

> >> 1) truely architecture independent and satisfies dependencies for any
> >> arch
> >> 2) meta packages that are placeholder for an arch:any package

> >> So why then forbid the use of the Multi-Arch field to let maintainers
> >> choose between the 2 types?

> > What? We didn't do that at all. This thread is about making it clear that
> > the Multi-Arch field *can* be set on Architecture: all packages! Case 1 is
> > Multi-Arch: foreign; case 2 is Multi-Arch: none.

> Yet Multi-Arch: same is forbidden in the specs. Which you would need for
> a meta package that depends on a library.

Please cite an example where this is not an absolutely preposterous thing to
do.

Library dependencies are satisfied via the dpkg-shlibdeps mechanism, which
generates direct dependencies from the package containing the binary to the
package containing the library it uses. I've never heard of anyone
depending on an architecture: all package as a proxy for *libraries*, and
can't think of any reason that would be sane to do.

> By smashing arch:all packages to :native you greatly reduce the
> installability of packages. Lets take a simple game (freeciv) as example
> and pretend this were a non-free package available for i386 only:

> Package: freeciv-client-gtk
> Architecture: i386
> Depends: libatk1.0-0 (>= 1.29.3), ..., freeciv-data (>= 2.2.0), ggzcore-bin

> Package: freeciv-data
> Architecture: all
> Version: 2.2.4-1

> That means freeciv will be uninstallable on amd64 for no good reason.

Yep - but under the current proposal we could easily fix that by annotating
freeciv-data as Multi-Arch: foreign.

The idea of being able to install foreign-arch packages *at all* is entirely
new in Debian. I don't think requiring the arch: all package to be marked
Multi-Arch: foreign is all that much of a burden.

> Could we default arch:all leaf packages (no further Depends) to
> Multi-Arch: foreign? That would cover data packages like the above
> example. That should be local enough to check.

Since that doesn't require dpkg to look beyond one level of dependencies
when calculating, I would be ok with that as a middle ground, if the dpkg
and apt maintainers agree. Though I suspect the cases where this matters
will be few in practice.

--
Steve Langasek Give me a lever long enough and a Free OS
Debian Developer to set it on, and I can move the world.
Ubuntu Developer http://www.debian.org/
slangasek@ubuntu.com vorlon@debian.org