FAQ Search Today's Posts Mark Forums Read
» Video Reviews

» Linux Archive

Linux-archive is a website aiming to archive linux email lists and to make them easily accessible for linux users/developers.


» Sponsor

» Partners

» Sponsor

Go Back   Linux Archive > Debian > Debian Development

 
 
LinkBack Thread Tools
 
Old 02-13-2011, 06:49 PM
Henrique de Moraes Holschuh
 
Default Cross-check autobuilt binary pkg with maintainer-provided pkg

On Sun, 13 Feb 2011, brian m. carlson wrote:
> On Sun, Feb 13, 2011 at 06:00:11PM +0000, Lars Wirzenius wrote:
> > That's something I don't understand. If I upload a broken package, why
> > should it be the buildd admin's job to deal with it? Should not I get
> > notified of the error, and told to fix it?
>
> I've seen packages that don't fail until the very end of the build.
> This can waste a lot of buildd time, especially on very large packages.
> So yes, as the maintainer, it may be your problem to fix, but failing to
> test packages that as a result fail on the buildds won't make the buildd
> admins very happy.

This is a good reason to forbid source-only uploads, but there is another,
which actually leverages what we do now, and the idea of not using the
maintainer-provided binary packages in order to have better determinism on
the build:

1. Instead of discarding upon upload the maintainer-provided binary uploads,
we should first retain some interesting metadata[1] about it, and then
discard.

* Packages that outright FTBFS won't get uploaded

2. Autobuild all binary packages (determinism, clean chroot, etc)

3. Compare the metadata on the autobuilt binary packages and the binary
packages originally uploaded by the maintainer

4. Notify of any important discrepancies (through PTS, *automated*):

1. Changed dependencies
2. Changed file names (new/missing)
3. (large?) changes to file sizes
4. Changed linking information
...

This can help with the build-conflicts issues, plus raise an alarm for the
maintainer about his building environment _or_ about something weird going
on with the autobuilder.

We are not adding any extra pain for any maintainers, they upload exactly
what they are already uploading today. We are also not adding to the normal
operational workload of the autobuilder operators, as we are not increasing
the FTBFS probability, nor adding any non-automated steps.

[1] Where metadata means dumps of the information we might want in step (4),
but could also be the entire binary package. Whatever is more practical.

Obviously, it should be discarded after a small while (one week? one
month?).

--
"One disk to rule them all, One disk to find them. One disk to bring
them all and in the darkness grind them. In the Land of Redmond
where the shadows lie." -- The Silicon Valley Tarot
Henrique Holschuh


--
To UNSUBSCRIBE, email to debian-devel-REQUEST@lists.debian.org
with a subject of "unsubscribe". Trouble? Contact listmaster@lists.debian.org
Archive: 20110213194931.GB2437@khazad-dum.debian.net">http://lists.debian.org/20110213194931.GB2437@khazad-dum.debian.net
 
Old 02-13-2011, 07:47 PM
Joey Hess
 
Default Cross-check autobuilt binary pkg with maintainer-provided pkg

Henrique de Moraes Holschuh wrote:
> 1. Instead of discarding upon upload the maintainer-provided binary uploads,
> we should first retain some interesting metadata[1] about it, and then
> discard.
>
> * Packages that outright FTBFS won't get uploaded
>
> 2. Autobuild all binary packages (determinism, clean chroot, etc)
>
> 3. Compare the metadata on the autobuilt binary packages and the binary
> packages originally uploaded by the maintainer
>
> 4. Notify of any important discrepancies (through PTS, *automated*):

Completely agree.

File sizes won't 100% stable unfortunatly but the rest should be.

> [1] Where metadata means dumps of the information we might want in step (4),
> but could also be the entire binary package. Whatever is more practical.
>
> Obviously, it should be discarded after a small while (one week? one
> month?).

Putting it in the .changes file seems to make the most sense?

--
see shy jo
 
Old 02-14-2011, 12:08 AM
Raphael Geissert
 
Default Cross-check autobuilt binary pkg with maintainer-provided pkg

Henrique de Moraes Holschuh wrote:
> This is a good reason to forbid source-only uploads, but there is another,
> which actually leverages what we do now, and the idea of not using the
> maintainer-provided binary packages in order to have better determinism on
> the build:
[...]

Sounds like a great idea, yes.
If ftpmasters don't want dak to do all the work, a new service under the QA
umbrella could be setup. Sending the original metadata to the QA host should
be enough for it to check the package built by the buildds whenever it is
uploaded to the archive.

Cheers,
--
Raphael Geissert - Debian Developer
www.debian.org - get.debian.net


--
To UNSUBSCRIBE, email to debian-devel-REQUEST@lists.debian.org
with a subject of "unsubscribe". Trouble? Contact listmaster@lists.debian.org
Archive: ij9v9j$e2e$2@dough.gmane.org">http://lists.debian.org/ij9v9j$e2e$2@dough.gmane.org
 
Old 02-14-2011, 03:38 PM
Peter Samuelson
 
Default Cross-check autobuilt binary pkg with maintainer-provided pkg

[Joey Hess]
> File sizes won't 100% stable unfortunatly but the rest should be.

Well, depending on how often I upgrade, vs. how often the buildd does
(and for that matter, how long my package sits in a queue before it is
built), there can be dependency changes due to build-depends shlibs /
symbols updates. Unfortunately, it probably isn't possible to tell
whether a dependency change is harmless or something the maintainer
should be notified about and a change that.

So there'll be some noise in the 'debdiff'. To minimize the noise,
_always_ upgrade before building and uploading, and time your build to
be just after a mirror pulse.
--
Peter Samuelson | org-tld!p12n!peter | http://p12n.org/


--
To UNSUBSCRIBE, email to debian-devel-REQUEST@lists.debian.org
with a subject of "unsubscribe". Trouble? Contact listmaster@lists.debian.org
Archive: 20110214163827.GC10272@p12n.org">http://lists.debian.org/20110214163827.GC10272@p12n.org
 
Old 02-15-2011, 10:05 PM
Joerg Jaspert
 
Default Cross-check autobuilt binary pkg with maintainer-provided pkg

>> This is a good reason to forbid source-only uploads, but there is another,
>> which actually leverages what we do now, and the idea of not using the
>> maintainer-provided binary packages in order to have better determinism on
>> the build:
> [...]
> Sounds like a great idea, yes.
> If ftpmasters don't want dak to do all the work, a new service under the QA
> umbrella could be setup. Sending the original metadata to the QA host should
> be enough for it to check the package built by the buildds whenever it is
> uploaded to the archive.

dak can do a lot of work. For some of it we "just" need the code.
The stuff described here very much sounds like dak is the optimal
place. Its just that we all do have some real life too (something needs
to pay for food and stuff in this world, damn), so we dont have as much
time as we want. But most of the time we are happy to babysit someone
developing stuff to

--
bye, Joerg
Some NM:
>A developer contacts you and asks you to met for a keysign. What is
>your response and why?
Do you like beer? When do we meet? [...]


--
To UNSUBSCRIBE, email to debian-devel-REQUEST@lists.debian.org
with a subject of "unsubscribe". Trouble? Contact listmaster@lists.debian.org
Archive: 87sjvo2912.fsf@gkar.ganneff.de">http://lists.debian.org/87sjvo2912.fsf@gkar.ganneff.de
 

Thread Tools




All times are GMT. The time now is 09:38 PM.

VBulletin, Copyright ©2000 - 2014, Jelsoft Enterprises Ltd.
Content Relevant URLs by vBSEO ©2007, Crawlability, Inc.
Copyright 2007 - 2008, www.linux-archive.org