Linux Archive

Linux Archive (http://www.linux-archive.org/)
-   Debian dpkg (http://www.linux-archive.org/debian-dpkg/)
-   -   Autobuilding dpkg master (http://www.linux-archive.org/debian-dpkg/559302-autobuilding-dpkg-master.html)

Raphael Hertzog 08-01-2011 12:53 PM

Autobuilding dpkg master
 
[ A bit of context for debian-dpkg@lists.debian.org: I discussed with
Michael Prokop during debconf and discovered his usage of Jenkins to
autobuild the latest version of all his grml related packages every time
that he commits a change ]

Hi Michael,

I would like to have dpkg's master branch auto-built for i386/amd64
and accessible via an APT repository.

The git repository is here:
git clone git://git.debian.org/git/dpkg/dpkg.git

In order to build from git, you need what's in Build-Depends but also
automake (and possibly autopoint IIRC). You also have to add
"~YYYYMMDDHHMM" to the current version in debian/changelog.

If the build fails on any architecture, I would like to have a mail sent to
debian-dpkg@lists.debian.org (with the 200 last lines of the build log if
possible).

As a supplementary bonus, I have a test-build branch here:
git clone git://git.debian.org/~hertzog/dpkg.git -b test-build

(this branch does not generally fast-forward, it's often recreated as a
merge of several pu/* branches)

If it could also be auto-built and be put in a Non-Automatic: yes
APT repository it would be nice too.

Can you arrange all this? Can I do something to help you?

I guess I will have to put something in the post-receive hook, I'll wait
for your instructions for this.

Cheers,
--
Raphaël Hertzog ◈ Debian Developer

Follow my Debian News ▶ http://RaphaelHertzog.com (English)
▶ http://RaphaelHertzog.fr (Français)


--
To UNSUBSCRIBE, email to debian-dpkg-REQUEST@lists.debian.org
with a subject of "unsubscribe". Trouble? Contact listmaster@lists.debian.org
Archive: 20110801125303.GA25541@rivendell.home.ouaza.com">h ttp://lists.debian.org/20110801125303.GA25541@rivendell.home.ouaza.com

Raphael Hertzog 08-04-2011 03:00 PM

Autobuilding dpkg master
 
On Mon, 01 Aug 2011, Raphael Hertzog wrote:
> I would like to have dpkg's master branch auto-built for i386/amd64
> and accessible via an APT repository.

Michael already did it for amd64.

For master in the official repo: http://jenkins.grml.org/job/dpkg-source+amd64
The APT repo:
deb http://jenkins.grml.org/job/dpkg-source+amd64/lastSuccessfulBuild/artifact/ /

For my test-build branch in my private repo:
http://jenkins.grml.org/job/dpkg-source-test+amd64
The APT repo:
deb http://jenkins.grml.org/job/dpkg-source-test+amd64/lastSuccessfulBuild/artifact/ /

My test-build branch contains currently both multiarch and the hardening
build flags (on top of latest master).

(Note that the repositories are not signed.)

It's working really well and is already hooked with the repositories
to auto-trigger new builds.

Many thanks to Michael!

Cheers,
--
Raphaël Hertzog ◈ Debian Developer

Follow my Debian News ▶ http://RaphaelHertzog.com (English)
▶ http://RaphaelHertzog.fr (Français)


--
To UNSUBSCRIBE, email to debian-dpkg-REQUEST@lists.debian.org
with a subject of "unsubscribe". Trouble? Contact listmaster@lists.debian.org
Archive: 20110804150033.GA25030@rivendell.home.ouaza.com">h ttp://lists.debian.org/20110804150033.GA25030@rivendell.home.ouaza.com

Michael Prokop 08-04-2011 10:19 PM

Autobuilding dpkg master
 
* Raphael Hertzog [Thu Aug 04, 2011 at 05:00:33PM +0200]:
> On Mon, 01 Aug 2011, Raphael Hertzog wrote:

> > I would like to have dpkg's master branch auto-built for i386/amd64
> > and accessible via an APT repository.

> Michael already did it for amd64.

> For master in the official repo: http://jenkins.grml.org/job/dpkg-source+amd64
> The APT repo:
> deb http://jenkins.grml.org/job/dpkg-source+amd64/lastSuccessfulBuild/artifact/ /

> For my test-build branch in my private repo:
> http://jenkins.grml.org/job/dpkg-source-test+amd64
> The APT repo:
> deb http://jenkins.grml.org/job/dpkg-source-test+amd64/lastSuccessfulBuild/artifact/ /

> My test-build branch contains currently both multiarch and the hardening
> build flags (on top of latest master).

> (Note that the repositories are not signed.)

> It's working really well and is already hooked with the repositories
> to auto-trigger new builds.

I just set up the according i386 builds and also set up signed
repositories. You can find all the dpkg-source* jobs listed at
http://jenkins.grml.org/view/Debian/

APT instructions for master branch of official repo:

deb http://jenkins.grml.org/debian dpkg-source main
deb-src http://jenkins.grml.org/debian dpkg-source main
wget -O - http://jenkins.grml.org/debian/C525F56752D4A654.asc | sudo apt-key add -

APT instructions for test-build branch of private repo:

deb http://jenkins.grml.org/debian dpkg-source-test main
deb-src http://jenkins.grml.org/debian dpkg-source-test main
wget -O - http://jenkins.grml.org/debian/C525F56752D4A654.asc | sudo apt-key add -


Note: the *sources* included in the apt repository are from the
amd64 build, the i386 packages are binary-only builds.

Raphael: No changes needed for auto-triggerering new builds from
your side. It's set up so as soon as the amd64 build succeeds it
automatically triggers the according i386 build.

Please let me know if you notice any problems or need any changes.

regards,
-mika-

Raphael Hertzog 08-04-2011 11:03 PM

Autobuilding dpkg master
 
On Fri, 05 Aug 2011, Michael Prokop wrote:
> Note: the *sources* included in the apt repository are from the
> amd64 build, the i386 packages are binary-only builds.

Hum, they are binary-only builds but from a different source package
apparently. Because I have different versions for arch: all and arch:
i386.

This is usually not a problem because we don't have strong dependencies
from arch: any to arch: all but it still seems wrong to me.

$ dpkg -l dpkg dpkg-dev
[...]
ii dpkg 1.16.1~1.gbpcca90 Debian package management system
ii dpkg-dev 1.16.1~10.gbpcca9 Debian package development tools

Cheers,
--
Raphaël Hertzog ◈ Debian Developer

Follow my Debian News ▶ http://RaphaelHertzog.com (English)
▶ http://RaphaelHertzog.fr (Français)


--
To UNSUBSCRIBE, email to debian-dpkg-REQUEST@lists.debian.org
with a subject of "unsubscribe". Trouble? Contact listmaster@lists.debian.org
Archive: 20110804230338.GA27539@rivendell.home.ouaza.com">h ttp://lists.debian.org/20110804230338.GA27539@rivendell.home.ouaza.com

Michael Prokop 08-04-2011 11:32 PM

Autobuilding dpkg master
 
* Raphael Hertzog [Fri Aug 05, 2011 at 01:03:38AM +0200]:
> On Fri, 05 Aug 2011, Michael Prokop wrote:
> > Note: the *sources* included in the apt repository are from the
> > amd64 build, the i386 packages are binary-only builds.

> Hum, they are binary-only builds but from a different source package
> apparently. Because I have different versions for arch: all and arch:
> i386.

> This is usually not a problem because we don't have strong dependencies
> from arch: any to arch: all but it still seems wrong to me.

> $ dpkg -l dpkg dpkg-dev
> [...]
> ii dpkg 1.16.1~1.gbpcca90 Debian package management system
> ii dpkg-dev 1.16.1~10.gbpcca9 Debian package development tools

The packages are based on the same git commit. Compare the *.changes
file from http://jenkins.grml.org/job/dpkg-source+i386/ with the one
from http://jenkins.grml.org/job/dpkg-source+amd64/

Also the version number behind the "1.16.1~1[0]" indicates that this
should be fine itself.

The confusing ~1 vs. ~10 part in the above dpkg listing results from
the build number which is different for the i386 vs amd64 jobs
inside Jenkins.

If that's an issue for you we could use a different naming schema.
In another project I'm e.g. using UNIXTIME=$(date +%s) inside the
autogenerated snapshot versions. Though I'm open for any suggestions. :)

regards,
-mika-

Raphael Hertzog 08-05-2011 06:01 AM

Autobuilding dpkg master
 
On Fri, 05 Aug 2011, Michael Prokop wrote:
> The confusing ~1 vs. ~10 part in the above dpkg listing results from
> the build number which is different for the i386 vs amd64 jobs
> inside Jenkins.

I underestood that but is it really required to update this build number
given that the i386 build is triggered by the amd64 build? It will be
automatically unique (and even if it isn't, is that a problem?).

> If that's an issue for you we could use a different naming schema.
> In another project I'm e.g. using UNIXTIME=$(date +%s) inside the
> autogenerated snapshot versions. Though I'm open for any suggestions. :)

This won't really help since both jobs are not happening at the same
second.

Cheers,
--
Raphaël Hertzog ◈ Debian Developer

Follow my Debian News ▶ http://RaphaelHertzog.com (English)
▶ http://RaphaelHertzog.fr (Français)


--
To UNSUBSCRIBE, email to debian-dpkg-REQUEST@lists.debian.org
with a subject of "unsubscribe". Trouble? Contact listmaster@lists.debian.org
Archive: 20110805060124.GC3120@rivendell.home.ouaza.com">ht tp://lists.debian.org/20110805060124.GC3120@rivendell.home.ouaza.com

Michael Prokop 08-06-2011 09:05 AM

Autobuilding dpkg master
 
* Raphael Hertzog [Fri Aug 05, 2011 at 08:01:24AM +0200]:
> On Fri, 05 Aug 2011, Michael Prokop wrote:

> > The confusing ~1 vs. ~10 part in the above dpkg listing results from
> > the build number which is different for the i386 vs amd64 jobs
> > inside Jenkins.

> I underestood that but is it really required to update this build number
> given that the i386 build is triggered by the amd64 build? It will be
> automatically unique (and even if it isn't, is that a problem?).

> > If that's an issue for you we could use a different naming schema.
> > In another project I'm e.g. using UNIXTIME=$(date +%s) inside the
> > autogenerated snapshot versions. Though I'm open for any suggestions. :)

> This won't really help since both jobs are not happening at the same
> second.

Thanks for your input. I've reworked the setup (sorry for the delay
but it turned out to be much trickier than expected).

Now the setup is as follows:

A job foobar-source builds the source package, then triggers a so
called matrix build named foobar-binaries which takes the resulting
files from the foobar-source job and builds the binary packages for
all defined architectures (being i386 and amd64 currently).

So having the dpkg (master branch) and the dpkg-test (test branch)
repositories we now get just 4 jobs, being:

* dpkg-binaries
* dpkg-source
* dpkg-test-binaries
* dpkg-test-source

-> http://jenkins.grml.org/view/Debian/

Raphael, please update your git hook(s) to match the new job names.
dpkg-source+amd64 became dpkg-source and dpkg-source-test+amd64
became dpkg-test-source.

To streamline the configuration I also updated the repository
definition, so now it's:

APT instructions (master branch):
deb http://jenkins.grml.org/debian dpkg main
deb-src http://jenkins.grml.org/debian dpkg main
wget -O - http://jenkins.grml.org/debian/C525F56752D4A654.asc | sudo apt-key add -

APT instructions (test branch of private repos):
deb http://jenkins.grml.org/debian dpkg-test main
deb-src http://jenkins.grml.org/debian dpkg-test main
wget -O - http://jenkins.grml.org/debian/C525F56752D4A654.asc | sudo apt-key add -

I hope this works as expected, please let me know if you need any
changes.

regards,
-mika-

Raphael Hertzog 08-06-2011 05:23 PM

Autobuilding dpkg master
 
Hi,

thank you for everything!

On Sat, 06 Aug 2011, Michael Prokop wrote:
> Raphael, please update your git hook(s) to match the new job names.
> dpkg-source+amd64 became dpkg-source and dpkg-source-test+amd64
> became dpkg-test-source.

Done.

> I hope this works as expected, please let me know if you need any
> changes.

The only problem left from my point of view is that the i386 Packages
file doesn't contain the Architecture: all .deb.

See http://jenkins.grml.org/debian/dists/dpkg-test/main/binary-i386/Packages
vs http://jenkins.grml.org/debian/dists/dpkg-test/main/binary-amd64/Packages

Cheers,
--
Raphaël Hertzog ◈ Debian Developer

Follow my Debian News ▶ http://RaphaelHertzog.com (English)
▶ http://RaphaelHertzog.fr (Français)


--
To UNSUBSCRIBE, email to debian-dpkg-REQUEST@lists.debian.org
with a subject of "unsubscribe". Trouble? Contact listmaster@lists.debian.org
Archive: 20110806172327.GE16024@rivendell.home.ouaza.com">h ttp://lists.debian.org/20110806172327.GE16024@rivendell.home.ouaza.com

Michael Prokop 08-08-2011 10:24 AM

Autobuilding dpkg master
 
* Raphael Hertzog [Sat Aug 06, 2011 at 07:23:27PM +0200]:

> thank you for everything!

You're welcome :)

> On Sat, 06 Aug 2011, Michael Prokop wrote:

> > I hope this works as expected, please let me know if you need any
> > changes.

> The only problem left from my point of view is that the i386 Packages
> file doesn't contain the Architecture: all .deb.

> See http://jenkins.grml.org/debian/dists/dpkg-test/main/binary-i386/Packages
> vs http://jenkins.grml.org/debian/dists/dpkg-test/main/binary-amd64/Packages

Thanks for catching this one, should be fixed by now.

regards,
-mika-


All times are GMT. The time now is 05:54 AM.

VBulletin, Copyright ©2000 - 2014, Jelsoft Enterprises Ltd.
Content Relevant URLs by vBSEO ©2007, Crawlability, Inc.