FAQ Search Today's Posts Mark Forums Read
» Video Reviews

» Linux Archive

Linux-archive is a website aiming to archive linux email lists and to make them easily accessible for linux users/developers.


» Sponsor

» Partners

» Sponsor

Go Back   Linux Archive > CentOS > CentOS Development

 
 
LinkBack Thread Tools
 
Old 09-28-2012, 05:43 AM
Lamar Owen
 
Default Interested in IA64 build.

On Monday, September 24, 2012 08:36:38 AM Lamar Owen wrote:
...
> In any case, the local rebuild of CentOS 5.5 using the SLC5.4 binaries as 'seed' started Saturday evening; as of 8:20A today it has successfully rebuilt 443 source RPMS, producing 1243 binary RPMS, and have seen 35 packages fail thus far.
...

And an update. Following an iterative process of building the failed packages as a set over and over against the 5.4 buildroot seed binaries, I was able to get it down to 118 source packages that failed to build, and 3,245 binaries successfully built. Two of those packages blocked me from uprevving my buildroot to 5.5: chkconfig, and lvm2. Since neither of those packages actually does any building, I seeded a 'reqs' repo with those two, plus the 32-bit valgraind-devel package, along with m4 1.4.8, and uprevved the buildroot (this is done by pointing the repos in the mock config to the newer binaries; mock configs are essentially modified yum configs, and work pretty much the same way, with quirks). This brought the number of successfully rebuilt packages up to 3,389 binaries, with 81 failed builds. Fortunately, none of the failed packages would prevent a yum update-style cross-grade from SLC 5.4 to CentOS 5.5. (the really key package that is still failing is anaconda; bu
t that's not a big problem right now since I'm not going to try to spin media until I get up to 5.8, I just need to be careful about backups and archives of the built repos!).

If you want the gory details, I'll be glad to share, but the short and sweet of it is that I now have one Altix box running my own local rebuild of CentOS 5.5 on ia64.

I'm now dong a 'self-hosting' build of 5.5 using the previously built 5.5 binaries as the buildroot, and once that's done I'll update/cross-grade the buildhost to the self-hosted 5.5 and start on uprevving to 5.6. The 5.5-> 5.6 upgrade wasn't the easiest one we've had, and I'm kindof dreading it in a way, but the process thus far has been very educational and enlightening when it comes to what kind of mechanics really go in to doing something like a rebuild of the upstream sources.

And many thanks to Karanbir and Johnny for the encouragement and assistance with the thought processes going into this. While I'm not currently using the same mock version as CentOS is, nor am I using the same process and hooks, the insights they've provided have been key to getting this far. And, of course, Russ Herrold's archived notes and correspondence were instrumental in me thinking I could even do such as thing as this in the first place.... thanks guys!

_______________________________________________
CentOS-devel mailing list
CentOS-devel@centos.org
http://lists.centos.org/mailman/listinfo/centos-devel
 
Old 10-04-2012, 03:45 PM
Lamar Owen
 
Default Interested in IA64 build.

On Friday, September 28, 2012 01:43:42 AM Lamar Owen wrote:
> ...
> > If you want the gory details, I'll be glad to share, but the short and sweet of it is that I now have one Altix box running my own local [partial] rebuild of CentOS 5.5 on ia64.
>
> I'm now dong a 'self-hosting' build of 5.5...

Ok, well, I've learned a lot since last Friday. Especially as it pertains to circular dependencies and how to properly (hopefully) step from one release up to another. While I still have 14 ia64-supported packages that aren't successfully rebuilding at the 5.5 level, I have enough of a fully built 5.5 tree, not fully self-hosted, but built, to try a run at building the 5.6 updates while I'm doing other things. Since I started that build, 18 builds have succeeded, producing 48 binary packages, and no builds have failed yet (I expect to see some failures, since I didn't prune the updates list of non-ia64-buildable packages). There aren't but 218 source packages to rebuild, so it shouldn't take more than overnight, I would think.

And I say 'ia64 supported' above for a reason. There are a number of packages that either don't have ia64 included or have ia64 specifically excluded, 47 of them in the 5.5 sources to be exact.

And I edited the above quote to better reflect the 'testing' 5.5 partial build to which I upgraded one box earlier.

My game plan is to use the nearly complete 5.5 tree to build the 5.6 updates, then use the 5.5 tree plus the 5.6 updates to build 5.7, and so on. Once a nearly complete 5.8 tree is built, I'll retry a self-hosting build of the complete 5.8 source package set using the 5.5+5.6updates+5.7updates+5.8updates trees to seed the buildroots, and then will build and mung things until a fully-self-hosted build is done. Mung is a highly descriptive piece of jargon for this process, being that 'mung' itself is a recursive acronym.... ('Mung until no good' as defined in the New Hacker Dictionary, aka the Jargon File).

And while it might seem like I could sidestep up to 5.8, I want to iterate through the whole process, using the full tree, for my own reasons.

This is certainly an iterative, and, at times, a hair-pulling process due to hidden and/or undocumented build requirements (upstream hidden and upstream undocumented; I've had to inject, using the chroot_setup_cmd macro in the mock config, a handful of packages that aren't listed as buildrequires: for packages that are being built). And that is somewhat frustrating!

And, again, the encouragement of and hints from Karanbir, Johnny, and Russ have been highly valuable.

But I want to note that I have done what I have done thus far using only publicly available information about the CentOS build process and my own knowledge of RPM building from my five years of maintaining the packages for PostgreSQL; it takes thought and iterative legwork to make this stuff rebuild. And I'm not using the centos-used version of mock, nor am I using any 'secret' configurations from the centos developers; just pointers to the right directions in general, which have been most helpful. And reading build logs and trying to ferret out why that package didn't build (was it the gcc version, or is there a missing dep, or do I need to turn off parallel builds, or....?).

_______________________________________________
CentOS-devel mailing list
CentOS-devel@centos.org
http://lists.centos.org/mailman/listinfo/centos-devel
 
Old 10-04-2012, 03:57 PM
R P Herrold
 
Default Interested in IA64 build.

On Thu, 4 Oct 2012, Lamar Owen wrote:

> But I want to note that I have done what I have done thus
> far using only publicly available information about the
> CentOS build process and my own knowledge of RPM building
> from my five years of maintaining the packages for
> PostgreSQL; it takes thought and iterative legwork to make
> this stuff rebuild. And I'm not using the centos-used
> version of mock, nor am I using any 'secret' configurations
> from the centos developers; just pointers to the right
> directions in general, which have been most helpful. And
> reading build logs and trying to ferret out why that package
> didn't build (was it the gcc version, or is there a missing
> dep, or do I need to turn off parallel builds, or....?).

Go, Lamar, go

If you need 'blank' artwork template files to amend the
branding, please let me know out of band

-- Russ herrold
_______________________________________________
CentOS-devel mailing list
CentOS-devel@centos.org
http://lists.centos.org/mailman/listinfo/centos-devel
 
Old 10-04-2012, 04:10 PM
Karanbir Singh
 
Default Interested in IA64 build.

On 10/04/2012 04:57 PM, R P Herrold wrote:
> Go, Lamar, go
>
> If you need 'blank' artwork template files to amend the
> branding, please let me know out of band
>
>

why would we not use the centos artwork here ?


--
Karanbir Singh
+44-207-0999389 | http://www.karan.org/ | twitter.com/kbsingh
ICQ: 2522219 | Yahoo IM: z00dax | Gtalk: z00dax
GnuPG Key : http://www.karan.org/publickey.asc
_______________________________________________
CentOS-devel mailing list
CentOS-devel@centos.org
http://lists.centos.org/mailman/listinfo/centos-devel
 
Old 10-04-2012, 04:11 PM
Florian La Roche
 
Default Interested in IA64 build.

> And while it might seem like I could sidestep up to 5.8, I want to iterate through the whole process, using the full tree, for my own reasons.


Hello Lamar Owen,

I think there is value in a self-hosted (can fully recompile itself) build of CentOS.

I sometimes use a recompile with koji as some test-load for kvm guest systems. A sample
recompile can be seen on http://jur-linux.org/koji/

Some rpms failing a recompile (not all will be relevant to IA64) are:
gpm, geronimo-specs, rhythmbox, evolution-connector, gstreamer-plugins-base,
gaim, totem

best regards,

Florian La Roche

_______________________________________________
CentOS-devel mailing list
CentOS-devel@centos.org
http://lists.centos.org/mailman/listinfo/centos-devel
 
Old 10-04-2012, 04:18 PM
Lamar Owen
 
Default Interested in IA64 build.

On Thursday, October 04, 2012 11:57:35 AM R P Herrold wrote:
> Go, Lamar, go

Oh, pshaw....

As of lunchtime, here's the status (there are 218 source packages in the 5.5->5.6 update):

[lowen@winterstar ~]$ buildstatus56
No IA64 target: 3
No Package Found: 0
Scratch total: 4
Built binaries: 118
Built SRPMS: 43
[lowen@winterstar ~]$

(I was expecting those 'No IA64 Target' errors). 20% through, going by just the number of source packages and not by time-to-compile. The kernel, glibc, and gcc have yet to build, and they take a while; as does the current package being built, tomcat5.....

The buildstatus56 script is very rudimentary, and rough around the edges...it won't tell me that builds are done, for instance, but I tail the nohup.out from the build for that. Anyway, the script:

[lowen@winterstar ~]$ cat /usr/local/bin/buildstatus56
#!/bin/sh
SMOCK=/opt/build/public_html/smock/yum
pushd $SMOCK >/dev/null
echo -n -e "No IA64 target: "
egrep -r -l "Architecture.is.(not.in|ex)cluded" scratch |cut -d "/" -f 2|wc -l
echo -n -e "No Package Found: "
egrep -r -l "No.Package.found" scratch |cut -d "/" -f 2|wc -l
echo -e -n "Scratch total: "
ls scratch|wc -l
echo -n -e "Built binaries: "
ls centos-56/ia64/RPMS/*.rpm|wc -l
echo -n -e "Built SRPMS: "
ls centos-56/src/SRPMS/*.rpm|wc -l
popd >/dev/null
[lowen@winterstar ~]$

No magic there, just basic sysadmin know-how from experience with Unix systems since 1987.... :-P

The key piece of my build setup is smock.pl. I'll be posting some time this afternoon more details on my setup, inlcuding a rough outline of the process I have thus far used (once I am satisfied that my process looks like it might actually work, that is, since I haven't gotten up to 5.6 yet.....).

And it *is* a manual process; there are steps that I have used that would be difficult to automate in any reasonable manner.

But that's for another e-mail.....
_______________________________________________
CentOS-devel mailing list
CentOS-devel@centos.org
http://lists.centos.org/mailman/listinfo/centos-devel
 
Old 10-04-2012, 04:41 PM
Karanbir Singh
 
Default Interested in IA64 build.

On 10/04/2012 05:18 PM, Lamar Owen wrote:
> No magic there, just basic sysadmin know-how from experience with Unix systems since 1987.... :-P
>
> The key piece of my build setup is smock.pl. I'll be posting some time this afternoon more details on my setup, inlcuding a rough outline of the process I have thus far used (once I am satisfied that my process looks like it might actually work, that is, since I haven't gotten up to 5.6 yet.....).
>
> And it *is* a manual process; there are steps that I have used that would be difficult to automate in any reasonable manner.
>

I think its worth doing the 'find srpms that exclude ia64' and work out
their dep chain - I suspect you have quite a few more RPMS than are
needed to be upstream compatible.

Also worth keeping in mind is that while its a good idea to build the
BuildArch: noarch ones' you should not block on those failing - I
remember a bunch were not happy on ia64's arch type.

--
Karanbir Singh
+44-207-0999389 | http://www.karan.org/ | twitter.com/kbsingh
ICQ: 2522219 | Yahoo IM: z00dax | Gtalk: z00dax
GnuPG Key : http://www.karan.org/publickey.asc
_______________________________________________
CentOS-devel mailing list
CentOS-devel@centos.org
http://lists.centos.org/mailman/listinfo/centos-devel
 
Old 10-04-2012, 05:30 PM
Lamar Owen
 
Default Interested in IA64 build.

On Thursday, October 04, 2012 12:41:42 PM Karanbir Singh wrote:
> On 10/04/2012 05:18 PM, Lamar Owen wrote:
> > And it *is* a manual process; there are steps that I have used that would be difficult to automate in any reasonable manner.

> I think its worth doing the 'find srpms that exclude ia64' and work out
> their dep chain -

Yes, that is a worthwhile thing to do. But, given the use of conditional buildrequires: in some packages (gdb, for instance), this would require more than just getting the buildrequires from the SRPM's headers, I would think. Possibly even going as far as using rpmbuild to interpret the specs. In the gdb case, the SRPM's header documents buildrequires on:

ncurses-devel
texinfo
gettext
flex
bison
expat-devel
readline-devel
zlib-devel
python-devel
libstdc++

But on ia64 there is a build requirement for libunwind, and the version of libunwind required is different between el5 and non-el5. The spec file segment (for those who might wonder how that sort of thing is done):

...
%ifarch ia64
%if 0%{!?el5:1}
BuildRequires: libunwind-devel >= 0.99-0.1.frysk20070405cvs
Requires: libunwind >= 0.99-0.1.frysk20070405cvs
%else
BuildRequires: libunwind >= 0.96-3
Requires: libunwind >= 0.96-3
%endif
%endif
...

(incidentally, this one tripped me up for a while. I had set %dist to my own string, but the %el5 macro won't be set if %dist is not .el5, thus I was getting a buildrequire for the cvs version of libunwind-0.99. And while I did find a src.rpm (in Karanbir's KBS Extras Testing repo) for that version, well, suffice to say that trying to build the 0.99 libunwind on an SGI Altix *softlocks all CPUs, hanging the machine* during the test phase. Needless to say, the 0.98 version, which is in upstreams sources now, is the version that will build. But it took a little longer than I would have liked to track down the fact that setting %dist to something custom would break the build of gdb. Oh, and this would not have shown up in any of the builds for i386 or x86_64, yet another place the ia64 build is just plain different (and follows a different build chain) from those.)

> I suspect you have quite a few more RPMS than are
> needed to be upstream compatible.

Yep, that's very possible. And I know of at least one that I haven't built yet that's not in the i386 or x86_64 centos5 releases, and that's the elilo bootloader.

> Also worth keeping in mind is that while its a good idea to build the
> BuildArch: noarch ones' you should not block on those failing - I
> remember a bunch were not happy on ia64's arch type.

Interesting; I hadn't run into that (I don't think, at least).

I'm using smock.pl's '--keepgoing' directive to tell smock to, well, keep going so no failures block the build.
_______________________________________________
CentOS-devel mailing list
CentOS-devel@centos.org
http://lists.centos.org/mailman/listinfo/centos-devel
 
Old 10-04-2012, 10:14 PM
Lamar Owen
 
Default Interested in IA64 build.

On Thursday, October 04, 2012 11:45:14 AM Lamar Owen wrote:
> Ok, well, I've learned a lot since last Friday. Especially as it pertains to circular dependencies and how to properly (hopefully) step from one release up to another. ...

And after I wrote that, I realized that I hadn't documented how I did that. So, in this admittedly longish e-mail, I'll attempt to do that.

Also, I failed to note that the precise build chain that I ended up following is likely not the same as the chain used building the supported CentOS builds, since I started with a different set of packages and did not build intervening updates. And my chain was compiled semi-automatically using iterations of invocations of smock.pl with possibly a different mix of repos each time, and thus the build I have is likely not completely reproducible from the c5-wip tree Karanbir posted a while back.

And, well, honestly, I'm not as concerned with reproducibility as I am with getting a self-hosting 5.8 build as an end result, from which I will then build the 5.8 updates and/or the 5.9 tree once it is released; if 5.9 comes out before I get to the 5.8 updates then 5.9 comes afterwards, and I go on to 5.9 updates. The intermediate steps will be different each time through, depending upon the base from where you start, and the order in which you hand-resolve each and every one of the several hidden dependencies; knowing that going into this process I specifically did not document every step in exact detail, as it's not important, at least in my opinion. I'll give enough info below for anyone who can figure out how to get mock working and building things, along with the use of createrepe, to follow.

Here's how I'm stepping from CentOS 5.5 to 5.6. To duplicate you'll need a working binary repo, one that is complete enough to satisfy the dependencies of buildsys-build (available on dev.centos.org, and its spec is available in the mock package itself). If you are building a complete tree using smock.pl, smock.pl will figure out the buildrequires that are documented in the SRPM header and will sequence the builds accordingly. It's the undocumented, hidden, or conditional buildrequires that will bite you, and you need binaries for those packages in your repos for the buildroots. I say buildroots (plural), and not buildroot (singular), since each mock invocation that smock.pl makes will start with the base buildroot and install the various buildrequires needed into it for the specific package that particular mock invocation is building.

First, get and install mock 1.0.28 plus all dependencies from EPEL. Also get some example source RPM's and make sure rpmbuild as a normal user works. If you want an experience closer to what the CentOS build is like, then forgo mock 1.0.28 from EPEL and use the patchedf mock-0.6.13 from CentOS-Extras; my configuration won't work for you, but you'll have a closer-to-CentOS-build experience, if that's what you want. I'm more concerned about getting up to date first, and 'correct' second. So I may downgrade mock to 0.6.13 and use the CentOS-provided configs (and other info that's been posted to the mailing list over the years) to do the final 'self-hosting' build at 5.8 (or 5.9). But anyway, back to the documenting....

Add yourself as a member of the 'mock' group.

For performance, mount /var/lib/mock as a tmpfs of several GB, if you have the RAM to spare (my ia64 builder, winterstar, has 54GB, so I gave it a 26GB tmpfs on /var/lib/mock). My experience was that this trimmed a pretty substantial amount of time off of some of the lengthier builds, like the kernel.

Now, obtain smock.pl. ( part of fedora-mingw-devel)
See: http://git.annexia.org/?p=fedora-mingw.git;a=tree;f=smock

Read the README, and read the script to see what it's going to do. You will need to create initial repos in the target directory, and you'll need to do the proper configuration for smock.pl to 'find itself'. I cheated and symlinked ~/public_html to my build tree in /opt/build/. Once you understand what smock.pl will, and won't, do, put it somewhere in your path (like /usr/local/bin ) and make it executable.

And, yeah, you need to understand what's going on; this is not a recipe with quantities of ingredients, exact cooking temperatures, and exact cooking times here. I can, for instance, tell you how to cook sausage gravy; but I can't give you a recipe, since the amount of grease rendered from the sausage will vary, and thus I need to say things like 'put in enough flour to barely absorb all the grease and to make a moist, not dry or wet, but moist, putty (not paste and not dough)' and ' pour in enough milk to quickly boil and 'batterize' the flour-grease mixture' or 'pour in enough milk to make the gravy look just almost too thin' : you need to understand what you're doing to follow these directions. The reason is simple: you're not going to be building things in exactly the same order as upstream, CentOS, or I did, and thus you're probably going to get stopped in your build attempt in different places than we did. I know I experienced at least one issue in my ia64 build than
wasn't experienced in the i386 or x86_64 builds.

Now, the mock configuration file is the piece that drives everything. Here's the one that I'm using right now to churn the 5.6 updates for ia64. Watch for line wrap. I'm going to notate each section; my notations will be prefixed with a ----> :

++++++++++++++++++++++++
[lowen@winterstar ~]$ cat /etc/mock/centos-56-ia64.cfg
#!/usr/bin/python -tt

----> The above line marks this configuration file as being a python script,
----> with all the syntax rules etc of a python script.

import os
config_opts['root'] = 'centos-56-ia64'
config_opts['target_arch'] = 'ia64'
config_opts['clean'] = True

----> These configuration options tell mock the buildroot chroot to use,
----> the architecture for which to build, and whether to clean the chroot

config_opts['chroot_setup_cmd'] = 'install buildsys-build zip zlib-devel autoconf pkgconfig libsmbclient-devel gstreamer-plugins-good-devel dhclient'

----> This previous line, which is one line, is where hidden buildrequires:
----> injection occurs in my buildsys. CentOS does things differently; see
----> the top three entries in the changelog for the mock in CentOS-Extras
----> for the patch CentOS uses to mock 0.6.13. I'm looking at eventually
----> updating the HintsAndMacros patch for mock 1.0.28, but that's after
----> I get things built.

config_opts['plugin_conf']['ccache_enable'] = False

----> This above line is critical to make the system work properly.

----> Note the triple double quotes below. This is a multiline string, being
----> assigned to config_opts['yum.conf'] and anything inside the paired
----> triple double quotes is a yum configuration file and not a python script.

config_opts['yum.conf'] = """
[main]
cachedir=/var/cache/yum
debuglevel=1
logfile=/var/log/yum.log
reposdir=/dev/null
retries=20
obsoletes=1
gpgcheck=0
assumeyes=1

# repos

----> The first repo stanza below was the bootstrapping SLC 5.4 binaries, which
----> I don't need anymore for building the 5.6 updates.

#[slc-64bit-local-repo]
#name=slc-64-local.repo
#baseurl=file:///opt/build/repos/slc54
#enabled=1
#gpgcheck=0
#

----> Comment below says it all. Well, almost all. It's as built under
----> SLC 54 plus an accumulation of 5.5 packages over at least four
----> full iterations.

# CentOS 5.5 as built under SLC 5.4
[c5-os]
name=c5-local.repo
baseurl=file:///opt/build/repos/centos/5.5/os/ia64/
enabled=1
gpgcheck=0

----> This section below was only for testing, but I left it in place as
----> reminder to myself....

#[c5-32-bit]
#name=c5-32-bit.repo
#baseurl=file:///opt/build/repos/centos/5.5/os/i386/
#enabled=1
#gpgcheck=0

#newer m4 for buildsys

----> This is documented by SL CERN on their SLC5 build page.

[m4-c5]
name=m4-c5.repo
baseurl=file:///opt/build/buildsys/m4-c5
enabled=1
gpgcheck=0

----> The repo below is the most dynamic piece of the whole
----> puzzle, and has changed with the multiple iterations.
----> This holds the built-from-source libunwind, among others.
----> As the contents of this repo changed over time, and with
----> each iteration, the exact contents isn't important, but I
----> will say that valgrind-devel i386 lives here, even for an ia64
----> build. I can try to reconstruct the order of packages in and
----> out of this repo, but, really, I just put the packages in here
----> that the error logs told me the build needed. Once the
----> 5.5 package of the same name built, out of this repo it
----> went.... and, of course, I ran createrepo on it for every
----> modification.....

# other things the buildsys needs, pulled from downrev
[buildroot-reqs]
name=buildroot-reqs.repo
baseurl=file:///opt/build/buildsys/reqs
enabled=1
gpgcheck=0

----> The repo below is the buildsys-build rpm, pulled from the centos
----> dev server, and basically verbatim from the mock rpm (do an
----> rpm -ql mock to find the spec file.....)

[groups]
name=groups
#baseurl=http://dev.centos.org/centos/buildsys/5/
baseurl=file:///opt/build/Factory-Install/groups/5/

# Change USERNAME and if necessary the distribution name and
# architecture, and INSERT this file to the relevant distributions
# in /etc/mock (eg. to /etc/mock/fedora-9-i386.cfg etc.)

----> The below repo is the dynamic repo that smock maintains.
----> in my instructions below I will try to consistently call it the [smock]
----> repo.

[smock]
name=smock
#baseurl=http://127.0.0.1/USERNAME-smock/yum/fedora-9/$basearch
# If you do not want to use a http server, you need to specify a direct path to the
# repository. Then comment out the other baseurl line.
baseurl=file:///opt/build/public_html/smock/yum/centos-56/ia64
enabled=1
keepcache=0

"""
----> Note the closing triple double quotes; we're not in Kansas anymore

----> The below syntax is for mock 1.0.28. This syntax will not work for
----> mock 0.6.13 as distributed in CentOS-Extras. See the mock config
----> Johnny posted several months ago for the syntax for 0.6.13

config_opts['macros'] ['%_topdir'] = "/builddir/build"
config_opts['macros'] ['%_rpmfilename'] = "%%{NAME}-%%{VERSION}-%%{RELEASE}.%%{ARCH}.rpm"

# Change the next two lines to reflect yourself.

config_opts['macros'] ['%packager'] = "Lamar Owen <lowen@pari.edu>"
config_opts['macros'] ['%vendor'] = "PARI"
config_opts['macros'] ['%distribution'] = "PARI Internal Altix EL5 Rebuild"

----> I initially followed the advice below, changing %dist to first be
---->"pari.lro" (which mangles the release) and then ".pari.lro" but then
----> I found that %dist must be .el5 for certain packages to
----> properly build. (gdb, for one, at least on ia64). A grep through all
----> spec files to find conditional buildrequires would be necessary to
----> find all of the packages that this macro impacts.
----> And, of course, I now have 'pari.lro' and '.pari.lro' tagged packages
----> in my 5.5 repos that will eventually be replaced when I mass
----> rebuild 5.8/9.

# please change this to reflect the Distro Tree and Repo hosting packages!
config_opts['macros'] ['%dist'] = ".el5"
config_opts['macros'] ['%centos_ver'] = "5"

#need this to get el5 builddeps from some packages.
config_opts['macros'] ['%el5'] = "1"
config_opts['macros'] ['%rhel'] = "5"
config_opts['macros'] ['%_smp_mflags'] = "-j30"

----> The above line is for my 30 CPU Altix; change this to match your number
----> of CPU's and thus the desired number of parallel compiles make will do.

[lowen@winterstar ~]$
+++++++++++++++++++++++++++

Ok, now we need to test mock, and we do this by initializing the chroot:

mock -r centos-56-ia64 --init

If this fails, you have other problems. Look at the output, it will tell you what your problem is. This has to work, or nothing else will. And I can't give you a recipe that is guaranteed to work, every time; you need to understand yum and how to populate a repository properly in order to fix issues with the chroot init. I figured it out; so can you, it just takes some thought and some work, you're capable of doing that.

Now, generate a list of packages to build, and put them in a single line (xargs -n 10000 works well for this), and put it in, say, ~/smock-packages.txt.

The first two packages to build are critical, and to build them you need to temporarily comment out the [smock] repo in the yum.conf section of the mock config. Change dir to the SRPMS directory containing the new centos-release package, and issue (I'm on ia64, so I use that here):
smock.pl --arch=ia64 --distro=centos-56 centos-release*src.rpm

This gets the centos-release package and the centos-release-notes package, and you need both, and they need each other, and if you put one in the [smock] repo without the other, and the [smock] repo is enabled for chroot populating, it will break the chrooted yum installs and nothing will build. So leave [smock] disabled for building these two.

Once those two build, uncomment the [smock] repo in the mock configuration. I know, I could probably just set enable=0 instead of commenting it out, but I chose to comment it out, and uncomment it back in instead.

For the build itself, I like to first see what smock.pl is going to do with a:
smock.pl --dryrun --arch=ia64 --distro=centos-56 `cat ~/smock-packages.txt`

Smock will send to stderr any dependency loops prefixed with a 'tsort:' and will send the package names in build order out stdout. If you have the downrev packages that will satisfy the deploops, you don't need to worry about them. If you don't have those packages, you'll need to build or find at least one package in each deploop and put it in the [buildroot-reqs] (which is part of the reason I put it there), being sure to run createrepo every time you hand modify any repo.

Save the output somewhere, if nothing else so you can see where the build is by looking at stdout later.

I use nohup; you can use screen or dtach or whatever. My actual building command line is:
nohup time smock.pl --keepgoing --arch=ia64 --distro=centos-56
`cat ~/smock-packages.txt` &

And I then tail -f nohup.out to watch the build.

Now, smock.pl has a buglet of sorts, and as the number of packages accumulates this buglet makes the build get progressively slower. Smock is maintaining a repository of its own; in the build loop it does a createrepo both before and after each and every package build. When a repo gets more than a few hundred packages in it, the createrepo can take longer than the package build does.

I made a modified 'smock2' that removed the automatic createrepo for the initial pass at the 5.5 packages, and any other time I knew the package set was going to be large, to save some on time, or when the buildroot seed repo contained a vast majority of the build dependencies. Or if I disabled the [smock] repo; if the [smock] repo isn't being used by mock, then why spend the time doing a createrepo on it?

Now, my 5.6 update build isn't complete, so I really don't know how well this 'stepwise' staging is going to work in practice. I should know tomorrow morning; the buid is at:

[lowen@winterstar ~]$ buildstatus56
No IA64 target: 4
No Package Found: 0
Scratch total: 8
Built binaries: 286
Built SRPMS: 86
[lowen@winterstar ~]$

Hmm, there are 8 builds in scratch; this is where failed build build, root, and state logs are kept, as well as where the currently running build, root, and state logs are put until a good build, when they get moved to the logs tree under the binary RPMS dir. So, that means 7 failed builds; 4 of those are due to having not IA64 target for the build. So 3 builds, so far, that I'll need to further investigate.

Tomorrow.

I hope this post is helpful to someone; if nothing else I'm putting it in my own archive for my own reference..... :-)
_______________________________________________
CentOS-devel mailing list
CentOS-devel@centos.org
http://lists.centos.org/mailman/listinfo/centos-devel
 
Old 10-04-2012, 10:36 PM
Lamar Owen
 
Default Interested in IA64 build.

On Thursday, October 04, 2012 06:14:12 PM Lamar Owen wrote:
> On Thursday, October 04, 2012 11:45:14 AM Lamar Owen wrote:
> > Ok, well, I've learned a lot since last Friday. Especially as it pertains to circular dependencies and how to properly (hopefully) step from one release up to another. ...
>
> And after I wrote that, I realized that I hadn't documented how I did that. So, in this admittedly longish e-mail, I'll attempt to do that.
...
> I hope this post is helpful to someone; if nothing else I'm putting it in my own archive for my own reference..... :-)

Oh, forgot to mention the most important things.

I'm not targeting strict binary compatibility with upstream for this build. If there were to be sufficient interest in an IA64 build that is binary compatible, that can be done, but that makes things that much more difficult to get right. And being there hasn't really been any interest, I haven't been concerned with the binary compatibility issue.

Nor am I doing serious QA on these bootstrapping builds; that will happen once I'm up to 5.8 or 5.9. So nothing in what I've written touches QA, or binary compatibility, or signing of the resulting packages, for that matter.

If I need to cross those bridges, I'll document how I crossed them.
_______________________________________________
CentOS-devel mailing list
CentOS-devel@centos.org
http://lists.centos.org/mailman/listinfo/centos-devel
 

Thread Tools




All times are GMT. The time now is 10:09 AM.

VBulletin, Copyright ©2000 - 2014, Jelsoft Enterprises Ltd.
Content Relevant URLs by vBSEO ©2007, Crawlability, Inc.
Copyright 2007 - 2008, www.linux-archive.org