FAQ Search Today's Posts Mark Forums Read
» Video Reviews

» Linux Archive

Linux-archive is a website aiming to archive linux email lists and to make them easily accessible for linux users/developers.


» Sponsor

» Partners

» Sponsor

Go Back   Linux Archive > CentOS > CentOS Development

 
 
LinkBack Thread Tools
 
Old 04-07-2012, 01:24 PM
Peter Penzov
 
Default How Centos compiles SRPMs from Red Hat

I have some more questions:
1. Do you use a cluster for building the source code or you take the SRPMs and manually copy/paste them into the building systems?
2. What tool do you use to find differences into SRPMs? I don't believe that you extract the packets manually and compate the code with diff tool every time.*


Best wishes

On Sat, Apr 7, 2012 at 2:10 PM, Johnny Hughes <johnny@centos.org> wrote:

On 04/06/2012 06:33 PM, Peter Penzov wrote:

> I have some questions to ask:

> 1. What build server do you use and how is configured? Can you paste the

> mock configuration? (If you use mock)

> 2. How much time is necessary to build the complete OS?(all packages)

> 3. When you download the SRPMs from Red Hat ftp server how do you remove

> the old packages from the latest?

> 4. Are there any hidden stones?

>

> Please share



Don't top post ... it is annoying and is not per our guidelines.



http://en.wikipedia.org/wiki/Posting_style



Our guidelines are here:



http://www.centos.org/modules/tinycontent/index.php?id=16



================================================== ======



Karanbir answered the rest of the questions ... but here is an answer to

the one about how long (your #2):



We have no idea how long it would to take to rebuild everything. *We

don't do that except for the first time we build a release. *We don't

even have any idea if the current SRPMS would build right now if you

tried it (although the SHOULD).



This is because we don't rebuild everything on every run. *We only

rebuild NEW things as they are released.



So, upstream releases 3 SRPMs today and we rebuild them today. *They

release 3 more SRPMS tomorrow and we rebuild them tomorrow, etc.



So our rebuild is Staged ... if you take all the SRPMS and rebuild them

all at the same time then that may or may not work and may or may not

produce identical results.



For example, things that remain from the original CentOS-6.0 build were

built against the repositories as they existed at that point in time.

If you rebuild them now against CentOS-6.2 (as it exists now), that

could introduce some inconsistencies as you would be using a different

gcc and glibc (the new ones not the 6.0 ones).



================================================== ======



Also, please note that the "CentOS Linux" distribution is open source

and we (the "CentOS Project") provide all source code as we build it.

(That would be the items from the SOURCE directories when you extract

the SRPMS).



We also provide all the scripts and methods required to rebuild it (that

would be the SPEC file and rpmbuild and our distro, if you install it).



Our goal is to provide you will a distribution that is free to use and

to provides sources as required by the licenses of the software that are

contained in CentOS. *You provide your own support, although you can use

our mailing lists, forums, and wiki to get help from others in the

"CentOS Community".



However, it has never been the intent of the "CentOS Project" to tell

you how to reproduce CentOS ... we just to provide the things we need to

provide because we are Open Source.




_______________________________________________

CentOS-devel mailing list

CentOS-devel@centos.org

http://lists.centos.org/mailman/listinfo/centos-devel




_______________________________________________
CentOS-devel mailing list
CentOS-devel@centos.org
http://lists.centos.org/mailman/listinfo/centos-devel
 
Old 04-07-2012, 01:25 PM
Itamar Reis Peixoto
 
Default How Centos compiles SRPMs from Red Hat

On Sat, Apr 7, 2012 at 10:24 AM, Peter Penzov <peter.penzov@gmail.com> wrote:
> I have some more questions:
> 1. Do you use a cluster for building the source code or you take the SRPMs

the best way is to use koji to do this, you can have thousands of
machines managed by koji

> and manually copy/paste them into the building systems?
> 2. What tool do you use to find differences into SRPMs? I don't believe that
> you extract the packets manually and compate the code with diff tool every
> time.
>
> Best wishes
>






--
------------

Itamar Reis Peixoto
msn, google talk: itamar@ispbrasil.com.br
+55 11 4063 5033 (FIXO SP)
+55 34 9158 9329 (TIM)
+55 34 8806 3989 (OI)
+55 34 3221 8599 (FIXO MG)
_______________________________________________
CentOS-devel mailing list
CentOS-devel@centos.org
http://lists.centos.org/mailman/listinfo/centos-devel
 
Old 04-07-2012, 02:42 PM
Johnny Hughes
 
Default How Centos compiles SRPMs from Red Hat

On 04/07/2012 08:24 AM, Peter Penzov wrote:
> I have some more questions:

You are still top posting ... please don't.

> 1. Do you use a cluster for building the source code or you take the
> SRPMs and manually copy/paste them into the building systems?

There are any number of build systems out there for different things.
There is the OBS (Open Build System from open SUSE ... Dell uses this
alot), Koji (the Scientific Linux system uses this), or plague (we use
this for CentOS-4 and CentOS-5).

However, we wrote our own build system for CentOS-6.

We basically use a beanstalk queue. However, what you use does not
matter. I still use scripts like the ones we initially used at the very
beginning of CentOS-3 as well.



> 2. What tool do you use to find differences into SRPMs? I don't believe
> that you extract the packets manually and compate the code with diff
> tool every time.

There is NO PROGRAM that I use. I am a Unix system admin. If I want to
do something, I use the tools I have been using for 20 years to do it.

In the case of looking at the inside of an SRPM, I would personally do
it this way.

1. Download the redhat SRPM from here (for example, lets do the latest
firefox):


ftp://ftp.redhat.com/pub/redhat/linux/enterprise/6Client/en/os/SRPMS/firefox-10.0.3-1.el6_2.src.rpm

and the centos SRPM

http://vault.centos.org/6.2/updates/Source/SPackages/firefox-10.0.3-1.el6.centos.src.rpm

2. Now you need to extract the SRPMS in to separate directories. There
are many ways to do this. 2 ways are are to use "rpm2cpio" ...OR...
"rpm -Uvh"

If all want to do is to see the differences and not rebuild the pack, I
would use rpm2cpio like this:

a. create a separate directory and put each SRPM in there.

b. extract the RedHat SRPM like this:
rpm2cpio firefox-10.0.3-1.el6_2.src.rpm | cpio -idv

c. go to the directory with the CentOS SRPM in it and do this:
rpm2cpio firefox-10.0.3-1.el6.centos.src.rpm | cpio -idv

d. now do a diff of the two directories:
diff -uNrp RHEL/ CentOS/

The output looks like this:

http://pastebin.centos.org/38641

So, you can see that besides the actual SRPM files (which you know are
different and line 1 and 2 in the link) ... there are 3 other differences.

We added a file named "firefox-centos-default-prefs.js" (lines 4-18) and
removed a file named "firefox-redhat-default-prefs.js" (lines 20-34).

We also modified the SPEC file (lines 36-58)

(The changes are changing the name redhat to centos for sources 12 and
13 and a changelog entry ... note that since this is centos6 there is no
source 13)

3. Every other file in those two directories are the same.

================================================== =======

How would I compare every file ... I would script something to do it.

You could also create one directory named firefox, and you could (if you
had rpm and rpmbuild installed) also do it this way:

1. download both files.

2. Install the CentOS SRPM like this:

rpm --define "_topdir `pwd`" firefox-10.0.3-1.el6.centos.src.rpm
mv SOURCES SOURCES.centos
mv SPECS SPECS.centos

3. Install the Red Hat SRPM like this:

rpm --define "_topdir `pwd`" firefox-10.0.3-1.el6_2.src.rpm
mv SOURCES SOURCES.rhel
mv SPECS SPECS.rhel

4. do diffs for each dir like this:

diff -uNrp SPECS.rhel/ SPECS.centos/ > SPECS.diff
diff -uNrp SOURCES.rhel/ SOURCES.centos/ > SOURCES.diff

================================================== ======
If you want to have a local git repo that tracks the differences, you
could do this instead:

1. download both files, put them in a firefox dir.

2. Install the RHEL SRPM (the original file):

rpm --define "_topdir `pwd`" firefox-10.0.3-1.el6_2.src.rpm

3. add the SPECS and SOURCES directories to the git repo (I don't like
to add bzipped or gzipped or tar files to git, so I exclude those from
the repos):

cd SPECS; git add *; cd ../

cd SOURCES; git add $(ls | egrep -v ".bz2|.gz|.tar"); cd ../

4. commit the changes to the git repo so you can track the centos
changes later:

git commit -m "initial rhel rpm imported"

5. Repeat steps 2, 3 for the CentOS SRPM ...

a. Now you can see the spec diff doing:

git diff SPECS/

b. or the SOURCES diff doing:

git diff SOURCES/

6. Now you can commit the CentOS changes to git:

git commit -m "initial centos rpm imported"

================================================== =============
I am not going to also tell you how to do it for svn or some other VCS
system ... the bottom line is that there are a hundred different ways to
extract the original SRPMS and make changes and track the changes.

Once you do the changes to the SPEC file and the SOURCES, you rebuild
the SRPM and then submit that SRPM somehow to a build system (mock,
build locally, plague, koji, brew, OBS, ect.). You can have any number
of build systems.

This is not rocket science ...

_______________________________________________
CentOS-devel mailing list
CentOS-devel@centos.org
http://lists.centos.org/mailman/listinfo/centos-devel
 
Old 04-07-2012, 03:34 PM
Peter Penzov
 
Default How Centos compiles SRPMs from Red Hat

On Sat, Apr 7, 2012 at 5:42 PM, Johnny Hughes <johnny@centos.org> wrote:

On 04/07/2012 08:24 AM, Peter Penzov wrote:

> I have some more questions:



You are still top posting ... please don't.



> 1. Do you use a cluster for building the source code or you take the

> SRPMs and manually copy/paste them into the building systems?



There are any number of build systems out there for different things.

There is the OBS (Open Build System from open SUSE ... Dell uses this

alot), Koji (the Scientific Linux system uses this), or plague (we use

this for CentOS-4 and CentOS-5).



However, we wrote our own build system for CentOS-6.



We basically use a beanstalk queue. *However, what you use does not

matter. *I still use scripts like the ones we initially used at the very

beginning of CentOS-3 as well.







> 2. What tool do you use to find differences into SRPMs? I don't believe

> that you extract the packets manually and compate the code with diff

> tool every time.



There is NO PROGRAM that I use. *I am a Unix system admin. *If I want to

do something, I use the tools I have been using for 20 years to do it.



In the case of looking at the inside of an SRPM, I would personally do

it this way.



1. *Download the redhat SRPM from here (for example, lets do the latest

firefox):





ftp://ftp.redhat.com/pub/redhat/linux/enterprise/6Client/en/os/SRPMS/firefox-10.0.3-1.el6_2.src.rpm




and the centos SRPM



http://vault.centos.org/6.2/updates/Source/SPackages/firefox-10.0.3-1.el6.centos.src.rpm



2. *Now you need to extract the SRPMS in to separate directories. *There

are many ways to do this. *2 ways are are to use "rpm2cpio" ...OR...

"rpm -Uvh"



If all want to do is to see the differences and not rebuild the pack, I

would use rpm2cpio like this:



*a. create a separate directory and put each SRPM in there.



*b. extract the RedHat SRPM like this:

* *rpm2cpio firefox-10.0.3-1.el6_2.src.rpm | cpio -idv



*c. go to the directory with the CentOS SRPM in it and do this:

* *rpm2cpio firefox-10.0.3-1.el6.centos.src.rpm | cpio -idv



*d. now do a diff of the two directories:

* *diff -uNrp RHEL/ CentOS/



The output looks like this:



http://pastebin.centos.org/38641



So, you can see that besides the actual SRPM files (which you know are

different and line 1 and 2 in the link) ... there are 3 other differences.



We added a file named "firefox-centos-default-prefs.js" (lines 4-18) and

removed a file named "firefox-redhat-default-prefs.js" (lines 20-34).



We also modified the SPEC file (lines 36-58)



(The changes are changing the name redhat to centos for sources 12 and

13 and a changelog entry ... note that since this is centos6 there is no

source 13)



3. *Every other file in those two directories are the same.



================================================== =======



How would I compare every file ... I would script something to do it.



You could also create one directory named firefox, and you could (if you

had rpm and rpmbuild installed) also do it this way:



1. *download both files.



2. *Install the CentOS SRPM like this:



rpm --define "_topdir `pwd`" firefox-10.0.3-1.el6.centos.src.rpm

mv SOURCES SOURCES.centos

mv SPECS SPECS.centos



3. Install the Red Hat SRPM like this:



rpm --define "_topdir `pwd`" firefox-10.0.3-1.el6_2.src.rpm

mv SOURCES SOURCES.rhel

mv SPECS SPECS.rhel



4. do diffs for each dir like this:



diff -uNrp SPECS.rhel/ SPECS.centos/ > SPECS.diff

diff -uNrp SOURCES.rhel/ SOURCES.centos/ > SOURCES.diff



================================================== ======

If you want to have a local git repo that tracks the differences, you

could do this instead:



1. download both files, put them in a firefox dir.



2. *Install the RHEL SRPM (the original file):



rpm --define "_topdir `pwd`" firefox-10.0.3-1.el6_2.src.rpm



3. *add the SPECS and SOURCES directories to the git repo (I don't like

to add bzipped or gzipped or tar files to git, so I exclude those from

the repos):



cd SPECS; git add *; cd ../



cd SOURCES; git add $(ls | egrep -v ".bz2|.gz|.tar"); cd ../



4. *commit the changes to the git repo so you can track the centos

changes later:



git commit -m "initial rhel rpm imported"



5. *Repeat steps 2, 3 for the CentOS SRPM ...



*a. *Now you can see the spec diff doing:



git diff SPECS/



*b. *or the SOURCES diff doing:



git diff SOURCES/



6. *Now you can commit the CentOS changes to git:



git commit -m "initial centos rpm imported"



================================================== =============

I am not going to also tell you how to do it for svn or some other VCS

system ... the bottom line is that there are a hundred different ways to

extract the original SRPMS and make changes and track the changes.



Once you do the changes to the SPEC file and the SOURCES, you rebuild

the SRPM and then submit that SRPM somehow to a build system (mock,

build locally, plague, koji, brew, OBS, ect.). *You can have any number

of build systems.



This is not rocket science ...




_______________________________________________

CentOS-devel mailing list

CentOS-devel@centos.org

http://lists.centos.org/mailman/listinfo/centos-devel



Yes it will be very difficult to create a custom build.

I want to ask you a very specific question:
If I decide to create my custom build from Red Hat Linux (with removed Red Hat logos, red color and etc in order not to violate the GPL license) and I just change the name with my own are there going to be any build problems?


I'm not interested in becoming a competitor in Red Hat's support businesses I just want open source OS certified for installation by Oracle database which I can custom modify. I'm not interested in any kernel code modification or package source code modification. I just want to build custom OS with just changed name and color. Maybe deployed on no more that 20 servers.


If I build from source Red Hat just with change name and color are there going to be any problems like dependency problems package source code problems and etc, left by Red Hat to make the life of the competitors terrible? I'm not interested what are they just answer me with YES or NO. I'm sure that you have insight development information about this.

_______________________________________________
CentOS-devel mailing list
CentOS-devel@centos.org
http://lists.centos.org/mailman/listinfo/centos-devel
 
Old 04-07-2012, 03:54 PM
Lamar Owen
 
Default How Centos compiles SRPMs from Red Hat

[Note: when posting replies, please take the time to trim the quoted
material (using the tag "[snip]" or points of ellipses (like I use
below) to denote where such trimming has been done). Bottom posting
without trimming is more annoying than top-posting. And, if I can do
it with Apple Mail on Mac OS X you can do it from your client, too.
Thank you!]


On Apr 7, 2012, at 11:34 AM, Peter Penzov wrote:
...
I'm not interested in becoming a competitor in Red Hat's support
businesses I just want open source OS certified for installation by
Oracle database which I can custom modify. I'm not interested in any
kernel code modification or package source code modification. I just
want to build custom OS with just changed name and color. Maybe
deployed on no more that 20 servers.

...

Hmm, while it's not CentOS you may want to look at how Scientific
Linux does 'sites' (in their terminology). As Scientific Linux is
built from the same source RPMS as CentOS is, much of the 'sites' info
should be at least somewhat applicable.


See:http://www.scientificlinux.org/documentation/howto/create.site
(note that that's currently throwing an internal server error; they
run Plone, and Plone has apparently had an issue).


They key is to not run afoul of the CentOS trademarks, if you use a
CentOS base (if you use an SL base, their usage guidelines apply, and
they are different from the CentOS ones I'm sure). And, as always, I
reserve the right to be wrong, and Johnny, Karanbir, or other CentOS
dev is more than welcome to correct me if need be.


_______________________________________________
CentOS-devel mailing list
CentOS-devel@centos.org
http://lists.centos.org/mailman/listinfo/centos-devel
 
Old 04-07-2012, 03:54 PM
Hakan Koseoglu
 
Default How Centos compiles SRPMs from Red Hat

On 7 April 2012 16:34, Peter Penzov <peter.penzov@gmail.com> wrote:
> I'm not interested in becoming a competitor in Red Hat's support businesses
> I just want open source OS certified for installation by Oracle database
> which I can custom modify. I'm not interested in any kernel code
> modification or package source code modification. I just want to build
> custom OS with just changed name and color. Maybe deployed on no more that
> 20 servers.
If you do this, your platform will not be certified by Oracle. You can
try and submit a new certification but them accepting it, in my
guess, would be an unlikely result. CentOS is not certified nor
supported by Oracle either.

In the end there is nothing stopping you to do what you have described
but I question the value. I presume the servers you have will be used
for production, not development. Especially with the Oracle licencing
costs for ~20 servers, you better have supported platforms. Using
CentOS+Oracle for development environments will mean your servers will
not be supported by Oracle but at least you will have the support of
the CentOS community for the OS. I wonder how much quicker you would
be than the CentOS guys respond to updated packages and for how long
you would be able to keep up with the service (which in both aspects
the CentOS guys are excellent).

The only Linux distributions supported by Oracle for Oracle database
are RHEL, OEL, SUSE SLES and finally Asianux. None of the other RHEL
derivatives are supported, neither Ubuntu (server or desktop variants,
TLS or non-TLS) nor openSUSE. (See Metalink ID 1304727.1).

By the way, the removing of logos etc. are to comply with trademark
rules, not GPL.
_______________________________________________
CentOS-devel mailing list
CentOS-devel@centos.org
http://lists.centos.org/mailman/listinfo/centos-devel
 
Old 04-07-2012, 04:12 PM
Peter Penzov
 
Default How Centos compiles SRPMs from Red Hat

On Sat, Apr 7, 2012 at 6:54 PM, Hakan Koseoglu <hakan@koseoglu.org> wrote:

On 7 April 2012 16:34, Peter Penzov <peter.penzov@gmail.com> wrote:

> I'm not interested in becoming a competitor in Red Hat's support businesses

> I just want open source OS certified for installation by Oracle database

> which I can custom modify. I'm not interested in any kernel code

> modification or package source code modification. I just want to build

> custom OS with just changed name and color. Maybe deployed on no more that

> 20 servers.

If you do this, your platform will not be certified by Oracle. You can

try and submit a new certification but them accepting it, in my

guess, would be an unlikely result. CentOS is not certified nor

supported by Oracle either.



In the end there is nothing stopping you to do what you have described

but I question the value. I presume the servers you have will be used

for production, not development. Especially with the Oracle licencing

costs for ~20 servers, you better have supported platforms. Using

CentOS+Oracle for development environments will mean your servers will

not be supported by Oracle but at least you will have the support of

the CentOS community for the OS. I wonder how much quicker you would

be than the CentOS guys respond to updated packages and for how long

you would be able to keep up with the service (which in both aspects

the CentOS guys are excellent).



The only Linux distributions supported by Oracle for Oracle database

are RHEL, OEL, SUSE SLES and finally Asianux. None of the other RHEL

derivatives are supported, neither Ubuntu (server or desktop variants,

TLS or non-TLS) nor openSUSE. (See Metalink ID 1304727.1).



By the way, the removing of logos etc. are to comply with trademark

rules, not GPL.


I suppose you mean that I have to provide the full source code in order to be GPL compatible? Sure, not a problem.


_______________________________________________

CentOS-devel mailing list

CentOS-devel@centos.org

http://lists.centos.org/mailman/listinfo/centos-devel



_______________________________________________
CentOS-devel mailing list
CentOS-devel@centos.org
http://lists.centos.org/mailman/listinfo/centos-devel
 
Old 04-12-2012, 07:29 PM
Charlie Brady
 
Default How Centos compiles SRPMs from Red Hat

On Sat, 7 Apr 2012, Karanbir Singh wrote:

> On 04/06/2012 08:38 PM, Peter Penzov wrote:
> > Do you have internal manual or a guide that you can share?
>
> well, converting srpms to rpms is covered by 'man rpmbuild', everything
> on top of that is to suit taste or process. Or: if just a case of
> grabbing the srpm and then :
>
> rpmbuild -ba <srpm>

Are you being disingenuous? The original question in this thread was:

I'm interested how Centos Team changes and compiles SRPMs from Red Hat?

So *you* just do "rpmbuild -ba <srpm>", do you?
_______________________________________________
CentOS-devel mailing list
CentOS-devel@centos.org
http://lists.centos.org/mailman/listinfo/centos-devel
 
Old 04-12-2012, 11:11 PM
Karanbir Singh
 
Default How Centos compiles SRPMs from Red Hat

On 04/12/2012 08:29 PM, Charlie Brady wrote:
>
> I'm interested how Centos Team changes and compiles SRPMs from Red Hat?
>
> So *you* just do "rpmbuild -ba <srpm>", do you?

pretty much.

--
Karanbir Singh
+44-207-0999389 | http://www.karan.org/ | twitter.com/kbsingh
ICQ: 2522219 | Yahoo IM: z00dax | Gtalk: z00dax
GnuPG Key : http://www.karan.org/publickey.asc
_______________________________________________
CentOS-devel mailing list
CentOS-devel@centos.org
http://lists.centos.org/mailman/listinfo/centos-devel
 
Old 04-12-2012, 11:12 PM
Karanbir Singh
 
Default How Centos compiles SRPMs from Red Hat

On 04/12/2012 08:29 PM, Charlie Brady wrote:
>
> I'm interested how Centos Team changes and compiles SRPMs from Red Hat?
>
> So *you* just do "rpmbuild -ba <srpm>", do you?

although, more accurately: rpmbuild --rebuild <srpm>

--
Karanbir Singh
+44-207-0999389 | http://www.karan.org/ | twitter.com/kbsingh
ICQ: 2522219 | Yahoo IM: z00dax | Gtalk: z00dax
GnuPG Key : http://www.karan.org/publickey.asc
_______________________________________________
CentOS-devel mailing list
CentOS-devel@centos.org
http://lists.centos.org/mailman/listinfo/centos-devel
 

Thread Tools




All times are GMT. The time now is 04:38 AM.

VBulletin, Copyright ©2000 - 2014, Jelsoft Enterprises Ltd.
Content Relevant URLs by vBSEO ©2007, Crawlability, Inc.
Copyright 2007 - 2008, www.linux-archive.org