Linux Archive

Linux Archive (http://www.linux-archive.org/)
-   Gentoo User (http://www.linux-archive.org/gentoo-user/)
-   -   The CHOST variable (http://www.linux-archive.org/gentoo-user/484928-chost-variable.html)

Alan McKinnon 02-03-2011 11:27 PM

The CHOST variable
 
Apparently, though unproven, at 01:43 on Friday 04 February 2011, Nils Holland
did opine thusly:

I'm not in a position to give a fully definitive answer to 1) ...

> 2) /etc/make.conf contains a note that one should not change the CHOST
> lightly (not that I'm planning to) and links to a nice document
> explaining how it can be done anyway (which, I have to admit, didn't
> make me any wiser, however). The question is, out of curiosity, why
> the CHOST should not be changed and what would happen if one did it
> anyway. I willingly believe that it would lead to problems, but would
> the actual cause of these problems actually be caused by the
> configuration of the machine being mixed up (for example, by the GNU
> build system / autoconf suddenly looking for a compiler or similiar
> tools / libraries under a path or by a name involving, for example,
> i486-pc-linux-gnu, which does not automatically exist of the
> appropriate tools have not been installed accordingly. Or would
> problems arise because code generated with the new CHOST does no
> longer "fit" to code generated with the previous / old CHOST?

The warning is actually there to stop users doing stupid things like blindly
trying to convert 32 bit systems to 64 bit. This is how that goes down:

1. Change CHOST
2. emerge -e world
3. ???
4. Fail!

Yes, if you are real smart it can be done. But "real smart" really does mean
"real smart" i.e. not for the faint of heart and certainly not worth being
officially supported.


--
alan dot mckinnon at gmail dot com

Florian Philipp 02-04-2011 04:58 AM

The CHOST variable
 
Am 04.02.2011 01:27, schrieb Alan McKinnon:
> Apparently, though unproven, at 01:43 on Friday 04 February 2011, Nils Holland
> did opine thusly:
>
> I'm not in a position to give a fully definitive answer to 1) ...
>
>> 2) /etc/make.conf contains a note that one should not change the CHOST
>> lightly (not that I'm planning to) and links to a nice document
>> explaining how it can be done anyway (which, I have to admit, didn't
>> make me any wiser, however). The question is, out of curiosity, why
>> the CHOST should not be changed and what would happen if one did it
>> anyway. I willingly believe that it would lead to problems, but would
>> the actual cause of these problems actually be caused by the
>> configuration of the machine being mixed up (for example, by the GNU
>> build system / autoconf suddenly looking for a compiler or similiar
>> tools / libraries under a path or by a name involving, for example,
>> i486-pc-linux-gnu, which does not automatically exist of the
>> appropriate tools have not been installed accordingly. Or would
>> problems arise because code generated with the new CHOST does no
>> longer "fit" to code generated with the previous / old CHOST?
>
> The warning is actually there to stop users doing stupid things like blindly
> trying to convert 32 bit systems to 64 bit. This is how that goes down:
>
> 1. Change CHOST
> 2. emerge -e world
> 3. ???
> 4. Fail!
>
> Yes, if you are real smart it can be done. But "real smart" really does mean
> "real smart" i.e. not for the faint of heart and certainly not worth being
> officially supported.
>

Is the same true for more compatible arches like i486 -> i686? I have a
system where I used the wrong stage-3 and now I'm stuck with an i486
CHOST on an Atom netbook where I could use every bit of performance.

Regards,
Florian

Alan McKinnon 02-04-2011 08:20 AM

The CHOST variable
 
Apparently, though unproven, at 11:29 on Friday 04 February 2011, Nils Holland
did opine thusly:

> Florian Philipp wrote:
> > Am 04.02.2011 01:27, schrieb Alan McKinnon:
> >> Yes, if you are real smart it can be done. But "real smart" really does
> >> mean
> >> "real smart" i.e. not for the faint of heart and certainly not worth
> >> being
> >> officially supported.
> >
> > Is the same true for more compatible arches like i486 -> i686? I have a
> > system where I used the wrong stage-3 and now I'm stuck with an i486
> > CHOST on an Atom netbook where I could use every bit of performance.
>
> Yep, that's about the same direction I was wondering about when asking my
> question.
>
> Specifically, I've been playing around with catalyst a bit in order to
> create my own stage3 tarballs and install discs. Works fine. But then I
> started to wonder about the following: I'd probably like to make my own
> stage3 and install disc i486 or even i386, so that it works "everywhere".
> Then, on whatever machine I install it, I would probably like to change
> the CHOST to something suitable for that machine, and that's actually what
> the comment in /etc/make.conf suggests against doing. As a result, I
> started to wonder why this is so, and what exactly would happen if done
> anyway.
>
> Of course, this is not really a big problem. I could create multiple
> stage3's with catalyst whenever I really need one to install on a machine
> and have that stage3 by of the right type for the machine right from the
> start. But still, always eager to learn, I thought I'd ask about the
> background behind all that stuff. ;-)

Interestingly, Ubuntu has always built for basic arches, and they seem to get
away with it.

IIRC they are now on i586 but for the longest time used i386. No performance
issues. You might want to investigate how they do their builds and see if you
can use their tricks.


--
alan dot mckinnon at gmail dot com

"Nils Holland" 02-04-2011 08:29 AM

The CHOST variable
 
Florian Philipp wrote:

> Am 04.02.2011 01:27, schrieb Alan McKinnon:
>>
>> Yes, if you are real smart it can be done. But "real smart" really does
>> mean
>> "real smart" i.e. not for the faint of heart and certainly not worth
>> being
>> officially supported.
>>
>
> Is the same true for more compatible arches like i486 -> i686? I have a
> system where I used the wrong stage-3 and now I'm stuck with an i486
> CHOST on an Atom netbook where I could use every bit of performance.

Yep, that's about the same direction I was wondering about when asking my
question.

Specifically, I've been playing around with catalyst a bit in order to
create my own stage3 tarballs and install discs. Works fine. But then I
started to wonder about the following: I'd probably like to make my own
stage3 and install disc i486 or even i386, so that it works "everywhere".
Then, on whatever machine I install it, I would probably like to change
the CHOST to something suitable for that machine, and that's actually what
the comment in /etc/make.conf suggests against doing. As a result, I
started to wonder why this is so, and what exactly would happen if done
anyway.

Of course, this is not really a big problem. I could create multiple
stage3's with catalyst whenever I really need one to install on a machine
and have that stage3 by of the right type for the machine right from the
start. But still, always eager to learn, I thought I'd ask about the
background behind all that stuff. ;-)

Greetings,
Nils

Neil Bothwick 02-04-2011 08:31 AM

The CHOST variable
 
On Fri, 04 Feb 2011 06:58:27 +0100, Florian Philipp wrote:

> Is the same true for more compatible arches like i486 -> i686? I have a
> system where I used the wrong stage-3 and now I'm stuck with an i486
> CHOST on an Atom netbook where I could use every bit of performance.

It's possible, there's even a Gentoo document on it,
http://www.gentoo.org/doc/en/change-chost.xml

Note the first line states

"Changing the CHOST is a big issue that can seriously screw up your
system"

Any time you consider a process that involves emerge -e world, you should
also consider a reinstall. A reinstall will certainly be quicker, the
only reason for an in place fix is if you cannot take the machine down
for that length of time.


--
Neil Bothwick

On the other hand, you have different fingers.

"Nils Holland" 02-04-2011 12:50 PM

The CHOST variable
 
Alan McKinnon wrote:

> Interestingly, Ubuntu has always built for basic arches, and they seem to
> get away with it.
>
> IIRC they are now on i586 but for the longest time used i386. No
> performance issues. You might want to investigate how they do
> their builds and see if you can use their tricks.

The question is, I guess, if the target host, when of the same arch (i.e.
i[3456]86) does actually have any influence on the code that gets
generated in terms of performance or ability to run on other sub-arches.
This is what I really couldn't find out so far and would find highly
interesting to know.

For example, why not just go (and stay) with CHOST="i386-pc-linux-gnu" and
on an i686 machine, set march or mcpu = i686 via CFLAGS if you want to
optimize for the particular subarch at hand? Why should it be necessary /
what would the (dis)advantages be of of such a setup vs. also having CHOST
set to "i686-pc-linux-gnu"?

Concering the Gentoo doc about changing the CHOST that was mentioned: Yep,
I've read that. If I understood it correctly, problems when changing CHOST
mainly seem to arise because of the way GCC and related basic build utils
install themselves (using the host triplet as part of their path or
executable name), leaving wrong / messed up references when changing the
CHOST.

For example, as I've said previously, the CHOST value gets passed to
./configure as --host for each package that gets build. That would make
configure try to select a compiler called <CHOST>-gcc in order to compile
the package, i.e. when CHOST is i486-pc-linux-gnu, a compiler called
i486-pc-linux-gnu-gcc would be used. Include file directories for glibc
and / or glibc itself sems to be selected by a similiar mechanism. All
right, no problem, so far this only determines how things are called and
where they are located, but are there any other "real" effects of all this
stuff?

The Gentoo CHOST document that was mentioned says something about nptl not
being available on i386. If true, and if the host variable generally
influences the availability of features, things would slowly start to make
sens as to why the CHOST might be important. On the other hand, I've read
through some documentation of the GNU C library (glibc) and didn't even
find anything about nptl not being available on i386, not to mention
anything else about different features on different subarches.

You see, I'm probably not entirely "getting it" yet. ;-)

Greetings,
Nils

Marius Vaitiekunas 02-04-2011 04:40 PM

The CHOST variable
 
On Fri, Feb 4, 2011 at 3:50 PM, Nils Holland <nhg@tisys.org> wrote:
> Alan McKinnon wrote:
>
>> Interestingly, Ubuntu has always built for basic arches, and they seem to
>> get away with it.
>>
>> IIRC they are now on i586 but for the longest time used i386. No
>> performance issues. You might want to investigate how they do
>> their builds and see if you can use their tricks.
>
> The question is, I guess, if the target host, when of the same arch (i.e.
> i[3456]86) does actually have any influence on the code that gets
> generated in terms of performance or ability to run on other sub-arches.
> This is what I really couldn't find out so far and would find highly
> interesting to know.
>
> For example, why not just go (and stay) with CHOST="i386-pc-linux-gnu" and
> on an i686 machine, set march or mcpu = i686 via CFLAGS if you want to
> optimize for the particular subarch at hand? Why should it be necessary /
> what would the (dis)advantages be of of such a setup vs. also having CHOST
> set to "i686-pc-linux-gnu"?
>
> Concering the Gentoo doc about changing the CHOST that was mentioned: Yep,
> I've read that. If I understood it correctly, problems when changing CHOST
> mainly seem to arise because of the way GCC and related basic build utils
> install themselves (using the host triplet as part of their path or
> executable name), leaving wrong / messed up references when changing the
> CHOST.
>
> For example, as I've said previously, the CHOST value gets passed to
> ./configure as --host for each package that gets build. That would make
> configure try to select a compiler called <CHOST>-gcc in order to compile
> the package, i.e. when CHOST is i486-pc-linux-gnu, a compiler called
> i486-pc-linux-gnu-gcc would be used. Include file directories for glibc
> and / or glibc itself sems to be selected by a similiar mechanism. All
> right, no problem, so far this only determines how things are called and
> where they are located, but are there any other "real" effects of all this
> stuff?
>
> The Gentoo CHOST document that was mentioned says something about nptl not
> being available on i386. If true, and if the host variable generally
> influences the availability of features, things would slowly start to make
> sens as to why the CHOST might be important. On the other hand, I've read
> through some documentation of the GNU C library (glibc) and didn't even
> find anything about nptl not being available on i386, not to mention
> anything else about different features on different subarches.
>
> You see, I'm probably not entirely "getting it" yet. ;-)
>
> Greetings,
> Nils
>
>
>

Hi,

I am not a big guru there, but i have changed CHOST variable
successfully few years ago. If I remember, the steps were like that:

Change CHOST variable.
Bootstrap system (like building from stage 1):
# /usr/portage/scripts/bootstrap.sh
# emerge -e system
# emerge -e world

Before gentoo has been providing daily stages, I was installing my
systems from stage1. It was a nice learning curve :)

--
mv

James 02-04-2011 06:57 PM

The CHOST variable
 
Nils Holland <nhg <at> tisys.org> writes:

> Florian Philipp wrote:

> Specifically, I've been playing around with catalyst a bit in order to
> create my own stage3 tarballs and install discs. Works fine. But then I
> started to wonder about the following: I'd probably like to make my own
> stage3 and install disc i486 or even i386, so that it works "everywhere".
> Then, on whatever machine I install it, I would probably like to change
> the CHOST to something suitable for that machine, and that's actually what
> the comment in /etc/make.conf suggests against doing. As a result, I
> started to wonder why this is so, and what exactly would happen if done
> anyway.


Ah music to my ears!

Several folks I know have been pondering your very ideas,
except as to how to apply the build out code for an
embedded Gentoo (x86) based tree. ALIX, ATOM, old
K5 and K6 machines, on and on and on. Surely
there is common ground amongst our efforts......?

I know this list is very sharp, but there are some
(deeply skilled) folks on the embedded-gentoo list:

gentoo-embedded@gentoo.org
http://www.gentoo.org/proj/en/base/embedded/handbook/

that if you pose your well-formed ideas and goals there
you'll find some of the best coding guidance
and pick up an ontourage of wanna_bee_follows (grin)
on your quest to build out for smaller x86 arches.

This aspect of gentoo's lineages is rich, but at
the moment, is in need of a champion......

Drop me some private email as I have lots
of links and resources, related to your quest.

sincerely,
James

Enrico Weigelt 02-04-2011 07:21 PM

The CHOST variable
 
* Nils Holland <nhg@tisys.org> wrote:

> 1) So a package using the GNU build system determines or is passed
> (via --host aka. CHOST) a target triplet specifying the system on
> which the resulting compiled code is supposed to run. What does the
> package do with that information? Does it only use it to determine
> what it has to compile (different / special code for different systems
> / architectures), or does this already have an influence on the
> optimization of the resulting code for a certain (sub-)architecture?

Exactly. Some packages have arch- and subarch-specific optimizations.
Those things IMHO should be primarily taken from the target triplet
(instead of other ./configure options), as it quite exactly defines
what cputype, platform and toolchain type you're using. It's very
important for crosscompiling (even some toolchains, eg. gcc., could,
and should, be explicitly asked for their target triplet).

> Forthermore: Does configuring a package with, say,
> --host=i686-pc-linux-gnu already automatically mean that said package
> would not be able to run on (for example) an i486-pc-linux-gnu host?

Maybe. Or maybe it's just quite slow, eg. when it's using certain
opcodes that are not supported by the older subarch and handled by
the kernel via exceptions (back in i386 times, that was the case for
fpops, when no fpu was present). It all depends on how the individual
package is actually coded.

> 2) /etc/make.conf contains a note that one should not change the CHOST
> lightly (not that I'm planning to) and links to a nice document
> explaining how it can be done anyway (which, I have to admit, didn't
> make me any wiser, however). The question is, out of curiosity, why
> the CHOST should not be changed and what would happen if one did it
> anyway. I willingly believe that it would lead to problems, but would
> the actual cause of these problems actually be caused by the
> configuration of the machine being mixed up (for example, by the GNU
> build system / autoconf suddenly looking for a compiler or similiar
> tools / libraries under a path or by a name involving, for example,
> i486-pc-linux-gnu, which does not automatically exist of the
> appropriate tools have not been installed accordingly. Or would
> problems arise because code generated with the new CHOST does no
> longer "fit" to code generated with the previous / old CHOST?

One of the most evil things that can happen (also w/ certain optimization
flags) is a mismatching calling convention between certain libraries
and executables. For example, code built for a newer subarch could put
more call arguments into registers which the called functions dont
expect - stack corruptions will happen.


cu
--
----------------------------------------------------------------------
Enrico Weigelt, metux IT service -- http://www.metux.de/

phone: +49 36207 519931 email: weigelt@metux.de
mobile: +49 151 27565287 icq: 210169427 skype: nekrad666
----------------------------------------------------------------------
Embedded-Linux / Portierung / Opensource-QM / Verteilte Systeme
----------------------------------------------------------------------

Enrico Weigelt 02-04-2011 07:39 PM

The CHOST variable
 
* Nils Holland <nhg@tisys.org> wrote:

> The question is, I guess, if the target host, when of the same arch (i.e.
> i[3456]86) does actually have any influence on the code that gets
> generated in terms of performance or ability to run on other sub-arches.
> This is what I really couldn't find out so far and would find highly
> interesting to know.

Yes, both. Passing arguments via register (instead of stack) can be
much faster in some cases, especially when you have a lot of calls
with small parameter sets, eg:

libfoo.c:

int foo(int a, int b)
{
... <do-something> ...
}

progbar.c:

int main()
{
...
for (int x=0; x<10000; x++)
foo(x, wurst);
...
}

On older subarchs, main() will have to push x and wurst on stack for
_each_ call, and foo() has to pop them down. With a newer subarch that
has more usable registers, the parameters a and b can directly sit in
registers (probably the counting of x might already happen in the same
one), so the parameters dont have to go through the stack. That all
heavily depends on the target ABI.

> For example, why not just go (and stay) with CHOST="i386-pc-linux-gnu" and
> on an i686 machine, set march or mcpu = i686 via CFLAGS if you want to
> optimize for the particular subarch at hand? Why should it be necessary /
> what would the (dis)advantages be of of such a setup vs. also having CHOST
> set to "i686-pc-linux-gnu"?

I'm not sure right now if/how -march and -mcpu affect calling convention,
variable alignments, etc (IMHO -march does, while -mcpu doesn't), but
as soon as you change these ABI elements, it can get really dangerous.

> Concering the Gentoo doc about changing the CHOST that was mentioned: Yep,
> I've read that. If I understood it correctly, problems when changing CHOST
> mainly seem to arise because of the way GCC and related basic build utils
> install themselves (using the host triplet as part of their path or
> executable name), leaving wrong / messed up references when changing the
> CHOST.

You're lucky if the compiler fails early this way, so you don't have
the unpleasant experience of sudden runtime breakages ;-p

> For example, as I've said previously, the CHOST value gets passed to
> ./configure as --host for each package that gets build. That would make
> configure try to select a compiler called <CHOST>-gcc in order to compile

... if the toolchain commands aren't passed explicitly ...
(which you *SHOULD* do in any crosscompiling scenario)

> the package, i.e. when CHOST is i486-pc-linux-gnu, a compiler called
> i486-pc-linux-gnu-gcc would be used. Include file directories for glibc
> and / or glibc itself sems to be selected by a similiar mechanism.

AFAIK they're not selected by this parameter (even some illdesigned
configure script might do such crazy things on its own ;-o), but built
into the toolchain commands themselves. You can override/change them,
but then you really should know what you're doing ;-p

> The Gentoo CHOST document that was mentioned says something about nptl not
> being available on i386. If true, and if the host variable generally
> influences the availability of features, things would slowly start to make
> sens as to why the CHOST might be important.

Exactly. nptl vs. linuxthreads is exactly an good example for
incompatible ABIs (while API more or less stays the same).

> On the other hand, I've read
> through some documentation of the GNU C library (glibc) and didn't even
> find anything about nptl not being available on i386, not to mention
> anything else about different features on different subarches.

I don't know much about their internals, but I can easily imagine
that nptl makes heavy use of newer registers which linuxthreads didn't.
For those things you should ask the gcc and libc folks.


cu
--
----------------------------------------------------------------------
Enrico Weigelt, metux IT service -- http://www.metux.de/

phone: +49 36207 519931 email: weigelt@metux.de
mobile: +49 151 27565287 icq: 210169427 skype: nekrad666
----------------------------------------------------------------------
Embedded-Linux / Portierung / Opensource-QM / Verteilte Systeme
----------------------------------------------------------------------


All times are GMT. The time now is 05:57 AM.

VBulletin, Copyright ©2000 - 2014, Jelsoft Enterprises Ltd.
Content Relevant URLs by vBSEO ©2007, Crawlability, Inc.