FAQ Search Today's Posts Mark Forums Read
» Video Reviews

» Linux Archive

Linux-archive is a website aiming to archive linux email lists and to make them easily accessible for linux users/developers.


» Sponsor

» Partners

» Sponsor

Go Back   Linux Archive > Gentoo > Gentoo Alt

 
 
LinkBack Thread Tools
 
Old 12-09-2010, 04:06 PM
Perry Smith
 
Default Adventures with Prefix on AIX 5.3

Hi,
I've been working in and on AIX since 1988 and have built many open source packages. *Recently I wanted a structure to help me out and I discovered Gentoo's Portage and the postfix port of it. *What follows is a small log of what I have done so far hoping it will help others in the future. *I have a few questions at the end.
I now have three levels in my path. *The first is the prefix area e.g. $EPREFIX/bin, the second is a previous area where I put open source packages I'll call $R. *This would be like /usr/local in most machines. *Then the normal /usr/bin directory.
I am roughly following the sets in the instructions for Solaris but I've hit a few very minor bumps in the road.
First, I put these packages, built by "hand" into my $R area:
Python-2.7.1 * * *gcc-4.5.0 * * * * make-3.82 * * * * tar-1.25bash-4.1 * * * * *gettext-0.18.1.1 *mpc-0.8.2 * * * * wget-1.12expat-2.0.1 * * * gmp-5.0.1 * * * * mpfr-2.4.2gawk-3.1.8 * * * *libiconv-1.13.1 * sed-4.2.1
This is gmp, mpfr, and mpc to make GCC along with iconv, expat, and gettext. *After that, I added wget, bash, and make. *I then started to try to do the bootstrap sequence but hit problems with the versions of sed, Python, and gawk so I built those two packages "by hand" in the $R area too. *I have copious notes about how these steps were done but I'll post them somewhere else. *This list doesn't seem quite the place. *
The issue I hit with sed is described here:
http://www.mail-archive.com/hlfs-dev@linuxfromscratch.org/msg01740.html
The issue I hit with gawk is described here:
https://trac.macports.org/ticket/4447
In both cases, I just pulled down the latest version and built them without incident. *Python did not really come close to building in the bootstrap sequence but the latest 2.7.1 builds using an "in tree" build method with the normal configure, make, make install sequence.
I am still progressing through the Solaris bring up instructions. *I'm doing the sequence of: emerge --oneshot <xyz> and things are going ok.
My questions:
1) The version of sed and gawk used in the bootstrap are fairly old. *I assume they are used because they are "stable" but they did not work for me which is why I got the latest version. *I get the idea that Python might fall into the same category. *If the latest version was used, it might built and work. *Should that be changed or is the boot strap process not really something folks want to spend a great deal of time fixing or modifying?
2) The one really weird thing that happened was I had a new version of sed in my $R area and when I tried to do:
emerge --oneshot sed
somehow, it used the sed from AIX and not the one in my $R area -- despite the fact that it was in the path. *This process must somehow modify the PATH briefly. *I guess? *The way I got around this was adding a symbolic link from the $EPREFIX area to the sed in the $R area.
3) My biggest question, before I get too deep is about binutils. *I see it coming up on the todo list. *I'm sure that I want to use AIX's ld(1) and as(1). *Should I not install binutils? *I know this is new territory but thought I would ask for some advice.
Thank you,Perry
 
Old 12-10-2010, 09:46 AM
Michael Haubenwallner
 
Default Adventures with Prefix on AIX 5.3

Hi Perry!

On 12/09/2010 06:06 PM, Perry Smith wrote:
> Hi,
>
> I've been working in and on AIX since 1988 and have built many open source packages.
> Recently I wanted a structure to help me out and I discovered Gentoo's Portage and the postfix port of it.
> What follows is a small log of what I have done so far hoping it will help others in the future.
> I have a few questions at the end.
<snip>

At the moment I'm doing an update on Prefix to work for AIX again,
specifically for AIX 6.1 where I have access to for a few months.
However, the (currently failing) Prefix snapshots run on AIX 5.3.

> I am roughly following the sets in the instructions for Solaris but I've hit a few very minor bumps in the road.

Unfortunately, all the bootstrap guides found for Solaris, Linux, OSX and
others are not really intended to work for AIX.

Instead, please use prefix-launcher[1] for AIX, where release 2.1.0 should
do for AIX 5.3.

Right now I've filled the mediawiki page there [2] with some howto,
any help to improve that is appreciated.

<snip>
> 3) My biggest question, before I get too deep is about binutils.
> I see it coming up on the todo list.
> I'm sure that I want to use AIX's ld(1) and as(1).
> Should I not install binutils?
> I know this is new territory but thought I would ask for some advice.

Currently, Gentoo Prefix uses native ld and as on AIX.
However, GNU binutils recently reclaimed to work on AIX, but I've not tested that yet.

[1] http://prefix-launcher.sourceforge.net/
[2] http://sourceforge.net/apps/mediawiki/prefix-launcher/index.php?title=Main_Page

HTH,
/haubi/
--
Michael Haubenwallner
Gentoo on a different level
 
Old 12-10-2010, 02:34 PM
Perry Smith
 
Default Adventures with Prefix on AIX 5.3

Hi,

Very good! I will look at the two references and let you know how I fair. I hit a stumbling block yesterday using the "Solaris" approach and was about to ask for help but I'll not do that at this point.

As I mentioned, I have done a lot with AIX and I'm really wanting this to work out. Please feel free to ping me about AIX questions and I'll try to help. I don't know much about libtool and autoconf. That is where I get lost.

Perry


On Dec 10, 2010, at 4:46 AM, Michael Haubenwallner wrote:

> Hi Perry!
>
> On 12/09/2010 06:06 PM, Perry Smith wrote:
>> Hi,
>>
>> I've been working in and on AIX since 1988 and have built many open source packages.
>> Recently I wanted a structure to help me out and I discovered Gentoo's Portage and the postfix port of it.
>> What follows is a small log of what I have done so far hoping it will help others in the future.
>> I have a few questions at the end.
> <snip>
>
> At the moment I'm doing an update on Prefix to work for AIX again,
> specifically for AIX 6.1 where I have access to for a few months.
> However, the (currently failing) Prefix snapshots run on AIX 5.3.
>
>> I am roughly following the sets in the instructions for Solaris but I've hit a few very minor bumps in the road.
>
> Unfortunately, all the bootstrap guides found for Solaris, Linux, OSX and
> others are not really intended to work for AIX.
>
> Instead, please use prefix-launcher[1] for AIX, where release 2.1.0 should
> do for AIX 5.3.
>
> Right now I've filled the mediawiki page there [2] with some howto,
> any help to improve that is appreciated.
>
> <snip>
>> 3) My biggest question, before I get too deep is about binutils.
>> I see it coming up on the todo list.
>> I'm sure that I want to use AIX's ld(1) and as(1).
>> Should I not install binutils?
>> I know this is new territory but thought I would ask for some advice.
>
> Currently, Gentoo Prefix uses native ld and as on AIX.
> However, GNU binutils recently reclaimed to work on AIX, but I've not tested that yet.
>
> [1] http://prefix-launcher.sourceforge.net/
> [2] http://sourceforge.net/apps/mediawiki/prefix-launcher/index.php?title=Main_Page
>
> HTH,
> /haubi/
> --
> Michael Haubenwallner
> Gentoo on a different level
 
Old 12-10-2010, 03:02 PM
Michael Haubenwallner
 
Default Adventures with Prefix on AIX 5.3

On 12/10/2010 04:34 PM, Perry Smith wrote:
> As I mentioned, I have done a lot with AIX and I'm really wanting this to work out.
> Please feel free to ping me about AIX questions and I'll try to help.

There is one longstanding question I did not manage yet to finally find a good decision upon,
which gives a result being easy-understandable on both the filesystem and the shared objects,
as well as easy-useable for an ELF-originated package manager like portage & the ebuilds:

How to implement the so-called "SONAME" feature with AIX shared objects (aka. shared libraries)

Eventually you have some thoughts after reading these:
http://bugs.gentoo.org/show_bug.cgi?id=213277
http://dev.gentoo.org/~haubi/aixrtl.txt

> I don't know much about libtool and autoconf.

Additionally, it is necessary to discuss this thing with libtool people.
I've tried this before already, but will have to do again, with more experience now.

Thank you!

/haubi/
--
Michael Haubenwallner
Gentoo on a different level
 
Old 12-10-2010, 11:41 PM
Perry Smith
 
Default Adventures with Prefix on AIX 5.3

Hi,

I'm still digesting this but I wanted to mention something that I have not seen mentioned in all the text I've read so far which is "64 bit".

When I built my gcc this last time, I built gcc twice and the dependent libraries four times each. The process breaks down into this.

For both passes, I do "out of tree" builds and build each library in 32 bit mode and again in 64 bit mode. I then rename the 32 bit shared object to e.g. libmpfr32.so.1, the 64 bit object to libmpfr64.so.1, and then create an archive called libmpfr.a.

On the first pass, I specify where each library can be found. e.g. when I build mpfr, I told it that gmp is /down/this/path via the ./configure option. I built all libraries and gcc and "installed them".

On the second pass (using the gcc and libs from the first pass), I set LIBPATH to /down/this/path:/lib but did not give any ./configure options to find the libraries. The build process finds and can use the libraries using LIBPATH. Again, I built all the libraries in both ABIs and finally built gcc.

The final link of gcc it seems added in an absolute path for gettext but generally the objects in the libraries do not have any absolute paths except for the PATH[0] in the header. i.e. dump -H mpc32.so.2 for the "Import File Strings" is:

***Import File Strings***
INDEX PATH BASE MEMBER
0 /a/b/c:/lib
1 libmpfr.a libmpfr32.so.1
2 libc.a shr.o
3 libgmp.a libgmp32.so.10

So, I could move my libmpfr.a somewhere else, set my LIBPATH, and the load process would find it and use it. To me, this is nice. I can't say its a hugh advantage but one that would come in useful from time to time.

I can now create 32 bit and 64 bit executables very easily. i.e.

gcc -o x x.c

creates a 32 bit executable while

gcc -maix64 -o x x.c

creates a 64 bit executable. The compile, link, and execute of either ABI doesn't require any gymnastics (LIBPATH is not set.) And, as an experiment, I moved my libmpfr.a library, pointed to it via LIBPATH, and could compile, link, and execute as well. So if things are in their normal position, no LIBPATH is needed. If they are moved elsewhere, setting LIBPATH allows them to be found and used.

Again, to repeat, being able to move the libraries isn't my main objective. My main point here is that every library, under my blue sky dream, would be compiled with both ABIs so that users could build complicated applications in 64 bit mode with very little trouble.

Let me get back to reading...

Thank you,
Perry

On Dec 10, 2010, at 10:02 AM, Michael Haubenwallner wrote:

>
>
> On 12/10/2010 04:34 PM, Perry Smith wrote:
>> As I mentioned, I have done a lot with AIX and I'm really wanting this to work out.
>> Please feel free to ping me about AIX questions and I'll try to help.
>
> There is one longstanding question I did not manage yet to finally find a good decision upon,
> which gives a result being easy-understandable on both the filesystem and the shared objects,
> as well as easy-useable for an ELF-originated package manager like portage & the ebuilds:
>
> How to implement the so-called "SONAME" feature with AIX shared objects (aka. shared libraries)
>
> Eventually you have some thoughts after reading these:
> http://bugs.gentoo.org/show_bug.cgi?id=213277
> http://dev.gentoo.org/~haubi/aixrtl.txt
>
>> I don't know much about libtool and autoconf.
>
> Additionally, it is necessary to discuss this thing with libtool people.
> I've tried this before already, but will have to do again, with more experience now.
>
> Thank you!
>
> /haubi/
> --
> Michael Haubenwallner
> Gentoo on a different level
 
Old 12-13-2010, 08:45 AM
Michael Haubenwallner
 
Default Adventures with Prefix on AIX 5.3

Hi Perry,

On 12/11/2010 01:41 AM, Perry Smith wrote:
> Hi,
>
> I'm still digesting this but I wanted to mention something that I have not seen mentioned in all the text I've read so far which is "64 bit".

Exactly - for now, there is only 32bit support in Prefix for AIX (as well as HP-UX),
as I do not have any /real/ need for 64bit. And iff you don't write database engines,
I'm pretty sure you don't have a /real/ need for 64bit too.
However - you're welcome to do the ppc64-aix keywords

> When I built my gcc this last time, I built gcc twice and the dependent libraries four times each.
> The process breaks down into this.

While gcc supports multilib on AIX out of the box, we have disabled any multilib
bit in Prefix in general. Instead, we support different Prefix installations
with different CHOST and KEYWORDs only.

> For both passes, I do "out of tree" builds and build each library in 32 bit mode and again in 64 bit mode.
> I then rename the 32 bit shared object to e.g. libmpfr32.so.1, the 64 bit object to libmpfr64.so.1, and then create an archive called libmpfr.a.

A Library called 'libmpfr.a' without any version number in plain filename
is calling for troubles[1].

<snip>
> And, as an experiment, I moved my libmpfr.a library, pointed to it via LIBPATH, and could compile, link, and execute as well.
> So if things are in their normal position, no LIBPATH is needed.
> If they are moved elsewhere, setting LIBPATH allows them to be found and used.

As you said - using LIBPATH is an /experiment/.
Anything like LIBPATH never is a solution, but a call for troubles.
So we don't allow LIBPATH/LD_LIBRARY_PATH/etc. in Prefix.

> Again, to repeat, being able to move the libraries isn't my main objective.

Good to hear

> My main point here is that every library, under my blue sky dream, would be compiled with both ABIs
> so that users could build complicated applications in 64 bit mode with very little trouble.

This is not supported by upstream portage yet. However, there are people
working in that direction[2]. But this does not necessarily mean we will
support that in Prefix whenever done upstream once.

[1] http://bugs.gentoo.org/show_bug.cgi?id=213277#c11
[2] http://archives.gentoo.org/gentoo-dev/msg_46b23feaa62cfbc11abca5d9613fcd05.xml

/haubi/
--
Michael Haubenwallner
Gentoo on a different level
 
Old 12-13-2010, 11:08 PM
Perry Smith
 
Default Adventures with Prefix on AIX 5.3

On Dec 13, 2010, at 3:45 AM, Michael Haubenwallner wrote:

> On 12/11/2010 01:41 AM, Perry Smith wrote:
>>
>> For both passes, I do "out of tree" builds and build each library in 32 bit mode and again in 64 bit mode.
>> I then rename the 32 bit shared object to e.g. libmpfr32.so.1, the 64 bit object to libmpfr64.so.1, and then create an archive called libmpfr.a.
>
> A Library called 'libmpfr.a' without any version number in plain filename
> is calling for troubles[1].
>
> [1] http://bugs.gentoo.org/show_bug.cgi?id=213277#c11

I can recreate this doing:

LIBPATH=/lib gcc -o x x.c && ./x
exec(): 0509-036 Cannot load program gcc because of the following errors:
0509-022 Cannot load module /gsa/ausgsa/projects/r/ruby/lib/libintl.a(libintl.so.8).
0509-150 Dependent module /lib/libiconv.a(libiconv.so.2) could not be loaded.
0509-152 Member libiconv.so.2 is not found in archive
0509-022 Cannot load module gcc.
0509-150 Dependent module /gsa/ausgsa/projects/r/ruby/lib/libintl.a(libintl.so.8) could not be loaded.
0509-022 Cannot load module .

I went back and rebuilt my libintl.a using absolute paths so the header looks like:

***Import File Strings***
INDEX PATH BASE MEMBER
0 /gsa/ausgsa/projects/r/ruby/lib:/usr/local/lib/gcc/powerpc-ibm-aix5.3.0.0/4.3.1:/usr/local/lib/gcc/powerpc-ibm-aix5.3.0.0/4.3.1/../../..:/usr/lib:/lib
1 libc.a shr.o
2 libpthread.a shr_xpg5.o
3 /gsa/ausgsa/projects/r/ruby/lib libiconv.a libiconv.so.2

and it still fails. Are you sure if you build with import files it is going to solve the problem? Seems like it would not but I'm curious what you have in mind.

Note that this is only going to be an issue for libraries that collide with the same base name in /lib. If I move /lib/libiconv.a out of the way, then the loader finds the correct libiconv.a.

The other question I have is if this is a show stopper for you? I would argue that what Java is doing is impolite (at least).

Perry
 
Old 12-13-2010, 11:33 PM
Perry Smith
 
Default Adventures with Prefix on AIX 5.3

> Drawback 1:
> It is impossible to specify undefined symbols in an import file for the
> shared object to allow for subsequent link steps keeping those symbols in the
> created shared object or executables, to be resolved by runtime linking when
> the executable is run.
> But as shared libraries with unresolved symbols are broken "by design" anyway,
> and real runtime dependency management isn't going to work with broken shared
> libraries, this becomes a non-problem as more and more packages get this right.
Does #! .. solve this? (from IBM pubs on the ld command) I'm not 100% clear when libraries with unresolved symbols are used.

#! .. (Two dots) Use this name to list symbols that will be resolved by the run-time linker. Use this file name to create shared objects that will be used by programs making use of the run-time linker. If you use a module that imports symbols from .. in a program that was not linked with the rtllib option, symbols will be unresolved, and references to such symbols will result in undefined behavior.
 
Old 12-14-2010, 07:20 AM
Michael Haubenwallner
 
Default Adventures with Prefix on AIX 5.3

On 12/14/10 01:33, Perry Smith wrote:
>> Drawback 1:
>> It is impossible to specify undefined symbols in an import file for the
>> shared object to allow for subsequent link steps keeping those symbols in the
>> created shared object or executables, to be resolved by runtime linking when
>> the executable is run.
>> But as shared libraries with unresolved symbols are broken "by design" anyway,
>> and real runtime dependency management isn't going to work with broken shared
>> libraries, this becomes a non-problem as more and more packages get this right.
> Does #! .. solve this? (from IBM pubs on the ld command) I'm not 100% clear when libraries with unresolved symbols are used.

Unfortunately not. A shared object name ".." seems to not have any special meaning
to the linker/binder, but the runtime loader only.
The problem is that it is a shared object referencing symbols in "..", while the
binary (executable or shared object) currently being linked does not need them,
so the binder does not see those symbols as undefined.
Import files are to inform the binder where to find symbols when they are needed,
but they aren't needed by the current binary.
It is even more complex: When one shared object references "..", the binder would
have to add a reference to a second shared object, when the symbols used from ".."
aren't available in some static object.

So there is one indirection too much for import files here.

> #! .. (Two dots) Use this name to list symbols that will be resolved by the run-time linker.
> Use this file name to create shared objects that will be used by programs making use of the run-time linker.
> If you use a module that imports symbols from .. in a program that was not linked with the rtllib option,
> symbols will be unresolved, and references to such symbols will result in undefined behavior.

I've stared at and tried this for weeks without success in my case...

/haubi/
--
Michael Haubenwallner
Gentoo on a different level
 
Old 12-14-2010, 07:22 AM
Michael Haubenwallner
 
Default Adventures with Prefix on AIX 5.3

(now with CC-ing list)

On 12/14/10 01:08, Perry Smith wrote:
>> A Library called 'libmpfr.a' without any version number in plain filename
>> is calling for troubles[1].
>>
>> [1] http://bugs.gentoo.org/show_bug.cgi?id=213277#c11
>
> I can recreate this doing:
>
> LIBPATH=/lib gcc -o x x.c && ./x
> exec(): 0509-036 Cannot load program gcc because of the following errors:
> 0509-022 Cannot load module /gsa/ausgsa/projects/r/ruby/lib/libintl.a(libintl.so.8).
> 0509-150 Dependent module /lib/libiconv.a(libiconv.so.2) could not be loaded.
> 0509-152 Member libiconv.so.2 is not found in archive

> I went back and rebuilt my libintl.a using absolute paths so the header looks like:

> 3 /gsa/ausgsa/projects/r/ruby/lib libiconv.a libiconv.so.2
>
> and it still fails.

This is interesting - I've not recognized that LIBPATH overrides absolute path too.

> Are you sure if you build with import files it is going to solve the problem?
> Seems like it would not but I'm curious what you have in mind.

What I'm doing with import files is to rename the /filename/ 'libiconv.a'
to something that contains the version number, so the filename-search doesn't
go for 'libiconv.a' but something like 'libiconv.so.2'. It is irrelevant if
this is an archive with an irrelevant member name or not - the /filename/ is
important to contain the version number when searched at runtime, while linking
has to be done without the version number.

> Note that this is only going to be an issue for libraries that collide with the same base name in /lib.
> If I move /lib/libiconv.a out of the way, then the loader finds the correct libiconv.a.

This is most often true, but it is a problem when LIBPATH is pointing to some
directory containing a same unversioned filename (maybe another Prefix) from a
different version. However, using LIBPATH is discuraged in general, and I have
to prevent from breaking with LIBPATH=/lib only.

> The other question I have is if this is a show stopper for you?

Yes, we do have the use-case shown in the bug.

> I would argue that what Java is doing is impolite (at least).

Looks like, but maybe they have some reason to do so.

/haubi/
--
Michael Haubenwallner
Gentoo on a different level
 

Thread Tools




All times are GMT. The time now is 05:04 PM.

VBulletin, Copyright ©2000 - 2014, Jelsoft Enterprises Ltd.
Content Relevant URLs by vBSEO ©2007, Crawlability, Inc.
Copyright 2007 - 2008, www.linux-archive.org