FAQ Search Today's Posts Mark Forums Read
» Video Reviews

» Linux Archive

Linux-archive is a website aiming to archive linux email lists and to make them easily accessible for linux users/developers.


» Sponsor

» Partners

» Sponsor

Go Back   Linux Archive > Redhat > Fedora Development

 
 
LinkBack Thread Tools
 
Old 07-27-2011, 03:48 PM
Neal Becker
 
Default howto not strip .so on install

Trying to help packaging dmtcp. There are 2 shared libs installed. They are
stripped by the rpm install, and then fail when attempting to dlopen them (or 1
of them).

1. Is that normal behaviour?
2. How can I avoid it?

--
devel mailing list
devel@lists.fedoraproject.org
https://admin.fedoraproject.org/mailman/listinfo/devel
 
Old 07-27-2011, 03:54 PM
Richard Shaw
 
Default howto not strip .so on install

On Wed, Jul 27, 2011 at 10:48 AM, Neal Becker <ndbecker2@gmail.com> wrote:
> Trying to help packaging dmtcp. *There are 2 shared libs installed. *They are
> stripped by the rpm install, and then fail when attempting to dlopen them (or 1
> of them).
>
> 1. Is that normal behaviour?
> 2. How can I avoid it?

Maybe a more experienced developer/packager knows exactly what you're
talking about but I need more information...

I know I recently ran into an issue where a small project was using a
simple makefile (no configure, autotools, scons, or cmake) and it was
stripping the libraries/binaries. I ended up fixing it by deleting the
line in the makefile from the spec file in the %prep section, i.e.:

sed -i '/strip/d' Makefile

Richard
--
devel mailing list
devel@lists.fedoraproject.org
https://admin.fedoraproject.org/mailman/listinfo/devel
 
Old 07-27-2011, 03:57 PM
Adam Jackson
 
Default howto not strip .so on install

On Wed, 2011-07-27 at 11:48 -0400, Neal Becker wrote:
> Trying to help packaging dmtcp. There are 2 shared libs installed. They are
> stripped by the rpm install, and then fail when attempting to dlopen them (or 1
> of them).
>
> 1. Is that normal behaviour?

No, the strip performed by the rpm install should not break normal usage
of dlopen or dlsym. X's input and video drivers, for example, have no
problem with it.

- ajax
--
devel mailing list
devel@lists.fedoraproject.org
https://admin.fedoraproject.org/mailman/listinfo/devel
 
Old 07-27-2011, 04:18 PM
Jerry James
 
Default howto not strip .so on install

On Wed, Jul 27, 2011 at 9:48 AM, Neal Becker <ndbecker2@gmail.com> wrote:
> Trying to help packaging dmtcp. *There are 2 shared libs installed. *They are
> stripped by the rpm install, and then fail when attempting to dlopen them (or 1
> of them).

Fail how? What's the error?
--
Jerry James
http://www.jamezone.org/
--
devel mailing list
devel@lists.fedoraproject.org
https://admin.fedoraproject.org/mailman/listinfo/devel
 
Old 07-27-2011, 05:34 PM
Neal Becker
 
Default howto not strip .so on install

Jerry James wrote:

> On Wed, Jul 27, 2011 at 9:48 AM, Neal Becker <ndbecker2@gmail.com> wrote:
>> Trying to help packaging dmtcp. There are 2 shared libs installed. They are
>> stripped by the rpm install, and then fail when attempting to dlopen them (or
>> 1 of them).
>
> Fail how? What's the error?
1. Get srpm here
http://www.ccs.neu.edu/home/kapil/fedora_rpms/dmtcp-1.2.3+svn1214-1.1.src.rpm

2. rpmbuild -ba dmtcp.spec

3. sudo rpm -U *dmtcp* (install all the rpms you just built)

4. Test:

dmtcp_checkpoint python
DMTCP/MTCP Copyright (C) 2006-2010 Jason Ansel, Michael Rieker,
Kapil Arya, and Gene Cooperman
This program comes with ABSOLUTELY NO WARRANTY.
This is free software, and you are welcome to redistribute it
under certain conditions; see COPYING file for details.
(Use flag "-q" to hide this message.)

dmtcp_coordinator starting...
Port: 7779
Checkpoint Interval: disabled (checkpoint manually instead)
Exit on last client: 1
Backgrounding...
[18446] NOTE at connectionmanager.cpp:623 in handlePreExistingFd; REASON='found
pre-existing socket... will not be restored'
fd = 10
device = pipe:[19269]
[18446] ERROR at mtcpinterface.cpp:91 in find_and_open_mtcp_so;
REASON='JASSERT(handle != NULL) failed'
mtcpso = /usr/lib64/dmtcp/libmtcp.so
dlerror() = /usr/lib64/dmtcp/libmtcp.so: undefined symbol:
mtcp_restore_start
Message: failed to load libmtcp.so
python (18446): Terminating...

5. Goto the rpmbuild/BUILD/dmtcpxxx directory, and reinstall the shared libs

sudo make install

6. Now it does not fail.

7. Verify that the only difference is replacing the shared libs installed via
rpm vs. the same shared libs installed without rpm.

So something that rpm does to the shared libs (in /usr/lib64/dmtcp) is breaking
things. The only thing I can think of is strip.

8. Notice the error:

dlerror() = /usr/lib64/dmtcp/libmtcp.so: undefined symbol:
mtcp_restore_start

But it is defined!
nm -D /usr/lib64/dmtcp/libmtcp.so | grep mtcp_restore_start
000000000000911c T mtcp_restore_start

That was the non-working version of libmtcp.so, which has been stripped.



--
devel mailing list
devel@lists.fedoraproject.org
https://admin.fedoraproject.org/mailman/listinfo/devel
 
Old 07-27-2011, 06:06 PM
Jerry James
 
Default howto not strip .so on install

On Wed, Jul 27, 2011 at 11:34 AM, Neal Becker <ndbecker2@gmail.com> wrote:
> 7. Verify that the only difference is replacing the shared libs installed via
> rpm vs. the same shared libs installed without rpm.
>
> So something that rpm does to the shared libs (in /usr/lib64/dmtcp) is breaking
> things. *The only thing I can think of is strip.
>
> 8. Notice the error:
>
> * * dlerror() = /usr/lib64/dmtcp/libmtcp.so: undefined symbol:
> mtcp_restore_start
>
> But it is defined!
> nm -D /usr/lib64/dmtcp/libmtcp.so | grep mtcp_restore_start
> 000000000000911c T mtcp_restore_start
>
> That was the non-working version of libmtcp.so, which has been stripped.

It isn't strip. If I run strip on the version of libmtcp.so left in
the build dir and copy it over /usr/lib64/dmtcp/libmtcp.so, then it
also works. But that version is different from the one installed by
rpm. Watch this:

$ ldd -r /usr/lib64/dmtcp/libmtcp.so
linux-vdso.so.1 => (0x00007fff7d7c3000)
libdl.so.2 => /lib64/libdl.so.2 (0x00007f913f396000)
libpthread.so.0 => /lib64/libpthread.so.0 (0x00007f913f17b000)
libgcc_s.so.1 => /lib64/libgcc_s.so.1 (0x00007f913ef65000)
libc.so.6 => /lib64/libc.so.6 (0x00007f913ebcc000)
/lib64/ld-linux-x86-64.so.2 (0x0000003520600000)
undefined symbol: mtcp_restore_start (/usr/lib64/dmtcp/libmtcp.so)
undefined symbol: dmtcp_exists (/usr/lib64/dmtcp/libmtcp.so)
undefined symbol: mtcp_restore_argv_start_addr (/usr/lib64/dmtcp/libmtcp.so)
undefined symbol: mtcp_sys_errno (/usr/lib64/dmtcp/libmtcp.so)
undefined symbol: mtcp_get_thread_sysinfo (/usr/lib64/dmtcp/libmtcp.so)
undefined symbol: mtcp_no (/usr/lib64/dmtcp/libmtcp.so)
undefined symbol: mtcp_dump_tls (/usr/lib64/dmtcp/libmtcp.so)
undefined symbol: mtcp_readdec (/usr/lib64/dmtcp/libmtcp.so)
undefined symbol: test_and_prepare_for_forked_ckpt (/usr/lib64/dmtcp/libmtcp.so)
undefined symbol: write_ckpt_to_file (/usr/lib64/dmtcp/libmtcp.so)
undefined symbol: mtcp_readchar (/usr/lib64/dmtcp/libmtcp.so)
$ ldd -r /tmp/libmtcp-stripped.so
linux-vdso.so.1 => (0x00007fffcd3ff000)
libdl.so.2 => /lib64/libdl.so.2 (0x00007f02082b3000)
libpthread.so.0 => /lib64/libpthread.so.0 (0x00007f0208098000)
libgcc_s.so.1 => /lib64/libgcc_s.so.1 (0x00007f0207e82000)
libc.so.6 => /lib64/libc.so.6 (0x00007f0207ae9000)
/lib64/ld-linux-x86-64.so.2 (0x0000003520600000)

The program /usr/lib/rpm/debugedit is used to extract debug
information from ELF objects when building RPMs. It looks like it is
damaging the shared object somehow.
--
Jerry James
http://www.jamezone.org/
--
devel mailing list
devel@lists.fedoraproject.org
https://admin.fedoraproject.org/mailman/listinfo/devel
 
Old 07-27-2011, 06:14 PM
Jerry James
 
Default howto not strip .so on install

On Wed, Jul 27, 2011 at 12:06 PM, Jerry James <loganjerry@gmail.com> wrote:
> It isn't strip. *If I run strip on the version of libmtcp.so left in
> the build dir and copy it over /usr/lib64/dmtcp/libmtcp.so, then it
> also works. *But that version is different from the one installed by
> rpm. *Watch this:
>
> $ ldd -r /usr/lib64/dmtcp/libmtcp.so
> * * * *linux-vdso.so.1 => *(0x00007fff7d7c3000)
> * * * *libdl.so.2 => /lib64/libdl.so.2 (0x00007f913f396000)
> * * * *libpthread.so.0 => /lib64/libpthread.so.0 (0x00007f913f17b000)
> * * * *libgcc_s.so.1 => /lib64/libgcc_s.so.1 (0x00007f913ef65000)
> * * * *libc.so.6 => /lib64/libc.so.6 (0x00007f913ebcc000)
> * * * */lib64/ld-linux-x86-64.so.2 (0x0000003520600000)
> undefined symbol: mtcp_restore_start * *(/usr/lib64/dmtcp/libmtcp.so)
> undefined symbol: dmtcp_exists *(/usr/lib64/dmtcp/libmtcp.so)
> undefined symbol: mtcp_restore_argv_start_addr *(/usr/lib64/dmtcp/libmtcp.so)
> undefined symbol: mtcp_sys_errno * * * *(/usr/lib64/dmtcp/libmtcp.so)
> undefined symbol: mtcp_get_thread_sysinfo * * * (/usr/lib64/dmtcp/libmtcp.so)
> undefined symbol: mtcp_no * * * (/usr/lib64/dmtcp/libmtcp.so)
> undefined symbol: mtcp_dump_tls (/usr/lib64/dmtcp/libmtcp.so)
> undefined symbol: mtcp_readdec *(/usr/lib64/dmtcp/libmtcp.so)
> undefined symbol: test_and_prepare_for_forked_ckpt * * *(/usr/lib64/dmtcp/libmtcp.so)
> undefined symbol: write_ckpt_to_file * *(/usr/lib64/dmtcp/libmtcp.so)
> undefined symbol: mtcp_readchar (/usr/lib64/dmtcp/libmtcp.so)
> $ ldd -r /tmp/libmtcp-stripped.so
> * * * *linux-vdso.so.1 => *(0x00007fffcd3ff000)
> * * * *libdl.so.2 => /lib64/libdl.so.2 (0x00007f02082b3000)
> * * * *libpthread.so.0 => /lib64/libpthread.so.0 (0x00007f0208098000)
> * * * *libgcc_s.so.1 => /lib64/libgcc_s.so.1 (0x00007f0207e82000)
> * * * *libc.so.6 => /lib64/libc.so.6 (0x00007f0207ae9000)
> * * * */lib64/ld-linux-x86-64.so.2 (0x0000003520600000)
>
> The program /usr/lib/rpm/debugedit is used to extract debug
> information from ELF objects when building RPMs. *It looks like it is
> damaging the shared object somehow.

Hmmmm, the dmtcp project is patching the default linker script:

make[1]: Entering directory
`/home/jamesjer/rpmbuild/BUILD/dmtcp-1.2.3+svn1214/mtcp'
rm -f mtcp.t
ld -shared --verbose > mtcp.t
sed -i -e '1,/========================/ d' mtcp.t
sed -i -e '/========================/,$ d' mtcp.t
rm -f mtcp.t-fail
if test x86_64 = x86_64; then
if patch mtcp.t mtcp.t.patch-x86_64; then
:;
else
mv mtcp.t mtcp.t-fail; false;
fi
else
if patch mtcp.t mtcp.t.patch-i386; then
:;
else
mv mtcp.t mtcp.t-fail; false;
fi
fi
patching file mtcp.t
Hunk #1 succeeded at 6 with fuzz 2.
Hunk #2 succeeded at 189 (offset 14 lines).
...
gcc -shared -fPIC -T mtcp.t -Wl,-Map,mtcp.map -o libmtcp.so
mtcp.o mtcp_restart_nolibc.o mtcp_maybebpt.o mtcp_printf.o
mtcp_util.o mtcp_safemmap.o mtcp_safe_open.o mtcp_state.o
mtcp_check_vdso.o mtcp_sigaction.o mtcp_ptrace.o mtcp_fastckpt.o -ldl
-lpthread

Here's the diff against the default linker script:

--- linker.default 2011-07-27 12:09:30.492379093 -0600
+++ mtcp.t 2011-07-27 11:48:10.803379078 -0600
@@ -1,12 +1,3 @@
-GNU ld version 2.21.51.0.6-6.fc15 20110118
- Supported emulations:
- elf_x86_64
- elf32_x86_64
- elf_i386
- i386linux
- elf_l1om
-using internal linker script:
-==================================================
/* Script for --shared -z combreloc: shared library, combine & sort relocs */
OUTPUT_FORMAT("elf64-x86-64", "elf64-x86-64",
"elf64-x86-64")
@@ -15,6 +6,9 @@
SEARCH_DIR("/usr/x86_64-redhat-linux/lib64");
SEARCH_DIR("/usr/local/lib64"); SEARCH_DIR("/lib64");
SEARCH_DIR("/usr/lib64"); SEARCH_DIR("/usr/x86_64-redhat-linux/lib");
SEARCH_DIR("/usr/lib64"); SEARCH_DIR("/usr/local/lib");
SEARCH_DIR("/lib"); SEARCH_DIR("/usr/lib");
SECTIONS
{
+ .the.begin : {
+ *(__shareable_begin)
+ }
/* Read-only sections, merged into text segment: */
. = SEGMENT_START("text-segment", 0) + SIZEOF_HEADERS;
.note.gnu.build-id : { *(.note.gnu.build-id) }
@@ -195,6 +189,9 @@
*(.ldata .ldata.* .gnu.linkonce.l.*)
. = ALIGN(. != 0 ? 64 / 8 : 1);
}
+ .the.end : {
+ *(__shareable_end)
+ }
. = ALIGN(64 / 8);
_end = .; PROVIDE (end = .);
. = DATA_SEGMENT_END (.);
@@ -239,4 +236,3 @@
}


-==================================================


That may or may not have something to do with your problem, but it
probably bears looking into.
--
Jerry James
http://www.jamezone.org/
--
devel mailing list
devel@lists.fedoraproject.org
https://admin.fedoraproject.org/mailman/listinfo/devel
 
Old 07-27-2011, 06:23 PM
Neal Becker
 
Default howto not strip .so on install

>Jerry James wrote:

> On Wed, Jul 27, 2011 at 12:06 PM, Jerry James <loganjerry@gmail.com> wrote:
>> It isn't strip. If I run strip on the version of libmtcp.so left in
>> the build dir and copy it over /usr/lib64/dmtcp/libmtcp.so, then it
>> also works. But that version is different from the one installed by
>> rpm. Watch this:
>>
>> $ ldd -r /usr/lib64/dmtcp/libmtcp.so
>> linux-vdso.so.1 => (0x00007fff7d7c3000)
>> libdl.so.2 => /lib64/libdl.so.2 (0x00007f913f396000)
>> libpthread.so.0 => /lib64/libpthread.so.0 (0x00007f913f17b000)
>> libgcc_s.so.1 => /lib64/libgcc_s.so.1 (0x00007f913ef65000)
>> libc.so.6 => /lib64/libc.so.6 (0x00007f913ebcc000)
>> /lib64/ld-linux-x86-64.so.2 (0x0000003520600000)
>> undefined symbol: mtcp_restore_start (/usr/lib64/dmtcp/libmtcp.so)
>> undefined symbol: dmtcp_exists (/usr/lib64/dmtcp/libmtcp.so)
>> undefined symbol: mtcp_restore_argv_start_addr (/usr/lib64/dmtcp/libmtcp.so)
>> undefined symbol: mtcp_sys_errno (/usr/lib64/dmtcp/libmtcp.so)
>> undefined symbol: mtcp_get_thread_sysinfo (/usr/lib64/dmtcp/libmtcp.so)
>> undefined symbol: mtcp_no (/usr/lib64/dmtcp/libmtcp.so)
>> undefined symbol: mtcp_dump_tls (/usr/lib64/dmtcp/libmtcp.so)
>> undefined symbol: mtcp_readdec (/usr/lib64/dmtcp/libmtcp.so)
>> undefined symbol: test_and_prepare_for_forked_ckpt
>> (/usr/lib64/dmtcp/libmtcp.so) undefined symbol: write_ckpt_to_file
>> (/usr/lib64/dmtcp/libmtcp.so) undefined symbol: mtcp_readchar
>> (/usr/lib64/dmtcp/libmtcp.so) $ ldd -r /tmp/libmtcp-stripped.so
>> linux-vdso.so.1 => (0x00007fffcd3ff000)
>> libdl.so.2 => /lib64/libdl.so.2 (0x00007f02082b3000)
>> libpthread.so.0 => /lib64/libpthread.so.0 (0x00007f0208098000)
>> libgcc_s.so.1 => /lib64/libgcc_s.so.1 (0x00007f0207e82000)
>> libc.so.6 => /lib64/libc.so.6 (0x00007f0207ae9000)
>> /lib64/ld-linux-x86-64.so.2 (0x0000003520600000)
>>
>> The program /usr/lib/rpm/debugedit is used to extract debug
>> information from ELF objects when building RPMs. It looks like it is
>> damaging the shared object somehow.
>
> Hmmmm, the dmtcp project is patching the default linker script:
>
> make[1]: Entering directory
> `/home/jamesjer/rpmbuild/BUILD/dmtcp-1.2.3+svn1214/mtcp'
> rm -f mtcp.t
> ld -shared --verbose > mtcp.t
> sed -i -e '1,/========================/ d' mtcp.t
> sed -i -e '/========================/,$ d' mtcp.t
> rm -f mtcp.t-fail
> if test x86_64 = x86_64; then
> if patch mtcp.t mtcp.t.patch-x86_64; then
> :;
> else
> mv mtcp.t mtcp.t-fail; false;
> fi
> else
> if patch mtcp.t mtcp.t.patch-i386; then
> :;
> else
> mv mtcp.t mtcp.t-fail; false;
> fi
> fi
> patching file mtcp.t
> Hunk #1 succeeded at 6 with fuzz 2.
> Hunk #2 succeeded at 189 (offset 14 lines).
> ...
> gcc -shared -fPIC -T mtcp.t -Wl,-Map,mtcp.map -o libmtcp.so
> mtcp.o mtcp_restart_nolibc.o mtcp_maybebpt.o mtcp_printf.o
> mtcp_util.o mtcp_safemmap.o mtcp_safe_open.o mtcp_state.o
> mtcp_check_vdso.o mtcp_sigaction.o mtcp_ptrace.o mtcp_fastckpt.o -ldl
> -lpthread
>
> Here's the diff against the default linker script:
>
> --- linker.default 2011-07-27 12:09:30.492379093 -0600
> +++ mtcp.t 2011-07-27 11:48:10.803379078 -0600
> @@ -1,12 +1,3 @@
> -GNU ld version 2.21.51.0.6-6.fc15 20110118
> - Supported emulations:
> - elf_x86_64
> - elf32_x86_64
> - elf_i386
> - i386linux
> - elf_l1om
> -using internal linker script:
> -==================================================
> /* Script for --shared -z combreloc: shared library, combine & sort relocs */
> OUTPUT_FORMAT("elf64-x86-64", "elf64-x86-64",
> "elf64-x86-64")
> @@ -15,6 +6,9 @@
> SEARCH_DIR("/usr/x86_64-redhat-linux/lib64");
> SEARCH_DIR("/usr/local/lib64"); SEARCH_DIR("/lib64");
> SEARCH_DIR("/usr/lib64"); SEARCH_DIR("/usr/x86_64-redhat-linux/lib");
> SEARCH_DIR("/usr/lib64"); SEARCH_DIR("/usr/local/lib");
> SEARCH_DIR("/lib"); SEARCH_DIR("/usr/lib");
> SECTIONS
> {
> + .the.begin : {
> + *(__shareable_begin)
> + }
> /* Read-only sections, merged into text segment: */
> . = SEGMENT_START("text-segment", 0) + SIZEOF_HEADERS;
> .note.gnu.build-id : { *(.note.gnu.build-id) }
> @@ -195,6 +189,9 @@
> *(.ldata .ldata.* .gnu.linkonce.l.*)
> . = ALIGN(. != 0 ? 64 / 8 : 1);
> }
> + .the.end : {
> + *(__shareable_end)
> + }
> . = ALIGN(64 / 8);
> _end = .; PROVIDE (end = .);
> . = DATA_SEGMENT_END (.);
> @@ -239,4 +236,3 @@
> }
>
>
> -==================================================
>
>
> That may or may not have something to do with your problem, but it
> probably bears looking into.

Sounds like the best course of action is to just disable debugedit?
1. Would this be acceptable for a fedora package?
2. Is it possible to disable debugedit?

--
devel mailing list
devel@lists.fedoraproject.org
https://admin.fedoraproject.org/mailman/listinfo/devel
 
Old 07-27-2011, 08:10 PM
Jerry James
 
Default howto not strip .so on install

On Wed, Jul 27, 2011 at 12:23 PM, Neal Becker <ndbecker2@gmail.com> wrote:
> Sounds like the best course of action is to just disable debugedit?
> 1. Would this be acceptable for a fedora package?
> 2. Is it possible to *disable debugedit?

I don't think disabling debugedit is necessary. I think I see the
problem. Caveat: I don't REALLY know what I'm talking about. You
should get someone who understands the linker to help out.

Upstream is adding new sections named .the.begin and .the.end,
containing __shareable_begin and __shareable_end, respectively, in
order to bracket the range of addresses they need to worry about.
However, their altered linker script does not put these new sections
in any particular segment, and since ld doesn't recognize the section
names, they aren't added to any of the default segments, so they
aren't marked as either loadable or allocatable. That means they can
be garbage collected, which debugedit dutifully does. In short, this
is an upstream bug.

If that shared library was in /usr/lib64 where prelink could see it, I
bet that prelink would nuke those sections, too.

The solution is for upstream to change their linker scripts to put
those new sections into a segment that won't be garbage collected.
Take a look at the output of "readelf -S libmtcp.so". Note the
absence of any flags on the ".the.begin" and ".the.end" sections.
Also, the output of "readelf -l libmtcp.so" shows that there is no
section-to-segment mapping for the new sections.

Hoping-this-is-not-utter-nonsense-ly yours,
--
Jerry James
http://www.jamezone.org/
--
devel mailing list
devel@lists.fedoraproject.org
https://admin.fedoraproject.org/mailman/listinfo/devel
 

Thread Tools




All times are GMT. The time now is 11:46 AM.

VBulletin, Copyright ©2000 - 2014, Jelsoft Enterprises Ltd.
Content Relevant URLs by vBSEO ©2007, Crawlability, Inc.
Copyright 2007 - 2008, www.linux-archive.org