Hi,

we have a package on DebianMed which has performance issue regarding
static build vs dynamic link build (latest gcc)

Has only impacted compile option is the static vs dynamic choice,
has anyone an idea on where we could investigate or how we could
speedup this?



For info, executable is quite large 11MB (static) vs 2M (dynamic).



Dynamic build exe is linked against:

osallou@debiansid:/tmp/ncbi-blast-2.2.25+-src/debian/ncbi-blast+/usr/bin$
ldd /usr/bin/blastx

*** linux-vdso.so.1 =>* (0x00007fffb01a8000)

*** libblastinput.so => /usr/lib/ncbi-blast+/libblastinput.so
(0x00007f06d9572000)

*** libncbi_xloader_blastdb.so =>
/usr/lib/ncbi-blast+/libncbi_xloader_blastdb.so (0x00007f06d9350000)

*** libncbi_xloader_blastdb_rmt.so =>
/usr/lib/ncbi-blast+/libncbi_xloader_blastdb_rmt.so
(0x00007f06d9127000)

*** libxblastformat.so => /usr/lib/ncbi-blast+/libxblastformat.so
(0x00007f06d8edd000)

*** libalign_format.so => /usr/lib/ncbi-blast+/libalign_format.so
(0x00007f06d8c32000)

*** libblastdb_format.so =>
/usr/lib/ncbi-blast+/libblastdb_format.so (0x00007f06d8a1a000)

*** libgene_info.so => /usr/lib/ncbi-blast+/libgene_info.so
(0x00007f06d8808000)

*** libxalnmgr.so => /usr/lib/ncbi-blast+/libxalnmgr.so
(0x00007f06d856a000)

*** libblastxml.so => /usr/lib/ncbi-blast+/libblastxml.so
(0x00007f06d8353000)

*** libxcgi.so => /usr/lib/ncbi-blast+/libxcgi.so
(0x00007f06d80c8000)

*** libxhtml.so => /usr/lib/ncbi-blast+/libxhtml.so
(0x00007f06d7e48000)

*** libxblast.so => /usr/lib/ncbi-blast+/libxblast.so
(0x00007f06d7a36000)

*** libxalgoblastdbindex.so =>
/usr/lib/ncbi-blast+/libxalgoblastdbindex.so (0x00007f06d77fa000)

*** libcomposition_adjustment.so =>
/usr/lib/ncbi-blast+/libcomposition_adjustment.so
(0x00007f06d75e6000)

*** libxalgodustmask.so =>
/usr/lib/ncbi-blast+/libxalgodustmask.so (0x00007f06d73dd000)

*** libxalgowinmask.so => /usr/lib/ncbi-blast+/libxalgowinmask.so
(0x00007f06d718a000)

*** libseqmasks_io.so => /usr/lib/ncbi-blast+/libseqmasks_io.so
(0x00007f06d6f7a000)

*** libseqdb.so => /usr/lib/ncbi-blast+/libseqdb.so
(0x00007f06d6cb2000)

*** libblast_services.so =>
/usr/lib/ncbi-blast+/libblast_services.so (0x00007f06d6aa1000)

*** libxobjutil.so => /usr/lib/ncbi-blast+/libxobjutil.so
(0x00007f06d67f5000)

*** libxobjread.so => /usr/lib/ncbi-blast+/libxobjread.so
(0x00007f06d64de000)

*** libcreaders.so => /usr/lib/ncbi-blast+/libcreaders.so
(0x00007f06d62d5000)

*** libxnetblastcli.so => /usr/lib/ncbi-blast+/libxnetblastcli.so
(0x00007f06d60c8000)

*** libxnetblast.so => /usr/lib/ncbi-blast+/libxnetblast.so
(0x00007f06d5e1b000)

*** libblastdb.so => /usr/lib/ncbi-blast+/libblastdb.so
(0x00007f06d5c06000)

*** libscoremat.so => /usr/lib/ncbi-blast+/libscoremat.so
(0x00007f06d59e2000)

*** libtables.so => /usr/lib/ncbi-blast+/libtables.so
(0x00007f06d57de000)

*** libncbi_xloader_genbank.so =>
/usr/lib/ncbi-blast+/libncbi_xloader_genbank.so (0x00007f06d55a1000)

*** libncbi_xreader_id1.so =>
/usr/lib/ncbi-blast+/libncbi_xreader_id1.so (0x00007f06d537b000)

*** libncbi_xreader_id2.so =>
/usr/lib/ncbi-blast+/libncbi_xreader_id2.so (0x00007f06d5160000)

*** libncbi_xreader_cache.so =>
/usr/lib/ncbi-blast+/libncbi_xreader_cache.so (0x00007f06d4f2c000)

*** libxconnect.so => /usr/lib/ncbi-blast+/libxconnect.so
(0x00007f06d4c94000)

*** libncbi_xreader.so => /usr/lib/ncbi-blast+/libncbi_xreader.so
(0x00007f06d49e5000)

*** libid1.so => /usr/lib/ncbi-blast+/libid1.so
(0x00007f06d47cb000)

*** libid2.so => /usr/lib/ncbi-blast+/libid2.so
(0x00007f06d457e000)

*** libseqsplit.so => /usr/lib/ncbi-blast+/libseqsplit.so
(0x00007f06d4324000)

*** libxcompress.so => /usr/lib/ncbi-blast+/libxcompress.so
(0x00007f06d40cf000)

*** libxobjmgr.so => /usr/lib/ncbi-blast+/libxobjmgr.so
(0x00007f06d3be3000)

*** libgenome_collection.so =>
/usr/lib/ncbi-blast+/libgenome_collection.so (0x00007f06d3981000)

*** libseqset.so => /usr/lib/ncbi-blast+/libseqset.so
(0x00007f06d3762000)

*** libseqedit.so => /usr/lib/ncbi-blast+/libseqedit.so
(0x00007f06d3519000)

*** libseq.so => /usr/lib/ncbi-blast+/libseq.so
(0x00007f06d2f16000)

*** libseqcode.so => /usr/lib/ncbi-blast+/libseqcode.so
(0x00007f06d2d03000)

*** libsequtil.so => /usr/lib/ncbi-blast+/libsequtil.so
(0x00007f06d2af2000)

*** libpub.so => /usr/lib/ncbi-blast+/libpub.so
(0x00007f06d28d2000)

*** libmedline.so => /usr/lib/ncbi-blast+/libmedline.so
(0x00007f06d26b2000)

*** libbiblio.so => /usr/lib/ncbi-blast+/libbiblio.so
(0x00007f06d246a000)

*** libgeneral.so => /usr/lib/ncbi-blast+/libgeneral.so
(0x00007f06d220c000)

*** libxser.so => /usr/lib/ncbi-blast+/libxser.so
(0x00007f06d1ec0000)

*** libxutil.so => /usr/lib/ncbi-blast+/libxutil.so
(0x00007f06d1bea000)

*** libxncbi.so => /usr/lib/ncbi-blast+/libxncbi.so
(0x00007f06d180e000)

*** libz.so.1 => /usr/lib/libz.so.1 (0x00007f06d15ed000)

*** libbz2.so.1.0 => /lib/libbz2.so.1.0 (0x00007f06d13dd000)

*** libdl.so.2 => /lib/x86_64-linux-gnu/libdl.so.2
(0x00007f06d11d8000)

*** libnsl.so.1 => /lib/x86_64-linux-gnu/libnsl.so.1
(0x00007f06d0fc0000)

*** librt.so.1 => /lib/x86_64-linux-gnu/librt.so.1
(0x00007f06d0db8000)

*** libpthread.so.0 => /lib/x86_64-linux-gnu/libpthread.so.0
(0x00007f06d0b9b000)

*** libstdc++.so.6 => /usr/lib/x86_64-linux-gnu/libstdc++.so.6
(0x00007f06d0891000)

*** libm.so.6 => /lib/x86_64-linux-gnu/libm.so.6
(0x00007f06d060f000)

*** libgcc_s.so.1 => /lib/x86_64-linux-gnu/libgcc_s.so.1
(0x00007f06d03f8000)

*** libc.so.6 => /lib/x86_64-linux-gnu/libc.so.6
(0x00007f06d0074000)

*** /lib64/ld-linux-x86-64.so.2 (0x00007f06d97e6000)





For a simple "-h" call, here is a benchmark:



# Static build"

osallou@debiansid:/tmp/ncbi-blast-2.2.25+-src/debian/ncbi-blast+/usr/bin$
time bash -c 'for (( c=1; c<=50; c++ )) ; do ./blastx -h >
/dev/null ; done'



real*** 0m0.403s

user*** 0m0.292s

sys*** 0m0.124s



# Dynamic build

osallou@debiansid:/tmp/ncbi-blast-2.2.25+-src/debian/ncbi-blast+/usr/bin$
time bash -c 'for (( c=1; c<=50; c++ )) ; do blastx -h >
/dev/null ; done'



real*** 0m8.002s

user*** 0m4.200s

sys*** 0m3.856s





-------- Message original --------



Sujet:
Re: BLAST+ speed & build issues


Date de
renvoi*:
Thu, 4 Aug 2011 09:31:54 +0000 (UTC)


De
(renvoi)*:
debian-med@lists.debian.org


Date*:
Thu, 04 Aug 2011 11:31:34 +0200


De*:
Olivier Sallou <olivier.sallou@irisa.fr>


Pour*:
Tim Booth <avarus@fastmail.fm>


Copie à*:

debian-med@lists.debian.org, "Aaron M. Ucko"
<amu@alum.mit.edu>







Hi,
I quickly build the ncbi-blast+ locally with static build using the
debian build process, and indeed,
static build binaries are much faster (blastx -h) than dynamically
linked ones.

I will investigate if there is a way to speedup this. Or if anyone has
ideas....

Olivier

Le 8/3/11 10:48 AM, Tim Booth a écrit :
> time bash -c 'for (( c=1; c<=50; c++ )) ; do ~/tings/ncbi-blast-2.2.25+/bin/blastx -h > /dev/null ; done'

--
Olivier Sallou
IRISA / University of Rennes 1
Campus de Beaulieu, 35000 RENNES - FRANCE
Tel: 02.99.84.71.95

gpg key id: 4096R/326D8438 (pgp.mit.edu)
Key fingerprint = 5FB4 6F83 D3B9 5204 6335 D26D 78DC 68DB 326D 8438



--
To UNSUBSCRIBE, email to debian-med-REQUEST@lists.debian.org
with a subject of "unsubscribe". Trouble? Contact listmaster@lists.debian.org
Archive: http://lists.debian.org/4E3A66F6.7040008@irisa.fr

On 08/04/2011 12:02 PM, Olivier Sallou wrote:
> Hi,
> we have a package on DebianMed which has performance issue regarding
> static build vs dynamic link build (latest gcc)
> Has only impacted compile option is the static vs dynamic choice, has
> anyone an idea on where we could investigate or how we could speedup this?
>
> For a simple "-h" call, here is a benchmark:
>
> # Static build"
> osallou@debiansid:/tmp/ncbi-blast-2.2.25+-src/debian/ncbi-blast+/usr/bin$ time
> bash -c 'for (( c=1; c<=50; c++ )) ; do ./blastx -h > /dev/null ; done'
>
> real 0m0.403s
> user 0m0.292s
> sys 0m0.124s
>
> # Dynamic build
> osallou@debiansid:/tmp/ncbi-blast-2.2.25+-src/debian/ncbi-blast+/usr/bin$ time
> bash -c 'for (( c=1; c<=50; c++ )) ; do blastx -h > /dev/null ; done'
>
> real 0m8.002s
> user 0m4.200s
> sys 0m3.856s
>
>

Hi,
unless the purpose of this program is to display help messages to stdout
this is hardly a meaningful benchmark.
Starting a dynamic linked library is slower than starting a static one,
but that does not yet mean the dynamic version will be so much slower
when its running.
You might be able to reduce startup time by only linking against the
libraries you need or lazyly dynamically loading them.
using ld --as-needed is the solution for the former, but see bug 633567
for problems with this.

Best Regards,
Julian Taylor

Julian Taylor, le Thu 04 Aug 2011 12:32:27 +0200, a écrit :
> You might be able to reduce startup time by only linking against the
> libraries you need or lazyly dynamically loading them.

Or use prelink.

Samuel


--
To UNSUBSCRIBE, email to debian-devel-REQUEST@lists.debian.org
with a subject of "unsubscribe". Trouble? Contact listmaster@lists.debian.org
Archive: 20110804112744.GF4985@type.bordeaux.inria.fr">http ://lists.debian.org/20110804112744.GF4985@type.bordeaux.inria.fr

On jeu., 2011-08-04 at 13:27 +0200, Samuel Thibault wrote:
> Julian Taylor, le Thu 04 Aug 2011 12:32:27 +0200, a écrit :
> > You might be able to reduce startup time by only linking against the
> > libraries you need or lazyly dynamically loading them.
>
> Or use prelink.

Which as issues especially wrt. ASLR.

Regards,
--
Yves-Alexis

On Thu, Aug 04, 2011 at 01:50:01PM +0200, Yves-Alexis Perez wrote:
> On jeu., 2011-08-04 at 13:27 +0200, Samuel Thibault wrote:
> > Julian Taylor, le Thu 04 Aug 2011 12:32:27 +0200, a écrit :
> > > You might be able to reduce startup time by only linking against the
> > > libraries you need or lazyly dynamically loading them.
> >
> > Or use prelink.
>
> Which as issues especially wrt. ASLR.

And doesn't help with the biggest problem: I/O

Mike


--
To UNSUBSCRIBE, email to debian-devel-REQUEST@lists.debian.org
with a subject of "unsubscribe". Trouble? Contact listmaster@lists.debian.org
Archive: 20110804115646.GA10717@glandium.org">http://lists.debian.org/20110804115646.GA10717@glandium.org

Linking against "needed" lib is quite difficult (would need to be done on upstream), package is rather complex regarding compilation.And each solution has its pros and cons....
I think we will have to discuss on validity to provide static builds ( takes a lot of space on package vs perf loss)

Thanks anyway

2011/8/4 Samuel Thibault <sthibault@debian.org>

Julian Taylor, le Thu 04 Aug 2011 12:32:27 +0200, a écrit :

> You might be able to reduce startup time by only linking against the

> libraries you need or lazyly dynamically loading them.



Or use prelink.



Samuel





--

To UNSUBSCRIBE, email to debian-devel-REQUEST@lists.debian.org

with a subject of "unsubscribe". Trouble? Contact listmaster@lists.debian.org

Archive: http://lists.debian.org/20110804112744.GF4985@type.bordeaux.inria.fr

Olivier,

Because "-h is slow" hardly seems like a good justification for having
static packages. Last I checked (which wasn't long ago..), BLAST is
typically a long-running process (at least on the stuff we're doing..).
Also, are subsequent calls (even '-h' ones) faster? I'd expect them to
be, once everything is cache'd, which would reduce the justification
that much further..

No, having the database stay in memory and being able to query against
it rather than having to start up blast every time (ala FastCGI or
something like you'd do for a webserver) might be a way to kill two
birds w/ one stone, as it was, but that'd be a fair bit of work, I
expect.

Thanks,

Stephen

* olivier sallou (olivier.sallou@gmail.com) wrote:
> Linking against "needed" lib is quite difficult (would need to be done on
> upstream), package is rather complex regarding compilation.
> And each solution has its pros and cons....
>
> I think we will have to discuss on validity to provide static builds ( takes
> a lot of space on package vs perf loss)
>
> Thanks anyway
>
> 2011/8/4 Samuel Thibault <sthibault@debian.org>
>
> > Julian Taylor, le Thu 04 Aug 2011 12:32:27 +0200, a écrit :
> > > You might be able to reduce startup time by only linking against the
> > > libraries you need or lazyly dynamically loading them.
> >
> > Or use prelink.
> >
> > Samuel
> >
> >
> > --
> > To UNSUBSCRIBE, email to debian-devel-REQUEST@lists.debian.org
> > with a subject of "unsubscribe". Trouble? Contact
> > listmaster@lists.debian.org
> > Archive:
> > http://lists.debian.org/20110804112744.GF4985@type.bordeaux.inria.fr
> >
> >

On Thu, Aug 04, 2011 at 12:02:25PM +0200, Olivier Sallou wrote:
> For info, executable is quite large 11MB (static) vs 2M (dynamic).

The 11MB probably contains a whole bunch of libraries instead,
making it larger. I don't see how this is relavant.

> Dynamic build exe is linked against:
> osallou@debiansid:/tmp/ncbi-blast-2.2.25+-src/debian/ncbi-blast+/usr/bin$ ldd
> /usr/bin/blastx
> linux-vdso.so.1 => (0x00007fffb01a8000)
> libblastinput.so => /usr/lib/ncbi-blast+/libblastinput.so
> (0x00007f06d9572000)
[...lots of blast libs...]

> For a simple "-h" call, here is a benchmark:

Like others have pointed out, this is not a useful benchmark.
You're basicly testing the startup time of your application. When
you start an applications that dynamic linker has to resolv
symbols and relocations. This takes time which for the static
version was done at link time. Having more libraries it needs to
check for the symbol means it will take longer.

What can help is that you reduce the total amount of symbols that
it needs to check be only exporting the symbols you need from the
shared libraries. There are various methods for that.

The basic 3 methods I know:
- Making things as much as possible static.
- Using symbol visibility: tell gcc which symbols should be visible.
- Using a linker map to only export those that should be visible.


Kurt


--
To UNSUBSCRIBE, email to debian-devel-REQUEST@lists.debian.org
with a subject of "unsubscribe". Trouble? Contact listmaster@lists.debian.org
Archive: 20110806103714.GA2861@roeckx.be">http://lists.debian.org/20110806103714.GA2861@roeckx.be