FAQ Search Today's Posts Mark Forums Read
» Video Reviews

» Linux Archive

Linux-archive is a website aiming to archive linux email lists and to make them easily accessible for linux users/developers.


» Sponsor

» Partners

» Sponsor

Go Back   Linux Archive > ArchLinux > ArchLinux Pacman Development

 
 
LinkBack Thread Tools
 
Old 01-22-2011, 09:25 AM
Allan McRae
 
Default makepkg: add soprovides support

On 22/01/11 19:31, Florian Pritz wrote:

On 22.01.2011 01:20, Allan McRae wrote:

And how can we do this portably and subject to the restrictions given...
Well, it has been established that using objdump or elfutils has
issues so lets just exclude them from the start. How else can we get
information about a file? file... That tells you whether a binary is
32 or 64 bit, and is already required by makepkg.


file doesn't tell you the library soname so I don't know what ld will
use for linking. So far readelf seems to be the best choice because I
can also use it to generate the dependencies (right now it seems like we
are too focused on the provides). Your example below can be achieved
using readelf.
Also file's output doesn't have to be the same on different platforms.



What are we going to do about OSX and its lack of elf binaries?
 
Old 01-22-2011, 09:54 AM
Xavier Chantry
 
Default makepkg: add soprovides support

On Sat, Jan 22, 2011 at 11:25 AM, Allan McRae <allan@archlinux.org> wrote:
>
> What are we going to do about OSX and its lack of elf binaries?
>

Who cares about soprovides/sodepends besides ArchLinux people ?
 
Old 01-22-2011, 09:59 AM
Florian Pritz
 
Default makepkg: add soprovides support

On 22.01.2011 11:54, Xavier Chantry wrote:
> On Sat, Jan 22, 2011 at 11:25 AM, Allan McRae <allan@archlinux.org> wrote:
>>
>> What are we going to do about OSX and its lack of elf binaries?
>>
>
> Who cares about soprovides/sodepends besides ArchLinux people ?

After a quick google search it looks like OS X uses .dylib so I won't
even look at those files anyway since my find command only search *.so*

--
Florian Pritz -- {flo,bluewind}@server-speed.net
 
Old 01-22-2011, 10:22 AM
Thomas Bächler
 
Default makepkg: add soprovides support

Am 22.01.2011 01:20, schrieb Allan McRae:
> I only pointed out the things that I see are wrong because they are what
> needed fixed. Also it was 2am when I replied in between dealing with a
> sick child so my time for pleasantries was limited.

Sorry, I sounded way harsher than I meant to in that post (as pointed
out by Dan already). Happens to the best of us.
 
Old 01-22-2011, 10:25 AM
Thomas Bächler
 
Default makepkg: add soprovides support

Am 22.01.2011 11:25, schrieb Allan McRae:
> What are we going to do about OSX and its lack of elf binaries?

A solution that fits all platforms probably doesn't exist here. We can
implement this for ELF now, and maybe someone can figure out the OS X
version at some point. (.dylib is very weird IIRC, I had to look into
that once).
 
Old 01-22-2011, 11:15 AM
Allan McRae
 
Default makepkg: add soprovides support

On 22/01/11 21:25, Thomas Bächler wrote:

Am 22.01.2011 11:25, schrieb Allan McRae:

What are we going to do about OSX and its lack of elf binaries?


A solution that fits all platforms probably doesn't exist here. We can
implement this for ELF now, and maybe someone can figure out the OS X
version at some point. (.dylib is very weird IIRC, I had to look into
that once).



That sounds fine. I did not realise that a different extension is used
for shared libraries on OSX (although I should have thinking about
it...). So just working with ELF for now seems fine.



Some other things to think about. I just built a bunch of packages
using soprovides/sodepends with readline providing libreadline.so.
With this, on an upgrade with a soname bump we will get a whole heap of
messages like:


warning: provider package was selected (readline provides libreadline.so)

So we need to consider whether we want to keep that message displayed,
or if there is some way to only display it once (see next).


I also noticed the performance hit in satisfying dependencies using
provides. I wonder if we can optimise dependency checking such that we
check for a given dependency only once, even if it is needed across
multiple packages? That would also solve the output issue above. I'm
not familiar enough with the dependency checking code to know if that is
viable.


Allan
 
Old 01-22-2011, 11:43 AM
Florian Pritz
 
Default makepkg: add soprovides support

On 22.01.2011 13:15, Allan McRae wrote:
> With this, on an upgrade with a soname bump we will get a whole heap of
> messages like:
>
> warning: provider package was selected (readline provides libreadline.so)

> I also noticed the performance hit in satisfying dependencies using
> provides.

When I tried that in August 2009 it only displayed that message once (at
least I think it did, but I'm pretty sure) and the time to resolve the
dependencies for ~900 packages with (using -U; all sodeps/provides and
not just readline iirc) went from 25 secs to 27 secs.

Could you stop the time so we can compare the results?

--
Florian Pritz -- {flo,bluewind}@server-speed.net
 
Old 01-22-2011, 01:04 PM
Allan McRae
 
Default makepkg: add soprovides support

On 22/01/11 22:43, Florian Pritz wrote:

On 22.01.2011 13:15, Allan McRae wrote:

With this, on an upgrade with a soname bump we will get a whole heap of
messages like:

warning: provider package was selected (readline provides libreadline.so)



I also noticed the performance hit in satisfying dependencies using
provides.


When I tried that in August 2009 it only displayed that message once (at
least I think it did, but I'm pretty sure)


Ugh... my bad... you have to weird things to your pacman databases get
this message to appear multiple times. Moving along...



and the time to resolve the
dependencies for ~900 packages with (using -U; all sodeps/provides and
not just readline iirc) went from 25 secs to 27 secs.

Could you stop the time so we can compare the results?


When installing two packages, changing the depend to a provide takes
8.7sec compared to 7.9sec. So about 10%, which is consistent with what
you got. I'm a bit surprised it scales so well to a much large number
of packages and provides.


Allan
 
Old 01-28-2011, 07:12 AM
Florian Pritz
 
Default makepkg: add soprovides support

From: Florian Pritz <bluewind@xssn.at>

The user adds libaries to the provides array without a version. These
must end with .so.
Example: provides=(readline libreadline.so)

find_soprovides() looks for .so files (not symlinks because these could
point outside of pkgdir) in $pkgdir, extracts the library soname (ld
links the binary to this name) and outputs provides seperated by spaces.
This list contains all libraries provided by the package.
Example: libfoo.so=3_64

write_pkginfo() only keeps .so provides with version information and warns
the user about unneded ones.

Support-by: Thomas Bächler <thomas@archlinux.org>
Support-by: Christoph Schied <Christoph.Schied@uni-ulm.de>
Signed-off-by: Florian Pritz <bluewind@server-speed.net>
---
scripts/makepkg.sh.in | 47 ++++++++++++++++++++++++++++++++++++++++++++++-
1 files changed, 46 insertions(+), 1 deletions(-)

diff --git a/scripts/makepkg.sh.in b/scripts/makepkg.sh.in
index 6ebfac0..06631f4 100644
--- a/scripts/makepkg.sh.in
+++ b/scripts/makepkg.sh.in
@@ -917,6 +917,34 @@ tidy_install() {
fi
}

+find_soprovides() {
+ local soprovides
+ local os=$(uname -s | tr [:upper:] [:lower:])
+ find "$pkgdir" -type f -name *.so* | while read filename
+ do
+ # check if we really have a shared object
+ if LC_ALL=C readelf -h "$filename" 2>/dev/null | grep -q '.*Type:.*DYN (Shared object file).*'; then
+ # 64
+ soarch=$(LC_ALL=C readelf -h "$filename" | sed -n 's/.*Class.*ELF(32|64)/1/p')
+ # get the string binaries link to: libfoo.so.1.2 -> libfoo.so.1
+ sofile=$(LC_ALL=C readelf -d "$filename" 2>/dev/null | sed -n 's/.*Library soname: [(.*)].*/1/p')
+ [ -z "$sofile" ] && sofile="${$filename##*/}"
+
+ # extract the library name: libfoo.so
+ soname="${sofile%%.so.*}.so"
+ # extract the major version: 1
+ soversion="${sofile##*.so.}"
+ if in_array "${soname}" ${provides[@]}; then
+ if ! in_array "${soname}=${soversion}-${soarch}" ${soprovides[@]}; then
+ # libfoo.so=1-elf_x86_64_linux
+ echo "${soname}=${soversion}-${soarch}"
+ soprovides=(${soprovides[@]} "${soname}=${soversion}-${soarch}")
+ fi
+ fi
+ fi
+ done
+}
+
write_pkginfo() {
local builddate=$(date -u "+%s")
if [[ -n $PACKAGER ]]; then
@@ -950,10 +978,27 @@ write_pkginfo() {
[[ $depends ]] && printf "depend = %s
" "${depends[@]}"
[[ $optdepends ]] && printf "optdepend = %s
" "${optdepends[@]}"
[[ $conflicts ]] && printf "conflict = %s
" "${conflicts[@]}"
- [[ $provides ]] && printf "provides = %s
" "${provides[@]}"
[[ $backup ]] && printf "backup = %s
" "${backup[@]}"

local it
+
+ soprovides=$(find_soprovides)
+ provides=("${provides[@]}" ${soprovides})
+
+ for it in "${provides[@]}"; do
+ # ignore versionless entires (those come from the PKGBUILD)
+ if [[ $it = *.so ]]; then
+ # check if the entry has been found by find_soprovides
+ # if not, it's unneeded; tell the user so he can remove it
+ if [[ ! $soprovides =~ (^|s)${it}=.* ]]; then
+ error "$(gettext "Can't find library listed in $provides: %s")" "$it"
+ return 1
+ fi
+ else
+ echo "provides = $it"
+ fi
+ done
+
for it in "${packaging_options[@]}"; do
local ret="$(check_option $it)"
if [[ $ret != "?" ]]; then
--
1.7.3.5
 
Old 01-31-2011, 06:15 PM
Florian Pritz
 
Default makepkg: add soprovides support

From: Florian Pritz <bluewind@xssn.at>

The user adds libaries to the provides array without a version. These
must end with .so.
Example: provides=(readline libreadline.so)

find_soprovides() looks for .so files (not symlinks because these could
point outside of pkgdir) in $pkgdir, extracts the library soname (ld
links the binary to this name) and outputs provides seperated by spaces.
This list contains all libraries provided by the package.
Example: libfoo.so=3-64

write_pkginfo() only keeps .so provides with version information and warns
the user about unneded ones.

Support-by: Thomas Bächler <thomas@archlinux.org>
Support-by: Christoph Schied <Christoph.Schied@uni-ulm.de>
Signed-off-by: Florian Pritz <bluewind@server-speed.net>
---
scripts/makepkg.sh.in | 46 +++++++++++++++++++++++++++++++++++++++++++++-
1 files changed, 45 insertions(+), 1 deletions(-)

diff --git a/scripts/makepkg.sh.in b/scripts/makepkg.sh.in
index b1b1b75..0d811b6 100644
--- a/scripts/makepkg.sh.in
+++ b/scripts/makepkg.sh.in
@@ -933,6 +933,33 @@ tidy_install() {
fi
}

+find_soprovides() {
+ local soprovides
+ find "$pkgdir" -type f -name *.so* | while read filename
+ do
+ # check if we really have a shared object
+ if LC_ALL=C readelf -h "$filename" 2>/dev/null | grep -q '.*Type:.*DYN (Shared object file).*'; then
+ # 64
+ soarch=$(LC_ALL=C readelf -h "$filename" | sed -n 's/.*Class.*ELF(32|64)/1/p')
+ # get the string binaries link to: libfoo.so.1.2 -> libfoo.so.1
+ sofile=$(LC_ALL=C readelf -d "$filename" 2>/dev/null | sed -n 's/.*Library soname: [(.*)].*/1/p')
+ [ -z "$sofile" ] && sofile="${$filename##*/}"
+
+ # extract the library name: libfoo.so
+ soname="${sofile%%.so.*}.so"
+ # extract the major version: 1
+ soversion="${sofile##*.so.}"
+ if in_array "${soname}" ${provides[@]}; then
+ if ! in_array "${soname}=${soversion}-${soarch}" ${soprovides[@]}; then
+ # libfoo.so=1-64
+ echo "${soname}=${soversion}-${soarch}"
+ soprovides=(${soprovides[@]} "${soname}=${soversion}-${soarch}")
+ fi
+ fi
+ fi
+ done
+}
+
write_pkginfo() {
local builddate=$(date -u "+%s")
if [[ -n $PACKAGER ]]; then
@@ -965,10 +992,27 @@ write_pkginfo() {
[[ $depends ]] && printf "depend = %s
" "${depends[@]}"
[[ $optdepends ]] && printf "optdepend = %s
" "${optdepends[@]}"
[[ $conflicts ]] && printf "conflict = %s
" "${conflicts[@]}"
- [[ $provides ]] && printf "provides = %s
" "${provides[@]}"
[[ $backup ]] && printf "backup = %s
" "${backup[@]}"

local it
+
+ soprovides=$(find_soprovides)
+ provides=("${provides[@]}" ${soprovides})
+
+ for it in "${provides[@]}"; do
+ # ignore versionless entires (those come from the PKGBUILD)
+ if [[ $it = *.so ]]; then
+ # check if the entry has been found by find_soprovides
+ # if not, it's unneeded; tell the user so he can remove it
+ if [[ ! $soprovides =~ (^|s)${it}=.* ]]; then
+ error "$(gettext "Can't find library listed in $provides: %s")" "$it"
+ return 1
+ fi
+ else
+ echo "provides = $it"
+ fi
+ done
+
for it in "${packaging_options[@]}"; do
local ret="$(check_option $it)"
if [[ $ret != "?" ]]; then
--
1.7.3.5
 

Thread Tools




All times are GMT. The time now is 11:44 AM.

VBulletin, Copyright ©2000 - 2014, Jelsoft Enterprises Ltd.
Content Relevant URLs by vBSEO ©2007, Crawlability, Inc.
Copyright ©2007 - 2008, www.linux-archive.org