FAQ Search Today's Posts Mark Forums Read
» Video Reviews

» Linux Archive

Linux-archive is a website aiming to archive linux email lists and to make them easily accessible for linux users/developers.


» Sponsor

» Partners

» Sponsor

Go Back   Linux Archive > ArchLinux > ArchLinux Pacman Development

 
 
LinkBack Thread Tools
 
Old 06-02-2008, 02:27 PM
Allan McRae
 
Default makepkg: add --allsource option

Miklos Vajna wrote:
> On Mon, Jun 02, 2008 at 06:47:36AM -0500, Dan McGee <dpmcgee@gmail.com> wrote:
>
>> So what are you saying, did anyone actually look at the patch? I
>> didn't change the existing --source option one bit, that *still* only
>> includes local source files.
>>
>
> yes, i read that patch
>
>
>> All I did was add a completely new --allsource option that allows you
>> to also build an src.tar.gz with all source, local and remote. How
>> much more flexible can we get than having two options here?
>>
>
> i just said that creating such a huge srpm-like tarball is a bad idea,
> and encouraging people to do so by adding an option for that is not
> something i would recommend. just my two cents, though.
>

I see no problem with this. It is an optional extra. Nobody is being
forced to use it or download the packages if someone else uses it. And
it is actually useful for those hosting custom repos as they can quickly
create a complete source package to host for GPL(2?) compliance.

The only cost here is one extra if statement if you are using the
--source option and another long option name to come up with an
abbreviation for (if that idea goes ahead). I.e. essentially zero.


Allan



_______________________________________________
pacman-dev mailing list
pacman-dev@archlinux.org
http://archlinux.org/mailman/listinfo/pacman-dev
 
Old 06-02-2008, 02:56 PM
Allan McRae
 
Default makepkg: add --allsource option

Dan McGee wrote:
> <snip>
> + local netfile
> + for netfile in ${source[@]}; do
> + local file=$(strip_url "$netfile")
> + if [ -f $netfile ]; then
> + msg2 "$(gettext "Adding %s...")" "$netfile"
> + comp_files="$comp_files $netfile"
> + elif [ "$SOURCEONLY" = "2" -a -f "$srcdir/$file" ]; then
> + msg2 "$(gettext "Adding %s...")" "$file"
> + comp_files="$comp_files -C $srcdir $file"
>

I'm a little confused here. The file $srcdir/$file is just a link to
$SRCDEST/$file which may be $startdir/$file. So does this not just add
the symbolic link to the tarball. Also, the -C option flags all
following files to be from the $srcdir directory. So, anything added
after this will also get its symbolic link added.

If I'm not going down the wrong track here, this should fix it...

+ local netfile
+ for netfile in ${source[@]}; do
+ local file=$(strip_url "$netfile")
+ if [ -f $netfile ]; then
+ msg2 "$(gettext "Adding %s...")" "$netfile"
+ comp_files="$comp_files -C $startdir $netfile"
+ elif [ "$SOURCEONLY" = "2" -a -f "$SRCDEST/$file" ]; then
+ msg2 "$(gettext "Adding %s...")" "$file"
+ comp_files="$comp_files -C $SRCDEST $file"





_______________________________________________
pacman-dev mailing list
pacman-dev@archlinux.org
http://archlinux.org/mailman/listinfo/pacman-dev
 
Old 06-02-2008, 03:13 PM
"Dan McGee"
 
Default makepkg: add --allsource option

On Mon, Jun 2, 2008 at 9:56 AM, Allan McRae <mcrae_allan@hotmail.com> wrote:
> Dan McGee wrote:
>> <snip>
>> + local netfile
>> + for netfile in ${source[@]}; do
>> + local file=$(strip_url "$netfile")
>> + if [ -f $netfile ]; then
>> + msg2 "$(gettext "Adding %s...")" "$netfile"
>> + comp_files="$comp_files $netfile"
>> + elif [ "$SOURCEONLY" = "2" -a -f "$srcdir/$file" ]; then
>> + msg2 "$(gettext "Adding %s...")" "$file"
>> + comp_files="$comp_files -C $srcdir $file"
>>
>
> I'm a little confused here. The file $srcdir/$file is just a link to
> $SRCDEST/$file which may be $startdir/$file. So does this not just add
> the symbolic link to the tarball. Also, the -C option flags all
> following files to be from the $srcdir directory. So, anything added
> after this will also get its symbolic link added.
>
> If I'm not going down the wrong track here, this should fix it...

Yeah, this wasn't tested super hard or anything so I think you are on
to something here. The -C option is definitely a bit odd but it kind
of does what we want. Since all files are available in src/ (although
some might be symlinks), can't we use 1 -C flag in combination with -H
to follow symlinks to our source files?

While we are making changes, can you see an easy way to get everything
under a pkgname/ dir (maybe with pkgver-pkgrel too) inside the
tarball? Unfortunately bsdtar has no --prefix option to easily do
this, and our friend Loui says it is doable but has not offered a line
of code.

>
> + local netfile
> + for netfile in ${source[@]}; do
> + local file=$(strip_url "$netfile")
> + if [ -f $netfile ]; then
> + msg2 "$(gettext "Adding %s...")" "$netfile"
> + comp_files="$comp_files -C $startdir $netfile"
> + elif [ "$SOURCEONLY" = "2" -a -f "$SRCDEST/$file" ]; then
> + msg2 "$(gettext "Adding %s...")" "$file"
> + comp_files="$comp_files -C $SRCDEST $file"
>
>
>
>
>
> _______________________________________________
> pacman-dev mailing list
> pacman-dev@archlinux.org
> http://archlinux.org/mailman/listinfo/pacman-dev
>

_______________________________________________
pacman-dev mailing list
pacman-dev@archlinux.org
http://archlinux.org/mailman/listinfo/pacman-dev
 
Old 06-02-2008, 03:56 PM
Allan McRae
 
Default makepkg: add --allsource option

Dan McGee wrote:
> On Mon, Jun 2, 2008 at 9:56 AM, Allan McRae <mcrae_allan@hotmail.com> wrote:
>
>> Dan McGee wrote:
>>
>>> <snip>
>>> + local netfile
>>> + for netfile in ${source[@]}; do
>>> + local file=$(strip_url "$netfile")
>>> + if [ -f $netfile ]; then
>>> + msg2 "$(gettext "Adding %s...")" "$netfile"
>>> + comp_files="$comp_files $netfile"
>>> + elif [ "$SOURCEONLY" = "2" -a -f "$srcdir/$file" ]; then
>>> + msg2 "$(gettext "Adding %s...")" "$file"
>>> + comp_files="$comp_files -C $srcdir $file"
>>>
>>>
>> I'm a little confused here. The file $srcdir/$file is just a link to
>> $SRCDEST/$file which may be $startdir/$file. So does this not just add
>> the symbolic link to the tarball. Also, the -C option flags all
>> following files to be from the $srcdir directory. So, anything added
>> after this will also get its symbolic link added.
>>
>> If I'm not going down the wrong track here, this should fix it...
>>
>
> Yeah, this wasn't tested super hard or anything so I think you are on
> to something here. The -C option is definitely a bit odd but it kind
> of does what we want. Since all files are available in src/ (although
> some might be symlinks), can't we use 1 -C flag in combination with -H
> to follow symlinks to our source files?
>

The files are not in src/ if the --source was used so we have to be
careful there.

> While we are making changes, can you see an easy way to get everything
> under a pkgname/ dir (maybe with pkgver-pkgrel too) inside the
> tarball? Unfortunately bsdtar has no --prefix option to easily do
> this, and our friend Loui says it is doable but has not offered a line
> of code.
>
>

How about:
- Make directory of form pkgname-pkgver-pkgrel in a temporary directory
- Instead of collecting files in "comp_files", just place links into
that directory.
- Use bsdtar with -L flag on that directory.
- Clean up

I think that is along the lines of what Loui was suggesting. It also
removes the whole -C issue without adding any further SOURCEONLY=2 checks.

Allan



_______________________________________________
pacman-dev mailing list
pacman-dev@archlinux.org
http://archlinux.org/mailman/listinfo/pacman-dev
 
Old 06-02-2008, 04:26 PM
"Dan McGee"
 
Default makepkg: add --allsource option

On Mon, Jun 2, 2008 at 10:56 AM, Allan McRae <mcrae_allan@hotmail.com> wrote:
> Dan McGee wrote:
>> On Mon, Jun 2, 2008 at 9:56 AM, Allan McRae <mcrae_allan@hotmail.com> wrote:
>>
>>> Dan McGee wrote:
>>>
>>>> <snip>
>>>> + local netfile
>>>> + for netfile in ${source[@]}; do
>>>> + local file=$(strip_url "$netfile")
>>>> + if [ -f $netfile ]; then
>>>> + msg2 "$(gettext "Adding %s...")" "$netfile"
>>>> + comp_files="$comp_files $netfile"
>>>> + elif [ "$SOURCEONLY" = "2" -a -f "$srcdir/$file" ]; then
>>>> + msg2 "$(gettext "Adding %s...")" "$file"
>>>> + comp_files="$comp_files -C $srcdir $file"
>>>>
>>>>
>>> I'm a little confused here. The file $srcdir/$file is just a link to
>>> $SRCDEST/$file which may be $startdir/$file. So does this not just add
>>> the symbolic link to the tarball. Also, the -C option flags all
>>> following files to be from the $srcdir directory. So, anything added
>>> after this will also get its symbolic link added.
>>>
>>> If I'm not going down the wrong track here, this should fix it...
>>>
>>
>> Yeah, this wasn't tested super hard or anything so I think you are on
>> to something here. The -C option is definitely a bit odd but it kind
>> of does what we want. Since all files are available in src/ (although
>> some might be symlinks), can't we use 1 -C flag in combination with -H
>> to follow symlinks to our source files?
>>
>
> The files are not in src/ if the --source was used so we have to be
> careful there.
>
>> While we are making changes, can you see an easy way to get everything
>> under a pkgname/ dir (maybe with pkgver-pkgrel too) inside the
>> tarball? Unfortunately bsdtar has no --prefix option to easily do
>> this, and our friend Loui says it is doable but has not offered a line
>> of code.
>>
>>
>
> How about:
> - Make directory of form pkgname-pkgver-pkgrel in a temporary directory
> - Instead of collecting files in "comp_files", just place links into
> that directory.
> - Use bsdtar with -L flag on that directory.
> - Clean up
>
> I think that is along the lines of what Loui was suggesting. It also
> removes the whole -C issue without adding any further SOURCEONLY=2 checks.

Yeah, I think I like this idea. Feel free to rip my patch apart.

We might want to take a closer look at how this is all done as the
original --source patch was written quickly, and the --allsource
option above was kind of jury-rigged in. I did like the fact that if
we download all sources we can verify the checksums- perhaps we can
still find a way to check md5sums for the local-only source as well
before we package it up in an src.tar.gz? We should be able to do some
kind of similar action:
for file in source:
if netfile && allsource then verify md5sum
if localfile then verify md5sum

Maybe add a flag or something to the checksum function to determine
whether to check netfiles or not.

Wow, this patch is going to turn into a sequence of 5.

-Dan

_______________________________________________
pacman-dev mailing list
pacman-dev@archlinux.org
http://archlinux.org/mailman/listinfo/pacman-dev
 
Old 06-02-2008, 10:09 PM
Loui
 
Default makepkg: add --allsource option

On Tue, 03 Jun 2008 01:56:49 +1000
Allan McRae <mcrae_allan@hotmail.com> wrote:

> Dan McGee wrote:
> > While we are making changes, can you see an easy way to get everything
> > under a pkgname/ dir (maybe with pkgver-pkgrel too) inside the
> > tarball? Unfortunately bsdtar has no --prefix option to easily do
> > this, and our friend Loui says it is doable but has not offered a line
> > of code.
>
> I think that is along the lines of what Loui was suggesting. It also
> removes the whole -C issue without adding any further SOURCEONLY=2 checks.

I can't quite grok the makepkg source code so I'll just demonstrate
what I do via a lil script. Enjoy!

#!/bin/bash
# Script for making source tarballs
# $1 = directory name
# $2 = release version

if [ "$1" == "" ]; then
echo "Usage: $0 <directory_name> <release_version>"
exit
fi

echo "Creating temporary symlink $1-$2"
ln -s $1 $1-$2

tar -cvvzf $1-$2.tar.gz $(find $1-$2/ -type f)

echo "Removing symlink $1-$2"
rm $1-$2
exit

_______________________________________________
pacman-dev mailing list
pacman-dev@archlinux.org
http://archlinux.org/mailman/listinfo/pacman-dev
 
Old 06-07-2008, 01:58 AM
Allan McRae
 
Default makepkg: add --allsource option

This supplements the --source option and does nearly the same thing, except
downloaded source files are included in the archive as well. The sources are
now packages with a pkgname/ prefix.

Original work by: Dan McGee <dan@archlinux.org>
Signed-off-by: Allan McRae <mcrae_allan@hotmail.com>
---
scripts/makepkg.sh.in | 46 ++++++++++++++++++++++++++++++++++------------
1 files changed, 34 insertions(+), 12 deletions(-)

diff --git a/scripts/makepkg.sh.in b/scripts/makepkg.sh.in
index 75db0c5..63187c0 100644
--- a/scripts/makepkg.sh.in
+++ b/scripts/makepkg.sh.in
@@ -921,14 +921,26 @@ create_xdelta() {

create_srcpackage() {
cd "$startdir"
+ if [ "$SOURCEONLY" = "2" ]; then
+ # get back to our src directory so we can begin with sources
+ mkdir -p "$srcdir"
+ cd "$srcdir"
+ download_sources
+ # we can only check checksums if we have all files
+ check_checksums
+ cd "$startdir"
+ fi
msg "$(gettext "Creating source package...")"
- local comp_files="$BUILDSCRIPT"
+ local srclinks="$(mktemp -d "$startdir"/srclinks.XXXXXXXXX)"
+ mkdir "${srclinks}"/${pkgname}
+
msg2 "$(gettext "Adding %s...")" "$BUILDSCRIPT"
+ ln -s "${startdir}/${BUILDSCRIPT}" "${srclinks}/${pkgname}/"

if [ "$install" != "" ]; then
if [ -f $install ]; then
msg2 "$(gettext "Adding install script...")"
- comp_files="$comp_files $install"
+ ln -s "${startdir}/$install" "${srclinks}/${pkgname}/"
else
error "$(gettext "Install script %s not found.")" "$install"
fi
@@ -936,14 +948,18 @@ create_srcpackage() {

if [ -f ChangeLog ]; then
msg2 "$(gettext "Adding %s...")" "ChangeLog"
- comp_files="$comp_files ChangeLog"
+ ln -s "${startdir}/ChangeLog" "${srclinks}/${pkgname}"
fi

- local i
- for i in ${source[@]}; do
- if [ -f $i ]; then
- msg2 "$(gettext "Adding %s...")" "$i"
- comp_files="$comp_files $i"
+ local netfile
+ for netfile in ${source[@]}; do
+ local file=$(strip_url "$netfile")
+ if [ -f $netfile ]; then
+ msg2 "$(gettext "Adding %s...")" "$netfile"
+ ln -s $netfile ${srclinks}/${pkgname}
+ elif [ "$SOURCEONLY" = "2" -a -f "$SRCDEST/$file" ]; then
+ msg2 "$(gettext "Adding %s...")" "$file"
+ ln -s "$SRCDEST/$file" "${srclinks}/${pkgname}/"
fi
done

@@ -951,10 +967,13 @@ create_srcpackage() {

# tar it up
msg2 "$(gettext "Compressing source package...")"
- if ! bsdtar -czf "$pkg_file" $comp_files; then
+ cd ${srclinks}
+ if ! bsdtar -czLf "$pkg_file" ${pkgname}; then
error "$(gettext "Failed to create source package file.")"
exit 1 # TODO: error code
fi
+ cd ${startdir}
+ rm -rf ${srclinks}
}

install_package() {
@@ -1126,8 +1145,10 @@ SRCDEST=${SRCDEST:-$startdir} #default to $startdir if undefined

# Parse Command Line Options.
OPT_SHORT="AbcCdefFghiLmop:rRsSV"
-OPT_LONG="ignorearch,asroot,builddeps,clean,cleanc ache,nodeps,noextract,force,forcever:,geninteg,hel p,holdver"
-OPT_LONG="$OPT_LONG,install,log,nocolor,nobuild,rm deps,repackage,source,syncdeps,usesudo,version"
+OPT_LONG="allsource,asroot,ignorearch,builddeps,c lean,cleancache,nodeps"
+OPT_LONG="$OPT_LONG,noextract,force,forcever:,gen integ,help,holdver"
+OPT_LONG="$OPT_LONG,install,log,nocolor,nobuild,r mdeps,repackage,source"
+OPT_LONG="$OPT_LONG,syncdeps,usesudo,version"
# Pacman Options
OPT_LONG="$OPT_LONG,noconfirm,noprogressbar"
OPT_TEMP="$(getopt -o "$OPT_SHORT" -l "$OPT_LONG" -n "$(basename "$0")" -- "$@" || echo 'GETOPT GO BANG!')"
@@ -1145,6 +1166,7 @@ while true; do
--noprogressbar) PACMAN_OPTS="$PACMAN_OPTS --noprogressbar" ;;

# Makepkg Options
+ --allsource) SOURCEONLY=2 ;;
--asroot) ASROOT=1 ;;
-A|--ignorearch) IGNOREARCH=1 ;;
-c|--clean) CLEANUP=1 ;;
@@ -1381,7 +1403,7 @@ if [ $EUID -eq 0 ]; then
fi

# if we are creating a source-only package, go no further
-if [ "$SOURCEONLY" = "1" ]; then
+if [ "$SOURCEONLY" != "0" ]; then
if [ -f "$PKGDEST/${pkgname}-${pkgver}-${pkgrel}${SRCEXT}"
-a "$FORCE" = "0" ]; then
error "$(gettext "A package has already been built. (use -f to overwrite)")"
--
1.5.5.3


_______________________________________________
pacman-dev mailing list
pacman-dev@archlinux.org
http://archlinux.org/mailman/listinfo/pacman-dev
 
Old 06-08-2008, 10:24 PM
"Dan McGee"
 
Default makepkg: add --allsource option

On Fri, Jun 6, 2008 at 8:58 PM, Allan McRae <mcrae_allan@hotmail.com> wrote:
> This supplements the --source option and does nearly the same thing, except
> downloaded source files are included in the archive as well. The sources are
> now packages with a pkgname/ prefix.
>
> Original work by: Dan McGee <dan@archlinux.org>
> Signed-off-by: Allan McRae <mcrae_allan@hotmail.com>

I like this. Any objections? This should make uploading to AUR super
easy as well as allowing individual packagers to satisfy the
requirements of the GPL.

-Dan

_______________________________________________
pacman-dev mailing list
pacman-dev@archlinux.org
http://archlinux.org/mailman/listinfo/pacman-dev
 
Old 06-09-2008, 08:36 AM
Xavier
 
Default makepkg: add --allsource option

On Mon, Jun 9, 2008 at 12:24 AM, Dan McGee <dpmcgee@gmail.com> wrote:
>
> I like this. Any objections? This should make uploading to AUR super
> easy as well as allowing individual packagers to satisfy the
> requirements of the GPL.
>

No objections, but just for clarification, the AUR already supported
the tarballs generated by --source. I only uploaded tarballs generated
this way to AUR.
But well, I find it much better with the pkgname prefix, and it would
also probably make AUR code simpler if it only supported these.
So that is very nice.

_______________________________________________
pacman-dev mailing list
pacman-dev@archlinux.org
http://archlinux.org/mailman/listinfo/pacman-dev
 

Thread Tools




All times are GMT. The time now is 08:14 PM.

VBulletin, Copyright ©2000 - 2014, Jelsoft Enterprises Ltd.
Content Relevant URLs by vBSEO ©2007, Crawlability, Inc.
Copyright 2007 - 2008, www.linux-archive.org