FAQ Search Today's Posts Mark Forums Read
» Video Reviews

» Linux Archive

Linux-archive is a website aiming to archive linux email lists and to make them easily accessible for linux users/developers.


» Sponsor

» Partners

» Sponsor

Go Back   Linux Archive > Gentoo > Gentoo Development

 
 
LinkBack Thread Tools
 
Old 02-06-2011, 09:59 PM
Ulrich Mueller
 
Default Please review: updates for bzr.eclass

Hello,

Please review an updated version of bzr.eclass.

This will fix the following main problems:

- So far, bzr.eclass used "lightweight" checkouts. These are optimised
for a use case very different from ours, where the data it accesses
is considered local. This led to massive performance problems (for
example, 700 MB of transferred data for a one week's update of the
Emacs repository). Therefore, the eclass has been changed to use
branches instead:

1) initial branch with "bzr branch --no-tree",
2) subsequent updates with "bzr pull",
3) export to ${WORKDIR} with "bzr export".

- There was no possibility to share several branches in one project,
which effectively meant that the same commits had to be downloaded
multiple times. The eclass now offers the possibility to use shared
repositories.

Furthermore, I've taken the opportunity to do some smaller cleanups of
the code.

I've tested the eclass with all ebuilds in the portage tree that are
inheriting bzr, and haven't noticed any problems. Especially, the
transition from checkout to branch was smooth in all cases.

The updated eclass is included below. Or, if you prefer a diff, you
can find it here:
<https://overlays.gentoo.org/proj/emacs/changeset?reponame=&new=1606%40emacs-overlay%2Feclass%2Fbzr.eclass&old=1593%40emacs-overlay%2Feclass%2Fbzr.eclass>

Ulrich


# Copyright 1999-2011 Gentoo Foundation
# Distributed under the terms of the GNU General Public License v2
# $Header: /var/cvsroot/gentoo-x86/eclass/bzr.eclass,v 1.10 2010/11/07 22:44:35 fauli Exp $
#
# @ECLASS: bzr.eclass
# @MAINTAINER:
# Jorge Manuel B. S. Vicetto <jmbsvicetto@gentoo.org>,
# Ulrich Mueller <ulm@gentoo.org>,
# Christian Faulhammer <fauli@gentoo.org>,
# Mark Lee <bzr-gentoo-overlay@lazymalevolence.com>,
# and anyone who wants to help
# @BLURB: generic fetching functions for the Bazaar VCS
# @DESCRIPTION:
# The bzr.eclass provides functions to fetch, unpack, patch, and
# bootstrap sources from repositories of the Bazaar distributed version
# control system. The eclass was originally derived from git.eclass.
#
# Note: Just set EBZR_REPO_URI to the URI of the branch and src_unpack()
# of this eclass will export the branch to ${WORKDIR}/${P}.

inherit eutils

EBZR="bzr.eclass"

case "${EAPI:-0}" in
0|1) EXPORT_FUNCTIONS src_unpack ;;
*) EXPORT_FUNCTIONS src_unpack src_prepare ;;
esac

HOMEPAGE="http://bazaar-vcs.org/"
DESCRIPTION="Based on the ${EBZR} eclass"

DEPEND=">=dev-vcs/bzr-2.0.1"
case "${EAPI:-0}" in
0|1) ;;
*) [[ ${EBZR_REPO_URI%%:*} = sftp ]]
&& DEPEND=">=dev-vcs/bzr-2.0.1[sftp]" ;;
esac

# @ECLASS-VARIABLE: EBZR_STORE_DIR
# @DESCRIPTION:
# The directory to store all fetched Bazaar live sources.
: ${EBZR_STORE_DIR:=${PORTAGE_ACTUAL_DISTDIR-${DISTDIR}}/bzr-src}

# @ECLASS-VARIABLE: EBZR_INIT_REPO_CMD
# @DESCRIPTION:
# The Bazaar command to initialise a shared repository.
: ${EBZR_INIT_REPO_CMD:="bzr init-repository --no-trees"}

# @ECLASS-VARIABLE: EBZR_FETCH_CMD
# @DESCRIPTION:
# The Bazaar command to fetch the sources.
: ${EBZR_FETCH_CMD="bzr branch --no-tree"}

# @ECLASS-VARIABLE: EBZR_UPDATE_CMD
# @DESCRIPTION:
# The Bazaar command to update the sources.
: ${EBZR_UPDATE_CMD:="bzr pull"}

# @ECLASS-VARIABLE: EBZR_EXPORT_CMD
# @DESCRIPTION:
# The Bazaar command to export a branch.
: ${EBZR_EXPORT_CMD:="bzr export"}

# @ECLASS-VARIABLE: EBZR_REVNO_CMD
# @DESCRIPTION:
# The Bazaar command to list a revision number of the branch.
: ${EBZR_REVNO_CMD:="bzr revno"}

# @ECLASS-VARIABLE: EBZR_OPTIONS
# @DEFAULT_UNSET
# @DESCRIPTION:
# The options passed to the fetch and update commands.

# @ECLASS-VARIABLE: EBZR_REPO_URI
# @DEFAULT_UNSET
# @DESCRIPTION:
# The repository URI for the source package.
#
# Note: If the ebuild uses an sftp:// URI, then in EAPI 0 or 1 it must
# make sure that dev-vcs/bzr was built with USE="sftp". In EAPI 2 or
# later, the eclass will depend on dev-vcs/bzr[sftp].

# @ECLASS-VARIABLE: EBZR_BOOTSTRAP
# @DEFAULT_UNSET
# @DESCRIPTION:
# Bootstrap script or command like autogen.sh or etc.

# @ECLASS-VARIABLE: EBZR_PATCHES
# @DEFAULT_UNSET
# @DESCRIPTION:
# bzr.eclass can apply patches in bzr_bootstrap(). You can use regular
# expressions in this variable like *.diff or *.patch and the like.
# Note: These patches will be applied before EBZR_BOOTSTRAP is processed.
#
# Patches are searched both in ${PWD} and ${FILESDIR}. If not found in
# either location, the installation dies.

# @ECLASS-VARIABLE: EBZR_PROJECT
# @DESCRIPTION:
# The project name of your ebuild. Normally, the branch will be stored
# in the ${EBZR_STORE_DIR}/${EBZR_PROJECT} directory.
#
# If EBZR_BRANCH is set (see below), then a shared repository will be
# created in that directory, and the branch will be located in
# ${EBZR_STORE_DIR}/${EBZR_PROJECT}/${EBZR_BRANCH}.
: ${EBZR_PROJECT:=${PN}}

# @ECLASS-VARIABLE: EBZR_BRANCH
# @DEFAULT_UNSET
# @DESCRIPTION:
# The directory where to store the branch within a shared repository,
# relative to ${EBZR_STORE_DIR}/${EBZR_PROJECT}.
#
# This variable should be set if there are several live ebuilds for
# different branches of the same upstream project. The branches can
# then share the same repository in EBZR_PROJECT, which will save both
# data traffic volume and disk space.
#
# If there is only a live ebuild for one single branch, EBZR_BRANCH
# needs not be set. In this case, the branch will be stored in a
# stand-alone repository directly in EBZR_PROJECT.

# @ECLASS-VARIABLE: EBZR_REVISION
# @DEFAULT_UNSET
# @DESCRIPTION:
# Revision to fetch, defaults to the latest
# (see http://bazaar-vcs.org/BzrRevisionSpec or bzr help revisionspec).

# @ECLASS-VARIABLE: EBZR_OFFLINE
# @DESCRIPTION:
# Set this variable to a non-empty value to disable automatic updating
# of a bzr source tree. This is intended to be set outside the ebuild
# by users.
: ${EBZR_OFFLINE:=${ESCM_OFFLINE}}

# @FUNCTION: bzr_initial_fetch
# @DESCRIPTION:
# Internal function, retrieves the source code from a repository for the
# first time, using ${EBZR_FETCH_CMD}.
bzr_initial_fetch() {
local repo_uri=$1 branch_dir=$2

if [[ -n "${EBZR_OFFLINE}" ]]; then
ewarn "EBZR_OFFLINE cannot be used when there is no local branch yet."
fi

# fetch branch
einfo "bzr branch start -->"
einfo " repository: ${repo_uri} => ${branch_dir}"

${EBZR_FETCH_CMD} ${EBZR_OPTIONS} "${repo_uri}" "${branch_dir}"
|| die "${EBZR}: can't branch from ${repo_uri}"
}

# @FUNCTION: bzr_update
# @DESCRIPTION:
# Internal function, updates the source code from a repository, using
# ${EBZR_UPDATE_CMD}.
bzr_update() {
local repo_uri=$1 branch_dir=$2

if [[ -n "${EBZR_OFFLINE}" ]]; then
einfo "skipping bzr pull -->"
einfo " repository: ${repo_uri}"
else
# update branch
einfo "bzr pull start -->"
einfo " repository: ${repo_uri}"

pushd "${branch_dir}" > /dev/null
|| die "${EBZR}: can't chdir to ${branch_dir}"
${EBZR_UPDATE_CMD} ${EBZR_OPTIONS} "${repo_uri}"
|| die "${EBZR}: can't pull from ${repo_uri}"
popd > /dev/null
fi
}

# @FUNCTION: bzr_fetch
# @DESCRIPTION:
# Wrapper function to fetch sources from a Bazaar repository with
# bzr branch or bzr pull, depending on whether there is an existing
# working copy.
bzr_fetch() {
local repo_dir branch_dir

# EBZR_REPO_URI is empty.
[[ ${EBZR_REPO_URI} ]] || die "${EBZR}: EBZR_REPO_URI is empty"

if [[ ! -d ${EBZR_STORE_DIR} ]] ; then
local save_sandbox_write=${SANDBOX_WRITE}
addwrite /
mkdir -p "${EBZR_STORE_DIR}"
|| die "${EBZR}: can't mkdir ${EBZR_STORE_DIR}"
SANDBOX_WRITE=${save_sandbox_write}
fi

pushd "${EBZR_STORE_DIR}" > /dev/null
|| die "${EBZR}: can't chdir to ${EBZR_STORE_DIR}"

repo_dir=${EBZR_STORE_DIR}/${EBZR_PROJECT}
branch_dir=${repo_dir}${EBZR_BRANCH:+/${EBZR_BRANCH}}

addwrite "${EBZR_STORE_DIR}"

# Clean up if the existing local copy is a checkout (as was the case
# with an older version of bzr.eclass).
if [[ ${EBZR_FETCH_CMD} != *checkout* && -d ${repo_dir}/.bzr/checkout ]]
then
ewarn "removing old bzr checkout"
rm -rf "${repo_dir}"
fi

if [[ ! -d ${branch_dir}/.bzr ]]; then
if [[ ${repo_dir} != "${branch_dir}" && ! -d ${repo_dir}/.bzr ]]; then
einfo "creating shared bzr repository"
${EBZR_INIT_REPO_CMD} "${repo_dir}"
|| die "${EBZR}: can't create shared repository"
fi

if [[ -z ${EBZR_MIRROR_URI} ]]; then
bzr_initial_fetch "${EBZR_REPO_URI}" "${branch_dir}"
else
# Workaround for faster initial download. This clones the
# branch from a fast mirror (which may be out of date), and
# subsequently pulls from the slow original repository.
bzr_initial_fetch "${EBZR_MIRROR_URI}" "${branch_dir}"
EBZR_UPDATE_CMD="${EBZR_UPDATE_CMD} --remember --overwrite"
EBZR_OFFLINE="" bzr_update "${EBZR_REPO_URI}" "${branch_dir}"
fi
else
bzr_update "${EBZR_REPO_URI}" "${branch_dir}"
fi

cd "${branch_dir}" || die "${EBZR}: can't chdir to ${branch_dir}"
export EBZR_REVNO=$(${EBZR_REVNO_CMD})

einfo "exporting ..."
${EBZR_EXPORT_CMD} ${EBZR_REVISION:+-r ${EBZR_REVISION}}
"${WORKDIR}/${P}" . || die "${EBZR}: export failed"
einfo "revision ${EBZR_REVISION:-${EBZR_REVNO}} is now in ${WORKDIR}/${P}"

popd > /dev/null
}

# @FUNCTION: bzr_bootstrap
# @DESCRIPTION:
# Apply patches in ${EBZR_PATCHES} and run ${EBZR_BOOTSTRAP} if specified.
bzr_bootstrap() {
local patch lpatch

pushd "${S}" > /dev/null || die "${EBZR}: can't chdir to ${S}"

if [[ -n ${EBZR_PATCHES} ]] ; then
einfo "apply patches -->"

for patch in ${EBZR_PATCHES} ; do
if [[ -f ${patch} ]] ; then
epatch ${patch}
else
# This loop takes care of wildcarded patches given via
# EBZR_PATCHES in an ebuild
for lpatch in "${FILESDIR}"/${patch} ; do
if [[ -f ${lpatch} ]] ; then
epatch ${lpatch}
else
die "${EBZR}: ${patch} is not found"
fi
done
fi
done
fi

if [[ -n ${EBZR_BOOTSTRAP} ]] ; then
einfo "begin bootstrap -->"

if [[ -f ${EBZR_BOOTSTRAP} ]] && [[ -x ${EBZR_BOOTSTRAP} ]] ; then
einfo " bootstrap with a file: ${EBZR_BOOTSTRAP}"
"./${EBZR_BOOTSTRAP}"
|| die "${EBZR}: can't execute EBZR_BOOTSTRAP"
else
einfo " bootstrap with commands: ${EBZR_BOOTSTRAP}"
"${EBZR_BOOTSTRAP}"
|| die "${EBZR}: can't eval EBZR_BOOTSTRAP"
fi
fi

popd > /dev/null
}

# @FUNCTION: bzr_src_unpack
# @DESCRIPTION:
# Default src_unpack(), calls bzr_fetch. For EAPIs 0 and 1, also calls
# bzr_src_prepare.
bzr_src_unpack() {
bzr_fetch
case "${EAPI:-0}" in
0|1) bzr_src_prepare ;;
esac
}

# @FUNCTION: bzr_src_prepare
# @DESCRIPTION:
# Default src_prepare(), calls bzr_bootstrap.
bzr_src_prepare() {
bzr_bootstrap
}
 
Old 02-07-2011, 06:51 PM
James Cloos
 
Default Please review: updates for bzr.eclass

>>>>> "UM" == Ulrich Mueller <ulm@gentoo.org> writes:

UM> 1) initial branch with "bzr branch --no-tree",
UM> 2) subsequent updates with "bzr pull",
UM> 3) export to ${WORKDIR} with "bzr export".

I applaud those changes, but please mv(1) old repos rather than rm(1)ing
them; bandwidth is often more of a premium than storage and a manual
cleanup is better than a magic behind-the-back permanent deletion. This
follows the same principles as portage does in $DISTFILES.

Thank you.

-JimC
--
James Cloos <cloos@jhcloos.com> OpenPGP: 1024D/ED7DAEA6
 
Old 02-07-2011, 08:04 PM
Ulrich Mueller
 
Default Please review: updates for bzr.eclass

>>>>> On Mon, 07 Feb 2011, James Cloos wrote:

UM> 1) initial branch with "bzr branch --no-tree",
UM> 2) subsequent updates with "bzr pull",
UM> 3) export to ${WORKDIR} with "bzr export".

> I applaud those changes,

Thanks.

> but please mv(1) old repos rather than rm(1)ing them; bandwidth is
> often more of a premium than storage and a manual cleanup is better
> than a magic behind-the-back permanent deletion. This follows the
> same principles as portage does in $DISTFILES.

Is the following better?
<https://overlays.gentoo.org/proj/emacs/changeset?reponame=&new=1609%40emacs-overlay%2Feclass%2Fbzr.eclass&old=1608%40emacs-overlay%2Feclass%2Fbzr.eclass>

Ulrich
 
Old 02-08-2011, 06:34 PM
James Cloos
 
Default Please review: updates for bzr.eclass

>>>>> "UM" == Ulrich Mueller <ulm@gentoo.org> writes:

>> but please mv(1) old repos rather than rm(1)ing them;

UM> Is the following better?
UM> <https://overlays.gentoo.org/proj/emacs/changeset?reponame=&new=1609%40emacs-overlay%2Feclass%2Fbzr.eclass&old=1608%40emacs-overlay%2Feclass%2Fbzr.eclass>

That looks perfect.

Thanks!

-JimC
--
James Cloos <cloos@jhcloos.com> OpenPGP: 1024D/ED7DAEA6
 

Thread Tools




All times are GMT. The time now is 07:01 PM.

VBulletin, Copyright ©2000 - 2014, Jelsoft Enterprises Ltd.
Content Relevant URLs by vBSEO ©2007, Crawlability, Inc.
Copyright 2007 - 2008, www.linux-archive.org