VCS used for development of portage
 

Hello!


Is moving portage development over to Git planned anytime soon?
Anything keeping you from the move?

Anything I can do to speed it up?



Sebastian

VCS used for development of portage
 

On 02/26/2010 07:18 PM, Sebastian Pipping wrote:
> Hello!
>
>
> Is moving portage development over to Git planned anytime soon?

Yes, we've been discussing it on this bug:

http://bugs.gentoo.org/show_bug.cgi?id=196025

> Anything keeping you from the move?

Well, the repository layout is somewhat non-trivial, but I think we
have a plan that will work. See discussion on bug 196025.

> Anything I can do to speed it up?

Robbin would know better than me.
--
Thanks,
Zac

VCS used for development of portage
 

Hello!


I've been playing with Git conversions of the portage repository.
The current "demo" conversion from anon SVN is up here:

http://git.goodpoint.de/?p=portage.git;a=summary

NOTE: Do not use it for development, yet - it's a demo!


At the end you can find the script I made for the conversion.

Notes on the script:
- An author map "portage-author-map.txt" is required, get from [1]
(Note to zmedico/robbat2: fixed version, do not use the older one)
- If updated, the latest version of this script is up at [2]
- It works with anon SVN and rsync. For the final run
- use developer SVN instead
- be sure to remove --rewrite-root !
- The final repository will be about 22MB in size
- Main conversion takes about two hours on my machine

Next we need to decide on
- a final location (git.overlays.gentoo.org/proj/portage?)
- when we convert (and freeze SVN)
- who runs the final conversion (zmedico, roobat2, me?)



Sebastian

[1] http://www.hartwork.org/public/portage-author-map.txt
[2] http://www.hartwork.org/public/portage-svn-to-git.sh


================================================== ===============
#!/usr/bin/env bash
starttime=$(date +'%Y-%m-%d--%H-%M-%S')
output_dir="portage-git-repo--${starttime}"

# Open logged subshell
(

# Print executed bash commands
set -x

# Rip/sync anon SVN using rsync
rsync -r rsync://anonvcs.gentoo.org/vcs-public-svnroot/portage/
portage-anon-svn-repo-dump/ || exit 1

# Init git-svn repo, note double use of --trunk
[ -d "${output_dir}" ] && exit 1
git svn init file://${PWD}/portage-anon-svn-repo-dump/
--rewrite-root=svn://anonsvn.gentoo.org/portage/
--trunk=main/trunk --tags=main/tags --branches=main/branches
"${output_dir}"
cd "${output_dir}" || exit 1

# Convert commits
git config svn.authorsfile ../portage-author-map.txt
time git svn fetch || exit 1

# Make real Git tags from remotes/tags/* (two special cases)
for branch_tag in $(git branch -r | grep 'tags/'
| fgrep -v 'tags/portage-2.1_pre5'); do
tag=$(sed 's|^tags/||' <<<"${branch_tag}")
git tag "v${tag}" "remotes/${branch_tag}" || exit 1
done
git tag 'trunk@1888' 'remotes/trunk@1888' || exit 1

# Make local branches from remotes/* (excluding tags and trunk)
for branch in $(git branch -r | grep -v 'tags|trunk'); do
git checkout -b "${branch}" "remotes/${branch}" || exit 1
done
git checkout master || exit 1

# Reduce size of repository
dotgitsize() { du --human --total .git | tail -n 1; }
dotgitsize
git gc --aggressive
dotgitsize

# Wipe all traces of SVN
git config --remove-section 'svn-remote.svn'
git config --remove-section 'svn'
rm -R .git/svn
rm -R .git/logs/refs/remotes
for file in .git/info/refs .git/packed-refs ; do
sed -e '/remotes//d' -i "${file}"
done

# Hide executed bash commands
set +x


cat <<INFO

DONE.


NEXT STEPS:
# cd "${output_dir}"

Verify that branches and tags
1. make sense
2. are unambiguous
3. are free of SVN
# git branch -a
# git show-branch --list
# git tag -l

Push full repository
# git remote add ${remote_name} ${push_url}
# git push --mirror ${remote_name}
INFO

) |& tee conversion--${starttime}.log
#================================================= ===============

VCS used for development of portage
 

On 03/02/2010 03:22 PM, Sebastian Pipping wrote:
> Hello!
>
>
> I've been playing with Git conversions of the portage repository.
> The current "demo" conversion from anon SVN is up here:
>
> http://git.goodpoint.de/?p=portage.git;a=summary
>
> NOTE: Do not use it for development, yet - it's a demo!

It looks very nice to me. I noticed that it preserved continuity
when branches got moved around, so the history seems like it will be
fully intact. Great job!
--
Thanks,
Zac

VCS used for development of portage
 

On 03/05/10 04:58, Zac Medico wrote:
>> http://git.goodpoint.de/?p=portage.git;a=summary
>>
>> NOTE: Do not use it for development, yet - it's a demo!
>
> It looks very nice to me. I noticed that it preserved continuity
> when branches got moved around, so the history seems like it will be
> fully intact. Great job!

Still, maybe we should not jump on this version yet:
- with svn2git we could split it to several repositories easily
(see [1] for status you on related experiments)
- neither svn2git nor git-svn seem to support proper conversion of
changes to svn:ignore

Summer of code could help about the latter:
http://en.gentoo-wiki.com/wiki/Google_Summer_of_Code_2010_ideas#Add_support_for_s vn:ignore_to_svn2git

But pushing the conversion further into the future could also be a
trade-off reducing efficiency and the number of contributions.

I don't feel like proposing anything on that matter at the moment. With
that said: what do you and Robin think?



Sebastian


[1] http://bugs.gentoo.org/show_bug.cgi?id=196025#c41

VCS used for development of portage
 

On Fri, Mar 05, 2010 at 04:33:14PM +0100, Sebastian Pipping wrote:
> I don't feel like proposing anything on that matter at the moment. With
> that said: what do you and Robin think?
Here's a related question.

Did the previous CVS -> SVN question generate the svn:ignore files from
.cvsignore, or simply discard them?

In either case, I'm starting to wonder if the change is just trivial
enough to get done in svn2git or git-svn directly. I think other
properties are already there.

--
Robin Hugh Johnson
Gentoo Linux: Developer, Trustee & Infrastructure Lead
E-Mail : robbat2@gentoo.org
GnuPG FP : 11AC BA4F 4778 E3F6 E4ED F38E B27B 944E 3488 4E85

VCS used for development of portage
 

On Saturday 06 March 2010, Robin H. Johnson wrote:
> In either case, I'm starting to wonder if the change is just trivial
> enough to get done in svn2git or git-svn directly. I think other
> properties are already there.

The git-svn man page states that a transformation is not trivial. I
don't know if in general, svn:ignore is a subset of gitignore.
Apprarantly, git-svnimport can handle this transformation, but it will
probably come with other problems.

In my opinion, we should just ignore this small history loss and after
the git conversion, commit a .gitignore file with the latest ignore
rules to the active branches. But that is just my opinion, and for the
actual portage developers to decide.



Robert

VCS used for development of portage
 

On 03/07/2010 01:49 AM, Robert Buchholz wrote:
> On Saturday 06 March 2010, Robin H. Johnson wrote:
>> In either case, I'm starting to wonder if the change is just trivial
>> enough to get done in svn2git or git-svn directly. I think other
>> properties are already there.
>
> The git-svn man page states that a transformation is not trivial. I
> don't know if in general, svn:ignore is a subset of gitignore.
> Apprarantly, git-svnimport can handle this transformation, but it will
> probably come with other problems.
>
> In my opinion, we should just ignore this small history loss and after
> the git conversion, commit a .gitignore file with the latest ignore
> rules to the active branches. But that is just my opinion, and for the
> actual portage developers to decide.

I think that's reasonable. Mainly, the files that need to be ignored
are *.py[co] and doc/*.html.
--
Thanks,
Zac