Hello!


Is moving portage development over to Git planned anytime soon?
Anything keeping you from the move?

Anything I can do to speed it up?



Sebastian

On 02/26/2010 07:18 PM, Sebastian Pipping wrote:
> Hello!
>
>
> Is moving portage development over to Git planned anytime soon?

Yes, we've been discussing it on this bug:

http://bugs.gentoo.org/show_bug.cgi?id=196025

> Anything keeping you from the move?

Well, the repository layout is somewhat non-trivial, but I think we
have a plan that will work. See discussion on bug 196025.

> Anything I can do to speed it up?

Robbin would know better than me.
--
Thanks,
Zac

Hello!


I've been playing with Git conversions of the portage repository.
The current "demo" conversion from anon SVN is up here:

http://git.goodpoint.de/?p=portage.git;a=summary

NOTE: Do not use it for development, yet - it's a demo!


At the end you can find the script I made for the conversion.

Notes on the script:
- An author map "portage-author-map.txt" is required, get from [1]
(Note to zmedico/robbat2: fixed version, do not use the older one)
- If updated, the latest version of this script is up at [2]
- It works with anon SVN and rsync. For the final run
- use developer SVN instead
- be sure to remove --rewrite-root !
- The final repository will be about 22MB in size
- Main conversion takes about two hours on my machine

Next we need to decide on
- a final location (git.overlays.gentoo.org/proj/portage?)
- when we convert (and freeze SVN)
- who runs the final conversion (zmedico, roobat2, me?)



Sebastian

[1] http://www.hartwork.org/public/portage-author-map.txt
[2] http://www.hartwork.org/public/portage-svn-to-git.sh


================================================== ===============
#!/usr/bin/env bash
starttime=$(date +'%Y-%m-%d--%H-%M-%S')
output_dir="portage-git-repo--${starttime}"

# Open logged subshell
(

# Print executed bash commands
set -x

# Rip/sync anon SVN using rsync
rsync -r rsync://anonvcs.gentoo.org/vcs-public-svnroot/portage/
portage-anon-svn-repo-dump/ || exit 1

# Init git-svn repo, note double use of --trunk
[ -d "${output_dir}" ] && exit 1
git svn init file://${PWD}/portage-anon-svn-repo-dump/
--rewrite-root=svn://anonsvn.gentoo.org/portage/
--trunk=main/trunk --tags=main/tags --branches=main/branches
"${output_dir}"
cd "${output_dir}" || exit 1

# Convert commits
git config svn.authorsfile ../portage-author-map.txt
time git svn fetch || exit 1

# Make real Git tags from remotes/tags/* (two special cases)
for branch_tag in $(git branch -r | grep 'tags/'
| fgrep -v 'tags/portage-2.1_pre5'); do
tag=$(sed 's|^tags/||' <<<"${branch_tag}")
git tag "v${tag}" "remotes/${branch_tag}" || exit 1
done
git tag 'trunk@1888' 'remotes/trunk@1888' || exit 1

# Make local branches from remotes/* (excluding tags and trunk)
for branch in $(git branch -r | grep -v 'tags|trunk'); do
git checkout -b "${branch}" "remotes/${branch}" || exit 1
done
git checkout master || exit 1

# Reduce size of repository
dotgitsize() { du --human --total .git | tail -n 1; }
dotgitsize
git gc --aggressive
dotgitsize

# Wipe all traces of SVN
git config --remove-section 'svn-remote.svn'
git config --remove-section 'svn'
rm -R .git/svn
rm -R .git/logs/refs/remotes
for file in .git/info/refs .git/packed-refs ; do
sed -e '/remotes//d' -i "${file}"
done

# Hide executed bash commands
set +x


cat <<INFO

DONE.


NEXT STEPS:
# cd "${output_dir}"

Verify that branches and tags
1. make sense
2. are unambiguous
3. are free of SVN
# git branch -a
# git show-branch --list
# git tag -l

Push full repository
# git remote add ${remote_name} ${push_url}
# git push --mirror ${remote_name}
INFO

) |& tee conversion--${starttime}.log
#================================================= ===============

On 03/02/2010 03:22 PM, Sebastian Pipping wrote:
> Hello!
>
>
> I've been playing with Git conversions of the portage repository.
> The current "demo" conversion from anon SVN is up here:
>
> http://git.goodpoint.de/?p=portage.git;a=summary
>
> NOTE: Do not use it for development, yet - it's a demo!

It looks very nice to me. I noticed that it preserved continuity
when branches got moved around, so the history seems like it will be
fully intact. Great job!
--
Thanks,
Zac

On 03/05/10 04:58, Zac Medico wrote:
>> http://git.goodpoint.de/?p=portage.git;a=summary
>>
>> NOTE: Do not use it for development, yet - it's a demo!
>
> It looks very nice to me. I noticed that it preserved continuity
> when branches got moved around, so the history seems like it will be
> fully intact. Great job!

Still, maybe we should not jump on this version yet:
- with svn2git we could split it to several repositories easily
(see [1] for status you on related experiments)
- neither svn2git nor git-svn seem to support proper conversion of
changes to svn:ignore

Summer of code could help about the latter:
http://en.gentoo-wiki.com/wiki/Google_Summer_of_Code_2010_ideas#Add_support_for_s vn:ignore_to_svn2git

But pushing the conversion further into the future could also be a
trade-off reducing efficiency and the number of contributions.

I don't feel like proposing anything on that matter at the moment. With
that said: what do you and Robin think?



Sebastian


[1] http://bugs.gentoo.org/show_bug.cgi?id=196025#c41

On Fri, Mar 05, 2010 at 04:33:14PM +0100, Sebastian Pipping wrote:
> I don't feel like proposing anything on that matter at the moment. With
> that said: what do you and Robin think?
Here's a related question.

Did the previous CVS -> SVN question generate the svn:ignore files from
.cvsignore, or simply discard them?

In either case, I'm starting to wonder if the change is just trivial
enough to get done in svn2git or git-svn directly. I think other
properties are already there.

--
Robin Hugh Johnson
Gentoo Linux: Developer, Trustee & Infrastructure Lead
E-Mail : robbat2@gentoo.org
GnuPG FP : 11AC BA4F 4778 E3F6 E4ED F38E B27B 944E 3488 4E85

On Saturday 06 March 2010, Robin H. Johnson wrote:
> In either case, I'm starting to wonder if the change is just trivial
> enough to get done in svn2git or git-svn directly. I think other
> properties are already there.

The git-svn man page states that a transformation is not trivial. I
don't know if in general, svn:ignore is a subset of gitignore.
Apprarantly, git-svnimport can handle this transformation, but it will
probably come with other problems.

In my opinion, we should just ignore this small history loss and after
the git conversion, commit a .gitignore file with the latest ignore
rules to the active branches. But that is just my opinion, and for the
actual portage developers to decide.



Robert

On 03/07/2010 01:49 AM, Robert Buchholz wrote:
> On Saturday 06 March 2010, Robin H. Johnson wrote:
>> In either case, I'm starting to wonder if the change is just trivial
>> enough to get done in svn2git or git-svn directly. I think other
>> properties are already there.
>
> The git-svn man page states that a transformation is not trivial. I
> don't know if in general, svn:ignore is a subset of gitignore.
> Apprarantly, git-svnimport can handle this transformation, but it will
> probably come with other problems.
>
> In my opinion, we should just ignore this small history loss and after
> the git conversion, commit a .gitignore file with the latest ignore
> rules to the active branches. But that is just my opinion, and for the
> actual portage developers to decide.

I think that's reasonable. Mainly, the files that need to be ignored
are *.py[co] and doc/*.html.
--
Thanks,
Zac