FAQ Search Today's Posts Mark Forums Read
» Video Reviews

» Linux Archive

Linux-archive is a website aiming to archive linux email lists and to make them easily accessible for linux users/developers.


» Sponsor

» Partners

» Sponsor

Go Back   Linux Archive > Redhat > Fedora Development

 
 
LinkBack Thread Tools
 
Old 12-03-2010, 11:34 PM
Jesse Keating
 
Default Request for comment: Potential change to dist-git branch structure

I'm working on fixing a few long standing bugs in fedpkg that have to do
with our branch structure
(https://bugzilla.redhat.com/show_bug.cgi?id=619979 and
https://bugzilla.redhat.com/show_bug.cgi?id=622592).

This has me examining our branch structure again and trying to remember
why exactly I chose it (obviously I did a poor job of documenting that...).

The original thought was to have top level branches that are named after
distribution releases, eg "f14", "f15", "el5". Then we would force
branches of those branches use a naming structure of "f14/topic". The
reason was so that our tooling could look at the name of the branch and
easily work back to the "f14" part. This would work even if it was
"f14/user/fred/topic/mybranch" or other such craziness. When I went to
test this, I realized that git won't allow you to have both "f14" and
"f14/topic" as branches, because of the way the git metadata works on
the filesystem. When I encountered this, I made "f14/master" become the
top level branch, and then "f14/somethingelse" could coexist.
Unfortunately I also wanted to keep things easy for users and tried to
maintain tooling that would allow you to just say "f14". This didn't
get enough real world testing and in hindsight was a bad idea. Things
go wrong quickly in git if your local branch name doesn't match the
remote branch name.

When thinking about the above, and the two bugs I'm working on, I
realized that we don't have any real strong need to be using "/" as a
delineator. It makes some code easier, but makes other things more
complex and difficult. So what if we changed it?

What I'm thinking about now is switching from / to - as a delineator.
This would improve a couple things. First, we could achieve upstream
top level branch names that are short and simple: "f14", "f15",
"master". We can have branches that build upon those names:
"f14-rebase", "f15-cve223", "f15-user-jkeating-private". We could keep
the simple fedpkg tooling that allows users to just type "f14" and the
like to reference a branch, and now the local branch will match the name
of the remote branch.

As for the first two bugs I mentioned, it doesn't directly help them.
However I would feel better about telling people that their local
branches must follow a naming scheme of <release>-<something> and then
we could easily guess what release the local branch is for if it isn't
tracking a remote branch. However the bug about what to do if there are
no remote branches is really not touched by any of this, it just got me
thinking about branches

What kind of user impact would this have?

My hope is that the impact would be minimal. Git allows branch renames,
and can successfully rename "f14/master" to "f14". All the history is
renamed. We should be able to do this without an outage of the git server.

The ACL system will need a slight tweaking, and a regeneration of the
ACLs but that is fairly quick and minor to accomplish.

However there will be some issues client side. We will not be able to
make use of git's symbolic-ref feature of "aliasing" a branch. We
cannot make "f14/master" an alias for "f14", again because of the
filesystem layout issues. These issues rear their head once again when
a client does a pull of an already checked out repo that had branches.
Because there was already a f14/master, when the client sees a new
branch just named "f14" it will fail to create it. Git has a command
that will fix this "git remote prune origin". That will remove the
local reference to f14/master and a subsequent pull sees the creation of
the "f14" branch happen successfully. However, if a user had a local
branch of f14 or f14/master they will be left with mismatched
.git/config entries. In this case it's easiest to delete the local
branch (git branch -d f14) and check it out again. If there are changes
on the branch one can fix the config to point it to the right upstream
location.

Also we would need to get a new fedpkg into the hands of all the
developers that handles the new branchnames. We could do a build that
handles both the oldnames and the new and have it out and available for
a reasonable period of time before we make the switch.

So, some pain, for some pretty good gain. This time around I can setup
pkgs.stg with this branch configuration and builds of fedpkg to use it
for a more through testing before rolling it into production.

So please, tell me what you think!

--
Jesse Keating
Fedora -- Freedom² is a feature!
identi.ca: http://identi.ca/jkeating
--
devel mailing list
devel@lists.fedoraproject.org
https://admin.fedoraproject.org/mailman/listinfo/devel
 
Old 12-04-2010, 04:33 AM
Garrett Holmstrom
 
Default Request for comment: Potential change to dist-git branch structure

On 12/3/2010 18:34, Jesse Keating wrote:
> The original thought was to have top level branches that are named after
> distribution releases, eg "f14", "f15", "el5". Then we would force
> branches of those branches use a naming structure of "f14/topic". The
> reason was so that our tooling could look at the name of the branch and
> easily work back to the "f14" part. This would work even if it was
> "f14/user/fred/topic/mybranch" or other such craziness. When I went to
> test this, I realized that git won't allow you to have both "f14" and
> "f14/topic" as branches, because of the way the git metadata works on
> the filesystem. When I encountered this, I made "f14/master" become the
> top level branch, and then "f14/somethingelse" could coexist.
> Unfortunately I also wanted to keep things easy for users and tried to
> maintain tooling that would allow you to just say "f14". This didn't
> get enough real world testing and in hindsight was a bad idea. Things
> go wrong quickly in git if your local branch name doesn't match the
> remote branch name.
>
> When thinking about the above, and the two bugs I'm working on, I
> realized that we don't have any real strong need to be using "/" as a
> delineator. It makes some code easier, but makes other things more
> complex and difficult. So what if we changed it?
>
> What I'm thinking about now is switching from / to - as a delineator.
> This would improve a couple things. First, we could achieve upstream
> top level branch names that are short and simple: "f14", "f15",
> "master". We can have branches that build upon those names:
> "f14-rebase", "f15-cve223", "f15-user-jkeating-private". We could keep
> the simple fedpkg tooling that allows users to just type "f14" and the
> like to reference a branch, and now the local branch will match the name
> of the remote branch.

Yes, please! Getting rid of the '/' strangeness ought to make things a
little easier to understand and use across the board. I suspect that
few enough packages use shared feature or bugfix branches that a
transition won't trip up very many people. Perhaps a hook on Fedora's
repositories that prints transition instructions when one attempts to
push to old-style branches in conjunction with a fedpkg command that
attempts to migrate existing local branches and remotes would help somewhat.

> As for the first two bugs I mentioned, it doesn't directly help them.
> However I would feel better about telling people that their local
> branches must follow a naming scheme of<release>-<something> and then
> we could easily guess what release the local branch is for if it isn't
> tracking a remote branch. However the bug about what to do if there are
> no remote branches is really not touched by any of this, it just got me
> thinking about branches

Why tie branch names down to specific releases? While that scheme makes
it easy for fedpkg to guess what release to attempt to build against
when one only cares about one release, it makes little sense to call a
branch "f14-rh123456" when in reality that branch will merge into "f13"
as well as "f14".
--
devel mailing list
devel@lists.fedoraproject.org
https://admin.fedoraproject.org/mailman/listinfo/devel
 
Old 12-04-2010, 09:19 AM
Matej Cepl
 
Default Request for comment: Potential change to dist-git branch structure

Dne 4.12.2010 06:33, Garrett Holmstrom napsal(a):
> Why tie branch names down to specific releases? While that scheme makes
> it easy for fedpkg to guess what release to attempt to build against
> when one only cares about one release, it makes little sense to call a
> branch "f14-rh123456" when in reality that branch will merge into "f13"
> as well as "f14".

+1 Why not just get out of all this silly business and leave branches to
be whatever we want them to be as God^H^H^HLinus intended them to be?
Really, branch rhbz1234567 doesn't have to have any relation to any
particular distribution (we usually don't clone Fedora bugs to all
distros where they happen, and that's The Right Thing).

Related issue I have with the Fedora git repositories is that one cannot
remove any branch once it is created. After I have created in bitlbee
repo two topic branches, only to find out that I cannot remove them
after the merge. I can understand need for documenting development of
the distribution, but cannot we lock just SOME branches (probably master
+ f* ones)? In this situation, I have moved my topical branches to
gitorious, where I can do whatever I want to do with them.

Best,

Matěj

--
devel mailing list
devel@lists.fedoraproject.org
https://admin.fedoraproject.org/mailman/listinfo/devel
 
Old 12-04-2010, 09:31 AM
Kalev Lember
 
Default Request for comment: Potential change to dist-git branch structure

On 12/04/2010 12:19 PM, Matej Cepl wrote:
> Related issue I have with the Fedora git repositories is that one cannot
> remove any branch once it is created. After I have created in bitlbee
> repo two topic branches, only to find out that I cannot remove them
> after the merge. I can understand need for documenting development of
> the distribution, but cannot we lock just SOME branches (probably master
> + f* ones)? In this situation, I have moved my topical branches to
> gitorious, where I can do whatever I want to do with them.

I think it makes sense to disallow removing official branches (f13, f14,
master) to make sure people don't change the history of branches which
are used for release builds.

On the other hand, for topic branches and personal branches I would very
much like to be able to do non fast-forward pushes and to be able to
delete them. With git it's common to create branches for preparing a
feature and merge them into the official branch once the feature is
ready. Allowing non fast-forward pushes in unofficial branches would
make it much easier to prepare a perfect history before merging it into
the official branches.


--
Regards,
Kalev
--
devel mailing list
devel@lists.fedoraproject.org
https://admin.fedoraproject.org/mailman/listinfo/devel
 
Old 12-04-2010, 02:24 PM
Severin Gehwolf
 
Default Request for comment: Potential change to dist-git branch structure

----- "Jesse Keating" <jkeating@redhat.com> wrote:

> I'm working on fixing a few long standing bugs in fedpkg that have to
> do
> with our branch structure
> (https://bugzilla.redhat.com/show_bug.cgi?id=619979 and
> https://bugzilla.redhat.com/show_bug.cgi?id=622592).
>
> This has me examining our branch structure again and trying to
> remember
> why exactly I chose it (obviously I did a poor job of documenting
> that...).
>
> The original thought was to have top level branches that are named
> after
> distribution releases, eg "f14", "f15", "el5". Then we would force
> branches of those branches use a naming structure of "f14/topic".
> The
> reason was so that our tooling could look at the name of the branch
> and
> easily work back to the "f14" part. This would work even if it was
> "f14/user/fred/topic/mybranch" or other such craziness. When I went
> to
> test this, I realized that git won't allow you to have both "f14" and
> "f14/topic" as branches, because of the way the git metadata works on
> the filesystem. When I encountered this, I made "f14/master" become
> the
> top level branch, and then "f14/somethingelse" could coexist.
> Unfortunately I also wanted to keep things easy for users and tried
> to
> maintain tooling that would allow you to just say "f14". This didn't
> get enough real world testing and in hindsight was a bad idea.
> Things
> go wrong quickly in git if your local branch name doesn't match the
> remote branch name.
>
> When thinking about the above, and the two bugs I'm working on, I
> realized that we don't have any real strong need to be using "/" as a
> delineator. It makes some code easier, but makes other things more
> complex and difficult. So what if we changed it?
>
> What I'm thinking about now is switching from / to - as a delineator.
> This would improve a couple things. First, we could achieve upstream
> top level branch names that are short and simple: "f14", "f15",
> "master".

I'm all for this change. I believe the majority of users will
have an outcome with f14/f13-kind of branches.

> We can have branches that build upon those names:
> "f14-rebase", "f15-cve223", "f15-user-jkeating-private". We could
> keep
> the simple fedpkg tooling that allows users to just type "f14" and
> the
> like to reference a branch, and now the local branch will match the
> name
> of the remote branch.
>
> As for the first two bugs I mentioned, it doesn't directly help them.
> However I would feel better about telling people that their local
> branches must follow a naming scheme of <release>-<something> and
> then
> we could easily guess what release the local branch is for if it
> isn't
> tracking a remote branch. However the bug about what to do if there
> are
> no remote branches is really not touched by any of this, it just got
> me
> thinking about branches
>
> What kind of user impact would this have?
>
> My hope is that the impact would be minimal. Git allows branch
> renames,
> and can successfully rename "f14/master" to "f14". All the history
> is
> renamed. We should be able to do this without an outage of the git
> server.
>
> The ACL system will need a slight tweaking, and a regeneration of the
> ACLs but that is fairly quick and minor to accomplish.
>
> However there will be some issues client side. We will not be able
> to
> make use of git's symbolic-ref feature of "aliasing" a branch. We
> cannot make "f14/master" an alias for "f14", again because of the
> filesystem layout issues. These issues rear their head once again
> when
> a client does a pull of an already checked out repo that had
> branches.
> Because there was already a f14/master, when the client sees a new
> branch just named "f14" it will fail to create it. Git has a command
> that will fix this "git remote prune origin". That will remove the
> local reference to f14/master and a subsequent pull sees the creation
> of
> the "f14" branch happen successfully. However, if a user had a local
> branch of f14 or f14/master they will be left with mismatched
> .git/config entries. In this case it's easiest to delete the local
> branch (git branch -d f14) and check it out again. If there are
> changes
> on the branch one can fix the config to point it to the right
> upstream
> location.
>
> Also we would need to get a new fedpkg into the hands of all the
> developers that handles the new branchnames. We could do a build
> that
> handles both the oldnames and the new and have it out and available
> for
> a reasonable period of time before we make the switch.

Would it make sense to add functionality to fedpkg which checks if there
exists configuration for remote branch tracking (i.e. local "f14"
tracks remote "f14/master"), and if that's the case, print
a warning (e.g. that it's recommended to delete the local branch and
recreate/check it out again)? This won't help much for the "git pull"
problem, but it may prevent some users from running into that problem
in the first place, because they saw the warning earlier when switching
branch or doing some other fedpkg operation.

--Severin

> So, some pain, for some pretty good gain. This time around I can
> setup
> pkgs.stg with this branch configuration and builds of fedpkg to use
> it
> for a more through testing before rolling it into production.
>
> So please, tell me what you think!
>
> --
> Jesse Keating
> Fedora -- Freedom² is a feature!
> identi.ca: http://identi.ca/jkeating
> --
> devel mailing list
> devel@lists.fedoraproject.org
> https://admin.fedoraproject.org/mailman/listinfo/devel
--
devel mailing list
devel@lists.fedoraproject.org
https://admin.fedoraproject.org/mailman/listinfo/devel
 
Old 12-04-2010, 04:52 PM
Bruno Wolff III
 
Default Request for comment: Potential change to dist-git branch structure

On Fri, Dec 03, 2010 at 16:34:05 -0800,
Jesse Keating <jkeating@redhat.com> wrote:
> "f14/user/fred/topic/mybranch" or other such craziness. When I went to
> test this, I realized that git won't allow you to have both "f14" and
> "f14/topic" as branches, because of the way the git metadata works on

Does there need to be some sort of check for / in maintainer created
branch names to prevent problems?

> My hope is that the impact would be minimal. Git allows branch renames,
> and can successfully rename "f14/master" to "f14". All the history is
> renamed. We should be able to do this without an outage of the git server.

Is this going to break things for people that having set up origin tracking
for multiple releases in the same repo?
--
devel mailing list
devel@lists.fedoraproject.org
https://admin.fedoraproject.org/mailman/listinfo/devel
 
Old 12-06-2010, 08:10 AM
Andreas Schwab
 
Default Request for comment: Potential change to dist-git branch structure

Jesse Keating <jkeating@redhat.com> writes:

> However, if a user had a local
> branch of f14 or f14/master they will be left with mismatched
> .git/config entries. In this case it's easiest to delete the local
> branch (git branch -d f14) and check it out again.

Or git branch --set-upstream.

Andreas.

--
Andreas Schwab, schwab@redhat.com
GPG Key fingerprint = D4E8 DBE3 3813 BB5D FA84 5EC7 45C6 250E 6F00 984E
"And now for something completely different."
--
devel mailing list
devel@lists.fedoraproject.org
https://admin.fedoraproject.org/mailman/listinfo/devel
 
Old 12-06-2010, 06:40 PM
Jesse Keating
 
Default Request for comment: Potential change to dist-git branch structure

On 12/03/2010 09:33 PM, Garrett Holmstrom wrote:
> On 12/3/2010 18:34, Jesse Keating wrote:
>> The original thought was to have top level branches that are named after
>> distribution releases, eg "f14", "f15", "el5". Then we would force
>> branches of those branches use a naming structure of "f14/topic". The
>> reason was so that our tooling could look at the name of the branch and
>> easily work back to the "f14" part. This would work even if it was
>> "f14/user/fred/topic/mybranch" or other such craziness. When I went to
>> test this, I realized that git won't allow you to have both "f14" and
>> "f14/topic" as branches, because of the way the git metadata works on
>> the filesystem. When I encountered this, I made "f14/master" become the
>> top level branch, and then "f14/somethingelse" could coexist.
>> Unfortunately I also wanted to keep things easy for users and tried to
>> maintain tooling that would allow you to just say "f14". This didn't
>> get enough real world testing and in hindsight was a bad idea. Things
>> go wrong quickly in git if your local branch name doesn't match the
>> remote branch name.
>>
>> When thinking about the above, and the two bugs I'm working on, I
>> realized that we don't have any real strong need to be using "/" as a
>> delineator. It makes some code easier, but makes other things more
>> complex and difficult. So what if we changed it?
>>
>> What I'm thinking about now is switching from / to - as a delineator.
>> This would improve a couple things. First, we could achieve upstream
>> top level branch names that are short and simple: "f14", "f15",
>> "master". We can have branches that build upon those names:
>> "f14-rebase", "f15-cve223", "f15-user-jkeating-private". We could keep
>> the simple fedpkg tooling that allows users to just type "f14" and the
>> like to reference a branch, and now the local branch will match the name
>> of the remote branch.
>
> Yes, please! Getting rid of the '/' strangeness ought to make things a
> little easier to understand and use across the board. I suspect that
> few enough packages use shared feature or bugfix branches that a
> transition won't trip up very many people. Perhaps a hook on Fedora's
> repositories that prints transition instructions when one attempts to
> push to old-style branches in conjunction with a fedpkg command that
> attempts to migrate existing local branches and remotes would help somewhat.

That's certainly something to look into. I'm not sure a hook would fire
off soon enough, or the client would notice that the upstream branch
doesn't exist anymore and balk before any upstream hooks could run.
Certainly worth looking into.

>
>> As for the first two bugs I mentioned, it doesn't directly help them.
>> However I would feel better about telling people that their local
>> branches must follow a naming scheme of<release>-<something> and then
>> we could easily guess what release the local branch is for if it isn't
>> tracking a remote branch. However the bug about what to do if there are
>> no remote branches is really not touched by any of this, it just got me
>> thinking about branches
>
> Why tie branch names down to specific releases? While that scheme makes
> it easy for fedpkg to guess what release to attempt to build against
> when one only cares about one release, it makes little sense to call a
> branch "f14-rh123456" when in reality that branch will merge into "f13"
> as well as "f14".

Couple reasons. First, the naming structure gives us the ability to
"easily" determine what Fedora your work is targeting. The vast
majority of Fedora packages have some macro or another that depends on
"dist" value, and they need to be defined any time the spec is parsed.
I prefer a scenario where this data is determined automatically, but
allowed to be overridden.

Also I don't envision a lot of these branches existing on the upstream
side. Downstream you can call the branch whatever you want, so if you
want to clone then branch for a bug to do test work, eventually merging
the work onto master, f14, f13 that's just fine. Only the shared
upstream branches would need a naming scheme.

Lastly by putting some soft of naming scheme in place it can help with
the ACL system, to provide ACLs for allowing non-ff changes in certain
branch types, or allowing all users to create branches of a package or
whatever. Although on that last point I think we need something like
github to easily allow users to 'fork' a repo when they don't have
commit rights to it, perhaps off to fedorapeople.org somewhere.
Rambling now.

--
Jesse Keating
Fedora -- Freedom² is a feature!
identi.ca: http://identi.ca/jkeating
--
devel mailing list
devel@lists.fedoraproject.org
https://admin.fedoraproject.org/mailman/listinfo/devel
 
Old 12-06-2010, 06:51 PM
Jesse Keating
 
Default Request for comment: Potential change to dist-git branch structure

On 12/04/2010 02:19 AM, Matej Cepl wrote:
> Dne 4.12.2010 06:33, Garrett Holmstrom napsal(a):
>> Why tie branch names down to specific releases? While that scheme makes
>> it easy for fedpkg to guess what release to attempt to build against
>> when one only cares about one release, it makes little sense to call a
>> branch "f14-rh123456" when in reality that branch will merge into "f13"
>> as well as "f14".
>
> +1 Why not just get out of all this silly business and leave branches to
> be whatever we want them to be as God^H^H^HLinus intended them to be?
> Really, branch rhbz1234567 doesn't have to have any relation to any
> particular distribution (we usually don't clone Fedora bugs to all
> distros where they happen, and that's The Right Thing).

Without some sort of naming scheme, it'd be quite hard for the fedpkg
client to fill in proper data for %{?dist} and other such macros when
parsing the spec file. It'd require manual action on the user to either
define it with a fedpkg option, or to set it in some sort of git config
(which doesn't traverse upstream/downstream so every cloner would have
to do that).

>
> Related issue I have with the Fedora git repositories is that one cannot
> remove any branch once it is created. After I have created in bitlbee
> repo two topic branches, only to find out that I cannot remove them
> after the merge. I can understand need for documenting development of
> the distribution, but cannot we lock just SOME branches (probably master
> + f* ones)? In this situation, I have moved my topical branches to
> gitorious, where I can do whatever I want to do with them.
>

That's another reason to have naming schemes so that we can design the
ACL system accordingly. However I'm reluctant to enable non-ff changes
in shared repos. Lots of ways for things to go wrong there,
particularly when "official" builds can come from anywhere within the
repo, no current restriction on "builds for dist-f14* must come from a
f14 branch" type thing. I honestly think we need to enable "forking" of
repos over to a fedorapeople place where you can do whatever you want
with them.

--
Jesse Keating
Fedora -- Freedom² is a feature!
identi.ca: http://identi.ca/jkeating
--
devel mailing list
devel@lists.fedoraproject.org
https://admin.fedoraproject.org/mailman/listinfo/devel
 
Old 12-06-2010, 07:17 PM
Jesse Keating
 
Default Request for comment: Potential change to dist-git branch structure

On 12/04/2010 02:31 AM, Kalev Lember wrote:
> On 12/04/2010 12:19 PM, Matej Cepl wrote:
>> Related issue I have with the Fedora git repositories is that one cannot
>> remove any branch once it is created. After I have created in bitlbee
>> repo two topic branches, only to find out that I cannot remove them
>> after the merge. I can understand need for documenting development of
>> the distribution, but cannot we lock just SOME branches (probably master
>> + f* ones)? In this situation, I have moved my topical branches to
>> gitorious, where I can do whatever I want to do with them.
>
> I think it makes sense to disallow removing official branches (f13, f14,
> master) to make sure people don't change the history of branches which
> are used for release builds.

There is no current restrictions on where "released builds" come from in
dist-git, particularly when there is a need from the likes of kernel and
KDE folks to do "official" builds from a user created branch (kernel and
KDE rebases can take a while and in the mean time they may need to issue
important updates of the current version of stuff)

>
> On the other hand, for topic branches and personal branches I would very
> much like to be able to do non fast-forward pushes and to be able to
> delete them. With git it's common to create branches for preparing a
> feature and merge them into the official branch once the feature is
> ready. Allowing non fast-forward pushes in unofficial branches would
> make it much easier to prepare a perfect history before merging it into
> the official branches.
>
>

I think it's best to do that kind of work in a separate repo, and not as
a in-repo branch of the main upstream repo. That's largely how the
kernel works, which is kinda the big example of git usage.


--
Jesse Keating
Fedora -- Freedom² is a feature!
identi.ca: http://identi.ca/jkeating
--
devel mailing list
devel@lists.fedoraproject.org
https://admin.fedoraproject.org/mailman/listinfo/devel
 

Thread Tools




All times are GMT. The time now is 08:35 PM.

VBulletin, Copyright ©2000 - 2014, Jelsoft Enterprises Ltd.
Content Relevant URLs by vBSEO ©2007, Crawlability, Inc.
Copyright ©2007 - 2008, www.linux-archive.org