FAQ Search Today's Posts Mark Forums Read
» Video Reviews

» Linux Archive

Linux-archive is a website aiming to archive linux email lists and to make them easily accessible for linux users/developers.


» Sponsor

» Partners

» Sponsor


 
 
LinkBack Thread Tools
 
Old 08-08-2011, 10:02 PM
Kevin Fenzi
 
Default Hosted plans

On Mon, 8 Aug 2011 14:30:15 -0500 (CDT)
Mike McGrath <mmcgrath@redhat.com> wrote:

>
> https://admin.fedoraproject.org/status/hosted01

Cool.

> We've got that. The IO is what I saw with git as well. It doesn't
> properly cache so all page loads have to craw the related git repo.

Yeah.

> Compounding this issue is web crawlers but I think we have/had
> something in place to help limit this.

Yeah, there's some robots.txt in place I think now, which has mitigated
this somewhat.

kevin
_______________________________________________
infrastructure mailing list
infrastructure@lists.fedoraproject.org
https://admin.fedoraproject.org/mailman/listinfo/infrastructure
 
Old 08-10-2011, 03:15 PM
Kevin Fenzi
 
Default Hosted plans

On Mon, 08 Aug 2011 14:32:25 -0400
Stephen Gallagher <sgallagh@redhat.com> wrote:

...snip...

> So I'm proposing the following options:
>
> 1) Move our existing ReviewBoard instance to one of the app servers.
> This will significantly improve the performance and responsiveness,
> but we'll still have no email notification support (due to
> as-yet-unknown negative interaction with FAS integration)

Yeah, that doesn't seem ideal. ;(

> 2) Move ReviewBoard to an app server and drop integration with FAS and
> allow standard enrollment for users, be they Fedora users or not. This
> will solve the performance and email issues, but results in a server
> running on Fedora systems that is not using Fedora accounts. Also I'm
> not sure we can maintain the existing review histories for the few
> projects currently using the system.

Ditto.

> 3) Turn ReviewBoard into a turnkey OpenShift virtual instance and
> allow any Fedora Hosted project to spin one up. This instance would
> use standard enrollment (rather than FAS integration, which is
> impossible outside the Infra firewall). Each project could have its
> own complete instance to maintain on its own. Upsides: less work for
> Fedora Admins, support for email and better performance. Downsides: no
> centrally-managed user accounts and projects need to do more of the
> maintaining of the system themselves.

This is pretty interesting... I assume after following the steps they
would have a persistent instance they could use moving forward. It
doesn't need anything special to talk to their project on hosted?
Does it end up costing the end project anything?

What happens if someone sets up an instance and then disappears?
Does the project have any way to deal with that? Or just make a new one?

Would someone be interested in trying this out and seeing how well it
actually works? Is there a project or two that are really wanting to
use reviewboard that we could ask?

> I'm all ears for a fourth (or fifth...) option.

Well, there's https://www.rbcommons.com/plans/ but thats a cost/month.
(Which might be worth it for some projects).

kevin
_______________________________________________
infrastructure mailing list
infrastructure@lists.fedoraproject.org
https://admin.fedoraproject.org/mailman/listinfo/infrastructure
 
Old 08-10-2011, 03:26 PM
Stephen Gallagher
 
Default Hosted plans

On Wed, 2011-08-10 at 09:15 -0600, Kevin Fenzi wrote:
> > 3) Turn ReviewBoard into a turnkey OpenShift virtual instance and
> > allow any Fedora Hosted project to spin one up. This instance would
> > use standard enrollment (rather than FAS integration, which is
> > impossible outside the Infra firewall). Each project could have its
> > own complete instance to maintain on its own. Upsides: less work for
> > Fedora Admins, support for email and better performance. Downsides: no
> > centrally-managed user accounts and projects need to do more of the
> > maintaining of the system themselves.
>
> This is pretty interesting... I assume after following the steps they
> would have a persistent instance they could use moving forward. It
> doesn't need anything special to talk to their project on hosted?
> Does it end up costing the end project anything?

I've submitted a patch upstream to ReviewBoard to add easy configuration
of Fedora Hosted source repositories:
http://reviews.reviewboard.org/r/2505/

I have confirmation from Christian Hammond (the upstream project lead)
that it will be included in the 1.5.6 and 1.6.0 releases (aka
imminently).

So there's very little that the projects need to do in order to connect
to the hosted repo. As I said above, they lose the centrally-managed
users available to FAS, so they'd need to manage their own groups
themselves. On the other side, this does mean that they gain much finer
control over permissions (since they can define their own
project-specific groups rather than relying on FAS groups).

>
> What happens if someone sets up an instance and then disappears?
> Does the project have any way to deal with that? Or just make a new one?
>

That's a good question for Mike McGrath. I suspect that it would be
prudent to recommend that projects set up several administrators so that
a disappearance of one doesn't result in the loss of all administrators.

Also, it's possible to promote a user to admin status if you have
database privileges as well by setting the admin flag on their user
account, but of course that assumes you have access to a DB admin.

A final option would be to modify the openshift instance to always
install a recovery admin with a random password that was escrowed by the
Fedora project, but I'm not sure whether that's realistic. Mike, can you
speak to that?

> Would someone be interested in trying this out and seeing how well it
> actually works? Is there a project or two that are really wanting to
> use reviewboard that we could ask?
>

Well, AutoQA has been using the FedoraHosted instance (without email
support and poor performance) very heavily. They might be interested.

I would also be interested in piloting it for the SSSD and FreeIPA
projects (for which I originally got invested in setting this up).

> > I'm all ears for a fourth (or fifth...) option.
>
> Well, there's https://www.rbcommons.com/plans/ but thats a cost/month.
> (Which might be worth it for some projects).

True, but as we already have the OpenShift option (thanks Mike!), I
think it's worth looking into making this part of our Hosted offering.
_______________________________________________
infrastructure mailing list
infrastructure@lists.fedoraproject.org
https://admin.fedoraproject.org/mailman/listinfo/infrastructure
 
Old 08-17-2011, 06:24 PM
seth vidal
 
Default Hosted plans

On Mon, 2011-08-08 at 10:54 -0600, Kevin Fenzi wrote:
> Greetings.
>
> I thought I would throw out some plans for hosted moving forward and
> see if we could hash out a plan and some timetable for implementing
> things.
>

> Some things we talked about:
>
> * A caching frontend of some kind. Could cache web requests and take
> load off the main machines.
> * Some way to distribute the data, so if serverbeach were off line we
> could still switch to and use another machine.
> * Adding more/different services.
> IRC commit bot
> redmine (still not in fedora, but people are working on it)
> your idea here.


We talked a bit about this yesterday and I wanted to write it up here:

1. move the mailman instance over to collab1 and have them share an
infrastructure - mailman is not (usually) easy to spread to lots of
servers in a very clean way and it is not (usually) the source of a
massive load (the archiver not withstanding)

2. I think we should seriously consider building things for hosted 2.0
such that slices are possible:

you still go to fedorahosted.org/projectname

but it redirects you to project-##.fedorahosted.org - based on the
first 2 letters of the name of your project or what not.

for commits you have to commit to:
git-##.fedorahosted.org (for example)
or svn-##.fedorahosted.org

it would allow us to expand out horizontally as things get busier.

It would require a config change and/or a flag day for our users but
it's not the end of the world now.
Advantages:
a. this keeps all of our eggs out of one basket
b. it means replicating the infrastructure is important,
repeatedly

it seems like an obvious course of action that scales well for what we
need.

3. Think HARD about limiting support for additional services on hosted.


thoughts?

-sv


_______________________________________________
infrastructure mailing list
infrastructure@lists.fedoraproject.org
https://admin.fedoraproject.org/mailman/listinfo/infrastructure
 
Old 08-17-2011, 06:25 PM
seth vidal
 
Default Hosted plans

On Wed, 2011-08-17 at 14:24 -0400, seth vidal wrote:

>
> We talked a bit about this yesterday and I wanted to write it up here:
>
> 1. move the mailman instance over to collab1 and have them share an
> infrastructure - mailman is not (usually) easy to spread to lots of
> servers in a very clean way and it is not (usually) the source of a
> massive load (the archiver not withstanding)
>
> 2. I think we should seriously consider building things for hosted 2.0
> such that slices are possible:
>
> you still go to fedorahosted.org/projectname
>
> but it redirects you to project-##.fedorahosted.org - based on the
> first 2 letters of the name of your project or what not.
>
> for commits you have to commit to:
> git-##.fedorahosted.org (for example)
> or svn-##.fedorahosted.org

Actually - on further thought just making it work for:

git-$projectname.fedorahosted.org (for example)

should work just fine and be no more effort - just an CNAME on the
backend.


Easier to document, too

-sv


_______________________________________________
infrastructure mailing list
infrastructure@lists.fedoraproject.org
https://admin.fedoraproject.org/mailman/listinfo/infrastructure
 
Old 08-17-2011, 07:01 PM
Stephen John Smoogen
 
Default Hosted plans

On Wed, Aug 17, 2011 at 12:24, seth vidal <skvidal@fedoraproject.org> wrote:
> On Mon, 2011-08-08 at 10:54 -0600, Kevin Fenzi wrote:
>> Greetings.
>>
>> I thought I would throw out some plans for hosted moving forward and
>> see if we could hash out a plan and some timetable for implementing
>> things.
>>
>
>> Some things we talked about:
>>
>> * A caching frontend of some kind. Could cache web requests and take
>> * load off the main machines.
>> * Some way to distribute the data, so if serverbeach were off line we
>> * could still switch to and use another machine.
>> * Adding more/different services.
>> * * * IRC commit bot
>> * * * redmine (still not in fedora, but people are working on it)
>> * * * your idea here.
>
>
> We talked a bit about this yesterday and I wanted to write it up here:
>
> 1. move the mailman instance over to collab1 and have them share an
> infrastructure - mailman is not (usually) easy to spread to lots of
> servers in a very clean way and it is not (usually) the source of a
> massive load (the archiver not withstanding)
>
> 2. I think we should seriously consider building things for hosted 2.0
> such that slices are possible:
>
> *you still go to fedorahosted.org/projectname
>
> *but it redirects you to project-##.fedorahosted.org - based on the
> *first 2 letters of the name of your project or what not.

Just an idea to bring up when we implement:

Due to the fact that we would have huge groupings on certain letters (
re, fe, li py, sy) use the first 2 letters of the md5sum of the name.
Hopefully that would spread out the load across enough. The project
build script would automate this so people don't have to figure it
out.

I like the below.

> *for commits you have to commit to:
> *git-##.fedorahosted.org (for example)
> *or svn-##.fedorahosted.org
>
> *it would allow us to expand out horizontally as things get busier.
>
> *It would require a config change and/or a flag day for our users but
> *it's not the end of the world now.
> *Advantages:
> * a. this keeps all of our eggs out of one basket
> * b. it means replicating the infrastructure is important,
> repeatedly
>
> *it seems like an obvious course of action that scales well for what we
> need.
>
> 3. Think HARD about limiting support for additional services on hosted.

4. Talk to something like github if we can meet up and grow something togehter?

>
> thoughts?
>
> -sv
>
>
> _______________________________________________
> infrastructure mailing list
> infrastructure@lists.fedoraproject.org
> https://admin.fedoraproject.org/mailman/listinfo/infrastructure
>



--
Stephen J Smoogen.
"The core skill of innovators is error recovery, not failure avoidance."
Randy Nelson, President of Pixar University.
"Let us be kind, one to another, for most of us are fighting a hard
battle." -- Ian MacLaren
_______________________________________________
infrastructure mailing list
infrastructure@lists.fedoraproject.org
https://admin.fedoraproject.org/mailman/listinfo/infrastructure
 
Old 08-17-2011, 07:26 PM
Jesse Keating
 
Default Hosted plans

On Aug 17, 2011, at 12:01 PM, Stephen John Smoogen wrote:
>
> 4. Talk to something like github if we can meet up and grow something togehter?
>

github is non-free. Gitorious is more inline with our values, but a big pile of mess to try and integrate with our environment.


- jlk

_______________________________________________
infrastructure mailing list
infrastructure@lists.fedoraproject.org
https://admin.fedoraproject.org/mailman/listinfo/infrastructure
 
Old 08-17-2011, 07:53 PM
seth vidal
 
Default Hosted plans

On Wed, 2011-08-17 at 13:01 -0600, Stephen John Smoogen wrote:

> Due to the fact that we would have huge groupings on certain letters (
> re, fe, li py, sy) use the first 2 letters of the md5sum of the name.
> Hopefully that would spread out the load across enough. The project
> build script would automate this so people don't have to figure it
> out.
>

md5sum is too complicated - maintaining a few thousand cnames, though,
isn't tough at all.


> 4. Talk to something like github if we can meet up and grow something togehter?

I agree with jesse that github, being closed source, is a complete
non-starter.

-sv


_______________________________________________
infrastructure mailing list
infrastructure@lists.fedoraproject.org
https://admin.fedoraproject.org/mailman/listinfo/infrastructure
 
Old 08-17-2011, 08:10 PM
Stephen John Smoogen
 
Default Hosted plans

On Wed, Aug 17, 2011 at 13:53, seth vidal <skvidal@fedoraproject.org> wrote:
> On Wed, 2011-08-17 at 13:01 -0600, Stephen John Smoogen wrote:
>
>> Due to the fact that we would have huge groupings on certain letters (
>> re, fe, li py, sy) use the first 2 letters of the md5sum of the name.
>> Hopefully that would spread out the load across enough. The project
>> build script would automate this so people don't have to figure it
>> out.
>>
>
> md5sum is too complicated - maintaining a few thousand cnames, though,
> isn't tough at all.
>
>
>> 4. Talk to something like github if we can meet up and grow something togehter?
>
> I agree with jesse that github, being closed source, is a complete
> non-starter.

Sorry I meant whichever one is the opensource one [Sorry, they are
all just names without faces to me]. However if their code is not
better than the problems we have with trac then I can move along.

> -sv
>
>
> _______________________________________________
> infrastructure mailing list
> infrastructure@lists.fedoraproject.org
> https://admin.fedoraproject.org/mailman/listinfo/infrastructure
>



--
Stephen J Smoogen.
"The core skill of innovators is error recovery, not failure avoidance."
Randy Nelson, President of Pixar University.
"Let us be kind, one to another, for most of us are fighting a hard
battle." -- Ian MacLaren
_______________________________________________
infrastructure mailing list
infrastructure@lists.fedoraproject.org
https://admin.fedoraproject.org/mailman/listinfo/infrastructure
 
Old 08-17-2011, 09:49 PM
Luke Macken
 
Default Hosted plans

Excerpts from seth vidal's message of Wed Aug 17 15:53:42 -0400 2011:
> On Wed, 2011-08-17 at 13:01 -0600, Stephen John Smoogen wrote:
> > 4. Talk to something like github if we can meet up and grow something togehter?
>
> I agree with jesse that github, being closed source, is a complete
> non-starter.

Allura, the code behind the shiny new SourceForge, is open source.

http://sourceforge.net/p/allura

Based on the many conversations I've had with them at PyCons, they would
definitely be interested in working together with us on hosting.

luke
_______________________________________________
infrastructure mailing list
infrastructure@lists.fedoraproject.org
https://admin.fedoraproject.org/mailman/listinfo/infrastructure
 

Thread Tools




All times are GMT. The time now is 10:39 AM.

VBulletin, Copyright ©2000 - 2014, Jelsoft Enterprises Ltd.
Content Relevant URLs by vBSEO ©2007, Crawlability, Inc.
Copyright 2007 - 2008, www.linux-archive.org