FAQ Search Today's Posts Mark Forums Read
» Video Reviews

» Linux Archive

Linux-archive is a website aiming to archive linux email lists and to make them easily accessible for linux users/developers.

» Sponsor

» Partners

» Sponsor

Go Back   Linux Archive > Gentoo > Gentoo Development

LinkBack Thread Tools
Old 02-25-2011, 11:31 AM
Matthew Marlowe
Default Community Development of Gentoo Server Mgmt Tools for VMware Clusters/etc - Puppet modules, github collaboration, etc - Fork of Dev Topic: Avoiding Urgent Stabilizations

Reader note -- Direct all respones to gentoo-server ML to avoid flooding

For those on server-ml who missed dev-ml:

I'm starting to put together a portage/stable server configuration for a large
number of gentoo VM's that will eventually be hosted on a VMware ESX 4.1U1
cluster - with the goal of limiting major changes to once/year and otherwise
only applying security/minimum necessary updates. I doubt it will be easy but
I'm doing my best at it

As part of that I'm maintaining on github several related repositories,
including portage mask/use/config files, cluster management utilities, etc.


I've also started to document work on my blog:


I'm not currently planning to utilize a separate overlay for packages/ebuilds,
but it's not out of the question.

I'd be happy to work with any other devs or users w/ similiar interests or
production networks to manage.....github makes group development relatively

On Friday, February 25, 2011 03:37:36 am Ed W wrote:
> I maintain a, likely much smaller, number of VMs using linux vservers.
> The approach here is to almost cut each machine down to a chroot that
> runs only one (or thereabouts) interesting service. To do this I have
> found customised portage profiles to be the under-plugged secret since
> they allow you to basically push a set of packages which should be
> installed and control "per type of vm" use flags and package keywords
> (eg I have www-nginx, www-apache, mail, proxy, mysql, ftp, etc
> profiles). Additionally I have a small overlay of local ebuilds that
> sit in the same tree

I'd rather not maintain a separate profile for each server/node type.
Instead it would seem to be easier to use to a buildhost to create a combined
repository of packages, and then use puppet to define nodetypes and tell each
individual node which packages to install, which users to create, and which
services to enable.

> Up until now I haven't really made any effort to sync this whole tree
> across multiple physical machines and it's a bit of an ad-hoc process.
> Using something like git would probably be perfect

Agreed -- git *is* perfect and widely used in enterprise configuration
management repositories. The main advantage is that it allows individual
users to maintain their own forks, and yet merge contributions from other
trees or to request that other repositories merge their changes w/o much
effort. Github makes this even easier w/ a web interface, tools to visual
changes and track contributions, and discussions/wiki's for each repository.

> The still missing step is configuration management across the machine
> types, eg I want to upgrade all my "Apache-WWW" class machines and merge
> in all changes in /etc in a certain way... At the moment I just run
> dispatch-conf across all machines, but it can be quite boring merging 20
> instances of sshd.conf... Seems like Puppet/Chef could be a solution
> here, but the step up and investment to make it work seems pretty large?

Yes, I'm currently planning that puppet will control installation of new
packages and pushing all configuration files. This works very well w/ RedHat
Enterprise Linux/etc and if we are careful should also work for gentoo-- For
updates, we may have to either extend puppet to be aware of revdep-rebuild and
gentoo package versioning better, or have an external script for updating the
relevant nodes after the buildhost has finished compiling them.

> It does appear like managing large numbers of virtual machines is one
> are that gentoo could score very well? Interested to see any chatter on
> how others solve this problem, or any general advocacy? Probably we
> should start a new thread though...

Agreed - With careful management, the time investment in QA and package
management/maintenance should be more than outweighed by the benefits for
reasonably sized server clusters. It would be even easier if we can have
gentoo server admins work together to help build puppet modules and general
mgmt utilities using a service like github.

> Regards
> Ed W

Matthew Marlowe / 858-400-7430 / DeployLinux Consulting, Inc
Professional Linux Hosting and Systems Administration Services
www.deploylinux.net * matt@deploylinux.net
'MattM' @ irc.freenode.net

Thread Tools

All times are GMT. The time now is 01:30 PM.

VBulletin, Copyright ©2000 - 2014, Jelsoft Enterprises Ltd.
Content Relevant URLs by vBSEO ©2007, Crawlability, Inc.
Copyright 2007 - 2008, www.linux-archive.org