FAQ Search Today's Posts Mark Forums Read
» Video Reviews

» Linux Archive

Linux-archive is a website aiming to archive linux email lists and to make them easily accessible for linux users/developers.


» Sponsor

» Partners

» Sponsor

Go Back   Linux Archive > CentOS > CentOS

 
 
LinkBack Thread Tools
 
Old 02-04-2010, 05:15 PM
Les Mikesell
 
Default best parallel / cluster SSH

On 2/4/2010 11:45 AM, Alan McKay wrote:
> Hey folks,
>
> I stumbled upon this while looking for something else
>
> http://www.linux.com/archive/feature/151340
>
> And it is something I could actually really make use of. But right on
> that site they list 3 different ones, and so I'm wondering what all is
> out there and what I should use.

It depends on what you need to do. If you really have enough machines
or long-running jobs that a shell loop through them isn't practical, you
might want something higher-level like cfengine or puppet, or at least
something running under cron to make them independent.

--
Les Mikesell
lesmikesell@gmail.com





_______________________________________________
CentOS mailing list
CentOS@centos.org
http://lists.centos.org/mailman/listinfo/centos
 
Old 02-04-2010, 05:31 PM
Alan McKay
 
Default best parallel / cluster SSH

> It depends on what you need to do. *If you really have enough machines
> or long-running jobs that a shell loop through them isn't practical, you
> might want something higher-level like cfengine or puppet, or at least
> something running under cron to make them independent.

cfengine or puppet (or something else - slackmaster?) are where I want
to be eventually - but in the immediate term something like this would
help a lot. e.g bouncing my 4 front-end apache servers on 4
different boxes. That sort of thing.

I was actually going to start another "configuration management redux"
thread as a follow up to a thread I started a few months ago.


--
“Don't eat anything you've ever seen advertised on TV”
- Michael Pollan, author of "In Defense of Food"
_______________________________________________
CentOS mailing list
CentOS@centos.org
http://lists.centos.org/mailman/listinfo/centos
 
Old 02-04-2010, 05:45 PM
"nate"
 
Default best parallel / cluster SSH

Alan McKay wrote:

> I was actually going to start another "configuration management redux"
> thread as a follow up to a thread I started a few months ago.

As Les mentioned, it's far more common in that situation to use
ssh key authentication and a for loop, if your ssh key has a pass
phrase use a ssh agent.

I still use it quite often even though I do have a fairly extensive
cfengine setup, sometimes I need something done right now such
as a mass restart and can't wait for cfengine to run on each host.

If you have servers say

web01 -> web30

sample script to restart apache -

for i in `seq -w 1 30`; do ssh root@web${i} "/etc/init.d/httpd restart"; done

nate


_______________________________________________
CentOS mailing list
CentOS@centos.org
http://lists.centos.org/mailman/listinfo/centos
 
Old 02-04-2010, 05:50 PM
Gavin Carr
 
Default best parallel / cluster SSH

On Thu, Feb 04, 2010 at 01:31:39PM -0500, Alan McKay wrote:
> > It depends on what you need to do. *If you really have enough machines
> > or long-running jobs that a shell loop through them isn't practical, you
> > might want something higher-level like cfengine or puppet, or at least
> > something running under cron to make them independent.
>
> cfengine or puppet (or something else - slackmaster?) are where I want
> to be eventually - but in the immediate term something like this would
> help a lot. e.g bouncing my 4 front-end apache servers on 4
> different boxes. That sort of thing.
>
> I was actually going to start another "configuration management redux"
> thread as a follow up to a thread I started a few months ago.

We've tried all the ones in that article you mentioned, and are currently
using classh - http://freshmeat.net/projects/classh - which is pretty nice,
in spite of being labelled alpha.

I've packaged it for c5 here:

http://www.openfusion.com.au/mrepo/centos5-i386/RPMS.of/classh-0.092-1.of.el5.noarch.rpm

if you'd like to give it a whirl.

Cheers,
Gavin

_______________________________________________
CentOS mailing list
CentOS@centos.org
http://lists.centos.org/mailman/listinfo/centos
 
Old 02-04-2010, 05:52 PM
Les Mikesell
 
Default best parallel / cluster SSH

On 2/4/2010 12:31 PM, Alan McKay wrote:
>> It depends on what you need to do. If you really have enough machines
>> or long-running jobs that a shell loop through them isn't practical, you
>> might want something higher-level like cfengine or puppet, or at least
>> something running under cron to make them independent.
>
> cfengine or puppet (or something else - slackmaster?) are where I want
> to be eventually - but in the immediate term something like this would
> help a lot. e.g bouncing my 4 front-end apache servers on 4
> different boxes. That sort of thing.

Usually for that sort of thing I explicitly _don't_ want all of a group
to be out of service at the same moment so I much prefer to:

for host in host1 host2 host3 host4
do
echo $host
ssh $host service httpd restart
done

which doesn't take much more thought then doing it locally and if I
expect to repeat it I can save the commands in a script.

> I was actually going to start another "configuration management redux"
> thread as a follow up to a thread I started a few months ago.

I'm interested in the topic, but most of what I've seen just add
complexity for no particular reason. Clustering ssh commands might be
an exception for a certain subset of things that need to happen at
approximately the same time on multiple machines (and you are willing to
risk breaking them all at once with a typo).

--
Les Mikesell
lesmikesell@gmail.com
_______________________________________________
CentOS mailing list
CentOS@centos.org
http://lists.centos.org/mailman/listinfo/centos
 
Old 02-04-2010, 05:52 PM
Steve Huff
 
Default best parallel / cluster SSH

On Feb 4, 2010, at 12:45 PM, Alan McKay wrote:


And it is something I could actually really make use of. But right on
that site they list 3 different ones, and so I'm wondering what all is
out there and what I should use.



the most comprehensive list of such things of which i am aware is here:

http://web.taranis.org/shmux/#related

at my workplace we use gsh. from RPMforge you can get pydsh, shmux,
and tentakel; i'm fond of shmux.


-steve

--
If this were played upon a stage now, I could condemn it as an
improbable fiction. - Fabian, Twelfth Night, III,v

http://five.sentenc.es

_______________________________________________
CentOS mailing list
CentOS@centos.org
http://lists.centos.org/mailman/listinfo/centos
 
Old 02-04-2010, 05:56 PM
Wade Hampton
 
Default best parallel / cluster SSH

I've been using clusterit for several years for multiple small
clusters. It works well
and was easy to install. I believe I got the Fedora source RPM and rebuilt
it for CentOS.

On Thu, Feb 4, 2010 at 1:50 PM, Gavin Carr <gavin@openfusion.com.au> wrote:
> On Thu, Feb 04, 2010 at 01:31:39PM -0500, Alan McKay wrote:
>> > It depends on what you need to do. *If you really have enough machines
>> > or long-running jobs that a shell loop through them isn't practical, you
>> > might want something higher-level like cfengine or puppet, or at least
>> > something running under cron to make them independent.
>>
>> cfengine or puppet (or something else - slackmaster?) are where I want
>> to be eventually - but in the immediate term something like this would
>> help a lot. * *e.g bouncing my 4 front-end apache servers on 4
>> different boxes. * That sort of thing.
>>
>> I was actually going to start another "configuration management redux"
>> thread as a follow up to a thread I started a few months ago.
>
> We've tried all the ones in that article you mentioned, and are currently
> using classh - http://freshmeat.net/projects/classh - which is pretty nice,
> in spite of being labelled alpha.
>
> I've packaged it for c5 here:
>
> *http://www.openfusion.com.au/mrepo/centos5-i386/RPMS.of/classh-0.092-1.of.el5.noarch.rpm
>
> if you'd like to give it a whirl.
>
> Cheers,
> Gavin
>
> _______________________________________________
> CentOS mailing list
> CentOS@centos.org
> http://lists.centos.org/mailman/listinfo/centos
>

Cheers,
--
Wade
_______________________________________________
CentOS mailing list
CentOS@centos.org
http://lists.centos.org/mailman/listinfo/centos
 
Old 02-04-2010, 06:03 PM
Les Mikesell
 
Default best parallel / cluster SSH

On 2/4/2010 12:45 PM, nate wrote:
> Alan McKay wrote:
>
>> I was actually going to start another "configuration management redux"
>> thread as a follow up to a thread I started a few months ago.
>
> As Les mentioned, it's far more common in that situation to use
> ssh key authentication and a for loop, if your ssh key has a pass
> phrase use a ssh agent.
>
> I still use it quite often even though I do have a fairly extensive
> cfengine setup, sometimes I need something done right now such
> as a mass restart and can't wait for cfengine to run on each host.
>
> If you have servers say
>
> web01 -> web30
>
> sample script to restart apache -
>
> for i in `seq -w 1 30`; do ssh root@web${i} "/etc/init.d/httpd restart"; done

Or keep the lists of servers in files so you can apply something like:

for i in `cat web-hosts`
to any ssh command loop

There's a big gray area between strict automation and manually logging
in to every machine for maintenance and I find it helpful to layer the
simple tools and steps you already know.

But, if someone ever gets cross-platform config management right or at
least close enough that it is worth learning yet another description
language I'd be very interested. Cfengine v3 might be getting there but
the windows version seems to be only available in the commercial build.

--
Les Mikesell
lesmikesell@gmail.com

_______________________________________________
CentOS mailing list
CentOS@centos.org
http://lists.centos.org/mailman/listinfo/centos
 
Old 02-04-2010, 08:11 PM
Karanbir Singh
 
Default best parallel / cluster SSH

On 02/04/2010 06:31 PM, Alan McKay wrote:
> cfengine or puppet (or something else - slackmaster?) are where I want

slackmaster, i dont know about - but are you refering to 'slack' ? its a
fairly easy way to get started, and is essentially a wrapper around
rsync. takes about 2 min to get setup and started, and will do all the
basic things including managing services etc.

The original centos buildsystem, which was spread over 12 machines, was
all managed using slack scripts

- KB
_______________________________________________
CentOS mailing list
CentOS@centos.org
http://lists.centos.org/mailman/listinfo/centos
 
Old 02-05-2010, 06:27 AM
Peter Kjellstrom
 
Default best parallel / cluster SSH

On Thursday 04 February 2010, Alan McKay wrote:
> Hey folks,
>
> I stumbled upon this while looking for something else
>
> http://www.linux.com/archive/feature/151340
>
> And it is something I could actually really make use of. But right on
> that site they list 3 different ones, and so I'm wondering what all is
> out there and what I should use.

In (HPC) clustering pdsh is very popular. It's available in .tgz with
spec-file and rebuilds nicely on c5 with rpmbuild -tb ...

https://computing.llnl.gov/linux/pdsh.html

A few examples of what pdsh can do:
- pick hosts with compact expressions: -w n[3-10,44]
- settable fanout (run on X hosts in parallel)
- remote command timeout
- pdcp command that allows the copying of file
- nifty post-processor that compats output:

$ pdsh -w n[1-3],n5 uname -r | dshbak -c
----------------
n[1-3,5]
----------------
2.6.18-164.11.1.el5

/Peter

> Is there one that is part of the standard CentOS?
>
> thanks,
> -Alan
_______________________________________________
CentOS mailing list
CentOS@centos.org
http://lists.centos.org/mailman/listinfo/centos
 

Thread Tools




All times are GMT. The time now is 04:38 PM.

VBulletin, Copyright ©2000 - 2014, Jelsoft Enterprises Ltd.
Content Relevant URLs by vBSEO ©2007, Crawlability, Inc.
Copyright 2007 - 2008, www.linux-archive.org