FAQ Search Today's Posts Mark Forums Read
» Video Reviews

» Linux Archive

Linux-archive is a website aiming to archive linux email lists and to make them easily accessible for linux users/developers.


» Sponsor

» Partners

» Sponsor

Go Back   Linux Archive > Debian > Debian ISP

 
 
LinkBack Thread Tools
 
Old 09-11-2008, 09:40 AM
"Wojciech Ziniewicz"
 
Default best FC-storage sharing/clustering techniques

Hi ,
I have two Servers.

First is a debian server on HP DL 380G5 server with fiberchannel card
connected to storage server ( fujitsu S16F R1430-M5 ) with 5 TB of
data.
The service that is mainly used on this server is apache. What i would
like to achieve is to connect another server to balance the www trafic
among those two servers.
There's no problem with the loadbalancing etc, but i was wondering
first how to connect those two servers to one fiberchannel physically
and second , how to make them "share" the same volume of data. I was
using GFS ( unluckily on redhat ;/ ) for another clustering scenario
but is this really necessary ? What other software can I use ?

regards and thank You for your time.

--
Wojciech Ziniewicz
Unix SEX :{look;gawk;find;sed;talk;grep;touch;finger;find;f l
ex;unzip;head;tail; mount;workbone;fsck;yes;gasp;fsck;more;yes;yes;eje
ct;umount;makeclean; zip;split;done;exit:xargs!!}


--
To UNSUBSCRIBE, email to debian-isp-REQUEST@lists.debian.org
with a subject of "unsubscribe". Trouble? Contact listmaster@lists.debian.org
 
Old 09-11-2008, 10:43 AM
"Wojciech Ziniewicz"
 
Default best FC-storage sharing/clustering techniques

2008/9/11 Jean-Paul Blaquiere <jeanpaul@blaquiere.id.au>:
> I big question for you. How much disk I/O are you in fact doing? It is
> possible that you are going to create something that will cause you
> grief, if it's not implemented correctly. Is an NFS server a feasible
> option? It is much simpler to implement, if a little less redundant to
> start out with. The scaling methodology is also quite different.

It would be countrywide service like youtube.pl but much more
specialized thus - smaller.
NFS won't do unless i would buy extra switch for "storage" layer with
gigabit ports etc etc.

Why I ask is that I have very bad experience with GFS and don't want
to deploy it.

Moreover I don't have any LVS servers yet but heard that it is very
efficient . Maybe I can cluster the servers connected to one storage
server with LVS without using GFS/ocfs ?

And the last question regarding LVS - is that possible to easily
migrate from one machine with normal kernel/system to two machines
stacked into lvs cluster ?

regards

> For a full clustered web server, look at the following :
> http://www.linuxvirtualserver.org/
> http://www.ultramonkey.org/
>
> We have one deployment of LVS managing some streaming media servers and
> it's working without a problem. (Along with the rest of our load balancer
> kit which is f5 and Foundry gear)
>
>
> ./jp
> --
> Jean-Paul Blaquiere
> jeanpaul@blaquiere.id.au
> http://www.blaquiere.id.au
> http://japester.ucc.asn.au/
>



--
Wojciech Ziniewicz
Unix SEX :{look;gawk;find;sed;talk;grep;touch;finger;find;f l
ex;unzip;head;tail; mount;workbone;fsck;yes;gasp;fsck;more;yes;yes;eje
ct;umount;makeclean; zip;split;done;exit:xargs!!}


--
To UNSUBSCRIBE, email to debian-isp-REQUEST@lists.debian.org
with a subject of "unsubscribe". Trouble? Contact listmaster@lists.debian.org
 
Old 09-11-2008, 03:57 PM
Jean-Paul Blaquiere
 
Default best FC-storage sharing/clustering techniques

There is a summary at the end of this email, it got kinda long, and
potentially side tracked away from your original question.


> On Sep 11, Wojciech Ziniewicz illuminated :

> It would be countrywide service like youtube.pl but much more
> specialized thus - smaller.
> NFS won't do unless i would buy extra switch for "storage" layer with
> gigabit ports etc etc.
>

What you could do, and this might be outside the realms of LVS, but in
the realm of f5's kit, is split your read and write requests between
servers (url based balancing). Web traffic is mostly read only, so you
could get away with having one server have direct access to the disk,
and export that via NFS to the read only slaves. The NFS cache
algorithms will make that feasibly quick too.

The big caveat, it won't scale well with increased write requests and
would be very customised for your deployment.

Can you put the web content into a database? That is the easiest way
to scale long term. Look at yotube, livejournal, etc. Creating
clustered scalable databases is a well solved problem.

Just out of interest, how much throughput are you expecting? Tens or
hundres of megabits to your customers?

> Why I ask is that I have very bad experience with GFS and don't want
> to deploy it.
>
There are alternatives to GFS. My only experience is with QFS (a
commercial cluster filesystem from Sun), so can't comment on the
good/bad sides of GFS and the rest. I only know that they exist.
It is also possible that you had a dodgy version of GFS or a non-
optimally configured system.

> Moreover I don't have any LVS servers yet but heard that it is very
> efficient . Maybe I can cluster the servers connected to one storage
> server with LVS without using GFS/ocfs ?
>
That is a very valid option. It depends on where you want to spend the
money. For example, we have a pair of NetApp filers managing some
300,000 user's email over NFS. The business choice here, was spend the
big $$$ on the storage devices, and less on the servers and network.

The alternative was, spend more money on servers and FC fabric, and spend
less money on the storage devices. Both extremes are likely to give
similar performance, but different kinds of management headaches.

We did theorise about migrating our mail servers to fibre channel
attached storage, and then went off and cried when we started
researching the pain and agony that would be involved in managing the
cluster file system for them. The easiest option for us to upgrade, in
this particular situation, is to buy bigger NetApps.

At the end of the day, if you're clustering services using LVS, you will
need three layers of servers. and I would definitely run LVS on
different hardware from the web servers.
- LVS load balancers (two)
- Web servers (as many as you need)
- storage

> And the last question regarding LVS - is that possible to easily
> migrate from one machine with normal kernel/system to two machines
> stacked into lvs cluster ?
>
Kind of. You need to insert the LVS server between the application
server and the end users. That will involve at least a small amount of
downtime. You should be able to make that less than a minute, if you
plan the steps right though.
There are no changes that need to be made to the kernel for a server to
be managed by LVS, only network configuration updates.

################################################## ##############################
Looking back over your original question, I am firmly of the belief that
a clustered file system will not be the right choice. At least, not yet.

It is too complex a solution that will cause grief in the future.

I think that an NFS server will provide you with more than enough
throughput to your web servers.
Deploy some LVS load balancers in front of them, and your throughput and
redundancy will be manageable.
If you can, push the web data into a database. It is much easier to
scale a database.


./jp
--
Jean-Paul Blaquiere
jeanpaul@blaquiere.id.au
http://www.blaquiere.id.au
http://japester.ucc.asn.au/


--
To UNSUBSCRIBE, email to debian-isp-REQUEST@lists.debian.org
with a subject of "unsubscribe". Trouble? Contact listmaster@lists.debian.org
 
Old 09-12-2008, 08:09 AM
"Wojciech Ziniewicz"
 
Default best FC-storage sharing/clustering techniques

2008/9/11 Jean-Paul Blaquiere <jeanpaul@blaquiere.id.au>:
> I think that an NFS server will provide you with more than enough
> throughput to your web servers.
> Deploy some LVS load balancers in front of them, and your throughput and
> redundancy will be manageable.
> If you can, push the web data into a database. It is much easier to
> scale a database.

Thank You for your opinion.

We've chosen to use fiberchannel for we see forward to have something
like 100 mbits of traffic and 10-20 servers more and I'm totally sure
that we will deploy LVS with two loadbalancers. Yesterday luckily i've
managed to browse all the documentation and it will be a very good
solution. There's a slight modification of the kernel possible but
it's included in vannilla tree so no problem.

The "tight-neck" od the system could be the FC throughput, i mean
indexing of the files, but I hope ocfs will do the job of
caching/indexing this huge amount of data. If not - we will move
everything to databases ;/

I didn't know that services like youtube keep the data in network
databases like oracle - of course it scale well then but what is the
overhead of such solution? Of course the bigger overhead, the more
machines you have to involve , but they probably dont have the
problems with money



regards

--
Wojciech Ziniewicz
Unix SEX :{look;gawk;find;sed;talk;grep;touch;finger;find;f l
ex;unzip;head;tail; mount;workbone;fsck;yes;gasp;fsck;more;yes;yes;eje
ct;umount;makeclean; zip;split;done;exit:xargs!!}


--
To UNSUBSCRIBE, email to debian-isp-REQUEST@lists.debian.org
with a subject of "unsubscribe". Trouble? Contact listmaster@lists.debian.org
 
Old 09-12-2008, 10:04 AM
Jean-Paul Blaquiere
 
Default best FC-storage sharing/clustering techniques

Ahh, your intended scale starts to make sense now.

http://kylecordes.com/2007/07/12/youtube-scalability/
http://video.google.com/videoplay?docid=-6304964351441328559

Have a look at the other HA videos that are related to the second link
there.

You should be able to push 400+MB/s through fibre channel. yes,
bytes, not bits. It should not be your bottleneck. That will be more
dependent upon the data type and how it is spread across disk spindles though.
YMWV (your milage will vary)

If you're scaling that large, and distributing your load, make sure your
design can handle that. Lots of thinking
and everyone has problems with money. Especially in the beginning.
and I think your scaling is past my knowledge sphere
I think the videos linked up above will give you the best start.


/jp
--
Jean-Paul Blaquiere
jeanpaul@blaquiere.id.au
http://www.blaquiere.id.au
http://japester.ucc.asn.au/


--
To UNSUBSCRIBE, email to debian-isp-REQUEST@lists.debian.org
with a subject of "unsubscribe". Trouble? Contact listmaster@lists.debian.org
 
Old 09-12-2008, 10:20 AM
"Wojciech Ziniewicz"
 
Default best FC-storage sharing/clustering techniques

2008/9/12 Jean-Paul Blaquiere <jeanpaul@blaquiere.id.au>:
> If you're scaling that large, and distributing your load, make sure your
> design can handle that. Lots of thinking
> and everyone has problems with money. Especially in the beginning.

The bright side of the case is that we have approx 40k $ on the beggining

one more time - thanks

regards

--
Wojciech Ziniewicz
Unix SEX :{look;gawk;find;sed;talk;grep;touch;finger;find;f l
ex;unzip;head;tail; mount;workbone;fsck;yes;gasp;fsck;more;yes;yes;eje
ct;umount;makeclean; zip;split;done;exit:xargs!!}


--
To UNSUBSCRIBE, email to debian-isp-REQUEST@lists.debian.org
with a subject of "unsubscribe". Trouble? Contact listmaster@lists.debian.org
 
Old 09-13-2008, 01:57 AM
"N.E.S.T. Solutions"
 
Default best FC-storage sharing/clustering techniques

Hi Jean-Paul,

With debian, LVS, and all other avail. softs, you can do it. But to be
able to scale,
you'll have to work hard at it, otherwise, it'll never get simpler to
manage.

The other thing you might want to do is look at www.isilon.com storage
devices, we work
with these as well as EMC & others, but for low cost serious filesystem
scalability, they
can rock your world. Isilon are Infiniband interconnected appliances.

I use them fully synced with file/folder level data protection factors
of n+1 to n+4, around
the world. I have not been deceived.

1h racking, 5 minutes to start a 32TB storage on a site, that's it,
done.

They use OneFS with BSD, so outta the box, it's all web interfaced with
lots of nice features
that would take you a lotta time to put together using GFS or Luster
based file systems, but
on top of that, the OS is open! We install our own packaged applications
in it for additionnal
monitoring, SNMP, etc...

Some large networks using Isilon are for example Kodak (online picture
albums stuff) and Canon,
plus many other ones of course.

If you want to add another 8TB to a 32TB site in one location for
example, you rack the unit,
plug it in (power), turn it on, and you're done! Login, click "add node"
and smile while the
FS gets fat before your eyes!

Throughput has been unreal for us. We use MANY heads-nodes cause each of
them will give you
an additionnal "pipe" for I/O. So, you could have for example 32TB made
of 4 8TB nodes, getting
4x the I/O, but if you do 32TB made of 4 8TB nodes + 4 metadata server
nodes, you get 8x the I/O.

Configuration and design is the key, but sooo simple.

Martin Houle
N.E.S.T. Solutions
Networking Electronics Security Telecoms

:: Owner & Founder :: Senior *nix Systems & Network Administrator ::
E-Mail : martin@nest-solutions.com
Web : http://www.nest-solutions.com

No virus found in this outgoing message.
Checked by AVG.
Version: 7.5.524 / Virus Database: 270.6.21/1668 - Release Date:
9/12/2008 6:56 AM



--
To UNSUBSCRIBE, email to debian-isp-REQUEST@lists.debian.org
with a subject of "unsubscribe". Trouble? Contact listmaster@lists.debian.org
 

Thread Tools




All times are GMT. The time now is 12:59 PM.

VBulletin, Copyright ©2000 - 2014, Jelsoft Enterprises Ltd.
Content Relevant URLs by vBSEO ©2007, Crawlability, Inc.
Copyright 2007 - 2008, www.linux-archive.org