FAQ Search Today's Posts Mark Forums Read
» Video Reviews

» Linux Archive

Linux-archive is a website aiming to archive linux email lists and to make them easily accessible for linux users/developers.


» Sponsor

» Partners

» Sponsor

Go Back   Linux Archive > Debian > Debian ISP

 
 
LinkBack Thread Tools
 
Old 10-21-2010, 06:48 PM
Wojciech Ziniewicz
 
Default question about modern lamp clustering technique

Hi debian-isp,
Probably vast of this group's members are admins dealing with system
engineering and general administration on a daily basis. Most of you
did many LAMP or LAMRoR or whatever installations throughout years.
What I would like to ask You - which suite of apps would you recommend
to create a farm serving one or multiple (doesn't matter) sites
,assumed that we have 1+ boxes
Why I ask is that since 1998 (when I started playing with unix/linux)
I know no good technique of having for example two very strong
physical servers (say bi-quad with 24GB ram each and raid1 1TB disk)
serving one site with apache/mysql/php on it that will squeeze whole
computing power from it. Even if I use XEN (which I like v.much) and
create VMs for separate services Below are problems that i'd like to
avoid :
- one standby server (WTF ? I want two of them sweating at a time)
shared storage :
- In my opinion _unefficient_ shared storage on NFS kernel mode (I
dont have access to my ISP's switches) what is _usually_
single-point-of-failure as it's hard to cluster NFS
- use of DRBD + GFS2 in multimaster mode is of course possible , but I
have really bad feelings about using GFS at all (especially on debian)
database:
- mysql ndb cluster ? Nay sir...
- mysql in multimaster mode : the only solution is
http://jayant7k.blogspot.com/2006/06/multi-master-replication-in-mysql.html
but how silly is that to do such workarounds that add extra several
hours to the after-crash recovery ?
www/proxy :
- generally the best part because there are so many
httpd/proxy/loadbalancers out there like
cherokee/haproxy/thin/nginx/even apache that I don't mention this part
of the problem.

So provided that you have 2 servers ,each capable of 4 strong VMs
(6GBram/2cpu) - how would You divide this lil'farm to have following
services : shared mysql database , shared postgresql database, shared
static content server, non-shared dynamic content (app) server with
php/ruby/whatever workers

Few words on the matter will be enough.
Thank You for reading and answering
Wojtek

--
Wojciech Ziniewicz
http://www.rfc-editor.org/rfc/rfc2324.txt


--
To UNSUBSCRIBE, email to debian-isp-REQUEST@lists.debian.org
with a subject of "unsubscribe". Trouble? Contact listmaster@lists.debian.org
Archive: AANLkTikfh6peVZbEH_BT5-shQvCguaDyRrzw0nqN9zpM@mail.gmail.com">http://lists.debian.org/AANLkTikfh6peVZbEH_BT5-shQvCguaDyRrzw0nqN9zpM@mail.gmail.com
 
Old 10-21-2010, 07:24 PM
Valentine Gostev
 
Default question about modern lamp clustering technique

Hello,

from my point of view load balancing and failover capabilities should
be provided on application level, not OS level.

In this particular case, when you are having 2 rather strong servers
and using virtualization solution I would suggest the following:

Let's look at load of every role you use:

mysql and postgres besides CPU and RAM consume good amount of disk
I/O, that's why the VMs with
mysql and postgres should be located on different hardware nodes to
distribute disk system load.

Dynamic content server (most likely it is Apache with PHP/RoR/Python I
believe) mostly uses CPU and RAM,
so maybe it will be better that we place it to the node where VM with
db server produces less load

nginx is a perfect solution for delivering static content to the site
visitors/app users and also it could perform
the role of a network load balancer,
http://wiki.nginx.org/LoadBalanceExample

And from my young experience I could say that DB clustering is a
rather complicated way and it may also
make some limitations, so, IMHO, the best way is to modify an
application a bit and place different tables
from DB at different DB servers and use them simultaneously.

Hope that my thoughts will help a bit in choosing a proper solution.


2010/10/22 Wojciech Ziniewicz <wojciech.ziniewicz@gmail.com>:
> Hi debian-isp,
> Probably vast of this group's members are admins dealing with system
> engineering and general administration on a daily basis. Most of you
> did many LAMP or LAMRoR or whatever installations throughout years.
> What I would like to ask You - which suite of apps would you recommend
> to create a farm serving one or multiple (doesn't matter) sites
> ,assumed that we have 1+ boxes
> Why I ask is that since 1998 (when I started playing with unix/linux)
> I know no good technique of having for example two very strong
> physical servers (say bi-quad with 24GB ram each and raid1 1TB disk)
> serving one site with apache/mysql/php on it that will squeeze whole
> computing power from it. Even if I use XEN (which I like v.much) and
> create VMs for separate services Below are problems that i'd like to
> avoid :
> - one standby server (WTF ? I want two of them sweating at a time)
> shared storage :
> - In my opinion _unefficient_ shared storage on NFS kernel mode (I
> dont have access to my ISP's switches) what is _usually_
> single-point-of-failure as it's hard to cluster NFS
> - use of DRBD + GFS2 in multimaster mode is of course possible , but I
> have really bad feelings about using GFS at all (especially on debian)
> database:
> - mysql ndb cluster ? Nay sir...
> - mysql in multimaster mode : the only solution is
> http://jayant7k.blogspot.com/2006/06/multi-master-replication-in-mysql.html
> but how silly is that to do such workarounds that add extra several
> hours to the after-crash recovery ?
> www/proxy :
> - generally the best part because there are so many
> httpd/proxy/loadbalancers out there like
> cherokee/haproxy/thin/nginx/even apache that I don't mention this part
> of the problem.
>
> So provided that you have 2 servers ,each capable of 4 strong VMs
> (6GBram/2cpu) - how would You divide this lil'farm to have following
> services : shared mysql database , shared postgresql database, shared
> static content server, non-shared dynamic content (app) server with
> php/ruby/whatever workers
>
> Few words on the matter will be enough.
> Thank You for reading and answering
> Wojtek
>
> --
> Wojciech Ziniewicz
> http://www.rfc-editor.org/rfc/rfc2324.txt
>
>
> --
> To UNSUBSCRIBE, email to debian-isp-REQUEST@lists.debian.org
> with a subject of "unsubscribe". Trouble? Contact listmaster@lists.debian.org
> Archive: http://lists.debian.org/AANLkTikfh6peVZbEH_BT5-shQvCguaDyRrzw0nqN9zpM@mail.gmail.com
>
>

--
Best regards,
Valentine Gostev


--
To UNSUBSCRIBE, email to debian-isp-REQUEST@lists.debian.org
with a subject of "unsubscribe". Trouble? Contact listmaster@lists.debian.org
Archive: AANLkTimWHyZOHe__nhJCZymUuCbtQ=LLO+xvEUv=KB8A@mail .gmail.com">http://lists.debian.org/AANLkTimWHyZOHe__nhJCZymUuCbtQ=LLO+xvEUv=KB8A@mail .gmail.com
 
Old 10-22-2010, 01:49 PM
Wojciech Ziniewicz
 
Default question about modern lamp clustering technique

2010/10/21 Valentine Gostev <core.longbow@gmail.com>:
> Hello,
>
> from my point of view load balancing and failover capabilities should
> be provided on application level, not OS level.

>From my point it looks the same, but it's typical sittuation that we
have software installed.

> In this particular case, when you are having 2 rather strong servers
> and using virtualization solution I would suggest the following:
>
> Let's look at load of every role you use:
>
> mysql and postgres besides CPU and RAM consume good amount of disk
> I/O, that's why the VMs with
> mysql and postgres should be located on different hardware nodes to
> distribute disk system load.

Generally vast of servers that I work with have mysql cache hit rate
about 80-90% so there's no big I/O utilization.
Resources partitioning on xen server is very simple and reliable -
generally VMs have almost direct access to hardware so having 3 or 4
DomU's on such machine is similar to having 3/4 physical 1Us (not
talking about cos-effectiveness)

[...]
> And from my young experience I could say that DB clustering is a
> rather complicated way and it may also
> make some limitations, so, IMHO, the best way is to modify an
> application a bit and place different tables
> from DB at different DB servers and use them simultaneously.

Yup, clustering mysql is EVIL and this is yet the very part that you
have to code your software so it will have loadbalancing hardcoded.

> Hope that my thoughts will help a bit in choosing a proper solution.

Sure, thanks


--
Wojciech Ziniewicz
http://www.rfc-editor.org/rfc/rfc2324.txt


--
To UNSUBSCRIBE, email to debian-isp-REQUEST@lists.debian.org
with a subject of "unsubscribe". Trouble? Contact listmaster@lists.debian.org
Archive: AANLkTi=_7U63LzhxzhyFtnCW1t-foH2pdxbtAW9p+sv3@mail.gmail.com">http://lists.debian.org/AANLkTi=_7U63LzhxzhyFtnCW1t-foH2pdxbtAW9p+sv3@mail.gmail.com
 
Old 10-22-2010, 04:02 PM
mimo
 
Default question about modern lamp clustering technique

Hi Wojciech. You asked for opinions here is mine but I think there might be a possibility I misunderstood your question all together.

On Thursday 21 October 2010 19:48:37 Wojciech Ziniewicz wrote:
> So provided that you have 2 servers ,each capable of 4 strong VMs
> (6GBram/2cpu) - how would You divide this lil'farm to have following
> services : shared mysql database , shared postgresql database, shared
> static content server, non-shared dynamic content (app) server with
> php/ruby/whatever workers

I would use DRBD for failover, run everything on one machine and let the second one just keep up in case the first one fails. Of course it's a waste but you don't want a single point of failure. Actually, if money is an issue (usually is), get a massive primary machine and a not so fast but bareable secondary as it will only be loaded heavily while your fixing the primary.
For reasons of caching by the OS, MySQL, Apache, PHP it's better to have NO virtualisation IMHO because it just duplicates those caches. I don't know - but I didn't really understand why you would use virtualisation.

mimo


--
To UNSUBSCRIBE, email to debian-isp-REQUEST@lists.debian.org
with a subject of "unsubscribe". Trouble? Contact listmaster@lists.debian.org
Archive: 201010221702.49443.mimo@restoel.net">http://lists.debian.org/201010221702.49443.mimo@restoel.net
 

Thread Tools




All times are GMT. The time now is 08:49 PM.

VBulletin, Copyright ©2000 - 2014, Jelsoft Enterprises Ltd.
Content Relevant URLs by vBSEO ©2007, Crawlability, Inc.
Copyright 2007 - 2008, www.linux-archive.org