FAQ Search Today's Posts Mark Forums Read
» Video Reviews

» Linux Archive

Linux-archive is a website aiming to archive linux email lists and to make them easily accessible for linux users/developers.


» Sponsor

» Partners

» Sponsor

Go Back   Linux Archive > Debian > Debian User

 
 
LinkBack Thread Tools
 
Old 01-22-2009, 02:43 AM
Ron Johnson
 
Default questions about lvm2

Hi,

I'm about to get a 1TB drive which I will use as the basis of an lvm
set.

1. Since sdX device names are non-deterministic, can I use it with
devices specified as UUIDs? Or does lvm handle that for me on boot?

2. Why should I partitioning this drive into smaller chunks if
they're all going to go into the same volume?


--
Ron Johnson, Jr.
Jefferson LA USA

"I am not surprised, for we live long and are celebrated poopers."


--
To UNSUBSCRIBE, email to debian-user-REQUEST@lists.debian.org
with a subject of "unsubscribe". Trouble? Contact listmaster@lists.debian.org
 
Old 01-22-2009, 02:59 AM
Matthew Moore
 
Default questions about lvm2

On Wednesday 21 January 2009 08:43:02 pm Ron Johnson wrote:
> 1. Since sdX device names are non-deterministic, can I use it with
> devices specified as UUIDs? Or does lvm handle that for me on boot?

lvm stores volume identifiers in a header at the start of the physical
volumes, so the device names/uuids don't matter.

> 2. Why should I partitioning this drive into smaller chunks if
> they're all going to go into the same volume?

It doesn't make sense to partition the drive into multiple smaller partitions
if they are all going onto the same volume group. If you want multiple volume
groups then you will have to bust the drive up into multiple partitions. If
you only want one volume group, then you can even get away with having no
partitions on the drive (though the lvm docs recommend against this for
compatibility with other lvm implementations).

MM


--
To UNSUBSCRIBE, email to debian-user-REQUEST@lists.debian.org
with a subject of "unsubscribe". Trouble? Contact listmaster@lists.debian.org
 
Old 01-22-2009, 03:13 AM
Ron Johnson
 
Default questions about lvm2

On 01/21/2009 09:59 PM, Matthew Moore wrote:

On Wednesday 21 January 2009 08:43:02 pm Ron Johnson wrote:

1. Since sdX device names are non-deterministic, can I use it with
devices specified as UUIDs? Or does lvm handle that for me on boot?


lvm stores volume identifiers in a header at the start of the physical
volumes, so the device names/uuids don't matter.



2. Why should I partitioning this drive into smaller chunks if
they're all going to go into the same volume?


It doesn't make sense to partition the drive into multiple smaller partitions
if they are all going onto the same volume group. If you want multiple volume
groups then you will have to bust the drive up into multiple partitions. If
you only want one volume group, then you can even get away with having no
partitions on the drive (though the lvm docs recommend against this for
compatibility with other lvm implementations).


Thanks.

If I have lots of existing data in JBODs, would I create a PV and VG
on the new drive, mv all the data from the existing drives to the
new VG, then add my existing drives (while also enlarging the fs) to
the one-drive VG, thus making an uber-device?


Also, what's a good HOWTO? Everything I've found via Google seems
out-of-date.


--
Ron Johnson, Jr.
Jefferson LA USA

"I am not surprised, for we live long and are celebrated poopers."


--
To UNSUBSCRIBE, email to debian-user-REQUEST@lists.debian.org
with a subject of "unsubscribe". Trouble? Contact listmaster@lists.debian.org
 
Old 01-22-2009, 04:30 AM
Matthew Moore
 
Default questions about lvm2

On Wednesday 21 January 2009 09:13:07 pm Ron Johnson wrote:
> If I have lots of existing data in JBODs, would I create a PV and VG
> on the new drive, mv all the data from the existing drives to the
> new VG, then add my existing drives (while also enlarging the fs) to
> the one-drive VG, thus making an uber-device?

That is exactly what I would do. Everyone warns about how cataclysmic things
will happen because your new uber-device has the same failrate as a raid-0
setup, but as long as you backup regularly (perhaps to another LV made from
PV's on different disks) this isn't too much to worry about.

> Also, what's a good HOWTO? Everything I've found via Google seems
> out-of-date.

When I first set up LVM I used the LVM2 howto here

http://tldp.org/HOWTO/LVM-HOWTO/

The "anatomy" and "common tasks" are really all you need.

Here are the notes I made when I set up LVM. First partition the drives you
want to use. Make sure to set the type of the partitions as linux LVM. Next
tell LVM2 about them with

$ pvcreate <device name>

You can see what devices are seen by lvm2 with pvdisplay. Next make a VG

$ vgcreate <vg name> <device1> <device2> ...

At this point you might need to activate the volume group

$ vgchange -a y <vg name>

Next make a LV in the VG

$ lvcreate -n <lv name> -l <extents> <vg name>

You can get the number of extents available in a volume group with vgdisplay.
There should now be a new device file at /dev/<vg name>/<lv name> that you
can put a filesystem on. You can only enter /dev/<vg name> as root, but the
device-mapper framework puts links to the LV's in /dev/mapper/<vg name>_<lv
name> when you it gets started at boot (this is the location most people
point to in their fstab).

Since you're already using the device-mapper, you also might want to set up
encryption with cryptsetup.

MM


--
To UNSUBSCRIBE, email to debian-user-REQUEST@lists.debian.org
with a subject of "unsubscribe". Trouble? Contact listmaster@lists.debian.org
 
Old 01-22-2009, 05:24 AM
Ron Johnson
 
Default questions about lvm2

On 01/21/2009 11:30 PM, Matthew Moore wrote:

On Wednesday 21 January 2009 09:13:07 pm Ron Johnson wrote:

If I have lots of existing data in JBODs, would I create a PV and VG
on the new drive, mv all the data from the existing drives to the
new VG, then add my existing drives (while also enlarging the fs) to
the one-drive VG, thus making an uber-device?


That is exactly what I would do. Everyone warns about how cataclysmic things
will happen because your new uber-device has the same failrate as a raid-0
setup, but as long as you backup regularly (perhaps to another LV made from
PV's on different disks) this isn't too much to worry about.


Backing up 2+ TB of data will require another 2x 1TB drives. And
*another* dual-drive external enclosure.


My wife will *not* be happy...


Also, what's a good HOWTO? Everything I've found via Google seems
out-of-date.


When I first set up LVM I used the LVM2 howto here

http://tldp.org/HOWTO/LVM-HOWTO/

The "anatomy" and "common tasks" are really all you need.

Here are the notes I made when I set up LVM. First partition the drives you
want to use. Make sure to set the type of the partitions as linux LVM. Next
tell LVM2 about them with


$ pvcreate <device name>

You can see what devices are seen by lvm2 with pvdisplay. Next make a VG

$ vgcreate <vg name> <device1> <device2> ...

At this point you might need to activate the volume group

$ vgchange -a y <vg name>

Next make a LV in the VG

$ lvcreate -n <lv name> -l <extents> <vg name>

You can get the number of extents available in a volume group with vgdisplay.
There should now be a new device file at /dev/<vg name>/<lv name> that you
can put a filesystem on. You can only enter /dev/<vg name> as root, but the
device-mapper framework puts links to the LV's in /dev/mapper/<vg name>_<lv
name> when you it gets started at boot (this is the location most people
point to in their fstab).


Thanks for the tips. I'm going to save this.

Since you're already using the device-mapper, you also might want to set up
encryption with cryptsetup.




Nothing but (non-pr0n) videos...

--
Ron Johnson, Jr.
Jefferson LA USA

"I am not surprised, for we live long and are celebrated poopers."


--
To UNSUBSCRIBE, email to debian-user-REQUEST@lists.debian.org
with a subject of "unsubscribe". Trouble? Contact listmaster@lists.debian.org
 
Old 01-22-2009, 04:20 PM
"Douglas A. Tutty"
 
Default questions about lvm2

On Thu, Jan 22, 2009 at 12:24:45AM -0600, Ron Johnson wrote:
> On 01/21/2009 11:30 PM, Matthew Moore wrote:
> >On Wednesday 21 January 2009 09:13:07 pm Ron Johnson wrote:

> >That is exactly what I would do. Everyone warns about how cataclysmic
> >things will happen because your new uber-device has the same failrate as a
> >raid-0 setup, but as long as you backup regularly (perhaps to another LV
> >made from PV's on different disks) this isn't too much to worry about.
>
> Backing up 2+ TB of data will require another 2x 1TB drives. And
> *another* dual-drive external enclosure.
>
> My wife will *not* be happy...

Remember, though, that you're adding older drives to the uber-device.
Even with new drives, the chance of failure increases with more devices.
This is why RAID was invented.

As for backing up multi-TB datasets, this is why tape drives were
invented.

Doug.


--
To UNSUBSCRIBE, email to debian-user-REQUEST@lists.debian.org
with a subject of "unsubscribe". Trouble? Contact listmaster@lists.debian.org
 
Old 01-22-2009, 04:41 PM
Ron Johnson
 
Default questions about lvm2

On 01/22/2009 11:20 AM, Douglas A. Tutty wrote:

On Thu, Jan 22, 2009 at 12:24:45AM -0600, Ron Johnson wrote:

On 01/21/2009 11:30 PM, Matthew Moore wrote:

On Wednesday 21 January 2009 09:13:07 pm Ron Johnson wrote:


That is exactly what I would do. Everyone warns about how cataclysmic
things will happen because your new uber-device has the same failrate as a
raid-0 setup, but as long as you backup regularly (perhaps to another LV
made from PV's on different disks) this isn't too much to worry about.
Backing up 2+ TB of data will require another 2x 1TB drives. And
*another* dual-drive external enclosure.


My wife will *not* be happy...


Remember, though, that you're adding older drives to the uber-device.
Even with new drives, the chance of failure increases with more devices.
This is why RAID was invented.


None are that old. Smallest is 640GB.


As for backing up multi-TB datasets, this is why tape drives were
invented.


She's very tolerant of my hobby. Bringing in tape drives and silos
would break that...


--
Ron Johnson, Jr.
Jefferson LA USA

"I am not surprised, for we live long and are celebrated poopers."


--
To UNSUBSCRIBE, email to debian-user-REQUEST@lists.debian.org
with a subject of "unsubscribe". Trouble? Contact listmaster@lists.debian.org
 
Old 01-23-2009, 04:28 PM
Mike Castle
 
Default questions about lvm2

On Wed, Jan 21, 2009 at 8:13 PM, Ron Johnson <ron.l.johnson@cox.net> wrote:
>
> If I have lots of existing data in JBODs, would I create a PV and VG on the
> new drive, mv all the data from the existing drives to the new VG, then add
> my existing drives (while also enlarging the fs) to the one-drive VG, thus
> making an uber-device?


That is how I started and how I've been, more or less, running with
LVM for years, though I think I'm about to change.

First, I've actually had different FS with different settings for
different purposes. Mostly different bytes-per-inode for file system
that have lots of big or lots of small files. Some have ext2 vs ext3
differences. Some are backups of retired windows machines that are
just readonly. And i'd do the occasional snapshot to do backups from.

As each FS would fill up, I would extend that particular FS. Some
would reach a steady state. One interesting side affect of this is
the fact that I would fragmentation at the filesystem level, as
opposed to the file level. Every once in a while, I'd move LVs around
to defrag (usually as part of adding a newer larger harddrive, and
retiring the smaller one).

One pain point I have is this: In order to run resize2fs(8), you have
to fsck the FS first. As they grow larger and larger, this takes
longer and longer. I have a 2TB FS now that I really don't want to
grow any more because of that.

Ext4 may solve the fscking issue, but in the one article I've read so
far, resizing wasn't mentioned in it. (I'm also considering ext4 for
other reasons.)

One thing I do tend to do with every disk is this:

I put a small swap partition on each disk, then the rest is an LVM
partition. Depending on the machine, it could be 128M to 256M; I try
to stay consistent across disks. I then set up each swap partition
with the same priority. That way I get more spindles in action for
swap. I have really no idea if it makes a difference or not, but it's
something I do. It may very well actually cause more contention
because if I'm doing something that is causing me to page, then I'm
probably processing data on all of those disks anyway.

Anyway, that's my experience with LVM. I like it, I've used it for,
mmmm... I'm not sure how long I've used it... has it been around for
10 years? I can't remember when I switched to it, but I'm pretty sure
it was before I moved to the South Bay Area. It took for years of
lobbying to get folks to start using it at work for one service I'm
responsible for; down time for backups dropped from one hour to one
minute, since they can now do LVM based snapshots and bring the
service right back up.

mrc


--
To UNSUBSCRIBE, email to debian-user-REQUEST@lists.debian.org
with a subject of "unsubscribe". Trouble? Contact listmaster@lists.debian.org
 
Old 01-23-2009, 09:32 PM
Ron Johnson
 
Default questions about lvm2

On 01/23/2009 11:28 AM, Mike Castle wrote:

On Wed, Jan 21, 2009 at 8:13 PM, Ron Johnson <ron.l.johnson@cox.net> wrote:

If I have lots of existing data in JBODs, would I create a PV and VG on the
new drive, mv all the data from the existing drives to the new VG, then add
my existing drives (while also enlarging the fs) to the one-drive VG, thus
making an uber-device?



That is how I started and how I've been, more or less, running with
LVM for years, though I think I'm about to change.


Over an 8 hour period, I grew my lv from one device to 4, resizing
an ext4 fs each time.


Note that I'm running 2.6.28 from:
deb http://kernel-archive.buildserver.net/debian-kernel trunk main

[snip]


One pain point I have is this: In order to run resize2fs(8), you have
to fsck the FS first. As they grow larger and larger, this takes
longer and longer. I have a 2TB FS now that I really don't want to
grow any more because of that.

Ext4 may solve the fscking issue, but in the one article I've read so


Yup, it does!!! (The required version of e2fsprogs is in Sid,
probably in Lenny, and probably isn't in Etch.)


# df -TH /data/big
Filesystem Type Size Used Avail Use% Mounted on
/dev/dm-0 ext4 3.0T 1.4T 1.5T 48% /data/big

# umount -v /data/big
/dev/dm-0 umounted

# time e2fsck -pfC0 /dev/main_huge_vg/main_huge_lv
/dev/main_huge_vg/main_huge_lv: 515937/180412416 files
(1.9% non-contiguous), 333951921/721649664 blocks

real 3m10.506s
user 2m21.589s
sys 0m4.020s


far, resizing wasn't mentioned in it. (I'm also considering ext4 for
other reasons.)
One thing I do tend to do with every disk is this:

I put a small swap partition on each disk, then the rest is an LVM


Stuffing your mobo with *dirt-cheap* RAM is the other way to solve
this particular problem!!!


[big snip]

--
Ron Johnson, Jr.
Jefferson LA USA

"I am not surprised, for we live long and are celebrated poopers."


--
To UNSUBSCRIBE, email to debian-user-REQUEST@lists.debian.org
with a subject of "unsubscribe". Trouble? Contact listmaster@lists.debian.org
 

Thread Tools




All times are GMT. The time now is 07:15 AM.

VBulletin, Copyright ©2000 - 2014, Jelsoft Enterprises Ltd.
Content Relevant URLs by vBSEO ©2007, Crawlability, Inc.
Copyright 2007 - 2008, www.linux-archive.org