FAQ Search Today's Posts Mark Forums Read
» Video Reviews

» Linux Archive

Linux-archive is a website aiming to archive linux email lists and to make them easily accessible for linux users/developers.


» Sponsor

» Partners

» Sponsor

Go Back   Linux Archive > CentOS > CentOS

 
 
LinkBack Thread Tools
 
Old 10-25-2010, 07:31 PM
Iain Morris
 
Default interesting kvm lvm collision issue

I've been running into a reproducible problem when using default LVM volume group names to present block devices for virtual machines in KVM, and I'm wondering why it is happening.
On dom0 I make a default VolGroup00 for the operating system. *I make a second VolGroup01 for logical volumes that will be block devices for virtual systems.

In VolGroup01, I make two lv's for one system: lv.sys1, and lv.sys1-data. *
I then build a new virtual machine called sys1, using lv.sys1 for the root filesystem, and lv.sys1-data for an independent data partition. *Everything works great after installation, and vgdisplay on both systems looks great.

If I then run vgscan, however, on the host system, it picks up the VolGroup01 I created _within_ the virtual machine, so I now have 2 VolGroup01's with different UUIDs showing up on dom0.

Now I can see how vgscan would mistakenly see the VolGroup01 of sys1 on the block device lv.sys1-data, but why are the VolGroup00 vg's not colliding as well?
When a pvdisplay is run, I have a new "physical volume" that is actually just a logical volume of the original VolGroup01:

[root@iain2 ~]# pvdisplay**WARNING: Duplicate VG name VolGroup01: Existing FNiKc9-BB3t-ziMg-prWW-n8RA-OMzk-obiKnf (created here) takes precedence over C8fNMV-aeSW-syIn-fWJZ-vJdK-N0As-Itrvfi
**WARNING: Duplicate VG name VolGroup01: Existing FNiKc9-BB3t-ziMg-prWW-n8RA-OMzk-obiKnf (created here) takes precedence over C8fNMV-aeSW-syIn-fWJZ-vJdK-N0As-Itrvfi**--- Physical volume ---**PV Name * * * * * * * /dev/VolGroup01/lv-sys1-data
**VG Name * * * * * * * VolGroup01**PV Size * * * * * * * 40.00 GB / not usable 4.00 MB**Allocatable * * * * * yes (but full)**PE Size (KByte) * * * 4096**Total PE * * * * * * *10239
**Free PE * * * * * * * 0**Allocated PE * * * * *10239**PV UUID * * * * * * * FTA4QU-ydZ7-e2Yy-nBsi-t4st-3jj7-IAkQH8*****--- Physical volume ---**PV Name * * * * * * * /dev/sda3
**VG Name * * * * * * * VolGroup00**PV Size * * * * * * * 39.06 GB / not usable 29.77 MB**Allocatable * * * * * yes (but full)**PE Size (KByte) * * * 32768**Total PE * * * * * * *1249
**Free PE * * * * * * * 0**Allocated PE * * * * *1249**PV UUID * * * * * * * tTViks-3lBM-HGzV-mnN9-zRsT-fFT0-ZsJRse*****--- Physical volume ---**PV Name * * * * * * * /dev/sda2
**VG Name * * * * * * * VolGroup01**PV Size * * * * * * * 240.31 GB / not usable 25.75 MB**Allocatable * * * * * yes***PE Size (KByte) * * * 32768**Total PE * * * * * * *7689
**Free PE * * * * * * * 5129**Allocated PE * * * * *2560**PV UUID * * * * * * * ZE5Io3-WYIO-EfOQ-h03q-zGdF-Frpa-tm63fX


Has anyone experienced this? *It's very unnerving to know your data is intact as you add new logical volumes for kvm systems. *I suppose the lesson learned here is to provide VGs with specific host names.



--
-- -
Iain Morris
iain.t.morris@gmail.com


_______________________________________________
CentOS mailing list
CentOS@centos.org
http://lists.centos.org/mailman/listinfo/centos
 
Old 10-25-2010, 09:18 PM
Gordon Messmer
 
Default interesting kvm lvm collision issue

On 10/25/2010 12:31 PM, Iain Morris wrote:
> I then build a new virtual machine called sys1, using lv.sys1 for the
> root filesystem, and lv.sys1-data for an independent data partition.
> Everything works great after installation, and vgdisplay on both
> systems looks great.
>
> If I then run vgscan, however, on the host system, it picks up the
> VolGroup01 I created _within_ the virtual machine, so I now have 2
> VolGroup01's with different UUIDs showing up on dom0.

Which block devices are you exporting to your guest? Post the libvirt
configuration file for it.
_______________________________________________
CentOS mailing list
CentOS@centos.org
http://lists.centos.org/mailman/listinfo/centos
 
Old 10-25-2010, 09:33 PM
Iain Morris
 
Default interesting kvm lvm collision issue

On Mon, Oct 25, 2010 at 2:18 PM, Gordon Messmer <yinyang@eburg.com> wrote:



Which block devices are you exporting to your guest? *Post the libvirt

configuration file for it.
See below. *It's specifically the second volume group that collides between virtual and physical systems. *Both dom0 and U have identical VolGroup00 VGs, but these do not collide.

Renaming the VG used by the domU within the domU removes the collision, but the newly-renamed VG still shows up in dom0 as a useable VG with space to be allocated. **
Here is the pvdisplay output from dom0. *Interestingly, it shows /dev/VolGroup01/lv-sys1-data as "physical volume" *when it's obviously just an LV in the original VolGroup01 VG. *And this only happens with the _second_ Volume Group created. *VolGroup00 is not an issue on this or any other systems I've used:

[root@iain2 qemu]# pvdisplay**--- Physical volume ---**PV Name * * * * * * * /dev/VolGroup01/lv-sys1-data**VG Name * * * * * * * vg-sys1**PV Size * * * * * * * 40.00 GB / not usable 4.00 MB
**Allocatable * * * * * yes (but full)**PE Size (KByte) * * * 4096**Total PE * * * * * * *10239**Free PE * * * * * * * 0**Allocated PE * * * * *10239**PV UUID * * * * * * * FTA4QU-ydZ7-e2Yy-nBsi-t4st-3jj7-IAkQH8
*****--- Physical volume ---**PV Name * * * * * * * /dev/sda3**VG Name * * * * * * * VolGroup00**PV Size * * * * * * * 39.06 GB / not usable 29.77 MB**Allocatable * * * * * yes (but full)
**PE Size (KByte) * * * 32768**Total PE * * * * * * *1249**Free PE * * * * * * * 0**Allocated PE * * * * *1249**PV UUID * * * * * * * tTViks-3lBM-HGzV-mnN9-zRsT-fFT0-ZsJRse
*****--- Physical volume ---**PV Name * * * * * * * /dev/sda2**VG Name * * * * * * * VolGroup01**PV Size * * * * * * * 240.31 GB / not usable 25.75 MB**Allocatable * * * * * yes*
**PE Size (KByte) * * * 32768**Total PE * * * * * * *7689**Free PE * * * * * * * 5129**Allocated PE * * * * *2560**PV UUID * * * * * * * ZE5Io3-WYIO-EfOQ-h03q-zGdF-Frpa-tm63fX



[root@iain2 qemu]# cat sys1.xml*<domain type='kvm'>**<name>sys1</name>**<uuid>37f34394-d380-d2c4-ac37-3263c16028ff</uuid>
**<memory>524288</memory>**<currentMemory>524288</currentMemory>**<vcpu>1</vcpu>**<os>** *<type arch='x86_64' machine='rhel5.4.0'>hvm</type>
** *<boot dev='hd'/>**</os>**<features>** *<acpi/>** *<apic/>** *<pae/>**</features>**<clock offset='utc'/>
**<on_poweroff>destroy</on_poweroff>**<on_reboot>restart</on_reboot>**<on_crash>restart</on_crash>**<devices>** *<emulator>/usr/libexec/qemu-kvm</emulator>
** *<disk type='block' device='disk'>** * *<driver name='qemu' cache='none'/>** * *<source dev='/dev/VolGroup01/lv-sys1'/>** * *<target dev='vda' bus='virtio'/>
** *</disk>** *<disk type='block' device='cdrom'>** * *<target dev='hdc' bus='ide'/>** * *<readonly/>** *</disk>
** *<disk type='block' device='disk'>** * *<source dev='/dev/VolGroup01/lv-sys1-data'/>** * *<target dev='vdb' bus='virtio'/>
** *</disk>** *<interface type='network'>** * *<mac address='54:52:00:3b:4a:f5'/>** * *<source network='default'/>** * *<model type='virtio'/>
** *</interface>** *<serial type='pty'>** * *<source path='/dev/pts/3'/>** * *<target port='0'/>** *</serial>
** *<console type='pty' tty='/dev/pts/3'>** * *<source path='/dev/pts/3'/>** * *<target port='0'/>** *</console>** *<input type='mouse' bus='ps2'/>
** *<graphics type='vnc' port='-1' autoport='yes' keymap='en-us'/>**</devices></domain>

*
--
-- -
Iain Morris
iain.t.morris@gmail.com

_______________________________________________
CentOS mailing list
CentOS@centos.org
http://lists.centos.org/mailman/listinfo/centos
 
Old 10-26-2010, 01:48 PM
Ross Walker
 
Default interesting kvm lvm collision issue

On Oct 25, 2010, at 3:31 PM, Iain Morris <iain.t.morris@gmail.com> wrote:

> I've been running into a reproducible problem when using default LVM volume group names to present block devices for virtual machines in KVM, and I'm wondering why it is happening.
>
> On dom0 I make a default VolGroup00 for the operating system. I make a second VolGroup01 for logical volumes that will be block devices for virtual systems.
>
> In VolGroup01, I make two lv's for one system: lv.sys1, and lv.sys1-data.
>
> I then build a new virtual machine called sys1, using lv.sys1 for the root filesystem, and lv.sys1-data for an independent data partition. Everything works great after installation, and vgdisplay on both systems looks great.
>
> If I then run vgscan, however, on the host system, it picks up the VolGroup01 I created _within_ the virtual machine, so I now have 2 VolGroup01's with different UUIDs showing up on dom0.
>
> Now I can see how vgscan would mistakenly see the VolGroup01 of sys1 on the block device lv.sys1-data, but why are the VolGroup00 vg's not colliding as well?
>
> When a pvdisplay is run, I have a new "physical volume" that is actually just a logical volume of the original VolGroup01:
>
> [root@iain2 ~]# pvdisplay
> WARNING: Duplicate VG name VolGroup01: Existing FNiKc9-BB3t-ziMg-prWW-n8RA-OMzk-obiKnf (created here) takes precedence over C8fNMV-aeSW-syIn-fWJZ-vJdK-N0As-Itrvfi
> WARNING: Duplicate VG name VolGroup01: Existing FNiKc9-BB3t-ziMg-prWW-n8RA-OMzk-obiKnf (created here) takes precedence over C8fNMV-aeSW-syIn-fWJZ-vJdK-N0As-Itrvfi
> --- Physical volume ---
> PV Name /dev/VolGroup01/lv-sys1-data
> VG Name VolGroup01
> PV Size 40.00 GB / not usable 4.00 MB
> Allocatable yes (but full)
> PE Size (KByte) 4096
> Total PE 10239
> Free PE 0
> Allocated PE 10239
> PV UUID FTA4QU-ydZ7-e2Yy-nBsi-t4st-3jj7-IAkQH8
>
> --- Physical volume ---
> PV Name /dev/sda3
> VG Name VolGroup00
> PV Size 39.06 GB / not usable 29.77 MB
> Allocatable yes (but full)
> PE Size (KByte) 32768
> Total PE 1249
> Free PE 0
> Allocated PE 1249
> PV UUID tTViks-3lBM-HGzV-mnN9-zRsT-fFT0-ZsJRse
>
> --- Physical volume ---
> PV Name /dev/sda2
> VG Name VolGroup01
> PV Size 240.31 GB / not usable 25.75 MB
> Allocatable yes
> PE Size (KByte) 32768
> Total PE 7689
> Free PE 5129
> Allocated PE 2560
> PV UUID ZE5Io3-WYIO-EfOQ-h03q-zGdF-Frpa-tm63fX
>
>
>
> Has anyone experienced this? It's very unnerving to know your data is intact as you add new logical volumes for kvm systems. I suppose the lesson learned here is to provide VGs with specific host names.

You need to exclude the LVs in the host VG from being scanned for sub-VGs. It's actually easier to just list what SHOULD be scanned rather than what shouldn't.

Look in /etc/lvm/lvm.conf

You can also avoid this by creating partition based PVs in the VMs rather then whole disk PVs which would need kpartx run on the host LV before LVM could scan it.

-Ross

_______________________________________________
CentOS mailing list
CentOS@centos.org
http://lists.centos.org/mailman/listinfo/centos
 
Old 10-27-2010, 07:02 PM
Iain Morris
 
Default interesting kvm lvm collision issue

On Tue, Oct 26, 2010 at 6:48 AM, Ross Walker <rswwalker@gmail.com> wrote:
*
You need to exclude the LVs in the host VG from being scanned for sub-VGs. It's actually easier to just list what SHOULD be scanned rather than what shouldn't.


Look in /etc/lvm/lvm.conf

This worked, thanks. *A couple of people emailed me separately on this. *For others' reference, I added the following filter to lvm.conf on dom0 and disabled the default "get everything" filter. *If anyone sees any pitfalls with this regex, I'm sure you'll let me know.

(assuming your physical disks are sas/sata)
filter = [ "a/^/dev/sd*/", "r/.*/" ]
Thanks for the help,
-Iain

--*-- -
Iain Morris
iain.t.morris@gmail.com

_______________________________________________
CentOS mailing list
CentOS@centos.org
http://lists.centos.org/mailman/listinfo/centos
 
Old 10-27-2010, 10:53 PM
Ross Walker
 
Default interesting kvm lvm collision issue

On Oct 27, 2010, at 3:02 PM, Iain Morris <iain.t.morris@gmail.com> wrote:



On Tue, Oct 26, 2010 at 6:48 AM, Ross Walker <rswwalker@gmail.com> wrote:
*
You need to exclude the LVs in the host VG from being scanned for sub-VGs. It's actually easier to just list what SHOULD be scanned rather than what shouldn't.


Look in /etc/lvm/lvm.conf

This worked, thanks. *A couple of people emailed me separately on this. *For others' reference, I added the following filter to lvm.conf on dom0 and disabled the default "get everything" filter. *If anyone sees any pitfalls with this regex, I'm sure you'll let me know.

(assuming your physical disks are sas/sata)
filter = [ "a/^/dev/sd*/", "r/.*/" ]
For all HDs this should work:
filter = [ "a/^/dev/[hs]d*/", "r/.*/" ]
Run a 'vgscan -vv' to test.
-Ross
_______________________________________________
CentOS mailing list
CentOS@centos.org
http://lists.centos.org/mailman/listinfo/centos
 

Thread Tools




All times are GMT. The time now is 12:11 AM.

VBulletin, Copyright ©2000 - 2014, Jelsoft Enterprises Ltd.
Content Relevant URLs by vBSEO ©2007, Crawlability, Inc.
Copyright 2007 - 2008, www.linux-archive.org