FAQ Search Today's Posts Mark Forums Read
» Video Reviews

» Linux Archive

Linux-archive is a website aiming to archive linux email lists and to make them easily accessible for linux users/developers.


» Sponsor

» Partners

» Sponsor

Go Back   Linux Archive > Debian > Debian Development

 
 
LinkBack Thread Tools
 
Old 03-16-2009, 05:13 PM
Joel Granados Moreno
 
Default Handle system crappyness.

* storage/devicelibs/lvm.py (zeroLvmMetadata): zero out the LVM metadata
only. This basically means from 513 byte and on. This will also
ignore the firs 512 bytes if we are in a partition, but that does not
matter because LVM does not touch those bits anyways.
* storage/devicelibs/lvm.py (pv_complete): Complete here means that all
the related PVs for the containing VG are present in the system.
Returns the Completeness value and a list of known devices.
* storage/devicelibs/lvm.py (vg_complete): likewise but for VG.
* storage/devicelibs/lvm.py (vgreduce): introduces a new argument to the
function. rm means use the lvm --removemissing option.

* storage/devicetree.py (_handleSysCrapyness): New function intended to
catch all unwanted behavior from the system before continuing.
* storage/devicetree.py (questionReinitializeIVG): New function intended
to ask the user what to do in case we find inconsistent LVM metadata.
---
storage/devicelibs/lvm.py | 127 +++++++++++++++++++++++++++++++++++++++++++--
storage/devicetree.py | 104 ++++++++++++++++++++++++++++++++++++
2 files changed, 226 insertions(+), 5 deletions(-)

diff --git a/storage/devicelibs/lvm.py b/storage/devicelibs/lvm.py
index 0a19711..df21ea9 100644
--- a/storage/devicelibs/lvm.py
+++ b/storage/devicelibs/lvm.py
@@ -99,6 +99,22 @@ def lvm_cc_addFilterRejectRegexp(regexp):
_composeConfig()
# End config_args handling code.

+def zeroLvmMetadata(device):
+ """ This dds from byte 513 to 200Mb+513 of the disk.
+
+ We assume that lvm has nothing to do with the first 512 bytes.
+ We assume that 200Mb is enough. My tests failed with 100Mb for
+ some unknown reason.
+ """
+ try:
+ fd = os.open(device, os.O_WRONLY)
+ os.lseek(fd, 513, os.SEEK_SET)
+ for i in range(200):
+ os.write(fd, ""*1024)
+ os.close(fd)
+ except:
+ raise LVMError("Falied to zero LVM data on %s." % device)
+
def getPossiblePhysicalExtents(floor=0):
"""Returns a list of integers representing the possible values for
the physical extent of a volume group. Value is in KB.
@@ -229,6 +245,100 @@ def pvinfo(device):

return info

+# Start of LVM consistency code
+#
+# PV_{,NOT}_COMPLETE: A PV is complete when it is part of a complete VG or
+# it is part of no VG. It is incomplete otherwise.
+# VG_{,NOT}_COMPLETE: A VG is complete when all its PVs are accounted for
+# by LVM metadata. More specifically, the `lvm vgs`
+# command has no PVs described as "unknown device"
+
+PV_COMPLETE = 0
+PV_NOT_COMPLETE = 1
+VG_COMPLETE = 0
+VG_NOT_COMPLETE = 1
+
+def pv_complete(device):
+ """ Check completeness of lvm structure with Physical Volume device.
+
+ Returns : (state, pv_list)
+ (None, None) means that device is not a PV.
+
+ state: [PV_COMPLETE | PV_NOT_COMPLETE]
+ pv_list: List of PVs related to VG. If no related VG is found then return
+ [device]
+ """
+ # Make sure this is a PV.
+ args = ["pvs"] +
+ ["--noheadings"] +
+ ["-o", "pv_name"] +
+ [device]
+ rc = iutil.execWithCapture("lvm", args,
+ stderr = "/dev/tty5")
+ if rc == None:
+ # This is not a PV.
+ return (None, None)
+
+ # Get the VG name related to the PV.
+ args = ["pvs"] +
+ ["--noheadings"] +
+ ["-o", "vg_name"] +
+ [device]
+ rc = iutil.execWithCapture("lvm", args,
+ stderr = "/dev/tty5")
+ if rc == None:
+ # Has no VG, but its ok
+ # FIXME: should look at stderr.
+ return (PV_COMPLETE, [device])
+ vg_name = rc.strip()
+
+ # Volume Group Completeness (vgs)
+ (vgc, devices) = vgstatus_ok(vg_name)
+ if vgc == None:
+ return (None, None)
+ elif vgc == VG_NOT_COMPLETE:
+ return (PV_NOT_COMPLETE, devices)
+ elif vgc == VG_COMPLETE:
+ return (PV_COMPLETE, devices)
+ else:
+ raise LVMError("Error checking completeness of %s" % vg_name)
+
+def vgcomplete(vg_name):
+ """ Check validity of lvm structure with Physical Volume device.
+
+ Returns : (state, pv_list)
+ (None, None) means that device is not a VG.
+
+ state: [VG_COMPLETE | VG_NOT_COMPLETE]
+ pv_list: List of PVs related to VG.
+ """
+ # Ask for the names of the PVs.
+ args = ["vgs"] +
+ ["--noheadings"] +
+ ["-o", "pv_name"] +
+ [vg_name]
+ rc = iutil.execWithCapture("lvm", args,
+ stderr = "/dev/tty5")
+
+ if rc == None:
+ # This is not a VG.
+ return (None, None)
+
+ # we make sure that every element is striped. Also, we don't want to
+ # return "unknown device"
+ pv_names = rc.strip("
").split("
")
+ ret_pv_names = []
+ completeness = VG_COMPLETE
+ for i in range(len(pv_names)):
+ if pv_names[i].strip() != "unknown device":
+ ret_pv_names.append(pv_names[i].strip())
+ else:
+ completeness = VG_NOT_COMPLETE
+
+ return (completeness, pv_names)
+#
+# End of LVM consistency code
+
def vgcreate(vg_name, pv_list, pe_size):
argv = ["vgcreate"]
if pe_size:
@@ -283,11 +393,18 @@ def vgdeactivate(vg_name):
if rc:
raise LVMError("vgdeactivate failed for %s" % vg_name)

-def vgreduce(vg_name, pv_list):
- args = ["vgreduce"] +
- config_args +
- [vg_name] +
- pv_list
+def vgreduce(vg_name, pv_list, rm=False):
+ """ Reduce a VG.
+
+ rm -> with RemoveMissing option.
+ Use pv_list when rm=False, otherwise ignore pv_list and call vgreduce with
+ the --removemissing option.
+ """
+ args = ["vgreduce"]
+ if rm:
+ args.extend(["--removemissing", vg_name])
+ else:
+ args.extend([vg_name] + pv_list)

rc = iutil.execWithRedirect("lvm", args,
stdout = "/dev/tty5",
diff --git a/storage/devicetree.py b/storage/devicetree.py
index 66d053e..1b4fcbd 100644
--- a/storage/devicetree.py
+++ b/storage/devicetree.py
@@ -136,6 +136,35 @@ def questionInitializeDisk(intf=None, name=None):
retVal = True
return retVal

+def questionReinitializeIVG(intf=None, vg_name=None, pv_names=None):
+ retVal = False # The less destructive default
+ if not intf or not pv_names: # We need at least the pv_names (list)
+ pass
+ else:
+ if vg_name is not None:
+ message_part = _("%s, that is an incomplete" % vg_name)
+ else:
+ message_part = _("an unknown")
+
+ rc = intf.messageWindow(_("Warning"),
+ _("Error processing drive(s) %s.
"
+ "It seems that there is inconsistent LVM data "
+ "(%s) make(s) up %s Volume Group. "
+ "You can reinitialize all related PVs, which will "
+ "erase all LVM metadata. Or ignore, which will "
+ "preserve contents.")
+ %(str(pv_names), str(pv_names), message_part),
+ type="custom",
+ custom_buttons = [ _("_Ignore drive(s)"),
+ _("_Re-initialize drive(s)") ],
+ custom_icon="question")
+ if rc == 0:
+ pass
+ else:
+ retVal = True # thie means clobber.
+
+ return retVal
+
class DeviceTree(object):
""" A quasi-tree that represents the devices in the system.

@@ -1296,6 +1325,76 @@ class DeviceTree(object):
log.info("setup of %s failed: %s"
% (lv_device.name, e))

+ def _handleSysCrapyness(self, device):
+ if device.type == "lvmvg":
+ # VG completeness (vgc)
+ (vgc, paths) = lvm.vgcomplete(device.name)
+ if vgc == lvm.VG_NOT_COMPLETE and
+ questionReinitializeIVG(intf=self.intf,
+ vg_name=device.name, pv_names=paths):
+ # We reinitialize all de parents.
+ # First we remove VG data
+ try:
+ lvm.vgreduce(device.name, [], rm=True)
+ lvm.vgremove(device.name) # now we can remove
+ except LVMError:
+ # the pvremoves will finish the job.
+ pass
+
+ for parent in device.parents:
+ try:
+ lvm.pvremove(parent.path)
+ except:
+ # does lvm.zerometadata make sence?
+ lvm.zeroLvmMetadata(device.path)
+
+ # Give the device the a default format
+ kwargs = {"uuid": parent.uuid,
+ "label": parent.diskLabel,
+ "device": parent.path,
+ "exists": parent.exists}
+ parent.format = formats.getFormat(*[""], **kwargs)
+
+ # to make sure that we have all the devices:
+ #device.remove(parent.path)
+
+ # Make sure all devices are handled
+ #for path in devices:
+ # pass # what to do here?
+
+ # remove VG device from list.
+ self._removeDevice(device)
+
+ return
+
+ elif device.format.type == "lvmpv" and
+ not lvm.pvstatus_ok(device.path):
+ if questionReinitializeIVG(intf = self.intf,
+ vg_name = device.format.vgName,
+ pv_names = device.path):
+ # For some reason there is a failure possibility here
+ # if the lvm commands dont work, just zero the device.
+ try:
+ if device.format.vgName is not None:
+ lvm.vgreduce(device.format.vgName, [], rm=True)
+ lvm.vgremove(device.format.vgName) # now we can remove
+ lvm.pvremove(device.path) # finally remove the PV
+ except LVMError:
+ # does lvm.zerometadata make sence?
+ lvm.zeroLvmMetadata(device.path)
+
+ # Give the device the a default format
+ kwargs = {"uuid": device.uuid,
+ "label": device.diskLabel,
+ "device": device.path,
+ "exists": device.exists}
+ device.format = formats.getFormat(*[""], **kwargs)
+
+ else:
+ self.addIgnoredDisk(device.name)
+
+ return
+
def populate(self):
""" Locate all storage devices. """
# each iteration scans any devices that have appeared since the
@@ -1325,6 +1424,11 @@ class DeviceTree(object):
for dev in devices:
self.addUdevDevice(dev)

+ # After having the complete tree we make sure that the system
+ # inconsistencies are ignored or resolved.
+ for leaf in self.leaves:
+ self._handleSysCrapyness(leaf)
+
self.teardownAll()

def teardownAll(self):
--
1.6.0.6

_______________________________________________
Anaconda-devel-list mailing list
Anaconda-devel-list@redhat.com
https://www.redhat.com/mailman/listinfo/anaconda-devel-list
 
Old 03-16-2009, 05:16 PM
Joel Granados
 
Default Handle system crappyness.

On Mon, Mar 16, 2009 at 07:13:43PM +0100, Joel Granados Moreno wrote:
> * storage/devicelibs/lvm.py (zeroLvmMetadata): zero out the LVM metadata
> only. This basically means from 513 byte and on. This will also
> ignore the firs 512 bytes if we are in a partition, but that does not
> matter because LVM does not touch those bits anyways.
> * storage/devicelibs/lvm.py (pv_complete): Complete here means that all
> the related PVs for the containing VG are present in the system.
> Returns the Completeness value and a list of known devices.
> * storage/devicelibs/lvm.py (vg_complete): likewise but for VG.
> * storage/devicelibs/lvm.py (vgreduce): introduces a new argument to the
> function. rm means use the lvm --removemissing option.
>
> * storage/devicetree.py (_handleSysCrapyness): New function intended to
> catch all unwanted behavior from the system before continuing.
> * storage/devicetree.py (questionReinitializeIVG): New function intended
> to ask the user what to do in case we find inconsistent LVM metadata.
> ---
> storage/devicelibs/lvm.py | 127 +++++++++++++++++++++++++++++++++++++++++++--
> storage/devicetree.py | 104 ++++++++++++++++++++++++++++++++++++
> 2 files changed, 226 insertions(+), 5 deletions(-)
>
> diff --git a/storage/devicelibs/lvm.py b/storage/devicelibs/lvm.py
> index 0a19711..df21ea9 100644
> --- a/storage/devicelibs/lvm.py
> +++ b/storage/devicelibs/lvm.py
> @@ -99,6 +99,22 @@ def lvm_cc_addFilterRejectRegexp(regexp):
> _composeConfig()
> # End config_args handling code.
>
> +def zeroLvmMetadata(device):
> + """ This dds from byte 513 to 200Mb+513 of the disk.
> +
> + We assume that lvm has nothing to do with the first 512 bytes.
> + We assume that 200Mb is enough. My tests failed with 100Mb for
> + some unknown reason.
> + """
> + try:
> + fd = os.open(device, os.O_WRONLY)
> + os.lseek(fd, 513, os.SEEK_SET)
> + for i in range(200):
> + os.write(fd, ""*1024)
> + os.close(fd)
> + except:
> + raise LVMError("Falied to zero LVM data on %s." % device)
> +
> def getPossiblePhysicalExtents(floor=0):
> """Returns a list of integers representing the possible values for
> the physical extent of a volume group. Value is in KB.
> @@ -229,6 +245,100 @@ def pvinfo(device):
>
> return info
>
> +# Start of LVM consistency code
> +#
> +# PV_{,NOT}_COMPLETE: A PV is complete when it is part of a complete VG or
> +# it is part of no VG. It is incomplete otherwise.
> +# VG_{,NOT}_COMPLETE: A VG is complete when all its PVs are accounted for
> +# by LVM metadata. More specifically, the `lvm vgs`
> +# command has no PVs described as "unknown device"
> +
> +PV_COMPLETE = 0
> +PV_NOT_COMPLETE = 1
> +VG_COMPLETE = 0
> +VG_NOT_COMPLETE = 1
> +
> +def pv_complete(device):
> + """ Check completeness of lvm structure with Physical Volume device.
> +
> + Returns : (state, pv_list)
> + (None, None) means that device is not a PV.
> +
> + state: [PV_COMPLETE | PV_NOT_COMPLETE]
> + pv_list: List of PVs related to VG. If no related VG is found then return
> + [device]
> + """
> + # Make sure this is a PV.
> + args = ["pvs"] +
> + ["--noheadings"] +
> + ["-o", "pv_name"] +
> + [device]
> + rc = iutil.execWithCapture("lvm", args,
> + stderr = "/dev/tty5")
> + if rc == None:
> + # This is not a PV.
> + return (None, None)
> +
> + # Get the VG name related to the PV.
> + args = ["pvs"] +
> + ["--noheadings"] +
> + ["-o", "vg_name"] +
> + [device]
> + rc = iutil.execWithCapture("lvm", args,
> + stderr = "/dev/tty5")
> + if rc == None:
> + # Has no VG, but its ok
> + # FIXME: should look at stderr.
> + return (PV_COMPLETE, [device])
> + vg_name = rc.strip()
> +
> + # Volume Group Completeness (vgs)
> + (vgc, devices) = vgstatus_ok(vg_name)
> + if vgc == None:
> + return (None, None)
> + elif vgc == VG_NOT_COMPLETE:
> + return (PV_NOT_COMPLETE, devices)
> + elif vgc == VG_COMPLETE:
> + return (PV_COMPLETE, devices)
> + else:
> + raise LVMError("Error checking completeness of %s" % vg_name)
> +
> +def vgcomplete(vg_name):
> + """ Check validity of lvm structure with Physical Volume device.
> +
> + Returns : (state, pv_list)
> + (None, None) means that device is not a VG.
> +
> + state: [VG_COMPLETE | VG_NOT_COMPLETE]
> + pv_list: List of PVs related to VG.
> + """
> + # Ask for the names of the PVs.
> + args = ["vgs"] +
> + ["--noheadings"] +
> + ["-o", "pv_name"] +
> + [vg_name]
> + rc = iutil.execWithCapture("lvm", args,
> + stderr = "/dev/tty5")
> +
> + if rc == None:
> + # This is not a VG.
> + return (None, None)
> +
> + # we make sure that every element is striped. Also, we don't want to
> + # return "unknown device"
> + pv_names = rc.strip("
").split("
")
> + ret_pv_names = []
> + completeness = VG_COMPLETE
> + for i in range(len(pv_names)):
> + if pv_names[i].strip() != "unknown device":
> + ret_pv_names.append(pv_names[i].strip())
> + else:
> + completeness = VG_NOT_COMPLETE
> +
> + return (completeness, pv_names)
> +#
> +# End of LVM consistency code
> +
> def vgcreate(vg_name, pv_list, pe_size):
> argv = ["vgcreate"]
> if pe_size:
> @@ -283,11 +393,18 @@ def vgdeactivate(vg_name):
> if rc:
> raise LVMError("vgdeactivate failed for %s" % vg_name)
>
> -def vgreduce(vg_name, pv_list):
> - args = ["vgreduce"] +
> - config_args +
> - [vg_name] +
> - pv_list
> +def vgreduce(vg_name, pv_list, rm=False):
> + """ Reduce a VG.
> +
> + rm -> with RemoveMissing option.
> + Use pv_list when rm=False, otherwise ignore pv_list and call vgreduce with
> + the --removemissing option.
> + """
> + args = ["vgreduce"]
> + if rm:
> + args.extend(["--removemissing", vg_name])
> + else:
> + args.extend([vg_name] + pv_list)
>
> rc = iutil.execWithRedirect("lvm", args,
> stdout = "/dev/tty5",
> diff --git a/storage/devicetree.py b/storage/devicetree.py
> index 66d053e..1b4fcbd 100644
> --- a/storage/devicetree.py
> +++ b/storage/devicetree.py
> @@ -136,6 +136,35 @@ def questionInitializeDisk(intf=None, name=None):
> retVal = True
> return retVal
>
> +def questionReinitializeIVG(intf=None, vg_name=None, pv_names=None):
> + retVal = False # The less destructive default
> + if not intf or not pv_names: # We need at least the pv_names (list)
> + pass
> + else:
> + if vg_name is not None:
> + message_part = _("%s, that is an incomplete" % vg_name)
> + else:
> + message_part = _("an unknown")
> +
> + rc = intf.messageWindow(_("Warning"),
> + _("Error processing drive(s) %s.
"
> + "It seems that there is inconsistent LVM data "
> + "(%s) make(s) up %s Volume Group. "
> + "You can reinitialize all related PVs, which will "
> + "erase all LVM metadata. Or ignore, which will "
> + "preserve contents.")
> + %(str(pv_names), str(pv_names), message_part),
> + type="custom",
> + custom_buttons = [ _("_Ignore drive(s)"),
> + _("_Re-initialize drive(s)") ],
> + custom_icon="question")
> + if rc == 0:
> + pass
> + else:
> + retVal = True # thie means clobber.
> +
> + return retVal
> +
> class DeviceTree(object):
> """ A quasi-tree that represents the devices in the system.
>
> @@ -1296,6 +1325,76 @@ class DeviceTree(object):
> log.info("setup of %s failed: %s"
> % (lv_device.name, e))
>
> + def _handleSysCrapyness(self, device):
> + if device.type == "lvmvg":
> + # VG completeness (vgc)
> + (vgc, paths) = lvm.vgcomplete(device.name)
> + if vgc == lvm.VG_NOT_COMPLETE and
> + questionReinitializeIVG(intf=self.intf,
> + vg_name=device.name, pv_names=paths):
> + # We reinitialize all de parents.
> + # First we remove VG data
> + try:
> + lvm.vgreduce(device.name, [], rm=True)
> + lvm.vgremove(device.name) # now we can remove
> + except LVMError:
> + # the pvremoves will finish the job.
> + pass
> +
> + for parent in device.parents:
> + try:
> + lvm.pvremove(parent.path)
> + except:
> + # does lvm.zerometadata make sence?
> + lvm.zeroLvmMetadata(device.path)
> +
> + # Give the device the a default format
> + kwargs = {"uuid": parent.uuid,
> + "label": parent.diskLabel,
> + "device": parent.path,
> + "exists": parent.exists}
> + parent.format = formats.getFormat(*[""], **kwargs)
> +
> + # to make sure that we have all the devices:
> + #device.remove(parent.path)
> +
> + # Make sure all devices are handled
> + #for path in devices:
> + # pass # what to do here?
> +
> + # remove VG device from list.
> + self._removeDevice(device)

Im missing the part where I ignore the devices here. I have it, but I
had not commited it .

> +
> + return
> +
> + elif device.format.type == "lvmpv" and
> + not lvm.pvstatus_ok(device.path):
> + if questionReinitializeIVG(intf = self.intf,
> + vg_name = device.format.vgName,
> + pv_names = device.path):
> + # For some reason there is a failure possibility here
> + # if the lvm commands dont work, just zero the device.
> + try:
> + if device.format.vgName is not None:
> + lvm.vgreduce(device.format.vgName, [], rm=True)
> + lvm.vgremove(device.format.vgName) # now we can remove
> + lvm.pvremove(device.path) # finally remove the PV
> + except LVMError:
> + # does lvm.zerometadata make sence?
> + lvm.zeroLvmMetadata(device.path)
> +
> + # Give the device the a default format
> + kwargs = {"uuid": device.uuid,
> + "label": device.diskLabel,
> + "device": device.path,
> + "exists": device.exists}
> + device.format = formats.getFormat(*[""], **kwargs)
> +
> + else:
> + self.addIgnoredDisk(device.name)
> +
> + return
> +
> def populate(self):
> """ Locate all storage devices. """
> # each iteration scans any devices that have appeared since the
> @@ -1325,6 +1424,11 @@ class DeviceTree(object):
> for dev in devices:
> self.addUdevDevice(dev)
>
> + # After having the complete tree we make sure that the system
> + # inconsistencies are ignored or resolved.
> + for leaf in self.leaves:
> + self._handleSysCrapyness(leaf)
> +
> self.teardownAll()
>
> def teardownAll(self):
> --
> 1.6.0.6
>
> _______________________________________________
> Anaconda-devel-list mailing list
> Anaconda-devel-list@redhat.com
> https://www.redhat.com/mailman/listinfo/anaconda-devel-list

--
Joel Andres Granados
Brno, Czech Republic, Red Hat.

_______________________________________________
Anaconda-devel-list mailing list
Anaconda-devel-list@redhat.com
https://www.redhat.com/mailman/listinfo/anaconda-devel-list
 
Old 03-16-2009, 07:35 PM
Joel Granados Moreno
 
Default Handle system crappyness.

* storage/devicelibs/lvm.py (zeroLvmMetadata): zero out the LVM metadata
only. This basically means from 513 byte and on. This will also
ignore the firs 512 bytes if we are in a partition, but that does not
matter because LVM does not touch those bits anyways.
* storage/devicelibs/lvm.py (vgreduce): introduces a new argument to the
function. rm means use the lvm --removemissing option.

* storage/devices.py (LVMVolumeGroupDevice.complete): New function to
evaluate if VG is consistent.
* storage/devices.py (LVMLogicalVolumeDevice.complete): Likewise for
LVs.

* storage/devicetree.py (_handleSysCrapyness): New function intended to
catch all unwanted behavior from the system before continuing.
* storage/devicetree.py (questionReinitILVM): New function intended
to ask the user what to do in case we find inconsistent LVM metadata.
---
storage/devicelibs/lvm.py | 33 ++++++++++++--
storage/devices.py | 29 +++++++++++-
storage/devicetree.py | 108 +++++++++++++++++++++++++++++++++++++++++++++
3 files changed, 162 insertions(+), 8 deletions(-)

diff --git a/storage/devicelibs/lvm.py b/storage/devicelibs/lvm.py
index 8b755a4..8a0e25b 100644
--- a/storage/devicelibs/lvm.py
+++ b/storage/devicelibs/lvm.py
@@ -99,6 +99,22 @@ def lvm_cc_addFilterRejectRegexp(regexp):
_composeConfig()
# End config_args handling code.

+def zeroLvmMetadata(device):
+ """ This dds from byte 513 to 200Mb+513 of the disk.
+
+ We assume that lvm has nothing to do with the first 512 bytes.
+ We assume that 200Mb is enough. My tests failed with 100Mb for
+ some unknown reason.
+ """
+ try:
+ fd = os.open(device, os.O_WRONLY)
+ os.lseek(fd, 513, os.SEEK_SET)
+ for i in range(200):
+ os.write(fd, ""*1024)
+ os.close(fd)
+ except:
+ raise LVMError("Falied to zero LVM data on %s." % device)
+
def getPossiblePhysicalExtents(floor=0):
"""Returns a list of integers representing the possible values for
the physical extent of a volume group. Value is in KB.
@@ -283,11 +299,18 @@ def vgdeactivate(vg_name):
if rc:
raise LVMError("vgdeactivate failed for %s" % vg_name)

-def vgreduce(vg_name, pv_list):
- args = ["vgreduce"] +
- config_args +
- [vg_name] +
- pv_list
+def vgreduce(vg_name, pv_list, rm=False):
+ """ Reduce a VG.
+
+ rm -> with RemoveMissing option.
+ Use pv_list when rm=False, otherwise ignore pv_list and call vgreduce with
+ the --removemissing option.
+ """
+ args = ["vgreduce"]
+ if rm:
+ args.extend(["--removemissing", vg_name])
+ else:
+ args.extend([vg_name] + pv_list)

rc = iutil.execWithRedirect("lvm", args,
stdout = "/dev/tty5",
diff --git a/storage/devices.py b/storage/devices.py
index 397a648..a5761a0 100644
--- a/storage/devices.py
+++ b/storage/devices.py
@@ -1692,9 +1692,15 @@ class LVMVolumeGroupDevice(DMDevice):

self.teardown()

- lvm.vgremove(self.name)
- self.notifyKernel()
- self.exists = False
+ # this sometimes fails for some reason.
+ try:
+ lvm.vgreduce(self.name, [], rm=True)
+ lvm.vgremove(self.name)
+ except lvm.LVMErorr:
+ raise DeviceError("Could not completely remove VG %s" % self.name)
+ finally:
+ self.notifyKernel()
+ self.exists = False

def reduce(self, pv_list):
""" Remove the listed PVs from the VG. """
@@ -1823,6 +1829,15 @@ class LVMVolumeGroupDevice(DMDevice):
""" A list of this VG's LVs """
return self._lvs[:] # we don't want folks changing our list

+ def complete(self):
+ """Check if the vg has all its pvs in the system
+ Return True if complete.
+ """
+ if len(self.pvs) == self.pvCount:
+ return True
+ else:
+ return False
+

class LVMLogicalVolumeDevice(DMDevice):
""" An LVM Logical Volume """
@@ -1992,6 +2007,14 @@ class LVMLogicalVolumeDevice(DMDevice):
self.format.teardown()
lvm.lvresize(self.vg.name, self._name, self.size)

+ def complete(self):
+ """ Test if vg exits and if it has all pvs. """
+ if not self.vg.exists:
+ return False
+
+ if not self.vg.complete():
+ return False
+

class MDRaidArrayDevice(StorageDevice):
""" An mdraid (Linux RAID) device.
diff --git a/storage/devicetree.py b/storage/devicetree.py
index fdcac7c..342140b 100644
--- a/storage/devicetree.py
+++ b/storage/devicetree.py
@@ -136,6 +136,36 @@ def questionInitializeDisk(intf=None, name=None):
retVal = True
return retVal

+def questionReinitILVM(intf=None, pv_names=None, lv_name=None, vg_name=None):
+ retVal = False # The less destructive default
+ if not intf or not pv_names or (lv_name is None and vg_name is None):
+ pass
+ else:
+ if vg_name is not None:
+ message = "%s Volume Group" % vg_name
+ elif lv_name is not None:
+ message = "%s Logical Volume" % lv_name
+
+
+ rc = intf.messageWindow(_("Warning"),
+ _("Error processing LVM.
"
+ "It seems that there is inconsistent LVM data. "
+ "(%s) make(s) up %s. "
+ "You can reinitialize all related PVs, which will "
+ "erase all LVM metadata. Or ignore, which will "
+ "preserve contents.")
+ %(str(pv_names), message),
+ type="custom",
+ custom_buttons = [ _("_Ignore drive(s)"),
+ _("_Re-initialize drive(s)") ],
+ custom_icon="question")
+ if rc == 0:
+ pass
+ else:
+ retVal = True # thie means clobber.
+
+ return retVal
+
class DeviceTree(object):
""" A quasi-tree that represents the devices in the system.

@@ -1305,12 +1335,85 @@ class DeviceTree(object):
size=lv_size,
exists=True)
self._addDevice(lv_device)
+
try:
lv_device.setup()
except DeviceError as e:
log.info("setup of %s failed: %s"
% (lv_device.name, e))

+ def _handleSysCrapyness(self, device):
+ def reinitializeVG(vg):
+ # First we remove VG data
+ try:
+ vg.destroy()
+ except DeviceError:
+ # the pvremoves will finish the job.
+ log.debug("There was an error destroying the VG %s." % vg.name)
+ pass
+
+ for parent in vg.parents:
+ try:
+ parent.destroy()
+ except:
+ # does lvm.zerometadata make sence?
+ lvm.zeroLvmMetadata(parent.path)
+
+ # Give the vg the a default format
+ kwargs = {"uuid": parent.uuid,
+ "label": parent.diskLabel,
+ "device": parent.path,
+ "exists": parent.exists}
+ parent.format = formats.getFormat(*[""], **kwargs)
+
+ # remove VG device from list.
+ self._removeDevice(vg)
+
+
+ if device.type == "lvmvg":
+ paths = []
+ for parent in device.parents:
+ paths.append(parent.path)
+
+ if not device.complete() and
+ questionReinitILVM(intf=self.intf,
+ vg_name=device.name, pv_names=paths):
+ reinitializeVG(device)
+
+ elif not device.complete():
+ # The user chose not to reinitialize.
+ # hopefully this will ignore the vg components too.
+ self.addIgnoredDisk(device.name)
+ return
+
+ elif device.type == "lvmlv":
+ # we might have already fixed this.
+ if device not in self._devices:
+ return
+
+ paths = []
+ for parent in device.vg.parents:
+ paths.append(parent.path)
+
+ if not device.complete() and
+ questionReinitILVM(intf=self.intf,
+ lv_name=device.name, pv_names=paths):
+
+ # destroy all lvs.
+ for lv in device.vg.lvs:
+ lv.destroy()
+ device.vg._removeLogVol(lv)
+ self._removeDevice(lv)
+
+ reinitializeVG(device.vg)
+
+ elif not device.complete():
+ # The user chose not to reinitialize.
+ # hopefully this will ignore the vg components too.
+ for lv in device.vg.lvs:
+ self.addIgnoredDisk(device.name)
+ return
+
def populate(self):
""" Locate all storage devices. """
# each iteration scans any devices that have appeared since the
@@ -1340,6 +1443,11 @@ class DeviceTree(object):
for dev in devices:
self.addUdevDevice(dev)

+ # After having the complete tree we make sure that the system
+ # inconsistencies are ignored or resolved.
+ for leaf in self.leaves:
+ self._handleSysCrapyness(leaf)
+
self.teardownAll()

def teardownAll(self):
--
1.6.0.6

_______________________________________________
Anaconda-devel-list mailing list
Anaconda-devel-list@redhat.com
https://www.redhat.com/mailman/listinfo/anaconda-devel-list
 
Old 03-16-2009, 07:57 PM
Jeremy Katz
 
Default Handle system crappyness.

On Monday, March 16 2009, Joel Granados Moreno said:
> +def zeroLvmMetadata(device):
> + """ This dds from byte 513 to 200Mb+513 of the disk.
> +
> + We assume that lvm has nothing to do with the first 512 bytes.
> + We assume that 200Mb is enough. My tests failed with 100Mb for
> + some unknown reason.
> + """
> + try:
> + fd = os.open(device, os.O_WRONLY)
> + os.lseek(fd, 513, os.SEEK_SET)
> + for i in range(200):
> + os.write(fd, ""*1024)
> + os.close(fd)
> + except:
> + raise LVMError("Falied to zero LVM data on %s." % device)

While maybe not a common case, this will traceback if the disk/partition
is < 200 megs in size.

Jeremy

_______________________________________________
Anaconda-devel-list mailing list
Anaconda-devel-list@redhat.com
https://www.redhat.com/mailman/listinfo/anaconda-devel-list
 
Old 03-16-2009, 08:06 PM
David Lehman
 
Default Handle system crappyness.

On Mon, 2009-03-16 at 21:35 +0100, Joel Granados Moreno wrote:
> * storage/devicelibs/lvm.py (zeroLvmMetadata): zero out the LVM metadata
> only. This basically means from 513 byte and on. This will also
> ignore the firs 512 bytes if we are in a partition, but that does not
> matter because LVM does not touch those bits anyways.

This should only be called from storage.formats.PhysicalVolume.destroy,
right?

> * storage/devicelibs/lvm.py (vgreduce): introduces a new argument to the
> function. rm means use the lvm --removemissing option.
>
> * storage/devices.py (LVMVolumeGroupDevice.complete): New function to
> evaluate if VG is consistent.
> * storage/devices.py (LVMLogicalVolumeDevice.complete): Likewise for
> LVs.

These should both be properties since they takes no arguments and, as
far as the user is concerned, don't actually perform any actions. The
parentheses just add noise and expose implementation details that nobody
needs to know.

>
> * storage/devicetree.py (_handleSysCrapyness): New function intended to
> catch all unwanted behavior from the system before continuing.

Why not call this "handleLVMInconsistency" or similar?

> * storage/devicetree.py (questionReinitILVM): New function intended
> to ask the user what to do in case we find inconsistent LVM metadata.

This seems okay, except it should take into account clearpart, right? We
don't want to prompt if they already gave 'clearpart --all'.

Dave

> ---
> storage/devicelibs/lvm.py | 33 ++++++++++++--
> storage/devices.py | 29 +++++++++++-
> storage/devicetree.py | 108 +++++++++++++++++++++++++++++++++++++++++++++
> 3 files changed, 162 insertions(+), 8 deletions(-)
>
> diff --git a/storage/devicelibs/lvm.py b/storage/devicelibs/lvm.py
> index 8b755a4..8a0e25b 100644
> --- a/storage/devicelibs/lvm.py
> +++ b/storage/devicelibs/lvm.py
> @@ -99,6 +99,22 @@ def lvm_cc_addFilterRejectRegexp(regexp):
> _composeConfig()
> # End config_args handling code.
>
> +def zeroLvmMetadata(device):
> + """ This dds from byte 513 to 200Mb+513 of the disk.
> +
> + We assume that lvm has nothing to do with the first 512 bytes.
> + We assume that 200Mb is enough. My tests failed with 100Mb for
> + some unknown reason.
> + """
> + try:
> + fd = os.open(device, os.O_WRONLY)
> + os.lseek(fd, 513, os.SEEK_SET)
> + for i in range(200):
> + os.write(fd, ""*1024)
> + os.close(fd)
> + except:
> + raise LVMError("Falied to zero LVM data on %s." % device)
> +
> def getPossiblePhysicalExtents(floor=0):
> """Returns a list of integers representing the possible values for
> the physical extent of a volume group. Value is in KB.
> @@ -283,11 +299,18 @@ def vgdeactivate(vg_name):
> if rc:
> raise LVMError("vgdeactivate failed for %s" % vg_name)
>
> -def vgreduce(vg_name, pv_list):
> - args = ["vgreduce"] +
> - config_args +
> - [vg_name] +
> - pv_list
> +def vgreduce(vg_name, pv_list, rm=False):
> + """ Reduce a VG.
> +
> + rm -> with RemoveMissing option.
> + Use pv_list when rm=False, otherwise ignore pv_list and call vgreduce with
> + the --removemissing option.
> + """
> + args = ["vgreduce"]
> + if rm:
> + args.extend(["--removemissing", vg_name])
> + else:
> + args.extend([vg_name] + pv_list)
>
> rc = iutil.execWithRedirect("lvm", args,
> stdout = "/dev/tty5",
> diff --git a/storage/devices.py b/storage/devices.py
> index 397a648..a5761a0 100644
> --- a/storage/devices.py
> +++ b/storage/devices.py
> @@ -1692,9 +1692,15 @@ class LVMVolumeGroupDevice(DMDevice):
>
> self.teardown()
>
> - lvm.vgremove(self.name)
> - self.notifyKernel()
> - self.exists = False
> + # this sometimes fails for some reason.
> + try:
> + lvm.vgreduce(self.name, [], rm=True)
> + lvm.vgremove(self.name)
> + except lvm.LVMErorr:
> + raise DeviceError("Could not completely remove VG %s" % self.name)
> + finally:
> + self.notifyKernel()
> + self.exists = False
>
> def reduce(self, pv_list):
> """ Remove the listed PVs from the VG. """
> @@ -1823,6 +1829,15 @@ class LVMVolumeGroupDevice(DMDevice):
> """ A list of this VG's LVs """
> return self._lvs[:] # we don't want folks changing our list
>
> + def complete(self):
> + """Check if the vg has all its pvs in the system
> + Return True if complete.
> + """
> + if len(self.pvs) == self.pvCount:
> + return True
> + else:
> + return False
> +
>
> class LVMLogicalVolumeDevice(DMDevice):
> """ An LVM Logical Volume """
> @@ -1992,6 +2007,14 @@ class LVMLogicalVolumeDevice(DMDevice):
> self.format.teardown()
> lvm.lvresize(self.vg.name, self._name, self.size)
>
> + def complete(self):
> + """ Test if vg exits and if it has all pvs. """
> + if not self.vg.exists:
> + return False
> +
> + if not self.vg.complete():
> + return False
> +
>
> class MDRaidArrayDevice(StorageDevice):
> """ An mdraid (Linux RAID) device.
> diff --git a/storage/devicetree.py b/storage/devicetree.py
> index fdcac7c..342140b 100644
> --- a/storage/devicetree.py
> +++ b/storage/devicetree.py
> @@ -136,6 +136,36 @@ def questionInitializeDisk(intf=None, name=None):
> retVal = True
> return retVal
>
> +def questionReinitILVM(intf=None, pv_names=None, lv_name=None, vg_name=None):
> + retVal = False # The less destructive default
> + if not intf or not pv_names or (lv_name is None and vg_name is None):
> + pass
> + else:
> + if vg_name is not None:
> + message = "%s Volume Group" % vg_name
> + elif lv_name is not None:
> + message = "%s Logical Volume" % lv_name
> +
> +
> + rc = intf.messageWindow(_("Warning"),
> + _("Error processing LVM.
"
> + "It seems that there is inconsistent LVM data. "
> + "(%s) make(s) up %s. "
> + "You can reinitialize all related PVs, which will "
> + "erase all LVM metadata. Or ignore, which will "
> + "preserve contents.")
> + %(str(pv_names), message),
> + type="custom",
> + custom_buttons = [ _("_Ignore drive(s)"),
> + _("_Re-initialize drive(s)") ],
> + custom_icon="question")
> + if rc == 0:
> + pass
> + else:
> + retVal = True # thie means clobber.
> +
> + return retVal
> +
> class DeviceTree(object):
> """ A quasi-tree that represents the devices in the system.
>
> @@ -1305,12 +1335,85 @@ class DeviceTree(object):
> size=lv_size,
> exists=True)
> self._addDevice(lv_device)
> +
> try:
> lv_device.setup()
> except DeviceError as e:
> log.info("setup of %s failed: %s"
> % (lv_device.name, e))
>
> + def _handleSysCrapyness(self, device):
> + def reinitializeVG(vg):
> + # First we remove VG data
> + try:
> + vg.destroy()
> + except DeviceError:
> + # the pvremoves will finish the job.
> + log.debug("There was an error destroying the VG %s." % vg.name)
> + pass
> +
> + for parent in vg.parents:
> + try:
> + parent.destroy()
> + except:
> + # does lvm.zerometadata make sence?
> + lvm.zeroLvmMetadata(parent.path)
> +
> + # Give the vg the a default format
> + kwargs = {"uuid": parent.uuid,
> + "label": parent.diskLabel,
> + "device": parent.path,
> + "exists": parent.exists}
> + parent.format = formats.getFormat(*[""], **kwargs)
> +
> + # remove VG device from list.
> + self._removeDevice(vg)
> +
> +
> + if device.type == "lvmvg":
> + paths = []
> + for parent in device.parents:
> + paths.append(parent.path)
> +
> + if not device.complete() and
> + questionReinitILVM(intf=self.intf,
> + vg_name=device.name, pv_names=paths):
> + reinitializeVG(device)
> +
> + elif not device.complete():
> + # The user chose not to reinitialize.
> + # hopefully this will ignore the vg components too.
> + self.addIgnoredDisk(device.name)
> + return
> +
> + elif device.type == "lvmlv":
> + # we might have already fixed this.
> + if device not in self._devices:
> + return
> +
> + paths = []
> + for parent in device.vg.parents:
> + paths.append(parent.path)
> +
> + if not device.complete() and
> + questionReinitILVM(intf=self.intf,
> + lv_name=device.name, pv_names=paths):
> +
> + # destroy all lvs.
> + for lv in device.vg.lvs:
> + lv.destroy()
> + device.vg._removeLogVol(lv)
> + self._removeDevice(lv)
> +
> + reinitializeVG(device.vg)
> +
> + elif not device.complete():
> + # The user chose not to reinitialize.
> + # hopefully this will ignore the vg components too.
> + for lv in device.vg.lvs:
> + self.addIgnoredDisk(device.name)
> + return
> +
> def populate(self):
> """ Locate all storage devices. """
> # each iteration scans any devices that have appeared since the
> @@ -1340,6 +1443,11 @@ class DeviceTree(object):
> for dev in devices:
> self.addUdevDevice(dev)
>
> + # After having the complete tree we make sure that the system
> + # inconsistencies are ignored or resolved.
> + for leaf in self.leaves:
> + self._handleSysCrapyness(leaf)
> +
> self.teardownAll()
>
> def teardownAll(self):

_______________________________________________
Anaconda-devel-list mailing list
Anaconda-devel-list@redhat.com
https://www.redhat.com/mailman/listinfo/anaconda-devel-list
 
Old 03-17-2009, 09:51 AM
Radek Vykydal
 
Default Handle system crappyness.

Joel Granados wrote:

On Mon, Mar 16, 2009 at 07:13:43PM +0100, Joel Granados Moreno wrote:


* storage/devicelibs/lvm.py (zeroLvmMetadata): zero out the LVM metadata
only. This basically means from 513 byte and on. This will also
ignore the firs 512 bytes if we are in a partition, but that does not
matter because LVM does not touch those bits anyways.
* storage/devicelibs/lvm.py (pv_complete): Complete here means that all
the related PVs for the containing VG are present in the system.
Returns the Completeness value and a list of known devices.
* storage/devicelibs/lvm.py (vg_complete): likewise but for VG.
* storage/devicelibs/lvm.py (vgreduce): introduces a new argument to the
function. rm means use the lvm --removemissing option.

* storage/devicetree.py (_handleSysCrapyness): New function intended to
catch all unwanted behavior from the system before continuing.
* storage/devicetree.py (questionReinitializeIVG): New function intended
to ask the user what to do in case we find inconsistent LVM metadata.
---
storage/devicelibs/lvm.py | 127 +++++++++++++++++++++++++++++++++++++++++++--
storage/devicetree.py | 104 ++++++++++++++++++++++++++++++++++++
2 files changed, 226 insertions(+), 5 deletions(-)

diff --git a/storage/devicelibs/lvm.py b/storage/devicelibs/lvm.py
index 0a19711..df21ea9 100644
--- a/storage/devicelibs/lvm.py
+++ b/storage/devicelibs/lvm.py
@@ -99,6 +99,22 @@ def lvm_cc_addFilterRejectRegexp(regexp):
_composeConfig()
# End config_args handling code.

+def zeroLvmMetadata(device):
+ """ This dds from byte 513 to 200Mb+513 of the disk.
+
+ We assume that lvm has nothing to do with the first 512 bytes.
+ We assume that 200Mb is enough. My tests failed with 100Mb for
+ some unknown reason.
+ """



Aren't you cleaning 200KB instead of 200MB?



+ try:
+ fd = os.open(device, os.O_WRONLY)
+ os.lseek(fd, 513, os.SEEK_SET)
+ for i in range(200):
+ os.write(fd, ""*1024)
+ os.close(fd)
+ except:
+ raise LVMError("Falied to zero LVM data on %s." % device)
+
def getPossiblePhysicalExtents(floor=0):
"""Returns a list of integers representing the possible values for
the physical extent of a volume group. Value is in KB.
@@ -229,6 +245,100 @@ def pvinfo(device):

return info

+# Start of LVM consistency code
+#
+# PV_{,NOT}_COMPLETE: A PV is complete when it is part of a complete VG or
+# it is part of no VG. It is incomplete otherwise.
+# VG_{,NOT}_COMPLETE: A VG is complete when all its PVs are accounted for
+# by LVM metadata. More specifically, the `lvm vgs`
+# command has no PVs described as "unknown device"
+
+PV_COMPLETE = 0
+PV_NOT_COMPLETE = 1
+VG_COMPLETE = 0
+VG_NOT_COMPLETE = 1
+
+def pv_complete(device):
+ """ Check completeness of lvm structure with Physical Volume device.
+
+ Returns : (state, pv_list)
+ (None, None) means that device is not a PV.
+
+ state: [PV_COMPLETE | PV_NOT_COMPLETE]
+ pv_list: List of PVs related to VG. If no related VG is found then return
+ [device]
+ """
+ # Make sure this is a PV.
+ args = ["pvs"] +
+ ["--noheadings"] +
+ ["-o", "pv_name"] +
+ [device]
+ rc = iutil.execWithCapture("lvm", args,
+ stderr = "/dev/tty5")
+ if rc == None:
+ # This is not a PV.
+ return (None, None)
+
+ # Get the VG name related to the PV.
+ args = ["pvs"] +
+ ["--noheadings"] +
+ ["-o", "vg_name"] +
+ [device]
+ rc = iutil.execWithCapture("lvm", args,
+ stderr = "/dev/tty5")
+ if rc == None:
+ # Has no VG, but its ok
+ # FIXME: should look at stderr.
+ return (PV_COMPLETE, [device])
+ vg_name = rc.strip()
+
+ # Volume Group Completeness (vgs)
+ (vgc, devices) = vgstatus_ok(vg_name)
+ if vgc == None:
+ return (None, None)
+ elif vgc == VG_NOT_COMPLETE:
+ return (PV_NOT_COMPLETE, devices)
+ elif vgc == VG_COMPLETE:
+ return (PV_COMPLETE, devices)
+ else:
+ raise LVMError("Error checking completeness of %s" % vg_name)
+
+def vgcomplete(vg_name):
+ """ Check validity of lvm structure with Physical Volume device.
+
+ Returns : (state, pv_list)
+ (None, None) means that device is not a VG.
+
+ state: [VG_COMPLETE | VG_NOT_COMPLETE]
+ pv_list: List of PVs related to VG.
+ """
+ # Ask for the names of the PVs.
+ args = ["vgs"] +
+ ["--noheadings"] +
+ ["-o", "pv_name"] +
+ [vg_name]
+ rc = iutil.execWithCapture("lvm", args,
+ stderr = "/dev/tty5")
+
+ if rc == None:
+ # This is not a VG.
+ return (None, None)
+
+ # we make sure that every element is striped. Also, we don't want to
+ # return "unknown device"
+ pv_names = rc.strip("
").split("
")
+ ret_pv_names = []
+ completeness = VG_COMPLETE
+ for i in range(len(pv_names)):
+ if pv_names[i].strip() != "unknown device":
+ ret_pv_names.append(pv_names[i].strip())
+ else:
+ completeness = VG_NOT_COMPLETE
+
+ return (completeness, pv_names)
+#
+# End of LVM consistency code
+
def vgcreate(vg_name, pv_list, pe_size):
argv = ["vgcreate"]
if pe_size:
@@ -283,11 +393,18 @@ def vgdeactivate(vg_name):
if rc:
raise LVMError("vgdeactivate failed for %s" % vg_name)

-def vgreduce(vg_name, pv_list):
- args = ["vgreduce"] +
- config_args +
- [vg_name] +
- pv_list
+def vgreduce(vg_name, pv_list, rm=False):
+ """ Reduce a VG.
+
+ rm -> with RemoveMissing option.
+ Use pv_list when rm=False, otherwise ignore pv_list and call vgreduce with
+ the --removemissing option.
+ """
+ args = ["vgreduce"]
+ if rm:
+ args.extend(["--removemissing", vg_name])
+ else:
+ args.extend([vg_name] + pv_list)

rc = iutil.execWithRedirect("lvm", args,
stdout = "/dev/tty5",
diff --git a/storage/devicetree.py b/storage/devicetree.py
index 66d053e..1b4fcbd 100644
--- a/storage/devicetree.py
+++ b/storage/devicetree.py
@@ -136,6 +136,35 @@ def questionInitializeDisk(intf=None, name=None):
retVal = True
return retVal

+def questionReinitializeIVG(intf=None, vg_name=None, pv_names=None):
+ retVal = False # The less destructive default
+ if not intf or not pv_names: # We need at least the pv_names (list)
+ pass
+ else:
+ if vg_name is not None:
+ message_part = _("%s, that is an incomplete" % vg_name)
+ else:
+ message_part = _("an unknown")
+
+ rc = intf.messageWindow(_("Warning"),
+ _("Error processing drive(s) %s.
"
+ "It seems that there is inconsistent LVM data "
+ "(%s) make(s) up %s Volume Group. "
+ "You can reinitialize all related PVs, which will "
+ "erase all LVM metadata. Or ignore, which will "
+ "preserve contents.")
+ %(str(pv_names), str(pv_names), message_part),
+ type="custom",
+ custom_buttons = [ _("_Ignore drive(s)"),
+ _("_Re-initialize drive(s)") ],
+ custom_icon="question")
+ if rc == 0:
+ pass
+ else:
+ retVal = True # thie means clobber.
+
+ return retVal
+
class DeviceTree(object):
""" A quasi-tree that represents the devices in the system.

@@ -1296,6 +1325,76 @@ class DeviceTree(object):
log.info("setup of %s failed: %s"
% (lv_device.name, e))

+ def _handleSysCrapyness(self, device):
+ if device.type == "lvmvg":
+ # VG completeness (vgc)
+ (vgc, paths) = lvm.vgcomplete(device.name)
+ if vgc == lvm.VG_NOT_COMPLETE and
+ questionReinitializeIVG(intf=self.intf,
+ vg_name=device.name, pv_names=paths):
+ # We reinitialize all de parents.
+ # First we remove VG data
+ try:
+ lvm.vgreduce(device.name, [], rm=True)
+ lvm.vgremove(device.name) # now we can remove
+ except LVMError:
+ # the pvremoves will finish the job.
+ pass
+
+ for parent in device.parents:
+ try:
+ lvm.pvremove(parent.path)
+ except:
+ # does lvm.zerometadata make sence?
+ lvm.zeroLvmMetadata(device.path)
+
+ # Give the device the a default format
+ kwargs = {"uuid": parent.uuid,
+ "label": parent.diskLabel,
+ "device": parent.path,
+ "exists": parent.exists}
+ parent.format = formats.getFormat(*[""], **kwargs)
+
+ # to make sure that we have all the devices:
+ #device.remove(parent.path)
+
+ # Make sure all devices are handled
+ #for path in devices:
+ # pass # what to do here?
+
+ # remove VG device from list.
+ self._removeDevice(device)



Im missing the part where I ignore the devices here. I have it, but I
had not commited it .



+
+ return
+
+ elif device.format.type == "lvmpv" and
+ not lvm.pvstatus_ok(device.path):
+ if questionReinitializeIVG(intf = self.intf,
+ vg_name = device.format.vgName,
+ pv_names = device.path):
+ # For some reason there is a failure possibility here
+ # if the lvm commands dont work, just zero the device.
+ try:
+ if device.format.vgName is not None:
+ lvm.vgreduce(device.format.vgName, [], rm=True)
+ lvm.vgremove(device.format.vgName) # now we can remove
+ lvm.pvremove(device.path) # finally remove the PV
+ except LVMError:
+ # does lvm.zerometadata make sence?
+ lvm.zeroLvmMetadata(device.path)
+
+ # Give the device the a default format
+ kwargs = {"uuid": device.uuid,
+ "label": device.diskLabel,
+ "device": device.path,
+ "exists": device.exists}
+ device.format = formats.getFormat(*[""], **kwargs)
+
+ else:
+ self.addIgnoredDisk(device.name)
+
+ return
+
def populate(self):
""" Locate all storage devices. """
# each iteration scans any devices that have appeared since the
@@ -1325,6 +1424,11 @@ class DeviceTree(object):
for dev in devices:
self.addUdevDevice(dev)

+ # After having the complete tree we make sure that the system
+ # inconsistencies are ignored or resolved.
+ for leaf in self.leaves:
+ self._handleSysCrapyness(leaf)
+
self.teardownAll()

def teardownAll(self):
--
1.6.0.6

_______________________________________________
Anaconda-devel-list mailing list
Anaconda-devel-list@redhat.com
https://www.redhat.com/mailman/listinfo/anaconda-devel-list









_______________________________________________
Anaconda-devel-list mailing list
Anaconda-devel-list@redhat.com
https://www.redhat.com/mailman/listinfo/anaconda-devel-list
 
Old 03-17-2009, 09:56 AM
Radek Vykydal
 
Default Handle system crappyness.

Joel Granados Moreno wrote:

* storage/devicelibs/lvm.py (zeroLvmMetadata): zero out the LVM metadata
only. This basically means from 513 byte and on. This will also
ignore the firs 512 bytes if we are in a partition, but that does not
matter because LVM does not touch those bits anyways.
* storage/devicelibs/lvm.py (vgreduce): introduces a new argument to the
function. rm means use the lvm --removemissing option.

* storage/devices.py (LVMVolumeGroupDevice.complete): New function to
evaluate if VG is consistent.
* storage/devices.py (LVMLogicalVolumeDevice.complete): Likewise for
LVs.

* storage/devicetree.py (_handleSysCrapyness): New function intended to
catch all unwanted behavior from the system before continuing.
* storage/devicetree.py (questionReinitILVM): New function intended
to ask the user what to do in case we find inconsistent LVM metadata.
---
storage/devicelibs/lvm.py | 33 ++++++++++++--
storage/devices.py | 29 +++++++++++-
storage/devicetree.py | 108 +++++++++++++++++++++++++++++++++++++++++++++
3 files changed, 162 insertions(+), 8 deletions(-)

diff --git a/storage/devicelibs/lvm.py b/storage/devicelibs/lvm.py
index 8b755a4..8a0e25b 100644
--- a/storage/devicelibs/lvm.py
+++ b/storage/devicelibs/lvm.py
@@ -99,6 +99,22 @@ def lvm_cc_addFilterRejectRegexp(regexp):
_composeConfig()
# End config_args handling code.

+def zeroLvmMetadata(device):

+ """ This dds from byte 513 to 200Mb+513 of the disk.
+
+ We assume that lvm has nothing to do with the first 512 bytes.
+ We assume that 200Mb is enough. My tests failed with 100Mb for
+ some unknown reason.

I noticed only now this take II of a patch I commented, so I copy my
comment

here too - seems that you are cleaning 200KB, not 200MB

+ """
+ try:
+ fd = os.open(device, os.O_WRONLY)
+ os.lseek(fd, 513, os.SEEK_SET)
+ for i in range(200):
+ os.write(fd, ""*1024)
+ os.close(fd)
+ except:
+ raise LVMError("Falied to zero LVM data on %s." % device)
+
def getPossiblePhysicalExtents(floor=0):
"""Returns a list of integers representing the possible values for
the physical extent of a volume group. Value is in KB.
@@ -283,11 +299,18 @@ def vgdeactivate(vg_name):
if rc:
raise LVMError("vgdeactivate failed for %s" % vg_name)

-def vgreduce(vg_name, pv_list):

- args = ["vgreduce"] +
- config_args +
- [vg_name] +
- pv_list
+def vgreduce(vg_name, pv_list, rm=False):
+ """ Reduce a VG.
+
+ rm -> with RemoveMissing option.
+ Use pv_list when rm=False, otherwise ignore pv_list and call vgreduce with
+ the --removemissing option.
+ """
+ args = ["vgreduce"]
+ if rm:
+ args.extend(["--removemissing", vg_name])
+ else:
+ args.extend([vg_name] + pv_list)

rc = iutil.execWithRedirect("lvm", args,

stdout = "/dev/tty5",
diff --git a/storage/devices.py b/storage/devices.py
index 397a648..a5761a0 100644
--- a/storage/devices.py
+++ b/storage/devices.py
@@ -1692,9 +1692,15 @@ class LVMVolumeGroupDevice(DMDevice):

self.teardown()

- lvm.vgremove(self.name)

- self.notifyKernel()
- self.exists = False
+ # this sometimes fails for some reason.
+ try:
+ lvm.vgreduce(self.name, [], rm=True)
+ lvm.vgremove(self.name)
+ except lvm.LVMErorr:
+ raise DeviceError("Could not completely remove VG %s" % self.name)
+ finally:
+ self.notifyKernel()
+ self.exists = False

def reduce(self, pv_list):

""" Remove the listed PVs from the VG. """
@@ -1823,6 +1829,15 @@ class LVMVolumeGroupDevice(DMDevice):
""" A list of this VG's LVs """
return self._lvs[:] # we don't want folks changing our list

+ def complete(self):

+ """Check if the vg has all its pvs in the system
+ Return True if complete.
+ """
+ if len(self.pvs) == self.pvCount:
+ return True
+ else:
+ return False
+

class LVMLogicalVolumeDevice(DMDevice):

""" An LVM Logical Volume """
@@ -1992,6 +2007,14 @@ class LVMLogicalVolumeDevice(DMDevice):
self.format.teardown()
lvm.lvresize(self.vg.name, self._name, self.size)

+ def complete(self):

+ """ Test if vg exits and if it has all pvs. """
+ if not self.vg.exists:
+ return False
+
+ if not self.vg.complete():
+ return False
+

class MDRaidArrayDevice(StorageDevice):

""" An mdraid (Linux RAID) device.
diff --git a/storage/devicetree.py b/storage/devicetree.py
index fdcac7c..342140b 100644
--- a/storage/devicetree.py
+++ b/storage/devicetree.py
@@ -136,6 +136,36 @@ def questionInitializeDisk(intf=None, name=None):
retVal = True
return retVal

+def questionReinitILVM(intf=None, pv_names=None, lv_name=None, vg_name=None):

+ retVal = False # The less destructive default
+ if not intf or not pv_names or (lv_name is None and vg_name is None):
+ pass
+ else:
+ if vg_name is not None:
+ message = "%s Volume Group" % vg_name
+ elif lv_name is not None:
+ message = "%s Logical Volume" % lv_name
+
+
+ rc = intf.messageWindow(_("Warning"),
+ _("Error processing LVM.
"
+ "It seems that there is inconsistent LVM data. "
+ "(%s) make(s) up %s. "
+ "You can reinitialize all related PVs, which will "
+ "erase all LVM metadata. Or ignore, which will "
+ "preserve contents.")
+ %(str(pv_names), message),
+ type="custom",
+ custom_buttons = [ _("_Ignore drive(s)"),
+ _("_Re-initialize drive(s)") ],
+ custom_icon="question")
+ if rc == 0:
+ pass
+ else:
+ retVal = True # thie means clobber.
+
+ return retVal
+
class DeviceTree(object):
""" A quasi-tree that represents the devices in the system.

@@ -1305,12 +1335,85 @@ class DeviceTree(object):

size=lv_size,
exists=True)
self._addDevice(lv_device)
+
try:
lv_device.setup()
except DeviceError as e:
log.info("setup of %s failed: %s"
% (lv_device.name, e))

+ def _handleSysCrapyness(self, device):

+ def reinitializeVG(vg):
+ # First we remove VG data
+ try:
+ vg.destroy()
+ except DeviceError:
+ # the pvremoves will finish the job.
+ log.debug("There was an error destroying the VG %s." % vg.name)
+ pass
+
+ for parent in vg.parents:
+ try:
+ parent.destroy()
+ except:
+ # does lvm.zerometadata make sence?
+ lvm.zeroLvmMetadata(parent.path)
+
+ # Give the vg the a default format
+ kwargs = {"uuid": parent.uuid,
+ "label": parent.diskLabel,
+ "device": parent.path,
+ "exists": parent.exists}
+ parent.format = formats.getFormat(*[""], **kwargs)
+
+ # remove VG device from list.
+ self._removeDevice(vg)
+
+
+ if device.type == "lvmvg":
+ paths = []
+ for parent in device.parents:
+ paths.append(parent.path)
+
+ if not device.complete() and
+ questionReinitILVM(intf=self.intf,
+ vg_name=device.name, pv_names=paths):
+ reinitializeVG(device)
+
+ elif not device.complete():
+ # The user chose not to reinitialize.
+ # hopefully this will ignore the vg components too.
+ self.addIgnoredDisk(device.name)
+ return
+
+ elif device.type == "lvmlv":
+ # we might have already fixed this.
+ if device not in self._devices:
+ return
+
+ paths = []
+ for parent in device.vg.parents:
+ paths.append(parent.path)
+
+ if not device.complete() and
+ questionReinitILVM(intf=self.intf,
+ lv_name=device.name, pv_names=paths):
+
+ # destroy all lvs.
+ for lv in device.vg.lvs:
+ lv.destroy()
+ device.vg._removeLogVol(lv)
+ self._removeDevice(lv)
+
+ reinitializeVG(device.vg)
+
+ elif not device.complete():
+ # The user chose not to reinitialize.
+ # hopefully this will ignore the vg components too.
+ for lv in device.vg.lvs:
+ self.addIgnoredDisk(device.name)
+ return
+
def populate(self):
""" Locate all storage devices. """
# each iteration scans any devices that have appeared since the
@@ -1340,6 +1443,11 @@ class DeviceTree(object):
for dev in devices:
self.addUdevDevice(dev)

+ # After having the complete tree we make sure that the system

+ # inconsistencies are ignored or resolved.
+ for leaf in self.leaves:
+ self._handleSysCrapyness(leaf)
+
self.teardownAll()

def teardownAll(self):



_______________________________________________
Anaconda-devel-list mailing list
Anaconda-devel-list@redhat.com
https://www.redhat.com/mailman/listinfo/anaconda-devel-list
 
Old 03-17-2009, 12:25 PM
Joel Granados
 
Default Handle system crappyness.

On Tue, Mar 17, 2009 at 11:51:11AM +0100, Radek Vykydal wrote:
> Joel Granados wrote:
>
> On Mon, Mar 16, 2009 at 07:13:43PM +0100, Joel Granados Moreno wrote:
>
> + """
>
>
> Aren't you cleaning 200KB instead of 200MB?

Yep, thx, will change. I actually want to clean 200MB. Wonder if thats
overkill ?

>

--
Joel Andres Granados
Brno, Czech Republic, Red Hat.

_______________________________________________
Anaconda-devel-list mailing list
Anaconda-devel-list@redhat.com
https://www.redhat.com/mailman/listinfo/anaconda-devel-list
 
Old 03-17-2009, 12:25 PM
Joel Granados
 
Default Handle system crappyness.

On Mon, Mar 16, 2009 at 04:57:07PM -0400, Jeremy Katz wrote:
> On Monday, March 16 2009, Joel Granados Moreno said:
> > +def zeroLvmMetadata(device):
> > + """ This dds from byte 513 to 200Mb+513 of the disk.
> > +
> > + We assume that lvm has nothing to do with the first 512 bytes.
> > + We assume that 200Mb is enough. My tests failed with 100Mb for
> > + some unknown reason.
> > + """
> > + try:
> > + fd = os.open(device, os.O_WRONLY)
> > + os.lseek(fd, 513, os.SEEK_SET)
> > + for i in range(200):
> > + os.write(fd, ""*1024)
> > + os.close(fd)
> > + except:
> > + raise LVMError("Falied to zero LVM data on %s." % device)
>
> While maybe not a common case, this will traceback if the disk/partition
> is < 200 megs in size.

Yep, you are right. I will handle the specific errno == 28 for this
case.
>
> Jeremy
>
> _______________________________________________
> Anaconda-devel-list mailing list
> Anaconda-devel-list@redhat.com
> https://www.redhat.com/mailman/listinfo/anaconda-devel-list

--
Joel Andres Granados
Brno, Czech Republic, Red Hat.

_______________________________________________
Anaconda-devel-list mailing list
Anaconda-devel-list@redhat.com
https://www.redhat.com/mailman/listinfo/anaconda-devel-list
 
Old 03-17-2009, 12:30 PM
Joel Granados
 
Default Handle system crappyness.

On Mon, Mar 16, 2009 at 04:06:09PM -0500, David Lehman wrote:
> On Mon, 2009-03-16 at 21:35 +0100, Joel Granados Moreno wrote:
> > * storage/devicelibs/lvm.py (zeroLvmMetadata): zero out the LVM metadata
> > only. This basically means from 513 byte and on. This will also
> > ignore the firs 512 bytes if we are in a partition, but that does not
> > matter because LVM does not touch those bits anyways.
>
> This should only be called from storage.formats.PhysicalVolume.destroy,
> right?

Yep, seems right.
>
> > * storage/devicelibs/lvm.py (vgreduce): introduces a new argument to the
> > function. rm means use the lvm --removemissing option.
> >
> > * storage/devices.py (LVMVolumeGroupDevice.complete): New function to
> > evaluate if VG is consistent.
> > * storage/devices.py (LVMLogicalVolumeDevice.complete): Likewise for
> > LVs.
>
> These should both be properties since they takes no arguments and, as
> far as the user is concerned, don't actually perform any actions. The
> parentheses just add noise and expose implementation details that nobody
> needs to know.

sure.

>
> >
> > * storage/devicetree.py (_handleSysCrapyness): New function intended to
> > catch all unwanted behavior from the system before continuing.
>
> Why not call this "handleLVMInconsistency" or similar?

yep, Although I had thought this might be used for other
inconsistencies. I will rename it to handleinconsistencies so we can
put other stuff in there if needed.
>
> > * storage/devicetree.py (questionReinitILVM): New function intended
> > to ask the user what to do in case we find inconsistent LVM metadata.
>
> This seems okay, except it should take into account clearpart, right? We
> don't want to prompt if they already gave 'clearpart --all'.

ok, will modify. Note that there are still corner cases that are not
handled with this patch. Mainly the PV that has not partition table. I
chose to ignore this corner case as the current storage code assumes a
partition table.
What will happen if we encounter this situation is that we will
partition a PV that might be in use. The result is a device that has
LVM metadata and partitioning metadata. For now there is no way of
avoiding this. (unless the user chooses to ignore the device when
prompted)

>
> Dave
>
> > ---
> > storage/devicelibs/lvm.py | 33 ++++++++++++--
> > storage/devices.py | 29 +++++++++++-
> > storage/devicetree.py | 108 +++++++++++++++++++++++++++++++++++++++++++++
> > 3 files changed, 162 insertions(+), 8 deletions(-)
> >
> > diff --git a/storage/devicelibs/lvm.py b/storage/devicelibs/lvm.py
> > index 8b755a4..8a0e25b 100644
> > --- a/storage/devicelibs/lvm.py
> > +++ b/storage/devicelibs/lvm.p> > +
> > + self.addIgnoredDisk(device.name)

--
Joel Andres Granados
Brno, Czech Republic, Red Hat.

_______________________________________________
Anaconda-devel-list mailing list
Anaconda-devel-list@redhat.com
https://www.redhat.com/mailman/listinfo/anaconda-devel-list
 

Thread Tools




All times are GMT. The time now is 12:20 AM.

VBulletin, Copyright ©2000 - 2014, Jelsoft Enterprises Ltd.
Content Relevant URLs by vBSEO ©2007, Crawlability, Inc.
Copyright 2007 - 2008, www.linux-archive.org