FAQ Search Today's Posts Mark Forums Read
» Video Reviews

» Linux Archive

Linux-archive is a website aiming to archive linux email lists and to make them easily accessible for linux users/developers.


» Sponsor

» Partners

» Sponsor

Go Back   Linux Archive > Redhat > Device-mapper Development

 
 
LinkBack Thread Tools
 
Old 11-03-2011, 02:54 PM
Tejun Heo
 
Default HPA unlock during partition scan of RAID components

Hello,

(cc'ing dm-devel)

On Thu, Nov 03, 2011 at 01:02:37AM +0000, Hawrylewicz Czarnowski, Przemyslaw wrote:
> Recently we have encountered a problem with unlocking of HPA on disks belonging to RAID array.
> The scenario in the simplest form is as follows:
> * take HPA resized drives
> * use SW Raid solution with metadata located at the end of disk (eg. md v1.0, IMSM)
> * create raid0/10/5 using whole devices (eg. /dev/sda) (raid size must extend capacity of single device; raid1 is not the case here)
> * create any partition with size exceeding HPA limit of single component
>
> According to the code of rescan_partitions(), if disk has capacity limit enabled (HPA) and boundary of partition extends beyond that limit, it is bypassed/unlocked (regardless of libata.ignore_hpa state).
> As for single block device it is proper action - for raid using whole block device it is not. The partition table despite the fact it is read from single disk belongs to raid array. Values in RAID context are set properly. But from the point of view of single device - they are not.
> Problem is that in general rescan_partition() has no knowledge of any raid array using that block device. Moreover, that raid is not assembled yet.
> And the result: as HPA limit is unlocked and metadata on this device in not recognizable anymore - in worst case raid is not assembled or failed.
>
> The simplest resolution is to conditionally forbid HPA unlocking (eg. extending ignore_hpa parameter) but I suppose a better and more 'intelligent' solution can be found.

This has come up a couple times and I think the proper solution is to
always unlock HPA and expose both sizes - locked and unlocked and let
dm, md or whatever do whatever is approriate. Block or driver layer
doesn't have any way to determine which one is the right bet - it
simply doesn't have enough information. I tried to bounce this idea
off people who whould actually be using this (dm/md) but haven't heard
back yet.

Thanks.

--
tejun

--
dm-devel mailing list
dm-devel@redhat.com
https://www.redhat.com/mailman/listinfo/dm-devel
 
Old 11-03-2011, 06:00 PM
Phillip Susi
 
Default HPA unlock during partition scan of RAID components

On 11/3/2011 11:54 AM, Tejun Heo wrote:

This has come up a couple times and I think the proper solution is to
always unlock HPA and expose both sizes - locked and unlocked and let
dm, md or whatever do whatever is approriate. Block or driver layer
doesn't have any way to determine which one is the right bet - it
simply doesn't have enough information. I tried to bounce this idea
off people who whould actually be using this (dm/md) but haven't heard
back yet.


Simply making the smaller size available does not solve the problem of
making the part of the drive that is supposed to remain hidden
accessible to user space, and it remains unlocked across a reboot, which
usually makes the bios fail to recognize such drives.


The only reason I am aware of for unlocking the hpa is to avoid problems
caused by upgrading an old system that was installed using the unlock
behavior and thus, incorrectly extended its partition into the protected
area. It seems the appropriate fix for that it for distribution upgrade
scripts to test for this and configure the boot loader to pass the
unlock flag ( or maybe fix the problem by shrinking the partition ),
rather than have the kernel continue to try unlocking things by default.


--
dm-devel mailing list
dm-devel@redhat.com
https://www.redhat.com/mailman/listinfo/dm-devel
 
Old 11-03-2011, 07:46 PM
NeilBrown
 
Default HPA unlock during partition scan of RAID components

On Thu, 3 Nov 2011 08:54:16 -0700 Tejun Heo <tj@kernel.org> wrote:

> Hello,
>
> (cc'ing dm-devel)
>
> On Thu, Nov 03, 2011 at 01:02:37AM +0000, Hawrylewicz Czarnowski, Przemyslaw wrote:
> > Recently we have encountered a problem with unlocking of HPA on disks belonging to RAID array.
> > The scenario in the simplest form is as follows:
> > * take HPA resized drives
> > * use SW Raid solution with metadata located at the end of disk (eg. md v1.0, IMSM)
> > * create raid0/10/5 using whole devices (eg. /dev/sda) (raid size must extend capacity of single device; raid1 is not the case here)
> > * create any partition with size exceeding HPA limit of single component
> >
> > According to the code of rescan_partitions(), if disk has capacity limit enabled (HPA) and boundary of partition extends beyond that limit, it is bypassed/unlocked (regardless of libata.ignore_hpa state).
> > As for single block device it is proper action - for raid using whole block device it is not. The partition table despite the fact it is read from single disk belongs to raid array. Values in RAID context are set properly. But from the point of view of single device - they are not.
> > Problem is that in general rescan_partition() has no knowledge of any raid array using that block device. Moreover, that raid is not assembled yet.
> > And the result: as HPA limit is unlocked and metadata on this device in not recognizable anymore - in worst case raid is not assembled or failed.
> >
> > The simplest resolution is to conditionally forbid HPA unlocking (eg. extending ignore_hpa parameter) but I suppose a better and more 'intelligent' solution can be found.
>
> This has come up a couple times and I think the proper solution is to
> always unlock HPA and expose both sizes - locked and unlocked and let
> dm, md or whatever do whatever is approriate. Block or driver layer
> doesn't have any way to determine which one is the right bet - it
> simply doesn't have enough information. I tried to bounce this idea
> off people who whould actually be using this (dm/md) but haven't heard
> back yet.
>
> Thanks.
>

What exactly do you mean by "expose both sizes" ??
A new ioctl - BLKGETHPASIZE64 ??

That might work, but I think it would be good if there were also an ioctl
BLKHBALOCK which changed BLKGETSIZE64 to match BLKGETHPASIZE64.
Then some user-space tools could examine the device with a full understanding
of md, dm, dmraid, partitions, filesystems etc etc and make a reasonably
informed decision. And then put that decision into effect.

??

NeilBrown
--
dm-devel mailing list
dm-devel@redhat.com
https://www.redhat.com/mailman/listinfo/dm-devel
 
Old 11-03-2011, 08:08 PM
Tejun Heo
 
Default HPA unlock during partition scan of RAID components

On Fri, Nov 04, 2011 at 07:46:52AM +1100, NeilBrown wrote:
> What exactly do you mean by "expose both sizes" ??
> A new ioctl - BLKGETHPASIZE64 ??
>
> That might work, but I think it would be good if there were also an ioctl
> BLKHBALOCK which changed BLKGETSIZE64 to match BLKGETHPASIZE64.
> Then some user-space tools could examine the device with a full understanding
> of md, dm, dmraid, partitions, filesystems etc etc and make a reasonably
> informed decision. And then put that decision into effect.

In kernel, just another size field. Out of kernel, I was thinking
more along the line of a new /sysfs field, but yeah maybe another
ioctl. At this point, I don't really think making unlocking
selectable is a good idea. That has to go through device detach /
attach cycle and what if someone else is already using first half of
the disk? We can try to be sneaky and slip in device size change
underneath it but that just sounds too crazy to me. IMHO, we should
unlock by default and just let everyone know what the size before
unlocking was in case that could be useful.

Thank you.

--
tejun

--
dm-devel mailing list
dm-devel@redhat.com
https://www.redhat.com/mailman/listinfo/dm-devel
 
Old 11-04-2011, 01:48 PM
Tejun Heo
 
Default HPA unlock during partition scan of RAID components

Hello,

On Thu, Nov 3, 2011 at 12:00 PM, Phillip Susi <psusi@cfl.rr.com> wrote:
> Simply making the smaller size available does not solve the problem of
> making the part of the drive that is supposed to remain hidden accessible to
> user space, and it remains unlocked across a reboot, which usually makes the
> bios fail to recognize such drives.
>
> The only reason I am aware of for unlocking the hpa is to avoid problems
> caused by upgrading an old system that was installed using the unlock
> behavior and thus, incorrectly extended its partition into the protected
> area. *It seems the appropriate fix for that it for distribution upgrade
> scripts to test for this and configure the boot loader to pass the unlock
> flag ( or maybe fix the problem by shrinking the partition ), rather than
> have the kernel continue to try unlocking things by default.

I frankly don't care about BIOSen locking up afterwards and I don't
think they are more common than the issues from not unlocking (e.g.
moving harddrive to another machine, hot-plugging - partition -
reboot). There will be a kernel param to disable unlocking behavior
and that would be it. On top of that if bios is that screwed it's
probably a good idea to not use bios raid which is a silly thing to
begin with. Currently the problem is that the kernel doesn't unlock
by default and the heuristics can't be turned off, so we have unhappy
customers on both sides. Anyways, if md/dm people are on board, I'm
gonna push for that but otherwise there isn't much point.

Thanks.

--
tejun

--
dm-devel mailing list
dm-devel@redhat.com
https://www.redhat.com/mailman/listinfo/dm-devel
 
Old 11-04-2011, 02:42 PM
Phillip Susi
 
Default HPA unlock during partition scan of RAID components

On 11/4/2011 10:48 AM, Tejun Heo wrote:

I frankly don't care about BIOSen locking up afterwards and I don't
think they are more common than the issues from not unlocking (e.g.
moving harddrive to another machine, hot-plugging - partition -
reboot). There will be a kernel param to disable unlocking behavior


Could you clarify what issues you refer to? Moving a drive to another
machine?



and that would be it. On top of that if bios is that screwed it's
probably a good idea to not use bios raid which is a silly thing to
begin with. Currently the problem is that the kernel doesn't unlock
by default and the heuristics can't be turned off, so we have unhappy
customers on both sides. Anyways, if md/dm people are on board, I'm
gonna push for that but otherwise there isn't much point.


It isn't only a problem for bios raid, it also affects mdadm raid.
Being able to defeat the heuristic would help, but I still don't see why
the heuristic should be implemented in the kernel instead of user space?



--
dm-devel mailing list
dm-devel@redhat.com
https://www.redhat.com/mailman/listinfo/dm-devel
 
Old 11-04-2011, 02:52 PM
Tejun Heo
 
Default HPA unlock during partition scan of RAID components

Hello,

On Fri, Nov 04, 2011 at 11:42:20AM -0400, Phillip Susi wrote:
> On 11/4/2011 10:48 AM, Tejun Heo wrote:
> >I frankly don't care about BIOSen locking up afterwards and I don't
> >think they are more common than the issues from not unlocking (e.g.
> >moving harddrive to another machine, hot-plugging - partition -
> >reboot). There will be a kernel param to disable unlocking behavior
>
> Could you clarify what issues you refer to? Moving a drive to
> another machine?

Yeah, that or hotplugging, or changing BIOS config, updating BIOS,
and, some (rare) BIOSen even fail to set up things consistently across
reboots.

> >and that would be it. On top of that if bios is that screwed it's
> >probably a good idea to not use bios raid which is a silly thing to
> >begin with. Currently the problem is that the kernel doesn't unlock
> >by default and the heuristics can't be turned off, so we have unhappy
> >customers on both sides. Anyways, if md/dm people are on board, I'm
> >gonna push for that but otherwise there isn't much point.
>
> It isn't only a problem for bios raid, it also affects mdadm raid.

Exposing alt size should be enough for md/dm no matter how hpa is
setup.

> Being able to defeat the heuristic would help, but I still don't see
> why the heuristic should be implemented in the kernel instead of
> user space?

Because sans raids, it was good enough and doesn't recent md's have
metadata at the beginning anyway?

The thing is that you have your problem, and other people have other
issues w/ HPA. At this point, it's a misfeature abused by some silly
BIOSen mostly for silly BIOS raids. I think we should expose the most
we can to our users and if some BIOSen are screwed on soft reboot,
well, just accept it or turn off HPA unlocking manually.

Thanks.

--
tejun

--
dm-devel mailing list
dm-devel@redhat.com
https://www.redhat.com/mailman/listinfo/dm-devel
 
Old 11-04-2011, 03:26 PM
Phillip Susi
 
Default HPA unlock during partition scan of RAID components

On 11/4/2011 11:52 AM, Tejun Heo wrote:

Could you clarify what issues you refer to? Moving a drive to
another machine?


Yeah, that or hotplugging, or changing BIOS config, updating BIOS,
and, some (rare) BIOSen even fail to set up things consistently across
reboots.


I'm asking for a clarification not just a restatement. What problem
occurs when you move a drive to another machine, or hotplug the drive?



Exposing alt size should be enough for md/dm no matter how hpa is
setup.


So you are saying that mdadm should store the metadata for 0.9 and 1.0
relative to the locked end of the disk, even if it is currently
unlocked? I suppose then grub also will need to be able to detect if
the disk has been unlocked and adjust to using the locked size.



Being able to defeat the heuristic would help, but I still don't see
why the heuristic should be implemented in the kernel instead of
user space?


Because sans raids, it was good enough and doesn't recent md's have
metadata at the beginning anyway?


It isn't about whether it is good enough or not, it's about whether the
kernel should be using heuristics to make decisions, or leaving it up to
user space, where distros can come up with all kinds of complex logic to
handle all of the goofy cases. You complain that the block layer does
not have enough information to get it right, so why not let user space
decide, where it has access to all of the information it needs.



The thing is that you have your problem, and other people have other
issues w/ HPA. At this point, it's a misfeature abused by some silly
BIOSen mostly for silly BIOS raids. I think we should expose the most
we can to our users and if some BIOSen are screwed on soft reboot,
well, just accept it or turn off HPA unlocking manually.


It is actually not used FOR the bios raid. The bios is using the HPA
for unrelated reasons and it just happens that unlocking it breaks the
bios raid ( and mdadm with format 0.9 or 1.0 ).


--
dm-devel mailing list
dm-devel@redhat.com
https://www.redhat.com/mailman/listinfo/dm-devel
 
Old 11-04-2011, 03:32 PM
Tejun Heo
 
Default HPA unlock during partition scan of RAID components

Hello,

On Fri, Nov 04, 2011 at 12:26:02PM -0400, Phillip Susi wrote:
> I'm asking for a clarification not just a restatement. What problem
> occurs when you move a drive to another machine, or hotplug the
> drive?

Umm... partioned / mkfs'd while unlocked and bios (on whatever
machine) locks it on later boots.

> >Exposing alt size should be enough for md/dm no matter how hpa is
> >setup.
>
> So you are saying that mdadm should store the metadata for 0.9 and
> 1.0 relative to the locked end of the disk, even if it is currently
> unlocked? I suppose then grub also will need to be able to detect
> if the disk has been unlocked and adjust to using the locked size.

No, with both sizes exposed, mdadm should be able to detect metadata
at the end even if it was created w/ hpa locked. This is still
fundamentally flawed as if you move it to another machine, it may be
locked in completely different way, but that's as far as we can go.

> >Because sans raids, it was good enough and doesn't recent md's have
> >metadata at the beginning anyway?
>
> It isn't about whether it is good enough or not, it's about whether
> the kernel should be using heuristics to make decisions, or leaving
> it up to user space, where distros can come up with all kinds of
> complex logic to handle all of the goofy cases. You complain that
> the block layer does not have enough information to get it right, so
> why not let user space decide, where it has access to all of the
> information it needs.

Upto userspace how? Unlock it after root is mounted?

> >The thing is that you have your problem, and other people have other
> >issues w/ HPA. At this point, it's a misfeature abused by some silly
> >BIOSen mostly for silly BIOS raids. I think we should expose the most
> >we can to our users and if some BIOSen are screwed on soft reboot,
> >well, just accept it or turn off HPA unlocking manually.
>
> It is actually not used FOR the bios raid. The bios is using the
> HPA for unrelated reasons and it just happens that unlocking it
> breaks the bios raid ( and mdadm with format 0.9 or 1.0 ).

It is also used for bios raids. Some BIOSen lock it for no reason.
Some lock it if BIOS raid is enabled. Some lock it for recovery
partition. Some are just crazy. This already has been discussed to
death.

Thanks.

--
tejun

--
dm-devel mailing list
dm-devel@redhat.com
https://www.redhat.com/mailman/listinfo/dm-devel
 
Old 11-04-2011, 08:08 PM
Phillip Susi
 
Default HPA unlock during partition scan of RAID components

On 11/4/2011 12:32 PM, Tejun Heo wrote:

Umm... partioned / mkfs'd while unlocked and bios (on whatever
machine) locks it on later boots.


So the first machine does not set an HPA at all, you format the drive,
then move it to a machine whose bios blindly adds an HPA to the drive?
A bios certainly shouldn't be blindly adding an HPA to an already
formatted drive that did not have one before. If it does, why is that
our problem?



No, with both sizes exposed, mdadm should be able to detect metadata
at the end even if it was created w/ hpa locked. This is still
fundamentally flawed as if you move it to another machine, it may be
locked in completely different way, but that's as far as we can go.


So mdadm could check both places after the kernel always unlocks,
without any heuristic? And when creating an array, it would write it to
the real, unlocked end of the disk, possibly trashing whatever the bios
has stored there?



Upto userspace how? Unlock it after root is mounted?


Or in the initramfs. Or just deal with it as part of the upgrade
process, as I suggested before.


--
dm-devel mailing list
dm-devel@redhat.com
https://www.redhat.com/mailman/listinfo/dm-devel
 

Thread Tools




All times are GMT. The time now is 03:22 PM.

VBulletin, Copyright ©2000 - 2014, Jelsoft Enterprises Ltd.
Content Relevant URLs by vBSEO ©2007, Crawlability, Inc.
Copyright 2007 - 2008, www.linux-archive.org