Check your /etc/default/grub, if you use raid 1.
 

Am 29.07.2012 16:19, schrieb Bruno Wolff III:
> On Sun, Jul 29, 2012 at 10:02:00 -0400,
> Sam Varshavchik <mrsam@courier-mta.com> wrote:
>> There's a long standing combination of two bugs: the list of rd.md.uuid boot parameters generated by anaconda for
>> /etc/default/grub may not include the raid uuid of non-stock partitions like /home; and although the ramfs
>> initscript autodiscovers all raid volumes present, sometimes (not always, I'll estimate 5% of the time) if a uuid
>> is not enumerated in the boot parameters, one of the drives in the raid 1 volume may not get assembled at boot.
>
> My raid info is /etc/mdadm.conf and that is what gets used by dracut when building an initramfs as far as I can tell.


in theory

sam is right and the problem exists since F16

you have to add all your UUIDs from /etc/mdadm.conf with MD_UUID=<youruuid>
entries to the kernel line or you are randomly boot with degrared arrays

it does not help to have the driver in initramfs if the arrays are not s
tarted correct, it must also be used properly from the system

maybe tehre are people affected even not recognize their degraded arrays
until it is too late

--
users mailing list
users@lists.fedoraproject.org
To unsubscribe or change subscription options:
https://admin.fedoraproject.org/mailman/listinfo/users
Guidelines: http://fedoraproject.org/wiki/Mailing_list_guidelines
Have a question? Ask away: http://ask.fedoraproject.org

Check your /etc/default/grub, if you use raid 1.
 

Am 29.07.2012 16:42, schrieb Sam Varshavchik:
> And, fail/remove/add did not resync the drive. Because the volume uses an internal bitmap: oh, the newly-added
> drive has a valid bitmap, apparently from the same volume, so let's add this drive without resyncing it!
>
> That, I think is a bug. Failing a drive should zero its superblock, to force a real resync if it gets added back to
> the array.

no, no and no!
if you have some mechanic problems that could end in zero the superblocks of all disks

in my last tests with a RAID1 i pulled the power from one SATA disk
and after reboot i was not able to re-add the drive before manually
zero the superblock which is the right behavior

this was some months ago and played with both physical drives
(pull, reboot, zero, re-add, resync)

--
users mailing list
users@lists.fedoraproject.org
To unsubscribe or change subscription options:
https://admin.fedoraproject.org/mailman/listinfo/users
Guidelines: http://fedoraproject.org/wiki/Mailing_list_guidelines
Have a question? Ask away: http://ask.fedoraproject.org

Check your /etc/default/grub, if you use raid 1.
 

On Sun, Jul 29, 2012 at 11:23:51 -0400,
Sam Varshavchik <mrsam@courier-mta.com> wrote:


I think I remember seeing someone reporting the array assembly bug
when it's not enumerated in rd.md.uuid, but I can't find the bug
right now.




I found two related bugs. The second is listed as closed.

mdadm raid10 incremental assembly fails at boot, creating a degraded array:
https://bugzilla.redhat.com/show_bug.cgi?id=823597

make mdadm --add not do stupid things:
https://bugzilla.redhat.com/show_bug.cgi?id=808774
--
users mailing list
users@lists.fedoraproject.org
To unsubscribe or change subscription options:
https://admin.fedoraproject.org/mailman/listinfo/users
Guidelines: http://fedoraproject.org/wiki/Mailing_list_guidelines
Have a question? Ask away: http://ask.fedoraproject.org

Check your /etc/default/grub, if you use raid 1.
 

On Sun, Jul 29, 2012 at 10:02 AM, Sam Varshavchik
<mrsam@courier-mta.com> wrote:
> To: For users of Fedora Core releases <users@lists.fedoraproject.org>
> Subject: Check your /etc/default/grub, if you use raid 1.
> Message-ID: <cone.1343570520.888812.3982.1000@monster.email-scan.com>
> Content-Type: text/plain; charset="utf-8"; Format="flowed";
> DelSp="yes"
>
> There's a long standing combination of two bugs: the list of rd.md.uuid boot
> parameters generated by anaconda for /etc/default/grub may not include the
> raid uuid of non-stock partitions like /home; and although the ramfs
> initscript autodiscovers all raid volumes present, sometimes (not always,
> I'll estimate 5% of the time) if a uuid is not enumerated in the boot
> parameters, one of the drives in the raid 1 volume may not get assembled at
> boot.
>
> There's probably a third bug in here: mdmonitor should've mailed me when an
> array came up degraded at boot (I suspect that because mdmonitor gets
> started so early in the boot process, not all the moving pieces are there
> for mail delivery to happen). Eventually, you'll boot again with both drives
> in the array somehow, except they'll be out of sync, resulting in massive
> corruption. If you're lucky, you'll boot just with the other drive, and
> discover that your filesystem's contents are weeks/months out of date, and
> maybe you'll be lucky enough to figure out what happen, and switch back to
> the other drive and resync. But, not everyone's so lucky.
>
> This first started happening in F16. It took me a while to figure out the
> cause for an occasionally raid assembly failure at boot. Fixed it, and
> forgot about it. Well, looks like the F17 anaconda brought back the broken
> /etc/default/grub, which found its way into my grub.cfg, and I just lost a
> full day, cleaning up this mess.
>
> So, if you use raid 1 and upgraded to F17, you may need to fix this before
> it's too late: put back the missing uuid into /etc/default/grub, and into
> every entry in grub.cfg
>
> Pissed.

Thanks for the explanation and fix/workaround, Sam. This happened to
me as well. I ran fsck on the two mirrors independently and was able
to recover most of the data from the lost+found's. But I had been
brooding over the root cause until now.
--
users mailing list
users@lists.fedoraproject.org
To unsubscribe or change subscription options:
https://admin.fedoraproject.org/mailman/listinfo/users
Guidelines: http://fedoraproject.org/wiki/Mailing_list_guidelines
Have a question? Ask away: http://ask.fedoraproject.org

Check your /etc/default/grub, if you use raid 1.
 

Sam Varshavchik wrote:
> So, if you use raid 1 and upgraded to F17, you may need to fix this
> before it's too late: put back the missing uuid into /etc/default/grub,
> and into every entry in grub.cfg

I took my F16 RAID1 server to F17 this past weekend using preupgrade. My
UUIDs are correct in my /etc/default/grub file. I do not seem to be
affected.

Thanks though.
--
users mailing list
users@lists.fedoraproject.org
To unsubscribe or change subscription options:
https://admin.fedoraproject.org/mailman/listinfo/users
Guidelines: http://fedoraproject.org/wiki/Mailing_list_guidelines
Have a question? Ask away: http://ask.fedoraproject.org

Check your /etc/default/grub, if you use raid 1.
 

Michael Cronenworth writes:


Sam Varshavchik wrote:
> So, if you use raid 1 and upgraded to F17, you may need to fix this
> before it's too late: put back the missing uuid into /etc/default/grub,
> and into every entry in grub.cfg

I took my F16 RAID1 server to F17 this past weekend using preupgrade. My
UUIDs are correct in my /etc/default/grub file. I do not seem to be
affected.

Thanks though.


Do you remember what you told the installer to do, whether you told it to
write out a new bootloader entry (it's one of the last steps).


--
users mailing list
users@lists.fedoraproject.org
To unsubscribe or change subscription options:
https://admin.fedoraproject.org/mailman/listinfo/users
Guidelines: http://fedoraproject.org/wiki/Mailing_list_guidelines
Have a question? Ask away: http://ask.fedoraproject.org

Check your /etc/default/grub, if you use raid 1.
 

Sam Varshavchik wrote:
> Do you remember what you told the installer to do, whether you told it
> to write out a new bootloader entry (it's one of the last steps).

Preupgrade doesn't ask any questions and the whole update happens
without any user interaction.
--
users mailing list
users@lists.fedoraproject.org
To unsubscribe or change subscription options:
https://admin.fedoraproject.org/mailman/listinfo/users
Guidelines: http://fedoraproject.org/wiki/Mailing_list_guidelines
Have a question? Ask away: http://ask.fedoraproject.org

Check your /etc/default/grub, if you use raid 1.
 

Am 29.07.2012 16:25, schrieb Reindl Harald:
>
>
> Am 29.07.2012 16:19, schrieb Bruno Wolff III:
>> On Sun, Jul 29, 2012 at 10:02:00 -0400,
>> Sam Varshavchik <mrsam@courier-mta.com> wrote:
>>> There's a long standing combination of two bugs: the list of rd.md.uuid boot parameters generated by anaconda for
>>> /etc/default/grub may not include the raid uuid of non-stock partitions like /home; and although the ramfs
>>> initscript autodiscovers all raid volumes present, sometimes (not always, I'll estimate 5% of the time) if a uuid
>>> is not enumerated in the boot parameters, one of the drives in the raid 1 volume may not get assembled at boot.
>>
>> My raid info is /etc/mdadm.conf and that is what gets used by dracut when building an initramfs as far as I can tell.
>
>
> in theory
>
> sam is right and the problem exists since F16
>
> you have to add all your UUIDs from /etc/mdadm.conf with MD_UUID=<youruuid>
> entries to the kernel line or you are randomly boot with degrared arrays
>
> it does not help to have the driver in initramfs if the arrays are not s
> tarted correct, it must also be used properly from the system
>
> maybe tehre are people affected even not recognize their degraded arrays
> until it is too late
>

I would consider it a dracut bug, if rd.md.uuid is specified and other raid
arrays are assembled in the initramfs.

--
users mailing list
users@lists.fedoraproject.org
To unsubscribe or change subscription options:
https://admin.fedoraproject.org/mailman/listinfo/users
Guidelines: http://fedoraproject.org/wiki/Mailing_list_guidelines
Have a question? Ask away: http://ask.fedoraproject.org