FAQ Search Today's Posts Mark Forums Read
» Video Reviews

» Linux Archive

Linux-archive is a website aiming to archive linux email lists and to make them easily accessible for linux users/developers.


» Sponsor

» Partners

» Sponsor

Go Back   Linux Archive > Debian > Debian User

 
 
LinkBack Thread Tools
 
Old 06-26-2011, 10:22 AM
Luke Kenneth Casson Leighton
 
Default mdadm and UUIDs for its component drives

at martin's request i'm forwarding this to debian-user so that it can
be found for archival purposes and general discussion. this is the
context: a follow-up question will be sent, without all the crap
below.

l.

[original]

allo martin,

haven't spoken to you for a while. got an interesting feature request
/ bug in mdadm to report. here's a bit of background, a lovely ubuntu
user trying to tell the world he's got it right, when it's all quite
likely to be spectacularly... inept and ineffective

http://ubuntuforums.org/archive/index.php/t-582775.html

i have set up a system which has WD external USB drives: they're
RAID-1 mirrored. the idea is to keep the temperature of the machine
down, by having the data drives external: that way, the main fan
doesn't fire up and it's all nice and quiet.

the issue is this: if put in, say, a USB memory stick in first, and it
pops up as /dev/sdc, and _then_ put in the two USB external drives,
what happens to mdadm? it sees /dev/sdc as being, instead of one of
the RAID drives, as a USB memory stick!

from the above thread (and i can confirm it - i've tried):
mdadm --manage /dev/md1 --add
/dev/disk/by-id/usb-WD_Ext_HDD_1021_574341565933303838393734-0:0

mdadm i can confirm goes and hunts down the symlinks and adds
/dev/sdd! i don't _want_ it to add /dev/sdd, i want it to add
/dev/disk/by-id/usb-WD_blah_blah

question is: how? or, does it not matter: does mdadm use UUIDs internally?

tia,

[martin's reply]


On Sat, Jun 25, 2011 at 8:26 PM, martin f krafft <madduck@debian.org> wrote:
> also sprach Luke Kenneth Casson Leighton <luke.leighton@gmail.com> [2011.06.25.1938 +0200]:
>> mdadm i can confirm goes and hunts down the symlinks and adds
>> /dev/sdd! *i don't _want_ it to add /dev/sdd, i want it to add
>> /dev/disk/by-id/usb-WD_blah_blah
>>
>> question is: how? *or, does it not matter: does mdadm use UUIDs internally?
>
> Yes, it uses UUIDs. The problem you describe should not happen.
>
> Please direct your questions to debian-user@lists.debian.org so that
> others can profit from this discussion.
>
> --
> *.'`. * martin f. krafft <madduck@d.o> * * *Related projects:
> : :' *: *proud Debian developer * * * * * * * http://debiansystem.info
> `. `'` * http://people.debian.org/~madduck * *http://vcs-pkg.org
> *`- *Debian - when you have better things to do than fixing systems
>
> "without a god, life is only a matter of opinion."
> * * * * * * * * * * * * * * * * * * * * * * * * * *-- douglas adams
>


--
To UNSUBSCRIBE, email to debian-user-REQUEST@lists.debian.org
with a subject of "unsubscribe". Trouble? Contact listmaster@lists.debian.org
Archive: BANLkTi=r-cA1TSwkGk=QQ1t0ANxmZDq7VA@mail.gmail.com">http://lists.debian.org/BANLkTi=r-cA1TSwkGk=QQ1t0ANxmZDq7VA@mail.gmail.com
 
Old 06-26-2011, 10:41 AM
Luke Kenneth Casson Leighton
 
Default mdadm and UUIDs for its component drives

moorning martin: thanks for responding. apologies for not thinking to
ask on debian-user earlier, and apologies for the long-winded style:
just got ddragged out of bed to go chase a lamb out of the garden that
was eating our flowers and vegetables. if i wasn't stumbling about
half-asleep or concerned for our future food supply i'd find a lamb
head-butting fence posts incredibly funny.

On Sat, Jun 25, 2011 at 8:26 PM, martin f krafft <madduck@debian.org> wrote:

> also sprach Luke Kenneth Casson Leighton <luke.leighton@gmail.com> [2011.06.25.1938 +0200]:
>> mdadm i can confirm goes and hunts down the symlinks and adds
>> /dev/sdd! *i don't _want_ it to add /dev/sdd, i want it to add
>> /dev/disk/by-id/usb-WD_blah_blah
>>
>> question is: how? *or, does it not matter: does mdadm use UUIDs internally?
>
> Yes, it uses UUIDs. The problem you describe should not happen.

ok, so the question therefore morphs into a long-winded
self-answering one: what is it about mdadm that causes people not to
be aware that UUIDs are used internally, such that they invest quite a
bit of time to e.g. modify their udev rules in /etc/ (rather than add
alternatives to /usr/local) and thus make their lives more awkward for
future upgrades, and search for "solutions" such as endeavouring to
use --add /dev/disk/by-id/XXX?

the answer is that mdadm tracks down the hardlink and displays, as
best i can tell, only that, with no immediately obvious options to get
it to display the disk UUIDs.

sooo.... here's some further questions:

* is there an option to mdadm to make it display UUIDs instead of or
as well as the disk name?

* if not, would adding one be a good idea?

* also, how about making mention of how mdadm works, in the man page
somewhere reaaasonably prominently?

the basic gist is that mdadm is a fantastic tool, does a far better
job than people believe or understand it to be doing, protects them
from themselves and any lack of knowledge of its inner workings, but
that means that unfortunately it's under-promoted and in danger of
being ignored (or worse, NIH-rewritten!)

i would hate to see a "better" tool being written which has, at the
very top of its home page, and in all freshmeat and sourceforget
prominent short descriptions, "yeah! we're l33t! our software RAID
tool uses UUIDs, which makes it better than mdadm. we r0ck!"



l.


--
To UNSUBSCRIBE, email to debian-user-REQUEST@lists.debian.org
with a subject of "unsubscribe". Trouble? Contact listmaster@lists.debian.org
Archive: BANLkTikbVC9ZsXT4SF-Lb3=gK2nqLmxgoQ@mail.gmail.com">http://lists.debian.org/BANLkTikbVC9ZsXT4SF-Lb3=gK2nqLmxgoQ@mail.gmail.com
 
Old 06-26-2011, 01:25 PM
Andrew McGlashan
 
Default mdadm and UUIDs for its component drives

Hi Luke,

Luke Kenneth Casson Leighton wrote:

the answer is that mdadm tracks down the hardlink and displays, as
best i can tell, only that, with no immediately obvious options to get
it to display the disk UUIDs.


I hear what you are saying, but I had a related problem which was similar.

When starting up a machine with two external 2TB drives which had been
set up as a mirror, it would sometimes only find one drive and then it
would happily mount the RAID1 array in a degraded state. Then, when the
other drive was added in, it had to do a rebuild of the array. It's not
much good having to rebuild the array after each boot when the mirror
should be perfectly fine.



So I solved it by adding the following to my /etc/rc.local

nohup /usr/local/bin/u1-mirror-drive.sh 2>&1 >/dev/null &



Note that external mirror drive, which mounts at /u1, has this
/etc/fstab entry:


/dev/mapper/vg--external--1-vg--external--1--u1 /u1 ext4
noauto,rw 0 0




I've masked the UUID below, but I don't see how it could cause any
trouble if I did not do that.....



The RAID1 is identified ONLY with UUID in /etc/mdadm/mdadm.conf

# grep ARRAY /etc/mdadm/mdadm.conf
ARRAY /dev/md0 UUID=06fd3d46-XXXX-XXXX-YYYY-ZZZZZZZZZZZZ



Here's my script that handles getting everything working after a boot:

# cat /usr/local/bin/u1-mirror-drive.sh
#!/bin/bash

RAID_DRIVE_ID=06fd3d46-XXXX-XXXX-YYYY-ZZZZZZZZZZZZ
RAID_DRIVES=2
TIME_LIMIT=60

echo RAID Drive ID: $RAID_DRIVE_ID
echo Number of Devices required: $RAID_DRIVES
echo Time Limit: $TIME_LIMIT

function error_drive_missing ()
{
echo
echo -en "aMissing drive(s) ... cannot assemble /dev/md0

"
/sbin/blkid|/bin/grep $RAID_DRIVE_ID
exit
}

(

echo "=================="
echo -en "Waiting for $RAID_DRIVES drives to be visible for
"linux_raid_member(s)" with blkid of: "$RAID_DRIVE_ID" ...
"

CNT=1
echo -en "00"
while [ $(/sbin/blkid|/bin/grep $RAID_DRIVE_ID|/usr/bin/wc -l) -lt
$RAID_DRIVES ]

do
if [ $CNT -lt 10 ]; then echo -en "$CNT"; else echo -en
"$CNT";fi
if [ $CNT -eq $TIME_LIMIT ]; then echo -en "$TIME_LIMIT
seconds ....
";error_drive_missing;fi

CNT=$(($CNT + 1))
/bin/sleep 1
done
echo -en "

All required drives found in $CNT seconds....
"
echo "=================="
echo -en "

"

cmds='/sbin/mdadm --assemble /dev/md0~
/sbin/vgscan~
/sbin/vgchange -ay vg-external-1~
/bin/mount /u1~
/bin/mount~
/bin/df -Th~
/bin/date~
/sbin/mdadm -D /dev/md0~
/bin/cat /proc/mdstat'

echo ".................."

IFS='~'
echo $cmds|
while read cmd
do
IFS=$'
'
echo "=================="
echo "${cmd}"
echo "------------------"
$cmd
echo -en "==================

"
done

echo ".................."

) 2>&1 | /usr/bin/tee /var/log/md0-vg-external-1-u1-wrk.$(date
+%Y%m%d%H%M).out




Right now, I am using 2x 2TB drives as mirrors, I plan to add a 3rd
drive as a 3-way mirror and to let it sync up, then remove the drive for
off-site storage. A 4th drive will come into play as well to rotate
off-site storage. Consequently, I catered for that scenario in the
"mount" script above -- ie I can easily change the number of RAID
devices to find before continuing if I choose to have more online or not
during boot. The script gives up if it cannot find the required number
of devices within 60 seconds, then I will have to manually intervene.


As you can see from the script, there is some logging taking place so
that I can check things over if necessary.


I may end up using multiple external mirrors at some stage; if I do that
then I'll likely have duplicated scripts for each metadevice and the
scripts will be [slightly] modified as required. I may end up with a
parameter file with a single script, but it's probably not worth the
further effort. Although using command line variables would be an easy
and viable option.


Anyway.... the long and short of it is, I can use mdadm without regard
to what devices are found, such as /dev/sda /dev/sdb /dev/sdc and the
like as I rely purely on the UUID functionality, which as you know,
mdadm handles perfectly well. ;-)



--
Kind Regards
AndrewM

Andrew McGlashan
Broadband Solutions now including VoIP


--
To UNSUBSCRIBE, email to debian-user-REQUEST@lists.debian.org
with a subject of "unsubscribe". Trouble? Contact listmaster@lists.debian.org

Archive: 4E07335A.4010504@affinityvision.com.au">http://lists.debian.org/4E07335A.4010504@affinityvision.com.au
 
Old 06-26-2011, 01:42 PM
Luke Kenneth Casson Leighton
 
Default mdadm and UUIDs for its component drives

On Sun, Jun 26, 2011 at 2:25 PM, Andrew McGlashan
<andrew.mcglashan@affinityvision.com.au> wrote:

> I hear what you are saying, but I had a related problem which was similar.

well... it's funny, because this is exactly what i need.

> Anyway.... the long and short of it is, I can use mdadm without regard to
> what devices are found, such as /dev/sda /dev/sdb /dev/sdc and the like as I
> rely purely on the UUID functionality, which as you know, mdadm handles
> perfectly well. *;-)



well. that was nice. the scenario you describe is precisely what i
sort-of had planned, but didn't have the expertise to do so was going
to recommend just two drives and then rsync to the other two.

_however_, given that you've solved exactly what is needed / best /
recommended for when you have 4 external drives (which i do) that's
bloody fantastic

ok, i bring in phil now, who i was talking to yesterday about this.
what he said was (and i may get this wrong: it only went in partly) -
something along the lines of "remember to build the drives with
individual mdadm bitmaps enabled". this will save a great deal of
arseing about when re-adding drives which didn't get properly added:
only 1/2 a 1Tb drive will need syncing, not an entire drive the
bitmap system he says has hierarchical granularity apparently.

also, he recommended taking at least one of the external drives *out*
of its external box and making it *internal*. the reason for that is
that a) if the drives ever get powered down accidentally (e.g. by
cleaning ladies) you're f****d b) if you move a drive or two
internally, it's possible to prioritise those drives as "reading"
ones, and to make the external ones "write" priority. so, the data
gets read from the internal one, and changes get propagated to all
drives.

... thoughts?

l.


--
To UNSUBSCRIBE, email to debian-user-REQUEST@lists.debian.org
with a subject of "unsubscribe". Trouble? Contact listmaster@lists.debian.org
Archive: BANLkTi=9WA9ayQ8sioeBsaocqJBigN+=7w@mail.gmail.com ">http://lists.debian.org/BANLkTi=9WA9ayQ8sioeBsaocqJBigN+=7w@mail.gmail.com
 
Old 06-26-2011, 01:55 PM
Luke Kenneth Casson Leighton
 
Default mdadm and UUIDs for its component drives

On Sun, Jun 26, 2011 at 2:25 PM, Andrew McGlashan
<andrew.mcglashan@affinityvision.com.au> wrote:

> Anyway.... the long and short of it is, I can use mdadm without regard to
> what devices are found, such as /dev/sda /dev/sdb /dev/sdc and the like as I
> rely purely on the UUID functionality, which as you know, mdadm handles
> perfectly well. *;-)



...mmmm you see, this is the bit that has me concerned. /dev/mdN can
be referred to by its unique UUID, but that's *not* what i'm referring
to. and, from what you're saying, you appear to be implying that yes,
the external drives can pop up as /dev/sda through /dev/sdc and be
"confused" - and thus it is pure luck (or actually design) that the
drives *happen* to all be part of the same identical RAID-1 mirroring
array.

so i realise martin that you've already answered, but it would be
really good if you could explicitly confirm:

yes, mdadm names its RAID drives by UUID (as can clearly be seen in
/dev/mdadm/mdadm.conf) but does it *also* refer to its *COMPONENT*
drives (internally, and non-obviously, and undocumentedly) by UUID and
then report to the outside world that it's using whatever name
(/dev/sdX) which can, under these external-drives scenario, change.

l.


--
To UNSUBSCRIBE, email to debian-user-REQUEST@lists.debian.org
with a subject of "unsubscribe". Trouble? Contact listmaster@lists.debian.org
Archive: BANLkTi=TZM+P-TGq1hJO6khLJg_Ufm9_dA@mail.gmail.com">http://lists.debian.org/BANLkTi=TZM+P-TGq1hJO6khLJg_Ufm9_dA@mail.gmail.com
 
Old 06-26-2011, 02:11 PM
martin f krafft
 
Default mdadm and UUIDs for its component drives

also sprach Luke Kenneth Casson Leighton <luke.leighton@gmail.com> [2011.06.26.1241 +0200]:
> * is there an option to mdadm to make it display UUIDs instead of or
> as well as the disk name?

mdadm -Es

> * also, how about making mention of how mdadm works, in the man page
> somewhere reaaasonably prominently?

Search manpage for "partitions". Please suggest patches if you find
the information insufficient.

--
.'`. martin f. krafft <madduck@d.o> Related projects:
: :' : proud Debian developer http://debiansystem.info
`. `'` http://people.debian.org/~madduck http://vcs-pkg.org
`- Debian - when you have better things to do than fixing systems

this product is under strict quality contril with perfect packing and
quality when leving the factory.please keep away from damp.high
temperature or sun expose.If found any detectives when purchasing.
please return the productby airmail to our administration section and
inform the time, place.and store of this purchase for our
improvement.We shall give you a satisfactory reply.Thanks for your
patronage and welcome your comments.
-- http://www.engrish.com
 
Old 06-26-2011, 02:34 PM
Luke Kenneth Casson Leighton
 
Default mdadm and UUIDs for its component drives

On Sun, Jun 26, 2011 at 3:11 PM, martin f krafft <madduck@debian.org> wrote:
> also sprach Luke Kenneth Casson Leighton <luke.leighton@gmail.com> [2011.06.26.1241 +0200]:
>> ** is there an option to mdadm to make it display UUIDs instead of or
>> as well as the disk name?
>
> mdadm -Es

oo! yaay! there is, however, no mention of the fact that these
options display UUIDs, and, confusingly, -s is listed as "only working
with the -R option"... oh wait, that's for Incremental Assembly mode
(eek!) ok, so -s (or --scan), scan /proc/mdstat, and -E for "show
components".



>> ** also, how about making mention of how mdadm works, in the man page
>> somewhere reaaasonably prominently?
>
> Search manpage for "partitions".

that's odd. i read around each part (man mdadm^M /partitions^M),
paragraph back and forwards: no mention of the UUIDs of drive
components of an array was clearly evident.

> Please suggest patches if you find the information insufficient.

ok. feeling slightly overwhelmed by the task, my lack of knowledge
on the detailed workings of mdadm somewhat getting in the way, but
i'll do my best.

l.


> --
> *.'`. * martin f. krafft <madduck@d.o> * * *Related projects:
> : :' *: *proud Debian developer * * * * * * * http://debiansystem.info
> `. `'` * http://people.debian.org/~madduck * *http://vcs-pkg.org
> *`- *Debian - when you have better things to do than fixing systems
>
> this product is under strict quality contril with perfect packing and
> quality when leving the factory.please keep away from damp.high
> temperature or sun expose.If found any detectives when purchasing.
> please return the productby airmail to our administration section and
> inform the time, place.and store of this purchase for our
> improvement.We shall give you a satisfactory reply.Thanks for your
> patronage and welcome your comments.
> * * * * * * * * * * * * * * * * * * * * * * -- http://www.engrish.com
>


--
To UNSUBSCRIBE, email to debian-user-REQUEST@lists.debian.org
with a subject of "unsubscribe". Trouble? Contact listmaster@lists.debian.org
Archive: BANLkTinX=Y6pxcPb2+Csk8un6LLV31RfMw@mail.gmail.com ">http://lists.debian.org/BANLkTinX=Y6pxcPb2+Csk8un6LLV31RfMw@mail.gmail.com
 
Old 06-26-2011, 03:02 PM
Andrew McGlashan
 
Default mdadm and UUIDs for its component drives

Luke Kenneth Casson Leighton wrote:

yes, mdadm names its RAID drives by UUID (as can clearly be seen in
/dev/mdadm/mdadm.conf) but does it *also* refer to its *COMPONENT*
drives (internally, and non-obviously, and undocumentedly) by UUID and
then report to the outside world that it's using whatever name
(/dev/sdX) which can, under these external-drives scenario, change.

l.


The other thing is that both drives in the array have the same UUID, so
you need to be able to tell them apart some way or another and the
/dev/sd* view is just fine.



And this works fine too fwiw :

mdadm -D /dev/disk/by-id/md-uuid-*

So long as mdadm can determine the drives in use, I don't care how it
uses them internally. However, if a drive goes bad, then I need to know
which one.


Let's say that /dev/sda has gone bad of a two drive RAID1 array; I can
visually detect the drive by doing the following:


dd if=/dev/sda of=/dev/null

Go look to see which drive is busy [hopefully it will show with a
flashing activity LED] and I can see which one has failed -- if that
doesn't work, then I can reverse the test and try all drives that are
meant to be okay to eliminate them.


--
Kind Regards
AndrewM

Andrew McGlashan
Broadband Solutions now including VoIP


--
To UNSUBSCRIBE, email to debian-user-REQUEST@lists.debian.org
with a subject of "unsubscribe". Trouble? Contact listmaster@lists.debian.org

Archive: 4E074A21.6000806@affinityvision.com.au">http://lists.debian.org/4E074A21.6000806@affinityvision.com.au
 
Old 06-26-2011, 03:26 PM
martin f krafft
 
Default mdadm and UUIDs for its component drives

also sprach Luke Kenneth Casson Leighton <luke.leighton@gmail.com> [2011.06.26.1634 +0200]:
> > Search manpage for "partitions".
>
> that's odd. i read around each part (man mdadm^M /partitions^M),
> paragraph back and forwards: no mention of the UUIDs of drive
> components of an array was clearly evident.

I was not trying to suggest that there was a mention of the UUIDs.
mdadm's manpage only mentions /proc/partitions; it scans that "file"
and then looks for UUIDs on each of the listed partitions, building
a list indexed by UUID [0]. This is called "scanning".

And now it has the list of devices (partitions) that constitute
individual arrays identified by UUID…

0. not sure this is the actual implementation…

> > Please suggest patches if you find the information insufficient.
>
> ok. feeling slightly overwhelmed by the task, my lack of
> knowledge on the detailed workings of mdadm somewhat getting in
> the way, but i'll do my best.

I'll try my best to provide feedback.

Thanks,

--
.'`. martin f. krafft <madduck@d.o> Related projects:
: :' : proud Debian developer http://debiansystem.info
`. `'` http://people.debian.org/~madduck http://vcs-pkg.org
`- Debian - when you have better things to do than fixing systems

"no, 'eureka' is greek for 'this bath is too hot.'"
-- dr. who
 
Old 06-26-2011, 03:29 PM
William Hopkins
 
Default mdadm and UUIDs for its component drives

On 06/27/11 at 01:02am, Andrew McGlashan wrote:
> Luke Kenneth Casson Leighton wrote:
> > yes, mdadm names its RAID drives by UUID (as can clearly be seen in
> >/dev/mdadm/mdadm.conf) but does it *also* refer to its *COMPONENT*
> >drives (internally, and non-obviously, and undocumentedly) by UUID and
> >then report to the outside world that it's using whatever name
> >(/dev/sdX) which can, under these external-drives scenario, change.
> >
> > l.
>
>
> Let's say that /dev/sda has gone bad of a two drive RAID1 array; I
> can visually detect the drive by doing the following:
>
> dd if=/dev/sda of=/dev/null
>
> Go look to see which drive is busy [hopefully it will show with a
> flashing activity LED] and I can see which one has failed -- if that
> doesn't work, then I can reverse the test and try all drives that
> are meant to be okay to eliminate them.

It seems to me that you'd be well served by simply using the UUID (by-uuid, not
by-id) in all things, including mounting and managing. Then you would never
need to figure out which disk sda was, you could just figure out which disk the
UUID was (and you'd only have to learn it once).

--
Liam
 

Thread Tools




All times are GMT. The time now is 03:11 AM.

VBulletin, Copyright ©2000 - 2014, Jelsoft Enterprises Ltd.
Content Relevant URLs by vBSEO ©2007, Crawlability, Inc.
Copyright ©2007 - 2008, www.linux-archive.org