FAQ Search Today's Posts Mark Forums Read
» Video Reviews

» Linux Archive

Linux-archive is a website aiming to archive linux email lists and to make them easily accessible for linux users/developers.


» Sponsor

» Partners

» Sponsor

Go Back   Linux Archive > Redhat > Fedora User

 
 
LinkBack Thread Tools
 
Old 02-12-2010, 06:23 AM
Suvayu Ali
 
Default Hardlinks and directories

Hi everyone,

When I tried to hardlink a directory today I ran into this,

$ ln muse test
ln: `muse': hard link not allowed for directory

So I did a little searching and found its not exactly a forbidden. So
far the closest to an understandable explanation/reasoning I came across
was a discussion in lwn[1]. So my question is how are hardlinks so
different from softlinks?

[1]http://lwn.net/Articles/249607/
--
Suvayu

Open source is the future. It sets us free.
--
users mailing list
users@lists.fedoraproject.org
To unsubscribe or change subscription options:
https://admin.fedoraproject.org/mailman/listinfo/users
Guidelines: http://fedoraproject.org/wiki/Communicate/MailingListGuidelines
 
Old 02-12-2010, 07:33 AM
g
 
Default Hardlinks and directories

Suvayu Ali wrote:

> So my question is how are hardlinks so different from softlinks?


back in 70's when i first started with unix, linking took several times of
rereading of manual before i really understood what and why.

have a look at these 2 links and see if it clears up for you.

http://en.wikipedia.org/wiki/Hard_link
http://en.wikipedia.org/wiki/Soft_link


hth.


--

peace out.

tc,hago.

g
.

****
in a free world without fences, who needs gates.
**
help microsoft stamp out piracy - give linux to a friend today.
**
to mess up a linux box, you need to work at it.
to mess up an ms windows box, you just need to *look* at it.
**
learn linux:
'Rute User's Tutorial and Exposition' http://rute.2038bug.com/index.html
'The Linux Documentation Project' http://www.tldp.org/
'LDP HOWTO-index' http://www.tldp.org/HOWTO/HOWTO-INDEX/index.html
'HowtoForge' http://howtoforge.com/
****

--
users mailing list
users@lists.fedoraproject.org
To unsubscribe or change subscription options:
https://admin.fedoraproject.org/mailman/listinfo/users
Guidelines: http://fedoraproject.org/wiki/Communicate/MailingListGuidelines
 
Old 02-12-2010, 12:41 PM
Patrick O'Callaghan
 
Default Hardlinks and directories

On Thu, 2010-02-11 at 23:23 -0800, Suvayu Ali wrote:
> Hi everyone,
>
> When I tried to hardlink a directory today I ran into this,
>
> $ ln muse test
> ln: `muse': hard link not allowed for directory
>
> So I did a little searching and found its not exactly a forbidden. So
> far the closest to an understandable explanation/reasoning I came across
> was a discussion in lwn[1]. So my question is how are hardlinks so
> different from softlinks?

A hard link is a directory entry that references an inode. Every
property of the file is represented in the inode, including its type,
ownership, permissions, size and pointers to the actual data, i.e. the
directory entry is simply a (name, inode) pair. As such, there can be
multiple directory entries referencing the same inode.

If the inode type is "directory", reading the data gives us another
level of the file hierarchy. However, consider what would happen if one
of the entries in this second level pointed (directly or indirectly)
back to the parent directory. Very early versions of Unix actually
allowed this, until people realized that you could create a circular
structure that was unreachable from the filesystem root, which would be
a Bad Thing (tm). (Note the analogy to garbage collection problems in
LISP and other languages).

In order to avoid this, the rule was established that you can't link to
directories, except when the directory itself is being created, at which
time two special entries are placed in it, called ".", and "..". These
are inserted directly by the system itself and are the only exceptions
allowed. This forces the entire structure to be a tree and not an
arbitrary graph.

However, as the space of inode numbers is local to a specific
filesystem, links cannot cross filesystem boundaries. Thus at a later
stage of development, the BSD version of Unix introduced symbolic links
as a way round this. A soft or symbolic link is just a file that
contains the name of another file (or directory or whatever). It's
marked as a soft link so the system can dereference it when it appears
in a path. Since it's just a name, there are no restrictions on what it
contains, in fact it may not point at anything that actually exists, and
of course can actually create a circular structure. Unix systems prevent
chaos by forbidding infinite deferencing in pathnames, i.e. the system
counts how many components have been traversed while trying to get to
the file, and throws an error when the count exceeds a certain number,
e.g. 17. This is a kludge but it keeps things sane. Note that no file
can exist which *only* has symbolic links to it. Once the last hard link
is broken, the file disappears. Note also that the inode maintains a
count of hard links, but not of symlinks, i.e. symlinks are in some ways
second-class citizens.

Soft links are similar to "aliases" on Windows. AFAIK Windows has no
concept analogous to hard links.

poc

--
users mailing list
users@lists.fedoraproject.org
To unsubscribe or change subscription options:
https://admin.fedoraproject.org/mailman/listinfo/users
Guidelines: http://fedoraproject.org/wiki/Communicate/MailingListGuidelines
 
Old 02-12-2010, 01:06 PM
Marko Vojinovic
 
Default Hardlinks and directories

On Friday 12 February 2010 07:23:02 Suvayu Ali wrote:
> $ ln muse test
> ln: `muse': hard link not allowed for directory
>
> So I did a little searching and found its not exactly a forbidden. So
> far the closest to an understandable explanation/reasoning I came across
> was a discussion in lwn[1]. So my question is how are hardlinks so
> different from softlinks?

If you are familiar with the concept of memory pointers in some (any)
programming language, this is basically the same thing.

A hard link is a pointer to some data on the disk.

A soft link is a pointer to a pointer.

Now, hard links are not allowed for directories since they would allow for
creation of loops (a directory containing itself), which is a Bad Idea, since
it breaks recursion. The filesystem needs to be a *tree* if recursion is to
function, so loops are forbidden (how would you delete a directory which
contains itself?).

However, soft links to directories are allowed, since they are just pointers
to other pointers, which makes them distinguishable from regular pointers
(hard links), and thus recursion algorithms can work around them ("don't
follow a symbolic link when recursing" is a typical option you can find in a
man page of various tools).

HTH, :-)
Marko




--
users mailing list
users@lists.fedoraproject.org
To unsubscribe or change subscription options:
https://admin.fedoraproject.org/mailman/listinfo/users
Guidelines: http://fedoraproject.org/wiki/Communicate/MailingListGuidelines
 
Old 02-12-2010, 02:04 PM
Don Quixote de la Mancha
 
Default Hardlinks and directories

On Fri, Feb 12, 2010 at 5:41 AM, Patrick O'Callaghan
<pocallaghan@gmail.com> wrote:
> Since it's just a name, there are no restrictions on what it
> contains, in fact it may not point at anything that actually exists, and
> of course can actually create a circular structure.

A Good Time Can Be Had By All, by symbolically linking a file that
does not *yet* exist, but will in the future.

The "ln -s" command requires the path of the file to which the link
will point, and the path of the link itself. But it doesn't care one
whit whether the linked-to file actually exists. If you can supply a
path to it, you could symbolically link to anywhere.

> Once the last hard link
> is broken, the file disappears. Note also that the inode maintains a
> count of hard links, but not of symlinks, i.e. symlinks are in some ways
> second-class citizens.

Actually that is not quite true. A seemingly bizarre and just about
always surprising but well-documented and surprisingly useful
requirement of UNIX filesystem symantics is that a file does not
actually disappear until *two* things both occur:

1. The last hard link disappears.

2. The file is closed.

Hilarity will ensue if you can find some way to open up a bunch of
giant files on a filesystem that is running out of space, then keep
them held open. No amount of trying to delete those files will cause
their payload data to disappear from the filesystem - they will only
disappear from the directory tree. Once you have the file open,
despite its being deleted, you can continue to read and write it just
like any other file.

Not just no other user, but no other process of any kind - not even
your own processes - will be able to locate the deleted files that you
are holding open. It will be very difficult for naive sysadmins to
even figure out what is going on. But the payload data will still
exist on the disk, just like any other file. I think the "lsof"
command file show that you have an un-named file open, but other than
reporting which filesystem it is part of, I don't think the actual
location of the file can be discovered in any way.

This causes a lot of grief to those who try to create file servers
that are completely faithful to UNIX filesystem semantics, especially
when the server is some other kind of operating system, and not any
kind of *NIX.

While it can be argued that keeping deleted files around until they
are closed is a completely asinine thing to do, consider how useful
they are as temporary files:

1. Create your temp file.
2. Open it.
3. *Immediately* delete it - yourself - by making an unlink(2) system call.
4. Use your temp file for whatever purpose suits you.

The cool thing about doing it this way, is that no matter how your
program should exit, your temp file will really and truly disappear
once you quit. Because the kernel takes care of closing open files
for your when your process exits, if you crash, or take some kind of
unexpected path to termination, you can be sure that that file will be
totally gone once your program terminates.

That is much, much harder to ensure on systems where one cannot delete
open files. On those systems, you have to ensure that you delete your
file *after* you are done with it, even if you crash or take some
weird error handling pathway to termination. Otherwise you will
eventually end up with a bunch of unwanted temp files all over the
place.

> Soft links are similar to "aliases" on Windows. AFAIK Windows has no
> concept analogous to hard links.

Windows aliases, like Macintosh Finder Aliases, are conceptually
similar to soft links, but aren't nearly as robust: Mac aliases and
Windows shortcuts are just plain regular files. To follow the alias
to the intended destination file, the user program has to make an API
call.

UNIX symlinks, however, are dereferenced entirely within the kernel;
user programs don't have to do anything at all to follow them.

If you try to open(2) a symlink, you'll immediately open the file it
points to, and not the symlink itself. In fact a separate system call
exists for the explicit purpose of reading a symlink when you *don't*
want to dereference it. I think that's what "ls -l" uses, when it
lists a symlink as well as the file it points to.

There is some advantage to Mac aliases over Windows shortcuts and UNIX
symlinks: they have the ability to hunt down the destination file no
matter where it may be. Dereferencing a Mac alias will mount file
servers, prompt you to insert removable media and the like, and the
operating system has a facility for automagically updating the alias
if the destination file has moved, provided that it is somehow able to
figure out the new location.

Interestingly, the Windows NTFS filesystem has full support for
*NIX-style symbolic links, but the Windows operating system does not
provide any manner of user interface for creating them.

I think that was done for the old POSIX subsystem in Windows NT4.
Back in the day, if one wanted to sell an operating system to a US
government agency, it had to be POSIX certified. So both Microsoft,
with the NT4 POSIX subsystem, and Apple, with A/UX, met that
certification by providing POSIX implementations that they never
expected the government employees to actually use. Instead the POSIX
implementations did nothing more than to facilitate the sale, with the
end-users running Win32 or Classic Mac OS applications on their new
computers.

While Microsoft doesn't really support the Windows POSIX subsystem
anymore, the symbolic link support is still present in NTFS. There
are I think some open source tools available for creating NTFS
symbolic links, if you want to play around with them. I think the
NTFS version of symbolic links are called "Joins" but my memory is
hazy.

I expect that the creation of an NTFS symbolic link is done with a
single Win32 system call, if you can figure out how to actually call
it.

Well that about beats the subject entirely to death.

Don Quixote
--
Don Quixote de la Mancha
quixote@dulcineatech.com
http://www.dulcineatech.com

Dulcinea Technologies Corporation: Software of Elegance and Beauty.
--
users mailing list
users@lists.fedoraproject.org
To unsubscribe or change subscription options:
https://admin.fedoraproject.org/mailman/listinfo/users
Guidelines: http://fedoraproject.org/wiki/Communicate/MailingListGuidelines
 
Old 02-12-2010, 02:10 PM
Tom H
 
Default Hardlinks and directories

> A hard link is a directory entry that references an inode. Every
> property of the file is represented in the inode, including its type,
> ownership, permissions, size and pointers to the actual data, i.e. the
> directory entry is simply a (name, inode) pair. As such, there can be
> multiple directory entries referencing the same inode.
...
> Soft links are similar to "aliases" on Windows. AFAIK Windows has no
> concept analogous to hard links.

Windows has had hard links since either Win2k or WinXP and calls them
junction points. AFAIK, they are only used in a default install in
WinVista and Win7 to hard link to/from deprecated user dirs.
--
users mailing list
users@lists.fedoraproject.org
To unsubscribe or change subscription options:
https://admin.fedoraproject.org/mailman/listinfo/users
Guidelines: http://fedoraproject.org/wiki/Communicate/MailingListGuidelines
 
Old 02-12-2010, 02:29 PM
Bruno Wolff III
 
Default Hardlinks and directories

On Fri, Feb 12, 2010 at 07:04:58 -0800,
Don Quixote de la Mancha <quixote@dulcineatech.com> wrote:
>
> Not just no other user, but no other process of any kind - not even
> your own processes - will be able to locate the deleted files that you
> are holding open. It will be very difficult for naive sysadmins to
> even figure out what is going on. But the payload data will still
> exist on the disk, just like any other file. I think the "lsof"
> command file show that you have an un-named file open, but other than
> reporting which filesystem it is part of, I don't think the actual
> location of the file can be discovered in any way.

lsof would allow you to find the processes that have the files open. And then
you could kill those processes to release the space. It looks like you should
be able to get size infomation of these files from lsof as well. So one
could script looking for processes that have large files open.
--
users mailing list
users@lists.fedoraproject.org
To unsubscribe or change subscription options:
https://admin.fedoraproject.org/mailman/listinfo/users
Guidelines: http://fedoraproject.org/wiki/Communicate/MailingListGuidelines
 
Old 02-12-2010, 03:00 PM
Roberto Ragusa
 
Default Hardlinks and directories

Don Quixote de la Mancha wrote:

> The "ln -s" command requires the path of the file to which the link
> will point, and the path of the link itself. But it doesn't care one
> whit whether the linked-to file actually exists. If you can supply a
> path to it, you could symbolically link to anywhere.

Add to that that you actually have to specify the first path from the
point of view of the second one (important if it is a relative path).
For example:

cd /tmp
ln -s passwd /etc/passwd2

now you have /etc/passwd2 linking to /etc/passwd (called "passwd"),
and /tmp has not been involved at all.

> I think the "lsof"
> command file show that you have an un-named file open, but other than
> reporting which filesystem it is part of, I don't think the actual
> location of the file can be discovered in any way.

Hmm?
>From lsof:

konsole 27162 rragusa 12u REG 8,8 55885008 1446680 /tmp/kde-rragusa/konsoleqbJrwb.tmp (deleted)

I also think you can regain access to the file content if you
use /proc/<pid>/fd/<fdnum>.

> The cool thing about doing it this way, is that no matter how your
> program should exit, your temp file will really and truly disappear
> once you quit. Because the kernel takes care of closing open files
> for your when your process exits, if you crash, or take some kind of
> unexpected path to termination, you can be sure that that file will be
> totally gone once your program terminates.
>
> That is much, much harder to ensure on systems where one cannot delete
> open files. On those systems, you have to ensure that you delete your
> file *after* you are done with it, even if you crash or take some
> weird error handling pathway to termination. Otherwise you will
> eventually end up with a bunch of unwanted temp files all over the
> place.

There is much bigger advantage. You can delete (and so recreate) executables
and libraries which are running.
This is the fundamental trick which creates the difference
between "yum update glibc" on Linux and "please reboot" on Windows.


--
Roberto Ragusa mail at robertoragusa.it
--
users mailing list
users@lists.fedoraproject.org
To unsubscribe or change subscription options:
https://admin.fedoraproject.org/mailman/listinfo/users
Guidelines: http://fedoraproject.org/wiki/Communicate/MailingListGuidelines
 
Old 02-12-2010, 04:08 PM
Patrick O'Callaghan
 
Default Hardlinks and directories

On Fri, 2010-02-12 at 07:04 -0800, Don Quixote de la Mancha wrote:
> Actually that is not quite true. A seemingly bizarre and just about
> always surprising but well-documented and surprisingly useful
> requirement of UNIX filesystem symantics is that a file does not
> actually disappear until *two* things both occur:
>
> 1. The last hard link disappears.
>
> 2. The file is closed.

Yes indeed. I was trying to simplify. The real criterion is "does the
inode have any links to it?" where "links" includes open file
descriptors.

Which of course explains why Unix has no "remove file" system call, just
"unlink()". The fact that the system reclaims space which can no longer
be accessed is merely an efficiency detail :-)

poc

--
users mailing list
users@lists.fedoraproject.org
To unsubscribe or change subscription options:
https://admin.fedoraproject.org/mailman/listinfo/users
Guidelines: http://fedoraproject.org/wiki/Communicate/MailingListGuidelines
 
Old 02-12-2010, 04:19 PM
Mikkel
 
Default Hardlinks and directories

On 02/12/2010 09:29 AM, Bruno Wolff III wrote:
>
> lsof would allow you to find the processes that have the files open. And then
> you could kill those processes to release the space. It looks like you should
> be able to get size infomation of these files from lsof as well. So one
> could script looking for processes that have large files open.

You might want to try using the +L1 and the -s options. It will
display the sizes of unlinked (deleted) files.

lsof +L1 -s

Mikkel
--

Do not meddle in the affairs of dragons,
for thou art crunchy and taste good with Ketchup!

--
users mailing list
users@lists.fedoraproject.org
To unsubscribe or change subscription options:
https://admin.fedoraproject.org/mailman/listinfo/users
Guidelines: http://fedoraproject.org/wiki/Communicate/MailingListGuidelines
 

Thread Tools




All times are GMT. The time now is 06:28 PM.

VBulletin, Copyright ©2000 - 2014, Jelsoft Enterprises Ltd.
Content Relevant URLs by vBSEO ©2007, Crawlability, Inc.
Copyright 2007 - 2008, www.linux-archive.org