FAQ Search Today's Posts Mark Forums Read
» Video Reviews

» Linux Archive

Linux-archive is a website aiming to archive linux email lists and to make them easily accessible for linux users/developers.


» Sponsor

» Partners

» Sponsor

Go Back   Linux Archive > Redhat > Fedora User

 
 
LinkBack Thread Tools
 
Old 04-01-2008, 07:39 AM
Nicholas Robinson
 
Default gorged harddrive

Ah, but who are you to say what its true feelings are? By intervening and
attempting to modify its behaviour are you not failing in your goal of
anarchy? Or does freedom to do what you wilt only apply to you?

See where true anarchy gets us?

On Monday 31 March 2008 23:32:59 charles f. zeitler wrote:
> --- Nicholas Robinson <npr@bottlehall.co.uk> wrote:
> > Surely your hard drive is only doing what it wilt.
> >
> > Why complain then?
>
> somethings not doing its 'true' will,
> and i'd like to fix it.
>
> charles zeitler
>
> : Do What Thou Wilt :
> :
> : Shall Be :
> :
> : The Whole of The Law :
>
> --Aleister Crowley--


--
fedora-list mailing list
fedora-list@redhat.com
To unsubscribe: https://www.redhat.com/mailman/listinfo/fedora-list
 
Old 04-01-2008, 12:41 PM
"Patrick O'Callaghan"
 
Default gorged harddrive

On Tue, 2008-04-01 at 10:57 +0800, Ed Greshko wrote:
> charles f. zeitler wrote:
> > --- Ed Greshko <Ed.Greshko@greshko.com> wrote:
> >
> >> charles f. zeitler wrote:
> >>> i've been pruning my "downloads" disk,
> >>> rather drastically, and not making a dent.
> >>>
> >>> today some more, less drastic but still
> >>> hefty, same result.
> >>>
> >>> revisited du- checked it twice - three
> >>> times- yup, it reports one directory at
> >>> 800+ gb- on a 400gb disk!
> >>>
> >>> fsck (forced) failed to report any problems,
> >>> there don't seem to be any symlinks,
> >>> and the sub-direcory sizes are sane...
> >>>
> >>> any ideas welcome, and appreciated.
> >> Instead of telling people what you are seeing it would be better to show the
> >> actual commands and output.
> >>
> > good point.
> >
> >
> > [fedora_8@Nyarlethotep ~]$ df
> > Filesystem 1K-blocks Used Available Use% Mounted on
> > /dev/sda8 13250836 11459264 1107608 92% /
> > /dev/sda9 1898468 825572 974904 46% /tmp
> > /dev/sda11 270882768 259964688 5414052 98% /home
> > /dev/sda10 1898468 1156484 643992 65% /var
> > /dev/sdc1 480719056 370452080 105383136 78% /home/fedora_8/music_vids
> > /dev/sda2 101105 17986 77898 19% /boot
> > tmpfs 1037552 248 1037304 1% /dev/shm
> > /dev/sdb1 384578164 330445976 34596748 91% /home/fedora_8/torrents_isos
> >
> > /dev/sdb1 is the drive under discussion.
> >
> >
> > [fedora_8@Nyarlethotep ~]$ du -sb t*s/*
> > 34256010522 torrents_isos/backup
> > 883393808812 torrents_isos/data
> > 58352749159 torrents_isos/finished
> > 75197043648 torrents_isos/finnished
> > 18222558607 torrents_isos/isos
> > 4781438 torrents_isos/logs
> > 16384 torrents_isos/lost+found
> > 4096 torrents_isos/lost_meta
> > 193903286 torrents_isos/meta
> > 1402434610 torrents_isos/new
> > 75585799469 torrents_isos/porn
> > 4096 torrents_isos/rar
> > 1318803 torrents_isos/shas
> > 4096 torrents_isos/tmp
> > 97996487 torrents_isos/total_meta.tar.bz2
> > 4096 torrents_isos/zip
> >
> > somethings wrong with t*s/data ....
> >
>
> OK.... I believe I know what the problem is. The torrents_isos/porn
> directory makes things seem larger than what they really are....
>
> No, just kidding.....
>
> I believe you may have a bunch of non-completed torrent downloads. When you
> start a torrent download the client will reserve the space and it will be
> reflected in the output of "du" but *not* in the output of "df". Thus with
> "du" you can have a situation where it "thinks" more disk space is being
> used than it actually is. FWIW, this is normal.

No.

If the space is reserved then it's used and du will see it. Otherwise
not. Try it and see. Don't be fooled by the apparent size of the
torrent, that's just the inode information. 'du' exists precisely
*because* the apparent inode size may be different from the real disk
usage because of sparse allocation, which BT uses to download sections
of the file out of sequence.

poc

--
fedora-list mailing list
fedora-list@redhat.com
To unsubscribe: https://www.redhat.com/mailman/listinfo/fedora-list
 
Old 04-01-2008, 12:50 PM
Ed Greshko
 
Default gorged harddrive

Patrick O'Callaghan wrote:

On Tue, 2008-04-01 at 10:57 +0800, Ed Greshko wrote:

charles f. zeitler wrote:

--- Ed Greshko <Ed.Greshko@greshko.com> wrote:


charles f. zeitler wrote:

i've been pruning my "downloads" disk,
rather drastically, and not making a dent.

today some more, less drastic but still
hefty, same result.

revisited du- checked it twice - three
times- yup, it reports one directory at
800+ gb- on a 400gb disk!

fsck (forced) failed to report any problems,
there don't seem to be any symlinks,
and the sub-direcory sizes are sane...

any ideas welcome, and appreciated.
Instead of telling people what you are seeing it would be better to show the
actual commands and output.



good point.


[fedora_8@Nyarlethotep ~]$ df
Filesystem 1K-blocks Used Available Use% Mounted on
/dev/sda8 13250836 11459264 1107608 92% /
/dev/sda9 1898468 825572 974904 46% /tmp
/dev/sda11 270882768 259964688 5414052 98% /home
/dev/sda10 1898468 1156484 643992 65% /var
/dev/sdc1 480719056 370452080 105383136 78% /home/fedora_8/music_vids
/dev/sda2 101105 17986 77898 19% /boot
tmpfs 1037552 248 1037304 1% /dev/shm
/dev/sdb1 384578164 330445976 34596748 91% /home/fedora_8/torrents_isos

/dev/sdb1 is the drive under discussion.


[fedora_8@Nyarlethotep ~]$ du -sb t*s/*
34256010522 torrents_isos/backup
883393808812 torrents_isos/data
58352749159 torrents_isos/finished
75197043648 torrents_isos/finnished
18222558607 torrents_isos/isos
4781438 torrents_isos/logs
16384 torrents_isos/lost+found
4096 torrents_isos/lost_meta
193903286 torrents_isos/meta
1402434610 torrents_isos/new
75585799469 torrents_isos/porn
4096 torrents_isos/rar
1318803 torrents_isos/shas
4096 torrents_isos/tmp
97996487 torrents_isos/total_meta.tar.bz2
4096 torrents_isos/zip

somethings wrong with t*s/data ....

OK.... I believe I know what the problem is. The torrents_isos/porn
directory makes things seem larger than what they really are....


No, just kidding.....

I believe you may have a bunch of non-completed torrent downloads. When you
start a torrent download the client will reserve the space and it will be
reflected in the output of "du" but *not* in the output of "df". Thus with
"du" you can have a situation where it "thinks" more disk space is being
used than it actually is. FWIW, this is normal.


No.


What do you mean "no"?

When starting the download of a torrent the output of "du" shows the disk
space has been used. But, in reality, it hasn't. df reflects that is
hasn't been actually used.




If the space is reserved then it's used and du will see it. Otherwise
not. Try it and see. Don't be fooled by the apparent size of the
torrent, that's just the inode information. 'du' exists precisely
*because* the apparent inode size may be different from the real disk
usage because of sparse allocation, which BT uses to download sections
of the file out of sequence.


Somehow I think you are sayihng/agreeing with what I've already said.

--
fedora-list mailing list
fedora-list@redhat.com
To unsubscribe: https://www.redhat.com/mailman/listinfo/fedora-list
 
Old 04-01-2008, 01:39 PM
Todd Denniston
 
Default gorged harddrive

Ed Greshko wrote, On 03/31/2008 10:57 PM:

charles f. zeitler wrote:

--- Ed Greshko <Ed.Greshko@greshko.com> wrote:


charles f. zeitler wrote:

i've been pruning my "downloads" disk,
rather drastically, and not making a dent.

today some more, less drastic but still
hefty, same result.

revisited du- checked it twice - three
times- yup, it reports one directory at
800+ gb- on a 400gb disk!

fsck (forced) failed to report any problems,
there don't seem to be any symlinks,
and the sub-direcory sizes are sane...

any ideas welcome, and appreciated.
Instead of telling people what you are seeing it would be better to
show the actual commands and output.



good point.


[fedora_8@Nyarlethotep ~]$ df
Filesystem 1K-blocks Used Available Use% Mounted on
/dev/sdb1 384578164 330445976 34596748 91%
/home/fedora_8/torrents_isos


/dev/sdb1 is the drive under discussion.


[fedora_8@Nyarlethotep ~]$ du -sb t*s/*
34256010522 torrents_isos/backup
883393808812 torrents_isos/data
58352749159 torrents_isos/finished

<snip>

somethings wrong with t*s/data ....



OK.... I believe I know what the problem is. The torrents_isos/porn
directory makes things seem larger than what they really are....


No, just kidding.....

I believe you may have a bunch of non-completed torrent downloads. When
you start a torrent download the client will reserve the space and it
will be reflected in the output of "du" but *not* in the output of
"df". Thus with "du" you can have a situation where it "thinks" more
disk space is being used than it actually is. FWIW, this is normal.




I think Ed is on the right track, but the strange thing is the reserve would
happen sparse...

cd /dev/shm/
mkdir normal sparse
dd if=/dev/zero of=normal/bunchozeros bs=1M count=20
cp --sparse=always normal/bunchozeros sparse/

ls -l normal/* sparse/*
-rw-r--r-- 1 root root 20971520 2008-04-01 09:28 normal/bunchozeros
-rw-r--r-- 1 root root 20971520 2008-04-01 09:30 sparse/bunchozeros

du -sb *
20971580 normal
20971580 sparse

du -s *
20504 normal
0 sparse

df -h .
Filesystem Size Used Avail Use% Mounted on
tmpfs 1014M 21M 994M 2% /dev/shm

OK!!!
df is showing how much disk is used/free based on inodes and filesizes (you
can run out of space if you have no inodes left).

du -s, du -sm show how much disk space is REALLY used on disk by the files.
du -b implies --apparent-size
du --apparent-size "print apparent sizes, rather than disk usage"


but you are probably looking to figure out WHAT for sure is causing this...
suggestion:
cd /home/fedora_8/torrents_isos/data
BIGONE=`du -sb * |sort -n |tail -1|awk '{print $2}'`
du -sb * |sort -n |tail -1
echo $BIGONE
cd $BIGONE
BIGONE=`du -sb * |sort -n |tail -1|awk '{print $2}'`
du -sb * |sort -n |tail -1
echo $BIGONE
cd $BIGONE
...
until the size seems to get reasonable, now you have probably located the
file/directory with the unreasonable file/directory.


knowing which file/directory is giving you strange results, you can look back
(in your mind or command history) to figure out what tried to create it, and
better yet you might be able to get rid of it, if you desire.
BTW I usually use `df -h` and `du -sm` when tracking these, as the numbers you
are getting are of the size that the human brain starts thinking 'was that as
many chars as last time???' instead of 'OK this is a reasonable number.'


--
Todd Denniston
Crane Division, Naval Surface Warfare Center (NSWC Crane)
Harnessing the Power of Technology for the Warfighter

--
fedora-list mailing list
fedora-list@redhat.com
To unsubscribe: https://www.redhat.com/mailman/listinfo/fedora-list
 
Old 04-01-2008, 02:53 PM
"Patrick O'Callaghan"
 
Default gorged harddrive

On Tue, 2008-04-01 at 20:50 +0800, Ed Greshko wrote:
> Patrick O'Callaghan wrote:
> > On Tue, 2008-04-01 at 10:57 +0800, Ed Greshko wrote:
> >> charles f. zeitler wrote:
> >>> --- Ed Greshko <Ed.Greshko@greshko.com> wrote:
> >>>
> >>>> charles f. zeitler wrote:
> >>>>> i've been pruning my "downloads" disk,
> >>>>> rather drastically, and not making a dent.
> >>>>>
> >>>>> today some more, less drastic but still
> >>>>> hefty, same result.
> >>>>>
> >>>>> revisited du- checked it twice - three
> >>>>> times- yup, it reports one directory at
> >>>>> 800+ gb- on a 400gb disk!
> >>>>>
> >>>>> fsck (forced) failed to report any problems,
> >>>>> there don't seem to be any symlinks,
> >>>>> and the sub-direcory sizes are sane...
> >>>>>
> >>>>> any ideas welcome, and appreciated.
> >>>> Instead of telling people what you are seeing it would be better to show the
> >>>> actual commands and output.
> >>>>
> >>> good point.
> >>>
> >>>
> >>> [fedora_8@Nyarlethotep ~]$ df
> >>> Filesystem 1K-blocks Used Available Use% Mounted on
> >>> /dev/sda8 13250836 11459264 1107608 92% /
> >>> /dev/sda9 1898468 825572 974904 46% /tmp
> >>> /dev/sda11 270882768 259964688 5414052 98% /home
> >>> /dev/sda10 1898468 1156484 643992 65% /var
> >>> /dev/sdc1 480719056 370452080 105383136 78% /home/fedora_8/music_vids
> >>> /dev/sda2 101105 17986 77898 19% /boot
> >>> tmpfs 1037552 248 1037304 1% /dev/shm
> >>> /dev/sdb1 384578164 330445976 34596748 91% /home/fedora_8/torrents_isos
> >>>
> >>> /dev/sdb1 is the drive under discussion.
> >>>
> >>>
> >>> [fedora_8@Nyarlethotep ~]$ du -sb t*s/*
> >>> 34256010522 torrents_isos/backup
> >>> 883393808812 torrents_isos/data
> >>> 58352749159 torrents_isos/finished
> >>> 75197043648 torrents_isos/finnished
> >>> 18222558607 torrents_isos/isos
> >>> 4781438 torrents_isos/logs
> >>> 16384 torrents_isos/lost+found
> >>> 4096 torrents_isos/lost_meta
> >>> 193903286 torrents_isos/meta
> >>> 1402434610 torrents_isos/new
> >>> 75585799469 torrents_isos/porn
> >>> 4096 torrents_isos/rar
> >>> 1318803 torrents_isos/shas
> >>> 4096 torrents_isos/tmp
> >>> 97996487 torrents_isos/total_meta.tar.bz2
> >>> 4096 torrents_isos/zip
> >>>
> >>> somethings wrong with t*s/data ....
> >>>
> >> OK.... I believe I know what the problem is. The torrents_isos/porn
> >> directory makes things seem larger than what they really are....
> >>
> >> No, just kidding.....
> >>
> >> I believe you may have a bunch of non-completed torrent downloads. When you
> >> start a torrent download the client will reserve the space and it will be
> >> reflected in the output of "du" but *not* in the output of "df". Thus with
> >> "du" you can have a situation where it "thinks" more disk space is being
> >> used than it actually is. FWIW, this is normal.
> >
> > No.
>
> What do you mean "no"?
>
> When starting the download of a torrent the output of "du" shows the disk
> space has been used. But, in reality, it hasn't. df reflects that is
> hasn't been actually used.

There are two scenarios, according to whether your torrent client
preallocates space or not. I use Ktorrent, which can be set to do
either. In the default case, no preallocation is done. After starting a
torrent an 'ls -l' of the file shows its full size, but a 'du' shows
only what has actually been downloaded. I checked this before my
previous post and I just checked it again to be sure.

If preallocation is on then of course 'du' will show the allocated
space, but 'df' will account for it. I also checked this BTW. The
numbers add up.

In other words 'df' and 'du' are mutually consistent.

poc

--
fedora-list mailing list
fedora-list@redhat.com
To unsubscribe: https://www.redhat.com/mailman/listinfo/fedora-list
 
Old 04-01-2008, 03:22 PM
"Patrick O'Callaghan"
 
Default gorged harddrive

On Tue, 2008-04-01 at 09:39 -0400, Todd Denniston wrote:
> df is showing how much disk is used/free based on inodes and filesizes

'df' is a wrapper round the statfs system call, which reads info
directly from the filesystem superblock. It takes no account whatever of
the apparent size of any individual files. The superblock knows how much
free space (and inodes) it has, and that's that. Assuming it's not
broken of course ...

> (you
> can run out of space if you have no inodes left).

True, and often forgotten.

> du -s, du -sm show how much disk space is REALLY used on disk by the
> files.

Including not only their contents but also the system overhead such as
indirect blocks etc. 'du' tells you 'how much space would I recover if
this file was deleted'.

poc

--
fedora-list mailing list
fedora-list@redhat.com
To unsubscribe: https://www.redhat.com/mailman/listinfo/fedora-list
 
Old 04-01-2008, 05:45 PM
Roger Heflin
 
Default gorged harddrive

Todd Denniston wrote:

Ed Greshko wrote, On 03/31/2008 10:57 PM:

charles f. zeitler wrote:

--- Ed Greshko <Ed.Greshko@greshko.com> wrote:


charles f. zeitler wrote:

i've been pruning my "downloads" disk,
rather drastically, and not making a dent.

today some more, less drastic but still
hefty, same result.

revisited du- checked it twice - three
times- yup, it reports one directory at
800+ gb- on a 400gb disk!

fsck (forced) failed to report any problems,
there don't seem to be any symlinks,
and the sub-direcory sizes are sane...

any ideas welcome, and appreciated.
Instead of telling people what you are seeing it would be better to
show the actual commands and output.



good point.


[fedora_8@Nyarlethotep ~]$ df
Filesystem 1K-blocks Used Available Use% Mounted on
/dev/sdb1 384578164 330445976 34596748 91%
/home/fedora_8/torrents_isos


/dev/sdb1 is the drive under discussion.


[fedora_8@Nyarlethotep ~]$ du -sb t*s/*
34256010522 torrents_isos/backup
883393808812 torrents_isos/data
58352749159 torrents_isos/finished

<snip>

somethings wrong with t*s/data ....



OK.... I believe I know what the problem is. The torrents_isos/porn
directory makes things seem larger than what they really are....


No, just kidding.....

I believe you may have a bunch of non-completed torrent downloads.
When you start a torrent download the client will reserve the space
and it will be reflected in the output of "du" but *not* in the output
of "df". Thus with "du" you can have a situation where it "thinks"
more disk space is being used than it actually is. FWIW, this is normal.




I think Ed is on the right track, but the strange thing is the reserve
would happen sparse...

cd /dev/shm/
mkdir normal sparse
dd if=/dev/zero of=normal/bunchozeros bs=1M count=20
cp --sparse=always normal/bunchozeros sparse/

ls -l normal/* sparse/*
-rw-r--r-- 1 root root 20971520 2008-04-01 09:28 normal/bunchozeros
-rw-r--r-- 1 root root 20971520 2008-04-01 09:30 sparse/bunchozeros

du -sb *
20971580 normal
20971580 sparse

du -s *
20504 normal
0 sparse

df -h .
Filesystem Size Used Avail Use% Mounted on
tmpfs 1014M 21M 994M 2% /dev/shm

OK!!!
df is showing how much disk is used/free based on inodes and filesizes
(you can run out of space if you have no inodes left).


Incorrect. df reads out the free blocks on the filesytem, file sizes and inodes
don't have anything to do with it.


The only way for du to differ from df (without a system issue) is for there to
be files that have been deleted but are still being accessed, see below for
things caused by system issues.


And if you run out of inodes it can show a lot of space being left free even
though you don't have any inodes to be able to use it.



du -s, du -sm show how much disk space is REALLY used on disk by the files.
du -b implies --apparent-size
du --apparent-size "print apparent sizes, rather than disk usage"


All correct.


I have seen corrupt directory entries that indicate that blocks are being used
(for a given file) that are not actually being used (or possibly don't even
exist at all) cause du to return wrong results, one would need to look through
the files in the funny directory and see if any of the sizes look wrong, given
that they are certain kinds of files there is a certain range that should be
valid, and an entry having something way outside of that range should be a hint
that that directory entry is corrupted. Often fsck will be unable to remove
the entries, I have had to use filesystem debuggers to remove this type of file
before, but even with those tools it is often difficult. I would expect one or
more files to be of a obviously wrong size.


If the systems crashes at exactly the wrong time, you can get some corrupted
entries.


Roger

--
fedora-list mailing list
fedora-list@redhat.com
To unsubscribe: https://www.redhat.com/mailman/listinfo/fedora-list
 
Old 04-01-2008, 11:29 PM
Ed Greshko
 
Default gorged harddrive

Patrick O'Callaghan wrote:

On Tue, 2008-04-01 at 20:50 +0800, Ed Greshko wrote:

Patrick O'Callaghan wrote:

On Tue, 2008-04-01 at 10:57 +0800, Ed Greshko wrote:

charles f. zeitler wrote:

--- Ed Greshko <Ed.Greshko@greshko.com> wrote:


charles f. zeitler wrote:

i've been pruning my "downloads" disk,
rather drastically, and not making a dent.

today some more, less drastic but still
hefty, same result.

revisited du- checked it twice - three
times- yup, it reports one directory at
800+ gb- on a 400gb disk!

fsck (forced) failed to report any problems,
there don't seem to be any symlinks,
and the sub-direcory sizes are sane...

any ideas welcome, and appreciated.
Instead of telling people what you are seeing it would be better to show the
actual commands and output.



good point.


[fedora_8@Nyarlethotep ~]$ df
Filesystem 1K-blocks Used Available Use% Mounted on
/dev/sda8 13250836 11459264 1107608 92% /
/dev/sda9 1898468 825572 974904 46% /tmp
/dev/sda11 270882768 259964688 5414052 98% /home
/dev/sda10 1898468 1156484 643992 65% /var
/dev/sdc1 480719056 370452080 105383136 78% /home/fedora_8/music_vids
/dev/sda2 101105 17986 77898 19% /boot
tmpfs 1037552 248 1037304 1% /dev/shm
/dev/sdb1 384578164 330445976 34596748 91% /home/fedora_8/torrents_isos

/dev/sdb1 is the drive under discussion.


[fedora_8@Nyarlethotep ~]$ du -sb t*s/*
34256010522 torrents_isos/backup
883393808812 torrents_isos/data
58352749159 torrents_isos/finished
75197043648 torrents_isos/finnished
18222558607 torrents_isos/isos
4781438 torrents_isos/logs
16384 torrents_isos/lost+found
4096 torrents_isos/lost_meta
193903286 torrents_isos/meta
1402434610 torrents_isos/new
75585799469 torrents_isos/porn
4096 torrents_isos/rar
1318803 torrents_isos/shas
4096 torrents_isos/tmp
97996487 torrents_isos/total_meta.tar.bz2
4096 torrents_isos/zip

somethings wrong with t*s/data ....

OK.... I believe I know what the problem is. The torrents_isos/porn
directory makes things seem larger than what they really are....


No, just kidding.....

I believe you may have a bunch of non-completed torrent downloads. When you
start a torrent download the client will reserve the space and it will be
reflected in the output of "du" but *not* in the output of "df". Thus with
"du" you can have a situation where it "thinks" more disk space is being
used than it actually is. FWIW, this is normal.

No.

What do you mean "no"?

When starting the download of a torrent the output of "du" shows the disk
space has been used. But, in reality, it hasn't. df reflects that is
hasn't been actually used.


There are two scenarios, according to whether your torrent client
preallocates space or not. I use Ktorrent, which can be set to do
either. In the default case, no preallocation is done. After starting a
torrent an 'ls -l' of the file shows its full size, but a 'du' shows
only what has actually been downloaded. I checked this before my
previous post and I just checked it again to be sure.

If preallocation is on then of course 'du' will show the allocated
space, but 'df' will account for it. I also checked this BTW. The
numbers add up.

In other words 'df' and 'du' are mutually consistent.


While it may be the case on your system, it isn't on mine and apparently
isn't on the OP's. And if you use azureus on your system you could prove it
to yourself too.


--
fedora-list mailing list
fedora-list@redhat.com
To unsubscribe: https://www.redhat.com/mailman/listinfo/fedora-list
 
Old 04-02-2008, 03:52 AM
"Patrick O'Callaghan"
 
Default gorged harddrive

On Wed, 2008-04-02 at 07:29 +0800, Ed Greshko wrote:
> Patrick O'Callaghan wrote:
> > On Tue, 2008-04-01 at 20:50 +0800, Ed Greshko wrote:
> >> Patrick O'Callaghan wrote:
> >>> On Tue, 2008-04-01 at 10:57 +0800, Ed Greshko wrote:
> >>>> charles f. zeitler wrote:
> >>>>> --- Ed Greshko <Ed.Greshko@greshko.com> wrote:
> >>>>>
> >>>>>> charles f. zeitler wrote:
> >>>>>>> i've been pruning my "downloads" disk,
> >>>>>>> rather drastically, and not making a dent.
> >>>>>>>
> >>>>>>> today some more, less drastic but still
> >>>>>>> hefty, same result.
> >>>>>>>
> >>>>>>> revisited du- checked it twice - three
> >>>>>>> times- yup, it reports one directory at
> >>>>>>> 800+ gb- on a 400gb disk!
> >>>>>>>
> >>>>>>> fsck (forced) failed to report any problems,
> >>>>>>> there don't seem to be any symlinks,
> >>>>>>> and the sub-direcory sizes are sane...
> >>>>>>>
> >>>>>>> any ideas welcome, and appreciated.
> >>>>>> Instead of telling people what you are seeing it would be better to show the
> >>>>>> actual commands and output.
> >>>>>>
> >>>>> good point.
> >>>>>
> >>>>>
> >>>>> [fedora_8@Nyarlethotep ~]$ df
> >>>>> Filesystem 1K-blocks Used Available Use% Mounted on
> >>>>> /dev/sda8 13250836 11459264 1107608 92% /
> >>>>> /dev/sda9 1898468 825572 974904 46% /tmp
> >>>>> /dev/sda11 270882768 259964688 5414052 98% /home
> >>>>> /dev/sda10 1898468 1156484 643992 65% /var
> >>>>> /dev/sdc1 480719056 370452080 105383136 78% /home/fedora_8/music_vids
> >>>>> /dev/sda2 101105 17986 77898 19% /boot
> >>>>> tmpfs 1037552 248 1037304 1% /dev/shm
> >>>>> /dev/sdb1 384578164 330445976 34596748 91% /home/fedora_8/torrents_isos
> >>>>>
> >>>>> /dev/sdb1 is the drive under discussion.
> >>>>>
> >>>>>
> >>>>> [fedora_8@Nyarlethotep ~]$ du -sb t*s/*
> >>>>> 34256010522 torrents_isos/backup
> >>>>> 883393808812 torrents_isos/data
> >>>>> 58352749159 torrents_isos/finished
> >>>>> 75197043648 torrents_isos/finnished
> >>>>> 18222558607 torrents_isos/isos
> >>>>> 4781438 torrents_isos/logs
> >>>>> 16384 torrents_isos/lost+found
> >>>>> 4096 torrents_isos/lost_meta
> >>>>> 193903286 torrents_isos/meta
> >>>>> 1402434610 torrents_isos/new
> >>>>> 75585799469 torrents_isos/porn
> >>>>> 4096 torrents_isos/rar
> >>>>> 1318803 torrents_isos/shas
> >>>>> 4096 torrents_isos/tmp
> >>>>> 97996487 torrents_isos/total_meta.tar.bz2
> >>>>> 4096 torrents_isos/zip
> >>>>>
> >>>>> somethings wrong with t*s/data ....
> >>>>>
> >>>> OK.... I believe I know what the problem is. The torrents_isos/porn
> >>>> directory makes things seem larger than what they really are....
> >>>>
> >>>> No, just kidding.....
> >>>>
> >>>> I believe you may have a bunch of non-completed torrent downloads. When you
> >>>> start a torrent download the client will reserve the space and it will be
> >>>> reflected in the output of "du" but *not* in the output of "df". Thus with
> >>>> "du" you can have a situation where it "thinks" more disk space is being
> >>>> used than it actually is. FWIW, this is normal.
> >>> No.
> >> What do you mean "no"?
> >>
> >> When starting the download of a torrent the output of "du" shows the disk
> >> space has been used. But, in reality, it hasn't. df reflects that is
> >> hasn't been actually used.
> >
> > There are two scenarios, according to whether your torrent client
> > preallocates space or not. I use Ktorrent, which can be set to do
> > either. In the default case, no preallocation is done. After starting a
> > torrent an 'ls -l' of the file shows its full size, but a 'du' shows
> > only what has actually been downloaded. I checked this before my
> > previous post and I just checked it again to be sure.
> >
> > If preallocation is on then of course 'du' will show the allocated
> > space, but 'df' will account for it. I also checked this BTW. The
> > numbers add up.
> >
> > In other words 'df' and 'du' are mutually consistent.
>
> While it may be the case on your system, it isn't on mine and apparently
> isn't on the OP's. And if you use azureus on your system you could prove it
> to yourself too.

Just installed Azureus and tested it. It behaves exactly the same as
Ktorrent, both with and without preallocation.

This is hardly surprising as we're talking about how the filesystem
works, not about about a specific application. As someone else pointed
out, a inconsistency between the results of 'du' and 'df' can occur when
a running process has an open file which has been unlinked, meaning the
space occupied by the file has not yet been reclaimed even though the
file no longer has a name. However, any inconsistency arising from this
would be in the *opposite* direction to what the OP is seeing, i.e. less
actual free space than accounted for by the filesystem size less the
space occupied by (visible) files.

In all of this I've been assuming that preallocation means what it has
always meant traditionally in Unix/Linux: write garbage into the file to
make sure disk space is allocated. It turns out the system can now do
this for you, with the fallocate() call. However this is a very recent
addition and people were still arguing about its semantics less than a
year ago -- see http://lwn.net/Articles/240571. One detail stands out:
with the correct parameter, the apparent size of the file (as reported
by the stat() call) does not change, even if space is allocated beyond
its end. The effect of this on 'df' and 'du' doesn't seem to be
documented anywhere. Furthermore, although it's not explicitly stated,
one presumes that the unused space is reclaimed when the file is closed,
so the OP's question still stands: how can 'du' report more space than
is being used, even if no processes have open files? For that matter,
how can the system preallocate more space than the size of the
filesystem? Doesn't make sense.

BTW, it might be worth knowing what filesystem the OP is using. In my
case it's ext3.

poc

--
fedora-list mailing list
fedora-list@redhat.com
To unsubscribe: https://www.redhat.com/mailman/listinfo/fedora-list
 
Old 04-02-2008, 04:29 AM
Ed Greshko
 
Default gorged harddrive

Patrick O'Callaghan wrote:


Just installed Azureus and tested it. It behaves exactly the same as
Ktorrent, both with and without preallocation.

This is hardly surprising as we're talking about how the filesystem
works, not about about a specific application. As someone else pointed
out, a inconsistency between the results of 'du' and 'df' can occur when
a running process has an open file which has been unlinked, meaning the
space occupied by the file has not yet been reclaimed even though the
file no longer has a name. However, any inconsistency arising from this
would be in the *opposite* direction to what the OP is seeing, i.e. less
actual free space than accounted for by the filesystem size less the
space occupied by (visible) files.

In all of this I've been assuming that preallocation means what it has
always meant traditionally in Unix/Linux: write garbage into the file to
make sure disk space is allocated. It turns out the system can now do
this for you, with the fallocate() call. However this is a very recent
addition and people were still arguing about its semantics less than a
year ago -- see http://lwn.net/Articles/240571. One detail stands out:
with the correct parameter, the apparent size of the file (as reported
by the stat() call) does not change, even if space is allocated beyond
its end. The effect of this on 'df' and 'du' doesn't seem to be
documented anywhere. Furthermore, although it's not explicitly stated,
one presumes that the unused space is reclaimed when the file is closed,
so the OP's question still stands: how can 'du' report more space than
is being used, even if no processes have open files? For that matter,
how can the system preallocate more space than the size of the
filesystem? Doesn't make sense.

BTW, it might be worth knowing what filesystem the OP is using. In my
case it's ext3.


FWIW, the OP is probably happy to have his problem solved. Chances are he
is running ext3 as most people take the default.


What would be more interesting would be to know what file system you are
running.


Never say Never.... :-)

--
fedora-list mailing list
fedora-list@redhat.com
To unsubscribe: https://www.redhat.com/mailman/listinfo/fedora-list
 

Thread Tools




All times are GMT. The time now is 11:21 AM.

VBulletin, Copyright ©2000 - 2014, Jelsoft Enterprises Ltd.
Content Relevant URLs by vBSEO ©2007, Crawlability, Inc.
Copyright 2007 - 2008, www.linux-archive.org