disk usage

06-29-2010, 11:13 AM

Hi,

I see df -h and du -hs difference.

root@test:/# du -hs * | grep G
2.4G*** home
du: cannot access `proc/1027/task/1027/fd/4': No such file or directory
du: cannot access `proc/1027/task/1027/fdinfo/4': No such file or directory

du: cannot access `proc/1027/fd/4': No such file or directory
du: cannot access `proc/1027/fdinfo/4': No such file or directory
9.5G*** usr
root@test:~# df -h
Filesystem*********** Size* Used Avail Use% Mounted on

/dev/sda1************* 70G** 55G** 15G* 79% /
varrun*************** 3.6G** 88K* 3.6G** 1% /var/run
udev***************** 3.6G** 32K* 3.6G** 1% /dev
/dev/sda5************ 464M** 28M* 412M** 7% /boot
root@test:~#

Please suggest whats eating up 43.1 GB disk space

Thanks

Kaushal

--
ubuntu-server mailing list
ubuntu-server@lists.ubuntu.com
https://lists.ubuntu.com/mailman/listinfo/ubuntu-server
More info: https://wiki.ubuntu.com/ServerTeam

06-29-2010, 11:33 AM

Hi,please be aware of /proc and its functionality.Your RAM is represented in /proc and I think it's counted within when you apply "du" on it.You really should atleast exclude /proc from doing "du".

http://www.linux.com/archive/feature/126718

Bye.
On Tue, Jun 29, 2010 at 13:13, Kaushal Shriyan <kaushalshriyan@gmail.com> wrote:

Hi,

I see df -h and du -hs difference.

root@test:/# du -hs * | grep G
2.4G*** home
du: cannot access `proc/1027/task/1027/fd/4': No such file or directory

du: cannot access `proc/1027/task/1027/fdinfo/4': No such file or directory

du: cannot access `proc/1027/fd/4': No such file or directory
du: cannot access `proc/1027/fdinfo/4': No such file or directory
9.5G*** usr
root@test:~# df -h
Filesystem*********** Size* Used Avail Use% Mounted on

/dev/sda1************* 70G** 55G** 15G* 79% /
varrun*************** 3.6G** 88K* 3.6G** 1% /var/run
udev***************** 3.6G** 32K* 3.6G** 1% /dev
/dev/sda5************ 464M** 28M* 412M** 7% /boot
root@test:~#

Please suggest whats eating up 43.1 GB disk space

Thanks

Kaushal

--

ubuntu-server mailing list

ubuntu-server@lists.ubuntu.com

https://lists.ubuntu.com/mailman/listinfo/ubuntu-server

More info: https://wiki.ubuntu.com/ServerTeam

--
To be or not to be -- Shakespeare | To do is to be -- Nietzsche | To be is to do -- Sartre | Do be do be do -- Sinatra

--
ubuntu-server mailing list
ubuntu-server@lists.ubuntu.com
https://lists.ubuntu.com/mailman/listinfo/ubuntu-server
More info: https://wiki.ubuntu.com/ServerTeam

06-29-2010, 11:35 AM

Ah well, /dev should be avoided too.

On Tue, Jun 29, 2010 at 13:33, Benjamin Griese <der.darude@gmail.com> wrote:

Hi,please be aware of /proc and its functionality.Your RAM is represented in /proc and I think it's counted within when you apply "du" on it.You really should atleast exclude /proc from doing "du".

http://www.linux.com/archive/feature/126718

Bye.

On Tue, Jun 29, 2010 at 13:13, Kaushal Shriyan <kaushalshriyan@gmail.com> wrote:

Hi,

I see df -h and du -hs difference.

root@test:/# du -hs * | grep G

2.4G*** home
du: cannot access `proc/1027/task/1027/fd/4': No such file or directory

du: cannot access `proc/1027/task/1027/fdinfo/4': No such file or directory

du: cannot access `proc/1027/fd/4': No such file or directory
du: cannot access `proc/1027/fdinfo/4': No such file or directory
9.5G*** usr
root@test:~# df -h
Filesystem*********** Size* Used Avail Use% Mounted on

/dev/sda1************* 70G** 55G** 15G* 79% /
varrun*************** 3.6G** 88K* 3.6G** 1% /var/run
udev***************** 3.6G** 32K* 3.6G** 1% /dev
/dev/sda5************ 464M** 28M* 412M** 7% /boot
root@test:~#

Please suggest whats eating up 43.1 GB disk space

Thanks

Kaushal

--

ubuntu-server mailing list

ubuntu-server@lists.ubuntu.com

https://lists.ubuntu.com/mailman/listinfo/ubuntu-server

More info: https://wiki.ubuntu.com/ServerTeam

--
To be or not to be -- Shakespeare | To do is to be -- Nietzsche | To be is to do -- Sartre | Do be do be do -- Sinatra

--
To be or not to be -- Shakespeare | To do is to be -- Nietzsche | To be is to do -- Sartre | Do be do be do -- Sinatra

--
ubuntu-server mailing list
ubuntu-server@lists.ubuntu.com
https://lists.ubuntu.com/mailman/listinfo/ubuntu-server
More info: https://wiki.ubuntu.com/ServerTeam

06-29-2010, 11:35 AM

Answering Kaushal Shriyan <kaushalshriyan@gmail.com>
(Tue, 29 Jun 2010 16:43:53 +0530):

Hi,

> Please suggest whats eating up 43.1 GB disk space

do you have any directories starting with a dot?

What result do you get?
du -hs /* /.[^.]*

Which filesystem are you running? Maybe you have an unbelievable count
of small files which are smaller than the cluster size of your disk.

Cheers

Dirk

--
Kontakt: http://d5e.org/contact
Blog: http://dirk.deimeke.net/
Podcast: http://deimhart.net/

--
ubuntu-server mailing list
ubuntu-server@lists.ubuntu.com
https://lists.ubuntu.com/mailman/listinfo/ubuntu-server
More info: https://wiki.ubuntu.com/ServerTeam

06-29-2010, 11:51 AM

On Tue, Jun 29, 2010 at 5:05 PM, Dirk Deimeke <dirk@deimeke.net> wrote:

Answering Kaushal Shriyan <kaushalshriyan@gmail.com>

(Tue, 29 Jun 2010 16:43:53 +0530):

Hi,

> Please suggest whats eating up 43.1 GB disk space

do you have any directories starting with a dot?

What result do you get?

du -hs /* /.[^.]*

Which filesystem are you running? Maybe you have an unbelievable count

of small files which are smaller than the cluster size of your disk.

Cheers

Dirk

--

Kontakt: http://d5e.org/contact

Blog: * *http://dirk.deimeke.net/

Podcast: http://deimhart.net/

--

ubuntu-server mailing list

ubuntu-server@lists.ubuntu.com

https://lists.ubuntu.com/mailman/listinfo/ubuntu-server

More info: https://wiki.ubuntu.com/ServerTeam

Hi Dirk,

/dev/sda1***** xfs**** 70G** 55G** 15G* 79% /

0*** /backuplogdir
0*** /backuplogdir_oct09
6.1M*** /bin
18M*** /boot
36K*** /dev
7.7M*** /etc
2.4G*** /home

0*** /initrd
0*** /initrd.img
143M*** /lib
0*** /lib64
0*** /log-test
0*** /media
0*** /mnt
0*** /opt
du: cannot access `/proc/2026/task/2026/fd/4': No such file or directory
du: cannot access `/proc/2026/task/2026/fdinfo/4': No such file or directory

du: cannot access `/proc/2026/fd/4': No such file or directory
du: cannot access `/proc/2026/fdinfo/4': No such file or directory
0*** /proc
0*** /REC-test
60K*** /root
7.8M*** /sbin
0*** /srv

0*** /sys
9.8G*** /usr
392M*** /var
0*** /vmlinuz
du: cannot access `/.[^.]*': No such file or directory

Please suggest further.

Thanks and Regards

Kaushal

--
ubuntu-server mailing list
ubuntu-server@lists.ubuntu.com
https://lists.ubuntu.com/mailman/listinfo/ubuntu-server
More info: https://wiki.ubuntu.com/ServerTeam

06-29-2010, 12:10 PM

Answering Kaushal Shriyan <kaushalshriyan@gmail.com>
(Tue, 29 Jun 2010 17:21:55 +0530):

Hi Kaushal

> Please suggest further.

I am not on one of my Ubuntu boxes, so please check the commands carefully,

What is the result of "xfs_info /"?

The other point to check:

If someone uses a large file. And the file is deleted. The space is
only freed, when the usage stops.

Cheers

Dirk

--
Kontakt: http://d5e.org/contact
Blog: http://dirk.deimeke.net/
Podcast: http://deimhart.net/

--
ubuntu-server mailing list
ubuntu-server@lists.ubuntu.com
https://lists.ubuntu.com/mailman/listinfo/ubuntu-server
More info: https://wiki.ubuntu.com/ServerTeam

06-29-2010, 12:13 PM

On Tue, Jun 29, 2010 at 5:40 PM, Dirk Deimeke <dirk@deimeke.net> wrote:

Answering Kaushal Shriyan <kaushalshriyan@gmail.com>

(Tue, 29 Jun 2010 17:21:55 +0530):

Hi Kaushal

Please suggest further.

I am not on one of my Ubuntu boxes, so please check the commands carefully,

What is the result of "xfs_info /"?

The other point to check:

If someone uses a large file. And the file is deleted. The space is only freed, when the usage stops.

Hi Dirk,
*
root@test:/# xfs_info /
meta-data="" isize=256*** agcount=16, agsize=1137225 blks
******** =********************** sectsz=512** attr=1

data**** =********************** bsize=4096** blocks=18195600, imaxpct=25
******** =********************** sunit=0***** swidth=0 blks, unwritten=1
naming** =version 2************* bsize=4096*
log***** =internal************** bsize=4096** blocks=8884, version=1

******** =********************** sectsz=512** sunit=0 blks, lazy-count=0
realtime =none****************** extsz=4096** blocks=0, rtextents=0
root@test:/#

Please suggest further

Thanks,

Kaushal

--
ubuntu-server mailing list
ubuntu-server@lists.ubuntu.com
https://lists.ubuntu.com/mailman/listinfo/ubuntu-server
More info: https://wiki.ubuntu.com/ServerTeam

06-29-2010, 12:48 PM

root@venus:~# du -bs /* | sort -nr | less

You can ignore /proc and /dev.

My top 2 are:
43667994365**** /home
30759426971**** /partimag

I had forgotten about partimag. I can move the contents to an external hard drive if I want to keep them.* As expected /home it the real culprit and between the two users, I am the culprit and among my directories - Videos is the largest by far and among those, my globaloneness shorts take up 25 GB. I have these backed up in at least two places so when I get into trouble, I know where to look.

On Tue, Jun 29, 2010 at 8:13 AM, Kaushal Shriyan <kaushalshriyan@gmail.com> wrote:

On Tue, Jun 29, 2010 at 5:40 PM, Dirk Deimeke <dirk@deimeke.net> wrote:

Answering Kaushal Shriyan <kaushalshriyan@gmail.com>

(Tue, 29 Jun 2010 17:21:55 +0530):

Hi Kaushal

Please suggest further.

I am not on one of my Ubuntu boxes, so please check the commands carefully,

What is the result of "xfs_info /"?

The other point to check:

If someone uses a large file. And the file is deleted. The space is only freed, when the usage stops.

Hi Dirk,
*
root@test:/# xfs_info /
meta-data="" isize=256*** agcount=16, agsize=1137225 blks
******** =********************** sectsz=512** attr=1

data**** =********************** bsize=4096** blocks=18195600, imaxpct=25
******** =********************** sunit=0***** swidth=0 blks, unwritten=1
naming** =version 2************* bsize=4096*
log***** =internal************** bsize=4096** blocks=8884, version=1

******** =********************** sectsz=512** sunit=0 blks, lazy-count=0
realtime =none****************** extsz=4096** blocks=0, rtextents=0
root@test:/#

Please suggest further

Thanks,

Kaushal

--

ubuntu-server mailing list

ubuntu-server@lists.ubuntu.com

https://lists.ubuntu.com/mailman/listinfo/ubuntu-server

More info: https://wiki.ubuntu.com/ServerTeam

--
Rev. Jim Tarvid, PCA
Galax, Virginia
http://ls.net

http://drupal.ls.net
http://crossleft.org

--
ubuntu-server mailing list
ubuntu-server@lists.ubuntu.com
https://lists.ubuntu.com/mailman/listinfo/ubuntu-server
More info: https://wiki.ubuntu.com/ServerTeam

06-29-2010, 12:49 PM

Am Di, 29.06.2010, 14:13 schrieb Kaushal Shriyan:
...
>>
>> Hi Dirk,
>
> root@test:/# xfs_info /
> meta-data=/dev/disk/by-uuid/48a2bff8-240d-49d9-b19d-ad287ec0c613
> isize=256 agcount=16, agsize=1137225 blks
> = sectsz=512 attr=1
> data = bsize=4096 blocks=18195600, imaxpct=25
> = sunit=0 swidth=0 blks, unwritten=1
> naming =version 2 bsize=4096
> log =internal bsize=4096 blocks=8884, version=1
> = sectsz=512 sunit=0 blks, lazy-count=0
> realtime =none extsz=4096 blocks=0, rtextents=0
> root@test:/#
>
> Please suggest further
>

Hi Kaushal,

not so easy to suggest any further steps and some more information would
be handy. What does your server do? Like Dirk suggested are you having a
lot of small files like you have it on a news server? Is it real hardware
or a virtual machine you are running? What programs are running? Maybe
important.

Have you rebootet your server? After a reboot do you have the same "lost"
43,1GB?

I would try and use a command line to find all files bigger than 1 GB and
see what is there. Then I would generate a large tarball of your server
and copy this tarball to another server. Then I would extract this tarball
on the other computer and analyze it carefully! Like does it needs the
same space as on the original server.

Hope this helps,

Michael

--
ubuntu-server mailing list
ubuntu-server@lists.ubuntu.com
https://lists.ubuntu.com/mailman/listinfo/ubuntu-server
More info: https://wiki.ubuntu.com/ServerTeam

06-29-2010, 01:08 PM

Answering Kaushal Shriyan <kaushalshriyan@gmail.com>
(Tue, 29 Jun 2010 17:43:52 +0530):

Hi,

are you sure, no one (program, daemon, human being) is using any
deleted files?

If you are not sure consider a shutdown of all services or eventually
a complete reboot.

> Please suggest further

Your blocksize is 4096, so regardless of the size of the file it will
use at minimum 4k.

If you want to take the effort you can check for wasted space.

For example
find /home /usr -size -4096c | wc -l
shows you a count of files which are smaller than 4096 bytes.

Cheers

Dirk

--
Kontakt: http://d5e.org/contact
Blog: http://dirk.deimeke.net/
Podcast: http://deimhart.net/

--
ubuntu-server mailing list
ubuntu-server@lists.ubuntu.com
https://lists.ubuntu.com/mailman/listinfo/ubuntu-server
More info: https://wiki.ubuntu.com/ServerTeam