No space left on device - bug?

08-08-2011, 03:39 PM

Hi

I am been noticing this odd error in my ubunut 11.04 64bit.

iqbala@ghar-iqbala:~$ cat file
this is a file
iqbala@ghar-iqbala:~$ tail file
this is a file
iqbala@ghar-iqbala:~$ tail -f file
this is a file
tail: cannot watch `file': No space left on device

I do not have any resource issue. Any suggestion what could be causing
it? It clears for few mins followed by a reboot. And then re-appears.

iqbala@ghar-iqbala:~$ top

top - 11:34:42 up 1:36, 5 users, load average: 0.13, 0.24, 0.27
Tasks: 187 total, 4 running, 182 sleeping, 0 stopped, 1 zombie
Cpu(s): 10.6%us, 11.3%sy, 0.7%ni, 76.2%id, 1.2%wa, 0.0%hi, 0.0%si, 0.0%st
Mem: 8094272k total, 7857888k used, 236384k free, 233272k buffers
Swap: 8284156k total, 20k used, 8284136k free, 3797280k cached

iqbala@ghar-iqbala:~$ df -h
Filesystem Size Used Avail Use% Mounted on
/dev/sda1 680G 313G 334G 49% /
none 3.9G 696K 3.9G 1% /dev
none 3.9G 8.3M 3.9G 1% /dev/shm
none 3.9G 120K 3.9G 1% /var/run
none 3.9G 0 3.9G 0% /var/lock
/home/iqbala/.Private
680G 313G 334G 49% /home/iqbala

iqbala@ghar-iqbala:~$ uname -a
Linux ghar-iqbala 2.6.38-10-generic #46-Ubuntu SMP Tue Jun 28 15:07:17
UTC 2011 x86_64 x86_64 x86_64 GNU/Linux

iqbala@ghar-iqbala:~$ lsb_release -a
LSB Version: core-2.0-amd64:core-2.0-noarch:core-3.0-amd64:core-3.0-noarch:core-3.1-amd64:core-3.1-noarch:core-3.2-amd64:core-3.2-noarch:core-4.0-amd64:core-4.0-noarch
Distributor ID: Ubuntu
Description: Ubuntu 11.04
Release: 11.04
Codename: natty

--
Asif Iqbal
PGP Key: 0xE62693C5 KeyServer: pgp.mit.edu
A: Because it messes up the order in which people normally read text.
Q: Why is top-posting such a bad thing?

--
ubuntu-users mailing list
ubuntu-users@lists.ubuntu.com
Modify settings or unsubscribe at: https://lists.ubuntu.com/mailman/listinfo/ubuntu-users

08-08-2011, 03:58 PM

On 8 August 2011 16:39, Asif Iqbal <vadud3@gmail.com> wrote:
> Hi
>
> I am been noticing this odd error in my ubunut 11.04 64bit.
>
> iqbala@ghar-iqbala:~$ cat file
> this is a file
> iqbala@ghar-iqbala:~$ tail file
> this is a file
> iqbala@ghar-iqbala:~$ tail -f file
> this is a file
> tail: cannot watch `file': No space left on device

Could this be relevant?
https://bugs.launchpad.net/ubuntu/+source/coreutils/+bug/700958 tail
-f gives misleading error message when inotify limit is reached

Otherwise googling for
tail no space left on device
throws up a number of links that might help

Colin

--
ubuntu-users mailing list
ubuntu-users@lists.ubuntu.com
Modify settings or unsubscribe at: https://lists.ubuntu.com/mailman/listinfo/ubuntu-users

08-08-2011, 11:25 PM

On Mon, Aug 8, 2011 at 11:58 AM, Colin Law <clanlaw@googlemail.com> wrote:
> On 8 August 2011 16:39, Asif Iqbal <vadud3@gmail.com> wrote:
>> Hi
>>
>> I am been noticing this odd error in my ubunut 11.04 64bit.
>>
>> iqbala@ghar-iqbala:~$ cat file
>> this is a file
>> iqbala@ghar-iqbala:~$ tail file
>> this is a file
>> iqbala@ghar-iqbala:~$ tail -f file
>> this is a file
>> tail: cannot watch `file': No space left on device
>
> Could this be relevant?
> https://bugs.launchpad.net/ubuntu/+source/coreutils/+bug/700958 *tail
> -f gives misleading error message when inotify limit is reached

yep exact same issue

# strace tail -f file
...
inotify_init() = 4
inotify_add_watch(4, "file",
IN_MODIFY|IN_ATTRIB|IN_DELETE_SELF|IN_MOVE_SELF) = -1 ENOSPC (No space
left on device)
open("/usr/share/locale/locale.alias", O_RDONLY) = 5
....

looks like I am using coreutils 8.5
iqbala@ghar-iqbala:~$ apt-cache policy coreutils
coreutils:
Installed: 8.5-1ubuntu6
Candidate: 8.5-1ubuntu6
Version table:
*** 8.5-1ubuntu6 0
500 http://archive.ubuntu.com/ubuntu/ natty/main amd64 Packages
100 /var/lib/dpkg/status

So I upgraded to latest coreutils using ppa
iqbala@ghar-iqbala:~$ sudo add-apt-repository ppa:hggdh2/ppa
iqbala@ghar-iqbala:~$ sudo apt-get update
iqbala@ghar-iqbala:~$ sudo apt-get install coreutils
iqbala@ghar-iqbala:~$ sudo apt-cache policy coreutils
coreutils:
Installed: 8.12.65-5c5576~20110530~ppa1
Candidate: 8.12.65-5c5576~20110530~ppa1
Version table:
*** 8.12.65-5c5576~20110530~ppa1 0
500 http://ppa.launchpad.net/hggdh2/ppa/ubuntu/ natty/main
amd64 Packages
100 /var/lib/dpkg/status
8.5-1ubuntu6 0
500 http://archive.ubuntu.com/ubuntu/ natty/main amd64 Packages

Now I truly get the inotify message
iqbala@ghar-iqbala:~$ tail -f file
this is a file
tail: inotify resources exhausted
tail: inotify cannot be used, reverting to polling

So why am I still getting this
iqbala@ghar-iqbala:~$ cat /proc/sys/fs/inotify/max_user_watches
8192

what is using inotify? how do I fix it?

>
> Otherwise googling for
> tail no space left on device
> throws up a number of links that might help
>
> Colin
>
> --
> ubuntu-users mailing list
> ubuntu-users@lists.ubuntu.com
> Modify settings or unsubscribe at: https://lists.ubuntu.com/mailman/listinfo/ubuntu-users
>

--
Asif Iqbal
PGP Key: 0xE62693C5 KeyServer: pgp.mit.edu
A: Because it messes up the order in which people normally read text.
Q: Why is top-posting such a bad thing?

--
ubuntu-users mailing list
ubuntu-users@lists.ubuntu.com
Modify settings or unsubscribe at: https://lists.ubuntu.com/mailman/listinfo/ubuntu-users

08-08-2011, 11:34 PM

On Mon, Aug 8, 2011 at 7:25 PM, Asif Iqbal <vadud3@gmail.com> wrote:
> On Mon, Aug 8, 2011 at 11:58 AM, Colin Law <clanlaw@googlemail.com> wrote:
>> On 8 August 2011 16:39, Asif Iqbal <vadud3@gmail.com> wrote:
>>> Hi
>>>
>>> I am been noticing this odd error in my ubunut 11.04 64bit.
>>>
>>> iqbala@ghar-iqbala:~$ cat file
>>> this is a file
>>> iqbala@ghar-iqbala:~$ tail file
>>> this is a file
>>> iqbala@ghar-iqbala:~$ tail -f file
>>> this is a file
>>> tail: cannot watch `file': No space left on device
>>
>> Could this be relevant?
>> https://bugs.launchpad.net/ubuntu/+source/coreutils/+bug/700958 *tail
>> -f gives misleading error message when inotify limit is reached
>
> yep exact same issue
>
> # strace tail -f file
> ...
> inotify_init() * * * * * * * * * * * * *= 4
> inotify_add_watch(4, "file",
> IN_MODIFY|IN_ATTRIB|IN_DELETE_SELF|IN_MOVE_SELF) = -1 ENOSPC (No space
> left on device)
> open("/usr/share/locale/locale.alias", O_RDONLY) = 5
> ....
>
> looks like I am using coreutils 8.5
> iqbala@ghar-iqbala:~$ apt-cache policy coreutils
> coreutils:
> *Installed: 8.5-1ubuntu6
> *Candidate: 8.5-1ubuntu6
> *Version table:
> **** 8.5-1ubuntu6 0
> * * * *500 http://archive.ubuntu.com/ubuntu/ natty/main amd64 Packages
> * * * *100 /var/lib/dpkg/status
>
> So I upgraded to latest coreutils using ppa
> iqbala@ghar-iqbala:~$ sudo add-apt-repository ppa:hggdh2/ppa
> iqbala@ghar-iqbala:~$ sudo apt-get update
> iqbala@ghar-iqbala:~$ sudo apt-get install coreutils
> iqbala@ghar-iqbala:~$ sudo apt-cache policy coreutils
> coreutils:
> *Installed: 8.12.65-5c5576~20110530~ppa1
> *Candidate: 8.12.65-5c5576~20110530~ppa1
> *Version table:
> **** 8.12.65-5c5576~20110530~ppa1 0
> * * * *500 http://ppa.launchpad.net/hggdh2/ppa/ubuntu/ natty/main
> amd64 Packages
> * * * *100 /var/lib/dpkg/status
> * * 8.5-1ubuntu6 0
> * * * *500 http://archive.ubuntu.com/ubuntu/ natty/main amd64 Packages
>
> Now I truly get the inotify message
> iqbala@ghar-iqbala:~$ tail -f file
> this is a file
> tail: inotify resources exhausted
> tail: inotify cannot be used, reverting to polling
>
> So why am I still getting this
> iqbala@ghar-iqbala:~$ cat /proc/sys/fs/inotify/max_user_watches
> 8192
>
> what is using inotify? how do I fix it?

I followed this
http://unix.stackexchange.com/questions/15509/whos-consuming-my-inotify-resources
and found only 25 process is using inotify. far less than 8192

>
>
>
>
>
>
>
>>
>> Otherwise googling for
>> tail no space left on device
>> throws up a number of links that might help
>>
>> Colin
>>
>> --
>> ubuntu-users mailing list
>> ubuntu-users@lists.ubuntu.com
>> Modify settings or unsubscribe at: https://lists.ubuntu.com/mailman/listinfo/ubuntu-users
>>
>
>
>
> --
> Asif Iqbal
> PGP Key: 0xE62693C5 KeyServer: pgp.mit.edu
> A: Because it messes up the order in which people normally read text.
> Q: Why is top-posting such a bad thing?
>

--
Asif Iqbal
PGP Key: 0xE62693C5 KeyServer: pgp.mit.edu
A: Because it messes up the order in which people normally read text.
Q: Why is top-posting such a bad thing?

--
ubuntu-users mailing list
ubuntu-users@lists.ubuntu.com
Modify settings or unsubscribe at: https://lists.ubuntu.com/mailman/listinfo/ubuntu-users