On 2012-05-30 12:08:29 +0200, Josselin Mouette wrote:
> Le samedi 26 mai 2012 Ã* 23:02 +0200, Carlos Alberto Lopez Perez a
> écrit :
> > With "tmpfs on /tmp" you are breaking many applications that assume that
> > they have enough space to write on /tmp like the flash player ( see
> > Debian bug #666096 ) or cdrecord software ( see #665634 ).
>
> Seriously, this is madness. You can’t expect to have “enough� space on
> *any* filesystem.

I think that the point is that in general, there is more space on
the local disk than on some tmpfs. What I mean is that if /tmp is
on the right partition on the disk, it will have more space than
any reasonable tmpfs.

--
Vincent Lefèvre <vincent@vinc17.net> - Web: <http://www.vinc17.net/>
100% accessible validated (X)HTML - Blog: <http://www.vinc17.net/blog/>
Work: CR INRIA - computer arithmetic / AriC project (LIP, ENS-Lyon)


--
To UNSUBSCRIBE, email to debian-devel-REQUEST@lists.debian.org
with a subject of "unsubscribe". Trouble? Contact listmaster@lists.debian.org
Archive: 20120530113506.GB24019@xvii.vinc17.org">http://lists.debian.org/20120530113506.GB24019@xvii.vinc17.org

Vincent Lefevre <vincent@vinc17.net> writes:
> On 2012-05-30 12:08:29 +0200, Josselin Mouette wrote:
>> Le samedi 26 mai 2012 Ã* 23:02 +0200, Carlos Alberto Lopez Perez a
>> écrit :
>> > With "tmpfs on /tmp" you are breaking many applications that assume that
>> > they have enough space to write on /tmp like the flash player ( see
>> > Debian bug #666096 ) or cdrecord software ( see #665634 ).
>>
>> Seriously, this is madness. You can’t expect to have “enough� space on
>> *any* filesystem.
>
> I think that the point is that in general, there is more space on
> the local disk than on some tmpfs. What I mean is that if /tmp is
> on the right partition on the disk, it will have more space than
> any reasonable tmpfs.

Does that make any difference at all? If an application is unable to
handle the out-of-space condition, then it will be unable to handle the
out-of-space condition no matter how big the file system is. Increasing
the file system size is futile. Fix the bug in the application instead.


Bjørn


--
To UNSUBSCRIBE, email to debian-devel-REQUEST@lists.debian.org
with a subject of "unsubscribe". Trouble? Contact listmaster@lists.debian.org
Archive: 87fwah96n9.fsf@nemi.mork.no">http://lists.debian.org/87fwah96n9.fsf@nemi.mork.no

On Wed, May 30, 2012 at 02:48:26PM +0200, Bjørn Mork wrote:
> Does that make any difference at all? If an application is unable to
> handle the out-of-space condition, then it will be unable to handle the
> out-of-space condition no matter how big the file system is. Increasing
> the file system size is futile. Fix the bug in the application instead.

I think this is totally tangential to the issue. If the effect of moving
to tmpfs is - practically - reducing the size of /tmp, then it is bringing
into range these size issues that may have not been hit by as many people
before. Yes the issues existed already, but how seriously we should treat
the issue should be a product of many factors, including the likelyhood of
the problem being hit.


--
To UNSUBSCRIBE, email to debian-devel-REQUEST@lists.debian.org
with a subject of "unsubscribe". Trouble? Contact listmaster@lists.debian.org
Archive: 20120530132203.GC32707@debian">http://lists.debian.org/20120530132203.GC32707@debian

On Wed, May 30, 2012 at 02:48:26PM +0200, Bjørn Mork wrote:
> Vincent Lefevre <vincent@vinc17.net> writes:
> > On 2012-05-30 12:08:29 +0200, Josselin Mouette wrote:
> >> Le samedi 26 mai 2012 Ã* 23:02 +0200, Carlos Alberto Lopez Perez a
> >> écrit :
> >> > With "tmpfs on /tmp" you are breaking many applications that assume that
> >> > they have enough space to write on /tmp like the flash player ( see
> >> > Debian bug #666096 ) or cdrecord software ( see #665634 ).

> >> Seriously, this is madness. You can’t expect to have “enough� space on
> >> *any* filesystem.

> > I think that the point is that in general, there is more space on
> > the local disk than on some tmpfs. What I mean is that if /tmp is
> > on the right partition on the disk, it will have more space than
> > any reasonable tmpfs.

> Does that make any difference at all? If an application is unable to
> handle the out-of-space condition, then it will be unable to handle the
> out-of-space condition no matter how big the file system is. Increasing
> the file system size is futile. Fix the bug in the application instead.

The problem is not whether applications gracefully handle ENOSPC. The
problem is whether we as a distribution are causing users to hit ENOSPC when
there's no justifiable reason for it.

Even if every application on the system handles ENOSPC gracefully, it's
*still* a bug if I have 100GB free on my disk and am hitting ENOSPC due to
decisions that Debian, and not me, has made regarding filesystem layout.

--
Steve Langasek Give me a lever long enough and a Free OS
Debian Developer to set it on, and I can move the world.
Ubuntu Developer http://www.debian.org/
slangasek@ubuntu.com vorlon@debian.org

Henrique de Moraes Holschuh <hmh@debian.org> writes:

> On Fri, 25 May 2012, Thomas Goirand wrote:
>> for small files, and in that case, it's faster. In reality, it's
>> not that much faster, thanks to Linux caching of the filesystem,
>
> Under heavy filesystem IO load, yes it is. By several orders of magnitude.

Even without load it is much faster because fsync() becomes a NOP. Disk
based filesystems add sequenze points where you have to wait for the
disk to catch up before continuing while tmpfs does not.

It also saves on wear of the disk if the files are small and short
lived, like temp files for gcc. No point writing the file to disk if it
gets deleted 10s later.

>> the point. Maybe we should add a /small-files-on-tmpfs (choose
>> a better name, of course...) folder so that apps know what to do,
>> that'd be a lot more graceful than just switching to whole /tmp
>> to tmpfs without any app knowing about it.
>
> Nice idea, but it would be worthless.
>
> In fact it is the other way. We have /var/tmp for the large file since
> about forever, and important platforms that have /tmp in memory since the
> early 2000's (Solaris)....
>
> And that STILL wasn't enough for people to not screw it up and dump large
> files in /tmp.

There is also no problem with having large files in tmpfs. Only
requirement is that you make tmpfs large enough and add enough ram
and/or swap to cope with it.

All the complaints about /tmp as tmpfs come down to one simple issue:
The size of the tmpfs isn't chosen well. It would be more constructive
to find a better heuristic for the size there.

MfG
Goswin


--
To UNSUBSCRIBE, email to debian-devel-REQUEST@lists.debian.org
with a subject of "unsubscribe". Trouble? Contact listmaster@lists.debian.org
Archive: 87lik7mhl9.fsf@frosties.localnet">http://lists.debian.org/87lik7mhl9.fsf@frosties.localnet

Carlos Alberto Lopez Perez <clopez@igalia.com> writes:

> On 25/05/12 12:14, Henrique de Moraes Holschuh wrote:
>> On Fri, 25 May 2012, Thomas Goirand wrote:
>>> for small files, and in that case, it's faster. In reality, it's
>>> not that much faster, thanks to Linux caching of the filesystem,
>>
>> Under heavy filesystem IO load, yes it is. By several orders of magnitude.
>>
> This is a corner case.
>
> Defaults should be sane for most of the cases, and not for corner cases.
> Also defaults should prioritize stability (and non-breakage) over
> performance.
>
> With "tmpfs on /tmp" you are breaking many applications that assume that
> they have enough space to write on /tmp like the flash player ( see
> Debian bug #666096 ) or cdrecord software ( see #665634 ).

And again, tmpfs isn't breaking it, only the size of /tmp. A 1GB real
/tmp filesystem breaks them just like a 1GB tmpfs breaks them.

So make /tmp large enough. Improve the heuristic that defines the size
for tmp.

> The FHS [2] does not define *nothing* about the size of /tmp or
> /var/tmp. It *only* says that programs using files under /tmp must not
> assume that this files are preserved between invocations of the program
>
>
> So please, *stop* saying that /tmp should be only for small files
> because this is simply *not* *true*. It is *not* defined on any standard
> that files on /tmp should be small. Period.

It is also *not* defined on any standard that files on /tmp may be
big. Period.

Sorry, that argument simply cuts both ways.

An application that stores files in /tmp without checking size and/or
handling ENOSPC properly is just broken and needs to be fixed
regardless.

MfG
Goswin


--
To UNSUBSCRIBE, email to debian-devel-REQUEST@lists.debian.org
with a subject of "unsubscribe". Trouble? Contact listmaster@lists.debian.org
Archive: 87hauvmgwm.fsf@frosties.localnet">http://lists.debian.org/87hauvmgwm.fsf@frosties.localnet

Nikolaus Rath <Nikolaus@rath.org> writes:

> Thomas Goirand <zigo@debian.org> writes:
>> On 05/25/2012 05:33 PM, Mehdi Dogguy wrote:
>>>> What if we're installing Debian on a very small system, and that we
>>>> need operations with big files in /tmp?
>>>>
>>>
>>> Increase your swap?
>>
>> So, in this case, we will have the following scenario:
>> - An app writes in /tmp
>> - There's not enough space, so the system starts swapping,
>> including some apps.

Which happens regardless wether tmp is tmpfs or a real filesystem. The
more IO there is the more likely some app gets swapped out.

>> - The file gets written to /tmp, then gets read
>> - Finally, the file gets deleted

With tmps that instantly frees up all the memory and swap used by the
file. With a real FS the file data remains in the dirty cache until such
a time as the disk has cought up with writing it all and then it is
thrown away. So potentially memory is freed up much later.

>> - Then we have randomly very sloppy reaction of apps
>> that were swapped out so that the file could be written
>> in /tmp.

Which, without tmpfs, then has to additionaly first wait for the dirty
cached data to be written out causing huge delays because you get two
seeks per page, 4k read/writes and no read-ahead.

> I believe tmpfs memory is swapped out preferentially, so your scenario
> doesn't have to play out like that. However, paging being a complex
> process, it's not impossible either. Is that something people are
> actually seeing? Because I haven't encountered this.

It happens. But that isn't to say it doesn't equally (or worse) happen
with a real FS.

Paging is a complex process and there are so many factors involved that
predicting the behaviour becomes pure guesswork. I would say both Thomas
and my scenario are equally likely to occur. No matter what the default
is there will be some users that hit the worst case.

MfG
Goswin


--
To UNSUBSCRIBE, email to debian-devel-REQUEST@lists.debian.org
with a subject of "unsubscribe". Trouble? Contact listmaster@lists.debian.org
Archive: 87d35jmgf9.fsf@frosties.localnet">http://lists.debian.org/87d35jmgf9.fsf@frosties.localnet

Vincent Danjean <vdanjean.ml@free.fr> writes:

> Le 25/05/2012 05:03, Russell Coker a écrit :
>> On Fri, 25 May 2012, Serge <sergemdev@gmail.com> wrote:
>>> Q: /tmp on tmpfs increases apps performance.
>>> A: What apps? Real apps don't write files during performance-critical
>>> operations. Even if they do, they write large files. And large files are
>>> written faster when they're written on real disk, rather then swapped
>>> out and slow down the entire system (see the "Who uses /tmp" part).
>>> The apps that can really benefit from tmpfs are too rare. And we're
>>> talking about default settings and most common cases.
>>
>> Any application which writes synchronously (through fsync(), fdatasync(), or
>> opening with O_SYNC) will get a massive performance benefit from using tmpfs.
>
> If some kind of sync is required by the application, I think this is
> because the application want to ensure the data are really written to
> the disk so that their state remains coherent even in case of crash.
> If the application is ok to have this kind of data written to
> tmpfs (ie in memory), I do not see the interest of using sync at
> first. Can someone shows me a valid use case of sync on tmpfs?
>
> Regards,
> Vince

You might also need to [fm]sync() to ensure the data written by one
application can be read by another, to ensure the state remains coherent
between multiple processes.

And don't forget that disk based filesystems add syncs internally to
ensure their own coherent state. Applications do get blocked by those
too.

MfG
Goswin


--
To UNSUBSCRIBE, email to debian-devel-REQUEST@lists.debian.org
with a subject of "unsubscribe". Trouble? Contact listmaster@lists.debian.org
Archive: 878vg7mg4h.fsf@frosties.localnet">http://lists.debian.org/878vg7mg4h.fsf@frosties.localnet

Salvo Tomaselli <tiposchi@tiscali.it> writes:

>> Because paging out a couple Gigabytes is veeeeeery different from
>> writing a couple Gigabytes to disk, of course.
>
> Yes because writing that on disk will only block the thread performing the
> write, not every thread that tries to allocate memory.

Wrong. The thread doing the write will just write to cache and not block
at all. That is until you run out of cache. And then any thread that
needs to allocate memory will block until such a time as some dirty
memory is written.

Now with multiple cores it becomes a bit more complex since then you
have seperate queues. So only one core might block anything needing
memory on that core.

If anyone wants to experience that then write out some GB of data over
NFS. After a short while processes will hang while others keep running.

MfG
Goswin


--
To UNSUBSCRIBE, email to debian-devel-REQUEST@lists.debian.org
with a subject of "unsubscribe". Trouble? Contact listmaster@lists.debian.org
Archive: 874nqvmfwu.fsf@frosties.localnet">http://lists.debian.org/874nqvmfwu.fsf@frosties.localnet

Carlos Alberto Lopez Perez <clopez@igalia.com> writes:

> On 25/05/12 12:20, Henrique de Moraes Holschuh wrote:
>> On Fri, 25 May 2012, Salvo Tomaselli wrote:
>>>> Because paging out a couple Gigabytes is veeeeeery different from
>>>> writing a couple Gigabytes to disk, of course.
>>>
>>> Yes because writing that on disk will only block the thread performing the
>>> write, not every thread that tries to allocate memory.
>>
>> What IO scheduler are you using? It must not be CFQ. CFQ will happly
>> screw it up and stall both readers and writers when the number of dirty
>> pages gets too large (which will happen if you write a gigabyte file to
>> disk).
>>
>> Either that, or you're cheating and your backend devices are simply "fast
>> enough" that it doesn't matter (fast RAID or fast SSD).
>>
>> The real question is: what does it better under CFQ+IO contention?
>> Several threads doing filesystem IO, or the swapper? Hint: it is not an
>> easy question to answer because it depends on the load, type of load,
>> and backend device.
>>
>
> I think that any user that has been using Linux for a while knows how
> ugly the things become when the systems starts swapping.
>
> When the system is swapping, the whole system will become so
> unresponsive while the swapping process is taking place that even your
> mouse pointer will stop moving.
>
> And this is *very* *very* annoying.
>
> This is so annoying, that I even had configured my laptop with a very
> low amount of swap, since I rather prefer the oom-killer to kill the
> application that is filling the ram than rather suffer this annoying
> situation.
>
> So please, don't argue about theoretical things about virtual memory or
> IO schedulers. If you are a desktop Linux user, you should know how ugly
> the things get when the system is swapping.

That is swapping out binaries or their data and needing to wait for it
to be swapped back in. The problem is the waiting for it to be swapped
in there, not that you are swapping.

With large data on tmpfs you will be swapping out lots of data but you
won't block waiting for stuff to be swapped back in in general. Only
something reading back the file will block, not your mouse cursor.

> I don't know the ultimate reason behind this ugly behaviour of Linux
> when the swapping process is happening, but I know this is real and it
> happens because I have experimented this situation myself more than a
> couple of times.

It's a matter of that gets swapped and linux choosing the wrong thing to
swap (far too often). There is some "bug" in linux that when you have
lots and lots of IO at some point the swap heuristics tip over and start
swapping apps and usefull data to create more cache space for
IO. Doesn't matter that you already have 3GB for cache, it still swaps
out your mouse cursor and then things go real bad.

MfG
Goswin


--
To UNSUBSCRIBE, email to debian-devel-REQUEST@lists.debian.org
with a subject of "unsubscribe". Trouble? Contact listmaster@lists.debian.org
Archive: 87zk8nl10x.fsf@frosties.localnet">http://lists.debian.org/87zk8nl10x.fsf@frosties.localnet