FAQ Search Today's Posts Mark Forums Read
» Video Reviews

» Linux Archive

Linux-archive is a website aiming to archive linux email lists and to make them easily accessible for linux users/developers.


» Sponsor

» Partners

» Sponsor

Go Back   Linux Archive > Debian > Debian Development

 
 
LinkBack Thread Tools
 
Old 11-13-2011, 10:04 AM
Lars Wirzenius
 
Default /tmp as tmpfs and consequence for imaging software

On Sun, Nov 13, 2011 at 06:30:27PM +0800, Thomas Goirand wrote:
> Agreed. Anyway, I'd be happy to have, by policy, some hard limits
> We can discuss of a tolerable size, like 100M? Anyway, anything
> bigger than 512 MB is obviously abusing /tmp, IMHO. But if we're
> to have /tmp using tmpfs by default, it becomes really important
> to have such policy.

Greetings from the real world.

There are systems where /tmp is on a real disk, and has terabytes
of free space, and it's totally OK to use most of that, temporarily.
I have one such system for running benchmarks of my backup program.
Having a Debian policy that programs are not allowed to use more
than 100 megabytes of temporary files would make my benchmarks
useless.

There are systems where /tmp is on a flash unit, and the total amount
of flash and RAM in the system is less than 100 megabytes. I have one,
it runs very well as a firewall. A Debian policy that requires a
program to be able to use at least 100 megs of temporary file space
would make it hard to use Debian on highly constrained systems.

A fixed policy is going to interact badly with real systems and
per-site decisions about, say, disk partitioning and provisioining
of RAM for various purposes.

The proper policy, IMHO, is that a) all software that uses temporary
files should obey TMPDIR if set (and fall back on /tmp if not)
and b) all software must deal with out-of-disk-space errors in a
sensible way (where the exact details may depend on the software).

It is then a sysadmin decision to setup /tmp or TMPDIR properly
so there's enough free space for temporary files for the software
they use. Debian should provide sensible defaults, but it's not
possible to pick defaults that are optimal for everyone, in
this situation. Given that MySQL, scientific software, and such
systems are mostly run on systems that have sysadmins, and desktops
are run by people who do not, it's sensible to favor the desktop
case by default.

--
Freedom-based blog/wiki/web hosting: http://www.branchable.com/
 
Old 11-13-2011, 10:31 AM
martin f krafft
 
Default /tmp as tmpfs and consequence for imaging software

also sprach Lars Wirzenius <liw@liw.fi> [2011.11.13.1204 +0100]:
> A fixed policy is going to interact badly with real systems and
> per-site decisions about, say, disk partitioning and provisioining
> of RAM for various purposes.
>
> The proper policy, IMHO, is that a) all software that uses temporary
> files should obey TMPDIR if set (and fall back on /tmp if not)
> and b) all software must deal with out-of-disk-space errors in a
> sensible way (where the exact details may depend on the software).

Fully agreed.

Please do not try to (ab)use policy to tell me how I have to
administer/manage my systems. Use policy to dictate how programs we
provide as part of The Debian System must behave.

I would welcome if TMPDIR-related bugs and bugs related to
temporary file handling would become release-critical.

I think it's great that Debian provides the flexibility to easily
make /tmp a tmpfs. However, I don't think that should be the
default (cf. RAMRUN, RAMLOCK).

--
.'`. martin f. krafft <madduck@d.o> Related projects:
: :' : proud Debian developer http://debiansystem.info
`. `'` http://people.debian.org/~madduck http://vcs-pkg.org
`- Debian - when you have better things to do than fixing systems

people with narrow minds usually have broad tongues.
 
Old 11-13-2011, 10:49 AM
Timo Juhani Lindfors
 
Default /tmp as tmpfs and consequence for imaging software

Lars Wirzenius <liw@liw.fi> writes:
> and b) all software must deal with out-of-disk-space errors in a
> sensible way (where the exact details may depend on the software).

That does not seem to be so easy:

http://www.gnu.org/ghm/2011/paris/slides/jim-meyering-goodbye-world.pdf


--
To UNSUBSCRIBE, email to debian-devel-REQUEST@lists.debian.org
with a subject of "unsubscribe". Trouble? Contact listmaster@lists.debian.org
Archive: 84obwgp7h7.fsf@sauna.l.org">http://lists.debian.org/84obwgp7h7.fsf@sauna.l.org
 
Old 11-13-2011, 11:12 AM
Thomas Goirand
 
Default /tmp as tmpfs and consequence for imaging software

On 11/13/2011 07:31 PM, martin f krafft wrote:
> also sprach Lars Wirzenius <liw@liw.fi> [2011.11.13.1204 +0100]:
>
>> A fixed policy is going to interact badly with real systems and
>> per-site decisions about, say, disk partitioning and provisioining
>> of RAM for various purposes.
>>
>> The proper policy, IMHO, is that a) all software that uses temporary
>> files should obey TMPDIR if set (and fall back on /tmp if not)
>> and b) all software must deal with out-of-disk-space errors in a
>> sensible way (where the exact details may depend on the software).
>>
> Fully agreed.
>
> Please do not try to (ab)use policy to tell me how I have to
> administer/manage my systems. Use policy to dictate how programs we
> provide as part of The Debian System must behave.
>
> I would welcome if TMPDIR-related bugs and bugs related to
> temporary file handling would become release-critical.
>
That was my though too (maybe badly expressed).
Of course, it's ok to use temporary space, and what
we need to enforce is a way to choose WHERE (even
if it was by other means than the TMPDIR, it would
be ok for me...).

Thomas


--
To UNSUBSCRIBE, email to debian-devel-REQUEST@lists.debian.org
with a subject of "unsubscribe". Trouble? Contact listmaster@lists.debian.org
Archive: 4EBFB44B.7050708@debian.org">http://lists.debian.org/4EBFB44B.7050708@debian.org
 
Old 11-13-2011, 11:36 AM
Bastien ROUCARIES
 
Default /tmp as tmpfs and consequence for imaging software

On Sun, Nov 13, 2011 at 12:04 PM, Lars Wirzenius <liw@liw.fi> wrote:
> On Sun, Nov 13, 2011 at 06:30:27PM +0800, Thomas Goirand wrote:
>> Agreed. Anyway, I'd be happy to have, by policy, some hard limits
>> We can discuss of a tolerable size, like 100M? Anyway, anything
>> bigger than 512 MB is obviously abusing /tmp, IMHO. But if we're
>> to have /tmp using tmpfs by default, it becomes really important
>> to have such policy.
>
> Greetings from the real world.
>
> There are systems where /tmp is on a real disk, and has terabytes
> of free space, and it's totally OK to use most of that, temporarily.
> I have one such system for running benchmarks of my backup program.
> Having a Debian policy that programs are not allowed to use more
> than 100 megabytes of temporary files would make my benchmarks
> useless.
>
> There are systems where /tmp is on a flash unit, and the total amount
> of flash and RAM in the system is less than 100 megabytes. I have one,
> it runs very well as a firewall. A Debian policy that requires a
> program to be able to use at least 100 megs of temporary file space
> would make it hard to use Debian on highly constrained systems.
>
> A fixed policy is going to interact badly with real systems and
> per-site decisions about, say, disk partitioning and provisioining
> of RAM for various purposes.
>
> The proper policy, IMHO, is that a) all software that uses temporary
> files should obey TMPDIR if set (and fall back on /tmp if not)
> and b) all software must deal with out-of-disk-space errors in a
> sensible way (where the exact details may depend on the software).
>
> It is then a sysadmin decision to setup /tmp or TMPDIR properly
> so there's enough free space for temporary files for the software
> they use. Debian should provide sensible defaults, but it's not
> possible to pick defaults that are optimal for everyone, in
> this situation. Given that MySQL, scientific software, and such
> systems are mostly run on systems that have sysadmins, and desktops
> are run by people who do not, it's sensible to favor the desktop
> case by default.

In my uni sysadmin said: we use distribution default by default, if
you want to change default fill a complain. It take about 3 month to
get some simple change from default like activing transparent hugpage
for hpc. I know it is crappy but it is also real world. Do not think
that sysadmin are competent.

To the contrary they care less about desktop because you could only
hit your own system. So you have modification in less than two days.

Bastien

> --
> Freedom-based blog/wiki/web hosting: http://www.branchable.com/
>
> -----BEGIN PGP SIGNATURE-----
> Version: GnuPG v1.4.10 (GNU/Linux)
>
> iQEcBAEBCAAGBQJOv6RWAAoJEIahkFub41rm2ukH/3Vjn/e21pcPWQI3W9ua7KuD
> SkMCqPJHnrKg0LXFCV0UfvV2IKa9Mf0xVI+WMv46CpLfpnLxgh P7gSpFpX90zZxt
> atuQ5QU1GkXAFsAEgl4j/GmymmR1uGsSiWhKcgRiFnslW7e1wc6oUs2fBgsMDGaV
> UOCS/lOsTMSNF6G3LMPvwTSVnMNtXOXM57yN42gGmojFlwzA9zJKAG3 EaZCSqA2l
> ESVKJP579FLWxMjMIN85dojSFulNIheAO8UNhL8hERViUkTbsK aN9MP3I9y58psp
> mvORbYNLT+41F1HIrUy7tK4ffpOW/d9z8s65Sn3f7cErNNiwsTRa37LYcHnoV40=
> =/6mu
> -----END PGP SIGNATURE-----
>
>


--
To UNSUBSCRIBE, email to debian-devel-REQUEST@lists.debian.org
with a subject of "unsubscribe". Trouble? Contact listmaster@lists.debian.org
Archive: CAE2SPAYXdoNOyCn=nGX04LMHGYqhCMw1-H-EXN5wLs9UHH5OKA@mail.gmail.com">http://lists.debian.org/CAE2SPAYXdoNOyCn=nGX04LMHGYqhCMw1-H-EXN5wLs9UHH5OKA@mail.gmail.com
 
Old 11-13-2011, 11:44 AM
Roger Leigh
 
Default /tmp as tmpfs and consequence for imaging software

On Sun, Nov 13, 2011 at 11:04:55AM +0000, Lars Wirzenius wrote:
> On Sun, Nov 13, 2011 at 06:30:27PM +0800, Thomas Goirand wrote:
> > Agreed. Anyway, I'd be happy to have, by policy, some hard limits
> > We can discuss of a tolerable size, like 100M? Anyway, anything
> > bigger than 512 MB is obviously abusing /tmp, IMHO. But if we're
> > to have /tmp using tmpfs by default, it becomes really important
> > to have such policy.
>
> The proper policy, IMHO, is that a) all software that uses temporary
> files should obey TMPDIR if set (and fall back on /tmp if not)
> and b) all software must deal with out-of-disk-space errors in a
> sensible way (where the exact details may depend on the software).

Agreed. Software which has unusually large space requirements
should, I think, make use of existing facilities such as statvfs(2)
to check that sufficient free space exists prior to attempting to
use several GiB. While not a guarantee of success, this would
permit such applications to display a suitably informative error
message, perhaps with instructions for rectifying the problem,
rather than inconveniently failing later with ENOSPC.

> It is then a sysadmin decision to setup /tmp or TMPDIR properly
> so there's enough free space for temporary files for the software
> they use. Debian should provide sensible defaults, but it's not
> possible to pick defaults that are optimal for everyone, in
> this situation. Given that MySQL, scientific software, and such
> systems are mostly run on systems that have sysadmins, and desktops
> are run by people who do not, it's sensible to favor the desktop
> case by default.

As mentioned in the original report, one of the "desktop" use cases
is burning DVD images, which may require over 4 GiB in temporary
space. Whether /tmp is suitable for such a purpose is IMO debatable.

Currently, the size limits for /tmp and other temporary filesystems
are set in /etc/default/tmpfs. It would I think be relatively
simple for the debian-installer to change these default values
according to the general usage of the system. This could also
influence the automated partitioner should it be desirable to
have a disc-backed /tmp. I would certainly like for the
partitioner to permit changing of the mount defaults, including
size limits, for the various tmpfses including /tmp, when
generating the initial /etc/fstab. I lack the d-i knowledge to
add support for tmpfs though.


Regards,
Roger

--
.'`. Roger Leigh
: :' : Debian GNU/Linux http://people.debian.org/~rleigh/
`. `' Printing on GNU/Linux? http://gutenprint.sourceforge.net/
`- GPG Public Key: 0x25BFB848 Please GPG sign your mail.


--
To UNSUBSCRIBE, email to debian-devel-REQUEST@lists.debian.org
with a subject of "unsubscribe". Trouble? Contact listmaster@lists.debian.org
Archive: 20111113124409.GY28537@codelibre.net">http://lists.debian.org/20111113124409.GY28537@codelibre.net
 
Old 11-13-2011, 01:39 PM
Salvo Tomaselli
 
Default /tmp as tmpfs and consequence for imaging software

> $HOME is not really nice but it could work. I have a tmp dir under my
> home directry and some script to clean up at every log on.

$HOME seems like a very bad idea to me. At least if used by default...

Many universities (and i guess other places too) keep the homes on a file
server and the rest locally.
This mean that having tmp on the home, potentially lot of network traffic
would be generated by software using temp files and well, it would just slow
everyone down for no good reason, since the file would be deleted soon.

Also, a temp file would use quota. That could be good or bad.. depends on what
is actually wanted.

--
Salvo Tomaselli
 
Old 11-13-2011, 01:59 PM
Josselin Mouette
 
Default /tmp as tmpfs and consequence for imaging software

Le dimanche 13 novembre 2011 à 15:39 +0100, Salvo Tomaselli a écrit :
> > $HOME is not really nice but it could work. I have a tmp dir under my
> > home directry and some script to clean up at every log on.
>
> $HOME seems like a very bad idea to me. At least if used by default...

On the contrary, it is a good default.

> Many universities (and i guess other places too) keep the homes on a file
> server and the rest locally.

It is a bad idea to do that on workstations, because network homes will
completely lock your computer on every network outage.

Those with specific, fully networked setups, like HPC clusters of which
case was mentioned, can specify another place if the application allows
it. This is why it is a good idea to have $HOME as default while making
this easily configurable - preferably both system-wide and per-user.

--
.'`. Josselin Mouette
: :' :
`. `'
`-
 
Old 11-13-2011, 02:24 PM
Tollef Fog Heen
 
Default /tmp as tmpfs and consequence for imaging software

]] Bastien ROUCARIES

Hi,

| No it is not true. Science and imaging software are better to use true
| disk baked file. For instance, if I want ot invert a big matrix they
| are pretty good algorithm that force only some part of the file to be
| keep on disk. They known better than kernel when to put somepart on
| the data on the slow disk.

I doubt the authors of said software is better at writing VMs than the
kernel authors are. Even if they are as good, the software isn't in a
position to know the needs ot the whole system, just what the particular
application needs.

| Using tmpfs under /tmpfs you break assumptions on the life expendancy
| of memory object.

How is this broken? It's not like /tmp is magically cleaned more often
just because it's a tmpfs.

| And you slow down this kind of software (that work perfecly for 40
| years).

If the architecture of a piece of software is the same as it was 40
years ago, it's not particularly well adapted to today's machines, since
it won't know how to take advantage of virtual memory, multiple cores,
etc.

Cheers,
--
Tollef Fog Heen
UNIX is user friendly, it's just picky about who its friends are


--
To UNSUBSCRIBE, email to debian-devel-REQUEST@lists.debian.org
with a subject of "unsubscribe". Trouble? Contact listmaster@lists.debian.org
Archive: 87fwhsrqot.fsf@qurzaw.varnish-software.com">http://lists.debian.org/87fwhsrqot.fsf@qurzaw.varnish-software.com
 
Old 11-13-2011, 02:29 PM
Tollef Fog Heen
 
Default /tmp as tmpfs and consequence for imaging software

]] Roger Leigh

Hi,

somewhat of a tangent, so sorry for hijacking the thread:

| Currently, the size limits for /tmp and other temporary filesystems
| are set in /etc/default/tmpfs.

Can we please stop doing things like that? It makes things harder for
alternative init systems that sysvinit hides this in a random file in
/etc/default rather than just putting it into fstab.

The same goes for RAMTMP, RAMRUN and whatnot.

(Yes, this means we need to handle /etc/fstab better from a maintainer
script when defaults change. Maybe have /etc/fstab.d. ;-)

Cheers,
--
Tollef Fog Heen
UNIX is user friendly, it's just picky about who its friends are


--
To UNSUBSCRIBE, email to debian-devel-REQUEST@lists.debian.org
with a subject of "unsubscribe". Trouble? Contact listmaster@lists.debian.org
Archive: 87bosgrqgl.fsf@qurzaw.varnish-software.com">http://lists.debian.org/87bosgrqgl.fsf@qurzaw.varnish-software.com
 

Thread Tools




All times are GMT. The time now is 11:00 AM.

VBulletin, Copyright ©2000 - 2014, Jelsoft Enterprises Ltd.
Content Relevant URLs by vBSEO ©2007, Crawlability, Inc.
Copyright ©2007 - 2008, www.linux-archive.org