FAQ Search Today's Posts Mark Forums Read
» Video Reviews

» Linux Archive

Linux-archive is a website aiming to archive linux email lists and to make them easily accessible for linux users/developers.


» Sponsor

» Partners

» Sponsor

Go Back   Linux Archive > Debian > Debian Development

 
 
LinkBack Thread Tools
 
Old 11-17-2011, 01:43 PM
Goswin von Brederlow
 
Default /tmp as tmpfs and consequence for imaging software

Salvo Tomaselli <tiposchi@tiscali.it> writes:

>> I think the problems you describe are quite uncommon. Yes, there are use
>> cases where tmpfs for /tmp isn't the best solution but I think most
>> people do not place 1.2GB files in their /tmp and benefit greatly from
>> tmpfs.
>
> I thought DVD burners were quite common... and almost every desktop or laptop
> has one. And a DVD is 4GiB when not 7. And usually burning software lets you
> decide if you want to 1st create an iso and then burn it.
>
> So please, don't say it is uncommon to have big files in /tmp.
>
> --
> Salvo Tomaselli

1) Laptops/netbooks without a optical drive at all are quite common too.
2) Why would you first create the iso? Waste of time.
3) I used to burn a lot of DVDs but I haven't burned one for at least 2
years now. Seems to be dying out. Somehow it is usb sticks or drives now
where people used to burn DVDs before.
4) Surely the software lets you store the iso somewhere else too.

MfG
Goswin


--
To UNSUBSCRIBE, email to debian-devel-REQUEST@lists.debian.org
with a subject of "unsubscribe". Trouble? Contact listmaster@lists.debian.org
Archive: 87r516zu5y.fsf@frosties.localnet">http://lists.debian.org/87r516zu5y.fsf@frosties.localnet
 
Old 11-17-2011, 01:46 PM
Goswin von Brederlow
 
Default /tmp as tmpfs and consequence for imaging software

Richard <richard.bown@blueyonder.co.uk> writes:

> On Wed, 16 Nov 2011 13:21:47 +0100
> Didier Raboud <odyx@debian.org> wrote:
>
>> Salvo Tomaselli wrote:
>>
>> >> I think the problems you describe are quite uncommon. Yes, there are use
>> >> cases where tmpfs for /tmp isn't the best solution but I think most
>> >> people do not place 1.2GB files in their /tmp and benefit greatly from
>> >> tmpfs.
>> >
>> > I thought DVD burners were quite common... and almost every desktop or
>> > laptop has one. And a DVD is 4GiB when not 7. And usually burning software
>> > lets you decide if you want to 1st create an iso and then burn it.
>>
>> Given that any burning software can (approximately) determine what size the
>> ISO file will be, it should really not start to write it in /tmp when the
>> /tmp size is not big enough (which the software can also check). Prompting a
>> user with "I will not be able to write ${file} in /tmp, please point me to a
>> location where I can." should not be too much of a problem.
>>
>>
>>
>
> Hi would the real solution be to make /tmp dynamic ?, that would allow for dual layer burning and
> blueray, or is that easier said than done.

You can make /tmp any size on the fly. But you need to have enough SWAP
setup to actualy use it. And that usualy needs some planing ahead.

> The other area where a large /tmp is needed is video streaming, especially if you are generating a
> video and putting down a USB port for external encoding to DVB-T or DVB-S., but that's a specialised
> application.

Again a case where you would not want /tmp to be on your / filesystem
where free space gets quickly fragmented. So if you don't have a
seperate /tmp filesystem you have already a suboptimal configuration and
having tmpfs for /tmp just makes that more noticeable.

MfG
Goswin


--
To UNSUBSCRIBE, email to debian-devel-REQUEST@lists.debian.org
with a subject of "unsubscribe". Trouble? Contact listmaster@lists.debian.org
Archive: 87mxbuztzw.fsf@frosties.localnet">http://lists.debian.org/87mxbuztzw.fsf@frosties.localnet
 
Old 11-17-2011, 01:54 PM
Goswin von Brederlow
 
Default /tmp as tmpfs and consequence for imaging software

Chow Loong Jin <hyperair@ubuntu.com> writes:

> On 16/11/2011 22:45, Salvo Tomaselli wrote:
>>
>>> Given that any burning software can (approximately) determine what size the
>>> ISO file will be, it should really not start to write it in /tmp when the
>>> /tmp size is not big enough (which the software can also check). Prompting
>>> a user with "I will not be able to write ${file} in /tmp, please point me
>>> to a location where I can." should not be too much of a problem.
>>
>> Yes it can (and does) if there is not enough space in /tmp. What mostly scares
>> me is the idea of paging out 4GiB to the swap partition (assuming it is so
>> big), because of a tmpfs configured to be very large.
>
> Doesn't tmpfs default to 50% of your memory? Unless you have 8GB of memory, you
> shouldn't be seeing 4GB worth of data getting into /tmp by mistake.

4GiB at 100MiB/s = 40s.

So the following code could block for a short while:

char *mem = malloc(4*1024*1024*1024);
memset(mem, 0, 4*1024*1024*1024);

So if you do use a lot of /tmp and have code that eats that much memory
at once then you might want to change your config from the default.

But seriously, how common is that?

MfG
Goswin


--
To UNSUBSCRIBE, email to debian-devel-REQUEST@lists.debian.org
with a subject of "unsubscribe". Trouble? Contact listmaster@lists.debian.org
Archive: 87ipmiztnr.fsf@frosties.localnet">http://lists.debian.org/87ipmiztnr.fsf@frosties.localnet
 
Old 11-17-2011, 01:58 PM
Goswin von Brederlow
 
Default /tmp as tmpfs and consequence for imaging software

Carlos Alberto Lopez Perez <clopez@igalia.com> writes:

> On 16/11/11 11:37, Goswin von Brederlow wrote:
>> Assuming you have increased your SWAP by the size of the tmpfs to
>> compensate for /tmp now using RAM+SWAP you can only ever get that effect
>> in cases where the OOMKiller would have already been triggered with /tmp
>> on disk.
>
> We are talking about the defaults... would the installer by default
> increase the swap by the same size of the tmpfs?

I would hope so.

The installer could even have a different partitioning templates for
small tmpfs (no swap), large tmpfs (large swap) or real filesystem.

MfG
Goswin


--
To UNSUBSCRIBE, email to debian-devel-REQUEST@lists.debian.org
with a subject of "unsubscribe". Trouble? Contact listmaster@lists.debian.org
Archive: 87ehx6zth8.fsf@frosties.localnet">http://lists.debian.org/87ehx6zth8.fsf@frosties.localnet
 
Old 11-17-2011, 02:04 PM
Goswin von Brederlow
 
Default /tmp as tmpfs and consequence for imaging software

Salvo Tomaselli <tiposchi@tiscali.it> writes:

>> While amazon.com "cloud" may have small RAM and large disks, many
>> mainframes are opposite (ie, IBM big blue, japan's world simulator). And
>> many new PCs may become that way: no spin Maybe not!
>
> I think we are focusing a bit too much on super expensive computers here.
> Of course on a computer with 1TiB of RAM it is faster to keep /tmp on ram.
> Also, given the availability of a reliable power source i would decrease the
> frequency of syncs to disk, in order to work mostly on buffers on ram.
>
> Unfortunately debian targets also different devices, such as ARM network
> attached storage with huge disk and 32MiB of RAM.
>
> --
> Salvo Tomaselli

So make the installer check and default to different partitioning
schemes.

That said I would still try tmpfs with large swap on a 32MiB system to
avoid the fua/flush and journal writes. tmpfs will use less erasures and
thereby make my flash last longer on my embedded arm device.

And on a 1TiB of RAM system you usualy run software that needs that much
ram and allocates it in big chunks. You wouldn't want the software to
hang to swap out 100GB of tmpfs. Better to have a real filesystem for
/tmp there that will already have written out the 100GB in the
background.

So you might have a point to the opposite effect than you intendet.


Overall any default will never work for all cases. It's a default, not
THE LAW, change it when it doesn't suite.

MfG
Goswin


--
To UNSUBSCRIBE, email to debian-devel-REQUEST@lists.debian.org
with a subject of "unsubscribe". Trouble? Contact listmaster@lists.debian.org
Archive: 87aa7uzt6h.fsf@frosties.localnet">http://lists.debian.org/87aa7uzt6h.fsf@frosties.localnet
 
Old 11-17-2011, 02:34 PM
Goswin von Brederlow
 
Default /tmp as tmpfs and consequence for imaging software

Russell Coker <russell@coker.com.au> writes:

> On Wed, 16 Nov 2011, Goswin von Brederlow <goswin-v-b@web.de> wrote:
>> With a filesystem it will write the dirty buffers to disk in the
>> background and then drop the clean pages from the cache quite
>> consistently. This leaves the code involved with moving the mouse
>> pointer alone and functioning smoothly.
>
> That's not what I have observed.
>
> The command "mv /mnt/usb/* /mnt/nfs" can cause X related stuff to be swapped
> out and the mouse pointer to freeze. It happens to me repeatedly running
> Unstable with kernel 3.0.0-2.

True. I do see the same thing with writing to NFS. The cache gets
quickly filled with dirty pages, the network can't (or simply doesn't
for some reason) keep up and then linux runs into the out-of-memory path
and starts swapping or otherwise blocks. But I've only seen that with NFS.

> On Wed, 16 Nov 2011, Goswin von Brederlow <goswin-v-b@web.de> wrote:
>> Having /tmp on / would quickly degrade performance as the filesystems
>> free space becomes fragmented over time. So if you end up with a tmpfs
>> for /tmp on such a system you already did install your systems wrong for
>> your use case under the old setup. So only suboptimal configurations are
>> hurt by defaulting to tmpfs.
>
> Could you please point me to a reference about some research on ext3/4
> fragmentation supporting this claim?

Can't give you any references but any research on fragmentation should
show you that if you mix short term and long term data then you will get
pices of the long term data stuck all over the place reducing the size
of continious blocks that are free.

>From personal experience I've managed to degrade an ext3 filesystem to
the point where reading of data sustained barely over 100KB/s by running
rtorrent with lots of downloads on it for a few month and always above
90% full.

MfG
Goswin


--
To UNSUBSCRIBE, email to debian-devel-REQUEST@lists.debian.org
with a subject of "unsubscribe". Trouble? Contact listmaster@lists.debian.org
Archive: 87wrayyd7k.fsf@frosties.localnet">http://lists.debian.org/87wrayyd7k.fsf@frosties.localnet
 
Old 11-17-2011, 04:44 PM
"John D. Hendrickson and Sara Darnell"
 
Default /tmp as tmpfs and consequence for imaging software

#1 Backup. Why is supporting optional kernel features injected at runlevel S and not runlevel 3?

At runlevel 3, being optional, no one will care which is done. At runlevel S, before a login can
fix things, it's a problem.


/tmp is legacy. You don't write history or future for others. What I mean is: /tmp is just a
directory of convenience guarunteed to be there. If you want Gnome to cache orbit using shared
memory - then using /tmp is not the way. configuring orbit is the way.


#2 I still don't think you offered proof that using LIMITED memory as a ramdisk for /tmp will
outperform other uses of that memory.


And this sounds like NEWBIES will end up on a red-herring chase figuring out why they are getting
tmp error messages and having to read about tmpfs to get their system up. NEWBIES have too much to
handle already with installing and booting: far too much in my opinion.


>> so is really only misbehaving when misconfigured.

No. It's also misbehaving if MIS-USED by newbies. What new user knows to increase swap beyond what
linuxdocs say and knows not to put files in /tmp ??? And me. I use /tmp for local scripts.


I saw some init scripts (was it ppp?) using / storing files in /dev/shm. I personally don't like
the policy "any package alters ppp and disperses what it's doing widely across the system". and ppp
scripts don't share memory why use /dev/shm ? !


it's a pain to check to see after each package installs if it has not badly altered ppp shell
scripts to allow rooting the system in some way.



--
To UNSUBSCRIBE, email to debian-devel-REQUEST@lists.debian.org
with a subject of "unsubscribe". Trouble? Contact listmaster@lists.debian.org
Archive: 4EC547FD.8060305@cox.net">http://lists.debian.org/4EC547FD.8060305@cox.net
 

Thread Tools




All times are GMT. The time now is 10:44 PM.

VBulletin, Copyright ©2000 - 2014, Jelsoft Enterprises Ltd.
Content Relevant URLs by vBSEO ©2007, Crawlability, Inc.
Copyright 2007 - 2008, www.linux-archive.org