FAQ Search Today's Posts Mark Forums Read
» Video Reviews

» Linux Archive

Linux-archive is a website aiming to archive linux email lists and to make them easily accessible for linux users/developers.


» Sponsor

» Partners

» Sponsor

Go Back   Linux Archive > Gentoo > Gentoo Embedded

 
 
LinkBack Thread Tools
 
Old 04-17-2008, 01:47 PM
Christopher Friedt
 
Default profiles for reduced disk usage

Does anyone know if there is a portage profile for reduced disk writes?
I just got an EEE pc and I'm experimenting on a USB stick before I write
to the SDD. I'm sort of guessing that such a profile would basically
just slightly alter the default configuration files of certain apps to
either turn off logging or to mount temporary directories as tmpfs.


Does such a profile sound closer to embedded, or would it be a desktop
profile without the logger, etc, etc.


It seems the line between embedded and default is growing thinner every day!

I guess I'm looking for something that's more or less a Desktop profile,
but something that would be more suitable for flash-based devices, like
the OpenMoko/Neo, EEE Pc, or a Maemo.


Nedd, I've been using qmerge with my arm packages by the way, and it's
fantastic. I'm really looking forward to using it more. At some point
I'm hoping to use, augment or write a graphical app for installing
packages locally and also remotely via ssh or some web interface.


Cheers,

Chris
--
gentoo-embedded@lists.gentoo.org mailing list
 
Old 04-17-2008, 02:06 PM
Joshua ChaitinPollak
 
Default profiles for reduced disk usage

Chris,
I think you'll still want to use the desktop profile, but configure your /tmp and /var/log to be mounted on a shared file system. What I've done in my embedded system is modify the inittab file to mount tmp on a shmfs file system, and the create directories called /tmp/var/log and /tmp/var/lock. The root file system (on disk), has symbolic links in place at /var/log and /var/lock that point to /tmp/var/log and /tmp/var/lock, respectively. This works fine, because by the time any process is ready to write to /var/log, the inittab has already mounted the shared file system and made the target directories of the symbolic links.
I think this would work for you, but its a bit of manual configuration you'll have to work through.
I don't know how you'd handle things like Firefox's cache, I don't know any way to reduce its writes or redirect them out of your profile.

On Apr 17, 2008, at 9:47 AM, Christopher Friedt wroteoes anyone know if there is a portage profile for reduced disk writes? I just got an EEE pc and I'm experimenting on a USB stick before I write to the SDD. I'm sort of guessing that such a profile would basically just slightly alter the default configuration files of certain apps to either turn off logging or to mount temporary directories as tmpfs.

Does such a profile sound closer to embedded, or would it be a desktop profile without the logger, etc, etc.

It seems the line between embedded and default is growing thinner every day!

I guess I'm looking for something that's more or less a Desktop profile, but something that would be more suitable for flash-based devices, like the OpenMoko/Neo, EEE Pc, or a Maemo.

Nedd, I've been using qmerge with my arm packages by the way, and it's fantastic. I'm really looking forward to using it more. At some point I'm hoping to use, augment or write a graphical app for installing packages locally and also remotely via ssh or some web interface.

Cheers,

Chris
--
gentoo-embedded@lists.gentoo.org mailing list



--*Joshua ChaitinPollak |*Software Engineer*Kiva Systems, Inc.,*225*Wildwood Ave, Woburn, MA 01970
 
Old 04-17-2008, 03:12 PM
"Christian Bricart"
 
Default profiles for reduced disk usage

Joshua ChaitinPollak wrote:
> [..]
> I don't know how you'd handle things like Firefox's cache, I don't
> know any way to reduce its writes or redirect them out of your profile.

http://www.mozilla.org/catalog/end-user/customizing/briefprefs.html
http://kb.mozillazine.org/Browser.cache.disk.parent_directory

drop a file in /usr/lib/mozilla-firefox/defaults/pref/ with content like:

+---
| pref("browser.cache.disk.parent_directory", "/tmp");
+---

or even:

+---
| pref("browser.cache.disk.enable", false);
+---

Christian


--
gentoo-embedded@lists.gentoo.org mailing list
 
Old 04-17-2008, 03:54 PM
Joshua ChaitinPollak
 
Default profiles for reduced disk usage

Here's a response from a co-worker who has done some work on his EEE pc:
Interesting stuff. As I understand it, with the Asus I've read that they've basically leveraged a write-balancing algorithm at the flash firmware level so that they fairly evenly distribute updates across the flash. So, bottom line, Asus doesn't worry too much about burn-in. They even ship with a writable ext3 partition, which would be really detrimental to disk due to the "duplicate write" overhead of continuous journaling (presumably in a fixed area of the disk?) if it weren't for the balancing logic.

W.r.t. keeping log size down, I think Asus's build of Xandros has turned off a bunch of debug-level logging within most of their packages (at build time?) and as such doesn't really demand much disk for logging operations, etc.

Same may apply to a Gentoo install - compile your stuff to not log at debug level (or turn off in syslog?) and you should be fine...

One thing that I did observe when I started using external ext2/ext3 partitions on the SD card slot was extremely lethargic writes. This was due to the fact that SD cards with ext2/ext3 partitions were being mounted with the 'sync' flag. From what I have read, this is extremely slow and extremely detrimental to flash. This is because every byte would be written individually, but due to the way flash works, you write a block at once. So, say the block size is 192 bytes... To write 192 bytes with sync on could cause 192 flashes of the block (or some fraction/multiple of 192). With async enabled (sync not enforced) the OS is able to cache up some data before writing, dramatically increasing the likelihood that it will have enough data to fill the block in 1 single write operation.

So, another tip for Gentoo users - make sure any ext2/ext3 flash partitions are mounted async (default) - not with the sync flag set like Xandros does by default with the SD card slot...

-Andrew

On Apr 17, 2008, at 11:12 AM, Christian Bricart wrote:Joshua ChaitinPollak wrote:
[..]
I don't know how you'd handle things like Firefox's cache, I don't
know any way to reduce its writes or redirect them out of your profile.

http://www.mozilla.org/catalog/end-user/customizing/briefprefs.html
http://kb.mozillazine.org/Browser.cache.disk.parent_directory

drop a file in /usr/lib/mozilla-firefox/defaults/pref/ with content like:

+---
| pref("browser.cache.disk.parent_directory", "/tmp");
+---

or even:

+---
| pref("browser.cache.disk.enable", false);
+---

Christian


--
gentoo-embedded@lists.gentoo.org mailing list



--*Joshua ChaitinPollak |*Software Engineer*Kiva Systems, Inc.,*225*Wildwood Ave, Woburn, MA 01970
 
Old 04-17-2008, 08:11 PM
Christopher Friedt
 
Default profiles for reduced disk usage

I've noticed a lot of the same things actually - I was using an Ubuntu
LiveUSB for a while, with persistent storage, and that would be just
brutal in terms of any disk I/O.


Actually it was with the latest one, Hardy Heron, so if anyone is using
that on a flash filesystem, be sure to have _loads_ of ram, a magnetic
disk swap, or be prepared to do a lot of extra configuration to prevent
disk writes.


Thanks for the tips,

Chris

Joshua ChaitinPollak wrote:

Here's a response from a co-worker who has done some work on his EEE pc:

Interesting stuff. As I understand it, with the Asus I've read that
they've basically leveraged a write-balancing algorithm at the flash
firmware level so that they fairly evenly distribute updates across
the flash. So, bottom line, Asus doesn't worry too much about burn-in.
They even ship with a writable ext3 partition, which would be really
detrimental to disk due to the "duplicate write" overhead of
continuous journaling (presumably in a fixed area of the disk?) if it
weren't for the balancing logic.


W.r.t. keeping log size down, I think Asus's build of Xandros has
turned off a bunch of debug-level logging within most of their
packages (at build time?) and as such doesn't really demand much disk
for logging operations, etc.


Same may apply to a Gentoo install - compile your stuff to not log at
debug level (or turn off in syslog?) and you should be fine...


One thing that I did observe when I started using external ext2/ext3
partitions on the SD card slot was extremely lethargic writes. This
was due to the fact that SD cards with ext2/ext3 partitions were being
mounted with the 'sync' flag. From what I have read, this is extremely
slow and extremely detrimental to flash. This is because every byte
would be written individually, but due to the way flash works, you
write a block at once. So, say the block size is 192 bytes... To write
192 bytes with sync on could cause 192 flashes of the block (or some
fraction/multiple of 192). With async enabled (sync not enforced) the
OS is able to cache up some data before writing, dramatically
increasing the likelihood that it will have enough data to fill the
block in 1 single write operation.


So, another tip for Gentoo users - make sure any ext2/ext3 flash
partitions are mounted async (default) - not with the sync flag set
like Xandros does by default with the SD card slot...


-Andrew


On Apr 17, 2008, at 11:12 AM, Christian Bricart wrote:


Joshua ChaitinPollak wrote:

[..]
I don't know how you'd handle things like Firefox's cache, I don't
know any way to reduce its writes or redirect them out of your profile.


http://www.mozilla.org/catalog/end-user/customizing/briefprefs.html
http://kb.mozillazine.org/Browser.cache.disk.parent_directory

drop a file in /usr/lib/mozilla-firefox/defaults/pref/ with content like:

+---
| pref("browser.cache.disk.parent_directory", "/tmp");
+---

or even:

+---
| pref("browser.cache.disk.enable", false);
+---

Christian


--
gentoo-embedded@lists.gentoo.org mailing list





--
gentoo-embedded@lists.gentoo.org mailing list
 
Old 04-17-2008, 08:16 PM
Christopher Friedt
 
Default profiles for reduced disk usage

I've taken a similar approach with the package db and the emerge cache,
which would usually reside in /var/db/pkg and /var/cache/edb.


good idea for the log, and lock dirs.

I guess /var/run, etc, etc can all go in tmpfs

C

Joshua ChaitinPollak wrote:

Chris,

I think you'll still want to use the desktop profile, but configure your
/tmp and /var/log to be mounted on a shared file system. What I've done
in my embedded system is modify the inittab file to mount tmp on a shmfs
file system, and the create directories called /tmp/var/log and
/tmp/var/lock. The root file system (on disk), has symbolic links in
place at /var/log and /var/lock that point to /tmp/var/log and
/tmp/var/lock, respectively. This works fine, because by the time any
process is ready to write to /var/log, the inittab has already mounted
the shared file system and made the target directories of the symbolic
links.


I think this would work for you, but its a bit of manual configuration
you'll have to work through.


I don't know how you'd handle things like Firefox's cache, I don't know
any way to reduce its writes or redirect them out of your profile.



On Apr 17, 2008, at 9:47 AM, Christopher Friedt wrote:

Does anyone know if there is a portage profile for reduced disk
writes? I just got an EEE pc and I'm experimenting on a USB stick
before I write to the SDD. I'm sort of guessing that such a profile
would basically just slightly alter the default configuration files of
certain apps to either turn off logging or to mount temporary
directories as tmpfs.


Does such a profile sound closer to embedded, or would it be a desktop
profile without the logger, etc, etc.


It seems the line between embedded and default is growing thinner
every day!


I guess I'm looking for something that's more or less a Desktop
profile, but something that would be more suitable for flash-based
devices, like the OpenMoko/Neo, EEE Pc, or a Maemo.


Nedd, I've been using qmerge with my arm packages by the way, and it's
fantastic. I'm really looking forward to using it more. At some point
I'm hoping to use, augment or write a graphical app for installing
packages locally and also remotely via ssh or some web interface.


Cheers,

Chris
--
gentoo-embedded@lists.gentoo.org mailing list





--
gentoo-embedded@lists.gentoo.org mailing list
 
Old 04-17-2008, 10:18 PM
Ed W
 
Default profiles for reduced disk usage

Christopher Friedt wrote:
I've noticed a lot of the same things actually - I was using an Ubuntu
LiveUSB for a while, with persistent storage, and that would be just
brutal in terms of any disk I/O.


I guess that was deliberate because most users assume that if they hit
save on their document and the screen says "saved", then they can rip
the stick out after a few seconds (that's how windows works after
all...). So based on the principle of least surprise I guess sync is a
sensible choice for the masses?


What would be a compromise is if it's possible to set the sync time per
device? Although isn't a flush implicit every 5 secs anyway on a
default kernel?


Ed W
--
gentoo-embedded@lists.gentoo.org mailing list
 

Thread Tools




All times are GMT. The time now is 06:59 PM.

VBulletin, Copyright ©2000 - 2014, Jelsoft Enterprises Ltd.
Content Relevant URLs by vBSEO ©2007, Crawlability, Inc.
Copyright 2007 - 2008, www.linux-archive.org