FAQ Search Today's Posts Mark Forums Read
» Video Reviews

» Linux Archive

Linux-archive is a website aiming to archive linux email lists and to make them easily accessible for linux users/developers.


» Sponsor

» Partners

» Sponsor

Go Back   Linux Archive > ArchLinux > ArchLinux General Discussion

 
 
LinkBack Thread Tools
 
Old 06-20-2012, 09:36 AM
Dennis Herbrich
 
Default back up /var/log before shutdown

On Wed, Jun 20, 2012 at 10:36:17AM +0200, Arno Gaboury wrote:
> add this in my */etc/rc.local.shutdown*:
>
> |*echo "Copying LOGs..."
>
> now=`date +"%Y%m%d_%Hh%M"`
> mkdir -p /logs_backup/$now
> cp -Rp /var/log/* ~/backup/logs_backup/$now/*
>
> My ~ folder is on another HD.
>
> Will this script be enough to do the job?

Depends on your job description.

If your system crashes (hah, as if ever!) or becomes unresponsive, you're
screwed, as rc.local.shutdown is likely not called and your logs are lost after
reboot. This is probably not what you want.

If you've got suitable network infrastructure, you may want to instruct
syslog-ng to forward the logs to a remote logging daemon in addition to the
local ramdisk. This is nice and clean, given a stable network connection to a
suitable machine to work as a logging daemon.

Just a desktop machine or no independent server available? It may be enough for
your purposes to use logrotate to copy over the logs properly to a safe mass
storage device in regular intervals. Maybe hourly. This, however, won't help
you much in case of kernel panic, either. It's better than rolling your own
regular rotation with cron and shell, though.

Feeling old-school? Setup a printer to receive critical(!) logs. For bonus
points, use a 9-pin matrix printer. If you hear it screeching, you know
something's horribly wrong. Free notification to boot!

Alternatively, if you need critical(!) log messages even after a crash, you may
want to configure syslog-ng to only(!) log such critical messages directly to a
file on you mass storage device. Output should be scarce as not to unduly put
your flash memory under stress.

Lots of possibilities. Choose wisely.

Best regards,
Dennis

--
"Den Rechtsstaat macht aus, dass Unschuldige wieder frei kommen."
Dr. Wolfgang Schäuble, Bundesinnenminister (14.10.08, TAZ-Interview)

0D21BE6C - F3DC D064 BB88 5162 56BE 730F 5471 3881 0D21 BE6C
 
Old 06-20-2012, 09:37 AM
Arno Gaboury
 
Default back up /var/log before shutdown

On 06/20/2012 11:23 AM, Gaetan Bisson wrote:

[2012-06-20 11:08:37 +0200] Arno Gaboury:

I fully understand these 3 lines, and thought it was not a
complicated way to back up /var/log at shutdown. But as it seems
there is much simplier way, I will then investigate.

I did not say those three lines were complicated; quite the opposite. My
point was that if you cannot understand them, you should not be doing
anything involving partitioning, experimental file systems such as
BTRFS, etc.

But since you do understand them, you can determine for yourself whether
they actually are an acceptable solution to your problem.

Cheers.


It seems to me now that the*asd* daemon is a better and cleaner solution.
 
Old 06-20-2012, 10:04 AM
Jan Steffens
 
Default back up /var/log before shutdown

On Wed, Jun 20, 2012 at 11:36 AM, Dennis Herbrich <dennis@archlinux.org> wrote:
> On Wed, Jun 20, 2012 at 10:36:17AM +0200, Arno Gaboury wrote:
>> add this in my */etc/rc.local.shutdown*:
>>
>> |*echo "Copying LOGs..."
>>
>> now=`date +"%Y%m%d_%Hh%M"`
>> mkdir -p /logs_backup/$now
>> cp -Rp /var/log/* ~/backup/logs_backup/$now/*
>>
>> My ~ folder is on another HD.
>>
>> Will this script be enough to do the job?
>
> Depends on your job description.
>
> If your system crashes (hah, as if ever!) or becomes unresponsive, you're
> screwed, as rc.local.shutdown is likely not called and your logs are lost after
> reboot. This is probably not what you want.
>
> If you've got suitable network infrastructure, you may want to instruct
> syslog-ng to forward the logs to a remote logging daemon in addition to the
> local ramdisk. This is nice and clean, given a stable network connection to a
> suitable machine to work as a logging daemon.
>
> Just a desktop machine or no independent server available? It may be enough for
> your purposes to use logrotate to copy over the logs properly to a safe mass
> storage device in regular intervals. Maybe hourly. This, however, won't help
> you much in case of kernel panic, either. It's better than rolling your own
> regular rotation with cron and shell, though.
>
> Feeling old-school? Setup a printer to receive critical(!) logs. For bonus
> points, use a 9-pin matrix printer. If you hear it screeching, you know
> something's horribly wrong. Free notification to boot!
>
> Alternatively, if you need critical(!) log messages even after a crash, you may
> want to configure syslog-ng to only(!) log such critical messages directly to a
> file on you mass storage device. Output should be scarce as not to unduly put
> your flash memory under stress.
>
> Lots of possibilities. Choose wisely.
>
> Best regards,
> *Dennis
>
> --
> "Den Rechtsstaat macht aus, dass Unschuldige wieder frei kommen."
> *Dr. Wolfgang Schäuble, Bundesinnenminister (14.10.08, TAZ-Interview)
>
> 0D21BE6C - F3DC D064 BB88 5162 56BE *730F 5471 3881 0D21 BE6C

If you use systemd with its accompanying log daemon (the journal), it
does not write its logs to disk by default. It only does so if the
directory /var/log/journal/ exists. Journald keeps volatile logs in
/run/log/journal, which is on a tmpfs.

If you create /var/log/journal/, you can use the MaxLevelStore=
setting (/etc/systemd/journald.conf) to control which messages get
written to disk, e.g. critical messages only. All other messages are
still recorded in volatile memory.

You can still run a classical syslog daemon and have the journal
forward to it, implementing all the fancy network or printer stuff
above.

Of course, the problem with any syslog (or journal) approach is that
it does not capture applications writing to /var/log/ directly.
 
Old 06-20-2012, 11:34 AM
Arno Gaboury
 
Default back up /var/log before shutdown

On 06/20/2012 11:03 AM, Gaetan Bisson wrote:

[2012-06-20 10:36:17 +0200] Arno Gaboury:

My /, /boot and /usr are on a BTRFS /(except /boot on ext2) on a ssd.
I want to add this line in my fstab to avoid too many writings on my ssd :

*tmpfs /var/log tmpfs nodev,nosuid,noexec 0 0*

So each time I reboot, my /var/log will be emptied, which could be a
problem in case of serious issue on my box. I was then thinking of a
way to backup this folder before I shutdown. I found this trick in
the Arch forum:

add this in my */etc/rc.local.shutdown*:

|*echo "Copying LOGs..."

now=`date +"%Y%m%d_%Hh%M"`
mkdir -p /logs_backup/$now
cp -Rp /var/log/* ~/backup/logs_backup/$now/*

My ~ folder is on another HD.

Will this script be enough to do the job?

Why are you inflicting such a complicated setup on yourself if you
cannot understand what those three little lines of shell do? That seems
to me like a completely backward way of taking the learning curve...

Also, please create new threads instead of hijacking random ones.

Cheers.

After hours of reading, here is the correct script to add in my
rc.shutdown.local.
I know this method will NOT work in case of system crash, but it is OK
for me as I understand what i am doing and it is simple.


*echo -n "Copying /var/log ..."
cd /home/gabx/backup
tar -zcf "./`date +'%d-%b-%y.tgz'`" /var/log

echo " done."*

I think it is much more simple as the one I mentioned earlier, as Gaetan
pointed out.


Cheers.
 
Old 06-20-2012, 12:22 PM
Andrew Hills
 
Default back up /var/log before shutdown

On Wed, Jun 20, 2012 at 7:34 AM, Arno Gaboury <arnaud.gaboury@gmail.com> wrote:
> tar -zcf *"./`date +'%d-%b-%y.tgz'`" /var/log

This date format will have you cursing yourself if you shut down more
than once in the same day.

--Andrew Hills
 
Old 06-20-2012, 12:36 PM
Allan McRae
 
Default back up /var/log before shutdown

On 20/06/12 22:22, Andrew Hills wrote:
> On Wed, Jun 20, 2012 at 7:34 AM, Arno Gaboury <arnaud.gaboury@gmail.com> wrote:
>> tar -zcf "./`date +'%d-%b-%y.tgz'`" /var/log
>
> This date format will have you cursing yourself if you shut down more
> than once in the same day.
>

I think he will be cursing himself when he realises the logs tend to be
most useful after a system crash and using this method he will not have
any...
 
Old 06-20-2012, 01:51 PM
Arno Gaboury
 
Default back up /var/log before shutdown

On 06/20/2012 02:36 PM, Allan McRae wrote:

On 20/06/12 22:22, Andrew Hills wrote:

On Wed, Jun 20, 2012 at 7:34 AM, Arno Gaboury <arnaud.gaboury@gmail.com> wrote:

tar -zcf "./`date +'%d-%b-%y.tgz'`" /var/log

This date format will have you cursing yourself if you shut down more
than once in the same day.


I think he will be cursing himself when he realises the logs tend to be
most useful after a system crash and using this method he will not have
any...



Will this following line avoid me cursing myself when multi reboot in a day?

*tar -zcf "./`date +'%D-%H-%M.tgz'`" /var/log

*AS for backing up the log after a system crash, you are perfectly right. Now I am trying to find something I understand to replace this "dirty" script in my rc.local.shutdwon. Until I found the clean way, I will stick to it.
 
Old 06-20-2012, 02:08 PM
Ralf Mardorf
 
Default back up /var/log before shutdown

I didn't follow this thread, but IIRC I read something about "not to
write to often to SSD"? So you take care regarding to noatime etc.?

FWIW

"Reliability and lifetime SSDs have no moving parts to fail
mechanically. Each block of a flash-based SSD can only be erased (and
therefore written) a limited number of times before it fails. The
controllers manage this limitation so that drives can last for many
years under normal use.[79][80][81][82][83] SSDs based on DRAM do not
have a limited number of writes. Firmware bugs are currently a common
cause for data loss.[citation needed] HDDs have moving parts, and are
subject to potential mechanical failures from the resulting wear and
tear." - Wiki

You'll make your Linux less reliable by handling log files in a dirty
way, to get a longer lifetime for a SSD drive? Nobody really knows how
long they'll last, but it's said they have a longer lifetime than
mechanical drives?

Perhaps not a good idea.
 
Old 06-20-2012, 02:09 PM
Andrew Hills
 
Default back up /var/log before shutdown

On Wed, Jun 20, 2012 at 9:51 AM, Arno Gaboury <arnaud.gaboury@gmail.com> wrote:
> *tar -zcf *"./`date +'%D-%H-%M.tgz'`" /var/log

This will cause a different problem... maybe you wanted %F instead of
%D. (Check "man date".)

--Andrew Hills
 
Old 06-20-2012, 02:57 PM
Leonid Isaev
 
Default back up /var/log before shutdown

On Wed, 20 Jun 2012 10:36:17 +0200
Arno Gaboury <arnaud.gaboury@gmail.com> wrote:

> Dear all,
>
> My /, /boot and /usr are on a BTRFS /(except /boot on ext2) on a ssd.
> I want to add this line in my fstab to avoid too many writings on my ssd :
>
> *tmpfs /var/log tmpfs nodev,nosuid,noexec 0 0*
>
> So each time I reboot, my /var/log will be emptied, which could be a
> problem in case of serious issue on my box. I was then thinking of a way
> to backup this folder before I shutdown. I found this trick in the Arch
> forum:
>
> add this in my */etc/rc.local.shutdown*:
>
> |*echo "Copying LOGs..."
>
> now=`date +"%Y%m%d_%Hh%M"`
> mkdir -p /logs_backup/$now
> cp -Rp /var/log/* ~/backup/logs_backup/$now/*
>
> My ~ folder is on another HD.
>
> Will this script be enough to do the job?
>
> TY for advising.
> |
>
>

Well, SSD's limited number of write cycles is largerly a myth these days (see
www.toshiba.com/taec/news/media_resources/docs/SSDmyths.pdf). Of course it
depends on your particular model/brand but practically, an SSD will most
likely outlive your machine anyways. So I wouldn't worry about /var/log too
much.

However, optimizing log writes is a good idea even on an HDD. I think putting
/var/log to RAM (as well as putting there firefox/thunderbird profiles) is
stupid and is asking for trouble. A much better approach is to properly
configure syslog-ng or rsyslog, specifically:

1. You don't have to write same log into messages.log, kernel.log etc...
2. If you have a univ. wifi with RADIUS, you most likely obtain lease every 15
min. this log (with level debug) goes into /var/log/dhcpcd.log.
3. Firewall logging is useful for network monitoring/debugging,
but /var/log/iptables.log will grow huge on a public network.

In cases 2 and 3 you can tell syslog to put the corresponding files
into /tmp/log (I assume /tmp is already tmpfs) since this info is not really
needed in the long term.

--
Leonid Isaev
GnuPG key: 0x164B5A6D
Fingerprint: C0DF 20D0 C075 C3F1 E1BE 775A A7AE F6CB 164B 5A6D
 

Thread Tools




All times are GMT. The time now is 08:17 PM.

VBulletin, Copyright ©2000 - 2014, Jelsoft Enterprises Ltd.
Content Relevant URLs by vBSEO ©2007, Crawlability, Inc.
Copyright ©2007 - 2008, www.linux-archive.org