FAQ Search Today's Posts Mark Forums Read
» Video Reviews

» Linux Archive

Linux-archive is a website aiming to archive linux email lists and to make them easily accessible for linux users/developers.


» Sponsor

» Partners

» Sponsor

Go Back   Linux Archive > CentOS > CentOS

 
 
LinkBack Thread Tools
 
Old 01-01-2010, 04:09 PM
Frank Cox
 
Default which program/command (perl, vi, or sed) is better

On Fri, 2010-01-01 at 08:45 -0800, ann kok wrote:
> Hi
>
> I have apache log file around 7.6G and record half year
>
> Which program/command (perl, vi, or sed) is better to extract the data by date?

Your question is a little too general. You want to extract what portion
of the data, into what?

If you want a complete record, you apparently have one already. If you
want to keep everything, just run it through bzip2 before storing it.

If you just want reports, look at something like webalizer.

If you want something "custom", you will have to decide exactly what you
want. Perl or sed or grep could probably extract it, or you could make
a program in python or C or whatever.

But the first task is to define exactly what you want.

--
MELVILLE THEATRE ~ Melville Sask ~ http://www.melvilletheatre.com

_______________________________________________
CentOS mailing list
CentOS@centos.org
http://lists.centos.org/mailman/listinfo/centos
 
Old 01-01-2010, 04:14 PM
Les Mikesell
 
Default which program/command (perl, vi, or sed) is better

ann kok wrote:
> Hi
>
> I have apache log file around 7.6G and record half year
>
> Which program/command (perl, vi, or sed) is better to extract the data by date? and finally I can remove that big file and still keep the record.
>
> I have problem when using vi and it uses up the server memory
>
> Thank you for your help
>

If the extraction is simple regex matches you could use sed, but I'd recommend
perl because it has additional features that you might need if the program
becomes more complex and it is easier to write in the first place.

But first I'd check to see if any of the available log processing programs will
already do what you want. If you are looking for summaries with counts by
page/time interval/client IP, etc. they may do what you want. I like analog
because it is very fast and can deal with multiple files even if the times
overlap and can uncompress them on the fly http://www.analog.cx/ or download rpm
from http://www.iddl.vt.edu/~jackie/analog/. Or you might like webalizer if
you just have one server/file.

--
Les Mikesell
lesmikesell@gmail.com
_______________________________________________
CentOS mailing list
CentOS@centos.org
http://lists.centos.org/mailman/listinfo/centos
 
Old 01-01-2010, 10:52 PM
Néstor
 
Default which program/command (perl, vi, or sed) is better

I have done similar programs in Perl (you can use PHP also) where, when I read the file in and using regexI select the records that I want to keep.
Feliz Año *:-)


On Fri, Jan 1, 2010 at 9:14 AM, Les Mikesell <lesmikesell@gmail.com> wrote:

ann kok wrote:

> Hi

>

> I have apache log file around 7.6G and record half year

>

> Which program/command (perl, vi, or sed) is better to extract the data by date? and finally I can remove that big file and still keep the record.

>

> I have problem when using vi and it uses up the server memory

>

> Thank you for your help

>



If the extraction is simple regex matches you could use sed, but I'd recommend

perl because it has additional features that you might need if the program

becomes more complex and it is easier to write in the first place.



But first I'd check to see if any of the available log processing programs will

already do what you want. *If you are looking for summaries with counts by

page/time interval/client IP, etc. they may do what you want. *I like analog

because it is very fast and can deal with multiple files even if the times

overlap and can uncompress them on the fly http://www.analog.cx/ or download rpm

from http://www.iddl.vt.edu/~jackie/analog/. * Or you might like webalizer if

you just have one server/file.



--

* Les Mikesell

* *lesmikesell@gmail.com

_______________________________________________

CentOS mailing list

CentOS@centos.org

http://lists.centos.org/mailman/listinfo/centos



_______________________________________________
CentOS mailing list
CentOS@centos.org
http://lists.centos.org/mailman/listinfo/centos
 
Old 01-01-2010, 11:08 PM
"Les Bell"
 
Default which program/command (perl, vi, or sed) is better

ann kok <oiyankok@yahoo.ca> wrote:

>>
Which program/command (perl, vi, or sed) is better to extract the data by
date?
<<

The book "Perl for System Administration ", (David N. Blank-Edelman,
O'Reilly, 2000) has a chapter on using Perl to analyse log files, rotate
and compress them, etc. Luckily, this chapter is the "free sample" they
provide online - see
http://oreilly.com/catalog/perlsysadm/chapter/ch09.html

Sorted.

I mean, problem solved.

Best,

--- Les Bell
[http://www.lesbell.com.au]
Tel: +61 2 9451 1144


_______________________________________________
CentOS mailing list
CentOS@centos.org
http://lists.centos.org/mailman/listinfo/centos
 

Thread Tools




All times are GMT. The time now is 11:54 PM.

VBulletin, Copyright ©2000 - 2014, Jelsoft Enterprises Ltd.
Content Relevant URLs by vBSEO ©2007, Crawlability, Inc.
Copyright ©2007 - 2008, www.linux-archive.org