FAQ Search Today's Posts Mark Forums Read
» Video Reviews

» Linux Archive

Linux-archive is a website aiming to archive linux email lists and to make them easily accessible for linux users/developers.


» Sponsor

» Partners

» Sponsor

Go Back   Linux Archive > Redhat > Fedora Infrastructure

 
 
LinkBack Thread Tools
 
Old 11-24-2007, 05:54 PM
"Michael Yingbull"
 
Default Log analyzer improvements, ticket #226

Hi all,

I'm following up from ticket #226, which is tracking improvements to the log analyzer system.
This would be what analyzers the logs on lockbox, which is the syslog host for infrastructure machines:

https://hosted.fedoraproject.org/projects/fedora-infrastructure/ticket/226

I wanted to capture what we wanted the new analyzer to do.**

Main feedback I had from discussion in #fedora-admin was a need for more signal, less noise:
the current 'analyzed' logs were too verbose and had too much cruft.

Did I capture that requirement?**

Are there other requirements besides improving the presentation?
Anything else that people feel they need from the log analyzer that they aren't getting?

Currently Epylog is used - I did some looking around, and I'm not seeing something that looks like its any better.

If someone knows another open source log analyzer they think would be much better, I'd like to hear.
Else, my plan is to continue with Epylog, reconfigure it... and if really needed to get what we need, patch it and contribute upstream.


Thanks all, hope everyone is having a good weekend.

Cheers,
Michael



_______________________________________________
Fedora-infrastructure-list mailing list
Fedora-infrastructure-list@redhat.com
https://www.redhat.com/mailman/listinfo/fedora-infrastructure-list
 
Old 11-26-2007, 01:33 PM
Mike McGrath
 
Default Log analyzer improvements, ticket #226

Michael Yingbull wrote:

Hi all,

I'm following up from ticket #226, which is tracking improvements to the log
analyzer system.
This would be what analyzers the logs on lockbox, which is the syslog host
for infrastructure machines:
https://hosted.fedoraproject.org/projects/fedora-infrastructure/ticket/226

I wanted to capture what we wanted the new analyzer to do.
Main feedback I had from discussion in #fedora-admin was a need for more
signal, less noise:
the current 'analyzed' logs were too verbose and had too much cruft.

Did I capture that requirement?



I think this is the biggest thing. Obviously we don't want to /dev/null
log lines but at the same time the current format is pretty useless to
us. I guess it might be best to do as much cleanup as possible and then
see where things are.


-Mike

_______________________________________________
Fedora-infrastructure-list mailing list
Fedora-infrastructure-list@redhat.com
https://www.redhat.com/mailman/listinfo/fedora-infrastructure-list
 
Old 11-26-2007, 01:35 PM
seth vidal
 
Default Log analyzer improvements, ticket #226

On Mon, 2007-11-26 at 08:33 -0600, Mike McGrath wrote:
> Michael Yingbull wrote:
> > Hi all,
> >
> > I'm following up from ticket #226, which is tracking improvements to the log
> > analyzer system.
> > This would be what analyzers the logs on lockbox, which is the syslog host
> > for infrastructure machines:
> > https://hosted.fedoraproject.org/projects/fedora-infrastructure/ticket/226
> >
> > I wanted to capture what we wanted the new analyzer to do.
> > Main feedback I had from discussion in #fedora-admin was a need for more
> > signal, less noise:
> > the current 'analyzed' logs were too verbose and had too much cruft.
> >
> > Did I capture that requirement?
> >
>
> I think this is the biggest thing. Obviously we don't want to /dev/null
> log lines but at the same time the current format is pretty useless to
> us. I guess it might be best to do as much cleanup as possible and then
> see where things are.

Actually, there's a huge portion of what is in the current logs that
needs to either:
1. be dumped out by epylog's weeder
2. be stopped from occurring on the system generating the message.

Michael, if you need any assistance with this, let me know, I have a
fair bit of experience adding weedlists to epylog.

-sv


_______________________________________________
Fedora-infrastructure-list mailing list
Fedora-infrastructure-list@redhat.com
https://www.redhat.com/mailman/listinfo/fedora-infrastructure-list
 
Old 11-26-2007, 01:42 PM
Mike McGrath
 
Default Log analyzer improvements, ticket #226

seth vidal wrote:

On Mon, 2007-11-26 at 08:33 -0600, Mike McGrath wrote:


Michael Yingbull wrote:


Hi all,

I'm following up from ticket #226, which is tracking improvements to the log
analyzer system.
This would be what analyzers the logs on lockbox, which is the syslog host
for infrastructure machines:
https://hosted.fedoraproject.org/projects/fedora-infrastructure/ticket/226

I wanted to capture what we wanted the new analyzer to do.
Main feedback I had from discussion in #fedora-admin was a need for more
signal, less noise:
the current 'analyzed' logs were too verbose and had too much cruft.

Did I capture that requirement?


I think this is the biggest thing. Obviously we don't want to /dev/null
log lines but at the same time the current format is pretty useless to
us. I guess it might be best to do as much cleanup as possible and then
see where things are.



Actually, there's a huge portion of what is in the current logs that
needs to either:
1. be dumped out by epylog's weeder
2. be stopped from occurring on the system generating the message.

Michael, if you need any assistance with this, let me know, I have a
fair bit of experience adding weedlists to epylog.



2) would be more favored by me where possible.

-Mike

_______________________________________________
Fedora-infrastructure-list mailing list
Fedora-infrastructure-list@redhat.com
https://www.redhat.com/mailman/listinfo/fedora-infrastructure-list
 
Old 11-26-2007, 02:40 PM
seth vidal
 
Default Log analyzer improvements, ticket #226

On Mon, 2007-11-26 at 08:42 -0600, Mike McGrath wrote:

> 2) would be more favored by me where possible.
>

No problem. From todays' report a couple of things we can do:

1. remove all user failure reports. They don't do us any good and
they're always ssh bruteforce attacks. Denyhosts will do its thing, or
not, but we can't be told about them all the time.

2. weed out pretty much everything beginning with:
rsyncd - informational messages about rsync processes - not useful
puppetd - notices on what is or is not done - not useful, either
- if we can turn off the syslog component of this and only have
this in the local puppet logs that'd be fine
ntpd - garbage noise - not useful for a log report
git-daemon - do I really need to explain why we can nuke this?


3. all of these lines:
crond[19403]: pam_unix(crond:session): session closed for user root

iirc, there is a new login module which handles these

4. puppetmasterd* - these appear to be errors/warnings from
puppetmasterd - these need to be fixed.

pruning out the items in 2 alone will nuke the better part of this
logreport.

-sv


_______________________________________________
Fedora-infrastructure-list mailing list
Fedora-infrastructure-list@redhat.com
https://www.redhat.com/mailman/listinfo/fedora-infrastructure-list
 

Thread Tools




All times are GMT. The time now is 04:55 PM.

VBulletin, Copyright ©2000 - 2014, Jelsoft Enterprises Ltd.
Content Relevant URLs by vBSEO ©2007, Crawlability, Inc.
Copyright 2007 - 2008, www.linux-archive.org