FAQ Search Today's Posts Mark Forums Read
» Video Reviews

» Linux Archive

Linux-archive is a website aiming to archive linux email lists and to make them easily accessible for linux users/developers.


» Sponsor

» Partners

» Sponsor

Go Back   Linux Archive > Redhat > Fedora Development

 
 
LinkBack Thread Tools
 
Old 10-10-2012, 08:54 PM
Lennart Poettering
 
Default replacing rsyslogd in minimal with journald

On Wed, 10.10.12 14:39, Chris Murphy (lists@colorremedies.com) wrote:

>
> On Oct 10, 2012, at 2:02 PM, Kay Sievers wrote:
>
> > Syslog is by fact today already an "add-on", and not a
> > required component, it is just installed by default today. I don't use
> > or run syslog on any of my boxes since quite a while.
>
> How is rsyslog properly disabled?
>
> sockets.target syslog.target rsyslog.service all seem related.

"systemctl disable rsyslog.service" should suffice.

Lennart

--
Lennart Poettering - Red Hat, Inc.
--
devel mailing list
devel@lists.fedoraproject.org
https://admin.fedoraproject.org/mailman/listinfo/devel
 
Old 10-10-2012, 09:17 PM
Chris Murphy
 
Default replacing rsyslogd in minimal with journald

On Oct 10, 2012, at 2:54 PM, Lennart Poettering wrote:

> On Wed, 10.10.12 14:39, Chris Murphy (lists@colorremedies.com) wrote:
>
>> How is rsyslog properly disabled?
>>
>> sockets.target syslog.target rsyslog.service all seem related.
>
> "systemctl disable rsyslog.service" should suffice.

I did that and now syslog.socket is angry, or at least its status is failed. I'm not sure if it's related, and if it's merely cosmetic.


Chris Murphy
--
devel mailing list
devel@lists.fedoraproject.org
https://admin.fedoraproject.org/mailman/listinfo/devel
 
Old 10-10-2012, 10:50 PM
Kevin Fenzi
 
Default replacing rsyslogd in minimal with journald

On Wed, 10 Oct 2012 22:02:26 +0200
Kay Sievers <kay@vrfy.org> wrote:

> On Wed, Oct 10, 2012 at 9:49 PM, Simo Sorce <simo@redhat.com> wrote:

...snip...

> > So make it really better and support time-based rotation. You don't
> > need to make time-based rotation the default, but you'll make a lot
> > of people happy to have the option.
>
> I really don't mind someone implementing a "maximum retention policy"
> for the journal, surely sounds useful for some setups, but I'm
> personally not really interested in implementing it.

Note there are more use cases than a "retention policy" type thing in
having time based log rotation.

"My laptop started acting up last tuesday, I should see whats in the
logs from then"

a) search each rotated journal file until you find last tuesday.
or
b) run journalctl on last tuesdays log since it was rotated daily and
you can clearly see what one is tuesdays.

"I'd like to run a daily report on my logs"

a) journalctl out the journal, figure out when the last day started,
cut things before that out.
or
b) journalctl after the daily rotate on the previous days journal.

"This thing might have messed up when I last booted... uptime shows 16
days"

a) Figure out what journal was from 16 days ago by hunting around.
or
b) journalctl out the one from 16 days ago

kevin
--
devel mailing list
devel@lists.fedoraproject.org
https://admin.fedoraproject.org/mailman/listinfo/devel
 
Old 10-10-2012, 11:18 PM
Chris Adams
 
Default replacing rsyslogd in minimal with journald

Once upon a time, Matthew Miller <mattdm@fedoraproject.org> said:
> On Wed, Oct 10, 2012 at 02:44:53PM -0400, Konstantin Ryabitsev wrote:
> > Well, hang on, Kay. My understanding was that we're trying to make
> > syslog an optional install in Fedora 18 (or is it 19?). If that is the
>
> The suggestion was to propose this as a feature for F19. I think there's
> some additional basic functionality we really need in place before that
> would be ready.

One additional thing related to log analysis: I have some logs that are
owned by different groups, and analysis tools that run under user
accounts (for example CGIs scanning for certain types of errors). How
does that work with journald?
--
Chris Adams <cmadams@hiwaay.net>
Systems and Network Administrator - HiWAAY Internet Services
I don't speak for anybody but myself - that's enough trouble.
--
devel mailing list
devel@lists.fedoraproject.org
https://admin.fedoraproject.org/mailman/listinfo/devel
 
Old 10-10-2012, 11:48 PM
Lennart Poettering
 
Default replacing rsyslogd in minimal with journald

On Wed, 10.10.12 16:50, Kevin Fenzi (kevin@scrye.com) wrote:

> "My laptop started acting up last tuesday, I should see whats in the
> logs from then"
>
> "I'd like to run a daily report on my logs"

These two are much better implemented via explicit time seeks. The
journal APIs support that just fine, journalctl currently
doesn't. However it's trivial to add that based on the lower level APIs,
the only thing that stopped me from doing that so far is that for that
we'd have to come up with a nice way to parse calendar timestamps, and I
want to be careful about that. that said the idea is to have two command
line args to journalctl where you can pass things such as:

$ journalctl --start-time=2012-10-01
...
$ journalctl --start-time=-5d
...
$ journalctl --start-time=2012-01-01 --end-time=2012-05-02
...

And this would do the right things. Since the journal will coalesce the
current journal and the rotated ones into one this will simply show you
everything that matches.

Of course the time expressions for this need to be powerful enough so
that people can trivially express things like "everything from today",
or "everything since two weeks ago" and suchlike.

> "This thing might have messed up when I last booted... uptime shows 16
> days"

For this we already have "journalctl -b" which only shows messages from
the current boot. We'll probably extend that later so that you can pass
"journalctl -b4" or so which would show you the messages from 4 boots
earlier only.

The takeaway here is that rotation is not a feature for finding
things. There are much better ways to find things and we should make
them available, and we can, because the backend allows that.

Lennart

--
Lennart Poettering - Red Hat, Inc.
--
devel mailing list
devel@lists.fedoraproject.org
https://admin.fedoraproject.org/mailman/listinfo/devel
 
Old 10-10-2012, 11:55 PM
Matthew Miller
 
Default replacing rsyslogd in minimal with journald

On Thu, Oct 11, 2012 at 01:48:07AM +0200, Lennart Poettering wrote:
> > "My laptop started acting up last tuesday, I should see whats in the
> > logs from then"
> > "I'd like to run a daily report on my logs"
> These two are much better implemented via explicit time seeks. The
> journal APIs support that just fine, journalctl currently
> doesn't. However it's trivial to add that based on the lower level APIs,
> the only thing that stopped me from doing that so far is that for that
> we'd have to come up with a nice way to parse calendar timestamps, and I
> want to be careful about that. that said the idea is to have two command
> line args to journalctl where you can pass things such as:

Not coincidentially, I filed an RFE bug for this yesterday:
https://bugzilla.redhat.com/show_bug.cgi?id=864672

> Of course the time expressions for this need to be powerful enough so
> that people can trivially express things like "everything from today",
> or "everything since two weeks ago" and suchlike.

+1 awesome.


--
Matthew Miller ☁☁☁ Fedora Cloud Architect ☁☁☁ <mattdm@fedoraproject.org>
--
devel mailing list
devel@lists.fedoraproject.org
https://admin.fedoraproject.org/mailman/listinfo/devel
 
Old 10-11-2012, 12:53 AM
Adam Williamson
 
Default replacing rsyslogd in minimal with journald

On Wed, 2012-10-10 at 14:37 -0400, Konstantin Ryabitsev wrote:
> On Wed, Oct 10, 2012 at 2:32 PM, Lennart Poettering
> <mzerqung@0pointer.de> wrote:
> >> Can journalctl send the logs via logwatch?
> >
> > Not sure I can parse this, but IIUC you are wondering whether logwatch
> > is compatible with the journal. Not to my knowledge, no. But adding this
> > should be fairly easy as the output of "journalctl" is a pixel-perfect
> > copy of the original format, so where it works on /var/log/messages it
> > should simply work on the output of journalctl and all should be good.
> >
> > Note however that with the capabilities of the journal it might be
> > interesting to add journal support to logwatch that goes beyond mere
> > compatibility. For example, tests such as "look for messages which are
> > claimed to come from PID xyz but actually came from uvw" and suchlike
> > would be really interesting to have. That information is not available
> > in the /var/log/messages format however...
>
> So, in other words, all our existing log analysis tools have to be
> modified if they are to be of any use in Fedora 18?

The signal seems to have been lost somewhere along the path, so apart
from anything else, no, because rsyslog is still installed by default in
F18, and the systemd journal doesn't do permanent logging by default
(/var/log/journal does not exist). rsyslog is still the primary system
logging mechanism in F18 and that is not going to change (he said
meaningfully, targeting the QA orbital laser on Lennart's home address.)

There is a proposal in this thread to stop installing rsyslog by default
and enable permanent logging by journal in F19, but that's just a
proposal so far, and does not affect F18.
--
Adam Williamson
Fedora QA Community Monkey
IRC: adamw | Twitter: AdamW_Fedora | identi.ca: adamwfedora
http://www.happyassassin.net

--
devel mailing list
devel@lists.fedoraproject.org
https://admin.fedoraproject.org/mailman/listinfo/devel
 
Old 10-11-2012, 02:23 AM
Kevin Fenzi
 
Default replacing rsyslogd in minimal with journald

On Thu, 11 Oct 2012 01:48:07 +0200
Lennart Poettering <mzerqung@0pointer.de> wrote:

> These two are much better implemented via explicit time seeks. The
> journal APIs support that just fine, journalctl currently
> doesn't. However it's trivial to add that based on the lower level
> APIs, the only thing that stopped me from doing that so far is that
> for that we'd have to come up with a nice way to parse calendar
> timestamps, and I want to be careful about that. that said the idea
> is to have two command line args to journalctl where you can pass
> things such as:
>
> $ journalctl --start-time=2012-10-01
> ...
> $ journalctl --start-time=-5d
> ...
> $ journalctl --start-time=2012-01-01 --end-time=2012-05-02
> ...
>
> And this would do the right things. Since the journal will coalesce
> the current journal and the rotated ones into one this will simply
> show you everything that matches.

Sounds great.

> Of course the time expressions for this need to be powerful enough so
> that people can trivially express things like "everything from today",
> or "everything since two weeks ago" and suchlike.

Yeah, I am reminded (pardon the pun) of the 'remind' program that did
this very well.

> > "This thing might have messed up when I last booted... uptime shows
> > 16 days"
>
> For this we already have "journalctl -b" which only shows messages
> from the current boot. We'll probably extend that later so that you
> can pass "journalctl -b4" or so which would show you the messages
> from 4 boots earlier only.

Excellent.

> The takeaway here is that rotation is not a feature for finding
> things. There are much better ways to find things and we should make
> them available, and we can, because the backend allows that.

Right, which is why I was trying to move to use cases over just asking
for time rotation. If these use cases can be solved better ways,
thats just fine with me.

kevin

--
devel mailing list
devel@lists.fedoraproject.org
https://admin.fedoraproject.org/mailman/listinfo/devel
 
Old 10-12-2012, 07:29 PM
Bill Nottingham
 
Default replacing rsyslogd in minimal with journald

Konstantin Ryabitsev (icon@fedoraproject.org) said:
> > Not sure I can parse this, but IIUC you are wondering whether logwatch
> > is compatible with the journal. Not to my knowledge, no. But adding this
> > should be fairly easy as the output of "journalctl" is a pixel-perfect
> > copy of the original format, so where it works on /var/log/messages it
> > should simply work on the output of journalctl and all should be good.
> >
> > Note however that with the capabilities of the journal it might be
> > interesting to add journal support to logwatch that goes beyond mere
> > compatibility. For example, tests such as "look for messages which are
> > claimed to come from PID xyz but actually came from uvw" and suchlike
> > would be really interesting to have. That information is not available
> > in the /var/log/messages format however...
>
> So, in other words, all our existing log analysis tools have to be
> modified if they are to be of any use in Fedora 18?

Right, you'll have to port them to understand CEE from updated rsyslog. HTH,
HAND. <- note: THIS IS A JOKE.

MORE SERIOUSLY....

There are a lot of usage cases that people want from their logging.

1) Administrators want their plain-text logs that they know and love (or at
least know and have gotten accustomed to) that they can use their normal
unix tools and their homegrown custom shell/awk/perl/python/whatever scripts
for parsing. (For the purposes of this discussion, consider logwatch one of
those homegrown things, as it basically is that writ large.)

2) System management authors would love to have a mechanism where they can
subscribe to particular alerts as they come in, without having to subscribe
to all messages, or try and parse the unstructured text of syslog

3) Application developers might want to be able to express stuff they log in
a more structured fashion rather than just:

"(function:line) bad juju happened here in frobnitz"

4) Administrators want to be able to do things like 'show me everything sshd
did/logged about', or 'show me what happened last Thursday, because I can
never get the hang of them.'

5) Standards People want to have messages in the new CEE format, so they can
use their new CEE tools on them and merge some of their homegrown tools.

6) Meanwhile, you've got the poor audit logger over there on the side doing its
own thing, and there are users who Really Like those audit logs.

And we've got a lot of technology going around. journald - that's
technology. rsyslog - that's technology. libumberlog & ceelog - that's
technology.

What we've got to do is take the usage cases we have, and the technology we
have, and get a coherent solution that covers them. And it's certainly not
clear at this point what that solution would be.

If people want CEE format logs, or plain text logs, maybe journald should
grow those as output formats. Or maybe rsyslog should produce those formats.
Maybe rsyslog should grow a journald plugin, so instead of duplicating some
of journald's code for associating entries with pid/exec/etc., it can read
the already annotated journal stream and add its own metadata & spit out
whatever formats it wants. (Maybe it already does this!) Maybe rsyslog or
journald should take over audit logging in some way.

But the point is, there's a lot of work in this space going on on all sides
(take ceelog, liumberlog, and journald - all relatively new bits of
technology touching portions of this space). And at this point I'd say it's
way too early to say that Fedora Shall Be XYZ, or to conversly say that
Fedora Shall Not Be XYZ. A full plan for hitting all the usage cases we
might want just isn't known. (Although it would be a lot easier to get there
if y'all would stop shouting AT & PAST each other.)

So no, you don't need to change anything for Fedora 18. rsyslog is there by
default, journald is there too if you want to look at that. And until we
actually have a Plan, rather than just Technology, I'm not sure why you'd
say that Fedora will do XYZ in F-19 either.

Well, you can probably say that Fedora 19 won't ship with sysklogd by
default; that should be safe.

Bill
--
devel mailing list
devel@lists.fedoraproject.org
https://admin.fedoraproject.org/mailman/listinfo/devel
 
Old 10-12-2012, 11:36 PM
Lennart Poettering
 
Default replacing rsyslogd in minimal with journald

On Fri, 12.10.12 15:29, Bill Nottingham (notting@redhat.com) wrote:

Heya,

> And we've got a lot of technology going around. journald - that's
> technology. rsyslog - that's technology. libumberlog & ceelog - that's
> technology.

THis really makes me wonder where CEE actually belongs in this. Is
anybody using this currently? What area is this supposed to cover that
is not already covered by the journal or rsyslog? Is there really room
for another format besides BSD syslog and journal records? So, what's
our story here with CEE?

> If people want CEE format logs, or plain text logs, maybe journald should
> grow those as output formats.

To me it appears that CEE isn't widely accepted so far (heck, not even
properly defined as multiple different vocabularies for fields are
floating around), and I am bit unsure where it really fits in the big
picture. I am a bit conservative in adding output formatting for CEE if
it isn't clear that there is a need for CEE, that it's going to stick
around for long and we actually have people using this.

> Or maybe rsyslog should produce those formats. Maybe rsyslog should
> grow a journald plugin, so instead of duplicating some of journald's
> code for associating entries with pid/exec/etc., it can read the
> already annotated journal stream and add its own metadata & spit out
> whatever formats it wants. (Maybe it already does this!)

Yes, this would certainly be useful. If rsyslog wants access to the full
data stream systemd generates then using our C APIs is a good choice, it
will get all meta data, and can process them the way they want.

> Maybe rsyslog or journald should take over audit logging in some way.

Since the audit logs contain a lot of useful data we definitely want to
acquire auditing as another input for the journal. In fact, Eric has
been working on kernel support to allow the journal to get a copy of the
audit stream without interfering with auditd.

Lennart

--
Lennart Poettering - Red Hat, Inc.
--
devel mailing list
devel@lists.fedoraproject.org
https://admin.fedoraproject.org/mailman/listinfo/devel
 

Thread Tools




All times are GMT. The time now is 10:40 AM.

VBulletin, Copyright ©2000 - 2014, Jelsoft Enterprises Ltd.
Content Relevant URLs by vBSEO ©2007, Crawlability, Inc.
Copyright 2007 - 2008, www.linux-archive.org