FAQ Search Today's Posts Mark Forums Read
» Video Reviews

» Linux Archive

Linux-archive is a website aiming to archive linux email lists and to make them easily accessible for linux users/developers.


» Sponsor

» Partners

» Sponsor

Go Back   Linux Archive > Ubuntu > Ubuntu Kernel Team

 
 
LinkBack Thread Tools
 
Old 05-02-2011, 06:05 PM
Julien Desfossez
 
Default Integrating LTTng in Ubuntu

Hello,

LTTng, the Linux Trace Toolkit Next Generation, is a project that aims
at producing a highly efficient full system tracing solution. It is
composed of several components to allow tracing of the kernel, of
userspace, trace viewing and analysis and trace streaming.

Ubuntu is more and more popular as a server-grade distribution and
having a comprehensive tracing infrastructure built into the kernel
would allow system administrators to investigate problems as they appear
without patching the kernel and rebooting the server.

For example, with LTTng you know really quickly what is the I/O
bandwidth (network or disk) associated with a particular process, and in
this process which file is the culprit for slowing down your whole
system. LTTng can also be used as a highly efficient userspace tracer
with UST. When the user-space and kernel-space traces are combined,
application developpers and sysadmins have a very detailed overview of
what is happening on their system.

Over the last months, the size of the LTTng kernel patchset has been
considerably shrinked for Linux distributions by removing all
LTTng-specific instrumentation from the tree.

You can find the latest patchset here [1].

Please note that the patches within this patchset are elected as the
first items from the LTTng project to eventually get into the mainline
Linux kernel. This includes TRACE_EVENT() changes, the LTTng trace
clocks and the Generic Ring Buffer Library. Please take into account
that most of those patches are really small patches that have been split
to facilitate mainlining by addressing them to the appropriate maintainers.

Also, you will notice that most of the bigger patches are actually new
functionalities that won't conflict with existing code base, because
they create files in new directories of the kernel tree.

The upcoming UDS would be a good timing to discuss the integration of
the new LTTng as an efficient tracer especially for the server flavour
of the Ubuntu kernel.

I submitted a blueprint [2] and following the discussion with Leann
Ogasawara I'm asking the Ubuntu Kernel-team their feedbacks and opinions
about making a full fledged blueprint/spec. I will be in Budapest next week.

The Linaro project already ships with LTTng. For the next Ubuntu
release, it would be really interesting to have a LTTng-ready kernel in
archive to see if it could become the default kernel in the next LTS.
Our mainlining expectations for the biggest patches are around september
2011.

Thanks,

Julien

[1]
http://lttng.org/files/lttng/patch-2.6.38.4-lttng-dev-snapshot-20110430.tar.bz2
[2] https://blueprints.launchpad.net/ubuntu/+spec/kernel-o-lttng

--
kernel-team mailing list
kernel-team@lists.ubuntu.com
https://lists.ubuntu.com/mailman/listinfo/kernel-team
 
Old 05-02-2011, 08:55 PM
Leann Ogasawara
 
Default Integrating LTTng in Ubuntu

On Mon, 2011-05-02 at 14:05 -0400, Julien Desfossez wrote:
> Hello,
>
> LTTng, the Linux Trace Toolkit Next Generation, is a project that aims
> at producing a highly efficient full system tracing solution. It is
> composed of several components to allow tracing of the kernel, of
> userspace, trace viewing and analysis and trace streaming.
>
> Ubuntu is more and more popular as a server-grade distribution and
> having a comprehensive tracing infrastructure built into the kernel
> would allow system administrators to investigate problems as they appear
> without patching the kernel and rebooting the server.
>
> For example, with LTTng you know really quickly what is the I/O
> bandwidth (network or disk) associated with a particular process, and in
> this process which file is the culprit for slowing down your whole
> system. LTTng can also be used as a highly efficient userspace tracer
> with UST. When the user-space and kernel-space traces are combined,
> application developpers and sysadmins have a very detailed overview of
> what is happening on their system.
>
> Over the last months, the size of the LTTng kernel patchset has been
> considerably shrinked for Linux distributions by removing all
> LTTng-specific instrumentation from the tree.
>
> You can find the latest patchset here [1].
>
> Please note that the patches within this patchset are elected as the
> first items from the LTTng project to eventually get into the mainline
> Linux kernel. This includes TRACE_EVENT() changes, the LTTng trace
> clocks and the Generic Ring Buffer Library. Please take into account
> that most of those patches are really small patches that have been split
> to facilitate mainlining by addressing them to the appropriate maintainers.
>
> Also, you will notice that most of the bigger patches are actually new
> functionalities that won't conflict with existing code base, because
> they create files in new directories of the kernel tree.
>
> The upcoming UDS would be a good timing to discuss the integration of
> the new LTTng as an efficient tracer especially for the server flavour
> of the Ubuntu kernel.
>
> I submitted a blueprint [2] and following the discussion with Leann
> Ogasawara I'm asking the Ubuntu Kernel-team their feedbacks and opinions
> about making a full fledged blueprint/spec. I will be in Budapest next week.

One of my concerns is the maintenance burden this brings upon the Ubuntu
Kernel Team. The LTTng patch set looks like it's 86 patches at the
moment. In comparison, our entire Ubuntu patch delta from the upstream
kernel when the Oneiric kernel opened was ~150 patches.

> The Linaro project already ships with LTTng. For the next Ubuntu
> release, it would be really interesting to have a LTTng-ready kernel in
> archive to see if it could become the default kernel in the next LTS.
> Our mainlining expectations for the biggest patches are around september
> 2011.

If the plan is to get these into mainline around the September 2011 time
frame, why not just re-target this spec/blueprint to the Oneiric+1 dev
cycle? We'd then have the patches incorporated with no additional
effort as they'd filter in via upstream. Is there a compelling reason
for us to have to carry these patches prior to them landing upstream?

Leann


--
kernel-team mailing list
kernel-team@lists.ubuntu.com
https://lists.ubuntu.com/mailman/listinfo/kernel-team
 
Old 05-03-2011, 04:09 AM
Julien Desfossez
 
Default Integrating LTTng in Ubuntu

On 11-05-02 04:55 PM, Leann Ogasawara wrote:
> On Mon, 2011-05-02 at 14:05 -0400, Julien Desfossez wrote:
>> Hello,
>>
>> LTTng, the Linux Trace Toolkit Next Generation, is a project that aims
>> at producing a highly efficient full system tracing solution. It is
>> composed of several components to allow tracing of the kernel, of
>> userspace, trace viewing and analysis and trace streaming.
>>
>> Ubuntu is more and more popular as a server-grade distribution and
>> having a comprehensive tracing infrastructure built into the kernel
>> would allow system administrators to investigate problems as they appear
>> without patching the kernel and rebooting the server.
>>
>> For example, with LTTng you know really quickly what is the I/O
>> bandwidth (network or disk) associated with a particular process, and in
>> this process which file is the culprit for slowing down your whole
>> system. LTTng can also be used as a highly efficient userspace tracer
>> with UST. When the user-space and kernel-space traces are combined,
>> application developpers and sysadmins have a very detailed overview of
>> what is happening on their system.
>>
>> Over the last months, the size of the LTTng kernel patchset has been
>> considerably shrinked for Linux distributions by removing all
>> LTTng-specific instrumentation from the tree.
>>
>> You can find the latest patchset here [1].
>>
>> Please note that the patches within this patchset are elected as the
>> first items from the LTTng project to eventually get into the mainline
>> Linux kernel. This includes TRACE_EVENT() changes, the LTTng trace
>> clocks and the Generic Ring Buffer Library. Please take into account
>> that most of those patches are really small patches that have been split
>> to facilitate mainlining by addressing them to the appropriate maintainers.
>>
>> Also, you will notice that most of the bigger patches are actually new
>> functionalities that won't conflict with existing code base, because
>> they create files in new directories of the kernel tree.
>>
>> The upcoming UDS would be a good timing to discuss the integration of
>> the new LTTng as an efficient tracer especially for the server flavour
>> of the Ubuntu kernel.
>>
>> I submitted a blueprint [2] and following the discussion with Leann
>> Ogasawara I'm asking the Ubuntu Kernel-team their feedbacks and opinions
>> about making a full fledged blueprint/spec. I will be in Budapest next week.
>
> One of my concerns is the maintenance burden this brings upon the Ubuntu
> Kernel Team. The LTTng patch set looks like it's 86 patches at the
> moment. In comparison, our entire Ubuntu patch delta from the upstream
> kernel when the Oneiric kernel opened was ~150 patches.
Yes but if you look closely you will see that most of the patches are
either small or add a new feature that doesn't conflict with the vanilla
kernel.
The 32 patches in the trace-event-semicolon-removal are split per file
they modify and usually modify very few lines. Moreover these patches
are being mainlined as we speak (really short ETA).
The lib-ring-buffer patches will enter in the mainline kernel as soon as
we make perf work with it (around september).
The trace-clock patches are architecture specific and work to mainline
these is in progress also (all the ARM related patches are already in
the Ubuntu kernel IIRC).

>> The Linaro project already ships with LTTng. For the next Ubuntu
>> release, it would be really interesting to have a LTTng-ready kernel in
>> archive to see if it could become the default kernel in the next LTS.
>> Our mainlining expectations for the biggest patches are around september
>> 2011.
>
> If the plan is to get these into mainline around the September 2011 time
> frame, why not just re-target this spec/blueprint to the Oneiric+1 dev
> cycle? We'd then have the patches incorporated with no additional
> effort as they'd filter in via upstream. Is there a compelling reason
> for us to have to carry these patches prior to them landing upstream?
Our hope is to have a complete working stack in Oneiric so that when the
next LTS comes up, it will be completely ready. As I said, our target is
mainly sysadmins, they will enjoy having a server LTS release with all
the tracing infrastructure they need fully tested, documented and ready,
that's why it could be interesting to have it enabled in the release
pre-LTS while we work on mainlining the remaining patches.
It would be great to have Oneiric with a LTTng kernel by default, but if
you think it's too much work, what about having an alternate source
kernel (like Linaro's) in Universe or have some kind of official PPA for
it ?

Thanks,

Julien

--
kernel-team mailing list
kernel-team@lists.ubuntu.com
https://lists.ubuntu.com/mailman/listinfo/kernel-team
 
Old 05-03-2011, 02:44 PM
Leann Ogasawara
 
Default Integrating LTTng in Ubuntu

On Tue, 2011-05-03 at 00:09 -0400, Julien Desfossez wrote:
> On 11-05-02 04:55 PM, Leann Ogasawara wrote:
> > On Mon, 2011-05-02 at 14:05 -0400, Julien Desfossez wrote:
> >> Hello,
> >>
> >> LTTng, the Linux Trace Toolkit Next Generation, is a project that aims
> >> at producing a highly efficient full system tracing solution. It is
> >> composed of several components to allow tracing of the kernel, of
> >> userspace, trace viewing and analysis and trace streaming.
> >>
> >> Ubuntu is more and more popular as a server-grade distribution and
> >> having a comprehensive tracing infrastructure built into the kernel
> >> would allow system administrators to investigate problems as they appear
> >> without patching the kernel and rebooting the server.
> >>
> >> For example, with LTTng you know really quickly what is the I/O
> >> bandwidth (network or disk) associated with a particular process, and in
> >> this process which file is the culprit for slowing down your whole
> >> system. LTTng can also be used as a highly efficient userspace tracer
> >> with UST. When the user-space and kernel-space traces are combined,
> >> application developpers and sysadmins have a very detailed overview of
> >> what is happening on their system.
> >>
> >> Over the last months, the size of the LTTng kernel patchset has been
> >> considerably shrinked for Linux distributions by removing all
> >> LTTng-specific instrumentation from the tree.
> >>
> >> You can find the latest patchset here [1].
> >>
> >> Please note that the patches within this patchset are elected as the
> >> first items from the LTTng project to eventually get into the mainline
> >> Linux kernel. This includes TRACE_EVENT() changes, the LTTng trace
> >> clocks and the Generic Ring Buffer Library. Please take into account
> >> that most of those patches are really small patches that have been split
> >> to facilitate mainlining by addressing them to the appropriate maintainers.
> >>
> >> Also, you will notice that most of the bigger patches are actually new
> >> functionalities that won't conflict with existing code base, because
> >> they create files in new directories of the kernel tree.
> >>
> >> The upcoming UDS would be a good timing to discuss the integration of
> >> the new LTTng as an efficient tracer especially for the server flavour
> >> of the Ubuntu kernel.
> >>
> >> I submitted a blueprint [2] and following the discussion with Leann
> >> Ogasawara I'm asking the Ubuntu Kernel-team their feedbacks and opinions
> >> about making a full fledged blueprint/spec. I will be in Budapest next week.
> >
> > One of my concerns is the maintenance burden this brings upon the Ubuntu
> > Kernel Team. The LTTng patch set looks like it's 86 patches at the
> > moment. In comparison, our entire Ubuntu patch delta from the upstream
> > kernel when the Oneiric kernel opened was ~150 patches.
> Yes but if you look closely you will see that most of the patches are
> either small or add a new feature that doesn't conflict with the vanilla
> kernel.
> The 32 patches in the trace-event-semicolon-removal are split per file
> they modify and usually modify very few lines. Moreover these patches
> are being mainlined as we speak (really short ETA).

Does "really short ETA" imply the patches will land in the 2.6.40
kernel? Are they already in a subsystem maintainer's tree?

> The lib-ring-buffer patches will enter in the mainline kernel as soon as
> we make perf work with it (around september).

How long do you think this work will take? Which mainline kernel would
you anticipate them to land in?

> The trace-clock patches are architecture specific and work to mainline
> these is in progress also (all the ARM related patches are already in
> the Ubuntu kernel IIRC).

Same question, which upstream kernel version do you expect the
trace-clock patches to land? 2.6.40? Also, I don't see any ARM related
trace-clock patches in the Ubuntu distro.

> >> The Linaro project already ships with LTTng. For the next Ubuntu
> >> release, it would be really interesting to have a LTTng-ready kernel in
> >> archive to see if it could become the default kernel in the next LTS.
> >> Our mainlining expectations for the biggest patches are around september
> >> 2011.

So the reason I keep inquiring about which upstream kernel you
anticipate patches to land is because it's a good possibility Oneiric's
kernel will at least be a 2.6.40 based kernel. If a good majority of
the patches land in 2.6.40 we can then re-evaluate what's left that
needs applied, if any. In my opinion, I think you effort is best spent
getting the patches applied upstream this cycle.

> > If the plan is to get these into mainline around the September 2011 time
> > frame, why not just re-target this spec/blueprint to the Oneiric+1 dev
> > cycle? We'd then have the patches incorporated with no additional
> > effort as they'd filter in via upstream. Is there a compelling reason
> > for us to have to carry these patches prior to them landing upstream?
> Our hope is to have a complete working stack in Oneiric so that when the
> next LTS comes up, it will be completely ready. As I said, our target is
> mainly sysadmins, they will enjoy having a server LTS release with all
> the tracing infrastructure they need fully tested, documented and ready,
> that's why it could be interesting to have it enabled in the release
> pre-LTS while we work on mainlining the remaining patches.
> It would be great to have Oneiric with a LTTng kernel by default, but if
> you think it's too much work, what about having an alternate source
> kernel (like Linaro's) in Universe or have some kind of official PPA for
> it ?

You are more than welcome to produce an LTTng kernel in Universe or
available for testing in a PPA but I can't commit any Ubuntu kernel team
resources for that.

Thanks,
Leann


--
kernel-team mailing list
kernel-team@lists.ubuntu.com
https://lists.ubuntu.com/mailman/listinfo/kernel-team
 
Old 05-03-2011, 08:04 PM
Julien Desfossez
 
Default Integrating LTTng in Ubuntu

On 11-05-03 10:44 AM, Leann Ogasawara wrote:
> On Tue, 2011-05-03 at 00:09 -0400, Julien Desfossez wrote:
>> On 11-05-02 04:55 PM, Leann Ogasawara wrote:
>>> On Mon, 2011-05-02 at 14:05 -0400, Julien Desfossez wrote:
>>>> Hello,
>>>>
>>>> LTTng, the Linux Trace Toolkit Next Generation, is a project that aims
>>>> at producing a highly efficient full system tracing solution. It is
>>>> composed of several components to allow tracing of the kernel, of
>>>> userspace, trace viewing and analysis and trace streaming.
>>>>
>>>> Ubuntu is more and more popular as a server-grade distribution and
>>>> having a comprehensive tracing infrastructure built into the kernel
>>>> would allow system administrators to investigate problems as they appear
>>>> without patching the kernel and rebooting the server.
>>>>
>>>> For example, with LTTng you know really quickly what is the I/O
>>>> bandwidth (network or disk) associated with a particular process, and in
>>>> this process which file is the culprit for slowing down your whole
>>>> system. LTTng can also be used as a highly efficient userspace tracer
>>>> with UST. When the user-space and kernel-space traces are combined,
>>>> application developpers and sysadmins have a very detailed overview of
>>>> what is happening on their system.
>>>>
>>>> Over the last months, the size of the LTTng kernel patchset has been
>>>> considerably shrinked for Linux distributions by removing all
>>>> LTTng-specific instrumentation from the tree.
>>>>
>>>> You can find the latest patchset here [1].
>>>>
>>>> Please note that the patches within this patchset are elected as the
>>>> first items from the LTTng project to eventually get into the mainline
>>>> Linux kernel. This includes TRACE_EVENT() changes, the LTTng trace
>>>> clocks and the Generic Ring Buffer Library. Please take into account
>>>> that most of those patches are really small patches that have been split
>>>> to facilitate mainlining by addressing them to the appropriate maintainers.
>>>>
>>>> Also, you will notice that most of the bigger patches are actually new
>>>> functionalities that won't conflict with existing code base, because
>>>> they create files in new directories of the kernel tree.
>>>>
>>>> The upcoming UDS would be a good timing to discuss the integration of
>>>> the new LTTng as an efficient tracer especially for the server flavour
>>>> of the Ubuntu kernel.
>>>>
>>>> I submitted a blueprint [2] and following the discussion with Leann
>>>> Ogasawara I'm asking the Ubuntu Kernel-team their feedbacks and opinions
>>>> about making a full fledged blueprint/spec. I will be in Budapest next week.
>>>
>>> One of my concerns is the maintenance burden this brings upon the Ubuntu
>>> Kernel Team. The LTTng patch set looks like it's 86 patches at the
>>> moment. In comparison, our entire Ubuntu patch delta from the upstream
>>> kernel when the Oneiric kernel opened was ~150 patches.
>> Yes but if you look closely you will see that most of the patches are
>> either small or add a new feature that doesn't conflict with the vanilla
>> kernel.
>> The 32 patches in the trace-event-semicolon-removal are split per file
>> they modify and usually modify very few lines. Moreover these patches
>> are being mainlined as we speak (really short ETA).
>
> Does "really short ETA" imply the patches will land in the 2.6.40
> kernel? Are they already in a subsystem maintainer's tree?
All I can tell you is that, they were posted yesterday and are already
being acked by the maintainers :
https://lkml.org/lkml/2011/5/2/350

>> The lib-ring-buffer patches will enter in the mainline kernel as soon as
>> we make perf work with it (around september).
>
> How long do you think this work will take? Which mainline kernel would
> you anticipate them to land in?
Probably Mathieu should answer this one.

>> The trace-clock patches are architecture specific and work to mainline
>> these is in progress also (all the ARM related patches are already in
>> the Ubuntu kernel IIRC).
>
> Same question, which upstream kernel version do you expect the
> trace-clock patches to land? 2.6.40? Also, I don't see any ARM related
> trace-clock patches in the Ubuntu distro.
The ARM related trace-clock patches are in Linaro kernel actually.

>>>> The Linaro project already ships with LTTng. For the next Ubuntu
>>>> release, it would be really interesting to have a LTTng-ready kernel in
>>>> archive to see if it could become the default kernel in the next LTS.
>>>> Our mainlining expectations for the biggest patches are around september
>>>> 2011.
>
> So the reason I keep inquiring about which upstream kernel you
> anticipate patches to land is because it's a good possibility Oneiric's
> kernel will at least be a 2.6.40 based kernel. If a good majority of
> the patches land in 2.6.40 we can then re-evaluate what's left that
> needs applied, if any. In my opinion, I think you effort is best spent
> getting the patches applied upstream this cycle.
I agree and that is our primary focus, the reason we would like to have
it "early" in Ubuntu is to make sure it will be in the next LTS.

The real number of patches to integrate for supported architectures in
Ubuntu is 43 patches, some of them are really just one liner and most of
create new files or add new functions.
Still I understand and agree that integrating out-of-tree patches has a
cost in maintenance. All I would like to make sure is that when the next
LTS comes up, we will have a good ground to ease the integration.

Thanks,

Julien

--
kernel-team mailing list
kernel-team@lists.ubuntu.com
https://lists.ubuntu.com/mailman/listinfo/kernel-team
 
Old 05-03-2011, 09:11 PM
Mathieu Desnoyers
 
Default Integrating LTTng in Ubuntu

* Julien Desfossez (ju@klipix.org) wrote:
> On 11-05-03 10:44 AM, Leann Ogasawara wrote:
> > On Tue, 2011-05-03 at 00:09 -0400, Julien Desfossez wrote:
> >> On 11-05-02 04:55 PM, Leann Ogasawara wrote:
> >>> On Mon, 2011-05-02 at 14:05 -0400, Julien Desfossez wrote:
> >>>> Hello,
> >>>>
> >>>> LTTng, the Linux Trace Toolkit Next Generation, is a project that aims
> >>>> at producing a highly efficient full system tracing solution. It is
> >>>> composed of several components to allow tracing of the kernel, of
> >>>> userspace, trace viewing and analysis and trace streaming.
> >>>>
> >>>> Ubuntu is more and more popular as a server-grade distribution and
> >>>> having a comprehensive tracing infrastructure built into the kernel
> >>>> would allow system administrators to investigate problems as they appear
> >>>> without patching the kernel and rebooting the server.
> >>>>
> >>>> For example, with LTTng you know really quickly what is the I/O
> >>>> bandwidth (network or disk) associated with a particular process, and in
> >>>> this process which file is the culprit for slowing down your whole
> >>>> system. LTTng can also be used as a highly efficient userspace tracer
> >>>> with UST. When the user-space and kernel-space traces are combined,
> >>>> application developpers and sysadmins have a very detailed overview of
> >>>> what is happening on their system.
> >>>>
> >>>> Over the last months, the size of the LTTng kernel patchset has been
> >>>> considerably shrinked for Linux distributions by removing all
> >>>> LTTng-specific instrumentation from the tree.
> >>>>
> >>>> You can find the latest patchset here [1].
> >>>>
> >>>> Please note that the patches within this patchset are elected as the
> >>>> first items from the LTTng project to eventually get into the mainline
> >>>> Linux kernel. This includes TRACE_EVENT() changes, the LTTng trace
> >>>> clocks and the Generic Ring Buffer Library. Please take into account
> >>>> that most of those patches are really small patches that have been split
> >>>> to facilitate mainlining by addressing them to the appropriate maintainers.
> >>>>
> >>>> Also, you will notice that most of the bigger patches are actually new
> >>>> functionalities that won't conflict with existing code base, because
> >>>> they create files in new directories of the kernel tree.
> >>>>
> >>>> The upcoming UDS would be a good timing to discuss the integration of
> >>>> the new LTTng as an efficient tracer especially for the server flavour
> >>>> of the Ubuntu kernel.
> >>>>
> >>>> I submitted a blueprint [2] and following the discussion with Leann
> >>>> Ogasawara I'm asking the Ubuntu Kernel-team their feedbacks and opinions
> >>>> about making a full fledged blueprint/spec. I will be in Budapest next week.
> >>>
> >>> One of my concerns is the maintenance burden this brings upon the Ubuntu
> >>> Kernel Team. The LTTng patch set looks like it's 86 patches at the
> >>> moment. In comparison, our entire Ubuntu patch delta from the upstream
> >>> kernel when the Oneiric kernel opened was ~150 patches.
> >> Yes but if you look closely you will see that most of the patches are
> >> either small or add a new feature that doesn't conflict with the vanilla
> >> kernel.
> >> The 32 patches in the trace-event-semicolon-removal are split per file
> >> they modify and usually modify very few lines. Moreover these patches
> >> are being mainlined as we speak (really short ETA).
> >
> > Does "really short ETA" imply the patches will land in the 2.6.40
> > kernel? Are they already in a subsystem maintainer's tree?
> All I can tell you is that, they were posted yesterday and are already
> being acked by the maintainers :
> https://lkml.org/lkml/2011/5/2/350

Yes. So it's not in a maintainer tree yet, but very close (received many
acked-by, strong support from Steven Rostedt). So this part could make
it into 2.6.40.

>
> >> The lib-ring-buffer patches will enter in the mainline kernel as soon as
> >> we make perf work with it (around september).
> >
> > How long do you think this work will take? Which mainline kernel would
> > you anticipate them to land in?
> Probably Mathieu should answer this one.

Probably .41 or .42: push happening during the .40-rc, aiming at the
merge window kicked off by the .40 release, which leads us to be merged
for .41 (if everything goes well), or .42 (pessimistic scenario).

>
> >> The trace-clock patches are architecture specific and work to mainline
> >> these is in progress also (all the ARM related patches are already in
> >> the Ubuntu kernel IIRC).
> >
> > Same question, which upstream kernel version do you expect the
> > trace-clock patches to land? 2.6.40? Also, I don't see any ARM related
> > trace-clock patches in the Ubuntu distro.
> The ARM related trace-clock patches are in Linaro kernel actually.

Yes.

>
> >>>> The Linaro project already ships with LTTng. For the next Ubuntu
> >>>> release, it would be really interesting to have a LTTng-ready kernel in
> >>>> archive to see if it could become the default kernel in the next LTS.
> >>>> Our mainlining expectations for the biggest patches are around september
> >>>> 2011.
> >
> > So the reason I keep inquiring about which upstream kernel you
> > anticipate patches to land is because it's a good possibility Oneiric's
> > kernel will at least be a 2.6.40 based kernel. If a good majority of
> > the patches land in 2.6.40 we can then re-evaluate what's left that
> > needs applied, if any. In my opinion, I think you effort is best spent
> > getting the patches applied upstream this cycle.

As I described above, I don't think porting Perf to the generic ring
buffer library is realistically going to happen for 2.6.40. But on the
other hand, the generic ring buffer patches are very much independent
from the rest of the kernel, so they would not be a big burden to
support.

Thank you,

Mathieu

> I agree and that is our primary focus, the reason we would like to have
> it "early" in Ubuntu is to make sure it will be in the next LTS.
>
> The real number of patches to integrate for supported architectures in
> Ubuntu is 43 patches, some of them are really just one liner and most of
> create new files or add new functions.
> Still I understand and agree that integrating out-of-tree patches has a
> cost in maintenance. All I would like to make sure is that when the next
> LTS comes up, we will have a good ground to ease the integration.
>
> Thanks,
>
> Julien

--
Mathieu Desnoyers
Operating System Efficiency R&D Consultant
EfficiOS Inc.
http://www.efficios.com

--
kernel-team mailing list
kernel-team@lists.ubuntu.com
https://lists.ubuntu.com/mailman/listinfo/kernel-team
 
Old 05-04-2011, 02:09 PM
Pete Graner
 
Default Integrating LTTng in Ubuntu

On 05/03/2011 04:04 PM, Julien Desfossez wrote:

On 11-05-03 10:44 AM, Leann Ogasawara wrote:

On Tue, 2011-05-03 at 00:09 -0400, Julien Desfossez wrote:

On 11-05-02 04:55 PM, Leann Ogasawara wrote:

On Mon, 2011-05-02 at 14:05 -0400, Julien Desfossez wrote:

Hello,

LTTng, the Linux Trace Toolkit Next Generation, is a project that aims
at producing a highly efficient full system tracing solution. It is
composed of several components to allow tracing of the kernel, of
userspace, trace viewing and analysis and trace streaming.

Ubuntu is more and more popular as a server-grade distribution and
having a comprehensive tracing infrastructure built into the kernel
would allow system administrators to investigate problems as they appear
without patching the kernel and rebooting the server.

For example, with LTTng you know really quickly what is the I/O
bandwidth (network or disk) associated with a particular process, and in
this process which file is the culprit for slowing down your whole
system. LTTng can also be used as a highly efficient userspace tracer
with UST. When the user-space and kernel-space traces are combined,
application developpers and sysadmins have a very detailed overview of
what is happening on their system.

Over the last months, the size of the LTTng kernel patchset has been
considerably shrinked for Linux distributions by removing all
LTTng-specific instrumentation from the tree.

You can find the latest patchset here [1].

Please note that the patches within this patchset are elected as the
first items from the LTTng project to eventually get into the mainline
Linux kernel. This includes TRACE_EVENT() changes, the LTTng trace
clocks and the Generic Ring Buffer Library. Please take into account
that most of those patches are really small patches that have been split
to facilitate mainlining by addressing them to the appropriate maintainers.

Also, you will notice that most of the bigger patches are actually new
functionalities that won't conflict with existing code base, because
they create files in new directories of the kernel tree.

The upcoming UDS would be a good timing to discuss the integration of
the new LTTng as an efficient tracer especially for the server flavour
of the Ubuntu kernel.

I submitted a blueprint [2] and following the discussion with Leann
Ogasawara I'm asking the Ubuntu Kernel-team their feedbacks and opinions
about making a full fledged blueprint/spec. I will be in Budapest next week.


One of my concerns is the maintenance burden this brings upon the Ubuntu
Kernel Team. The LTTng patch set looks like it's 86 patches at the
moment. In comparison, our entire Ubuntu patch delta from the upstream
kernel when the Oneiric kernel opened was ~150 patches.

Yes but if you look closely you will see that most of the patches are
either small or add a new feature that doesn't conflict with the vanilla
kernel.
The 32 patches in the trace-event-semicolon-removal are split per file
they modify and usually modify very few lines. Moreover these patches
are being mainlined as we speak (really short ETA).


Does "really short ETA" imply the patches will land in the 2.6.40
kernel? Are they already in a subsystem maintainer's tree?

All I can tell you is that, they were posted yesterday and are already
being acked by the maintainers :
https://lkml.org/lkml/2011/5/2/350


The lib-ring-buffer patches will enter in the mainline kernel as soon as
we make perf work with it (around september).


How long do you think this work will take? Which mainline kernel would
you anticipate them to land in?

Probably Mathieu should answer this one.


The trace-clock patches are architecture specific and work to mainline
these is in progress also (all the ARM related patches are already in
the Ubuntu kernel IIRC).


Same question, which upstream kernel version do you expect the
trace-clock patches to land? 2.6.40? Also, I don't see any ARM related
trace-clock patches in the Ubuntu distro.

The ARM related trace-clock patches are in Linaro kernel actually.


The Linaro project already ships with LTTng. For the next Ubuntu
release, it would be really interesting to have a LTTng-ready kernel in
archive to see if it could become the default kernel in the next LTS.
Our mainlining expectations for the biggest patches are around september
2011.


So the reason I keep inquiring about which upstream kernel you
anticipate patches to land is because it's a good possibility Oneiric's
kernel will at least be a 2.6.40 based kernel. If a good majority of
the patches land in 2.6.40 we can then re-evaluate what's left that
needs applied, if any. In my opinion, I think you effort is best spent
getting the patches applied upstream this cycle.

I agree and that is our primary focus, the reason we would like to have
it "early" in Ubuntu is to make sure it will be in the next LTS.

The real number of patches to integrate for supported architectures in
Ubuntu is 43 patches, some of them are really just one liner and most of
create new files or add new functions.
Still I understand and agree that integrating out-of-tree patches has a
cost in maintenance. All I would like to make sure is that when the next
LTS comes up, we will have a good ground to ease the integration.


Yes it is very expensive. We carry a high maintenance overhead for
supporting non-LTS releases & 3 active LTS releases. Every patch that is
not in mainline has to be carried and fixed up quite frequently
depending where in the tree it is. Additionaly we have to take extra
care with CVEs to make sure we are non vulnerable due to extra patches.
Our policy is only to do it for items that directly benefit Ubuntu where
we are willing to pay the extra cost aka aufs used for live cd's.


Thanks
--
Pete Graner <pgraner@canonical.com>
Manager
Ubuntu Kernel Team
Canonical Ltd. http://www.canonical.com/

--
kernel-team mailing list
kernel-team@lists.ubuntu.com
https://lists.ubuntu.com/mailman/listinfo/kernel-team
 

Thread Tools




All times are GMT. The time now is 09:35 AM.

VBulletin, Copyright ©2000 - 2014, Jelsoft Enterprises Ltd.
Content Relevant URLs by vBSEO ©2007, Crawlability, Inc.
Copyright 2007 - 2008, www.linux-archive.org