Package: linux-2.6
Version: 2.6.32-38
Severity: important
Tags: squeeze



-- Package-specific info:
** Version:
Linux version 2.6.32-5-amd64 (Debian 2.6.32-38) (ben@decadent.org.uk) (gcc version 4.3.5 (Debian 4.3.5-4) ) #1 SMP Mon Oct 3 03:59:20 UTC 2011

-- System Information:
Debian Release: 6.0.3
APT prefers proposed-updates
APT policy: (992, 'proposed-updates'), (991, 'stable-updates'), (500, 'stable')
Architecture: amd64 (x86_64)

Kernel: Linux 2.6.32-5-amd64 (SMP w/4 CPU cores)
Locale: LANG=ru_RU.UTF-8, LC_CTYPE=ru_RU.UTF-8 (charmap=UTF-8)
Shell: /bin/sh linked to /bin/dash

-- debconf information excluded

The config option XEN_MAX_DOMAIN_MEMORY controls how much memory a Xen instance is seeing. The default for 64bit is 32GB, which is the reason that m2.4xlarge Amazon EC2 instances only report this amount of memory.
Please set this limit to 70GB as there is a known restriction for t1.micro instances at about 80GB.
Similar bug exists and Ubuntu where it's already fixed (https://bugs.launchpad.net/ubuntu/+source/linux-ec2/+bug/667796)



--
To UNSUBSCRIBE, email to debian-kernel-REQUEST@lists.debian.org
with a subject of "unsubscribe". Trouble? Contact listmaster@lists.debian.org
Archive: 20111012044608.5378.84714.reportbug@disconnect.dom ">http://lists.debian.org/20111012044608.5378.84714.reportbug@disconnect.dom

On Wed, 2011-10-12 at 08:46 +0300, Dmitry Musatov wrote:
> The config option XEN_MAX_DOMAIN_MEMORY controls how much memory a
> Xen instance is seeing. The default for 64bit is 32GB, which is the
> reason that m2.4xlarge Amazon EC2 instances only report this amount of
> memory.
> Please set this limit to 70GB as there is a known restriction for
> t1.micro instances at about 80GB.
> Similar bug exists and Ubuntu where it's already fixed
> (https://bugs.launchpad.net/ubuntu/+source/linux-ec2/+bug/667796)

Is this the sort of change we can consider making in a stable update?
I'm not at all sure, although my gut feeling is that it would be safe.

The 3.0 kernels in unstable (and testing?) right now have
XEN_MAX_DOMAIN_MEMORY=128, I think they use a more dynamic scheme for
this limit than the 2.6.32 kernels did too.

Ian.

--
Ian Campbell


Captain Penny's Law:
You can fool all of the people some of the
time, and some of the people all of the
time, but you can't fool mom.

On Wed, 2011-10-12 at 08:26 +0100, Ian Campbell wrote:
> On Wed, 2011-10-12 at 08:46 +0300, Dmitry Musatov wrote:
> > The config option XEN_MAX_DOMAIN_MEMORY controls how much memory a
> > Xen instance is seeing. The default for 64bit is 32GB, which is the
> > reason that m2.4xlarge Amazon EC2 instances only report this amount of
> > memory.
> > Please set this limit to 70GB as there is a known restriction for
> > t1.micro instances at about 80GB.
> > Similar bug exists and Ubuntu where it's already fixed
> > (https://bugs.launchpad.net/ubuntu/+source/linux-ec2/+bug/667796)
>
> Is this the sort of change we can consider making in a stable update?
> I'm not at all sure, although my gut feeling is that it would be safe.
[...]

I think so. But what is the trade-off? There must be some reason why
this isn't set to however many TB the kernel can support.

Ben.

--
Ben Hutchings
If at first you don't succeed, you're doing about average.

On Wed, 2011-10-12 at 14:11 +0100, Ben Hutchings wrote:
> On Wed, 2011-10-12 at 08:26 +0100, Ian Campbell wrote:
> > On Wed, 2011-10-12 at 08:46 +0300, Dmitry Musatov wrote:
> > > The config option XEN_MAX_DOMAIN_MEMORY controls how much memory a
> > > Xen instance is seeing. The default for 64bit is 32GB, which is the
> > > reason that m2.4xlarge Amazon EC2 instances only report this amount of
> > > memory.
> > > Please set this limit to 70GB as there is a known restriction for
> > > t1.micro instances at about 80GB.
> > > Similar bug exists and Ubuntu where it's already fixed
> > > (https://bugs.launchpad.net/ubuntu/+source/linux-ec2/+bug/667796)
> >
> > Is this the sort of change we can consider making in a stable update?
> > I'm not at all sure, although my gut feeling is that it would be safe.
> [...]
>
> I think so. But what is the trade-off? There must be some reason why
> this isn't set to however many TB the kernel can support.

It effects the amount of space set aside for the P2M table (the mapping
of physical to machine addresses). In the kernel in Squeeze this space
is statically reserved in BSS so increasing it will waste some more
memory, according to the Kconfig comment it is 1 page per GB.

In a more up to date kernel the space comes from BRK and is reclaimed if
it is not used, MAX_DOMAIN_MEMORY was bumped to default to 128G in the
same change.

Ian.
--
Ian Campbell
Current Noise: Devin Townsend - Namaste

The only way to learn a new programming language is by writing programs in it.
-- Brian Kernighan




--
To UNSUBSCRIBE, email to debian-kernel-REQUEST@lists.debian.org
with a subject of "unsubscribe". Trouble? Contact listmaster@lists.debian.org
Archive: 1318427918.21903.719.camel@zakaz.uk.xensource.com" >http://lists.debian.org/1318427918.21903.719.camel@zakaz.uk.xensource.com

On Wed, 2011-10-12 at 14:58 +0100, Ian Campbell wrote:
> On Wed, 2011-10-12 at 14:11 +0100, Ben Hutchings wrote:
> > On Wed, 2011-10-12 at 08:26 +0100, Ian Campbell wrote:
> > > On Wed, 2011-10-12 at 08:46 +0300, Dmitry Musatov wrote:
> > > > The config option XEN_MAX_DOMAIN_MEMORY controls how much memory a
> > > > Xen instance is seeing. The default for 64bit is 32GB, which is the
> > > > reason that m2.4xlarge Amazon EC2 instances only report this amount of
> > > > memory.
> > > > Please set this limit to 70GB as there is a known restriction for
> > > > t1.micro instances at about 80GB.
> > > > Similar bug exists and Ubuntu where it's already fixed
> > > > (https://bugs.launchpad.net/ubuntu/+source/linux-ec2/+bug/667796)
> > >
> > > Is this the sort of change we can consider making in a stable update?
> > > I'm not at all sure, although my gut feeling is that it would be safe.
> > [...]
> >
> > I think so. But what is the trade-off? There must be some reason why
> > this isn't set to however many TB the kernel can support.
>
> It effects the amount of space set aside for the P2M table (the mapping
> of physical to machine addresses). In the kernel in Squeeze this space
> is statically reserved in BSS so increasing it will waste some more
> memory, according to the Kconfig comment it is 1 page per GB.
>
> In a more up to date kernel the space comes from BRK and is reclaimed if
> it is not used, MAX_DOMAIN_MEMORY was bumped to default to 128G in the
> same change.

How intrusive is the change? Could we reasonably backport it?

Ben.

--
Ben Hutchings
Quantity is no substitute for quality, but it's the only one we've got.

On Thu, 2011-10-13 at 03:27 +0100, Ben Hutchings wrote:
> On Wed, 2011-10-12 at 14:58 +0100, Ian Campbell wrote:
> > On Wed, 2011-10-12 at 14:11 +0100, Ben Hutchings wrote:
> > > On Wed, 2011-10-12 at 08:26 +0100, Ian Campbell wrote:
> > > > On Wed, 2011-10-12 at 08:46 +0300, Dmitry Musatov wrote:
> > > > > The config option XEN_MAX_DOMAIN_MEMORY controls how much memory a
> > > > > Xen instance is seeing. The default for 64bit is 32GB, which is the
> > > > > reason that m2.4xlarge Amazon EC2 instances only report this amount of
> > > > > memory.
> > > > > Please set this limit to 70GB as there is a known restriction for
> > > > > t1.micro instances at about 80GB.
> > > > > Similar bug exists and Ubuntu where it's already fixed
> > > > > (https://bugs.launchpad.net/ubuntu/+source/linux-ec2/+bug/667796)
> > > >
> > > > Is this the sort of change we can consider making in a stable update?
> > > > I'm not at all sure, although my gut feeling is that it would be safe.
> > > [...]
> > >
> > > I think so. But what is the trade-off? There must be some reason why
> > > this isn't set to however many TB the kernel can support.
> >
> > It effects the amount of space set aside for the P2M table (the mapping
> > of physical to machine addresses). In the kernel in Squeeze this space
> > is statically reserved in BSS so increasing it will waste some more
> > memory, according to the Kconfig comment it is 1 page per GB.
> >
> > In a more up to date kernel the space comes from BRK and is reclaimed if
> > it is not used, MAX_DOMAIN_MEMORY was bumped to default to 128G in the
> > same change.
>
> How intrusive is the change? Could we reasonably backport it?

It was 58e05027b530 "xen: convert p2m to a 3 level tree" which I think
is too big. IIRC there was a bunch of subsequent fixups to it as well,
it was quite a subtle change.

Ian.

>
> Ben.
>

--
Ian Campbell


Aliquid melius quam pessimum optimum non est.

On Thu, 2011-10-13 at 07:05 +0100, Ian Campbell wrote:
> On Thu, 2011-10-13 at 03:27 +0100, Ben Hutchings wrote:
> > On Wed, 2011-10-12 at 14:58 +0100, Ian Campbell wrote:
> > > On Wed, 2011-10-12 at 14:11 +0100, Ben Hutchings wrote:
> > > > On Wed, 2011-10-12 at 08:26 +0100, Ian Campbell wrote:
> > > > > On Wed, 2011-10-12 at 08:46 +0300, Dmitry Musatov wrote:
> > > > > > The config option XEN_MAX_DOMAIN_MEMORY controls how much memory a
> > > > > > Xen instance is seeing. The default for 64bit is 32GB, which is the
> > > > > > reason that m2.4xlarge Amazon EC2 instances only report this amount of
> > > > > > memory.
> > > > > > Please set this limit to 70GB as there is a known restriction for
> > > > > > t1.micro instances at about 80GB.
> > > > > > Similar bug exists and Ubuntu where it's already fixed
> > > > > > (https://bugs.launchpad.net/ubuntu/+source/linux-ec2/+bug/667796)
> > > > >
> > > > > Is this the sort of change we can consider making in a stable update?
> > > > > I'm not at all sure, although my gut feeling is that it would be safe.
> > > > [...]
> > > >
> > > > I think so. But what is the trade-off? There must be some reason why
> > > > this isn't set to however many TB the kernel can support.
> > >
> > > It effects the amount of space set aside for the P2M table (the mapping
> > > of physical to machine addresses). In the kernel in Squeeze this space
> > > is statically reserved in BSS so increasing it will waste some more
> > > memory, according to the Kconfig comment it is 1 page per GB.
> > >
> > > In a more up to date kernel the space comes from BRK and is reclaimed if
> > > it is not used, MAX_DOMAIN_MEMORY was bumped to default to 128G in the
> > > same change.
> >
> > How intrusive is the change? Could we reasonably backport it?
>
> It was 58e05027b530 "xen: convert p2m to a 3 level tree" which I think
> is too big. IIRC there was a bunch of subsequent fixups to it as well,
> it was quite a subtle change.

You didn't directly answer the questions, but that sounds like 'fairly'
and 'no'.

If I understand correctly, the memory cost of expanding the table to
cover 70GB is (70GB - 32GB) * 4KB / 1GB = 156KB. Is that right?

Since we don't have a specific flavour to support EC2, and since some
people like to run domains with much less memory, I'm inclined to say
that this is 'wontfix' for squeeze. But I'm not sure just how small
they are likely to be (while still running Debian). Maybe the cost
isn't that significant.

Ben.

--
Ben Hutchings
No political challenge can be met by shopping. - George Monbiot

On Thu, 2011-10-13 at 07:05 +0100, Ian Campbell wrote:
> On Thu, 2011-10-13 at 03:27 +0100, Ben Hutchings wrote:
> > On Wed, 2011-10-12 at 14:58 +0100, Ian Campbell wrote:
> > > On Wed, 2011-10-12 at 14:11 +0100, Ben Hutchings wrote:
> > > > On Wed, 2011-10-12 at 08:26 +0100, Ian Campbell wrote:
> > > > > On Wed, 2011-10-12 at 08:46 +0300, Dmitry Musatov wrote:
> > > > > > The config option XEN_MAX_DOMAIN_MEMORY controls how much memory a
> > > > > > Xen instance is seeing. The default for 64bit is 32GB, which is the
> > > > > > reason that m2.4xlarge Amazon EC2 instances only report this amount of
> > > > > > memory.
> > > > > > Please set this limit to 70GB as there is a known restriction for
> > > > > > t1.micro instances at about 80GB.
> > > > > > Similar bug exists and Ubuntu where it's already fixed
> > > > > > (https://bugs.launchpad.net/ubuntu/+source/linux-ec2/+bug/667796)
> > > > >
> > > > > Is this the sort of change we can consider making in a stable update?
> > > > > I'm not at all sure, although my gut feeling is that it would be safe.
> > > > [...]
> > > >
> > > > I think so. But what is the trade-off? There must be some reason why
> > > > this isn't set to however many TB the kernel can support.
> > >
> > > It effects the amount of space set aside for the P2M table (the mapping
> > > of physical to machine addresses). In the kernel in Squeeze this space
> > > is statically reserved in BSS so increasing it will waste some more
> > > memory, according to the Kconfig comment it is 1 page per GB.
> > >
> > > In a more up to date kernel the space comes from BRK and is reclaimed if
> > > it is not used, MAX_DOMAIN_MEMORY was bumped to default to 128G in the
> > > same change.
> >
> > How intrusive is the change? Could we reasonably backport it?
>
> It was 58e05027b530 "xen: convert p2m to a 3 level tree" which I think
> is too big. IIRC there was a bunch of subsequent fixups to it as well,
> it was quite a subtle change.

You didn't directly answer the questions, but that sounds like 'fairly'
and 'no'.

If I understand correctly, the memory cost of expanding the table to
cover 70GB is (70GB - 32GB) * 4KB / 1GB = 156KB. Is that right?

Since we don't have a specific flavour to support EC2, and since some
people like to run domains with much less memory, I'm inclined to say
that this is 'wontfix' for squeeze. But I'm not sure just how small
they are likely to be (while still running Debian). Maybe the cost
isn't that significant.

Ben.

--
Ben Hutchings
No political challenge can be met by shopping. - George Monbiot

On Tue, 2011-10-18 at 04:40 +0100, Ben Hutchings wrote:
> On Thu, 2011-10-13 at 07:05 +0100, Ian Campbell wrote:
> > On Thu, 2011-10-13 at 03:27 +0100, Ben Hutchings wrote:
> > > On Wed, 2011-10-12 at 14:58 +0100, Ian Campbell wrote:
> > > > On Wed, 2011-10-12 at 14:11 +0100, Ben Hutchings wrote:
> > > > > On Wed, 2011-10-12 at 08:26 +0100, Ian Campbell wrote:
> > > > > > On Wed, 2011-10-12 at 08:46 +0300, Dmitry Musatov wrote:
> > > > > > > The config option XEN_MAX_DOMAIN_MEMORY controls how much memory a
> > > > > > > Xen instance is seeing. The default for 64bit is 32GB, which is the
> > > > > > > reason that m2.4xlarge Amazon EC2 instances only report this amount of
> > > > > > > memory.
> > > > > > > Please set this limit to 70GB as there is a known restriction for
> > > > > > > t1.micro instances at about 80GB.
> > > > > > > Similar bug exists and Ubuntu where it's already fixed
> > > > > > > (https://bugs.launchpad.net/ubuntu/+source/linux-ec2/+bug/667796)
> > > > > >
> > > > > > Is this the sort of change we can consider making in a stable update?
> > > > > > I'm not at all sure, although my gut feeling is that it would be safe.
> > > > > [...]
> > > > >
> > > > > I think so. But what is the trade-off? There must be some reason why
> > > > > this isn't set to however many TB the kernel can support.
> > > >
> > > > It effects the amount of space set aside for the P2M table (the mapping
> > > > of physical to machine addresses). In the kernel in Squeeze this space
> > > > is statically reserved in BSS so increasing it will waste some more
> > > > memory, according to the Kconfig comment it is 1 page per GB.
> > > >
> > > > In a more up to date kernel the space comes from BRK and is reclaimed if
> > > > it is not used, MAX_DOMAIN_MEMORY was bumped to default to 128G in the
> > > > same change.
> > >
> > > How intrusive is the change? Could we reasonably backport it?
> >
> > It was 58e05027b530 "xen: convert p2m to a 3 level tree" which I think
> > is too big. IIRC there was a bunch of subsequent fixups to it as well,
> > it was quite a subtle change.
>
> You didn't directly answer the questions, but that sounds like 'fairly'
> and 'no'.

Correct. Sorry, I thought "too big" covered both.

> If I understand correctly, the memory cost of expanding the table to
> cover 70GB is (70GB - 32GB) * 4KB / 1GB = 156KB. Is that right?

Yes.

> Since we don't have a specific flavour to support EC2,

One option might be to increase the limit only for the xen flavour and
leave the normal flavour where it is. That adds a rather unfortunate
matrix to the selection of which flavour to use though, ideally I would
prefer folks to be using the regular flavour in domU wherever possible.

> and since some
> people like to run domains with much less memory, I'm inclined to say
> that this is 'wontfix' for squeeze. But I'm not sure just how small
> they are likely to be (while still running Debian). Maybe the cost
> isn't that significant.

http://www.debian.org/releases/stable/amd64/ch03s04.html.en and
http://www.debian.org/releases/stable/i386/ch03s04.html.en both say the
minimum is 64M.

We are talking about going from 128KB to 280KB reserved for the p2m.
Which for a 64M machine is going from 0.20% to 0.43% of RAM overhead.

I'm not sure if <64M is realistic. I have a (32-bit, physical) machine I
use as a firewall which has 32M and apt-get and friends really do grind
along (it's also an old Pentium with a tiny disk, so there are other
factors in that).

I think we are only talking about the limit for a 64 bit guest? I would
guess that those are more unlikely to be given tiny amounts of RAM
compared with 32 bit.

Ian.

--
Ian Campbell
Current Noise: Monster Magnet - Bored With Sorcery

Your picture of the world often changes just before you get it into focus.

On Tue, 2011-10-18 at 10:57 +0100, Ian Campbell wrote:
> On Tue, 2011-10-18 at 04:40 +0100, Ben Hutchings wrote:
[...]
> > and since some
> > people like to run domains with much less memory, I'm inclined to say
> > that this is 'wontfix' for squeeze. But I'm not sure just how small
> > they are likely to be (while still running Debian). Maybe the cost
> > isn't that significant.
>
> http://www.debian.org/releases/stable/amd64/ch03s04.html.en and
> http://www.debian.org/releases/stable/i386/ch03s04.html.en both say the
> minimum is 64M.
>
> We are talking about going from 128KB to 280KB reserved for the p2m.
> Which for a 64M machine is going from 0.20% to 0.43% of RAM overhead.
>
> I'm not sure if <64M is realistic. I have a (32-bit, physical) machine I
> use as a firewall which has 32M and apt-get and friends really do grind
> along (it's also an old Pentium with a tiny disk, so there are other
> factors in that).
>
> I think we are only talking about the limit for a 64 bit guest? I would
> guess that those are more unlikely to be given tiny amounts of RAM
> compared with 32 bit.

Yes, that seems reasonable. Let's do it (but after -39).

Ben.

--
Ben Hutchings
73.46% of all statistics are made up.