Bug#637234: linux-image-3.0.0-1-686-pae: I/O errors using ext4 under xen (also affects ext3 as of linux-image-3.1.0-1-amd64 et al)
 

retitle 637234 linux-image-3.0.0-1-686-pae: I/O errors using ext4 under xen (also affects ext3 as of linux-image-3.1.0-1-amd64 et al)
thanks

[note that I've stripped off the CC to xen-devel in this comment, but
have no objection to it being put back if it makes sense]

On Wed, Sep 07, 2011 at 08:29:38AM -0400, Konrad Rzeszutek Wilk wrote:
> On Wed, Sep 07, 2011 at 02:51:04AM +0100, Ben Hutchings wrote:
> > On Mon, 2011-08-29 at 10:08 -0400, Konrad Rzeszutek Wilk wrote:
> > [...]
> > > Oh, I think I know _exactly_ what bug that is:
> > >
> > > This git commit:
> > > 280802657fb95c52bb5a35d43fea60351883b2af "xen/blkback: When writting barriers set the sector number to zero"
> > > has to be reverted. Specifically:
> > >
> > > commit 3f963cae3ef35d26fdd899c08797a598c5ca3e9b
> > > Author: Jeremy Fitzhardinge <jeremy.fitzhardinge@citrix.com>
> > > Date: Tue Jul 19 16:44:42 2011 -0700
> > >
> > > Revert "xen/blkback: When writting barriers set the sector number to zero..."
> > [...]
> > > and this one added:
> > >
> > > 25266338a41470a21e9b3974445be09e0640dda7
> > > xen/blkback: don't fail empty barrier requests
> > [...]
> >
> > Which repository are these in?
>
> Jeremy's: git://git.kernel.org/pub/scm/linux/kernel/git/jeremy/xen.git

This problem is also present with ext3 by default now, following the
barriers-by-default change in Linux 3.1 (I'm seeing the problems with
3.1.4-1 domU and squeeze dom0; the barrier=0 workaround works).

If I'm not confused, this message from Konrad is about the same problem,
and points to another possible fix:

<http://lists.xen.org/archives/html/xen-devel/2011-12/msg00306.html>

I suspect that over time this will hit a lot of people on domU upgrades
from squeeze to wheezy - and it needs to be fixed in the dom0 kernel, which
presumably means a point release update.

Cheers,
Dominic.

--
Dominic Hargreaves | http://www.larted.org.uk/~dom/
PGP key 5178E2A5 from the.earth.li (keyserver,web,email)



--
To UNSUBSCRIBE, email to debian-kernel-REQUEST@lists.debian.org
with a subject of "unsubscribe". Trouble? Contact listmaster@lists.debian.org
Archive: 20111211220245.GQ4460@urchin.earth.li">http://lists.debian.org/20111211220245.GQ4460@urchin.earth.li

Bug#637234: linux-image-3.0.0-1-686-pae: I/O errors using ext4 under xen (also affects ext3 as of linux-image-3.1.0-1-amd64 et al)
 

On Sun, 2011-12-11 at 22:02 +0000, Dominic Hargreaves wrote:
> retitle 637234 linux-image-3.0.0-1-686-pae: I/O errors using ext4 under xen (also affects ext3 as of linux-image-3.1.0-1-amd64 et al)
> thanks
>
> [note that I've stripped off the CC to xen-devel in this comment, but
> have no objection to it being put back if it makes sense]
>
> On Wed, Sep 07, 2011 at 08:29:38AM -0400, Konrad Rzeszutek Wilk wrote:
> > On Wed, Sep 07, 2011 at 02:51:04AM +0100, Ben Hutchings wrote:
> > > On Mon, 2011-08-29 at 10:08 -0400, Konrad Rzeszutek Wilk wrote:
> > > [...]
> > > > Oh, I think I know _exactly_ what bug that is:
> > > >
> > > > This git commit:
> > > > 280802657fb95c52bb5a35d43fea60351883b2af "xen/blkback: When writting barriers set the sector number to zero"
> > > > has to be reverted. Specifically:
> > > >
> > > > commit 3f963cae3ef35d26fdd899c08797a598c5ca3e9b
> > > > Author: Jeremy Fitzhardinge <jeremy.fitzhardinge@citrix.com>
> > > > Date: Tue Jul 19 16:44:42 2011 -0700
> > > >
> > > > Revert "xen/blkback: When writting barriers set the sector number to zero..."
> > > [...]
> > > > and this one added:
> > > >
> > > > 25266338a41470a21e9b3974445be09e0640dda7
> > > > xen/blkback: don't fail empty barrier requests
> > > [...]
> > >
> > > Which repository are these in?
> >
> > Jeremy's: git://git.kernel.org/pub/scm/linux/kernel/git/jeremy/xen.git
>
> This problem is also present with ext3 by default now, following the
> barriers-by-default change in Linux 3.1 (I'm seeing the problems with
> 3.1.4-1 domU and squeeze dom0; the barrier=0 workaround works).
>
> If I'm not confused, this message from Konrad is about the same problem,
> and points to another possible fix:
>
> <http://lists.xen.org/archives/html/xen-devel/2011-12/msg00306.html>

That fix looks plausible and is in any case correct in its own right so
I have added it to SVN for 2.6.32-40.

Ian.

>
> I suspect that over time this will hit a lot of people on domU upgrades
> from squeeze to wheezy - and it needs to be fixed in the dom0 kernel, which
> presumably means a point release update.
>
> Cheers,
> Dominic.
>
> --
> Dominic Hargreaves | http://www.larted.org.uk/~dom/
> PGP key 5178E2A5 from the.earth.li (keyserver,web,email)
>
>
>

--
Ian Campbell


Will it improve my CASH FLOW?

Bug#637234: linux-image-3.0.0-1-686-pae: I/O errors using ext4 under xen (also affects ext3 as of linux-image-3.1.0-1-amd64 et al)
 

found 637234 2.6.32-39
notfound 637234 3.0.0-1
thanks

On Mon, Dec 12, 2011 at 08:35:47AM +0000, Ian Campbell wrote:
> On Sun, 2011-12-11 at 22:02 +0000, Dominic Hargreaves wrote:

> > This problem is also present with ext3 by default now, following the
> > barriers-by-default change in Linux 3.1 (I'm seeing the problems with
> > 3.1.4-1 domU and squeeze dom0; the barrier=0 workaround works).
> >
> > If I'm not confused, this message from Konrad is about the same problem,
> > and points to another possible fix:
> >
> > <http://lists.xen.org/archives/html/xen-devel/2011-12/msg00306.html>
>
> That fix looks plausible and is in any case correct in its own right so
> I have added it to SVN for 2.6.32-40.

Great, thanks!

> > I suspect that over time this will hit a lot of people on domU upgrades
> > from squeeze to wheezy - and it needs to be fixed in the dom0 kernel, which
> > presumably means a point release update.

Marking 'found' accordingly.

--
Dominic Hargreaves | http://www.larted.org.uk/~dom/
PGP key 5178E2A5 from the.earth.li (keyserver,web,email)



--
To UNSUBSCRIBE, email to debian-kernel-REQUEST@lists.debian.org
with a subject of "unsubscribe". Trouble? Contact listmaster@lists.debian.org
Archive: 20111212224737.GE4460@urchin.earth.li">http://lists.debian.org/20111212224737.GE4460@urchin.earth.li

Bug#637234: linux-image-3.0.0-1-686-pae: I/O errors using ext4 under xen (also affects ext3 as of linux-image-3.1.0-1-amd64 et al)
 

package linux-2.6
notfound 637234 3.0.0-3
found 637234 3.1.0-1~experimental.1
found 637234 3.1.1-1
found 637234 3.1.8-2
found 637234 3.2.4-1
thanks

dom0
====

amd64 squeeze with
Linux version 2.6.32-5-xen-amd64 (Debian 2.6.32-39) (dannf@debian.org) (gcc version 4.3.5 (Debian 4.3.5-4) ) #1 SMP Thu Nov 3 05:42:31 UTC 2011

config
------

name = 'lindi2'
vcpus = '4'
memory = '1024'
disk = [ 'file:/local/xen/lindi2/disk.img,xvda,w' ]
vif = [ 'mac=00:01:01:99:80:02' ]
on_crash = 'restart'

domU
====

amd64 wheezy with
Linux version 3.1.0-1-amd64 (Debian 3.1.0-1~experimental.1) (waldi@debian.org) (gcc version 4.6.1 (Debian 4.6.1-16) ) #1 SMP Thu Nov 3 19:35:59 UTC 2011

(also occurs with 3.1.1-1, 3.1.8-2 and 3.2.4-1 but listing the first
version where the bug occurs)

Using ext3 for rootfs. dmesg has

...
blkfront: barrier: empty write xvda op failed
blkfront: xvda: barrier or flush: disabled
end_request: I/O error, dev xvda, sector 16519664
end_request: I/O error, dev xvda, sector 16519664
...

Full dmesg attached.


workaround
==========

Setting barrier=0 for / in fstab seems to help. Thanks Konrad for the
tip, now I can continue working :-)

Bug#637234: linux-image-3.0.0-1-686-pae: I/O errors using ext4 under xen (also affects ext3 as of linux-image-3.1.0-1-amd64 et al)
 

notfound 637234 3.2.6-1

notfound 637234 2.6.32-41

thanks



Timo & all,



This has been a bug in the dom0 kernel. If I've been following this
bug correctly, then the only thing about the 3.0+ kernels is that
they have barriers enabled by default, which we then disable as a
workaround. But this is supposed to work, and it was mishandled by
the kernel in dom0. I therefore see no point in marking this bug as
found in 3.0+ kernels used in domU's.



This bug was apparently fixed in 2.6.32-40 or 2.6.32-41, I waited
for 2.6.32-41 and now I'm running that kernel in my dom0. You have
to upgrade your dom0 to the latest kernel. I now have various wheezy
domU's with barriers enabled again, running with no issues.



Regards,



Gedalya





On 02/07/2012 05:17 AM, Timo Juhani Lindfors wrote:

package linux-2.6
notfound 637234 3.0.0-3
found 637234 3.1.0-1~experimental.1
found 637234 3.1.1-1
found 637234 3.1.8-2
found 637234 3.2.4-1
thanks

dom0
====

amd64 squeeze with
Linux version 2.6.32-5-xen-amd64 (Debian 2.6.32-39) (dannf@debian.org) (gcc version 4.3.5 (Debian 4.3.5-4) ) #1 SMP Thu Nov 3 05:42:31 UTC 2011

config
------

name = 'lindi2'
vcpus = '4'
memory = '1024'
disk = [ 'file:/local/xen/lindi2/disk.img,xvda,w' ]
vif = [ 'mac=00:01:01:99:80:02' ]


domU
====

amd64 wheezy with
Linux version 3.1.0-1-amd64 (Debian 3.1.0-1~experimental.1) (waldi@debian.org) (gcc version 4.6.1 (Debian 4.6.1-16) ) #1 SMP Thu Nov 3 19:35:59 UTC 2011

(also occurs with 3.1.1-1, 3.1.8-2 and 3.2.4-1 but listing the first
version where the bug occurs)

Using ext3 for rootfs. dmesg has

...
blkfront: barrier: empty write xvda op failed
blkfront: xvda: barrier or flush: disabled
end_request: I/O error, dev xvda, sector 16519664
end_request: I/O error, dev xvda, sector 16519664
...

Full dmesg attached.








workaround
==========

Setting barrier=0 for / in fstab seems to help. Thanks Konrad for the
tip, now I can continue working :-)