FAQ Search Today's Posts Mark Forums Read
» Video Reviews

» Linux Archive

Linux-archive is a website aiming to archive linux email lists and to make them easily accessible for linux users/developers.


» Sponsor

» Partners

» Sponsor


 
 
LinkBack Thread Tools
 
Old 09-21-2011, 04:40 PM
Michael Mol
 
Default Bcache

Has anyone played with Bcache? (Volker, I'm looking sideways at you...)

http://bcache.evilpiepirate.org/

--
:wq
 
Old 09-21-2011, 05:53 PM
Volker Armin Hemmann
 
Default Bcache

Am Mittwoch 21 September 2011, 12:40:14 schrieb Michael Mol:
> Has anyone played with Bcache? (Volker, I'm looking sideways at you...)
>
> http://bcache.evilpiepirate.org/

hm, no updates to the website for almost a year and a gb+ git clone, no
patches. I pass.
--
#163933
 
Old 03-14-2012, 02:53 PM
Vivek Goyal
 
Default Bcache

On Wed, Mar 14, 2012 at 09:32:28AM -0400, Kent Overstreet wrote:
> I'm already registered to attend, but would it be too late in the
> process to give a talk? I'd like to give a short talk about bcache, what
> it does and where it's going (more than just caching).

[CCing dm-devel list]

I am curious if you considered writing a device mapper driver for this? If
yes, why that is not a good choice. It seems to be stacked device and device
mapper should be good at that. All the configuration through sysfs seems
little odd to me.

On a side note, I was playing with bcache a bit. I tried to register the
cache device and it crashes. (I guess I should post this on relevant mailing
list).

# echo /dev/sdc > /sys/fs/bcache/register

[ 6758.314093] Pid: 0, comm: kworker/0:1 Not tainted 3.1.0-bcache+ #2
Hewlett-Packard HP xw6600 Workstation/0A9Ch
[ 6758.314093] RIP: 0010:[<ffffffff8146625b>] [<ffffffff8146625b>]
closure_put+0x5b/0xe0
[ 6758.314093] RSP: 0018:ffff88013fc83c60 EFLAGS: 00010246
[ 6758.314093] RAX: 6b6b6b6b6b6b6b6b RBX: ffff8801281204a0 RCX:
0000000000000000
[ 6758.314093] RDX: 0000000000000000 RSI: 00000000ffffffff RDI:
ffff88013906ec48
[ 6758.314093] RBP: ffff88013fc83c60 R08: 0000000000000000 R09:
0000000000000001
[ 6758.314093] R10: 0000000000000000 R11: 0000000000000000 R12:
0000000000000000
[ 6758.314093] R13: ffff880130b58560 R14: 0000000000080000 R15:
0000000000000000
[ 6758.314093] FS: 0000000000000000(0000) GS:ffff88013fc80000(0000)
knlGS:0000000000000000
[ 6758.314093] CS: 0010 DS: 0000 ES: 0000 CR0: 000000008005003b
[ 6758.314093] CR2: 00007f9becec7000 CR3: 0000000137fe0000 CR4:
00000000000006e0
[ 6758.314093] DR0: 0000000000000000 DR1: 0000000000000000 DR2:
0000000000000000
[ 6758.314093] DR3: 0000000000000000 DR6: 00000000ffff0ff0 DR7:
0000000000000400
[ 6758.314093] Process kworker/0:1 (pid: 0, threadinfo ffff88013a44a000,
task ffff88013a458000)
[ 6758.314093] Stack:
[ 6758.314093] ffff88013fc83c80 ffffffff8145ee6d ffffffff00000000
ffff8801281204a0
[ 6758.314093] ffff88013fc83c90 ffffffff81173a9d ffff88013fc83cc0
ffffffff812d15d3
[ 6758.314093] ffff88013a44a000 0000000000000000 ffff8801281204a0
0000000000080000
[ 6758.314093] Call Trace:
[ 6758.314093] <IRQ>
[ 6758.314093] [<ffffffff8145ee6d>] uuid_endio+0x3d/0x50
[ 6758.314093] [<ffffffff81173a9d>] bio_endio+0x1d/0x40
[ 6758.314093] [<ffffffff812d15d3>] req_bio_endio+0x83/0xc0
[ 6758.314093] [<ffffffff812d4f71>] blk_update_request+0x101/0x5c0
[ 6758.314093] [<ffffffff812d51a2>] ? blk_update_request+0x332/0x5c0
[ 6758.314093] [<ffffffff812d5461>] blk_update_bidi_request+0x31/0x90
[ 6758.314093] [<ffffffff812d54ec>] blk_end_bidi_request+0x2c/0x80
[ 6758.314093] [<ffffffff812d5580>] blk_end_request+0x10/0x20
[ 6758.314093] [<ffffffff81471b7c>] scsi_io_completion+0x9c/0x5f0
[ 6758.314093] [<ffffffff81468940>] scsi_finish_command+0xb0/0xe0
[ 6758.314093] [<ffffffff81471965>] scsi_softirq_done+0xa5/0x140
[ 6758.314093] [<ffffffff812db70b>] blk_done_softirq+0x7b/0x90
[ 6758.314093] [<ffffffff8104fc65>] __do_softirq+0xc5/0x3a0
[ 6758.314093] [<ffffffff817f6dac>] call_softirq+0x1c/0x30
[ 6758.314093] [<ffffffff8100419d>] do_softirq+0x8d/0xc0
[ 6758.314093] [<ffffffff8105027e>] irq_exit+0xae/0xe0
[ 6758.314093] [<ffffffff817f74b3>] do_IRQ+0x63/0xe0
[ 6758.314093] [<ffffffff817ecc30>] common_interrupt+0x70/0x70
[ 6758.314093] <EOI>
[ 6758.314093] [<ffffffff8100a1f6>] ? mwait_idle+0xb6/0x470
[ 6758.314093] [<ffffffff8100a1ed>] ? mwait_idle+0xad/0x470
[ 6758.314093] [<ffffffff810011df>] cpu_idle+0x8f/0xd0
[ 6758.314093] [<ffffffff817da107>] start_secondary+0x1be/0x1c2
[ 6758.314093] Code: 00 48 8b 50 48 83 e2 08 0f 85 9c 00 00 00 48 8b 50 48
83 e2 10 0f 85 8d 00 00 00 48 83 78 18 00 75 46 48 8b 40 40 48 85 c0 74 24
[ 6758.314093] 8b 50 48 48 c1 ea 04 89 d1 89 f2 83 e1 01 f0 0f c1 50 4c
83
[ 6758.314093] RIP [<ffffffff8146625b>] closure_put+0x5b/0xe0
[ 6758.314093] RSP <ffff88013fc83c60>

Thanks
Vivek

--
dm-devel mailing list
dm-devel@redhat.com
https://www.redhat.com/mailman/listinfo/dm-devel
 
Old 03-14-2012, 04:24 PM
Kent Overstreet
 
Default Bcache

On Wed, Mar 14, 2012 at 11:53 AM, Vivek Goyal <vgoyal@redhat.com> wrote:
> On Wed, Mar 14, 2012 at 09:32:28AM -0400, Kent Overstreet wrote:
>> I'm already registered to attend, but would it be too late in the
>> process to give a talk? I'd like to give a short talk about bcache, what
>> it does and where it's going (more than just caching).
>
> [CCing dm-devel list]
>
> I am curious if you considered writing a device mapper driver for this? If
> yes, why that is not a good choice. It seems to be stacked device and device
> mapper should be good at that. All the configuration through sysfs seems
> little odd to me.

Everyone asks this. Yeah, I considered it, I tried to make it work for
a couple weeks but it was far more trouble than it was worth. I'm not
opposed to someone else working on it but I'm not going to spend any
more time on it myself.
>
> On a side note, I was playing with bcache a bit. I tried to register the
> cache device and it crashes. (I guess I should post this on relevant mailing
> list).

Can you post the full log? There was a bug where if it encountered an
error during registration, it wouldn't wait for a uuid read or write
before tearing everything down - that's what your backtrace looks like
to me.

You could try the bcache-3.2-dev branch, too. I have a newer branch
with a ton of bugfixes but I'm waiting until it's seen more testing
before I post it.

>
> # echo /dev/sdc > /sys/fs/bcache/register
>
> [ 6758.314093] Pid: 0, comm: kworker/0:1 Not tainted 3.1.0-bcache+ #2
> Hewlett-Packard HP xw6600 Workstation/0A9Ch
> [ 6758.314093] RIP: 0010:[<ffffffff8146625b>] *[<ffffffff8146625b>]
> closure_put+0x5b/0xe0
> [ 6758.314093] RSP: 0018:ffff88013fc83c60 *EFLAGS: 00010246
> [ 6758.314093] RAX: 6b6b6b6b6b6b6b6b RBX: ffff8801281204a0 RCX:
> 0000000000000000
> [ 6758.314093] RDX: 0000000000000000 RSI: 00000000ffffffff RDI:
> ffff88013906ec48
> [ 6758.314093] RBP: ffff88013fc83c60 R08: 0000000000000000 R09:
> 0000000000000001
> [ 6758.314093] R10: 0000000000000000 R11: 0000000000000000 R12:
> 0000000000000000
> [ 6758.314093] R13: ffff880130b58560 R14: 0000000000080000 R15:
> 0000000000000000
> [ 6758.314093] FS: *0000000000000000(0000) GS:ffff88013fc80000(0000)
> knlGS:0000000000000000
> [ 6758.314093] CS: *0010 DS: 0000 ES: 0000 CR0: 000000008005003b
> [ 6758.314093] CR2: 00007f9becec7000 CR3: 0000000137fe0000 CR4:
> 00000000000006e0
> [ 6758.314093] DR0: 0000000000000000 DR1: 0000000000000000 DR2:
> 0000000000000000
> [ 6758.314093] DR3: 0000000000000000 DR6: 00000000ffff0ff0 DR7:
> 0000000000000400
> [ 6758.314093] Process kworker/0:1 (pid: 0, threadinfo ffff88013a44a000,
> task ffff88013a458000)
> [ 6758.314093] Stack:
> [ 6758.314093] *ffff88013fc83c80 ffffffff8145ee6d ffffffff00000000
> ffff8801281204a0
> [ 6758.314093] *ffff88013fc83c90 ffffffff81173a9d ffff88013fc83cc0
> ffffffff812d15d3
> [ 6758.314093] *ffff88013a44a000 0000000000000000 ffff8801281204a0
> 0000000000080000
> [ 6758.314093] Call Trace:
> [ 6758.314093] *<IRQ>
> [ 6758.314093] *[<ffffffff8145ee6d>] uuid_endio+0x3d/0x50
> [ 6758.314093] *[<ffffffff81173a9d>] bio_endio+0x1d/0x40
> [ 6758.314093] *[<ffffffff812d15d3>] req_bio_endio+0x83/0xc0
> [ 6758.314093] *[<ffffffff812d4f71>] blk_update_request+0x101/0x5c0
> [ 6758.314093] *[<ffffffff812d51a2>] ? blk_update_request+0x332/0x5c0
> [ 6758.314093] *[<ffffffff812d5461>] blk_update_bidi_request+0x31/0x90
> [ 6758.314093] *[<ffffffff812d54ec>] blk_end_bidi_request+0x2c/0x80
> [ 6758.314093] *[<ffffffff812d5580>] blk_end_request+0x10/0x20
> [ 6758.314093] *[<ffffffff81471b7c>] scsi_io_completion+0x9c/0x5f0
> [ 6758.314093] *[<ffffffff81468940>] scsi_finish_command+0xb0/0xe0
> [ 6758.314093] *[<ffffffff81471965>] scsi_softirq_done+0xa5/0x140
> [ 6758.314093] *[<ffffffff812db70b>] blk_done_softirq+0x7b/0x90
> [ 6758.314093] *[<ffffffff8104fc65>] __do_softirq+0xc5/0x3a0
> [ 6758.314093] *[<ffffffff817f6dac>] call_softirq+0x1c/0x30
> [ 6758.314093] *[<ffffffff8100419d>] do_softirq+0x8d/0xc0
> [ 6758.314093] *[<ffffffff8105027e>] irq_exit+0xae/0xe0
> [ 6758.314093] *[<ffffffff817f74b3>] do_IRQ+0x63/0xe0
> [ 6758.314093] *[<ffffffff817ecc30>] common_interrupt+0x70/0x70
> [ 6758.314093] *<EOI>
> [ 6758.314093] *[<ffffffff8100a1f6>] ? mwait_idle+0xb6/0x470
> [ 6758.314093] *[<ffffffff8100a1ed>] ? mwait_idle+0xad/0x470
> [ 6758.314093] *[<ffffffff810011df>] cpu_idle+0x8f/0xd0
> [ 6758.314093] *[<ffffffff817da107>] start_secondary+0x1be/0x1c2
> [ 6758.314093] Code: 00 48 8b 50 48 83 e2 08 0f 85 9c 00 00 00 48 8b 50 48
> 83 e2 10 0f 85 8d 00 00 00 48 83 78 18 00 75 46 48 8b 40 40 48 85 c0 74 24
> [ 6758.314093] *8b 50 48 48 c1 ea 04 89 d1 89 f2 83 e1 01 f0 0f c1 50 4c
> 83
> [ 6758.314093] RIP *[<ffffffff8146625b>] closure_put+0x5b/0xe0
> [ 6758.314093] *RSP <ffff88013fc83c60>
>
> Thanks
> Vivek

--
dm-devel mailing list
dm-devel@redhat.com
https://www.redhat.com/mailman/listinfo/dm-devel
 
Old 03-14-2012, 05:12 PM
chetan loke
 
Default Bcache

On Wed, Mar 14, 2012 at 11:53 AM, Vivek Goyal <vgoyal@redhat.com> wrote:
> On Wed, Mar 14, 2012 at 09:32:28AM -0400, Kent Overstreet wrote:
>> I'm already registered to attend, but would it be too late in the
>> process to give a talk? I'd like to give a short talk about bcache, what
>> it does and where it's going (more than just caching).
>
> [CCing dm-devel list]
>
> I am curious if you considered writing a device mapper driver for this? If
> yes, why that is not a good choice. It seems to be stacked device and device
> mapper should be good at that. All the configuration through sysfs seems
> little odd to me.

I'm not a dm guru but a quick scan at flash-cache seems like it does
what you are saying. Now, if performance isn't acceptable then hashes
can be replaced with trees and what-not. Also no one would need to
re-invent the stacking mechanism. I saw thin support(atleast
documented) for dm. Plus, no matter what cache you come up with you
may have to persist/store the meta-data associated with it. And dm
seems like the right place to abstract that.



> Vivek

Chetan

--
dm-devel mailing list
dm-devel@redhat.com
https://www.redhat.com/mailman/listinfo/dm-devel
 
Old 03-14-2012, 05:17 PM
Kent Overstreet
 
Default Bcache

On Wed, Mar 14, 2012 at 2:12 PM, chetan loke <loke.chetan@gmail.com> wrote:
> I'm not a dm guru but a quick scan at flash-cache seems like it does
> what you are saying. Now, if performance isn't acceptable then hashes
> can be replaced with trees and what-not. Also no one would need to
> re-invent the stacking mechanism. I saw thin support(atleast
> documented) for dm. Plus, no matter what cache you come up with you
> may have to persist/store the meta-data associated with it. And dm
> seems like the right place to abstract that.

Bcache kills flash cache on performance - bcache can do around a
million iops on 4k random reads, and beats it on real world
applications and hardware too (i.e. mysql).

I'm not aware of any real features I'm missing out on by not using dm...

--
dm-devel mailing list
dm-devel@redhat.com
https://www.redhat.com/mailman/listinfo/dm-devel
 
Old 03-14-2012, 05:33 PM
chetan loke
 
Default Bcache

On Wed, Mar 14, 2012 at 2:17 PM, Kent Overstreet <koverstreet@google.com> wrote:
> On Wed, Mar 14, 2012 at 2:12 PM, chetan loke <loke.chetan@gmail.com> wrote:
>> I'm not a dm guru but a quick scan at flash-cache seems like it does
>> what you are saying. Now, if performance isn't acceptable then hashes
>> can be replaced with trees and what-not. Also no one would need to
>> re-invent the stacking mechanism. I saw thin support(atleast
>> documented) for dm. Plus, no matter what cache you come up with you
>> may have to persist/store the meta-data associated with it. And dm
>> seems like the right place to abstract that.
>
> Bcache kills flash cache on performance - bcache can do around a
> million iops on 4k random reads, and beats it on real world
> applications and hardware too (i.e. mysql).
>

Don't get too carried away with the perf numbers. re-read what I said:
"if performance isn't acceptable then hashes can be replaced with
trees and what-not".

> I'm not aware of any real features I'm missing out on by not using dm...

But you are not explaining why dm is not the right stack. Just because
it crashed when you tried doesn't mean it's not the right place.
flash-cache works, doesn't it? flash-cache's limitation is because
it's a dm-target or because it is using hashing or something else?
There are start-ups who are doing quite great with SSD-cache+dm. So
please stop kidding yourself.

--
dm-devel mailing list
dm-devel@redhat.com
https://www.redhat.com/mailman/listinfo/dm-devel
 
Old 03-14-2012, 05:41 PM
Kent Overstreet
 
Default Bcache

On Wed, Mar 14, 2012 at 2:33 PM, chetan loke <loke.chetan@gmail.com> wrote:
> On Wed, Mar 14, 2012 at 2:17 PM, Kent Overstreet <koverstreet@google.com> wrote:
>> On Wed, Mar 14, 2012 at 2:12 PM, chetan loke <loke.chetan@gmail.com> wrote:
>>> I'm not a dm guru but a quick scan at flash-cache seems like it does
>>> what you are saying. Now, if performance isn't acceptable then hashes
>>> can be replaced with trees and what-not. Also no one would need to
>>> re-invent the stacking mechanism. I saw thin support(atleast
>>> documented) for dm. Plus, no matter what cache you come up with you
>>> may have to persist/store the meta-data associated with it. And dm
>>> seems like the right place to abstract that.
>>
>> Bcache kills flash cache on performance - bcache can do around a
>> million iops on 4k random reads, and beats it on real world
>> applications and hardware too (i.e. mysql).
>>
>
> Don't get too carried away with the perf numbers. re-read what I said:
> *"if performance isn't acceptable then hashes can be replaced with
> trees and what-not".

Nobody's stopping you.

>> I'm not aware of any real features I'm missing out on by not using dm...
>
> But you are not explaining why dm is not the right stack. Just because
> it crashed when you tried doesn't mean it's not the right place.
> flash-cache works, doesn't it? flash-cache's limitation is because
> it's a dm-target or because it is using hashing or something else?
> There are start-ups who are doing quite great with SSD-cache+dm. So
> please stop kidding yourself.

If you want me to implement bcache differently, shouldn't you explain
why? I'm not sure why I _have_ to justify my decisions to you.

--
dm-devel mailing list
dm-devel@redhat.com
https://www.redhat.com/mailman/listinfo/dm-devel
 
Old 03-14-2012, 05:47 PM
Christoph Hellwig
 
Default Bcache

On Wed, Mar 14, 2012 at 02:41:35PM -0400, Kent Overstreet wrote:
> If you want me to implement bcache differently, shouldn't you explain
> why? I'm not sure why I _have_ to justify my decisions to you.

You don't have to - unless you want to get bcache merged into the
mainline kernel. If you don't want to you probably shouldn't bother
appearing at the LSF, though.

--
dm-devel mailing list
dm-devel@redhat.com
https://www.redhat.com/mailman/listinfo/dm-devel
 
Old 03-14-2012, 05:54 PM
"Ted Ts'o"
 
Default Bcache

On Wed, Mar 14, 2012 at 02:33:25PM -0400, chetan loke wrote:
> But you are not explaining why dm is not the right stack. Just because
> it crashed when you tried doesn't mean it's not the right place.
> flash-cache works, doesn't it? flash-cache's limitation is because
> it's a dm-target or because it is using hashing or something else?
> There are start-ups who are doing quite great with SSD-cache+dm. So
> please stop kidding yourself.

SATA-attached flash is not the only kind of flash out there you know.
There is also PCIe-attached flash which is a wee bit faster (where wee
is defined as multiple orders of magnitude --- SATA-attached SSD's
typically have thousands of IOPS; Fusion I/O is shipping product today
with hundreds of thousands of IOPS, and has demonstrated a billion
IOPS early this year). And Fusion I/O isn't the only company shipping
PCIe-attached flash products.

Startups may be doing great on SSD's; you may want to accept the fact
that there is stuff which is way, way, way better out there than
SSD's which are available on the market *today*.

And it's not like bache which is a new project. It's working code,
just like flash cache is today. So it's not like it needs to justify
its existence.

Best regards,

- Ted

--
dm-devel mailing list
dm-devel@redhat.com
https://www.redhat.com/mailman/listinfo/dm-devel
 

Thread Tools




All times are GMT. The time now is 10:27 PM.

VBulletin, Copyright ©2000 - 2014, Jelsoft Enterprises Ltd.
Content Relevant URLs by vBSEO ©2007, Crawlability, Inc.
Copyright 2007 - 2008, www.linux-archive.org