FAQ Search Today's Posts Mark Forums Read
» Video Reviews

» Linux Archive

Linux-archive is a website aiming to archive linux email lists and to make them easily accessible for linux users/developers.


» Sponsor

» Partners

» Sponsor

Go Back   Linux Archive > Redhat > Device-mapper Development

 
 
LinkBack Thread Tools
 
Old 01-16-2012, 12:27 AM
Yukihito HARA
 
Default About the thin provision function @ kernel 3.2 or later.

Hi, I'm Yukihito, and interested in Linux system.

Recently days, I became happy to hear that Linux kernel start to support thin-provisioning function.
And not trying to how it work on latest kernel.

According to the word "thin provisioning", I'm imaging that, I can get larger space than actual disk size. This image is born from thin-provisioning system working on VMWare series.

And can write the files until reach to the physical maximum. Once reach to the physical maximum, I can't write any more files to that volume, but I can write more files after add additional physical disks to thin-pool.


I could create the pool according to the documents included in the kernel source(Documentation/device-mapper/thin-provisioning.txt).
But I'm not sure, it works as I expecting like above.

So can you provide more detailed documentation about the thin-provisioning, if you have?

There're no documentation written in Japanese, so I'll also output this communication to wiki or my twitter in Japanese. It helpful to make your work more major among Japanese IT engineers. And you can get more feedback if the challengers in Japan will be increased.

The worst point of the current documentation is, it's not instruct about how to use like above. Can't get image how to use this like VMware system from that document.

Sorry, this is my private testing, not for work. So I can't get much time to this, but very interested in.



I'm looking forward to your answer.

Thank you.
Yukihito.

--
dm-devel mailing list
dm-devel@redhat.com
https://www.redhat.com/mailman/listinfo/dm-devel
 
Old 01-16-2012, 01:38 PM
Joe Thornber
 
Default About the thin provision function @ kernel 3.2 or later.

On Mon, Jan 16, 2012 at 10:27:34AM +0900, Yukihito HARA wrote:
> Hi, I'm Yukihito, and interested in Linux system.
>
> There're no documentation written in Japanese, so I'll also output this
> communication to wiki or my twitter in Japanese. It helpful to make your
> work more major among Japanese IT engineers. And you can get more feedback
> if the challengers in Japan will be increased.

There are a couple of documents here:

https://github.com/jthornber/storage-papers/tree/master/thinp-snapshots-2011

I'll be updating the presentation soon for a talk next month.

- Joe

--
dm-devel mailing list
dm-devel@redhat.com
https://www.redhat.com/mailman/listinfo/dm-devel
 
Old 01-16-2012, 02:59 PM
"Ted Ts'o"
 
Default About the thin provision function @ kernel 3.2 or later.

On Mon, Jan 16, 2012 at 02:38:17PM +0000, Joe Thornber wrote:
>
> There are a couple of documents here:
>
> https://github.com/jthornber/storage-papers/tree/master/thinp-snapshots-2011
>
> I'll be updating the presentation soon for a talk next month.

Hi Joe,

Thanks for making a pointer to the paper available! I have a question
about one of the slides, where you say: "snapshots of external
origins". Am I correct in suppose this means you can take a snapshot
of something which is currently not a thin-provisioned volume?

If so, is this currently implemented? Looking at the documentation in
the v3.2's Documentation/device-mapper/thin-provisioning.txt, I don't
see a way of doing it:

create_snap <dev id> <origin id>

Create a new snapshot of another thinly-provisioned device.
<dev id> is an arbitrary unique 24-bit identifier chosen by
the caller.
<origin id> is the identifier of the thinly-provisioned device
of which the new device will be a snapshot.

Is this a feature which is yet to be implemented? And if so, what's
the timeline for implementing it.

If this is not a feature to be implemented, can I please put it on the
wishlist? I can imagine a lot of ext4 users who would be interested
in thin-provisioning, especially if there is discard support such than
when we delete a file, we send a discard which causes the
thin-provisioned space in the snapshot to be dropped. (This is also
apparently not implemented, from what I can tell from the source; do
you have an approximate idea of where in the priority list / when it
is scheduled to be implemented?)

Many thanks,

- Ted

--
dm-devel mailing list
dm-devel@redhat.com
https://www.redhat.com/mailman/listinfo/dm-devel
 
Old 01-16-2012, 03:21 PM
Heinz Mauelshagen
 
Default About the thin provision function @ kernel 3.2 or later.

On 01/16/2012 02:27 AM, Yukihito HARA wrote:
Hi, I'm Yukihito, and interested in Linux system.




Hi.





Recently days, I became happy to hear that Linux kernel start to
support thin-provisioning function.

And not trying to how it work on latest kernel.



According to the word "thin provisioning", I'm imaging that, I can
get larger space than actual disk size. This image is born from
thin-provisioning system working on VMWare series.




Thin provisioning iis a general design to save storage costs.



There's solutions in software and hardware in the market, including
VMWare.

Higher range storage systems feature it.




And can write the files until reach to the physical maximum. Once
reach to the physical maximum, I can't write any more files to
that volume, but I can write more files after add additional
physical disks to thin-pool.




Yes.






I could create the pool according to the documents included in the
kernel source(Documentation/device-mapper/thin-provisioning.txt).

But I'm not sure, it works as I expecting like above.



So can you provide more detailed documentation about the
thin-provisioning, if you have?




Did you read Documentation/device-mapper/thin-provisioning.txt in
the kernel source tree to use it yet?

It provides advice how to create thin-provisioned devices and
snapshots (see dmsetup message examples).




There're no documentation written in Japanese, so I'll also output
this communication to wiki or my twitter in Japanese. It helpful
to make your work more major among Japanese IT engineers. And you
can get more feedback if the challengers in Japan will be
increased.

The worst point of the current documentation is, it's not instruct
about how to use like above. Can't get image how to use this like
VMware system from that document.




Keep in mind that there'll be LVM2 support available later in
distributions.

You can pull that code from the HEAD of the LVM2 upstream repository
for testing purposes already (not meant for production use
obviously).





Sorry, this is my private testing, not for work. So I can't get
much time to this, but very interested in.




Thanks for trying it, we're interested in external feedback.



Regards,

Heinz








I'm looking forward to your answer.



Thank you.

Yukihito.






--
dm-devel mailing list
dm-devel@redhat.com
https://www.redhat.com/mailman/listinfo/dm-devel



--
================================================== =============
Heinz Mauelshagen +49 2626 141200
Consulting Development Engineer FAX +49 2626 924446
Red Hat GmbH
Am Sonnenhang 11
56242 Marienrachdorf
Germany heinzm@redhat.com
================================================== =============


--
dm-devel mailing list
dm-devel@redhat.com
https://www.redhat.com/mailman/listinfo/dm-devel
 
Old 01-17-2012, 09:13 AM
Joe Thornber
 
Default About the thin provision function @ kernel 3.2 or later.

Hi Ted,

Thanks for taking time to look at thinp.

On Mon, Jan 16, 2012 at 10:59:59AM -0500, Ted Ts'o wrote:
> Thanks for making a pointer to the paper available! I have a question
> about one of the slides, where you say: "snapshots of external
> origins". Am I correct in suppose this means you can take a snapshot
> of something which is currently not a thin-provisioned volume?

...

> Is this a feature which is yet to be implemented? And if so, what's
> the timeline for implementing it.

Yes, that's exactly it. I do want to support it in the near future
(2-3 months). There are a few options on this front.

i) We only support a read-only external origin. Of course people can
use snapshots if they want writeable versions of the current state.
An explicit merge facility will need to be added if people want to
copy the snap back over the origin. This is by far the simplest
approach, and I'll probably start with this - at least in a dev branch.

ii) We support read/write origins. When I last looked at this 15
months ago I struggled to come up with a metadata design that allowed
both efficient snapshot lookup _and_ efficient snapshot deletion. I'm
expecting people to use many, many more snapshots with thinp; deletion
performance really is a problem.

iii) We add support to patch metadata changes into the running
metadata device. This would allow us to map the origin directly into
the pools data device (eg, using a linear target). And then introduce
a mapping for that origin. This would mean the origin would appear as
a thin dev within the pool. This approach greatly improves the
performance since we wouldn't have to always COW on the first write to
an origin block (i.e. the overlapping block optimisation is still
valid).

> If this is not a feature to be implemented, can I please put it on the
> wishlist? I can imagine a lot of ext4 users who would be interested
> in thin-provisioning, especially if there is discard support such than
> when we delete a file, we send a discard which causes the
> thin-provisioned space in the snapshot to be dropped. (This is also
> apparently not implemented, from what I can tell from the source; do
> you have an approximate idea of where in the priority list / when it
> is scheduled to be implemented?)

Discard was in until recently. There were some race conditions
associated with it that I didn't have time to work through before the
merge window. Expect this to reappear sometime in the next 2-3 months.

- Joe

--
dm-devel mailing list
dm-devel@redhat.com
https://www.redhat.com/mailman/listinfo/dm-devel
 
Old 01-17-2012, 02:59 PM
"Ted Ts'o"
 
Default About the thin provision function @ kernel 3.2 or later.

On Tue, Jan 17, 2012 at 10:13:31AM +0000, Joe Thornber wrote:
> iii) We add support to patch metadata changes into the running
> metadata device. This would allow us to map the origin directly into
> the pools data device (eg, using a linear target). And then introduce
> a mapping for that origin. This would mean the origin would appear as
> a thin dev within the pool.

Have you considered just doing this part first? If there's a way that
an existing linear region could be considered a data source for a
pool, then wouldn't you get full support of existing LVM2 volumes
(read-only and read/write) for free, at one fell swoop?

Given that it's relatively rare that volume groups get moved, this
could even be something which is done as a off-line conversion step so
that existing LVM volumes are treated as "thin-provisioned voumes"
that happened to be fully provisioned, with some flag so that a
discard operation doesn't cause the blocks to be freed for use by some
other thin-provisioned volume, but instead is propagated down to the
hardware (for better SSD performance).

- Ted

--
dm-devel mailing list
dm-devel@redhat.com
https://www.redhat.com/mailman/listinfo/dm-devel
 
Old 01-19-2012, 08:14 AM
Joe Thornber
 
Default About the thin provision function @ kernel 3.2 or later.

On Tue, Jan 17, 2012 at 10:59:30AM -0500, Ted Ts'o wrote:
> On Tue, Jan 17, 2012 at 10:13:31AM +0000, Joe Thornber wrote:
> > iii) We add support to patch metadata changes into the running
> > metadata device. This would allow us to map the origin directly into
> > the pools data device (eg, using a linear target). And then introduce
> > a mapping for that origin. This would mean the origin would appear as
> > a thin dev within the pool.
>
> Have you considered just doing this part first? If there's a way that
> an existing linear region could be considered a data source for a
> pool, then wouldn't you get full support of existing LVM2 volumes
> (read-only and read/write) for free, at one fell swoop?

Yes, this is the solution that I like best. But I've only recently
thought of it whilst I coded up the thinp_dump and thin_restore tools.
Note that by using these tools and doing some manipulation of the
metadata you can do this today in an offline mode.

> Given that it's relatively rare that volume groups get moved, this
> could even be something which is done as a off-line conversion step so
> that existing LVM volumes are treated as "thin-provisioned voumes"
> that happened to be fully provisioned, with some flag so that a
> discard operation doesn't cause the blocks to be freed for use by some
> other thin-provisioned volume, but instead is propagated down to the
> hardware (for better SSD performance).

Certainly the discard support needs to do two things; release the
block and pass the discard down to the device. Making the first part
optional sounds sensible to me.

- Joe

--
dm-devel mailing list
dm-devel@redhat.com
https://www.redhat.com/mailman/listinfo/dm-devel
 
Old 01-27-2012, 03:04 PM
Joe Thornber
 
Default About the thin provision function @ kernel 3.2 or later.

Hi Ted,

On Mon, Jan 16, 2012 at 10:59:59AM -0500, Ted Ts'o wrote:
> (This is also
> apparently not implemented, from what I can tell from the source; do
> you have an approximate idea of where in the priority list / when it
> is scheduled to be implemented?)

The 'thin-dev' branch of my github repo now has external origin and
discard support.

https://github.com/jthornber/linux-2.6/tree/thin-dev

I'll give it 2 or 3 weeks of testing before pushing it upstream.

- Joe

--
dm-devel mailing list
dm-devel@redhat.com
https://www.redhat.com/mailman/listinfo/dm-devel
 

Thread Tools




All times are GMT. The time now is 07:36 AM.

VBulletin, Copyright ©2000 - 2014, Jelsoft Enterprises Ltd.
Content Relevant URLs by vBSEO ©2007, Crawlability, Inc.
Copyright 2007 - 2008, www.linux-archive.org