FAQ Search Today's Posts Mark Forums Read
» Video Reviews

» Linux Archive

Linux-archive is a website aiming to archive linux email lists and to make them easily accessible for linux users/developers.


» Sponsor

» Partners

» Sponsor

Go Back   Linux Archive > Redhat > Cluster Development

 
 
LinkBack Thread Tools
 
Old 04-26-2012, 02:55 PM
Steven Whitehouse
 
Default gfs2: Get rid of I_MUTEX_QUOTA usage

Hi,

On Thu, 2012-04-26 at 15:47 +0200, Jan Kara wrote:
> Hello,
>
> On Thu 26-04-12 14:28:39, Steven Whitehouse wrote:
> > On Thu, 2012-04-26 at 15:20 +0200, Jan Kara wrote:
> > > GFS2 uses i_mutex on its system quota inode to synchronize writes to
> > > quota file. Since this is an internal inode to GFS2 (not part of directory
> > > hiearchy or visible by user) we are safe to define locking rules for it. So
> > > let's just get it its own locking class to make it clear.
> > >
> > It is visible to the user if the gfs2 metadata filesystem is mounted.
> > That used to be the way in which the quotas were set by gfs2_quota.
> > However that is really considered obsolete now, so maybe we don't have
> > to support that interface any more... the gfs2_quota package is no
> > longer in distros as the generic quota package does the job for us these
> > days.
> Ah, OK. But then if users could write to (or even truncate?) the quota
> file, wasn't it really deadlockable? mutex_lock_nested would silence
> lockdep but the deadlock won't change. Another advantage of my change is
> that if there is some problem, lockdep will warn about it because all
> places which end up taking i_mutex on quota file will use the same locking
> class. Using of lockdep subclasses (i.e. the _nested variant of mutex_lock)
> essentially tells lockdep - I promise that uses of the lock tagged as
> one locking subclass cannot interact with uses tagged as another locking
> class.
>
> Honza
>
It has been fairly well tested over a period of time, so I'm pretty
confident that it does work at the moment. The locking is not that
obvious though and could certainly do with a clean up. I'd prefer to
clean up the locking than just silence the messages.

I am taking a look at it currently to see what we might do. It may take
me a little while to get something together though, bearing in mind all
the possible cases that need checking. If we can use sd_quota_mutex
everywhere, then that should at least reduce the complexity and remove
the need for I_MUTEX_QUOTA, so thats my preferred solution if possible.
Otherwise I'm open to suggestions,

Steve.

> > > CC: Steven Whitehouse <swhiteho@redhat.com>
> > > Signed-off-by: Jan Kara <jack@suse.cz>
> > > ---
> > > fs/gfs2/ops_fstype.c | 8 ++++++++
> > > fs/gfs2/quota.c | 2 +-
> > > 2 files changed, 9 insertions(+), 1 deletions(-)
> > >
> > > So this is probably the simplest what can be done for GFS2 (actually OCFS2
> > > does the same for internal system files). Compile tested only. Steven?
> > >
> > > diff --git a/fs/gfs2/ops_fstype.c b/fs/gfs2/ops_fstype.c
> > > index 6f3a18f..ae8f225 100644
> > > --- a/fs/gfs2/ops_fstype.c
> > > +++ b/fs/gfs2/ops_fstype.c
> > > @@ -19,6 +19,7 @@
> > > #include <linux/mount.h>
> > > #include <linux/gfs2_ondisk.h>
> > > #include <linux/quotaops.h>
> > > +#include <linux/lockdep.h>
> > >
> > > #include "gfs2.h"
> > > #include "incore.h"
> > > @@ -767,6 +768,7 @@ fail:
> > > return error;
> > > }
> > >
> > > +static struct lock_class_key gfs2_quota_imutex_key;
> > >
> > > static int init_inodes(struct gfs2_sbd *sdp, int undo)
> > > {
> > > @@ -804,6 +806,12 @@ static int init_inodes(struct gfs2_sbd *sdp, int undo)
> > > fs_err(sdp, "can't get quota file inode: %d
", error);
> > > goto fail_rindex;
> > > }
> > > + /*
> > > + * i_mutex on quota files is special. Since this inode is hidden system
> > > + * file, we are safe to define locking ourselves.
> > > + */
> > > + lockdep_set_class(&sdp->sd_quota_inode->i_mutex,
> > > + &gfs2_quota_imutex_key);
> > >
> > > error = gfs2_rindex_update(sdp);
> > > if (error)
> > > diff --git a/fs/gfs2/quota.c b/fs/gfs2/quota.c
> > > index 6019da3..970598b 100644
> > > --- a/fs/gfs2/quota.c
> > > +++ b/fs/gfs2/quota.c
> > > @@ -772,7 +772,7 @@ static int do_sync(unsigned int num_qd, struct gfs2_quota_data **qda)
> > > return -ENOMEM;
> > >
> > > sort(qda, num_qd, sizeof(struct gfs2_quota_data *), sort_qd, NULL);
> > > - mutex_lock_nested(&ip->i_inode.i_mutex, I_MUTEX_QUOTA);
> > > + mutex_lock(&ip->i_inode.i_mutex);
> > > for (qx = 0; qx < num_qd; qx++) {
> > > error = gfs2_glock_nq_init(qda[qx]->qd_gl, LM_ST_EXCLUSIVE,
> > > GL_NOCACHE, &ghs[qx]);
> >
> >
 
Old 04-26-2012, 03:19 PM
Jan Kara
 
Default gfs2: Get rid of I_MUTEX_QUOTA usage

Hi,

On Thu 26-04-12 15:55:33, Steven Whitehouse wrote:
> On Thu, 2012-04-26 at 15:47 +0200, Jan Kara wrote:
> > Hello,
> >
> > On Thu 26-04-12 14:28:39, Steven Whitehouse wrote:
> > > On Thu, 2012-04-26 at 15:20 +0200, Jan Kara wrote:
> > > > GFS2 uses i_mutex on its system quota inode to synchronize writes to
> > > > quota file. Since this is an internal inode to GFS2 (not part of directory
> > > > hiearchy or visible by user) we are safe to define locking rules for it. So
> > > > let's just get it its own locking class to make it clear.
> > > >
> > > It is visible to the user if the gfs2 metadata filesystem is mounted.
> > > That used to be the way in which the quotas were set by gfs2_quota.
> > > However that is really considered obsolete now, so maybe we don't have
> > > to support that interface any more... the gfs2_quota package is no
> > > longer in distros as the generic quota package does the job for us these
> > > days.
> > Ah, OK. But then if users could write to (or even truncate?) the quota
> > file, wasn't it really deadlockable? mutex_lock_nested would silence
> > lockdep but the deadlock won't change. Another advantage of my change is
> > that if there is some problem, lockdep will warn about it because all
> > places which end up taking i_mutex on quota file will use the same locking
> > class. Using of lockdep subclasses (i.e. the _nested variant of mutex_lock)
> > essentially tells lockdep - I promise that uses of the lock tagged as
> > one locking subclass cannot interact with uses tagged as another locking
> > class.
> >
> > Honza
> >
> It has been fairly well tested over a period of time, so I'm pretty
> confident that it does work at the moment. The locking is not that
> obvious though and could certainly do with a clean up. I'd prefer to
> clean up the locking than just silence the messages.
>
> I am taking a look at it currently to see what we might do. It may take
> me a little while to get something together though, bearing in mind all
> the possible cases that need checking. If we can use sd_quota_mutex
> everywhere, then that should at least reduce the complexity and remove
> the need for I_MUTEX_QUOTA, so thats my preferred solution if possible.
> Otherwise I'm open to suggestions,
Ok, sounds good.

Honza

> > > > CC: Steven Whitehouse <swhiteho@redhat.com>
> > > > Signed-off-by: Jan Kara <jack@suse.cz>
> > > > ---
> > > > fs/gfs2/ops_fstype.c | 8 ++++++++
> > > > fs/gfs2/quota.c | 2 +-
> > > > 2 files changed, 9 insertions(+), 1 deletions(-)
> > > >
> > > > So this is probably the simplest what can be done for GFS2 (actually OCFS2
> > > > does the same for internal system files). Compile tested only. Steven?
> > > >
> > > > diff --git a/fs/gfs2/ops_fstype.c b/fs/gfs2/ops_fstype.c
> > > > index 6f3a18f..ae8f225 100644
> > > > --- a/fs/gfs2/ops_fstype.c
> > > > +++ b/fs/gfs2/ops_fstype.c
> > > > @@ -19,6 +19,7 @@
> > > > #include <linux/mount.h>
> > > > #include <linux/gfs2_ondisk.h>
> > > > #include <linux/quotaops.h>
> > > > +#include <linux/lockdep.h>
> > > >
> > > > #include "gfs2.h"
> > > > #include "incore.h"
> > > > @@ -767,6 +768,7 @@ fail:
> > > > return error;
> > > > }
> > > >
> > > > +static struct lock_class_key gfs2_quota_imutex_key;
> > > >
> > > > static int init_inodes(struct gfs2_sbd *sdp, int undo)
> > > > {
> > > > @@ -804,6 +806,12 @@ static int init_inodes(struct gfs2_sbd *sdp, int undo)
> > > > fs_err(sdp, "can't get quota file inode: %d
", error);
> > > > goto fail_rindex;
> > > > }
> > > > + /*
> > > > + * i_mutex on quota files is special. Since this inode is hidden system
> > > > + * file, we are safe to define locking ourselves.
> > > > + */
> > > > + lockdep_set_class(&sdp->sd_quota_inode->i_mutex,
> > > > + &gfs2_quota_imutex_key);
> > > >
> > > > error = gfs2_rindex_update(sdp);
> > > > if (error)
> > > > diff --git a/fs/gfs2/quota.c b/fs/gfs2/quota.c
> > > > index 6019da3..970598b 100644
> > > > --- a/fs/gfs2/quota.c
> > > > +++ b/fs/gfs2/quota.c
> > > > @@ -772,7 +772,7 @@ static int do_sync(unsigned int num_qd, struct gfs2_quota_data **qda)
> > > > return -ENOMEM;
> > > >
> > > > sort(qda, num_qd, sizeof(struct gfs2_quota_data *), sort_qd, NULL);
> > > > - mutex_lock_nested(&ip->i_inode.i_mutex, I_MUTEX_QUOTA);
> > > > + mutex_lock(&ip->i_inode.i_mutex);
> > > > for (qx = 0; qx < num_qd; qx++) {
> > > > error = gfs2_glock_nq_init(qda[qx]->qd_gl, LM_ST_EXCLUSIVE,
> > > > GL_NOCACHE, &ghs[qx]);
> > >
> > >
>
>
--
Jan Kara <jack@suse.cz>
SUSE Labs, CR
 
Old 09-26-2012, 08:25 AM
Steven Whitehouse
 
Default GFS2: Get rid of I_MUTEX_QUOTA usage

From: Jan Kara <jack@suse.cz>

GFS2 uses i_mutex on its system quota inode to synchronize writes to
quota file. Since this is an internal inode to GFS2 (not part of directory
hiearchy or visible by user) we are safe to define locking rules for it. So
let's just get it its own locking class to make it clear.

Signed-off-by: Jan Kara <jack@suse.cz>
Signed-off-by: J. Bruce Fields <bfields@redhat.com>
Signed-off-by: Steven Whitehouse <swhiteho@redhat.com>

diff --git a/fs/gfs2/ops_fstype.c b/fs/gfs2/ops_fstype.c
index e5af9dc..e443966 100644
--- a/fs/gfs2/ops_fstype.c
+++ b/fs/gfs2/ops_fstype.c
@@ -19,6 +19,7 @@
#include <linux/mount.h>
#include <linux/gfs2_ondisk.h>
#include <linux/quotaops.h>
+#include <linux/lockdep.h>

#include "gfs2.h"
#include "incore.h"
@@ -766,6 +767,7 @@ fail:
return error;
}

+static struct lock_class_key gfs2_quota_imutex_key;

static int init_inodes(struct gfs2_sbd *sdp, int undo)
{
@@ -803,6 +805,12 @@ static int init_inodes(struct gfs2_sbd *sdp, int undo)
fs_err(sdp, "can't get quota file inode: %d
", error);
goto fail_rindex;
}
+ /*
+ * i_mutex on quota files is special. Since this inode is hidden system
+ * file, we are safe to define locking ourselves.
+ */
+ lockdep_set_class(&sdp->sd_quota_inode->i_mutex,
+ &gfs2_quota_imutex_key);

error = gfs2_rindex_update(sdp);
if (error)
diff --git a/fs/gfs2/quota.c b/fs/gfs2/quota.c
index 420bc38..4021dec 100644
--- a/fs/gfs2/quota.c
+++ b/fs/gfs2/quota.c
@@ -782,7 +782,7 @@ static int do_sync(unsigned int num_qd, struct gfs2_quota_data **qda)
return -ENOMEM;

sort(qda, num_qd, sizeof(struct gfs2_quota_data *), sort_qd, NULL);
- mutex_lock_nested(&ip->i_inode.i_mutex, I_MUTEX_QUOTA);
+ mutex_lock(&ip->i_inode.i_mutex);
for (qx = 0; qx < num_qd; qx++) {
error = gfs2_glock_nq_init(qda[qx]->qd_gl, LM_ST_EXCLUSIVE,
GL_NOCACHE, &ghs[qx]);
--
1.7.4
 

Thread Tools




All times are GMT. The time now is 05:13 AM.

VBulletin, Copyright ©2000 - 2014, Jelsoft Enterprises Ltd.
Content Relevant URLs by vBSEO ©2007, Crawlability, Inc.
Copyright 2007 - 2008, www.linux-archive.org