FAQ Search Today's Posts Mark Forums Read
» Video Reviews

» Linux Archive

Linux-archive is a website aiming to archive linux email lists and to make them easily accessible for linux users/developers.


» Sponsor

» Partners

» Sponsor

Go Back   Linux Archive > Redhat > Fedora User

 
 
LinkBack Thread Tools
 
Old 09-29-2010, 01:12 PM
Ian Chapman
 
Default NFS Buffering

On 28/09/10 22:07, Simon Andrews wrote:

> Does anyone know how to either make this buffer smaller, or get rid of
> it all together so the scp can accruately report on its progress?

You could try adjusting /proc/sys/vm/dirty_ratio.

"Contains, as a percentage of total system memory, the number of pages
at which a process which is generating disk writes will itself start
writing out dirty data"

The default seems to be 20 on Fedora 13. For example to set it to 5%
(which is the lowest it will go I believe):

echo 5 > /proc/sys/vm/dirty_ratio

This should effectively lower the size of the buffer, but note this will
affect all file systems, not just NFS. However you may find that acceptable.


--
Ian Chapman.
--
users mailing list
users@lists.fedoraproject.org
To unsubscribe or change subscription options:
https://admin.fedoraproject.org/mailman/listinfo/users
Guidelines: http://fedoraproject.org/wiki/Mailing_list_guidelines
 
Old 09-29-2010, 02:48 PM
Simon Andrews
 
Default NFS Buffering

On 29/09/2010 12:19, JB wrote:
> Simon Andrews<simon.andrews<at> bbsrc.ac.uk> writes:
>
>> ...
>
> Hi,
> thanks.
>
>> It looks like the nfs daemon is caching write data (around 2GB of it)
>> which lets scp think its finished when actually there's loads of data
>> sitting in a write buffer. The hanging is presumably the time it takes
>> to flush the buffer (there is a process called nfsiod which is active
>> during this time and df shows data is still be
>
> Your nfs client is a Fedora 13.
> Can you tell me what nfsiod is and where is came from ?
> Can you see it ?
> $ ps aux | grep -i nfsiod

I think it must be part of the kernel. There's no nfsiosd binary
anywhere on the system, but:

$ ps aux | grep -i nfsiod
root 8974 0.0 0.0 0 0 ? S Sep27 0:15 [nfsiod]

Where the parent process is kthread.

Simon.
--
users mailing list
users@lists.fedoraproject.org
To unsubscribe or change subscription options:
https://admin.fedoraproject.org/mailman/listinfo/users
Guidelines: http://fedoraproject.org/wiki/Mailing_list_guidelines
 
Old 09-29-2010, 02:51 PM
Simon Andrews
 
Default NFS Buffering

On 29/09/2010 14:12, Ian Chapman wrote:
> On 28/09/10 22:07, Simon Andrews wrote:
>
>> Does anyone know how to either make this buffer smaller, or get rid of
>> it all together so the scp can accruately report on its progress?
>
> You could try adjusting /proc/sys/vm/dirty_ratio.
>
> "Contains, as a percentage of total system memory, the number of pages
> at which a process which is generating disk writes will itself start
> writing out dirty data"
>
> The default seems to be 20 on Fedora 13. For example to set it to 5%
> (which is the lowest it will go I believe):

That sounds like it might be an answer. It's a shame there's no way to
specify this per-process, but this machine does have quite a bit of RAM
in it, so having this specified as a percentage of RAM might make it
larger than we'd want.

I'll have a play and see what effect this has.

Cheers

Simon.
--
users mailing list
users@lists.fedoraproject.org
To unsubscribe or change subscription options:
https://admin.fedoraproject.org/mailman/listinfo/users
Guidelines: http://fedoraproject.org/wiki/Mailing_list_guidelines
 
Old 09-29-2010, 03:27 PM
Ian Chapman
 
Default NFS Buffering

On 29/09/10 22:51, Simon Andrews wrote:

> That sounds like it might be an answer. It's a shame there's no way to
> specify this per-process, but this machine does have quite a bit of RAM
> in it, so having this specified as a percentage of RAM might make it
> larger than we'd want.
>
> I'll have a play and see what effect this has.

No problem, there's also a good page here which explains a bit more how
all this works, as well as some explanation on other tunables that might
suit you better.

http://www.westnet.com/~gsmith/content/linux-pdflush.htm

--
Ian Chapman.
--
users mailing list
users@lists.fedoraproject.org
To unsubscribe or change subscription options:
https://admin.fedoraproject.org/mailman/listinfo/users
Guidelines: http://fedoraproject.org/wiki/Mailing_list_guidelines
 
Old 09-29-2010, 03:29 PM
JB
 
Default NFS Buffering

Simon Andrews <simon.andrews <at> bbsrc.ac.uk> writes:

> ...
> I think it must be part of the kernel. There's no nfsiod binary
> anywhere on the system, but:
>
> $ ps aux | grep -i nfsiod
> root 8974 0.0 0.0 0 0 ? S Sep27 0:15 [nfsiod]
>
> Where the parent process is kthread.
>
> Simon.

Thanks.
Could you please give us (on Fedora 13) an unedited output of:
# cat /etc/fstab
# mount

JB


--
users mailing list
users@lists.fedoraproject.org
To unsubscribe or change subscription options:
https://admin.fedoraproject.org/mailman/listinfo/users
Guidelines: http://fedoraproject.org/wiki/Mailing_list_guidelines
 
Old 09-29-2010, 04:43 PM
JD
 
Default NFS Buffering

On 09/29/2010 02:03 AM, Simon Andrews wrote:
> On 29/09/2010 03:19, JD wrote:
>>
>>> <simon.andrews@bbsrc.ac.uk<mailto:simon.andrews@bb src.ac.uk>> wrote:
>>> The problem is that after scp reports that it's 100% complete the
>>> program will hang for ~20 mins before it will move on to another file.
>>> At this point it can't be killed.
>> Hey! Simon,
>> Listen: buffering is done by the filesystem internals
>> in collaboration with the block io layer. Once the filesystem
>> commits the write to block io layer, the write call returns to
>> the calling program, and there is not an iota you can do about
>> it!
> I think I get the general process by which the caching happens, and I'm
> not necessarily aiming to get rid of it, just set the cache size to
> something which is appropriate for the speed of the link I'm operating over.
YOU CANNOT!!! IT IS DONE BY THE KERNEL!!! AND IT IS SET IN CONCRETE!!
READ MY SUBSEQUENT REPLY AS WELL.
>
>> In the case of nfs, buffering is done by the nfsiod.
>> Buffering will be done at both the server AND the client.
>> This is especially noticeable when the nfs client writes onto
>> and nfs mounted filesystem.
>> nfsiod is the "helper" kernel thead. There will be as many of these
>> as the admin configures the system for.
> OK, so I'm the admin on the client system. How to I configure nfsiod?
You can only set the NUMBER of how many to run.
But I have forgotten where it is done - perhaps in
some .conf file in /etc, or a file in /etc/sysconfig/
>
>> Ditto with the main dispatcher,
>> the nfsd process.
>> The nfsiod is what buffers writes on the client side.
> Which means that that is the part which is causing my problems. The
> main problem is the disparity between the size of the cache on the
> client (somewhere around 2GB) and the speed of transfer onto the NFS
> mounted share (around 2MB/s). This means that every time the cache
> needs to be flushed there is a ~20min wait during which the client is
> completely blocked (so no chance to kill it or interact with it in any
> way). Give me a 5MB cache and I'm a happy man
>
>> If you want the scp to function more synchronously, you
>> need to rewrite scp, so that it calls fsync after each write!
>> This will force scp process to wait for the data to be flushed
>> before the write call returns.
> I don't need to be that draconian, just configure a sensible cache size.
> I just can't see where I can set that.
>
Just give it up dude!
I told you already - cache amount CAN NOT BE CHANGED!!

> Thanks for any advice
>
> Simon.
>
>
>
>
--
users mailing list
users@lists.fedoraproject.org
To unsubscribe or change subscription options:
https://admin.fedoraproject.org/mailman/listinfo/users
Guidelines: http://fedoraproject.org/wiki/Mailing_list_guidelines
 
Old 09-29-2010, 06:27 PM
JB
 
Default NFS Buffering

Simon Andrews <simon.andrews <at> bbsrc.ac.uk> writes:

> ...
> >
> >> It looks like the nfs daemon is caching write data (around 2GB of it)
> >> which lets scp think its finished when actually there's loads of data
> >> sitting in a write buffer. The hanging is presumably the time it takes
> >> to flush the buffer (there is a process called nfsiod which is active
> >> during this time and df shows data is still being written).
> > ...
> ...
> $ ps aux | grep -i nfsiod
> root 8974 0.0 0.0 0 0 ? S Sep27 0:15 [nfsiod]
>
> Where the parent process is kthread.
>
> Simon.

This is from HP Tru64 UNIX.
$ man 8 nfsiod
http://h30097.www3.hp.com/docs/base_doc/DOCUMENTATION/V51_HTML/MAN/MAN8/0043____.HTM

JB


--
users mailing list
users@lists.fedoraproject.org
To unsubscribe or change subscription options:
https://admin.fedoraproject.org/mailman/listinfo/users
Guidelines: http://fedoraproject.org/wiki/Mailing_list_guidelines
 
Old 09-29-2010, 07:09 PM
JB
 
Default NFS Buffering

JB <jb.1234abcd <at> gmail.com> writes:

>
> Simon Andrews <simon.andrews <at> bbsrc.ac.uk> writes:
>
> > ...
> > I think it must be part of the kernel. There's no nfsiod binary
> > anywhere on the system, but:
> >
> > $ ps aux | grep -i nfsiod
> > root 8974 0.0 0.0 0 0 ? S Sep27 0:15 [nfsiod]
> >
> > Where the parent process is kthread.
> >
> > Simon.
>
> Thanks.
> Could you please give us (on Fedora 13) an unedited output of:
> # cat /etc/fstab
> # mount
>
> JB
>

Well, actually these :-) :

$ man 8 mount
...
It is possible that files /etc/mtab and /proc/mounts don’t match. The
first file is based only on the mount command options, but the content
of the second file also depends on the kernel and others settings (e.g.
remote NFS server. In particular case the mount command may reports
unreliable information about a NFS mount point and the /proc/mounts
file usually contains more reliable information.)

# cat /etc/mtab
# cat /proc/mounts

JB


--
users mailing list
users@lists.fedoraproject.org
To unsubscribe or change subscription options:
https://admin.fedoraproject.org/mailman/listinfo/users
Guidelines: http://fedoraproject.org/wiki/Mailing_list_guidelines
 
Old 09-29-2010, 07:17 PM
Alan Cox
 
Default NFS Buffering

> YOU CANNOT!!! IT IS DONE BY THE KERNEL!!! AND IT IS SET IN CONCRETE!!
> READ MY SUBSEQUENT REPLY AS WELL.

Your capslock key appears to have failed.


Alan
--
users mailing list
users@lists.fedoraproject.org
To unsubscribe or change subscription options:
https://admin.fedoraproject.org/mailman/listinfo/users
Guidelines: http://fedoraproject.org/wiki/Mailing_list_guidelines
 
Old 09-30-2010, 12:16 AM
Gordon Messmer
 
Default NFS Buffering

On 09/28/2010 07:07 AM, Simon Andrews wrote:
> Does anyone know how to either make this buffer smaller, or get rid of
> it all together so the scp can accruately report on its progress?

Mount the NFS filesystem with the "sync" option.
--
users mailing list
users@lists.fedoraproject.org
To unsubscribe or change subscription options:
https://admin.fedoraproject.org/mailman/listinfo/users
Guidelines: http://fedoraproject.org/wiki/Mailing_list_guidelines
 

Thread Tools




All times are GMT. The time now is 07:04 PM.

VBulletin, Copyright ©2000 - 2014, Jelsoft Enterprises Ltd.
Content Relevant URLs by vBSEO ©2007, Crawlability, Inc.
Copyright ©2007 - 2008, www.linux-archive.org