FAQ Search Today's Posts Mark Forums Read
» Video Reviews

» Linux Archive

Linux-archive is a website aiming to archive linux email lists and to make them easily accessible for linux users/developers.


» Sponsor

» Partners

» Sponsor

Go Back   Linux Archive > CentOS > CentOS

 
 
LinkBack Thread Tools
 
Old 10-17-2008, 05:32 PM
MHR
 
Default ls and rm: "argument list too long"

On Fri, Oct 17, 2008 at 4:09 AM, Jeremy Sanders
<jeremy@jeremysanders.net> wrote:
>
> This limitation has been removed from more recent kernels.
>
> http://git.kernel.org/?p=linux/kernel/git/torvalds/linux-2.6.git;a=commit;h=b6a2fea39318e43fee84fa7b0b90d68 bed92d2ba
>
> http://www.gnu.org/software/coreutils/faq/#Argument-list-too-long
>

This is usually not a kernel issue at all - it is a shell issue. The
limitation is the len
_______________________________________________
CentOS mailing list
CentOS@centos.org
http://lists.centos.org/mailman/listinfo/centos
 
Old 10-17-2008, 06:49 PM
Scott Silva
 
Default ls and rm: "argument list too long"

on 10-17-2008 2:30 AM Jussi Hirvi spake the following:
> Since when is there a limit in how long directory listings CentOS can show
> (ls), or how large directories can be removed (rm). It is really annoying to
> say, for example
>
> rm -rf /var/amavis/tmp
>
> and get only "argument list too long" as feedback.
>
> Is there a way to go round this problem?
>
> I have CentOS 5.2.
>
It isn't a problem with the commands, it is a problem of how long a command
line can be when piped to a command.

rm -rf /var/amavis/tmp is effectively the same as rm -rf /var/amavis/tmp/1
/var/amavis/tmp/2 /var/amavis/tmp/3 /var/amavis/tmp/4 /var/amavis/tmp/5 ...
etc. The number of diles and directories in that folder is the limiting factor.

And yes, Fedora would have the same limitation.

--
MailScanner is like deodorant...
You hope everybody uses it, and
you notice quickly if they don't!!!!

_______________________________________________
CentOS mailing list
CentOS@centos.org
http://lists.centos.org/mailman/listinfo/centos
 
Old 10-17-2008, 08:38 PM
John Kordash
 
Default ls and rm: "argument list too long"

> > rm -rf /var/amavis/tmp
> >
> > and get only "argument list too long" as feedback.
> >
> > Is there a way to go round this problem?
> >
> > I have CentOS 5.2.
> >
> It isn't a problem with the commands, it is a problem of how
> long a command
> line can be when piped to a command.
>
> rm -rf /var/amavis/tmp is effectively the same as rm -rf
> /var/amavis/tmp/1
> /var/amavis/tmp/2 /var/amavis/tmp/3 /var/amavis/tmp/4
> /var/amavis/tmp/5 ...
> etc. The number of diles and directories in that folder is
> the limiting factor.

I don't believe this is correct. The command "rm -rf /path/to/dir" doesn't expand on the shell the same way "rm -rf /path/to/dir/*" would.

Unless I'm misunderstanding your comment, "rm -rf /path/to/dir" will remove everything as intended without blowing out the argument list.

Dealing with file removal and getting 'argument list too long' is a FAQish question, and there is more than one way to get around the issue. Common workarounds include find piped to xargs rm, the above mentioned recursive directory nuke, one line perl scripts, etc.

-John
_______________________________________________
CentOS mailing list
CentOS@centos.org
http://lists.centos.org/mailman/listinfo/centos
 
Old 10-18-2008, 12:58 AM
thad
 
Default ls and rm: "argument list too long"

Satchel Paige - "Don't look back. Something might be gaining on you."


On Fri, Oct 17, 2008 at 4:36 AM, Laurent Wandrebeck
<l.wandrebeck@gmail.com> wrote:
> 2008/10/17 Jussi Hirvi <greenspot@greenspot.fi>:
>> Since when is there a limit in how long directory listings CentOS can show
>> (ls), or how large directories can be removed (rm). It is really annoying to
>> say, for example
>>
>> rm -rf /var/amavis/tmp
>>
>> and get only "argument list too long" as feedback.
>>
>> Is there a way to go round this problem?
>>
>> I have CentOS 5.2.
>>
>> - Jussi
> try something like:
> for i in /var/amavis/tmp/*
> do
> rm -rf $i
> done

it should be:

for i in `ls /var/amavis/tmp`
do
rm $i
done
_______________________________________________
CentOS mailing list
CentOS@centos.org
http://lists.centos.org/mailman/listinfo/centos
 
Old 10-18-2008, 02:11 AM
Les Mikesell
 
Default ls and rm: "argument list too long"

thad wrote:

Satchel Paige - "Don't look back. Something might be gaining on you."


On Fri, Oct 17, 2008 at 4:36 AM, Laurent Wandrebeck
<l.wandrebeck@gmail.com> wrote:

2008/10/17 Jussi Hirvi <greenspot@greenspot.fi>:

Since when is there a limit in how long directory listings CentOS can show
(ls), or how large directories can be removed (rm). It is really annoying to
say, for example

rm -rf /var/amavis/tmp

and get only "argument list too long" as feedback.

Is there a way to go round this problem?

I have CentOS 5.2.

- Jussi

try something like:
for i in /var/amavis/tmp/*
do
rm -rf $i
done


it should be:

for i in `ls /var/amavis/tmp`
do
rm $i
done


These shouldn't make any difference. The limit is on the size of the
expanded shell command line. The original example won't cause it. The
ones that expand a list with a * or the output of ls may. The right
solution is to let rm recurse with -r or to potentially long list to xargs.


--
Les Mikesell
lesmikesell@gmail.com
_______________________________________________
CentOS mailing list
CentOS@centos.org
http://lists.centos.org/mailman/listinfo/centos
 
Old 10-18-2008, 02:56 AM
Robert Nichols
 
Default ls and rm: "argument list too long"

Les Mikesell wrote:

thad wrote:


it should be:

for i in `ls /var/amavis/tmp`
do
rm $i
done


These shouldn't make any difference. The limit is on the size of the
expanded shell command line.


Really?

$ M=0; N=0; for W in `find /usr -xdev 2>/dev/null`; do M=$(($M+1)); N=$(($N+${#W}+1)); done; echo $M $N
156304 7677373

vs.

$ /bin/echo `find /usr -xdev 2>/dev/null`
bash: /bin/echo: Argument list too long

For the first case, the shell never tries to pass the list as command arguments.
It builds the list internally, limited only by memory size, and processes the
words one by one. As a final test case, by using the shell's builtin 'echo'
the whole 7-plus megabytes gets echoed to the terminal:

$ echo `find /usr -xdev 2>/dev/null`
(no errors -- just lots of output)

Anyway, the "for i in `ls ...`" solution breaks for paths that include
embedded white space.

--
Bob Nichols "NOSPAM" is really part of my email address.
Do NOT delete it.

_______________________________________________
CentOS mailing list
CentOS@centos.org
http://lists.centos.org/mailman/listinfo/centos
 
Old 10-18-2008, 04:52 AM
Les Mikesell
 
Default ls and rm: "argument list too long"

Robert Nichols wrote:


These shouldn't make any difference. The limit is on the size of the
expanded shell command line.


Really?

$ M=0; N=0; for W in `find /usr -xdev 2>/dev/null`; do M=$(($M+1));
N=$(($N+${#W}+1)); done; echo $M $N

156304 7677373

vs.

$ /bin/echo `find /usr -xdev 2>/dev/null`
bash: /bin/echo: Argument list too long

For the first case, the shell never tries to pass the list as command
arguments.
It builds the list internally, limited only by memory size, and
processes the

words one by one.


Is that peculiar to bash? I thought the `command` construct was
expanded by shells into the command line before being evaluated.



--
Les Mikesell
lesmikesell@gmail.com
_______________________________________________
CentOS mailing list
CentOS@centos.org
http://lists.centos.org/mailman/listinfo/centos
 
Old 10-18-2008, 10:00 AM
"William L. Maltby"
 
Default ls and rm: "argument list too long"

On Fri, 2008-10-17 at 23:52 -0500, Les Mikesell wrote:
> Robert Nichols wrote:
> >
> >> These shouldn't make any difference. The limit is on the size of the
> >> expanded shell command line.
> >
> > Really?
> >
> > $ M=0; N=0; for W in `find /usr -xdev 2>/dev/null`; do M=$(($M+1));
> > N=$(($N+${#W}+1)); done; echo $M $N
> > 156304 7677373
> >
> > vs.
> >
> > $ /bin/echo `find /usr -xdev 2>/dev/null`
> > bash: /bin/echo: Argument list too long
> >
> > For the first case, the shell never tries to pass the list as command
> > arguments.
> > It builds the list internally, limited only by memory size, and
> > processes the
> > words one by one.
>
> Is that peculiar to bash? I thought the `command` construct was
> expanded by shells into the command line before being evaluated.

IIRC, none of the above make a "command line". Everything but the

`find /usr -xdev 2>/dev/null`

is a bash "internal command". IIRC, what should happen here is a new
instance of bash is spawned as part of a pipeline that sends the output
of the find (which is "exec'd" by that new instance of bash, the child)
into the parent. The parent reads the input from the pipe and can do
whatever it wants, in this case build an array. It then uses the array
as data to the loop.

The "command line" is never constructed with the long list. It is only
passed to the child (the new instance of bash that is part of the
pipeline). That instance receives an argument count and an array of
pointers to the arguments. In "C" parlance it looks something like this.

main(argc, *argv[]) /* could be **argv instead */
{
/* stuff to do */
.
.
.
}

The "*argv[]" pointers point to the parts of the "command line",

find /usr -xdev

The child execs find, passing the "/usr" and "-xdev" as arguments to
find (which has a similar "main" construct), another "command line". The
I/O redirection was already done by the parent, so the child need not
even know that "stdout" is a pipe.

The longest command line in this case is "find /usr -xdev', 15
characters. Find "sees" only 10 characters.

I hope I've remembered correctly, that this is not FUD, and that it
helps someone.

> <snip>

--
Bill

_______________________________________________
CentOS mailing list
CentOS@centos.org
http://lists.centos.org/mailman/listinfo/centos
 
Old 10-18-2008, 10:09 AM
"William L. Maltby"
 
Default ls and rm: "argument list too long"

On Sat, 2008-10-18 at 06:00 -0400, William L. Maltby wrote:
> <snip>

> The longest command line in this case is "find /usr -xdev', 15
> characters. Find "sees" only 10 characters.

Uh, +1 for the that terminates each parameter? Need more java here.

> <snip>

--
Bill

_______________________________________________
CentOS mailing list
CentOS@centos.org
http://lists.centos.org/mailman/listinfo/centos
 
Old 10-18-2008, 11:40 AM
"William L. Maltby"
 
Default ls and rm: "argument list too long"

On Sat, 2008-10-18 at 06:00 -0400, William L. Maltby wrote:
> <snip>

Ok. 3rd cup of coffee has made its way into various of my systems. A
minor correction (but important for us pedantic typers) is below.

> main(argc, *argv[]) /* could be **argv instead */
main(int argc, char *argv[]) /* could be **argv instead */
> {
> /* stuff to do */
> .
> .
> .
> }
> <snip>

--
Bill

_______________________________________________
CentOS mailing list
CentOS@centos.org
http://lists.centos.org/mailman/listinfo/centos
 

Thread Tools




All times are GMT. The time now is 07:47 AM.

VBulletin, Copyright ©2000 - 2014, Jelsoft Enterprises Ltd.
Content Relevant URLs by vBSEO ©2007, Crawlability, Inc.
Copyright 2007 - 2008, www.linux-archive.org