FAQ Search Today's Posts Mark Forums Read
» Video Reviews

» Linux Archive

Linux-archive is a website aiming to archive linux email lists and to make them easily accessible for linux users/developers.


» Sponsor

» Partners

» Sponsor

Go Back   Linux Archive > Debian > Debian User

 
 
LinkBack Thread Tools
 
Old 02-03-2009, 04:14 PM
"Gorka"
 
Default Slow Script

Hi! I've got a perl script with this for:

for (my $j=0;$j<=$#fichero1;$j++)
{
if (@fichero1[$j] eq $valor1)
{
$token = 1;
}
}

The problem is that fichero1 has 32 millions of records and moreover I've
got to repeat this for several millions times, so this way it would take
years to finish.
Does anybody know a way to optimize this script? Is there any other linux
programing language I could make this more quickly whith?
Thank you!




--
To UNSUBSCRIBE, email to debian-user-REQUEST@lists.debian.org
with a subject of "unsubscribe". Trouble? Contact listmaster@lists.debian.org
 
Old 02-03-2009, 04:24 PM
Dave Sherohman
 
Default Slow Script

On Tue, Feb 03, 2009 at 06:14:48PM +0100, Gorka wrote:
> Hi! I've got a perl script with this for:
>
> for (my $j=0;$j<=$#fichero1;$j++)
> {
> if (@fichero1[$j] eq $valor1)
> {
> $token = 1;
> }
> }
>
> The problem is that fichero1 has 32 millions of records and moreover I've
> got to repeat this for several millions times, so this way it would take
> years to finish.
> Does anybody know a way to optimize this script? Is there any other linux
> programing language I could make this more quickly whith?
> Thank you!

Although the Perl could definitely be optimized (and you've already been
shown one way to do so), your core issue is that you're doing several
million passes over 32 million records. That's not going to be fast in
any language. (Even if you can check a million records per second,
that's 32 seconds per pass, or about 6 hours for 1,000 passes, or just
over a year for a million passes.)

Given the small piece of code that you posted and the magnitude of the
numbers you've stated, I strongly suspect that you probably want to use
a database for this, since it's exactly the kind of thing they're
designed to handle, but we can't say much more than that without
additional information about what you're actually trying to accomplish.

--
Dave Sherohman
NomadNet, Inc.
http://nomadnetinc.com/


--
To UNSUBSCRIBE, email to debian-user-REQUEST@lists.debian.org
with a subject of "unsubscribe". Trouble? Contact listmaster@lists.debian.org
 
Old 02-03-2009, 04:27 PM
"Eugene V. Lyubimkin"
 
Default Slow Script

Gorka wrote:
> Hi! I've got a perl script with this for:
>
> for (my $j=0;$j<=$#fichero1;$j++)
> {
> if (@fichero1[$j] eq $valor1)
> {
> $token = 1;
> }
> }
>
Try

foreach (@fichero1)
{
if ($_ eq $valor1)
{
$token = 1;
break;
}
}

--
Eugene V. Lyubimkin aka JackYF, JID: jackyf.devel(maildog)gmail.com
Ukrainian C++ Developer, Debian Maintainer, APT contributor
 

Thread Tools




All times are GMT. The time now is 08:04 AM.

VBulletin, Copyright ©2000 - 2014, Jelsoft Enterprises Ltd.
Content Relevant URLs by vBSEO ©2007, Crawlability, Inc.
Copyright 2007 - 2008, www.linux-archive.org