FAQ Search Today's Posts Mark Forums Read
» Video Reviews

» Linux Archive

Linux-archive is a website aiming to archive linux email lists and to make them easily accessible for linux users/developers.


» Sponsor

» Partners

» Sponsor

Go Back   Linux Archive > Debian > Debian User

 
 
LinkBack Thread Tools
 
Old 10-09-2010, 11:51 AM
Zhang Weiwu
 
Default a data rate threshold tool?

Hello. I found my application, which was supposed to run 10 hours,
either run and starve every other service on it, or doesn't run at all.
Even with nice and ionice, I cannot avoid other web services starved by
this application. And it has to run on the production server.

One trick that comes to my mind is to use a data rate threshold tool to
limit the data rate input is fed to the application.

$ threshold --rate 20KB/s < data_source | my_greedy_application

Is there such a tool in Debian?

Thanks. Running Debian 5.


--
To UNSUBSCRIBE, email to debian-user-REQUEST@lists.debian.org
with a subject of "unsubscribe". Trouble? Contact listmaster@lists.debian.org
Archive: 4CB05732.1000100@realss.com">http://lists.debian.org/4CB05732.1000100@realss.com
 
Old 10-09-2010, 12:23 PM
John Hasler
 
Default a data rate threshold tool?

Please tell us exactly what your application does, what you mean by
"data rate", and what your "data_source" is. It sounds like the problem
is a bug in your application.
--
John Hasler


--
To UNSUBSCRIBE, email to debian-user-REQUEST@lists.debian.org
with a subject of "unsubscribe". Trouble? Contact listmaster@lists.debian.org
Archive: 87lj67368a.fsf@thumper.dhh.gt.org">http://lists.debian.org/87lj67368a.fsf@thumper.dhh.gt.org
 
Old 10-09-2010, 12:58 PM
Chris Jackson
 
Default a data rate threshold tool?

Zhang Weiwu wrote:

> > Hello. I found my application, which was supposed to run 10 hours,
> > either run and starve every other service on it, or doesn't run at all.
> > Even with nice and ionice, I cannot avoid other web services starved by
> > this application. And it has to run on the production server.
> >
> > One trick that comes to my mind is to use a data rate threshold tool to
> > limit the data rate input is fed to the application.
> >
> > $ threshold --rate 20KB/s < data_source | my_greedy_application
> >
> > Is there such a tool in Debian?


I don't know about a tool, but assuming it's not important that the rate
is precise as long as it's somewhere about right, a simple perl script
will do it. You can probably do similar things with python etc.

The following takes a rate in bytes a second and optionally a block size
in bytes, and writes a block of data at the appropriate interval (no
--rate - so if you save it as "threshold" invoke it as threshold 20480
- for 20kB/s). Default block size is 1k. Note that I only just wrote
it, so YMMV and take it as an example for experimentation Should be
fairly obvious how it works; note it'll do silly things if you give it
silly arguments, in particular very large block sizes.

-- start of script

#!/usr/bin/perl

use strict;
use warnings;

use Time::HiRes qw(sleep);

my $rate=$ARGV[0];
if(!$rate) {
print STDERR "No rate given
";
exit 1;
}

my $block_sz=$ARGV[1];
$block_sz=1024 unless $block_sz;

my $delay=$block_sz/$rate;

my $bytes;
my $nbytes;
while($nbytes=sysread STDIN, $bytes, $block_sz) {
while($nbytes) {
my $wbytes=syswrite STDOUT, $bytes, $nbytes;
if(!defined($wbytes)) {
print STDERR "error writing: $!
";
exit 1;
}
$nbytes-=$wbytes;
}
sleep($delay);
}

if(!defined($nbytes)) {
print STDERR "error reading: $!
";
exit 1;
}

exit 0;

-- end of script

--
Chris Jackson
Shadowcat Systems Ltd.


--
To UNSUBSCRIBE, email to debian-user-REQUEST@lists.debian.org
with a subject of "unsubscribe". Trouble? Contact listmaster@lists.debian.org

Archive: 4CB066FC.30003@shadowcat.co.uk">http://lists.debian.org/4CB066FC.30003@shadowcat.co.uk
 
Old 10-09-2010, 01:48 PM
Timo Juhani Lindfors
 
Default a data rate threshold tool?

Zhang Weiwu <zhangweiwu@realss.com> writes:
> $ threshold --rate 20KB/s < data_source | my_greedy_application

pv --rate-limit 20k < data_source | my_greedy_application


--
To UNSUBSCRIBE, email to debian-user-REQUEST@lists.debian.org
with a subject of "unsubscribe". Trouble? Contact listmaster@lists.debian.org
Archive: 84iq1bfped.fsf@sauna.l.org">http://lists.debian.org/84iq1bfped.fsf@sauna.l.org
 
Old 10-09-2010, 04:18 PM
green
 
Default a data rate threshold tool?

Zhang Weiwu wrote at 2010-10-09 05:51 -0600:
> One trick that comes to my mind is to use a data rate threshold tool to
> limit the data rate input is fed to the application.
>
> $ threshold --rate 20KB/s < data_source | my_greedy_application
>
> Is there such a tool in Debian?

If you want to control network usage, you could try trickle.
 
Old 10-15-2010, 02:09 AM
Zhang Weiwu
 
Default a data rate threshold tool?

Thanks for all of you. The answer using pv(1) is what I need.

pv is a 'pipe viewer' that display the progress of a command-line pipe.
It might also able to display other pipes. It is new to me that the tool
perform data rate limit task as well.

A brief check shows trickle, as suggested, is a tool for network
bandwidth limit, that might also be useful in cases.


--
To UNSUBSCRIBE, email to debian-user-REQUEST@lists.debian.org
with a subject of "unsubscribe". Trouble? Contact listmaster@lists.debian.org
Archive: 4CB7B7DB.2050809@realss.com">http://lists.debian.org/4CB7B7DB.2050809@realss.com
 

Thread Tools




All times are GMT. The time now is 01:37 PM.

VBulletin, Copyright ©2000 - 2014, Jelsoft Enterprises Ltd.
Content Relevant URLs by vBSEO ©2007, Crawlability, Inc.
Copyright 2007 - 2008, www.linux-archive.org