FAQ Search Today's Posts Mark Forums Read
» Video Reviews

» Linux Archive

Linux-archive is a website aiming to archive linux email lists and to make them easily accessible for linux users/developers.


» Sponsor

» Partners

» Sponsor

Go Back   Linux Archive > CentOS > CentOS

 
 
LinkBack Thread Tools
 
Old 08-18-2011, 09:36 AM
Marc Deop i Argemí
 
Default which firewall to automatically block bandwidth abusers?

On 18/08/2011 4:13, Craig White wrote:
> On Wed, 2011-08-17 at 21:50 +0200, Rudi Ahlers wrote:
>> Hi,
>>
>> I'm looking for a firewall (preferably on Linux / UNIX) that could
>> automatically block bandwidth abusers as soon as a connection goes
>> over a certain speed, or limit - i.e. either more than say 3Mb/s or
>> 10GB in a giving period (like weekly / monthly).
>>
>> But, I need it to block the IP to, or where the traffic comes from, or
>> goes to. i.e. a user logs into a web server and upload a LOT of data,
>> then the firewall should block him, but not other people.
>>
>> Or, someone uploads a small bit of data but downloads a lot of data
>> and then get's blocked.
>> But I need to set thresholds
>> And I should be able to exclude certain IP's / domains from the limits.
>>
>> Does this make sense?
>>
>> Can this be done with iptables? If so, how?
>>
>> If not, what else could I use for this?
>>
>>
>> A normal DDOS prevention firewall doesn't really work since it only
>> blocks traffic coming in. But I need to limit traffic going out as
>> well.
>>
>> The servers behind the firewall will serve mail, http, ftp, sql and SSH
> ----
> http://tinyurl.com/3n5yn8u

Would you mind providing the url without using such url shorteners?

Thanks,

Regards
_______________________________________________
CentOS mailing list
CentOS@centos.org
http://lists.centos.org/mailman/listinfo/centos
 
Old 08-18-2011, 10:14 AM
Rudi Ahlers
 
Default which firewall to automatically block bandwidth abusers?

On Thu, Aug 18, 2011 at 4:13 AM, Craig White <craigwhite@azapple.com> wrote:
> On Wed, 2011-08-17 at 21:50 +0200, Rudi Ahlers wrote:
>> Hi,
>>
>> I'm looking for a firewall (preferably on Linux / UNIX) that could
>> automatically block bandwidth abusers as soon as a connection goes
>> over a certain speed, or limit - i.e. either more than say 3Mb/s or
>> 10GB in a giving period (like weekly / monthly).
>>
>> But, I need it to block the IP to, or where the traffic comes from, or
>> goes to. i.e. a user logs into a web server and upload a LOT of data,
>> then the firewall should block him, but not other people.
>>
>> Or, someone uploads a small bit of data but downloads a lot of data
>> and then get's blocked.
>> But I need to set thresholds
>> And I should be able to exclude certain IP's / domains from the limits.
>>
>> Does this make sense?
>>
>> Can this be done with iptables? If so, how?
>>
>> If not, what else could I use for this?
>>
>>
>> A normal DDOS prevention firewall doesn't really work since it only
>> blocks traffic coming in. But I need to limit traffic going out as
>> well.
>>
>> The servers behind the firewall will serve mail, http, ftp, sql and SSH
> ----
> http://tinyurl.com/3n5yn8u
>
> Craig


We already monitor traffic usage on the switches with cacti via SNMP.


But, I need to block traffic abusers automatically. from any IP
address, to any IP address.

The firewalls we have, and have tested all need a set of IP addresses
to throttle, which won't work in this case.
A user can login from any IP address on the internet, and either
upload or download exsesively and we need to block that IP address as
soon as it's reaches a certain (pre-set by us) threshold


--
Kind Regards
Rudi Ahlers
SoftDux

Website: http://www.SoftDux.com
Technical Blog: http://Blog.SoftDux.com
Office: 087 805 9573
Cell: 082 554 7532
_______________________________________________
CentOS mailing list
CentOS@centos.org
http://lists.centos.org/mailman/listinfo/centos
 
Old 08-18-2011, 06:45 PM
Rudi Ahlers
 
Default which firewall to automatically block bandwidth abusers?

On Thu, Aug 18, 2011 at 7:20 PM, Patrick Lists
<centos-list@puzzled.xs4all.nl> wrote:
> On 08/18/2011 12:06 PM, Always Learning wrote:
>>
>> On Thu, 2011-08-18 at 11:36 +0200, Marc Deop i Argemí wrote:
>>
>>>> http://tinyurl.com/3n5yn8u
>>
>> It gives me:-
>>
>> http://lmgtfy.com/?q=traffic+accounting
>>
>> which displays a Google search box and an advertisement with nothing
>> about Traffic or about Accounting.
>
> Lmgtfy means "let me google that for you". Posting such an url is a
> pretty standard response to people who ask for help without first making
> an effort to find some answers (by googling, etc.). The hint is: do your
> homework first and don't expect spoonfeeding.
>
> Regards,
> Patrick
> _______________________________________________



And you obviously think I didn't do my homework?


Did you see my specific requirement? Or did you just see "how" and
"firewall" and assumed "google" ?


--
Kind Regards
Rudi Ahlers
SoftDux

Website: http://www.SoftDux.com
Technical Blog: http://Blog.SoftDux.com
Office: 087 805 9573
Cell: 082 554 7532
_______________________________________________
CentOS mailing list
CentOS@centos.org
http://lists.centos.org/mailman/listinfo/centos
 
Old 08-18-2011, 07:01 PM
Rudi Ahlers
 
Default which firewall to automatically block bandwidth abusers?

Let's try again:


I need to automatically block any user who abuses bandwidth, either
incoming or outgoing. I should be able to set the limits, in either
rate/s or usage/s: 1Mb/s or 10GB/h, for example.

Then, any users, connecting from anywhere, on any IP should be blocked
- either if he uploads or downloads (i.e ingres & outgres) for a
specific amount of time.


My research:

The firewalls which we've tried (both normal Linux iptables and
hardware based firewalls) can do this, as long as I can specify the
IP's to block - this is standard for an office-type firewall.
BUT, I don't have a range of IP's to specify since these particular
servers are on the internet, thus any possible IP on the net could
connect to the server.


I also need to exclude certain IP's from this rule (i.e. for backup
servers which actually need to transfer a lot of traffic).

To some degree this would mean "traffic accounting", but that just
keeps a log of traffic usage. And we already measure traffic use with
cacti & SNMP. Cacti can send us an email if a certain amount of
bandwidth is used up, but it doesn't tell the firewall to block the
offending IP address.

DDOS protection type firewalls doesn't help much either since they
only block incoming "attacks", but not really normal uploads. They
also don't block outgoing traffic once the condition is met.

--
Kind Regards
Rudi Ahlers
SoftDux

Website: http://www.SoftDux.com
Technical Blog: http://Blog.SoftDux.com
Office: 087 805 9573
Cell: 082 554 7532
_______________________________________________
CentOS mailing list
CentOS@centos.org
http://lists.centos.org/mailman/listinfo/centos
 
Old 08-18-2011, 07:15 PM
Rudi Ahlers
 
Default which firewall to automatically block bandwidth abusers?

On Thu, Aug 18, 2011 at 9:09 PM, Always Learning <centos@u61.u22.net> wrote:
>
> On Thu, 2011-08-18 at 21:01 +0200, Rudi Ahlers wrote:
>
>> I need to automatically block any user who abuses bandwidth, either
>> incoming or outgoing. I should be able to set the limits, in either
>> rate/s or usage/s: 1Mb/s or 10GB/h, for example.
>
> First question is:
>
> (a) how can you get the IP address ?

I don't fully understand your question?
How do you get any IP address from any machine that connects to a
server on the internet? netstat shows the IP's,
/var/log/http/access.log shows the IP's and I'm sure it's listed in
other places as well.

We currently use ntop to monitor the server's usage, but there's no
way to automatically block an abusive IP.


>
> (b) how can you introduce a, or use an existing, system to record and
> store the data amounts (bandwidth) and IP addresses ?

What do you mean?


>
> (c) how long will this information be retained before being discarded ?

How long will what information be retained? And what for? I don't
understand the nature of this question?

>
> (d) how can you monitor on every change to the data amount ?

Again, I don't understand what you mean?


>
> (e) will it do both IP4 and IP6 ?

Does it matter? IPV6 is already being used on a wide scale. iptables
support both

>
> (f) what mechanism can you use to block the IP address ... IP Tables via
> simple BASH command ?

if that will do the trick, yes. Any way to block the IP would be fine.
iptables would probably be easiest.


Ideally I would like to get a dedicated firewall, or dedicated Linux /
UNIX firewall appliance for this purpose as it needs to monitor and
protect a whole bunch of servers

>
>
> Its an interesting requirement.
>
>
>
>
> --
> With best regards,
>
> Paul.
> England,
> EU.
>







--
Kind Regards
Rudi Ahlers
SoftDux

Website: http://www.SoftDux.com
Technical Blog: http://Blog.SoftDux.com
Office: 087 805 9573
Cell: 082 554 7532
_______________________________________________
CentOS mailing list
CentOS@centos.org
http://lists.centos.org/mailman/listinfo/centos
 
Old 08-18-2011, 07:21 PM
Les Mikesell
 
Default which firewall to automatically block bandwidth abusers?

On 8/18/2011 2:01 PM, Rudi Ahlers wrote:
> Let's try again:
>
>
> I need to automatically block any user who abuses bandwidth, either
> incoming or outgoing. I should be able to set the limits, in either
> rate/s or usage/s: 1Mb/s or 10GB/h, for example.
>
> Then, any users, connecting from anywhere, on any IP should be blocked
> - either if he uploads or downloads (i.e ingres& outgres) for a
> specific amount of time.

Those requirements don't mesh very well with the real world. That is,
people use use a network that they've been provided or paid for aren't
necessarily 'abusing' anything, and blocking access at times when the
network isn't fully loaded doesn't help anyone. What's the big picture
here? Don't you really need QOS to throttle certain things at peak
times only?

--
Les Mikesell
lesmikesell@gmail.com

_______________________________________________
CentOS mailing list
CentOS@centos.org
http://lists.centos.org/mailman/listinfo/centos
 
Old 08-18-2011, 07:25 PM
Mike
 
Default which firewall to automatically block bandwidth abusers?

On Thu, 18 Aug 2011, Rudi Ahlers wrote:

> Let's try again:
>
>
> I need to automatically block any user who abuses bandwidth, either
> incoming or outgoing. I should be able to set the limits, in either
> rate/s or usage/s: 1Mb/s or 10GB/h, for example.
>
> Then, any users, connecting from anywhere, on any IP should be blocked
> - either if he uploads or downloads (i.e ingres & outgres) for a
> specific amount of time.
>

As one might imagine there is at least one commercial product that seems
to fit the bill.

http://www.aspirantinfotech.com/downloads/Cyberoam/pdf/Managing-bandwidth-the-User-based-approach.pdf

I mention this as I thought it was well written and thorough. After
reading the pdf seems to me there ought to be something open source based
upon perhaps this: http://lartc.org/lartc.html

Anyway maybe some food for thought.
_______________________________________________
CentOS mailing list
CentOS@centos.org
http://lists.centos.org/mailman/listinfo/centos
 
Old 08-18-2011, 07:27 PM
Rudi Ahlers
 
Default which firewall to automatically block bandwidth abusers?

On Thu, Aug 18, 2011 at 9:21 PM, Les Mikesell <lesmikesell@gmail.com> wrote:
> On 8/18/2011 2:01 PM, Rudi Ahlers wrote:
>> Let's try again:
>>
>>
>> I need to automatically block any user who abuses bandwidth, either
>> incoming or outgoing. I should be able to set the limits, in either
>> rate/s or usage/s: 1Mb/s or 10GB/h, for example.
>>
>> Then, any users, connecting from anywhere, on any IP should be blocked
>> - either if he uploads or downloads (i.e ingres& *outgres) for a
>> specific amount of time.
>
> Those requirements don't mesh very well with the real world. *That is,
> people use use a network that they've been provided or paid for aren't
> necessarily 'abusing' anything, and blocking access at times when the
> network isn't fully loaded doesn't help anyone. *What's the big picture
> here? *Don't you really need QOS to throttle certain things at peak
> times only?
>
> --
> * Les Mikesell
> * *lesmikesell@gmail.com
>
> _______________________________________________


Les, it's not really about blocking people who paid.

the servers in question provide a free service and no money is
generated from it, but the client still pays for bandwidth so we'd
like to cap heavy users a bit to avoid expensive bills.


I know the requirements are strange, but I'm really hoping I could
find something that could do this for us.
Right now they have someone who monitors ntop and block IP's that way
around, but it's inefficient and a salary which could have been spent
elsewhere.

Bandwidth in our country is exuberantly expensive, probably about 20x
the price of bandwidth in the USA



--
Kind Regards
Rudi Ahlers
SoftDux

Website: http://www.SoftDux.com
Technical Blog: http://Blog.SoftDux.com
Office: 087 805 9573
Cell: 082 554 7532
_______________________________________________
CentOS mailing list
CentOS@centos.org
http://lists.centos.org/mailman/listinfo/centos
 
Old 08-18-2011, 07:29 PM
Les Mikesell
 
Default which firewall to automatically block bandwidth abusers?

On 8/18/2011 2:15 PM, Rudi Ahlers wrote:
> On Thu, Aug 18, 2011 at 9:09 PM, Always Learning<centos@u61.u22.net> wrote:
>>
>> On Thu, 2011-08-18 at 21:01 +0200, Rudi Ahlers wrote:
>>
>>> I need to automatically block any user who abuses bandwidth, either
>>> incoming or outgoing. I should be able to set the limits, in either
>>> rate/s or usage/s: 1Mb/s or 10GB/h, for example.
>>
>> First question is:
>>
>> (a) how can you get the IP address ?
>
> I don't fully understand your question?
> How do you get any IP address from any machine that connects to a
> server on the internet? netstat shows the IP's,

You said 'user' which may or may not map to a consistent, single, IP
address.

> /var/log/http/access.log shows the IP's and I'm sure it's listed in
> other places as well.

Are these web browser clients, locally attached PCs, or what?

> We currently use ntop to monitor the server's usage, but there's no
> way to automatically block an abusive IP.

What's 'abusive'? If they are using a web app, let the app monitor the
connection of a logged in user and handle them appropriately.

>
> Ideally I would like to get a dedicated firewall, or dedicated Linux /
> UNIX firewall appliance for this purpose as it needs to monitor and
> protect a whole bunch of servers

A separate box won't know what is going on. Suppose you have a remote
mail server relaying in or out for a large number of users. The
intermediate box will see a lot of smtp traffic to/from one IP, but it
will correspond to a lot of users. Likewise for web users behind a
company proxy.

--
Les Mikesell
lesmikesell@gmail.com
_______________________________________________
CentOS mailing list
CentOS@centos.org
http://lists.centos.org/mailman/listinfo/centos
 
Old 08-18-2011, 07:31 PM
Rudi Ahlers
 
Default which firewall to automatically block bandwidth abusers?

On Thu, Aug 18, 2011 at 9:25 PM, Mike <mike@microdel.org> wrote:
> On Thu, 18 Aug 2011, Rudi Ahlers wrote:
>
>> Let's try again:
>>
>>
>> I need to automatically block any user who abuses bandwidth, either
>> incoming or outgoing. I should be able to set the limits, in either
>> rate/s or usage/s: 1Mb/s or 10GB/h, for example.
>>
>> Then, any users, connecting from anywhere, on any IP should be blocked
>> - either if he uploads or downloads (i.e ingres & outgres) for a
>> specific amount of time.
>>
>
> As one might imagine there is at least one commercial product that seems
> to fit the bill.
>
> http://www.aspirantinfotech.com/downloads/Cyberoam/pdf/Managing-bandwidth-the-User-based-approach.pdf
>
> I mention this as I thought it was well written and thorough. *After
> reading the pdf seems to me there ought to be something open source based
> upon perhaps this: *http://lartc.org/lartc.html
>
> Anyway maybe some food for thought.
> _______________________________________________
>


Thanx. We already tried the cyberoams, but they didn't work as
expected since they manage bandwidth on a per-user basis, and our
"users" come from the world-wide-web.


I have read through that document link on
http://lartc.org/lartc.html#AEN1393 and the closest I could get is
rate limiting, but that doesn't actually block the IP if it goes over
a certain threshold, it just slows everything down.


--
Kind Regards
Rudi Ahlers
SoftDux

Website: http://www.SoftDux.com
Technical Blog: http://Blog.SoftDux.com
Office: 087 805 9573
Cell: 082 554 7532
_______________________________________________
CentOS mailing list
CentOS@centos.org
http://lists.centos.org/mailman/listinfo/centos
 

Thread Tools




All times are GMT. The time now is 09:04 AM.

VBulletin, Copyright ©2000 - 2014, Jelsoft Enterprises Ltd.
Content Relevant URLs by vBSEO ©2007, Crawlability, Inc.
Copyright ©2007 - 2008, www.linux-archive.org