Walton Hoops wrote:
> I could use some help from the Ubuntu wizards out there.
> I run a home server, using Ubuntu 8.10. It very low traffic, with most of
> the traffic being e-mail. Last night, over the course of an hour it
> recorded roughly 8 GB (4 up and 4 down) of traffic over the course of 2
> hours (monitoring with vnstat) and then dropped back to normal. Looking at
> the logs, the traffic did not come through apache, sendmail,or SSH. Judging
> from the fact that the up/down are equal, I'm guessing I've was used as a
> proxy for something (I don't have a proxy server installed), but I know not
> what. So, I have two questions.
> 1.) Any suggestions on how to further investigate this? At this point I'm
> at a loss.
> 2.) How would you suggest further hardening my security, since it seems it
> was compromised? I use Firestarter to lock down my ports, Fail2Ban to stop
> those pesky SSH brute force attacks, and Snort to keep an eye out for other
> attacks.
> Any input would be appreciated.
> Walton
>
>

I am no security expert, but I would suggest you take out the network
cable of that machine till you are sure it has not been compromised.

Are there other machines on this network?

--

Please reply to this list only. I read this list on its corresponding
newsgroup on gmane.org. Replies sent to my email address are just
filtered to a folder in my mailbox and get periodically deleted without
ever having been read.


--
ubuntu-users mailing list
ubuntu-users@lists.ubuntu.com
Modify settings or unsubscribe at: https://lists.ubuntu.com/mailman/listinfo/ubuntu-users

On Thu, 2009-02-12 at 13:43 -0500, H.S. wrote:


Walton Hoops wrote:
> I could use some help from the Ubuntu wizards out there.
> I run a home server, using Ubuntu 8.10. It very low traffic, with most of
> the traffic being e-mail. Last night, over the course of an hour it
> recorded roughly 8 GB (4 up and 4 down) of traffic over the course of 2
> hours (monitoring with vnstat) and then dropped back to normal. Looking at
> the logs, the traffic did not come through apache, sendmail,or SSH. Judging
> from the fact that the up/down are equal, I'm guessing I've was used as a
> proxy for something (I don't have a proxy server installed), but I know not
> what. So, I have two questions.
> 1.) Any suggestions on how to further investigate this? At this point I'm
> at a loss.
> 2.) How would you suggest further hardening my security, since it seems it
> was compromised? I use Firestarter to lock down my ports, Fail2Ban to stop
> those pesky SSH brute force attacks, and Snort to keep an eye out for other
> attacks.
> Any input would be appreciated.
> Walton
>
>

I am no security expert, but I would suggest you take out the network
cable of that machine till you are sure it has not been compromised.

Are there other machines on this network?




First place I would start is to check your logs for successful ssh connections:

grep Accepted /var/log/auth.log

This will show you date/time and IP address of successful SSH connections.

To check if any of these have gained root permissions through su:

grep Successful /var/log/auth.log

What services are open on the machine in question?

I would then check the logs for each of the services in question around the time of the activity to see in detail exactly which service generated/consumed the traffic.

At what point are you observing the bandwidth, is this on a router, on the eth0 interface of the machine or an aggregate of all interfaces?

Is it theoretical that 4G out and then in could be on a loopback interface?



Guy.


--
ubuntu-users mailing list
ubuntu-users@lists.ubuntu.com
Modify settings or unsubscribe at: https://lists.ubuntu.com/mailman/listinfo/ubuntu-users

I had already checked the SSH logs, and just checked 'em again using the grep lines you suggested. The last time anyone sshed in was 3 days prior, and it was me :-). Su was not used at all.
The open services on the machine are:
SSH - which we covered
IMAPS (Dovecot) - Showed no unusual activity, just the usual spam from my filters
STMP/STMPS (Sendmail) - Also showed no unusual activity
MySQL - Shows only logins from Wordpress and PHPBB
HTTP/HTTPS (Apache) - Just googlebot (my page doesn't get many visitors), and me checking vnstat.

All other ports are closed both on the router and by the firestarter firewall. A port scan confirms this.

The bandwidth is being recorded for eth1 by vnstat, (I wish it could all be explained away by loopback).

I am noticing something else strange though, I just checked collectd, and it's not showing any significant bandwidth on the interface for that time. Could vnstat have gone crazy? (God I wish it was that easy, but it never is :-)).

Any more thoughts?
Thanks again!
Walton



From: ubuntu-users-bounces@lists.ubuntu.com [mailto:ubuntu-users-bounces@lists.ubuntu.com] On Behalf Of Guy Thouret
Sent: Thursday, February 12, 2009 11:55 AM
To: Ubuntu user technical support, not for general discussions
Subject: Re: Security Issue

On Thu, 2009-02-12 at 13:43 -0500, H.S. wrote:

Walton Hoops wrote:
> I could use some help from the Ubuntu wizards out there.
> I run a home server, using Ubuntu 8.10. It very low traffic, with most of
> the traffic being e-mail. Last night, over the course of an hour it
> recorded roughly 8 GB (4 up and 4 down) of traffic over the course of 2
> hours (monitoring with vnstat) and then dropped back to normal. Looking at
> the logs, the traffic did not come through apache, sendmail,or SSH. Judging
> from the fact that the up/down are equal, I'm guessing I've was used as a
> proxy for something (I don't have a proxy server installed), but I know not
> what. So, I have two questions.
> 1.) Any suggestions on how to further investigate this? At this point I'm
> at a loss.
> 2.) How would you suggest further hardening my security, since it seems it
> was compromised? I use Firestarter to lock down my ports, Fail2Ban to stop
> those pesky SSH brute force attacks, and Snort to keep an eye out for other
> attacks.
> Any input would be appreciated.
> Walton
>
>

I am no security expert, but I would suggest you take out the network
cable of that machine till you are sure it has not been compromised.

Are there other machines on this network?

First place I would start is to check your logs for successful ssh connections:

grep Accepted /var/log/auth.log
This will show you date/time and IP address of successful SSH connections.
To check if any of these have gained root permissions through su:

grep Successful /var/log/auth.log
What services are open on the machine in question?
I would then check the logs for each of the services in question around the time of the activity to see in detail exactly which service generated/consumed the traffic.
At what point are you observing the bandwidth, is this on a router, on the eth0 interface of the machine or an aggregate of all interfaces?
Is it theoretical that 4G out and then in could be on a loopback interface?

Guy.


--
ubuntu-users mailing list
ubuntu-users@lists.ubuntu.com
Modify settings or unsubscribe at: https://lists.ubuntu.com/mailman/listinfo/ubuntu-users

On Feb 12, 2009, at 1:23 PM, Walton Hoops wrote:

> I had already checked the SSH logs, and just checked 'em again using
> the grep lines you suggested. The last time anyone sshed in was 3
> days prior, and it was me :-). Su was not used at all.
> The open services on the machine are:
> SSH - which we covered
> IMAPS (Dovecot) - Showed no unusual activity, just the usual spam
> from my filters
> STMP/STMPS (Sendmail) - Also showed no unusual activity
> MySQL - Shows only logins from Wordpress and PHPBB
> HTTP/HTTPS (Apache) - Just googlebot (my page doesn't get many
> visitors), and me checking vnstat.

Is your PHPBB installation up to date with the latest version/
patches? PHPBB is notorious for being a vector for security
breaches. It is possible someone hacked your machine through PHPBB.
I would double-check your apache logs for any odd transfers during
that time-frame. Also, do a netstat -tap to double check those are
the only services open on your box.

Outside of that, do you have your system set up to automatically
download / install Ubuntu updates? I know this is a new option in
Intrepid. It is possible that's when your system decided to run its
updates. I don't use it personally, but I believe the logs for it are
stored in /var/log/unattended-upgrades You can also check /var/log/
apt/term.log* and/or /var/log/aptitude to see if apt did anything
during that time.


--
ubuntu-users mailing list
ubuntu-users@lists.ubuntu.com
Modify settings or unsubscribe at: https://lists.ubuntu.com/mailman/listinfo/ubuntu-users

Preston Kutzner wrote:

>
> Outside of that, do you have your system set up to automatically
> download / install Ubuntu updates? I know this is a new option in
> Intrepid. It is possible that's when your system decided to run its

4 GB of updates? Is that realistic? Yes, it should at least be ruled out
first.

To the OP: BTW, 4 GB is fairly close to a typical DVD image. Does that
ring any bell?




--

Please reply to this list only. I read this list on its corresponding
newsgroup on gmane.org. Replies sent to my email address are just
filtered to a folder in my mailbox and get periodically deleted without
ever having been read.


--
ubuntu-users mailing list
ubuntu-users@lists.ubuntu.com
Modify settings or unsubscribe at: https://lists.ubuntu.com/mailman/listinfo/ubuntu-users

Automatic updates did run at 7 a.m., but this happened at 10 p.m. Besides,
4 Gigs of updates seems excessive :-P.
I did double check the apache logs, and for good measure did a grep for
POST. Nothing... just Google Bot and some script kiddies attempt at the
roundcube exploit (I wish they'd figure out I don't run roundcube).
Your right about PHPBB being a security nightmare though, and it's not in
use anymore so I think I'll get rid of it to be safe.
Oh, the output from netstat -tap:
Proto Recv-Q Send-Q Local Address Foreign Address State
PID/Program name
tcp 0 0 *:imaps *:* LISTEN
6925/dovecot
tcp 0 0 *:mysql *:* LISTEN
5814/mysqld
tcp 0 0 *:submission *:* LISTEN
7047/sendmail: MTA:
tcp 0 0 *:www *:* LISTEN
8568/apache2
tcp 0 0 *:sieve *:* LISTEN
6925/dovecot
tcp 0 0 *:ssmtp *:* LISTEN
7047/sendmail: MTA:
tcp 0 0 192.168.1.100:domain *:* LISTEN
5690/named
tcp 0 0 <removed>:domain *:* LISTEN
5690/named
tcp 0 0 *:ssh *:* LISTEN
5715/sshd
tcp 0 0 <removed>:ipp *:* LISTEN
5902/cupsd
tcp 0 0 *:smtp *:* LISTEN
7047/sendmail: MTA:
tcp 0 0 <removed>:953 *:* LISTEN
5690/named
tcp 0 0 <removed>:6010 *:* LISTEN
24435/0
tcp 0 0 *:https *:* LISTEN
8568/apache2
tcp 0 0 <removed>:ssh <removed>:38048
ESTABLISHED 9686/sshd: walton [
tcp 0 0 192.168.1.100:imaps <removed>:32238
ESTABLISHED 22971/imap-login
tcp 0 0 192.168.1.100:imaps <removed>:22279
ESTABLISHED 22288/imap-login
tcp 0 0 <removed>:38048 <removed>:ssh
ESTABLISHED 9681/ssh
tcp 0 17013 192.168.1.100:imaps <removed>:31897
ESTABLISHED 23859/imap-login
tcp 0 300 192.168.1.100:ssh <removed>:30936
ESTABLISHED 24427/sshd: walton
tcp6 0 0 [::]:imaps [::]:* LISTEN
6925/dovecot
tcp6 0 0 [::]:domain [::]:* LISTEN
5690/named
tcp6 0 0 [::]:ssh [::]:* LISTEN
5715/sshd
tcp6 0 0 ip6-localhost:953 [::]:* LISTEN
5690/named
tcp6 0 0 ip6-localhost:6010 [::]:* LISTEN
24435/0

nothing suspicious looking to me.
Walton
-----Original Message-----
From: ubuntu-users-bounces@lists.ubuntu.com
[mailto:ubuntu-users-bounces@lists.ubuntu.com] On Behalf Of Preston Kutzner
Sent: Thursday, February 12, 2009 1:09 PM
To: Ubuntu user technical support, not for general discussions
Subject: Re: Security Issue

On Feb 12, 2009, at 1:23 PM, Walton Hoops wrote:

> I had already checked the SSH logs, and just checked 'em again using
> the grep lines you suggested. The last time anyone sshed in was 3
> days prior, and it was me :-). Su was not used at all.
> The open services on the machine are:
> SSH - which we covered
> IMAPS (Dovecot) - Showed no unusual activity, just the usual spam
> from my filters
> STMP/STMPS (Sendmail) - Also showed no unusual activity
> MySQL - Shows only logins from Wordpress and PHPBB
> HTTP/HTTPS (Apache) - Just googlebot (my page doesn't get many
> visitors), and me checking vnstat.

Is your PHPBB installation up to date with the latest version/
patches? PHPBB is notorious for being a vector for security
breaches. It is possible someone hacked your machine through PHPBB.
I would double-check your apache logs for any odd transfers during
that time-frame. Also, do a netstat -tap to double check those are
the only services open on your box.

Outside of that, do you have your system set up to automatically
download / install Ubuntu updates? I know this is a new option in
Intrepid. It is possible that's when your system decided to run its
updates. I don't use it personally, but I believe the logs for it are
stored in /var/log/unattended-upgrades You can also check /var/log/
apt/term.log* and/or /var/log/aptitude to see if apt did anything
during that time.


--
ubuntu-users mailing list
ubuntu-users@lists.ubuntu.com
Modify settings or unsubscribe at:
https://lists.ubuntu.com/mailman/listinfo/ubuntu-users


--
ubuntu-users mailing list
ubuntu-users@lists.ubuntu.com
Modify settings or unsubscribe at: https://lists.ubuntu.com/mailman/listinfo/ubuntu-users

Yeah I too noted that it's about the right size to be a DVD image, and I do
use that server for file storage, but those files aren't web accessible,
only through SSH, and I was asleep at the time :-). One of the first things
I did was to double check that I wasn't still hosting an Ubuntu install DVD
on my web server (I did for a short time), and made sure that all my torrent
client wasn't running.
Good thinking though!
Walton

-----Original Message-----
From: ubuntu-users-bounces@lists.ubuntu.com
[mailto:ubuntu-users-bounces@lists.ubuntu.com] On Behalf Of H.S.
Sent: Thursday, February 12, 2009 2:07 PM
To: ubuntu-users@lists.ubuntu.com
Subject: Re: Security Issue

Preston Kutzner wrote:

>
> Outside of that, do you have your system set up to automatically
> download / install Ubuntu updates? I know this is a new option in
> Intrepid. It is possible that's when your system decided to run its

4 GB of updates? Is that realistic? Yes, it should at least be ruled out
first.

To the OP: BTW, 4 GB is fairly close to a typical DVD image. Does that
ring any bell?




--

Please reply to this list only. I read this list on its corresponding
newsgroup on gmane.org. Replies sent to my email address are just
filtered to a folder in my mailbox and get periodically deleted without
ever having been read.


--
ubuntu-users mailing list
ubuntu-users@lists.ubuntu.com
Modify settings or unsubscribe at:
https://lists.ubuntu.com/mailman/listinfo/ubuntu-users


--
ubuntu-users mailing list
ubuntu-users@lists.ubuntu.com
Modify settings or unsubscribe at: https://lists.ubuntu.com/mailman/listinfo/ubuntu-users

On Feb 12, 2009, at 3:06 PM, H.S. wrote:

> Preston Kutzner wrote:
>
>>
>> Outside of that, do you have your system set up to automatically
>> download / install Ubuntu updates? I know this is a new option in
>> Intrepid. It is possible that's when your system decided to run its
>
> 4 GB of updates? Is that realistic? Yes, it should at least be ruled
> out
> first.

I concede that 4GB of updates is a little on the far-fetched side, but
it is something to look at. Just trying to rule out all possibilities
with regards to something legitimate causing traffic. Yes, 4GB sounds
suspiciously like a DVD image.

As another suggestion, you could use 'find' to search for files added/
updated during your time period as well. I would suggest looking for
files added/updated during that *day* in case your box was compromised
a little before the suspicious net traffic. It might give you some
clues as to what happened.

>
>
> To the OP: BTW, 4 GB is fairly close to a typical DVD image. Does that
> ring any bell?
>
>
>
>
> --
>
> Please reply to this list only. I read this list on its corresponding
> newsgroup on gmane.org. Replies sent to my email address are just
> filtered to a folder in my mailbox and get periodically deleted
> without
> ever having been read.
>
>
> --
> ubuntu-users mailing list
> ubuntu-users@lists.ubuntu.com
> Modify settings or unsubscribe at: https://lists.ubuntu.com/mailman/listinfo/ubuntu-users


--
ubuntu-users mailing list
ubuntu-users@lists.ubuntu.com
Modify settings or unsubscribe at: https://lists.ubuntu.com/mailman/listinfo/ubuntu-users

Good thought with the find! I have it running now!

-----Original Message-----
From: ubuntu-users-bounces@lists.ubuntu.com
[mailto:ubuntu-users-bounces@lists.ubuntu.com] On Behalf Of Preston Kutzner
Sent: Thursday, February 12, 2009 3:24 PM
To: Ubuntu user technical support, not for general discussions
Subject: Re: Security Issue

On Feb 12, 2009, at 3:06 PM, H.S. wrote:

> Preston Kutzner wrote:
>
>>
>> Outside of that, do you have your system set up to automatically
>> download / install Ubuntu updates? I know this is a new option in
>> Intrepid. It is possible that's when your system decided to run its
>
> 4 GB of updates? Is that realistic? Yes, it should at least be ruled
> out
> first.

I concede that 4GB of updates is a little on the far-fetched side, but
it is something to look at. Just trying to rule out all possibilities
with regards to something legitimate causing traffic. Yes, 4GB sounds
suspiciously like a DVD image.

As another suggestion, you could use 'find' to search for files added/
updated during your time period as well. I would suggest looking for
files added/updated during that *day* in case your box was compromised
a little before the suspicious net traffic. It might give you some
clues as to what happened.

>
>
> To the OP: BTW, 4 GB is fairly close to a typical DVD image. Does that
> ring any bell?
>
>
>
>
> --
>
> Please reply to this list only. I read this list on its corresponding
> newsgroup on gmane.org. Replies sent to my email address are just
> filtered to a folder in my mailbox and get periodically deleted
> without
> ever having been read.
>
>
> --
> ubuntu-users mailing list
> ubuntu-users@lists.ubuntu.com
> Modify settings or unsubscribe at:
https://lists.ubuntu.com/mailman/listinfo/ubuntu-users


--
ubuntu-users mailing list
ubuntu-users@lists.ubuntu.com
Modify settings or unsubscribe at:
https://lists.ubuntu.com/mailman/listinfo/ubuntu-users


--
ubuntu-users mailing list
ubuntu-users@lists.ubuntu.com
Modify settings or unsubscribe at: https://lists.ubuntu.com/mailman/listinfo/ubuntu-users

On 02/12/2009 10:11 AM, Walton Hoops wrote:
> I could use some help from the Ubuntu wizards out there.
> I run a home server, using Ubuntu 8.10. It very low traffic, with most of
> the traffic being e-mail. Last night, over the course of an hour it
> recorded roughly 8 GB (4 up and 4 down) of traffic over the course of 2
> hours (monitoring with vnstat) and then dropped back to normal. Looking at
> the logs, the traffic did not come through apache, sendmail,or SSH. Judging
> from the fact that the up/down are equal, I'm guessing I've was used as a
> proxy for something (I don't have a proxy server installed), but I know not
> what. So, I have two questions.
> 1.) Any suggestions on how to further investigate this? At this point I'm
> at a loss.
> 2.) How would you suggest further hardening my security, since it seems it
> was compromised? I use Firestarter to lock down my ports, Fail2Ban to stop
> those pesky SSH brute force attacks, and Snort to keep an eye out for other
> attacks.
> Any input would be appreciated.
> Walton
>
>

Do you mind if I run a portscan (zenmap) against your server? Perhaps it
might reveal something open that shouldn't be. I can drop you the
results off-list, but need your permission to do so first. If OK I'll
send you an email directly with my current IP so that you know who/where
it's coming from.




--
ubuntu-users mailing list
ubuntu-users@lists.ubuntu.com
Modify settings or unsubscribe at: https://lists.ubuntu.com/mailman/listinfo/ubuntu-users