Linux Archive

Linux Archive (http://www.linux-archive.org/)
-   Fedora User (http://www.linux-archive.org/fedora-user/)
-   -   intrusion tracking (http://www.linux-archive.org/fedora-user/481129-intrusion-tracking.html)

"Wolfgang S. Rupprecht" 01-25-2011 09:34 PM

intrusion tracking
 
Once again I find myself trying to help someone piece together how an
intruder managed to get into their system. The system was way out of
date (FC6) so it is no surprise that they got compromised. What I can
tell, the intruder managed to get root which allowed them to remove the
iptables file and lower the protection on ssh to allow unix passwords.
The attacker then installed an ssh-probing client that was installed in
/root. That lowered ssh security allowed a second intrusion at user
level (probably by password guessing) where an IRC bot was installed and
run from cron with normal user permissions.

I would have been nice to know when and how they initially got in. The
site runs a handful of daemons (postix, named, ntp, apache, dovecot), so
any of them could have allowed the initial intrution. They didn't have
selinux enabled, so that compounded problems. Clearly the top level
answer is to just impress upon them the fact that they need to stay
current and keep selinux enabled. It still would be nice to know how
the attackers got in though.

The real issue is that there isn't a good activity log. While I can
install tripwire to watch for changed files, it probably won't tell me
how they got in. Is there something that addresses that problem? Some
poor sucker always has to be the first victim of a new attack. It would
be nice to know which service to disable or reconfigure until a fix is
distributed. Is there some way to track intruders that I'm missing?

-wolfgang
--
Wolfgang S. Rupprecht http://www.wsrcc.com/wolfgang/ (IPv6-only)
--
users mailing list
users@lists.fedoraproject.org
To unsubscribe or change subscription options:
https://admin.fedoraproject.org/mailman/listinfo/users
Guidelines: http://fedoraproject.org/wiki/Mailing_list_guidelines

Steven Stern 01-25-2011 11:24 PM

intrusion tracking
 
On 01/25/2011 04:34 PM, Wolfgang S. Rupprecht wrote:
>
> Once again I find myself trying to help someone piece together how an
> intruder managed to get into their system. The system was way out of
> date (FC6) so it is no surprise that they got compromised. What I can
> tell, the intruder managed to get root which allowed them to remove the
> iptables file and lower the protection on ssh to allow unix passwords.
> The attacker then installed an ssh-probing client that was installed in
> /root. That lowered ssh security allowed a second intrusion at user
> level (probably by password guessing) where an IRC bot was installed and
> run from cron with normal user permissions.
>
> I would have been nice to know when and how they initially got in. The
> site runs a handful of daemons (postix, named, ntp, apache, dovecot), so
> any of them could have allowed the initial intrution. They didn't have
> selinux enabled, so that compounded problems. Clearly the top level
> answer is to just impress upon them the fact that they need to stay
> current and keep selinux enabled. It still would be nice to know how
> the attackers got in though.
>
> The real issue is that there isn't a good activity log. While I can
> install tripwire to watch for changed files, it probably won't tell me
> how they got in. Is there something that addresses that problem? Some
> poor sucker always has to be the first victim of a new attack. It would
> be nice to know which service to disable or reconfigure until a fix is
> distributed. Is there some way to track intruders that I'm missing?
>
> -wolfgang

I like OSSEC. It's pretty good at detecting break in attempts and file
system changes. At the very least, OSSEC would have said something as
the intruder made changes that would disable it.

--
-- Steve
--
users mailing list
users@lists.fedoraproject.org
To unsubscribe or change subscription options:
https://admin.fedoraproject.org/mailman/listinfo/users
Guidelines: http://fedoraproject.org/wiki/Mailing_list_guidelines

Joe Zeff 01-26-2011 12:40 AM

intrusion tracking
 
On 01/25/2011 02:34 PM, Wolfgang S. Rupprecht wrote:
> That lowered ssh security allowed a second intrusion at user
> level (probably by password guessing)

No need. Once they had root they could add a user and use that for their
user-level work.
--
users mailing list
users@lists.fedoraproject.org
To unsubscribe or change subscription options:
https://admin.fedoraproject.org/mailman/listinfo/users
Guidelines: http://fedoraproject.org/wiki/Mailing_list_guidelines

Marko Vojinovic 01-26-2011 12:59 AM

intrusion tracking
 
On Tuesday 25 January 2011 22:34:16 Wolfgang S. Rupprecht wrote:
> Once again I find myself trying to help someone piece together how an
> intruder managed to get into their system. The system was way out of
> date (FC6) so it is no surprise that they got compromised. What I can
> tell, the intruder managed to get root which allowed them to remove the
> iptables file and lower the protection on ssh to allow unix passwords.
> The attacker then installed an ssh-probing client that was installed in
> /root. That lowered ssh security allowed a second intrusion at user
> level (probably by password guessing) where an IRC bot was installed and
> run from cron with normal user permissions.

Shouldn't this be the other way around? I mean, ordinary user gets compromized
first, and then root gets compromized later?

If the intruder has root access, he has absolutely no need to brute-force the
user passwords through ssh. It is enough to change the password interactively
or by modifying /etc/shadow. That is, unless the intruder is just plain
stupid. ;-)

> The real issue is that there isn't a good activity log. While I can
> install tripwire to watch for changed files, it probably won't tell me
> how they got in. Is there something that addresses that problem? Some
> poor sucker always has to be the first victim of a new attack. It would
> be nice to know which service to disable or reconfigure until a fix is
> distributed. Is there some way to track intruders that I'm missing?

The only safe way to track and analyze intrusion details of a live system is
to have the machine log all activities to another machine on the net. That way
the logs are physically append-only, and even after the intrusion happens, the
intruder has no way of deleting the logs and otherwise covering up how the
machine got compromized.

Other than that, once the intruder becomes root, all bets are off, there is no
safe way to know anything about the intrusion and what exactly happened. The
only thing you can do is wipe the hard disk and reinstall the system from
scratch. Forensic research of a rooted system is (a) very painful and tough
job (even for experts) and (b) almost impossible, in most cases.

If you are into intrusion detection research, you can set up a honeypot
machine, make an exact cloned copy of the hard disk, log all activity to
another server, monitor all network traffic with a transparent machine-in-the-
middle, and then sit and wait for the machine to get hacked. Then take it off
the net, do a diff of the entire hard disk against the initial copy, analyze
logs and network traffic, etc. Those are the "laboratory conditions" in which
you can do proper forensics.

Other than that, the only thing that can give you a trustworthy clue what
happened is the remote log server, if you have one set up. If you don't, well,
the only thing you can do is to keep guessing what happened... ;-)

HTH, :-)
Marko








--
users mailing list
users@lists.fedoraproject.org
To unsubscribe or change subscription options:
https://admin.fedoraproject.org/mailman/listinfo/users
Guidelines: http://fedoraproject.org/wiki/Mailing_list_guidelines

Heinz Diehl 01-26-2011 02:00 PM

intrusion tracking
 
On 26.01.2011, Wolfgang S. Rupprecht wrote:

> The real issue is that there isn't a good activity log. While I can
> install tripwire to watch for changed files

I would have used "aide" instead of tripwire.

> it probably won't tell me how they got in.
> Is there something that addresses that problem?

No way. Once the attacker has become root, all your logs could be
deleted and/or manipulated. You can't rely on them any longer.


--
users mailing list
users@lists.fedoraproject.org
To unsubscribe or change subscription options:
https://admin.fedoraproject.org/mailman/listinfo/users
Guidelines: http://fedoraproject.org/wiki/Mailing_list_guidelines

"Wolfgang S. Rupprecht" 01-26-2011 07:51 PM

intrusion tracking
 
Joe Zeff <joe@zeff.us> writes:
> On 01/25/2011 02:34 PM, Wolfgang S. Rupprecht wrote:
>> That lowered ssh security allowed a second intrusion at user
>> level (probably by password guessing)
>
> No need. Once they had root they could add a user and use that for their
> user-level work.

I understand. I believe they were two unrelated breakins by different
people, the second breakin was enabled by the lax security caused by the
first.

-wolfgang
--
Wolfgang S. Rupprecht http://www.wsrcc.com/wolfgang/ (IPv6-only)
--
users mailing list
users@lists.fedoraproject.org
To unsubscribe or change subscription options:
https://admin.fedoraproject.org/mailman/listinfo/users
Guidelines: http://fedoraproject.org/wiki/Mailing_list_guidelines

"Wolfgang S. Rupprecht" 01-26-2011 08:06 PM

intrusion tracking
 
Marko Vojinovic <vvmarko@gmail.com> writes:
> Shouldn't this be the other way around? I mean, ordinary user gets compromized
> first, and then root gets compromized later?

Oh, I'm sure there was an initial user-level attack that I haven't found
yet and probably won't. Apache will all that dynamic stuff run from the
web server scares me. I don't think the guys running the system
appreciated how much of a security vulnarability that is.

> If the intruder has root access, he has absolutely no need to
> brute-force the user passwords through ssh. It is enough to change the
> password interactively or by modifying /etc/shadow. That is, unless
> the intruder is just plain stupid. ;-)

It was most likely a double infection with the second intruder just
taking advantage of the now lower security.

> The only safe way to track and analyze intrusion details of a live
> system is to have the machine log all activities to another machine on
> the net. That way the logs are physically append-only, and even after
> the intrusion happens, the intruder has no way of deleting the logs
> and otherwise covering up how the machine got compromized.

Agreed.

I was just wondering if there is a package that already does that or
something close.

> Other than that, once the intruder becomes root, all bets are off,
> there is no safe way to know anything about the intrusion and what
> exactly happened. The only thing you can do is wipe the hard disk and
> reinstall the system from scratch. Forensic research of a rooted
> system is (a) very painful and tough job (even for experts) and (b)
> almost impossible, in most cases.

I'll certainy take a snapshot of the disk and pick at it as inspiration
strikes. If it was a server attack (apache? dovecot?), there might be
probes in log file. They might have been lazy and not deleted the log
entries.

-wolfgang
--
Wolfgang S. Rupprecht http://www.wsrcc.com/wolfgang/ (IPv6-only)
--
users mailing list
users@lists.fedoraproject.org
To unsubscribe or change subscription options:
https://admin.fedoraproject.org/mailman/listinfo/users
Guidelines: http://fedoraproject.org/wiki/Mailing_list_guidelines

Joe Zeff 01-26-2011 11:07 PM

intrusion tracking
 
On 01/26/2011 01:06 PM, Wolfgang S. Rupprecht wrote:
> Oh, I'm sure there was an initial user-level attack that I haven't found
> yet and probably won't.

Check /etc/passwd for users you don't recognize.

grep -v nologin /etc/passwd

will give you a list of users who can log in. The few who aren't
regular users, such as halt and shutdown will probably have obvious
"shells." On my system, the only such "user" with /bin/bash is mysql.
If one of the intruders did create a new account, it should jump out at
you. And, of course, if you haven't changed the root password, do it now!
--
users mailing list
users@lists.fedoraproject.org
To unsubscribe or change subscription options:
https://admin.fedoraproject.org/mailman/listinfo/users
Guidelines: http://fedoraproject.org/wiki/Mailing_list_guidelines

Rick Stevens 01-27-2011 04:07 PM

intrusion tracking
 
On 01/26/2011 07:00 AM, Heinz Diehl wrote:
> On 26.01.2011, Wolfgang S. Rupprecht wrote:
>
>> The real issue is that there isn't a good activity log. While I can
>> install tripwire to watch for changed files
>
> I would have used "aide" instead of tripwire.
>
>> it probably won't tell me how they got in.
>> Is there something that addresses that problem?
>
> No way. Once the attacker has become root, all your logs could be
> deleted and/or manipulated. You can't rely on them any longer.

There are patches to the bash, korn and c shells that log every command
line entered to syslog. Get those and install them, then set up syslog
to log to a remote logging server.

It ain't perfect, but we've nabbed a couple of baddies that way.
----------------------------------------------------------------------
- Rick Stevens, Systems Engineer, C2 Hosting ricks@nerd.com -
- AIM/Skype: therps2 ICQ: 22643734 Yahoo: origrps2 -
- -
- I'm afraid my karma just ran over your dogma -
----------------------------------------------------------------------
--
users mailing list
users@lists.fedoraproject.org
To unsubscribe or change subscription options:
https://admin.fedoraproject.org/mailman/listinfo/users
Guidelines: http://fedoraproject.org/wiki/Mailing_list_guidelines


All times are GMT. The time now is 03:32 AM.

VBulletin, Copyright ©2000 - 2014, Jelsoft Enterprises Ltd.
Content Relevant URLs by vBSEO ©2007, Crawlability, Inc.