Someone’s been having fun.

Jester was setting up spam assassin on the server and checked the running processes to verify that it was working. What he saw caused him concern.

Apparently, someone had gotten into the server and had left a few goodies: IRC bots, mail relay clients, etc. They were all running under the same rights as apache, so they didn’t get root access, but it was a wakeup call to us.

We upgraded all applications to the latest, greatest versions and did some research. We found that there was a security update a few days ago to one of the applications I run that fixed this kind of security hole (so we’re pretty sure that’s how they got in). Jester removed all the offending applications (which were spread all over the system in hidden directories) and a quick reboot got rid of all the zombie processes (all 53 of them).

Crap like this makes me really cranky.

Things look good for now, but I’ll be keeping a much closer eye on it in the future.