Superiocity (my company) just took on a new client. My client was complaining that he had to restart his Linux server a few times a day. Upon looking into the issue, I noticed a process that was running called eggdrop. Eggdrop is an IRC bot. That means that it monitors remote IRC chat rooms.
This service should not have been running on this server. The most common use of these IRC bots is to listen to a command issued by someone in the chat root. If the bot sees this command, it (along with several other bots) attack a remote server with many connections to that server so the remote server can not function properly. This is known as a Distributed Denial of Service (DDOS) attack. The infected machine is now considered to be a ‘drone’.
To secure this server is a multi-step process.
- Kill the eggdrop process
- Kill related processes
- Remove the programs from the server
- Secure the server to keep hackers from installing this code again
KILL THE EGGDROP PROCESS
This is done easily enough with kill -9 <pid>
KILL RELATED PROCESSES
This wasn’t as easy. I first had to find any and all references to ‘eggdrop’ in the system. More on that in a minute. Basically, the only other service I found running was being masked as apache. Running ps auxw | grep http will show you all your apache processes (if running that daemon as opposed to apache2). This server had three primary httpd servers running. One from Plesk (man I hate plesk), another as the userland apache server for serving the legitimate web sites, and another which was supposedly called with /usr/local/apache/bin/httpd. Um…. that binary doesn’t exist on the system! It was being masked. I killed all those masked processes.
REMOVE THE PROGRAMS FROM THE SERVER
I found two instances where the programs were located:
- /tmp/.user
- /tmp/.psy
A quick recursive remove got rid of them. rm -rf /tmp/.user /tmp/.psy Additionally, a crontab was installed for the user ‘apache’ that would restart these services if they were to go down. I removed the lines in the crontab by using crontab -e and then changed the mode and attributes of apache’s crontab so that it could not be modified and was immutable.. chmod 000 /var/spool/cron/apache chown root:root /var/spool/cron/apache chattr +i /var/spool/cron/apache
SECURE THE SERVER
This is easier than it sounds in this case. There are a number of basic processes you should do to secure any server:
- Update all software on the server to the latest versions
- Shutdown and remove any unnecessary services
- Properly setup a firewall
In this case, I can’t update any software on the server. There’s no package manager! This particular Linux distribution is old so finding packages and required libraries online isn’t easy. I contacted the service provider and asked about this lack of a package manager. Hopefully I’ll hear back from them soon.
There are currently three ways that this particular server could have been hacked:
- Outdated Plesk application with known vulnerabilities
- Outdated SSL libraries that allow remote code execution via SSH
- Existing, poorly written web applications
I found out about the SSH vulnerability by scanning the server with Nessus.
In conclusion, I’m not sure how this server got hacked. I may never know. I hopefully stopped the services from running again but this server is still vulnerable until I can update the software. It may still be vulnerable after that due to the existing proprietary web applications. In any case, I will advise the client that we should get another server that I lock-down from the get-go and then install the web applications on that server. Wish me luck!
Be First to Comment