If you are a sysadmin or if configuring Linux servers is part of your job description, you’ve probably been asked to set up all kinds of software packages under tight deadlines and cost constraints. After you installed mail, DNS, MySQL, Apache, and your application stack was up and running, you might have briefly paused to wonder, “is my server secure?” It’s a very important question. Let us put aside for a moment the acrid debate of whether open source software is more or less safe than closed source software and heed famous security expert Bruce Schneier who argues that security is not something you can buy, it is something your must get for yourself.
If you’ve ever received a phone call from someone wailing that the “server is down” and they are “losing money by the hour” then you probably logged into the poor machine and restarted a few services (or the entire machine) and then started sniffing around the log files to see what happened. It is during this sort of rescue operation that one really starts to wonder about the security of a given machine. It’s quite disconcerting to see endless brute force login attempts in the sshd log. It’s infuriating to see SQL injection attempts in an Apache log. Almost always, an organization just wants to get the machine running again with a minimum of expenditure. Your instructions are to just get it running again.
It is entirely understandable that organizations want to avoid a real security audit in order to save money, but — make no mistake — saving money in this way inherently involves risk, especially if your servers process financial transactions or other sensitive data. Consider the damage dealt to Sony’s reputation by the Playstation Network hack in Spring 2011. Consider also the breach of the Dutch certificate authority Diginotar that came to light in summer 2011. These large institutions have suffered staggering blows to their reputation and credibility because of their failure to guard privacy entrusted to them.
To secure your Linux servers, there are a few very important and fairly easy things you can do to keep the bad guys out:
Close every port that doesn’t need to be open and disable every service you do not need that might open a port. Limit administrative access (e.g., SSH access on port 22) to networks that belong to you. This iota of prevention is worth megatons of cure. The iptables command is an amazingly powerful tool that can lock your server up tighter than Fort Knox. Amazon’s EC2 service offers Security Groups. Make use of these if you have them at your disposal.
Disable direct login as root via SSH. It’s in the sshd configuration. Go even further, disable password login and require key pair authentication. If you must use passwords, make sure you pick good ones. It’s not as easy as you think. Also, when users must authenticate themselves, make absolutely certain that the passwords are encrypted in transit. These means using Secure FTP and not plain old FTP. It also means requiring SSL/TSL for mail connections.
Simple packages like fail2ban allow you to create ‘jails’ which monitor logs so that, using simple rulesets, you can temporarily ban any remote address which is repeatedly failing to login or which is requesting nonexistent pages, etc. If you must have ports open, fail2ban helps you protect them.
Update Your System Regularly
When security patches are released, your need to update your software to take advantage of the patches. The longer you wait, the longer you might have a hole in your system. Unfortunately, this usually requires some human interaction. If you automatically update your servers, the security patch may break something in your system. Build some time in your budget to apply regular updates — but test them first!
A file integrity checker such as samhain can provide tremendous peace of mind. It’s an automated daemon process that calculates a hash on your precious binaries and critical folders and lets you know if they ever change. If someone breaks in, samhain will let you know soon after. If you are worried about someone halting samhain before it gets a chance to inform you, you can go the extra mile and conceal samhain’s operation using generic process names and steganography.
Good Coding Practices
At the end of the day, your application looks like swiss cheese compared to the core applications that support it. Linux, Apache, MySQL, etc. are all grizzled, hardened veterans compared to the script kids you hired on Craiglist to write your email form. Choose your developers carefully. Make sure you check out the Top 25 Coding Mistakes. Your coders should know what these mistakes are and how to avoid them. If you are considering hiring a developer, quiz them on this. A good coder will know a thing or two.