Recently I have dealt with a number of websites that were showing abnormal behavior as stated by the person that asked for my assistance. All websites used WordPress as Blog/CMS and a first look at the websites in question in fact was showing that some items were not showing. Mainly graphics were effected by the “error”, no text-elements were modified or replaced.
Investigating the issue a bit further by copying the image-url of one of the graphics that was not showing to the clipboard and opening the URL directly, I found that the link in fact worked. However, the request was redirected to an external website that was not available any more. Knowing that this behavior was not wanted and installed by the website owner and his webmaster, it was certainly the product of a malicious activity on the webserver itself. Therefore the options either were that the file that displays the image to the end-user was modified, or a mod-rewrite rule was set up on the server that produced the requests to graphics in being re-directed.
Looking at the link to the graphic, one first potential security issue became apparent. The WordPress installation used the “timthumb.php” file as part of its Hermes-Template installation. In outdated versions “timthumb.php” is known to open doors to hackers that intend to modify your server content or harm you in other ways. As reference see the corresponding entries on the Exploit-Database and OSVDB. Explaining the security hole in the “timthumb.php” would certainly go too far in this post. With the knowledge of “timthumb.php” being present, I was almost certain of the webserver having been compromised. Therefore, full access details to the server were requested in order to conduct a high-level analysis of the server content with forensic approaches.
Having obtained access to the system, the first step was to bring back the appearance of the website to normal. A mod-rewrite rule was located in one of the specific WordPress folders, which clearly indicated the redirection rule to the external website. This file was deleted to bring back the old look of the WordPress installation. I tend to do this on business websites rather than setting them to a “maintenance mode”, as anything that looks different to your customer might make them worry about things and keep them away from your website in future.
Following to the step of de-activating the mod-rewrite in the “.htaccess” file, a full backup of the website was taken, including directories out of the Apache document-root, the so called Server Root. This step is important as in that moment the website was still vulnerable. The fact that I had tackled the first modification done by the attacker, could have led into a scenario where after noticing of being found either files are removed and evidence is being destroyed, or a total attack that would have brought down the website entirely. With a full backup on my local system, a re-store of the website would have been possible in no matter of time, also all evidence was stored and available for later analysis as well as for investigation in my testing laboratory environment.
A further look at the server indicated php-files that I could not relate to WordPress itself, compared to a previously taken backup of about a year ago (the website did not change much since then) was showing that the files were new. Furthermore, the timestamp of file creation and modification were recent, not older than 14-days. Looking at the files (which were called default.php if that sounds bells among you) did not show much source code at all. The entire functionality of the script was obfuscated through using gz-compression and base64-encoding, please note that I say “obfuscated” the use of base64 in no matter can be considered as crypting as it is an insecure mechanism that anyone can revert.
The only code I was able to see was “eval(gzinflate(base64_decode($source)))”, so you might be wondering what this does. The biggest part of the script is a base64 encoded data-string, you cannot really read it but it contains the core functionality of the program. This base64-encoded data gets decoded to “normal” code by using the base64_decode($source) function, which in this case does not return the directly readable php-code, but gz-compressed data, which again cannot be read from scratch. n order to turn the gz-compressed code into something usable, the gzinflate($source) function is applied, returning readable php-code. Still this would not be too bad, if there was not the eval-function, which is the part of the malicious script, which actually executes the code that was “decoded” and “decompressed”.
In this case a 22 pass obfuscation was used, I decoded it by a php script that I ran in my local test environment to get a further impression of what the malicious script did. This step is not required, but in the event of raising legal action the source may contain additional information on the source of the attack, even though such is highly unlikely. The most important factor on back-engineering the obfuscation is, that it permits me to identify further modifications the script may have made.
Knowing that the “timthumb.php” vulnerability was used in order to compromise the server, an additional look was taken at the cache folder of the Hermes template. This inspection indicated various files which were created by the “timthumb.php” but carried the extension “php” which makes them executable php-scripts that could harm your server environment. During the backup procedure, my local anti virus system already informed me that the files found in there were malicious code, to be more precise a php-shell was installed, as well as a mass-mailing system, which in the end also caused abuse reports being sent to the hosting provider who then suspended the website until the investigation and script removal was performed.
Having cleaned everything and seeing that the website runs properly again, most people would just have stopped further actions. This would be wrong. The investigation has shown, that files were installed that were able to modify the server content. This includes reading any file that is installed on the server, or taking a full copy of at least the server html-document root. WordPress itself is a content management system, which operates from a database. WordPress stores the configuration in the wp-config.php file, this file contains the database-name used by WordPress, the database-username, the database-password and the database-host. Looking at the fact that an intruder could have read this file, it would permit the intruder to use your MySQL-DB-Admin tool, such as phpmyadmin or others. Therefore MySQL-Database has to be considered compromised. If your server configuration uses the same master-password for the database as for other configuration items and FTP access, you can be sure that your server has been entirely compromised. The intruder has FULL access to your server configuration, the FTP Server, the MySQL Server and everything else your hosting provider provides.
A check of the FTP-Log was showing non-authorized access to the FTP, of which a few dated back a while and some were more recent. Several accounts were used to access the FTP server, which indicates that the configuration of the server itself has been accessed to find that data.
To my disappointment the server was set to only keep the HTTP-Access-Log of the current day. As pointed out before the modifications on the server were made through PHP-Shells and a vulnerable file in the base-install of the WordPress blogging system. Therefore the only way to get an idea who accessed and modified the server content at a certain time would be to analyse the access to the PHP-shells and other malicious items, there are even tools to identify potentially dangerous operations from a server-access-log such as the Apache server does create.
Due to this I could not preclude that only the files I have spotted at that point were modified. It could also have happened that malicious code was appended to existing files on the server. Checking for this by opening each file and looking at the source code would be a time taking procedure.
Therefore I programmed a PHP-Tool which works comparable to an anti-virus scanner. The script scans all files, reads their code and checks them for pre-defined text-items/source-code items, which is comparable to looking for a virus-signature based on the virus definition file your anti-virus software keeps downloading from the provider. Once the script has scanned the files, it generates a simple report sorting the findings into high risk and medium risk as well as counting the occurrences A list of the files that were found is given, this contains the file location and the item that has been found. Of course on less significant functions this may produce a high number of false positives, however, the high risk classification has shown a high accuracy as this contains only the elements that I have found as combined code in malicious items.
Interface for the Server Scanner
Server Analysis Result
The two screenshots above show you the interface for the custom development server scanner as well as the result listing after the analysis was conducted. This tool is still in development but already has proven to be highly useful at its current development stage.
As result, all passwords to all accounts needed to be changed. At the beginning the MySQL-Password remained unchanged, as the change would impose a short unavailability of the website which could in worse case result in error messages that are disclosing further data on the server that I would rather prefer to stay hidden. I tend to do such password updates in the middle of the night and in a well prepared scenario where I just update a file right in the moment I have decided to change a password.
However, the solution that should be taken would be to create a new database user and database for your CMS / Webblog. I would NEVER use the master-account for anything that operates openly, and in a perfect environment you would not use the master FTP account for uploading/downloading data to your site, but create another account that only has access to the document root of the server instead of full-server-access. Such procedures should become CODEX for anyone that works in hosting environment. Working in a different way is highly negligent and will cause issues.
If you had a neighbor looking after your letterbox while you are leaving for vacation, you most likely would not give them all your keys, but only the one to the letterbox. If you act in a way of using the master account everywhere it can be compared to giving your house keys and car keys to some not well known neighbor who should just empty your letterbox. Once you return your car is gone, your house is empty and the letterbox may well be full of overdue notices and you get to know that your neighbor was using a fake identity to rent his other apartment. Would you want that to happen to you in the non-digital real life?
I therefore would recommend everyone to keep their systems up to date, create user accounts instead of using the master-account for everything you do and if you feel like something is going wrong, do not wait until your service provider pulls the plug on you. If that happens you have already been a part of criminal activities and your web resources may have been used to harm others. In front of court it certainly would not be a good point to have to declare that the harm happened through negligent behavior and stating that you were “unaware” of what could be done does never protect you from penalties – “ignorantia legis non excusat”.
As well as you would be suspicious about giving your neighbor too much access to your house, you should be suspicious about anything that happens with your webspace. If you notice tiny changes without having changed something yourself you should not be waiting for too long to ask for help if you cannot handle the issue yourself.
Websites such as Blogs and CMS can be considered pretty stable if you are not using a release version that still needs testing. This means that something that worked for quite a while before, does usually not break simply, especially not if it is part of your site and not an external plugin like for integrating social networks where only their code gets displayed on your site.