I suppose Kickstarter would object to a project to put out a hit on these guys…

So, here’s the what happened.

TMI is hosted on my own Linux server in downtown Portland. Not exactly the easiest place for me to get to, but I’m getting a good deal. (SpiritOne – http://www.spiritone.com/ – Tell ’em I sent you.) The system itself is pretty robust with a quad-core CPU, 8GB of memory, and mirrored 1TB hard drives. It’s set to automatically update.

The problem with running your own server is that you have no one to turn too when things go South. Since administrating a server isn’t a daily thing for me, I was not up on current attacks. I’ve sure learned a lot in the last couple of weeks though. So I guess it’s not a total loss.

My failing was being slow about updating WordPress. I’ve been burned by updates being not quite compatible with Comicpress. My delay probably left me open to an exploit kit. But I had at least two plugins and maybe a theme or two that appear to also have been vulnerable.

The first thing an exploit kit does on intrusion is to install a web shell. A web shell gives command line access to your system. This is why your webserver ‘user’ should never have administrative privileges. Other attacks may gain access through an existing web shell, and usually they install their own web shell immediately. I lost track of how many web shell scripts had been installed on my server.

Fortunately they only had the rights as the webserver account itself, so I have no worries of being owned.

Once they’re in, they can start changing things. Commonly they may set up a script to redirect traffic or even insert their own web content (from pharmaceuticals to porn – THAT porn), so they can put your website address into spam. It’s no skin off their nose if you get hate mail and shutdown. Another nasty thing they do is set your system up to try to infect the computers of site visitors. Once your site gets flagged for malware it can take days to get off the list and other sites will drop links to you to prevent getting blocked for linking to a malware site.

I’m not alone. It appears that a huge number of WordPress sites have been compromised recently.

What I’ve learned:

Protecting your WordPress site: Update immediately. Most updates are for security; the longer you delay, the longer your pants are down. Delete any themes and plugins you aren’t using to cut down on possible exploits. They can’t break in to something that’s not there. Check access logs and error logs for suspicious activity, such as accessing PHP scripts in odd locations. Install a security plugin or two. I found one that can immediately blocks access if an IP address exceeds the number of bad URLS that a human visitor might try.

Detection: Pop-up warnings where there shouldn’t be a pop-up, download windows opening up, or server errors (500) may indicate tampering. Your access logs may show connections to PHP scripts in your upload and media folders or in the image subdirectories of themes and plugins. Logs showing “POST” connections to those PHP files is a dead give-a-way. If one of these scripts is gobbledygook, that’s probably a web shell. Check your index.php files. If there’s gobbledygook in the first few lines, that’s probably an attempt to infect visitors or pop up ads. The gobbledygook is obfuscation meant to make it harder to detect. I’m also trying out a plugin that scans all my WordPress files for changes, among other things.

Cleaning up:

1) Disable access to your site. (At one point I had web shells getting injected faster than I could delete them.) If you can’t halt the webserver program or disconnect the system from the Internet, you may be able to add a firewall or .htaccess rule to only allow yourself in.
2) Delete any plugins and themes you aren’t using. You can always try them again at a later date.
3) Delete and reinstall ALL of your themes and plugins. (Some plugins have an option to remove data upon being uninstalled. You might want to disable that first.) It’s easier than going over each and every PHP script. Don’t bother looking for file date mismatches as the exploit kits hitting me appear to be able set it to match the rest of the files.
4) Look for PHP scripts in your upload (or blogs.dir if you’re running Multi Site mode) and media folders and subfolders. I don’t know of any good reason for them to be there. Remove any that you don’t know should be there and make sure any you keep haven’t been altered. (Again, you can reinstall something later.)
5) Re-upload WordPress. The easy way is to go to the Updates page and simply hit the Reinstall button (or upgrade if you haven’t.
6) Now you should look for PHP files that don’t have matching file dates. The only one that should differ is your wp-config.php file. Anything else is likely to be obsolete or from an exploit. But make sure to check them before deleting them. (Don’t be shy. Again, you can re-upload if you make a mistake.)
7) Change your system and WordPress passwords. (You never know.) You might also want to have anyone that with access higher than Subscriber change theirs as well.
8) Cross your fingers and re enable your website. Make sure everything works. Watch for suspicious activity.
9) Have a cold (or warm) beverage of your choice.

Cheers!