I find it interesting that the discussion of how to best deal with spam is ongoing and evolving. Part of the problem is that there are many different email clients and servers, which means there is no one-size-fits-all anti-spam solution. Some solutions work only with Outlook, some work on the email server, some are hardware-based solutions, some are open source solutions. Every solution I’ve seen includes trade-offs. And certainly spammers change their tactics constantly, which means that anti-spam solutions must always stay one step ahead if they are to be effective.

So which is the best anti-spam solution for you? There are two primary considerations:

1. The email client used. Do you use Outlook or Thunderbird to read your email? Or do you use a web interface?
2. The server environment where your email account is hosted. This article deals with Linux/cpanel-based servers.

Preventive Strategies

This would not be a complete article without mentioning some popular preventive strategies. The idea here is that if you can prevent the spammers from harvesting your email address, you are invisible to them. The first and most popular preventive strategy is to prevent “data scraping” of your website. If your email address appears on your website in plain text (you can highlight and “copy” it into your paste buffer) chances are sooner or later a spammer will scrape your site and add your email address to their database. The primary preventive strategy is to make your email address un-parseable. There are a few techniques for this. Two of the most popular are:

1. Putting your email address in a bitmap (an image). This way, your email address still displays and is readable to humans, but parsers typically cannot read the image.
2. Obfuscation through Javascript. This method involves burying the plain-text email into a jumble of javascript display statements. This is possibly easier to defeat for spammers though it can be adjusted quite easily. Of course, they only need to grab the address once and then they have it.

Antidotal Strategies

Okay, so the spammers have your email address. This is the most common, and in some sense, inevitable, situation most email aficionados find themselves in eventually. What to do now?

Server Side Anti-Spam Solutions

• Spam Assassin – this software is essentially the first line of defense and being open source, it is fairly well understood by spammers. Most sophisticated spammers have little trouble getting around Spam Assassin. However, it does do a decent job of blocking unsophisticated spammers, of which there are many. Spam Assassin uses an algorithm intended to mark emails as spam which include common spammer tactics or “signatures”. The spam emails can be auto-deleted or simply passed on to the email client marked as spam (typically in the subject line). Spam Assassin is nice because it’s basically a set-and-forget piece of software that does its job behind the scenes. Downsides include incorrectly marking good emails as spam and the fact that it seems in practice to be fairly easily defeated by the more sophisticated spammers.
• Box Trapper – many cpanel installations now include Box Trapper. Box Trapper is a tougher nut to crack for spammers though not surprisingly, it causes a bit more pain for the email user than Spam Assassin. The basic idea of Box Trapper is that everyone who emails you is essentially considered “guilty until proven innocent”. When Box Trapper is activated, everyone who emails you will receive a “challenge” email which requires a response from the emailer. Generally this consists of a simple “reply to” (you need only reply to the challenge email, no other modifications are necessary). This works because most spam reply-to email addresses are fake and have no monitoring. You might consider it amusing that the reason that most spammers have fake reply-to addresses is that they do not wish to be spammed or electronically assaulted in any other way. The most classic response to spam for a newbie is to send an email asking to be removed from the mailing list. This has the unintended effect of confirming to the spammer that a real, live human is monitoring the account and reading all the crappy spam. Ironically, this only makes the spammer consider your address more valuable!
• Other Solutions – hardware anti-spam systems and other software will not be covered here as they are not available at most low-cost hosting companies.

Client Side Anti-Spam Solutions

The final barrier to spam is the client software. If you use a web interface to read email on your inexpensive email hosting account, you’re primary hopes are pinned on the server-side solutions.

Open Source Solutions

This was a category that didn’t even exist a few years ago. Since then a few Windows add-ons have been written for popular clients like Outlook and Thunderbird. The two I have used before are SpamBully and SpamBayes. After using SpamBully for several months on my Windows box, I’ve determined that SpamBully is unfit for my needs. SpamBully consistently marked obvious spam as not-spam and emails from safe senders, friends and even my own website (which I’d specifically white-listed as well as marking the emails as “not spam”) as spam. Basically, it required more effort to read my emails and to mark spam emails as junk than would have been required by simply not using it.

Commercial Solutions

I have a very lengthy experience with a product called SpamFighter. SpamFighter initially seemed to work well, but after several months (actually over a year) of constant use, it has the same problems as SpamBully, namely incorrectly marking good emails as spam and (far more often) marking spam emails as good. Again, my yardstick was to determine if SpamFighter required more time to use than not, and again, this solution did not work often enough for me to want to continue to use it.

Junk Email Filter on Outlook and Thunderbird

This brings us all the way back to email client software spam filtering solutions which are oft-overlooked solutions. The Junk email filter on Outlook has various settings, as seen here:

A major revelation for me was the “level of junk protection” setting. I had always had it set “Low” previously, as I was concerned about missing real emails. This had the effect of passing a great deal of spam to my inbox, to the point where I had spent time working with SpamFighter and SpamBayes. However, merely changing the protection level to “High”, meant that almost all spam was caught, as well as a small handful of legitimate emails.

This led to a large revelation: it’s much easier to “un-junk” legitimate emails than it is to “junk” spam emails, simply because there are so many more spam emails sent than legitimate emails.

This change in my own mind-set really made my Outlook Junk filter start working for me. The few legit emails caught in the Outlook Junk email filter are easily marked “Not Junk” and from then on, delivered straight to my inbox.

Summary

If you’ve given up on client-side solutions, especially those built-in solutions so often overlooked, try reversing your mindset: crank that junk mail sensitivity setting to “High”, unjunk the legitimate emails and see how many junk emails get through to your inbox. If your experience is similar to mine, the junk emails in your inbox will drop to near-zero.

Happy emailing!

If you’ve ever received a phone call from someone wailing that the “server is down” and they are “losing money by the hour” then you probably logged into the poor machine and restarted a few services (or the entire machine) and then started sniffing around the log files to see what happened. It is during this sort of rescue operation that one really starts to wonder about the security of a given machine. It’s quite disconcerting to see endless brute force login attempts in the sshd log.  It’s infuriating to see SQL injection attempts in an Apache log. Almost always, an organization just wants to get the machine running again with a minimum of expenditure.  Your instructions are to just get it running again.

It is entirely understandable that organizations want to avoid a real security audit in order to save money, but — make no mistake — saving money in this way inherently involves risk, especially if your servers process financial transactions or other sensitive data. Consider the damage dealt to Sony’s reputation by the Playstation Network hack in Spring 2011.  Consider also the breach of the Dutch certificate authority Diginotar that came to light in summer 2011. These large institutions have suffered staggering blows to their reputation and credibility because of their failure to guard privacy entrusted to them.

To secure your Linux servers, there are a few very important and fairly easy things you can do to keep the bad guys out:

Firewalls

Close every port that doesn’t need to be open and disable every service you do not need that might open a port. Limit administrative access (e.g., SSH access on port 22) to networks that belong to you. This iota of prevention is worth megatons of cure. The iptables command is an amazingly powerful tool that can lock your server up tighter than Fort Knox.  Amazon’s EC2 service offers Security Groups.  Make use of these if you have them at your disposal.

Secure Authentication

Disable direct login as root via SSH.  It’s in the sshd configuration.  Go even further, disable password login and require key pair authentication.  If you must use passwords, make sure you pick good ones.  It’s not as easy as you think. Also, when users must authenticate themselves, make absolutely certain that the passwords are encrypted in transit. These means using Secure FTP and not plain old FTP.  It also means requiring SSL/TSL for mail connections.

Autoban

Simple packages like fail2ban allow you to create ‘jails’ which monitor logs so that, using simple rulesets, you can temporarily ban any remote address which is repeatedly failing to login or which is requesting nonexistent pages, etc. If you must have ports open, fail2ban helps you protect them.

Update Your System Regularly

When security patches are released, your need to update your software to take advantage of the patches.  The longer you wait, the longer you might have a hole in your system.  Unfortunately, this usually requires some human interaction.  If you automatically update your servers, the security patch may break something in your system.  Build some time in your budget to apply regular updates — but test them first!

Integrity Checking

A file integrity checker such as samhain can provide tremendous peace of mind.  It’s an automated daemon process that calculates a hash on your precious binaries and critical folders and lets you know if they ever change.  If someone breaks in, samhain will let you know soon after.  If you are worried about someone halting samhain before it gets a chance to inform you, you can go the extra mile and conceal samhain’s operation using generic process names and steganography.

Good Coding Practices

At the end of the day, your application looks like swiss cheese compared to the core applications that support it.  Linux, Apache, MySQL, etc. are all grizzled, hardened veterans compared to the script kids you hired on Craiglist to write your email form.  Choose your developers carefully.  Make sure you check out the Top 25 Coding Mistakes.  Your coders should know what these mistakes are and how to avoid them. If you are considering hiring a developer, quiz them on this.  A good coder will know a thing or two.

Think of it as a snapshot, as if you’re looking at a photo of the night sky — the sky will look different tomorrow night…and 3 months from now. The internet changes quickly. Cloud vendors will continue to come and go, so longevity, stability and portability will become increasingly important factors when choosing vendors.

Where do SMB owners fit in?
As a small business owner, you and your staff will most likely only have direct experience with SaaS – the applications that deliver software like Microsoft Office and Google Docs.

Here’s an example of how the three layers work together.

SaaS (Software as a Service) – You and your employees use it as off-the-shelf application software. You may decide to use SalesForce to manage your customer database. The SalesForce software is hosted on the cloud, rather than individual hard drives on your office computers, or on the network server in your office. In order to use SalesForce, you download apps that connect with the cloud version, so you’re using SaaS.

PaaS (Platform as a Service) – You hire a software developer who uses this to create custom software. (As a small business owner, you won’t have to know the technical details about PaaS or IaaS, but we’ll give you an overview just for fun.)

Now, let’s say you decide that the off-the-shelf version of SalesForce still requires too many hours of manual data manipulation.

For example, you’re paying overtime in the accounting and sales departments because they need to use different spreadsheets every month to sort and extract data that’s specific to your company.

SalesForce also sells access to bare-bones software platform versions (Force.com and others) — PaaS — on which you can develop, build and host your own version of SalesForce, customized to your individual business needs.

That’s the point where you hire a custom software developer. The custom software development team codes the custom software on the PaaS — uses it as a testing and developing platform — and saves the time it would have taken to set up the development environment on a separate server.

There are many other PaaS providers besides Force.com. And there’s fierce competition among providers to give developers more choices of languages, frameworks and platforms as PaaS evolves, especially in the Open Source community, according to Wazi, the most current news source for Open Source development.

Honestly, the further you go into learning about PaaS and IaaS, the geekier it gets. We touch lightly upon IaaS in another blog post. But trust me — your time and energy will be better spent running your business than trying to figure out the differences between PaaS and IaaS. Their functions often overlap and  they’re moving targets — evolving even as you read this.  The developers who use them don’t even agree on their definitions.

You’re better off finding a software development team that writes brilliant code, will handle the PaaS and IaaS for you, and can communicate with you so you get the software you want — really!

Small and mid-sized businesses in the market for custom software

One of the reasons is that corporate IT departments are slower to make decisions because of their hierarchy. IT managers often need to spend weeks or months developing proposals and attending meetings in order to get buy-in and budget allocated to move forward.

But SMB owners who work directly with a knowledgeable and experienced custom software developers can say ‘Yes’ and get the project launched as soon as the planning is complete. They can make decisions faster — and save money in the process.

Developers love IaaS (Infrastructure as a Service) because it provides everything they need for their basic development environment in a push-button scalable form: processing, storage and networking can all be sized to fit the project almost instantaneously. They don’t have to wait around for a human to show up to configure and network a new server if they need more storage space. There’s no down time waiting for more physical memory to be installed to increase computing power.

SMB owners love IaaS because it It keeps development costs down. There’s no large up-front hardware investment needed to buy, configure and network larger servers. Yet the pay-per-use cost is scalable. When computing power and databases need to expand as the business grows, it’s easily done. Again, there’s no large up-front hardware investment necessary — just an incrementally higher monthly bill for the additional resources from the IaaS provider.

Key takeaway: If you want to save time and money on your small business custom software project in the long run, look for a software developer who has experience with Public IaaS cloud — like Amazon or Rackspace.

If you’ve ever owned a dilapidated old car or if you’ve been forced to travel a lot for work, then you probably know how the venerable American Automobile Association can be a real life saver.  I once owned a 1987 Honda Prelude with a list of mechanical problems longer than the Midgard Serpent. When that faulty oxygen valved caused me to stall on the 101 Freeway in downtown Los Angeles, I called AAA and they rescued me.  When work had me driving 100,000 miles in just 3 years, AAA provided me with enormously helpful and up-to-date highway maps and traffic data that kept me out of the nightmare that is the Washington, DC beltway at quitting time on a Friday.

If you have a broken down old website or if you are planning a journey into the Amazon cloud, Boomcycle can help.  We would like to offer you our own Triple A Service: Amazon Acronym Assistance.  Everyone is talking about the cloud and CTOs everywhere are convinced they need cloud technologies.  But what does it mean exactly?  Who can explain what EC2, RDS and SES actually do and how to take advantage of them for your business?  Who can help you plan your cloud migration effectively so that you don’t end up in a website migration quagmire that costs you a fortune? Boomcycle can.

Boomcycle has hands-on software development experience with cloud technologies and has experienced first-hand some of the remarkable benefits:  scalability, flexibility, and cost savings.

For starters, setting up a development machine to get your new site started is a lot easier with Amazon EC2.  The old way involved calling your go-to hosting company and muddling through all the details they need to give you either a) a bloated machine with Webhost Manager designed to host a thousand websites or b) a bare-bones server with nothing but CentOS off the CDROM installed on it.  With Amazon Elastic Compute Cloud (EC2), you can set up your favorite distro once with all of your favorite development tools and store a snapshot of it as an Amazon Machine Image (AMI).  If you need a new server quickly, you can instantiate a virtual machine from one of these images almost immediately.  It gets you up and running faster.  Additionally, when you need to scale up your service, the ability to instantiate new servers is invaluable and can be automated. When you need to scale down for cost savings, you can delete them just as quickly and just as automatically.

Boomcycle is also familiar with the pitfalls and hazards of Amazon Web Services (AWS).  The AWS ecosystem is a patchwork of APIs, code libaries, documentation, and outright quirks that can present challenges.  For example, Amazon’s DNS service — Route 53 — currently has no interface in the AWS web console and one must make use of the command line tools to create and delete DNS records.  Additionally, there is no edit function.  One must first delete an existing record and then create a new one to replace it.  Even so, it’s a lot easier and more secure than maintaining BIND on one’s own servers. Even integration of two different Amazon services — EC2 and Amazon’s Simple Email Service (SES) — can be a challenge in certain contexts.  This EC2/SES integration is in many cases a necessary chore as the IPs one can assign to an Amazon EC2 compute instances are all on a policy block list at spamhaus so one must find a different way to send outgoing email.  Boomcycle has done this before.  We know that the perl scripts provided by Amazon have numerous dependencies and we know how to get those installed.

If you think you need to use the Amazon cloud for your IT project, call us.  We can help you understand not only the alphabet soup of acronyms, but also the IT implications and cost considerations. We’d love to help.

Following a concerted effort by Boomcycle open source senior engineer (and genius!) Jason, the system is functioning much more smoothly.  The database duties are still handled by their primary dedicated machine, and our monitor scripts still complain every now and then about excessive load when there are a lot of visitors, but the 30+ Gigabytes of images are now happily hosted by the Rackspace Content Delivery Network.  These images bypass the primary server entirely and are fetched directly from their remote sources by a Rackspace Cloud Server and fed directly into the content delivery network.  The multithreaded script that handles the images has fetched as many as 17,000 images, generated 6 different thumbnails for each image, and uploaded the original image and all its thumbnails into the Rackspace CDN in a single hour.

Boomcycle’s remedy for this client has not just stabilized their system and improved performance for their clients and customers, but we have significantly reduced their monthly cost.  Thanks to our new system, the client was able to shut down a second image server that is no longer needed.  Our client estimates the Rackspace Cloud Server and CDN costs at around $50 — considerably less than the hundreds of dollars per month they were paying for a dedicated machine to host all these images.  The Rackspace system also makes redundant copies of the images and manages backups.  It’s an all around win.

Our client is happy and we’re talking to them about some further initiatives to help them keep up with their customer demand. Boomcycle is happy to help our clients grow!

Page Speed Before

Here is a screen shot of the Google Page Speed plugin before tweaking WordPress. It was clocking in at 47/100, which is noticeably slow to visitors:

There are several things going on here, some of which are outside the scope of this post. There are also several ways to analyze the web site to determine where exactly the bottlenecks are, some more complicated than others. We’re going to keep it simple.

For starters, the Firebug plugin for Firefox is your friend when troubleshooting speed issues on web pages. Use the “Net” time line to quickly identify the biggest delays when loading your web page. (If you’re feeling extra frisky, check out xdebug for PHP.) In boomcycle.com’s case, there were 3 images that stuck out immediately, as they were each taking over 4 seconds to load. The browser was downloading them concurrently, so it wasn’t as big of a delay as it could have been.

On to the tweaks…

MySQL Query Cache

OK, this is an easy one. Just add the following lines to your MySQL configuration file (usually my.cnf), and then restart MySQL. Unfortunately this little tweak is probably not available unless you have root access to a dedicated or VPS server. If that is the case, it doesn’t hurt to open a support ticket and ask if they would be willing to add it globally.

query-cache-type = 1
query-cache-size = 20M

These settings tell MySQL to use up to 20 Megabytes of cache to store common queries and their result sets. If there is more memory available on your server, go ahead and crank up the setting higher, but don’t go too crazy. More info on MySQL query caching can be found here.

timthumb.php

This particular theme, Minimal, makes use of an image resizing (among other functions) script called timthumb. While this is a handy script, the version installed was fairly old and was taking over 4 seconds to read, resize, and then serve a thumbnail from a 300K image. The newest version shaved almost a full 3 seconds off the image load time due to better processing and caching.

The easiest way to tell if your theme is using TimThumb is to use the Media section of FireFox’s built-in Page Info utility. Just look for a URL with timthumb.php in it like this one:

http://boomcycle.com/wp-content/themes/Minimal/timthumb.php?src=http://boomcycle.com/wp-content/uploads/2011/01/web_development_sample_3.png&h=226&w=406&zc=1&q=90

That URL also helps you identify where the file is stored on your server. In this case, it is in wp-content/themes/Minimal, which is relative to the public html directory of the web user. Use that information to save a copy of the original and then upload the new version in its place.

gzip compression

We are constantly amazed at how many web sites out there don’t take advantage of this very useful setting. gzip compression allows the server to compress almost everything except images before sending it to the visitor, saving both time and bandwidth. Web browsers have been supporting gzip compression for years, so there’s almost never a reason not to use it.

For boomcycle.com, enabling gzip compression saved over 180k per initial page load. (After the initial view the browser has usually cached the content, unless another web server configuration setting is not enabled – more on Expires Headers in a minute.) Sure, 180KB of bandwidth savings doesn’t sound like much, but multiply that by 10,000 visitors and you’ve just saved over 1.8GB of bandwidth! That also means your web server can spend more time handling other requests instead of being tied up delivering large, single-threaded resources like Javascript (more on that subject in a minute as well.)

First, check to see if gzip compression is enabled on the web server in question. There are several ways to do this, but by far the easiest is with this web site:

http://www.whatsmyip.org/http_compression/

That compression test will also report how much there is to gain by enabling gzip compression. One thing to keep in mind is that all modern image formats are already highly compressed, so enabling gzip compression for images will actually have the opposite result.

Various speed checking plugins for Firefox will also tell you if gzip compression is enabled, but we have experienced false information due to aggressive browser caching.  There is nothing more frustrating than chasing your tail trying to troubleshoot a issue that doesn’t really exist. Instead, use a third party point of view that hasn’t cached the web site in question.

In order to enable gzip compression on your web server, it has to be compiled in and available. If your web site is on a shared hosting provider, then you’ll have to open a ticket to request enabling gzip compression globally. It’s better for you; it’s better for them; it’s better for all of their customers.

The following settings are typical gzip compression settings for Apache 2.x web servers. The settings are surrounded by an IF statement to ensure it doesn’t break your web site if for some reason the compression module is unavailable. This configuration can be placed in the main .htaccess, or in a global included configuration file.

<IfModule mod_deflate.c>
    SetOutputFilter DEFLATE
    <IfModule mod_setenvif.c>
        # Netscape 4.x has some problems
        BrowserMatch ^Mozilla/4 gzip-only-text/html
        # Netscape 4.06-4.08 have some more problems
        BrowserMatch ^Mozilla/4\.0[678] no-gzip
        # MSIE masquerades as Netscape, but it is fine
        BrowserMatch \bMSIE !no-gzip !gzip-only-text/html
        # Don't compress already-compressed files
        SetEnvIfNoCase Request_URI .(?:gif|jpe?g|png)$ no-gzip dont-vary
        SetEnvIfNoCase Request_URI .(?:exe|t?gz|zip|bz2|sit|rar)$ no-gzip dont-vary
        SetEnvIfNoCase Request_URI .(?:avi|mov|mp3|mp4|rm|flv|swf|mp?g)$ no-gzip dont-vary
        SetEnvIfNoCase Request_URI .pdf$ no-gzip dont-vary
        </IfModule>
    <IfModule mod_headers.c>
        #Make sure proxies don't deliver the wrong content
        Header append Vary User-Agent env=!dont-vary
    </IfModule>
</IfModule>

If the settings are put into .htaccess, the changes will take place immediately upon next page load. If they are in a main Apache include file, the service must be reloaded or restarted for the changes to take effect.

Expires Headers

Most web sites have static content that rarely or never changes. Images, style sheets, javascript, etc. This content should be delivered with appropriate expires headers so that the visitors to your site will have the content cached in their browsers instead of having to request it for every single page load. The result is less bandwidth and resources used on your server, and a much faster user experience for your visitors.

The configuration settings are similar to the gzip compression settings. They can go into the .htaccess file, or a globally included configuration file:

<IfModule mod_expires.c>
    # Remove ETags if mod_expires is controlling caching
    Header unset ETag
    FileETag None

    ExpiresActive on
    ExpiresByType image/jpg "access plus 60 days"
    ExpiresByType image/png "access plus 60 days"
    ExpiresByType image/gif "access plus 60 days"
    ExpiresByType image/jpeg "access plus 60 days"

    ExpiresByType text/css "access plus 1 days"

    ExpiresByType image/x-icon "access plus 1 month"

    ExpiresByType application/pdf "access plus 1 month"
    ExpiresByType audio/x-wav "access plus 1 month"
    ExpiresByType audio/mpeg "access plus 1 month"
    ExpiresByType video/mpeg "access plus 1 month"
    ExpiresByType video/mp4 "access plus 1 month"
    ExpiresByType video/quicktime "access plus 1 month"
    ExpiresByType video/x-ms-wmv "access plus 1 month"
    ExpiresByType application/x-shockwave-flash "access 1 month"

    ExpiresByType text/javascript "access plus 1 week"
    ExpiresByType application/x-javascript "access plus 1 week"
    ExpiresByType application/javascript "access plus 1 week"
</IfModule>

If you’re curious what the ETags setting is for, feel free to Google it. In our experience ETags is not a big deal one way or another, but it always shows up in the speed test results if it is not disabled.

The other settings are perhaps self-explanatory. Adjust them according to your web site usage and update habits. If you frequently alter flash content, then just change the application/x-shockwave-flash expires setting to “access plus 1 day” instead of 1 month. If you have a WordPress blog where existing content never changes, then it’s probably safe to crank most of the settings to several months instead.

WordPress Plugins – DB Cache Reloaded and Hyper Cache

There are plenty of WordPress caching plugins to choose from out there, so how do you decide which one is right for your web site? The answer is by a lot of trial and error, or a lot of reading comments regarding other people’s experience.

We’ve tried many different caching plugins, and by far the most impressive is a combination of DB Cache Reloaded and Hyper Cache. They work wonders with out-of-the-box configuration, and should not interfere with anything WordPress is trying to do. Having said that, they might not be compatible with other caching plugins, so it’s best to delete the old cache stores and then disable the other plugins before activating DB Cache Reloaded and Hyper Cache.

Page Speed After Tweaks

Here is a screen shot of Page Speed for boomcycle.com after all the above tweaks were implemented. The increase in speed was significant, both on paper (66/100)  and most importantly, in user experience (<2 second load times):

NOTE: When testing with Page Speed, be sure to hold down the SHIFT or CTRL key while pressing F5 in your browser. This will force the browser to request all of the content from the web server, regardless of what is in the browser’s cache. Do this a couple of times to ensure the caching has been activated in the WordPress plugins, and then run the Page Speed test. The result should be obvious in Firebug. For example – once TimThumb has cached the thumbnails, the image load times dropped from over 4 seconds to 500 – 900 milliseconds.

Interestingly, the Firefox version of the Page Speed plugin reports an even better score than the Chrome version – 79/100. That’s on par with some of the biggest and most popular web sites on the internet today.

The only way to get a much better rating is to combine JavaScript and convert everything to static content.

Other WordPress Speed Tweaks

You might have noticed that Page Speed mentions Javascript in several places in all the reports. There is a good reason for this. When the web browser requests Javascript content, it does so one at a time, essentially blocking all other requests for content until the Javascript is downloaded completely. (The reason for this seemingly odd behavior is beyond the scope of this post. Google it if you’re curious.) The impact in this case is  a 66/100 Page Speed instead of a much higher result, since the browser had to request 6 different Javascript files.

There is only one way to resolve this problem, and it can cause serious issues and upgrade headaches in the future. We do not recommend implementing the change unless you really know what you are doing. The resolution is to combine all javascript sources into one large file, and then update the source html/php/template so that it only references one file. If your WordPress theme makes proper use of headers and footers, this might actually be a fairly easy change to make. Upgrading the theme or making other WordPress changes could easily break what you’ve changed, however.

The other recommended change relating to Javascript is to move all the Javascript to the end of the page instead of at the top. The result is that all the visual content is loaded before the Javascript loads, making it appear that the page loaded much faster than it really did. This is also not a change to take lightly, as it could very easily break your web site.

By far the best method of speeding up a WordPress web site is by using a plugin that pre-caches all of the content, and then serves it as static html files instead of dynamic content. The result is blazing Page Speed times in the low to high 90′s. However, this particular method of caching will not work for a majority of WordPress blogs due to the liberal use of dynamic content – ads, tag words, comments, etc. Usually if the WordPress site is busy enough to warrant this type of pre-caching, the visitors will be expecting constant updates, defeating the whole purpose of pre-caching to begin with (and if the site is that large and busy, then it’s probably time to invest in some dedicated servers with all of that ad revenue coming in!)

We hope you’ve found this information helpful for speeding up your WordPress website. If you still need faster website speed, Boomcycle would love to help!

Amazon RDS Scaling

Scaling refers to the ability to increase the capacity of the database server to meet customer demand. As customer demand increases, database servers may be unable to keep up with this demand. They will become slower and slower as their capacity to serve requests is exceeded and eventually will begin dropping requests and causing errors. Amazon RDS provides cloud-based tools for scaling the database to head off this trouble in the form of both horizontal and vertical scaling.

Horizontal Scaling

Horizontal scaling refers to increasing the capacity of the database by adding more machines. Amazon RDS facilitates this by providing a simple way to implement database replication. With database replication, an installation of Amazon RDS serves as the “master” and one or more instances of Amazon RDS are set up as separate and distinct “read replicas.” All updates to the database take place on the single master instances, and those updates propagate out to the read replicas. Because the typical choke point with database-driven web applications is in many simultaneous attempts to read the database, this provides the value benefit of sharing the load of retrieving the data among multiple servers. Amazon RDS supports establishing as many read replicas as necessary.

One drawback with database replication is that it is asynchronous. Because updates to the database must propagate out from the master to the read replicas, and because this is not instantaneous, the data on the read replicas may differ from the data on the master. Special consideration is required in the design of a software application where this type of scaling is utilized to prevent data inconsistency errors.

Another type of horizontal scaling of database is called “sharding”. Sharding is the act of splitting up pieces of the database to separate servers so that a single server doesn’t have to bear the entire load. A database scaled in this manner does not suffer the disadvantages of being asynchronous, but sharding requires very special design considerations, both in terms of database and software design. For extra scaling power, this may be employed in tandem with database replication, where each piece of the database is being read by its own set of read replicas.

Vertical Scaling

Vertical scaling refers to increasing the database capacity without adding more machines. This usually means faster processors, more RAM, bigger disks, faster network connections, and so on. Because the instance of Amazon RDS is running on the cloud, increasing the amount of RAM, number of processor cores, or disk space is a simple matter and may be done “on-the-fly.”

Fault Tolerance

Amazon RDS has an option called a Multiple Availability Zone, or Multi-AZ deployment. This means that instead of a single server instance, an additional server instance with identical configuration is established at a geographically distinct location. If the primary server is ever unreachable (network outages or hardware failures), this second server immediately takes over in a completely transparent manner. The second server’s data is “synchronous” with the primary server, meaning the databases are always identical. This provides an effective backup of the database and goes a very long way towards ensuring the database is always available.

Recent Amazon Outage

In April of 2011, Amazon suffered a much-publicized outage of aspects of its cloud computing platform, illustrating quite dramatically that even their system is not perfect. At a high level, the outage occurred in multiple availability zones in their “US-EAST-1” region. This caused some high profile web sites to be unavailable and some actual permanent data loss. Although outages of this nature are rare, they are a risk that must be accounted for. Risks of this type may be mitigated by ensuring availability zones are in distinct broad geographic regions and having a backup plan for the database that is independent of Amazon RDS.

Combining Scaling with Fault Tolerance

Scaling and fault tolerance may be combined to maximize the availability of databases to clients. In an environment with a single master and multiple read replicas, the master may be configured as a Multi-AZ deployment. In the event the master is unavailable, its invisible backup will immediately take over its duties and changes will automatically propagate from it to the replicas. It is also possible to configure the read replicas as Multi-AZ deployments, though the advantage of doing so is not as obvious. Multiple read replicas serve as available backups of each other, though they may not be located in separate geographic locations (which increases risk).

Finding the right solution

Boomcycle employs benchmarking and load testing of the Amazon RDS instances using real data scenarios from the application to determine capacity. These data are balanced with the requirements and budget of the client to arrive at an ideal configuration to meet clients’ needs.

Most business people know they need a mobile presence but many don’t know where to start or even what an “app” truly is. Technology is moving quickly and as usual the business owner whose core business is not technology-centric is left behind in the ongoing and ever-changing technology conversation. Recently that conversation has centered around mobile devices. Mobile is an exploding field, with new devices seemingly entering the market every day and an ever expanding user base demanding more convenience. If the business does not enjoy a mobile presence, the marketplace may leave them behind.

Here at Boomcycle, we hear similar questions again and again:

• What is an “app”?
• What are the various mobile platforms I should care about?
• What types of apps can be built?
• What does an app cost to build?
• How does an app get into in the iTunes App Store?
• Can I build one app that works on all devices?

So let us address the “mobile basics” so that you have a better idea of what you need and how to most effectively implement your businesses mobile presence.

What Is An App?

Some would say that Apple did the world a service by using television ads to make people aware of programs that ran on mobile devices, specially the iPod, iPad and most popularly, the iPhone. The word “app” is short for “application”, meaning a program that performs some function, written in a computer programming language. Prior to the introduction of the word “app” into the common vernacular, we often noticed an immediate glazing of the eyes whenever we used the word “application” in front of many clients. So thank you, Apple: the word “app” is now commonly understood to mean “little programs that run on iPhones”. Such a definition, while far too vague and far too broad, at least allows “techies” and “civilians” to start a conversation!

And while Apple does not have a monopoly on mobile devices, they do currently seem to enjoy a monopoly on mobile mind-share. When most people think of apps, they think “i-something” first (iPhone, iPod, iPad) and every other platform (if they are even aware there are other platforms) second. This is changing rapidly, though as Apple popularized the app first, Apple iOS apps are usually considered first by our clients looking to go mobile.

What Are The Most Popular Mobile Devices?

You’ve no doubt heard of at least some other mobile platforms. The biggest mobile platforms at the time of this writing are:

• iPhone, iPod
• iPad
• Android
• Blackberry

If you are thinking about “going mobile” with your website or building a totally custom app, these are the main devices on which you should focus. Naturally as time marches on, newer devices are bound to displace or obviate the current list.

Why did we separate the iPhone/iPod from the iPad? The size of the display. The larger sized iPad enables comfortable use of different kinds of apps. Reading apps like Kindle or iBooks are the most popular for these larger devices. While some people considered the iPad a ho-hum usability improvement, they fail to see the tectonic shift happening in sales, medicine and electronic publishing, to name but a few.

What Types of Apps Can Be Built?

Here’s where the definitions get a tad foggy, but bear with us, we’ll do our best to make it clear!

The Mobile Friendly Website

Your website has been around for years, and you’ve probably tried to pull it up on your mobile device’s web browser. Remember what an annoying experience that was? The text is too small, the menus are a mess and the website is generally unusable. So you never look at it that way again. What you need is to “re-purpose” your existing “traditional” website as a Mobile Friendly or Mobile Compatible website.

Building a mobile friendly website is the easiest way to “go mobile” and retain the eyeballs you’ve worked so hard to get on your traditional website. You will need to do a few things to create a mobile website:

• Ensure that your current website is powered by a Content Management System (CMS) like WordPress
• Evaluate your current site and make a list of the most important factors for mobile display
• Create screen mock-ups and get feedback from stakeholders, clients and customers
• Build the mobile version of your website based on the mock-ups

Your CMS will provide data to your traditional website and your mobile website, insuring consistent content on both websites.

Native App

These are what most people are referring to when they say “app”. This is an app, typically downloaded from the mobile platform’s store (EX: an app from the Apple’s iTunes App Store) and running “natively” — on the phone’s hardware. No connection to a wireless network is necessarily required to run a native app. Native apps can be ebook readers, games, calculators or any number of useful programs that don’t necessarily require a connection to the internet. The look-and-feel of an iPhone app is fairly well established thanks to Apple’s user interface conventions. Other platforms are more akin to the wild west, design-wise.

The big problem with “native apps” are that these apps are native to the devices on which they run. So if you want to develop native apps for mobile devices, you must select which mobile platforms on which you wish to make your app available. Most companies that want to develop a native app start with the iPhone/iPod/iPad and Android, though many also choose to develop on Blackberry simultaneously. The former are definitely the Big Three when it comes to mobile development currently.

Web App

A “web app” is a special type of app that looks like a Native App except that a web app runs on your mobile device’s web browser (for example, on the iPhone, a web app runs in Safari, the built-in web browser). A web app feels like a native app to the user, except that it relies on a data connection to work. For example, a web app that uses GPS needs a data connection to determine the location of the mobile device. A simple example of this type of web app is the Google Maps application that comes standard on Apple mobile devices.

What Does an App Cost to Build?

If this is not the first question we hear, it is a close second. We wish there was a simple answer to this question! Programming for mobile devices is still programming: it can be complex and it requires education and experience in order to do well. The complexity of any program is the key cost factor. The greater the complexity, the greater the number of hours to design and develop the app.

The other driver in the mobile equation is supply vs. demand. There are a limited number of good mobile developers and an exploding demand for the development of new mobile apps. Mobile development is currently more expensive than traditional software development.

One of the most important considerations in mobile development is the location of the data that the app uses: where does the data for the app come from?

Complexity may be found in surprising places. For example, to “mobil-ify” a standard website, one of the first things to consider is whether or not the website uses a CMS as its data source. If the site is “plain vanilla” HTML/CSS or Flash-based, the information from the website may not be usable by the mobile friendly website. In this case, the first step of the mobile friendly website initiative may be converting the current website so it uses a CMS (Project #1). The second step is then building a mobile friendly website that uses the data from the CMS.

If your app’s data can be consumed freely from the web (e.g., Google, Craigslist, eBay, etc.) your app can pull the data and with a bit of user customization such as a saved search (e.g., “I want to look at houses for sale in Santa Monica for $750K”) — bang! You have a useful app because freely available and current data is present.

However if your data is custom and requires effort to maintain — such as your own company’s data (products in stock, prices for software, the number of locations to serve your customers) then this data must be maintained by back-office systems. These back-office systems may exist already, in which case the effort may be in connecting those systems to the outside world so that a mobile device may use the data. And if these back-office systems must be built to support a mobile app, developing said systems will add to the cost of developing the “app”. In this latter scenario, the app itself may be thought of as simply the display of the data that’s maintained through more traditional systems.

Beware of mobile app developers who give you quotes for apps that have yet to be designed and formally specified through consultation with a competent programming consultancy.

How Does an App Get In the App Store?

A detailed explanation of why an app may or may not get into Apple’s iTunes Store is beyond the scope of this article. Apple may not approve an application for obvious reasons (buggy or poorly-performing) or reasons that may be far less “obvious”. The iPhone Developer Program License Agreement is a lengthy document which attempts to codify these issues. Suffice it to say, getting your app in iTunes is not a slam-dunk and may involve some time and effort on the part of your business and your consultant.

The challenging Apple approval queue is one reason why a mobile website (which doesn’t require approval) may be a good first step for businesses looking to dip their toes into mobile development.

Can I Build One App That Works On All Devices?

Strictly speaking, an app that is “native” to Device “A” will not run on Device “B”. For example, a native iPhone app will not run on an Android device, or any other manufacturer’s device. This means if you want to develop native apps, you will need to develop an app for each device you wish to target. Again, the most popular targets these days are iPhone/iPod and iPad, Android and Blackberry, in that order.

Web apps and mobile websites entail less deployment risk and can generally run on multiple mobile browsers. Your development team must be certain to check the app against the most popular mobile browsers, just like with traditional web applications on standard desktop computer web browsers.

Go Mobile!

Mobile is exploding and those businesses who go mobile first will be the first to capture the eyeballs and business of the users of these devices. If your website is not mobile-friendly, this is an obvious place to start your mobile initiative.

Ideas for native and web apps are everywhere and one need only survey the iTunes App Store or Android Market to get a plethora of ideas.

So let your imagination run wild: you may create that killer mobile app that your competitor doesn’t have! Do so, and you’ll be tapping into a whole new generation of clients and consumers of your products and services.