The full text from Jay Allen of Six Apart is contained in the extended entry.
Hi everyone, my name is Jay Allen and I am the Product Manager for Movable Type. I’m writing today to address—what else?—comment spam.
This is an issue that, as many of you know, I have spent several thousand waking hours working on since its first appearance back in the fall of 2003, both as the author of MT-Blacklist and as the maintainer of the Comment Spam Clearinghouse. This is an issue which Six Apart takes very seriously, as evidenced not only in the improvements in Movable Type v3.x but also, in some part, by my hiring to this position.
Over the last month, we have been devoting a great deal of resources to solving the comment spam problem once and for all and making it a non-issue, not just for us in the Movable Type/TypePad world, but also for all weblogs regardless of publishing tool. Our preference is towards solutions that scale to the entire weblog medium, not those which merely move the burden from one site to another, from one tool to another, or from spammers to users.
Identifying the Problem
Recently, however, there have been a number of reports about the escalating effect of comment spam on Movable Type installations, especially evident in shared hosting environments. At first, we assumed that these problems were caused mainly on legacy systems (i.e. MT 2.x) running without the benefit of the modern anti-spam measures (e.g. TypeKey, comment moderation, MT-Blacklist v2.x, etc.) built to protect Movable Type installations. After further analysis and load testing, we’ve actually found that this is not the case.
In fact, we have found that there is a fairly major bug (in terms of effect, but not code size) which causes page rebuilding even in the case of a comment submission which would be moderated and hence should have no effect on the live page. This means that even if you are using comment moderation in Movable Type and even force moderation in MT-Blacklist, your server load is impacted just as if a comment had been posted to the live site. This bug has been fixed in development.
In addition, we have found another less severe instance of unnecessary database connections which would normally be associated with dynamic pages, even if dynamic templates are not in use. This would adversely affect any customer not using static pages by adding the overhead of dynamic files on top of the normal load caused by rebuilding of static files. This has also been fixed in development.
These two bugs are, in high probability, the causes of the extreme server loads that our customers have been experiencing under the load of a severe spam attack.
We are currently testing these fixes both in-house and with a number of web hosts who were among the first affected by the problem. We will have these fixes released to you as soon as the testing is complete. There is no higher priority to us than making sure that our customers and their websites are protected from the effects of these malicious attacks. We expect to give you a firm date for availability of this patch within 48 hours.
What To Do Now
In the meantime, one way you can help protect your system and mitigate the effects of both problems is by enabling dynamic templates. Under normal conditions, there are many factors to consider in choosing dynamic templates vs. static templates. In general, the higher your site’s traffic is, the more beneficial static templates are to you. However, since spam attacks are rapid requests that would cause rebuilding in the case of static pages, the sweet spot is moved far towards dynamic templates, even for high traffic sites.
If you would like to change your templates to dynamic, you should check out the Dynamic Publishing section in the Movable Type documentation and also Elise Bauer and Arvind Satyanarayan’s tutorial on the subject.
When setting up your dynamic pages, choose the “Build Only Archive Templates Dynamically” option. This choice means your archive templates will not be rebuilt upon comment or TrackBack submission. If you still experience high loads, you can choose the third “custom” option and set all of your templates to dynamic.
This is also a good time to mention our TypeKey authentication service which has proven very effective in stopping weblog spam. If you are interested in setting up TypeKey, check out our public TypeKey tutorial posted today on ProNet.
What’s Next
“
While we realize that these recommendations may not be your normal preference, they should keep your servers responsive despite any severe attacks until we can release the patch. We are sorry for any inconvenience this may have caused and I assure you that we are working tirelessly to remedy the situation.
On a more personal note, I would have preferred the circumstances surrounding my first post here to be somewhat (or completely) different, but there will be time once this issue is solved for me to address the past, present and future of this software as it deserves.
I want to thank you all for your perseverence through these serious problems and for helping us see clearly where the problems lie.