especially HTTP GET attacks can be crippling to a web server. These attacks can come in two varieties, large amounts of requests per second, or small amounts of requests per second from any given IP, so that it becomes difficult to distinguish between a legitimate request and one that is powered by a botnet.
Assuming you have some form of rate control filtering in the front of your servers to counter the flood of requests, many 100s of requests will still leak through – which will be even more difficult to filter in the absence of a definitive User Agent signature.
There are efforts underway to implement the “Click Here” to proceed or Captcha to proceed on to the website.
Such plug-ins/modules, are very important.
Captchas or Click Through (which are Session based) are a great way to filter out bot traffic. Currently a plug-in/module does not exist for nginx but is reportedly under development.
Apache Web Server has a couple of such modules out there but how efficient they are in handling large volumes of request, remains to be seen. Lighttpd has a few Captcha plug-ins as well.
The Captcha solution works only if you have front-end rate limiting gear/bandwidth, otherwise the DDoS will drown (read: saturate) your link and server.
Session based click-through are also good, but inherently we have seen that a lot many requests leak through, presumably because the programming at hand to detect the various types of User Agents is not detailed enough to capture all the requests, and hence some slip through.
Captcha albeit a nuisance, is that one solution that works really well under such circumstances where slow HTTP GET requests are becoming more and more difficult to filter.
