Thursday, July 21, 2011

Distributed Denial of Service Attacks - A Primer

This primer on DDoS attacks is for network, security and business managers. Existing information on DDoS attacks is outdated and has not kept up with current state of the art in attack tools and attack mitigation techniques. The scope of this tutorial is limited to internet data centers, webhosts and is not relevant to very large ISPs with several routers and several Internet links.


Defining DDoS Attacks

Denial of service (DoS) attacks are attacks that are deliberate attacks on your network properties to deny service to legitimate users. When these attacks seemingly come from distributed sources, they become distributed denial of service (DDoS) attacks.
 
Few years back, it was common to use spoofing techniques where a hacker would actually use very few machines (or just one machine) and spoof mutliple IP addresses. To the attacked destination it would seem that the attack is coming from multiple IP addresses. However in the recent times, with the advent of infected PCs, increasing number of smart mobile phones, many botnets are available around the world, which can be used to launch a real DDoS attack.
 

Scenarios Under Attack

 
  • Under attack, your team does not know details of the attack. They understand the symptoms, but they can’t figure out the cause and the solution.
  • Your routers and switches are overloaded and they don’t have the capability to stop such attacks. Firewalls simply allow these packets. IPS appliances (if you have them), don’t have the rules to block such attacks. Your equipment doesn’t match up in performance that’s required.
  • May be, you have multiple links to the Internet. The attackers are attacking from different links.
  • The attack is seemingly coming from all over the world. You cannot simply identify a Net-block to deny so that the attack can be stopped !! And you cannot simply block everyone !!
  • The attack is no different from legitimate users accessing your web pages from the point of your edge equipment.
  • Your team is unable to figure out the solutions quickly when the attackers are constantly changing the tactics.
  • You have too much collateral damage. When attack happens on one part of the network, the others bleed too.
  • Software solutions such as mod_evasive, iptables, Apache / LiteSpeed tuning, kernel tuning, not capable of handling the load.
  • You are not as rich as others to over-provision your bandwidth and to buy high-bandwidth gear.
  • The only tool your service provider has is Null Routing your IP address!!
 

Causes of Attacks

  • Sometimes script kiddies’ youthful exuberance gets transformed to rivalry and they try to prove their might and you just get involved in their cross fire. Sometimes it may simply be your own rivals who are getting at you.
  • In some cases, some script kiddies may have recently learnt the tricks of the trade. They may be doing this for sheer personal pleasure. They are randomly choosing targets and you just happened to be on their radar by mistake. They may incrementally advance to more sophisticated attacks as they learn the tricks.
  •  For some attackers it is prestigious to attack you and bring you down despite all your attack defenses. Some times, a software bug on your server may be causing too many users to come to your site again and again. Your servers can’t handle the load and they keep trying. It’s like an avalanche.
  •  Sometimes, someone wants you to pay or else they will break your site. May be there is a reason you are not reporting this to authorities – or may be authorities/ISPs don’t help and don’t know how to help.

Motivation Behind Attacks

There can be many reasons why DDoS attacks are launched.
 
  • Some attacks are principle driven attacks. The attackers desire to silence you since your values are different from theirs. E.g. you own porn sites and they are web-vigilante. Sometimes your rival religious sects may be flooding if you own a religious site. Your web properties may be site capitalistic and the attackers may be anti-capitalism. You may be American and the attackers could be anti-American. You may be managing debt and the attackers may have sympathy with poor or those in debt. Such principle driven attacks are very common and are difficult to solve on the principle grounds. May be you have a gay/lesbian site and someone doesn’t agree with you.
  • Sometimes attacks have a business reason. You may be in the way of someone else’s business growth. They will then hire a botmaster who will then launch an attack on your properties. Sometimes the attackers may have ethical objection to your business.
  • Angry customers are another cause of the attacks. If you host IRC servers, gambling - especially offshore, or porn sites and you have some angry customers, they may come back to extract revenge. If you have recently banned someone from your servers or if someone has lost a lot of money on your site, they may be behind the attacks.
  • Another type of attacks we have seen are related to social networking. If you are a social networking site and one of the users has written a page against a foreign government. That government may now be after you. Until you remove that page, you will be attacked.

Virus Infections, Botnets and Distributed Attack Tools

Millions of new users join the Internet daily. These are in the far-flung corners of the world. If you are reading this primer, you are savvy - but they are not. If they get an enticing email from someone, they open it and their machine now has bot code which can be remotely controlled by botmasters. Millions of such machines around the world are under control of bot-herds who buy, sell and rent them for monetary gains. Some foreign governments are also known to control them for possible cyber-warfare.
  

Fig. 1 A botnet control panel showing how easy it is to launch a DDoS attack

The above pictures shows a bot-controller that can be easily used to launch a DDoS attack sitting somewhere using bots around the world. The renter uses this software after paying the rent. She can simply enter the URL of the target site to be attacked and choose the type of the attack and launch the attacks including a blended attack. This program controls the bots on the Internet.
 
Following is another example of another bot controller panel. As you can see the panel consists of a display of available bots in different countries and ability to launch different tasks through the bots.
 
Fig. 2 A botnet control panel example also showing status of available bots
 
Another picture below shows ability of the bot-master to upload new functionality into the botnet and update the bots remotely and have an ability to remotely see the status of the bots.
 
Fig. 3 Another botnet control panel example showing ability to update bots remotely

Most Common Current Generation DDoS Attacks

Therea are many kinds of legacy and current generation attacks that are prevalent today. SYN flood and HTTP GET floods being the most common.
 
  • SYN Flood

    Spoofed SYN Packets fill the connection table of servers, and all other devices in your network path. Low volume SYN flood can be easily stopped by software firewalls. High bandwidth SYN floods needs specialized equipment with SYN proxy capability.
 
  • Zombie Flood

    In zombie or botnet floods, non-spoofed connections overload the services. The attacking IPs are able to do three way handshakes. These are difficult to stop unless you have behavioral mitigation. High bandwidth zombie floods needs specialized logic for discriminating legitimate traffic within zombie flood.
 
  • ICMP Flood

    In these floods, ICMP packets, such as those used for ping, overload the servers and the network pipe. Low volume ICMP flood can be easily stopped by ACLs on routers and switches. High bandwidth ICMP floods needs specialized equipment.
 
  • TCP/UDP Port Flood

    In these floods, TCP/UDP packets overload the servers and the pipe on ports not being used for service, e.g. TCP port 81.  Low volume port floods on non-service ports are easily stopped by ACLs. Higher volume need specialized equipment for automatic detection and mitigation unless you have totally blocked all non-service ports by default. Sites that use services such as FTP or IRC that use dynamic ports need to be careful with the stateful traffic on the dynamic ports. Most large attacks that are greater than 1 Gbps involve UDP floods because they are easy to generate by spoofing IPs.

    When packets overload the servers and the pipe on service ports, e.g. TCP port 80. Firewall, switches, routers, IPS appliances cannot stop these attacks. In these cases, you need specialized equipment for discrimination.
 
  • Fragment Flood

    In these flood, fragmented packets overload the servers. Many firewalls, switches, routers cannot stop these attacks unless they have rulesets for dropping fragmented packets. Sometimes you may need specialized equipment.
 
  • Anomalous Packet Flood

    Hackers create most floods with scripts. Sometimes deliberately and sometimes due to errors in scripts, packets are anomalous. These anomalies may be headers at layer 3, 4 or 7. They may be in TCP or UDP states or protocols. These packets overload the CPU of the servers and other networking equipment on the way. Some firewalls, and IPS appliances can stop these attacks. Specialized equipment for DDoS easily stop these attacks.
 
  • HTTP GET Flood

    These attacks involved connection-oriented bots overload the servers and the pipe on service ports, e.g. on HTTP, mimicking legitimate users.  Since firewalls, switches, routers, IPS appliances don't have behavioral anomaly prevention, they cannot stop these attacks. Therefore you need specialized equipment to stop these attacks.
 
  • Blended Attacks

    When multiple types of above attacks are blended on the server, they confuse the conventional equipment further. Firewall, switches, routers, IPS appliances cannot stop these attacks. You need specialized equipment to stop these attacks
 
  • Floods from Unwanted Geographical Areas

    These are very common. For example, you have most of your customers in China and you are getting too many packets from Russia. Such floods are easy to stop with simple access control lists in Anti-DDoS equipment or in switches or routers before the network.
 

Myths and Realities about DDoS Attacks

 
  • Most Network and Security Operations engineers only hear about DDoS attacks happening to others.  They think that they don’t have enemies. In reality, their perceptions of risk factors and suscpetability is most often misplaced. If you have a web presence, you can be attacked easily – sometimes even by mistake.
  •  Many engineers think that they can custom compile kernel, set some options in Apache, install mod_dosevasive and DDoS attacks can be taken care of. In Reality, most servers do not have the capacity to handle DDoS attacks. Under most average sized DDoS attacks, your server CPUs will be too overloaded to give Apache modules a chance.
  •  Another myth that exists is that simple iptables commands can block DDoS attacks. In reality, NetFilter/iptables can block very tiny attacks and tiny percentage of DDoS attacks. Real DDoS attacks require specialized equipment because the CPU running iptables will be too busy handling attack packets.
  •  Many Network and Security Operations engineers think that their webhosts will take care of DDoS attacks. Many webhosts are happy to just null-route an attacked IP domain unless they have specialized equipment. Many webhosts do not have the skills to manually isolate issues - unless they specially advertise such capability.
  • Some think that their ISPs, to whom their webhosting data center is connected to, cooperate under attack and they can find the source of the attack. Most ISPs are too busy. They have strict and bureaucratic processes to reach each other. Typical response time for ISPs are in days if not in hours – whereas you want the solution NOW !!
  •  It is also easy to think that under attack, we can report to law enforcement to solve the problem. In reality, most law enforcement departments will not bother about needle in hay-stack attacks – for them that’s what most attacks are. Unless you are important and the attacks are in multiple 10s of Gigabits per second, don’t waste their time and yours.
  • You may also think that you can determine that ACLs for your routers and switches to block the attacks. DDoS attacks are moving targets. The hackers are smart, their tools are smarter and techniques are sophisticated.
  • Another myth that surrounds DDoS attacks are filled pipes. Many wonder if there is any point in buying any specialized Anti-DDoS equipment. In reality, 90% of the attacks are sub-1Gbps today and if you have that much pipe, you will be better off having a DDoS mitigation solution than not having one at all. Pain from the most complex attacks can be reduced with specialized equipment. Without the DDoS mitigation equipment, your servers will be thoroughly exposed to even the most ordinary attacks. Take the first step. DDoS mitigation eqauipment is not as expensive as you may think. DDoS mitigation costs are proportional to number of links, bandwidth, complexity of policies and type of attacks.  If you have a reasonable sized business, it should not cost you an arm and a leg. There are cost-effective solutions available that are effective.
 

Home Remedies For Simple and Small DDoS Attacks

  • Update kernel to the latest release
  • Install all security updates
  • Disable unused and insecure services
  • Remove unused packages
  • Memory resources can be exhausted by filling up various kernel tables that are not tuned to be sufficiently large. Ensure that you understand various kernel tables.
  • Network card is gateway to the packets.  Better network card means better handling of large number of packets. Better network card driver means better performance.
  • Choose a vendor such as Intel and model which is proven and a driver that’s already hardened.
  • Use NetFilter/iptables firewall to deny bad packets
  • Use Hashlimit module to identify IPs that are consuming resources
  • Use ipset module to block-lists of up to IP addresses that can be queried, loaded and unloaded from user-space.
  • Use command : netstat -plan|grep :80 |awk '{print $5}' |cut -d: -f1 |sort |uniq -c |sort -n to find out if port 80 is being attacked by too many IPs.
  • Use modules such as mod_evasive, mod_limitipconn to limit attacks from limited number of IPs.
  • Try mod_qos to improve quality of service.
  • Apache has its limits. You can try LiteSpeed.

Anti-DDoS Appliances

There are primarily following categories of appliances in the market for DDoS mitigation:
 

      • Carrier DDoS mitigation solutions
        • These solutions are useful for global networks and carriers and ISPs.
        • They employ IP flow-based and deep packet inspection technologies, and protect entire networks consisting of multiple routers and switches and services behind them.
        • Example of such solutions are Arbor Networks.
        • These solutions are too expensive for individual IDCs, webhosts or web properties.
        • These solutions have been designed around early 2000 and therefore are not keeping up with the current generation of DDoS attacks which involve botnets that mimic legitimate clients.
        • These solutions work very well at global level and the residual attacks from such solutions may be too much for an individual web property which in turn may have to employ a solution such as 2 below.
            • Custom Logic (FPGA or ASIC) based Internet Data Center (IDC), Web hosting and Web Property DDoS Mitigation Solutions
              • These solutions are useful for large IDCs, large web hosts and large web properties.
              • They work to protect one or several Internet links.
              • The behavioral solutions are implemented in custom hardware logic and provide line rate performance for large attacks.
              • IntruGuard has one such solution.
              • These solutions are cost-effective and effective for IDCs, webhosts and web properties.
                  • Software based Web Property DDoS Mitigation Solutions
                    • These solutions are useful for smaller web properties with very minimal traffic.
                    • The behavioral solutions are implemented in off-the-shelf CPUs and have issues at large attack traffic volumes in terms of keeping up.
                    • Some appliances have IPS functionality implemented in hardware but have their DDoS mitigation logic in software and suffer from the same issues.
                          

                        Few Things to Look for in Anti-DDoS Equipment

                        • Latest Technology

                          The hackers are pretty up-to-date on techniques. If your DDoS mitigation appliance is built around technology that was developed in early 2000s, it won't help you much as most of the current generation attacks would pass through.

                        • Centralized monitoring

                          Look for appliances that allow you to centrally monitor all DDoS events and traffic in your network.  You can use SNMP, Cacti, MRTG to monitor traffic and attack levels and attack events. You can configure Syslog to get all attack events on a centralized server as well.
                         
                        • Visibility into normal network traffic patterns

                          Look for appliances that allow you to get extremely granular visibility into your network traffic. Typically you should look for a 12 month round robin view of what normal traffic looks like and incorporate this information into a correlation engine for threat detection, alerts, and reporting.
                         
                        • Alerting Mechanisms

                          Look for appliances that give you a threshold based alerting mechanism for DDoS specific events. You can set threshold for different people to get alerts depending on the quantum of attack. You should be able to query a database for Top Attacks, Top Attackers, Top Attacked Destination, etc. You should be able to create custom queries in your custom applications/reports.
                         
                        • Filtering Mechanisms to Reduce False Positives

                          Look for appliances that filter traffic in different network layers as they inspect incoming packets using dynamic profiling (based on monitoring and analysis of normal behavior), anti-spoofing algorithms, and other technology to progressively filter harmful traffic upstream of the network.
                         
                        • Low Latency

                          Latency, in this context, is the amount of time it takes a packet to go through an appliance. Look for appliances that don't affect your mission critical traffic by adding additional significant latency. Most switches and routers have low latency in the range of a < 50 microseconds. The anti-DDoS equipment should maintain similar latency levels. This latency should be maintained even during attacks.
                         
                        • Hardware Logic for Anti-DDoS

                          These days it is common for a $100 home router to claim that it has DDoS attack mitigation capability. Such claims have to withstand third party tests and real life. It is also easy to build Intel CPU based appliance running Linux with some behavioral capability built-in to claim anti-DDoS features. Many IPS appliances have IPS in hardware logic but anti-DDoS capability in software. Such appliances cannot handle attacks beyond a certain Mbps.

                          Look for custom DDoS mitigation logic implemented in hardware as that alone can withstand large DDoS attacks. A granular approach to DDoS mitigation selectively mitigate attacks at highest possible layer so that attacks are stopped at most specific layer. This reduces the false positives.

                          Ability to monitor a large number of ports, sources, destinations, connections etc. helps in proper identification of attacks and attackers.
                         
                        • Bypass and Redundancy

                          Look for internal or external bypass capability that ensures that your network traffic continues even if the appliance fails. For multiple links, look for ability to cross conect appliances in a fail-over configuration. In addition, look for asymmetric traffic support because you may have traffic coming from one link and going through another.
                         
                        • Extensible Architecture

                          Anti-DDoS equipment must grow with your business. Look for appliances that have such capability to grow through licenses.
                        • Third Party Validation

                          Look for third party validation for a solution you choose. That will mitigate some risks of your inability to actually do a test in your own labs.

                         

                        Conclusion

                        DDoS attacks are on the rise. The potential threats and volumes are increasing as more machines including mobile devices join the Internet. If you have a web property, the likelihood of your getting attacked is on the rise. Script kiddies are moving from fame to fortune. It is prudent to plan protection of the infrastructure rather than wait for the attacks to strike. 

                        http://knol.google.com/k/distributed-denial-of-service-attacks-a-primer#