SNMP Enabler for Windows
Save time by remotely installing, enabling, and configuring SNMP on multiple Windows servers and workstations.
Download Now »
Showing posts with label traffic. Show all posts
Showing posts with label traffic. Show all posts
Saturday, August 6, 2011
Saturday, September 11, 2010
Traffic shaping with tc
Sometimes it's necessary to limit traffic bandwidth from and to a container. You can do it using ordinary tc tool.
Contents
* 1 Packet routes
* 2 Limiting outgoing bandwidth
* 3 Limiting incoming bandwidth
* 4 Limiting CT to HN talks
* 5 Limiting packets per second rate from container
* 6 An alternate approch using HTB
* 7 External links
First of all, a few words about how packets travel from and to a container. Suppose we have Hardware Node (HN) with a container (CT) on it, and this container talks to some Remote Host (RH). HN has one "real" network interface eth0 and, thanks to OpenVZ, there is also "virtual" network interface venet0. Inside the container we have interface venet0:0.
venet0:0 venet0 eth0
CT >------------->-------------> HN >--------->--------> RH
venet0:0 venet0 eth0
CT <-------------<-------------< HN <---------<--------< RH
DEV=eth0
tc qdisc del dev $DEV root
tc qdisc add dev $DEV root handle 1: cbq avpkt 1000 bandwidth 100mbit
tc class add dev $DEV parent 1: classid 1:1 cbq rate 256kbit allot 1500 prio 5 bounded isolated
tc filter add dev $DEV parent 1: protocol ip prio 16 u32 match ip src X.X.X.X flowid 1:1
tc qdisc add dev $DEV parent 1:1 sfq perturb 10
X.X.X.X is an IP address of container.
DEV=venet0
tc qdisc del dev $DEV root
tc qdisc add dev $DEV root handle 1: cbq avpkt 1000 bandwidth 100mbit
tc class add dev $DEV parent 1: classid 1:1 cbq rate 256kbit allot 1500 prio 5 bounded isolated
tc filter add dev $DEV parent 1: protocol ip prio 16 u32 match ip dst X.X.X.X flowid 1:1
tc qdisc add dev $DEV parent 1:1 sfq perturb 10
Note that X.X.X.X is an IP address of container.
DEV=venet0
tc filter add dev $DEV parent 1: protocol ip prio 20 u32 match u32 1 0x0000 police rate 2kbit buffer 10k drop flowid :1
To prevent dos atacks from the container you can limit packets per second rate using iptables.
DEV=eth0
iptables -I FORWARD 1 -o $DEV -s X.X.X.X -m limit --limit 200/sec -j ACCEPT
iptables -I FORWARD 2 -o $DEV -s X.X.X.X -j DROP
Here X.X.X.X is an IP address of container.
An alternate approch using HTB
#!/bin/sh
#
# Incoming traffic control
#
CT_IP1=$1
CT_IP2=$2
DEV=venet0
#
tc qdisc del dev $DEV root
#
tc qdisc add dev $DEV root handle 1: htb default 10
#
tc class add dev $DEV parent 1: classid 1:1 htb rate 100mbit burst 15k
tc class add dev $DEV parent 1:1 classid 1:10 htb rate 10mbit ceil 10mbit burst 15k
tc class add dev $DEV parent 1:1 classid 1:20 htb rate 20mbit ceil 20mbit burst 15k
tc class add dev $DEV parent 1:1 classid 1:30 htb rate 30mbit ceil 30mbit burst 15k
#
tc qdisc add dev $DEV parent 1:10 handle 10: sfq perturb 10
tc qdisc add dev $DEV parent 1:20 handle 20: sfq perturb 10
tc qdisc add dev $DEV parent 1:30 handle 30: sfq perturb 10
#
if [ ! -z $CT_IP1 ]; then
tc filter add dev $DEV protocol ip parent 1:0 prio 1 u32 match ip dst "$CT_IP1" flowid 1:20
fi
if [ ! -z $CT_IP2 ]; then
tc filter add dev $DEV protocol ip parent 1:0 prio 1 u32 match ip dst "$CT_IP2" flowid 1:30
fi
#
echo;echo "tc configuration for $DEV:"
tc qdisc show dev $DEV
tc class show dev $DEV
tc filter show dev $DEV
#
# Outgoing traffic control
#
DEV=eth0
#
tc qdisc del dev $DEV root
#
tc qdisc add dev $DEV root handle 1: htb default 10
#
tc class add dev $DEV parent 1: classid 1:1 htb rate 100mbit burst 15k
tc class add dev $DEV parent 1:1 classid 1:10 htb rate 10mbit ceil 10mbit burst 15k
tc class add dev $DEV parent 1:1 classid 1:20 htb rate 20mbit ceil 20mbit burst 15k
tc class add dev $DEV parent 1:1 classid 1:30 htb rate 30mbit ceil 30mbit burst 15k
#
tc qdisc add dev $DEV parent 1:10 handle 10: sfq perturb 10
tc qdisc add dev $DEV parent 1:20 handle 20: sfq perturb 10
tc qdisc add dev $DEV parent 1:30 handle 30: sfq perturb 10
#
if [ ! -z $CT_IP1 ]; then
tc filter add dev $DEV protocol ip parent 1:0 prio 1 u32 match ip src "$CT_IP1" flowid 1:20
fi
if [ ! -z $CT_IP2 ]; then
tc filter add dev $DEV protocol ip parent 1:0 prio 1 u32 match ip src "$CT_IP2" flowid 1:30
fi
#
echo;echo "tc configuration for $DEV:"
tc qdisc show dev $DEV
tc class show dev $DEV
tc filter show dev $DEV
Courtesy : http://wiki.openvz.org/Traffic_shaping_with_tc
Contents
* 1 Packet routes
* 2 Limiting outgoing bandwidth
* 3 Limiting incoming bandwidth
* 4 Limiting CT to HN talks
* 5 Limiting packets per second rate from container
* 6 An alternate approch using HTB
* 7 External links
Packet routes
First of all, a few words about how packets travel from and to a container. Suppose we have Hardware Node (HN) with a container (CT) on it, and this container talks to some Remote Host (RH). HN has one "real" network interface eth0 and, thanks to OpenVZ, there is also "virtual" network interface venet0. Inside the container we have interface venet0:0.
venet0:0 venet0 eth0
CT >------------->-------------> HN >--------->--------> RH
venet0:0 venet0 eth0
CT <-------------<-------------< HN <---------<--------< RH
Limiting outgoing bandwidth
We can limit container outgoing bandwidth by setting the tc filter on eth0.DEV=eth0
tc qdisc del dev $DEV root
tc qdisc add dev $DEV root handle 1: cbq avpkt 1000 bandwidth 100mbit
tc class add dev $DEV parent 1: classid 1:1 cbq rate 256kbit allot 1500 prio 5 bounded isolated
tc filter add dev $DEV parent 1: protocol ip prio 16 u32 match ip src X.X.X.X flowid 1:1
tc qdisc add dev $DEV parent 1:1 sfq perturb 10
X.X.X.X is an IP address of container.
Limiting incoming bandwidth
This can be done by setting the tc filter on venet0:DEV=venet0
tc qdisc del dev $DEV root
tc qdisc add dev $DEV root handle 1: cbq avpkt 1000 bandwidth 100mbit
tc class add dev $DEV parent 1: classid 1:1 cbq rate 256kbit allot 1500 prio 5 bounded isolated
tc filter add dev $DEV parent 1: protocol ip prio 16 u32 match ip dst X.X.X.X flowid 1:1
tc qdisc add dev $DEV parent 1:1 sfq perturb 10
Note that X.X.X.X is an IP address of container.
Limiting CT to HN talks
As you can see, two filters above don't limit container to HN talks. I mean a container can emit as much traffic as it wishes. To make such a limitation from the HN, it is necessary to use tc police on venet0:DEV=venet0
tc filter add dev $DEV parent 1: protocol ip prio 20 u32 match u32 1 0x0000 police rate 2kbit buffer 10k drop flowid :1
Limiting packets per second rate from container
To prevent dos atacks from the container you can limit packets per second rate using iptables.
DEV=eth0
iptables -I FORWARD 1 -o $DEV -s X.X.X.X -m limit --limit 200/sec -j ACCEPT
iptables -I FORWARD 2 -o $DEV -s X.X.X.X -j DROP
Here X.X.X.X is an IP address of container.
An alternate approch using HTB
For details refer to the HTB Home Page
#!/bin/sh
#
# Incoming traffic control
#
CT_IP1=$1
CT_IP2=$2
DEV=venet0
#
tc qdisc del dev $DEV root
#
tc qdisc add dev $DEV root handle 1: htb default 10
#
tc class add dev $DEV parent 1: classid 1:1 htb rate 100mbit burst 15k
tc class add dev $DEV parent 1:1 classid 1:10 htb rate 10mbit ceil 10mbit burst 15k
tc class add dev $DEV parent 1:1 classid 1:20 htb rate 20mbit ceil 20mbit burst 15k
tc class add dev $DEV parent 1:1 classid 1:30 htb rate 30mbit ceil 30mbit burst 15k
#
tc qdisc add dev $DEV parent 1:10 handle 10: sfq perturb 10
tc qdisc add dev $DEV parent 1:20 handle 20: sfq perturb 10
tc qdisc add dev $DEV parent 1:30 handle 30: sfq perturb 10
#
if [ ! -z $CT_IP1 ]; then
tc filter add dev $DEV protocol ip parent 1:0 prio 1 u32 match ip dst "$CT_IP1" flowid 1:20
fi
if [ ! -z $CT_IP2 ]; then
tc filter add dev $DEV protocol ip parent 1:0 prio 1 u32 match ip dst "$CT_IP2" flowid 1:30
fi
#
echo;echo "tc configuration for $DEV:"
tc qdisc show dev $DEV
tc class show dev $DEV
tc filter show dev $DEV
#
# Outgoing traffic control
#
DEV=eth0
#
tc qdisc del dev $DEV root
#
tc qdisc add dev $DEV root handle 1: htb default 10
#
tc class add dev $DEV parent 1: classid 1:1 htb rate 100mbit burst 15k
tc class add dev $DEV parent 1:1 classid 1:10 htb rate 10mbit ceil 10mbit burst 15k
tc class add dev $DEV parent 1:1 classid 1:20 htb rate 20mbit ceil 20mbit burst 15k
tc class add dev $DEV parent 1:1 classid 1:30 htb rate 30mbit ceil 30mbit burst 15k
#
tc qdisc add dev $DEV parent 1:10 handle 10: sfq perturb 10
tc qdisc add dev $DEV parent 1:20 handle 20: sfq perturb 10
tc qdisc add dev $DEV parent 1:30 handle 30: sfq perturb 10
#
if [ ! -z $CT_IP1 ]; then
tc filter add dev $DEV protocol ip parent 1:0 prio 1 u32 match ip src "$CT_IP1" flowid 1:20
fi
if [ ! -z $CT_IP2 ]; then
tc filter add dev $DEV protocol ip parent 1:0 prio 1 u32 match ip src "$CT_IP2" flowid 1:30
fi
#
echo;echo "tc configuration for $DEV:"
tc qdisc show dev $DEV
tc class show dev $DEV
tc filter show dev $DEV
Courtesy : http://wiki.openvz.org/Traffic_shaping_with_tc
Monday, September 6, 2010
Traffic shaping TorrentFlux
TorrentFlux is a great program/interface to download your torrents remotely on a linux machine. It is based on php and it uses a modified bittornado client to download the torrents.
The problem: The bittornado client is able to put specific limits on a per torrent basis. That means that you can put a 100kb/sec download and 50kb/sec upload limit per torrent through torrentflux’s web interface. If you have 20 torrents though, this easily becomes 20*50=1Mb/sec upload “limit”. There are cases that you don’t want this to happen and you want both a per torrent limit (eg 50kb/sec) and a global limit (eg 300kb/sec).
The solution: My solution is based on iptables, layer7 filter and tc (iproute2). I am using layer7 filter to pick out the bittorrent packets, iptables to mark those packets with specific values and tc to shape those marked packets into categories. Beware that the method I am using works mostly on the “uploading” part (outgoing traffic). It is not that hard to make it work for the incoming traffic as well, but it is my personal view that downloading with a few Mb/sec is not as harmfull as uploading with a few Mb/sec. I usually have my downloads seeded over many weeks…so it’s good for my ratio to have the torrent downloaded as fast as possible and then seed it endlessly. I usually like to seed until i get a ratio over 1000% per torrent (that means 10 times as much uploaded traffic than downloaded). The following example configs are created for use on a 100mbit line and keeping in mind that outgoing torrent traffic should not exceed 2-2.5Mbits (~250-300kb/sec).
The procedure:
0) Before you begin make sure you have the kernel sources on /usr/src/linux.
1) Then, you need to patch your kernel for layer7 filtering and enable marking. On gentoo linux you only need to:
and then configure your kernel for marking.
Here’s how my netfilter configuration looks like:
Rebuild your kernel and install the proper modules. If you need to reboot your machine to apply the new kernel do it now.
2) Now it’s time to install iptables and iproute2 if you don’t have them already. On gentoo linux:
3) Now it’s the iptables marking time. I am going to show you (some of) the output of my iptables-save command. Change it to fit your neeeds:
You need to change IP.OF.MACHINE with the IP of your linux box.
4) And now the traffic shaping part:
The rules are pretty straightforward…so I am not going to fully explain them. The basic concept is that you create a “shaped” partition of your bandwith and you add classes (high priority, normal , bittorrent) there. The trick is that you can skip anything you don’t want shaped by marking it with iptables 1001 mark.
In my iptables example above, I mark as 1001 the outgoing ssh and http traffic. This way I can shape the seeding of my torrents using TorrentFlux but I can download via http without any traffic shaping the torrents to my PC at home. I can also ssh to the machine without any latency caused by the shaping because the sshd port (22) is marked with 1001.
The only problem I faced with those scripts was that sometimes the layer7 filter for bittorrent let’s some torrent traffic pass by. My solution to that was to change NORMAL_MAX=$SHAPEDLINK to NORMAL_MAX=2048 for example. Then, even “normal traffic” was shaped. Remember that anything I didn’t want shaped, was marked as 1001 on the iptables script…so the machine was still very responsive even after shaping the “normal traffic”.
To check how your scripts are doing in terms of shaping you can download this excellent perl script: tc-viewer. Click here for a screenshot: tc-viewer htb screenshot
The above example configs are very very generic. If you have a server that serves many other duties apart from ssh, http and bittorrent, then this script might not work out of the box for you.
*Update*
It looks like the problem I had with layer7 bittorrent filter missing packets was not actually a layer7’s “problem”, but rather a new feature of the latest version of bittornado. I was using bittornado version 0.3.18 (experimental) which is the first bittornado version that comes with Message_Stream_Encryption. What this means: whenever bittornado finds another peer with encryption capabilities, it encrypts all traffic between you and the other peer, so the layer7 filter cannot understand that these flows are torrent traffic anymore, and categorizes them as “normal” traffic. That’s why I needed to “shape” normal traffic as well.
There are three ways to cope with encrypted bittorrent traffic. The first one is the one I described above without even knowing about it (shaping normal traffic). The second way is to go back to a version without encryption (0.3.17), which I think is a _really_ bad idea. Encryption came to help us hide our traffic from ISP filters, and is a step we can all take to protect ourselves. The third way is to mark the port range that torrentflux uses (check the admin panel of torrentflux for it) as torrent traffic by our iptables script. If the port range is high enough it can be almost certain that no other service will use those ports, so no priority traffic will be mis-matched as “torrent traffic”. If, for example, you have defined your port range to be from port 61000 to port 63000, then inject a command like:
just below the
command of the iptables script above.
Enjoy shaped encrypted bittorent uploads! Keep seeding…
Source : http://www.void.gr/kargig/blog/2007/01/14/traffic-shaping-torrentflux/
The problem: The bittornado client is able to put specific limits on a per torrent basis. That means that you can put a 100kb/sec download and 50kb/sec upload limit per torrent through torrentflux’s web interface. If you have 20 torrents though, this easily becomes 20*50=1Mb/sec upload “limit”. There are cases that you don’t want this to happen and you want both a per torrent limit (eg 50kb/sec) and a global limit (eg 300kb/sec).
The solution: My solution is based on iptables, layer7 filter and tc (iproute2). I am using layer7 filter to pick out the bittorrent packets, iptables to mark those packets with specific values and tc to shape those marked packets into categories. Beware that the method I am using works mostly on the “uploading” part (outgoing traffic). It is not that hard to make it work for the incoming traffic as well, but it is my personal view that downloading with a few Mb/sec is not as harmfull as uploading with a few Mb/sec. I usually have my downloads seeded over many weeks…so it’s good for my ratio to have the torrent downloaded as fast as possible and then seed it endlessly. I usually like to seed until i get a ratio over 1000% per torrent (that means 10 times as much uploaded traffic than downloaded). The following example configs are created for use on a 100mbit line and keeping in mind that outgoing torrent traffic should not exceed 2-2.5Mbits (~250-300kb/sec).
The procedure:
0) Before you begin make sure you have the kernel sources on /usr/src/linux.
1) Then, you need to patch your kernel for layer7 filtering and enable marking. On gentoo linux you only need to:
#emerge -avt net-misc/l7-filter net-misc/l7-protocols and then configure your kernel for marking.
Here’s how my netfilter configuration looks like:
CONFIG_NETFILTER=yYou can clearly see layer7 being enabled as a module: CONFIG_IP_NF_MATCH_LAYER7=m
# CONFIG_NETFILTER_DEBUG is not set
# CONFIG_BRIDGE_NETFILTER is not set
#
# Core Netfilter Configuration
#
# CONFIG_NETFILTER_NETLINK is not set
CONFIG_NETFILTER_XTABLES=y
CONFIG_NETFILTER_XT_TARGET_CLASSIFY=m
# CONFIG_NETFILTER_XT_TARGET_CONNMARK is not set
CONFIG_NETFILTER_XT_TARGET_MARK=m
CONFIG_NETFILTER_XT_TARGET_NFQUEUE=m
CONFIG_NETFILTER_XT_MATCH_COMMENT=m
CONFIG_NETFILTER_XT_MATCH_CONNBYTES=m
CONFIG_NETFILTER_XT_MATCH_CONNMARK=m
CONFIG_NETFILTER_XT_MATCH_CONNTRACK=m
CONFIG_NETFILTER_XT_MATCH_DCCP=m
CONFIG_NETFILTER_XT_MATCH_ESP=m
CONFIG_NETFILTER_XT_MATCH_HELPER=m
CONFIG_NETFILTER_XT_MATCH_LENGTH=m
CONFIG_NETFILTER_XT_MATCH_LIMIT=m
CONFIG_NETFILTER_XT_MATCH_MAC=m
CONFIG_NETFILTER_XT_MATCH_MARK=m
# CONFIG_NETFILTER_XT_MATCH_POLICY is not set
CONFIG_NETFILTER_XT_MATCH_MULTIPORT=m
CONFIG_NETFILTER_XT_MATCH_PKTTYPE=m
# CONFIG_NETFILTER_XT_MATCH_QUOTA is not set
CONFIG_NETFILTER_XT_MATCH_REALM=m
CONFIG_NETFILTER_XT_MATCH_SCTP=m
CONFIG_NETFILTER_XT_MATCH_STATE=m
# CONFIG_NETFILTER_XT_MATCH_STATISTIC is not set
CONFIG_NETFILTER_XT_MATCH_STRING=m
CONFIG_NETFILTER_XT_MATCH_TCPMSS=m
#
# IP: Netfilter Configuration
#
CONFIG_IP_NF_CONNTRACK=m
CONFIG_IP_NF_CT_ACCT=y
CONFIG_IP_NF_CONNTRACK_MARK=y
# CONFIG_IP_NF_CONNTRACK_EVENTS is not set
CONFIG_IP_NF_CT_PROTO_SCTP=m
CONFIG_IP_NF_FTP=m
CONFIG_IP_NF_IRC=m
CONFIG_IP_NF_NETBIOS_NS=m
CONFIG_IP_NF_TFTP=m
CONFIG_IP_NF_AMANDA=m
CONFIG_IP_NF_PPTP=m
CONFIG_IP_NF_H323=m
CONFIG_IP_NF_SIP=m
CONFIG_IP_NF_QUEUE=m
CONFIG_IP_NF_IPTABLES=y
CONFIG_IP_NF_MATCH_IPRANGE=y
CONFIG_IP_NF_MATCH_LAYER7=m
# CONFIG_IP_NF_MATCH_LAYER7_DEBUG is not set
CONFIG_IP_NF_MATCH_TOS=y
CONFIG_IP_NF_MATCH_RECENT=m
CONFIG_IP_NF_MATCH_ECN=m
CONFIG_IP_NF_MATCH_DSCP=m
CONFIG_IP_NF_MATCH_AH=m
CONFIG_IP_NF_MATCH_TTL=m
CONFIG_IP_NF_MATCH_OWNER=m
CONFIG_IP_NF_MATCH_ADDRTYPE=m
CONFIG_IP_NF_MATCH_HASHLIMIT=m
CONFIG_IP_NF_FILTER=y
CONFIG_IP_NF_TARGET_REJECT=y
CONFIG_IP_NF_TARGET_LOG=y
CONFIG_IP_NF_TARGET_ULOG=m
CONFIG_IP_NF_TARGET_TCPMSS=y
CONFIG_IP_NF_NAT=m
CONFIG_IP_NF_NAT_NEEDED=y
CONFIG_IP_NF_TARGET_MASQUERADE=m
CONFIG_IP_NF_TARGET_REDIRECT=m
CONFIG_IP_NF_TARGET_NETMAP=m
CONFIG_IP_NF_TARGET_SAME=m
CONFIG_IP_NF_NAT_SNMP_BASIC=m
CONFIG_IP_NF_NAT_IRC=m
CONFIG_IP_NF_NAT_FTP=m
CONFIG_IP_NF_NAT_TFTP=m
CONFIG_IP_NF_NAT_AMANDA=m
CONFIG_IP_NF_NAT_PPTP=m
CONFIG_IP_NF_NAT_H323=m
CONFIG_IP_NF_NAT_SIP=m
CONFIG_IP_NF_MANGLE=m
CONFIG_IP_NF_TARGET_TOS=m
CONFIG_IP_NF_TARGET_ECN=m
CONFIG_IP_NF_TARGET_DSCP=m
CONFIG_IP_NF_TARGET_TTL=m
CONFIG_IP_NF_TARGET_CLUSTERIP=m
# CONFIG_IP_NF_RAW is not set
CONFIG_IP_NF_ARPTABLES=m
CONFIG_IP_NF_ARPFILTER=m
CONFIG_IP_NF_ARP_MANGLE=m
Rebuild your kernel and install the proper modules. If you need to reboot your machine to apply the new kernel do it now.
2) Now it’s time to install iptables and iproute2 if you don’t have them already. On gentoo linux:
#echo "net-firewall/iptables extensions l7filter" >> /etc/portage/package.use
#emerge -avt net-firewall/iptables sys-apps/iproute2
3) Now it’s the iptables marking time. I am going to show you (some of) the output of my iptables-save command. Change it to fit your neeeds:
# Generated by iptables-save v1.3.5 on Fri Jan 12 20:50:52 2007
*mangle
REROUTING ACCEPT [1102387:193393325]
:INPUT ACCEPT [1102372:193390208]
:FORWARD ACCEPT [0:0]
:OUTPUT ACCEPT [2100485:2922693566]
OSTROUTING ACCEPT [2100483:2922690566]
-A PREROUTING -s IP.OF.MACHINE -p tcp -m multiport --sports 22,80 -j MARK --set-mark 1001
-A PREROUTING -d IP.OF.MACHINE -p tcp -m multiport --dports 22,80 -j MARK --set-mark 1001
-A PREROUTING -m layer7 --l7proto ssh -j MARK --set-mark 1001
#-A PREROUTING -m layer7 --l7proto bittorrent -j MARK --set-mark 11090
-A PREROUTING -m mark --mark 1001 -j RETURN
-A POSTROUTING -s IP.OF.MACHINE -p tcp -m multiport --sports 22,80 -j MARK --set-mark 1001
-A POSTROUTING -d IP.OF.MACHINE -p tcp -m multiport --dports 22,80 -j MARK --set-mark 1001
-A POSTROUTING -m mark --mark 1001 -j RETURN
-A POSTROUTING -m connmark --mark 0x0 -j MARK --set-mark 11030
-A POSTROUTING -m layer7 --l7proto dns -j MARK --set-mark 11010
-A POSTROUTING -m layer7 --l7proto ssh -j MARK --set-mark 11010
-A POSTROUTING -p tcp -m tcp --tcp-flags SYN,RST,ACK SYN -j MARK --set-mark 11010
-A POSTROUTING -p icmp -j MARK --set-mark 11010
-A POSTROUTING -m layer7 --l7proto bittorrent -j MARK --set-mark 11090
COMMIT
# Completed on Fri Jan 12 20:50:52 2007
# Generated by iptables-save v1.3.5 on Fri Jan 12 20:50:52 2007
*nat
REROUTING ACCEPT [407:30699]
OSTROUTING ACCEPT [111:6662]
:OUTPUT ACCEPT [111:6662]
COMMIT
# Completed on Fri Jan 12 20:50:52 2007
# Generated by iptables-save v1.3.5 on Fri Jan 12 20:50:52 2007
*filter
:INPUT ACCEPT [266369:32040284]
:FORWARD ACCEPT [0:0]
:OUTPUT ACCEPT [479227:676859047]
COMMIT
# Completed on Fri Jan 12 20:50:52 2007
You need to change IP.OF.MACHINE with the IP of your linux box.
4) And now the traffic shaping part:
# Main Link
LINK=100000
SHAPEDLINK=50000
# High Priority
HIGHPRIO=10000
HIGHPRIO_MAX=$SHAPEDLINK
# Normal
NORMAL=512
NORMAL_MAX=$SHAPEDLINK
# Downloads
TOR=512
TOR_MAX=2048
# del old
tc qdisc del dev $DEV root 2> /dev/null > /dev/null
# add root
tc qdisc add dev $DEV root handle 100: htb default 1
tc class add dev $DEV parent 100: classid 100:1 htb rate ${LINK}kbit
tc qdisc add dev $DEV parent 100:1 handle 1: htb
tc class add dev $DEV parent 1: classid 1:1 htb rate ${SHAPEDLINK}kbit
# some more rules
tc class add dev $DEV parent 100: classid 100:1 htb rate ${LINK}kbit
tc qdisc add dev $DEV parent 100:1 sfq perturb 10
tc filter add dev $DEV parent 100:0 protocol ip prio 1 handle 1001 fw flowid 100:1
tc class add dev $DEV parent 1:1 classid 1:10 htb rate ${SHAPEDLINK}kbit ceil ${SHAPEDLINK}kbit prio 5
tc qdisc add dev $DEV parent 1:10 sfq perturb 10
# High priority
tc class add dev $DEV parent 1:10 classid 1:1010 htb rate ${HIGHPRIO}kbit ceil ${HIGHPRIO_MAX}kbit prio 0
tc qdisc add dev $DEV parent 1:1010 sfq perturb 10
tc filter add dev $DEV parent 1:0 protocol ip prio 0 handle 11010 fw flowid 1:1010
# normal
tc class add dev $DEV parent 1:10 classid 1:1030 htb rate ${NORMAL}kbit ceil ${NORMAL_MAX}kbit prio 5
tc qdisc add dev $DEV parent 1:1030 sfq perturb 10
tc filter add dev $DEV parent 1:0 protocol ip prio 5 handle 11030 fw flowid 1:1030
# bittorent
tc class add dev $DEV parent 1:10 classid 1:1090 htb rate ${TOR}kbit ceil ${TOR_MAX}kbit prio 10
tc qdisc add dev $DEV parent 1:1090 sfq perturb 10
tc filter add dev $DEV parent 1:0 protocol ip prio 10 handle 11090 fw flowid 1:1090
The rules are pretty straightforward…so I am not going to fully explain them. The basic concept is that you create a “shaped” partition of your bandwith and you add classes (high priority, normal , bittorrent) there. The trick is that you can skip anything you don’t want shaped by marking it with iptables 1001 mark.
In my iptables example above, I mark as 1001 the outgoing ssh and http traffic. This way I can shape the seeding of my torrents using TorrentFlux but I can download via http without any traffic shaping the torrents to my PC at home. I can also ssh to the machine without any latency caused by the shaping because the sshd port (22) is marked with 1001.
The only problem I faced with those scripts was that sometimes the layer7 filter for bittorrent let’s some torrent traffic pass by. My solution to that was to change NORMAL_MAX=$SHAPEDLINK to NORMAL_MAX=2048 for example. Then, even “normal traffic” was shaped. Remember that anything I didn’t want shaped, was marked as 1001 on the iptables script…so the machine was still very responsive even after shaping the “normal traffic”.
To check how your scripts are doing in terms of shaping you can download this excellent perl script: tc-viewer. Click here for a screenshot: tc-viewer htb screenshot
The above example configs are very very generic. If you have a server that serves many other duties apart from ssh, http and bittorrent, then this script might not work out of the box for you.
*Update*
It looks like the problem I had with layer7 bittorrent filter missing packets was not actually a layer7’s “problem”, but rather a new feature of the latest version of bittornado. I was using bittornado version 0.3.18 (experimental) which is the first bittornado version that comes with Message_Stream_Encryption. What this means: whenever bittornado finds another peer with encryption capabilities, it encrypts all traffic between you and the other peer, so the layer7 filter cannot understand that these flows are torrent traffic anymore, and categorizes them as “normal” traffic. That’s why I needed to “shape” normal traffic as well.
There are three ways to cope with encrypted bittorrent traffic. The first one is the one I described above without even knowing about it (shaping normal traffic). The second way is to go back to a version without encryption (0.3.17), which I think is a _really_ bad idea. Encryption came to help us hide our traffic from ISP filters, and is a step we can all take to protect ourselves. The third way is to mark the port range that torrentflux uses (check the admin panel of torrentflux for it) as torrent traffic by our iptables script. If the port range is high enough it can be almost certain that no other service will use those ports, so no priority traffic will be mis-matched as “torrent traffic”. If, for example, you have defined your port range to be from port 61000 to port 63000, then inject a command like:
-A POSTROUTING -p tcp --sport 61000:63000 -j MARK --set-mark 11090just below the
-A POSTROUTING -m layer7 --l7proto bittorrent -j MARK --set-mark 11090command of the iptables script above.
Enjoy shaped encrypted bittorent uploads! Keep seeding…
Source : http://www.void.gr/kargig/blog/2007/01/14/traffic-shaping-torrentflux/
During some QoS tests on Linux I needed to measure the traffic of the system in realtime without being able to compile any new software on it. The system had already perl installed so I googled to find a script that could monitor in/out traffic of an interface. The first script I found was this: http://perlmonks.org/?node_id=635792
While it’s actually doing what it says, it only runs just once. I wanted the script to run for a period of time. So I changed it a bit.
Here’s the outcome:
I’ve changed it so that it’s:
a) running continuously until someone presses ctrl+c to stop it,
b) parsing the /proc/net/dev output instead of the ifconfig output. I think this is more efficient/fast than parsing the ifconfig output.
Sample output:
You can also download a version with comments that I made so that one can make the script run for X number of repetitions instead of running until someone stops it.
http://www.void.gr/kargig/blog/wp-content/iftraffic.pl
Source : http://www.void.gr/kargig/blog/2009/05/04/iftraffic-perl-script-to-measure-inout-traffic-in-realtime/
While it’s actually doing what it says, it only runs just once. I wanted the script to run for a period of time. So I changed it a bit.
Here’s the outcome:
#!/usr/bin/perl
my $dev=$ARGV[0];
sub get_measures {
my $data = `cat /proc/net/dev | grep "$dev" | head -n1`;
$data =~ /$dev\:(\d+)\D+\d+\D+\d+\D+\d+\D+\d+\D+\d+\D+\d+\D+\d+\D+(\d+)\D+/;
my $recv = int($1/1024);
my $sent= int($2/1024);
return ($recv,$sent);
}
my @m1 = get_measures;
while(1) {
sleep 1;
my @m2 = get_measures;
my @rates = ($m2[0] - $m1[0], $m2[1]-$m1[1]);
foreach ('received' , ' transmit') {
printf "$_ rate:%sKB",shift @rates;
}
print "\n";
@m1=@m2;
}I’ve changed it so that it’s:
a) running continuously until someone presses ctrl+c to stop it,
b) parsing the /proc/net/dev output instead of the ifconfig output. I think this is more efficient/fast than parsing the ifconfig output.
Sample output:
$iftraffic.pl eth0I am sure that someone with more insight into perl than me can make it even more efficient.
received rate:1564KB transmit rate:71KB
received rate:1316KB transmit rate:44KB
received rate:1415KB transmit rate:48KB
received rate:1579KB transmit rate:76KB
You can also download a version with comments that I made so that one can make the script run for X number of repetitions instead of running until someone stops it.
http://www.void.gr/kargig/blog/wp-content/iftraffic.pl
Source : http://www.void.gr/kargig/blog/2009/05/04/iftraffic-perl-script-to-measure-inout-traffic-in-realtime/
traffic shaping a dsl line with linux
The case is like this:
[code]
Internet < --> [dsl modem] < --> [linux box] < --> [Lan]
[/code]
DSL modem is connected on eth2 on linux box and the rest of the Lan on eth0. I had a serious problem with people leaving edonkey clients opens all night..limiting the download speed to 20kb/sec but forgetting to limit the upload. The current dsl line is 384/128 so having the uploads unlimited…is like killing the line.
The solution was to setup a QOS script. And here it is:
[code]
#!/bin/bash
DEV="eth2"
LOCALIF="eth2"
# Reset everything to a known state (cleared)
tc qdisc del dev $DEV root 2> /dev/null > /dev/null
tc qdisc del dev imq0 root 2> /dev/null > /dev/null
iptables -t mangle -F POSTROUTING 2> /dev/null > /dev/null
iptables -t mangle -Z POSTROUTING 2> /dev/null > /dev/null
iptables -t mangle -X POSTROUTING 2> /dev/null > /dev/null
iptables -t mangle -F tosfix
iptables -t mangle -F ack
ip link set imq0 down 2> /dev/null > /dev/null
rmmod imq 2> /dev/null > /dev/null
if [ "$1" = "stop" ]
then
echo "Shaping removed on $DEV."
exit
fi
tc qdisc add dev $DEV root handle 1: tbf rate 85kbit burst 1600 limit 1
tc qdisc add dev $DEV parent 1:1 handle 2: prio bands 4
tc qdisc add dev $DEV parent 2:1 handle 10: sfq perturb 10
tc qdisc add dev $DEV parent 2:2 handle 20: sfq perturb 10
tc qdisc add dev $DEV parent 2:3 handle 30: sfq perturb 10
tc qdisc add dev $DEV parent 2:4 handle 40: tbf rate 40kbit burst 1600 limit 3000
tc qdisc add dev $DEV parent 40:1 handle 41: pfifo limit 10
iptables -t mangle -N tosfix
iptables -t mangle -A tosfix -p tcp -m length --length 0:64 -j RETURN
iptables -t mangle -A tosfix -m limit --limit 2/s --limit-burst 10 -j RETURN
iptables -t mangle -A tosfix -j TOS --set-tos Maximize-Throughput
iptables -t mangle -A tosfix -j RETURN
iptables -t mangle -N ack
iptables -t mangle -A ack -m tos ! --tos Normal-Service -j RETURN
iptables -t mangle -A ack -p tcp -m length --length 0:64 \
-j TOS --set-tos Minimize-Delay
iptables -t mangle -A ack -p tcp -m length --length 64: \
-j TOS --set-tos Maximize-Throughput
iptables -t mangle -A ack -j RETURN
# Is our TOS broken? Fix it for TCP ACK and OpenSSH.
iptables -t mangle -A POSTROUTING -p tcp -m tcp --tcp-flags SYN,RST,ACK ACK -j ack
iptables -t mangle -A POSTROUTING -p tcp -m tos --tos Minimize-Delay -j tosfix
# Here we deal with ACK, SYN, and RST packets
# Match SYN and RST packets
iptables -t mangle -A POSTROUTING -o $LOCALIF -p tcp -m tcp --tcp-flags ! SYN,RST,ACK ACK \
-j CLASSIFY --set-class 2:1
# Match ACK packets
iptables -t mangle -A POSTROUTING -o $LOCALIF -p tcp -m tcp --tcp-flags SYN,RST,ACK ACK \
-m length --length :128 -m tos --tos Minimize-Delay \
-j CLASSIFY --set-class 2:1
# Match packets with TOS Minimize-Delay
iptables -t mangle -A POSTROUTING -o $LOCALIF -p tcp -m tos --tos Minimize-Delay \
-j CLASSIFY --set-class 2:1
### Actual traffic shaping classifications with CLASSIFY
# ICMP (ping)
iptables -t mangle -A POSTROUTING -o $LOCALIF -p icmp -j CLASSIFY --set-class 2:1
# Outbound client requests for HTTP, IRC and AIM (dport matches)
iptables -t mangle -A POSTROUTING -o $LOCALIF -p tcp --dport 80 -j CLASSIFY --set-class 2:2
iptables -t mangle -A POSTROUTING -o $LOCALIF -p tcp --dport 6667 -j CLASSIFY --set-class 2:2
iptables -t mangle -A POSTROUTING -o $LOCALIF -p tcp --dport 5190 -j CLASSIFY --set-class 2:2
iptables -t mangle -A POSTROUTING -o $LOCALIF -p tcp --sport 80 -j CLASSIFY --set-class 2:3
iptables -t mangle -A POSTROUTING -o $LOCALIF -p tcp --dport 1024: -j CLASSIFY --set-class 2:4
[/code]
It WORKS for me…I don’t know whether it will work for you though. I take no responsibility. I will explain it no further because comments do exists and it’s really easy to understand what’s going on if you read a couple of tc tutorials from the net. Many ideas about this script were “stolen” from other scripts I studied while trying to make mine.
Have fun with it…
Source : http://www.void.gr/kargig/blog/2005/07/27/traffic-shaping-a-dsl-line-with-linux/
[code]
Internet < --> [dsl modem] < --> [linux box] < --> [Lan]
[/code]
DSL modem is connected on eth2 on linux box and the rest of the Lan on eth0. I had a serious problem with people leaving edonkey clients opens all night..limiting the download speed to 20kb/sec but forgetting to limit the upload. The current dsl line is 384/128 so having the uploads unlimited…is like killing the line.
The solution was to setup a QOS script. And here it is:
[code]
#!/bin/bash
DEV="eth2"
LOCALIF="eth2"
# Reset everything to a known state (cleared)
tc qdisc del dev $DEV root 2> /dev/null > /dev/null
tc qdisc del dev imq0 root 2> /dev/null > /dev/null
iptables -t mangle -F POSTROUTING 2> /dev/null > /dev/null
iptables -t mangle -Z POSTROUTING 2> /dev/null > /dev/null
iptables -t mangle -X POSTROUTING 2> /dev/null > /dev/null
iptables -t mangle -F tosfix
iptables -t mangle -F ack
ip link set imq0 down 2> /dev/null > /dev/null
rmmod imq 2> /dev/null > /dev/null
if [ "$1" = "stop" ]
then
echo "Shaping removed on $DEV."
exit
fi
tc qdisc add dev $DEV root handle 1: tbf rate 85kbit burst 1600 limit 1
tc qdisc add dev $DEV parent 1:1 handle 2: prio bands 4
tc qdisc add dev $DEV parent 2:1 handle 10: sfq perturb 10
tc qdisc add dev $DEV parent 2:2 handle 20: sfq perturb 10
tc qdisc add dev $DEV parent 2:3 handle 30: sfq perturb 10
tc qdisc add dev $DEV parent 2:4 handle 40: tbf rate 40kbit burst 1600 limit 3000
tc qdisc add dev $DEV parent 40:1 handle 41: pfifo limit 10
iptables -t mangle -N tosfix
iptables -t mangle -A tosfix -p tcp -m length --length 0:64 -j RETURN
iptables -t mangle -A tosfix -m limit --limit 2/s --limit-burst 10 -j RETURN
iptables -t mangle -A tosfix -j TOS --set-tos Maximize-Throughput
iptables -t mangle -A tosfix -j RETURN
iptables -t mangle -N ack
iptables -t mangle -A ack -m tos ! --tos Normal-Service -j RETURN
iptables -t mangle -A ack -p tcp -m length --length 0:64 \
-j TOS --set-tos Minimize-Delay
iptables -t mangle -A ack -p tcp -m length --length 64: \
-j TOS --set-tos Maximize-Throughput
iptables -t mangle -A ack -j RETURN
# Is our TOS broken? Fix it for TCP ACK and OpenSSH.
iptables -t mangle -A POSTROUTING -p tcp -m tcp --tcp-flags SYN,RST,ACK ACK -j ack
iptables -t mangle -A POSTROUTING -p tcp -m tos --tos Minimize-Delay -j tosfix
# Here we deal with ACK, SYN, and RST packets
# Match SYN and RST packets
iptables -t mangle -A POSTROUTING -o $LOCALIF -p tcp -m tcp --tcp-flags ! SYN,RST,ACK ACK \
-j CLASSIFY --set-class 2:1
# Match ACK packets
iptables -t mangle -A POSTROUTING -o $LOCALIF -p tcp -m tcp --tcp-flags SYN,RST,ACK ACK \
-m length --length :128 -m tos --tos Minimize-Delay \
-j CLASSIFY --set-class 2:1
# Match packets with TOS Minimize-Delay
iptables -t mangle -A POSTROUTING -o $LOCALIF -p tcp -m tos --tos Minimize-Delay \
-j CLASSIFY --set-class 2:1
### Actual traffic shaping classifications with CLASSIFY
# ICMP (ping)
iptables -t mangle -A POSTROUTING -o $LOCALIF -p icmp -j CLASSIFY --set-class 2:1
# Outbound client requests for HTTP, IRC and AIM (dport matches)
iptables -t mangle -A POSTROUTING -o $LOCALIF -p tcp --dport 80 -j CLASSIFY --set-class 2:2
iptables -t mangle -A POSTROUTING -o $LOCALIF -p tcp --dport 6667 -j CLASSIFY --set-class 2:2
iptables -t mangle -A POSTROUTING -o $LOCALIF -p tcp --dport 5190 -j CLASSIFY --set-class 2:2
iptables -t mangle -A POSTROUTING -o $LOCALIF -p tcp --sport 80 -j CLASSIFY --set-class 2:3
iptables -t mangle -A POSTROUTING -o $LOCALIF -p tcp --dport 1024: -j CLASSIFY --set-class 2:4
[/code]
It WORKS for me…I don’t know whether it will work for you though. I take no responsibility. I will explain it no further because comments do exists and it’s really easy to understand what’s going on if you read a couple of tc tutorials from the net. Many ideas about this script were “stolen” from other scripts I studied while trying to make mine.
Have fun with it…
Source : http://www.void.gr/kargig/blog/2005/07/27/traffic-shaping-a-dsl-line-with-linux/
Traffic Shaping (Explanation 1)
The most fundamental question is why is it necessary traffic regulation or traffic shaping?
1. You use your home office speedy to three computers. You do not need traffic shapping.
2. You use FastNet 384 kbps at home for three computers. You do not need traffic shapping.
3. If your users and your bandwidth is a little big, tell your users 100, 8 Mbps symmetris your bandwidth, you seem to not need traffic shaping (especially if it's also debatable use voip or video conferencing).
4. If your users only one to five people can say you do not need traffic shaping, because your bandwidth is still sufficient to serve your users. But what if your users more than 15 people and each perform a remote ssh connection, in addition to browse and download. You could say you'll have a problem, if you do not shape traffic to upload and create policy for your downlink. I experienced it myself with about 60 users of bandwidth-hungry.
5. Maintain low latency for interactive traffic. This means that the process of downloading and uploading should not disturb SSH, telnet and the like. It is the most important. With a latency of 200ms is enough to make working with SSH very uncomfortable.
6. Keeping the user can still browse the internet with a comfortable speed while doing the uploading or downloading.
7. Ensuring that the upload process does not compromise the process of downloading and vice versa. Please understand that there is a big queue at the device like a cable modem or ADSL modem will make uploading, downloading and interactive traffic will play each other each other.
In my previous article, I gave the script to perform traffic shaping in openSUSE (well, also for other Linux distributions). Now I will explain what the intent of the script. When was the first time to learn tc, ip, and HTB I realized that this was the most difficult.
#!/bin/sh
#
#
# /etc/init.d/mebwshaper_eth1
#
### BEGIN INIT INFO
# Provides: mebwshaper_eth1
# Required-Start: $network
# Should-Start:
# Required-Stop:
# Should-Stop:
# Default-Start: 3 5
# Default-Stop: 0 1 2 6
# Short-Description: Custom bandwidth shaping by medwinz@gmail.com
# Description: Custom bandwidth shaping by medwinz@gmail.com
### END INIT INFO
#
Courtesy : http://medwinz.blogsome.com/2008/11/18/traffic-shaping-lagi-huh/
1. You use your home office speedy to three computers. You do not need traffic shapping.
2. You use FastNet 384 kbps at home for three computers. You do not need traffic shapping.
3. If your users and your bandwidth is a little big, tell your users 100, 8 Mbps symmetris your bandwidth, you seem to not need traffic shaping (especially if it's also debatable use voip or video conferencing).
4. If your users only one to five people can say you do not need traffic shaping, because your bandwidth is still sufficient to serve your users. But what if your users more than 15 people and each perform a remote ssh connection, in addition to browse and download. You could say you'll have a problem, if you do not shape traffic to upload and create policy for your downlink. I experienced it myself with about 60 users of bandwidth-hungry.
5. Maintain low latency for interactive traffic. This means that the process of downloading and uploading should not disturb SSH, telnet and the like. It is the most important. With a latency of 200ms is enough to make working with SSH very uncomfortable.
6. Keeping the user can still browse the internet with a comfortable speed while doing the uploading or downloading.
7. Ensuring that the upload process does not compromise the process of downloading and vice versa. Please understand that there is a big queue at the device like a cable modem or ADSL modem will make uploading, downloading and interactive traffic will play each other each other.
In my previous article, I gave the script to perform traffic shaping in openSUSE (well, also for other Linux distributions). Now I will explain what the intent of the script. When was the first time to learn tc, ip, and HTB I realized that this was the most difficult.
#!/bin/sh
#
#
# /etc/init.d/mebwshaper_eth1
#
### BEGIN INIT INFO
# Provides: mebwshaper_eth1
# Required-Start: $network
# Should-Start:
# Required-Stop:
# Should-Stop:
# Default-Start: 3 5
# Default-Stop: 0 1 2 6
# Short-Description: Custom bandwidth shaping by medwinz@gmail.com
# Description: Custom bandwidth shaping by medwinz@gmail.com
### END INIT INFO
#
Courtesy : http://medwinz.blogsome.com/2008/11/18/traffic-shaping-lagi-huh/
Subscribe to:
Posts (Atom)