Ride the Storm: Navigating Through Unstable Periods / Katerina Rudko (Belka G...
Mikrotik Per traffic loadbalance
1. Per-Traffic Load Balancing 1
Per-Traffic Load Balancing
Introduction
Bandwidth management is an essential part of every day operation for typical ISP's, business, and even everyday
home users. There are many different types of management tools available to RouterOS users, QoS, rate-limiting,
packet-limiting, to name few
I personally operate a wireless ISP in an area that has no other type of conventional high-speed internet conection
(ie. cable, fibre or DSL). Not having access to fibre myself, I am in a situation where the single fastest backbone
connection I can get does not provide enough bandwidth for me to have only one connection. As a result of this
limitation load-balancing multiple internet backbone connections is very important.
In the past I have used ECMP, persistent per connection styled load-balancing (see Routing), as well as various other
methods. However, I found all of then lacking in various different areas (not load-balancing correctly, broken large
HTTP downloads, IM problems, to name a few issues). I then investigated a way to give me more control over my
bandwidth while minimizing the potential problems. The end result was a per-traffic type of load-balancing. This
tutorial is designed address that one specific area in depth, at a later date, I may expand/add additional info on
fail-over, and other topics
Functions of RouterOS used
• Firewall mangle rules
• Firewall address-lists
• Routing
Step 1 - How to break up to traffic
Before you even log onto your RouterOS box, you should have an idea of how you want to divide your traffic, and
have an understanding of what traffic can and what traffic can't be broken up.
Here is an example of what you may want to attempt to separate (in no particular order)
• HTTP traffic (port 80)
• SSL traffic (port 443)
• POP3 traffic (port 110)
• SMTP traffic (port 25)
• P2P traffic (various port)
• Unknown traffic (various port)
After making a list of the traffic type, and the ports they operate on, you need to look at the list and decide if it is at
all even capable of being forced out a certain internet connection.
Using the above list as an example here is what I came up with
• HTTP traffic (no problems found yet)
• SSL traffic (some issues, for normal SSL websites and 90% of all software it is not a problem, I'll explain why
issues can arise later)
• POP3 traffic (no problems found yet)
• SMTP traffic (no problems found yet)
• P2P traffic (must go out same internet connection as Unknown Traffic, I'll explain why this is later)
• Unknown traffic (must go out same internet connection as P2P traffic, I'll explain why this is later)
2. Per-Traffic Load Balancing 2
Now for a quick explanation of why there can be some issues depending on the traffic type. Some websites/programs
do not play friendly with multiple requests from different IP addresses, this is the reason why ECMP has so many
problems. I will provide a simple solution to the rare conditions where load-balancing can't be completed.
Again using the above as an example lets explain where and why issues can be arise. SSL - The beauty of a website
is that are separate requests for different data, ie. loading pictures from 3 different sources would be a request to each
respective server. The result is if a we have a website that uses both SSL and HTTP traffic we know that in most
cases the website will just answer the request without care of the originating IP address. However, and this is a
special case, if the website/program developer checks where the requests are originating and they find that the IP are
different, they may not succesfully answer those requests (this can be by design or by accident). I had 2 cases of this,
in both cases it was secured medical websites that where using it as a method of protecting their data.
P2P and Unknown - I am addressing these together because the issue is one and the same. RouterOS doesn't identify
P2P based on any single condition, but instead analyzes the packets! This means RouterOS needs time to watch the
data before it realizes that is in fact P2P traffic. As a result RouterOS doesn't know the data is P2P until AFTER the
connection is made. This is important because the only way to send data out a specific internet connection you must
know that traffic is BEFORE the connection is made. So as with P2P traffic, unknown traffic is just that, it is
unknown. By marking the unknown traffic though you can control what internet connection is used for both P2P and
the left over unknown traffic (very useful!)
Step 2 - Setting Up the Network
Before we get to the real point of this tutorial we need a fictious network we can use as an example.
• client computers (172.18.1.0/24)
• Internet Gateways (10.0.1.1/24, 10 0.2.1/24)
• RouterOS IPs (10.0.1.2/24, 10.0.2.2/24)
Asuming that the IPs, default routes, and DNS settings are already in place the following allow users to get internet
access.
3. Per-Traffic Load Balancing 3
Create Address-List for permitted use of internet
/ ip firewall address-list
add list="Allowed-Internet" address=172.18.1.0/24 comment="" disabled=no
Create Address-List to bypass load-balancing
/ ip firewall address-list
add list="WAN-01" address=172.18.1.24/32 comment="" disabled=no
add list="WAN-02" address=172.18.1.76/32 comment="" disabled=no
Apply 'Masquerading'to the traffic leaving the WAN interfaces
/ ip firewall nat
add chain=srcnat action=masquerade out-interface="WAN - 01" src-address-list="Allowed-Internet" comment="Gateway 10.0.1.1/24" disabled=no
add chain=srcnat action=masquerade out-interface="WAN - 02" src-address-list="Allowed-Internet" comment="Gateway 10.0.2.1/24" disabled=no
Clients should be able to browse the internet, however only one internet connection would be used (the current
default route in RouterOS)
Step 3 - Using RouterOS's Mangle Tool to mark specific traffic
The following is the necessary RouterOS commands to mark specific traffic for a specific route. We will use the
same example traffic types as in the beginning,i.e. HTTP, SSL, POP3, SMTP, P2P, and Unknown
/ ip firewall mangle
add chain=prerouting action=mark-routing new-routing-mark="WAN-01" src-address-list="WAN-01" passthrough=no comment="" disabled=no
add chain=prerouting action=mark-routing new-routing-mark="WAN-02" src-address-list="WAN-02" passthrough=no comment="" disabled=no
add chain=prerouting action=mark-routing new-routing-mark="HTTP traffic" passthrough=no dst-port=80 protocol=tcp comment="" disabled=no
add chain=prerouting action=mark-routing new-routing-mark="SSL traffic" passthrough=no dst-port=443 protocol=tcp comment="" disabled=no
add chain=prerouting action=mark-routing new-routing-mark="POP3 traffic" passthrough=no dst-port=110 protocol=tcp comment="" disabled=no
add chain=prerouting action=mark-routing new-routing-mark="SMTP traffic" passthrough=no dst-port=25 protocol=tcp comment="" disabled=no
add chain=prerouting action=mark-routing new-routing-mark="P2P traffic" passthrough=no p2p=all-p2p comment="" disabled=no
add chain=prerouting action=mark-routing new-routing-mark="Unknown traffic" passthrough=no comment="" disabled=no
The first two lines provide a method of marking certain clients to bypass the load-balancing and use only a specific
internet connection for all of their traffic.
The next lines then will mark traffic based on the dst-port, notice how we are not passing through, also notice that we
are marking all traffic, even if not known and that there is always a different internet connection to be specified for
the Unknown and P2P instead of using the router's default route. I also mark P2P separate, even though it must go
out the same internet connection as Unknown traffic.
I do this for a couple of reasons, one is that I could easily stop all traffic by simply disabling a route, and the other is
in my RouterOS configuration I use a lot of QoS, it's very easy to remember how everything is configured if the QoS
mirrors the load-balancing.
So now that we are marking traffic for their respective routes, we next have to add those actual routes.
4. Per-Traffic Load Balancing 4
Step 4 - Using the routing functions of RouterOS to force traffic out to certain
internet connections
This following is the necessary RouterOS commands to provide routes for the marked HTTP, SSL, POP3, SMTP,
P2P, and Unknown Traffic
/ ip route
add dst-address=0.0.0.0/0 gateway=10.0.1.1 scope=255 target-scope=10 routing-mark="WAN - 01" comment="" disabled=no
add dst-address=0.0.0.0/0 gateway=10.0.2.1 scope=255 target-scope=10 routing-mark="WAN - 02" comment="" disabled=no
add dst-address=0.0.0.0/0 gateway=10.0.2.1 scope=255 target-scope=10 routing-mark="HTTP traffic" comment="" disabled=no
add dst-address=0.0.0.0/0 gateway=10.0.2.1 scope=255 target-scope=10 routing-mark="SSL traffic" comment="" disabled=no
add dst-address=0.0.0.0/0 gateway=10.0.2.1 scope=255 target-scope=10 routing-mark="POP3 traffic" comment="" disabled=no
add dst-address=0.0.0.0/0 gateway=10.0.1.1 scope=255 target-scope=10 routing-mark="SMTP traffic" comment="" disabled=no
add dst-address=0.0.0.0/0 gateway=10.0.1.1 scope=255 target-scope=10 routing-mark="P2P traffic" comment="" disabled=no
add dst-address=0.0.0.0/0 gateway=10.0.1.1 scope=255 target-scope=10 routing-mark="Unknown traffic" comment="" disabled=no
The first two lines provide the routes necessary to give the clients that are not being load-balanced via traffic type the
correct gateway to the internet. The remaining lines are the routes necessary to provide the appropiate gateway based
on traffic type
Step 5 - Review what you've created
What did you do: * You intelligently broke up your internet traffic into different types * Your marked that traffic
using the Mangle Tool of RouterOS * You created a bypass list to allow certain IP to bypass the Per-Traffic
Load-Balancing * You assigned particular routes based on the traffic you marked with Mangle
Conclusion
What you have done is very powerful and this tutorial provides just the tip of the iceberg so to say. The traffic types I
have listed here are only a small amout of the total and you may want to add many others (ie. DNS, Terminal
Services, ICMP, etc). This can be used is a great deal of different ways be it in a small offfice enviroment (separating
your email usage from your browsing) or a large wireless ISP (for load-balancing). Some tips and final thougths
1. Think though what you want to achieve before you start 2. Pay attention to the traffic of type on each line (is it
incoming intensive or outgoing intensive) 3. Finally experiment, nothing better then finding better, faster, and more
intelligent ways to improve your services with a little capital costs
