双线路问题

开始

还是去年的这个时候，李硕问我在FreeBSD上有没有网吧（电信、网通）双线路的解决办法，共同研究。我说有，但是上网一找却没找到。后来李硕再没提此事，就忘了。

后来，韩冬媳妇问我会不会做网吧双线，说可以用来联系电信的业务。我说可以试试看。再后来，结婚，去哈尔滨学习水手业务，从哈尔滨回来就回家过年，然后找工作，上班。这件事一直没有结果。

直到上个月，给吉林教育杂志社做网络共享连接，一时兴起用 m0n0Wall 做了，又想起这件事来。

离开方正维修站后。暂时没有好的想法和打算，于是趁机研究了一下。

实践

起初，在网上找相关的文章，大多是用专用路由器解决。除了那些专用设备以外我能接触上的只有：Route OS 、 m0n0Wall 、Iptables 、ipfw、ipf 和 pf 。我一个一个的试。其实这些东西都是我所不了解的，下载找资料安装测试就花去了很多时间。后来我决定从 pf 开始着手，因为这个我手上的资料最多，FreeBSD 是我相对来说最熟悉的。

以前用过 ipfw 、 ipfilter 之后就没用过 pf ，主要是和他们老是弄混。一个功能的实现方式不一样，所以学着很别扭。用过了以后才知道 pf 真的是很好的一个防火墙系统，我很喜欢。

我给自己搭建了以下的环境：


pc1 ： ---------------------| | pc2 : --------------------------------- || modem：

FreeBSD 4.11 + ipfilter ----| | FreeBSD 6.2 + pf

rl0 192.168.1.254 ---------|-|----------------------------------------> (to modem)

rl1 172.16.11.1/24 <-------|-|-> fxp0 172.16.11.2/24 (default 172.16.11.1)

rl2 172.17.12.1/24 <------- |-|-> fxp1 172.16.12.2/24


pc3 :Windowx 2003 -------|
192.168.0.2/24 -default- <-|---> rl0 192.168.0.1/24
Windows 2003

pc1 设置：

自动 PPPoE 拨号
ipfilter 共享上网，一切从简。
pass in quick all
pass out quick all
map tun0 172.16.0.0/16 -> 0/32 portmap tcp/udp 10000:40000
map tun0 172.16.0.0/16 -> 0/32

中间忘记了启用ip转发，上不了网，浪费了一些时间，晕一下。

pc2 设置：

1.表

table 《表名》 persist file "/etc/telecom_ip"

2.规则
ext_if1 = "fxp0"
ext_gw2 = "172.16.11.1"
ext_if2 = "fxp1"
ext_gw2 = "172.16.12.1"
int_if = "rl0:
int_net = "192.168.0.0/24"

table perstst files "/etc/telecom_ip"

下面是手册中的说明，我按照他给出的例子做了修改。

#下面的例子通过2条到因特网的连接平衡输出流量：
#pass in on $int_if route-to {($ext_if1 $ext_gw2), ($ext_if2 $ext_gw2)} round-robin from $int_net to any keep state

# route-to 选项用来在收到流量的内部接口上指定平衡的流量经过各自的网关到输出的网络接口。
# 注意route-to 选项必须在每个需要均衡的过滤规则上出现。返回的数据包会路由到它们出去时的外部接口（这是由ISP做的），然后正常路由回内部网络。
# 要保证带有属于$ext_if1源地址的数据包总是路由到$ext_gw1（$ext_if2 和 $ext_gw2也是同样的），下面2行必须包括在规则集中：
pass out on $ext_if1 route-to ($ext_if2 $ext_gw2) from $ext_if2 to any
pass out on $ext_if2 route-to ($ext_if2 $ext_gw2) from $ext_if1 to any

# 最后，NAT也可以使用在输出接口中：
nat on $ext_if2 from $int_net to -> $ext_if2
nat on $ext_if1 from $int_net to any -> $ext_if1

实验到晚上，我发现，问题的关键在于静态路由表：不添加静态路由表的话，走另外一条线路的访问不会回来。但是做负载均衡的可以。目前来看，只要加了静态路由表，做一下两个方向的NAT 规则就可已各走各的路。不加静态路由表，怎么做都没有用。明天加一台机器再测试。

下面是第二天（12:56 2007-6-6）做的实验。

以下的这个规则是从手册上修改过来的，他的确可以实现负载均衡。

# cat pf.2
lan_net = "192.168.0.0/24"
int_if = "rl0"
ext_if1 = "fxp0"
ext_if2 = "fxp1"
ext_gw1 = "172.16.11.1"
ext_gw2 = "172.16.12.1"

table persist file "/etc/telecom_ip"


# nat outgoing connections on each internet interface
nat on $ext_if1 from $lan_net to any -> ($ext_if1)
nat on $ext_if2 from $lan_net to any -> ($ext_if2)

# default deny
block in from any to any
block out from any to any

# pass all outgoing packets on internal interface
pass out on $int_if from any to $lan_net

# pass in quick any packets destined for the gateway itself
pass in quick on $int_if from $lan_net to $int_if

# load balance outgoing tcp traffic from internal network.
pass in on $int_if route-to \
{ ($ext_if1 $ext_gw1), ($ext_if2 $ext_gw2) } round-robin \
proto tcp from $lan_net to any flags S/SA modulate state

# load balance outgoing udp and icmp traffic from internal network
pass in on $int_if route-to \
{ ($ext_if1 $ext_gw1), ($ext_if2 $ext_gw2) } round-robin \
proto { udp, icmp } from $lan_net to any keep state

# general "pass out" rules for external interfaces
pass out on $ext_if1 proto tcp from any to any flags S/SA modulate state
pass out on $ext_if1 proto { udp, icmp } from any to any keep state
pass out on $ext_if2 proto tcp from any to any flags S/SA modulate state
pass out on $ext_if2 proto { udp, icmp } from any to any keep state

# route packets from any IPs on $ext_if1 to $ext_gw1 and the same for
# $ext_if2 and $ext_gw2
pass out on $ext_if1 route-to ($ext_if2 $ext_gw2) from $ext_if2 to any
pass out on $ext_if2 route-to ($ext_if1 $ext_gw1) from $ext_if1 to any

既然双线路出口都有数据通过，似乎加不加默认网关已经没什么作用了。于是我删除了默认网关。没错，在pc2 上已经不能 nslookup 了，但是在 2003 上可以，同样可以下载文件，同样是负载均衡状态，两条出口线路都有数据经过。

最后我发现：问题出现在问题出在局域网入口上：

# load balance outgoing tcp traffic from internal network.
pass in on $int_if route-to \
{ ($ext_if1 $ext_gw1), ($ext_if2 $ext_gw2) } round-robin \
proto tcp from $lan_net to any flags S/SA modulate state

# load balance outgoing udp and icmp traffic from internal network
pass in on $int_if route-to \
{ ($ext_if1 $ext_gw1), ($ext_if2 $ext_gw2) } round-robin \
proto { udp, icmp } from $lan_net to any keep state

这两条就是让从连接局域网的网卡上进入的数据平均分摊到两条出口上。所以我做了如下修改：

pass in quick on $int_if route-to ($ext_if2 $ext_gw2) proto tcp from $lan_net to 《表名》 flags S/SA modulate state

pass in quick on $int_if route-to ($ext_if1 $ext_gw1) proto tcp from $lan_net to any flags S/SA modulate state

现在似乎是成功了。
下面是我最终的修改结果，让 tcp 和 icmp 分线走，udp 到 dns 的走默认线路。其实我想可以在局域网网关机器上添加默认路由并做 DNS 转发，这样就可以都分开了。

# cat pf.2
lan_net = "192.168.0.0/24"
int_if = "rl0"
ext_if1 = "fxp0"
ext_if2 = "fxp1"
ext_gw1 = "172.16.11.1"
ext_gw2 = "172.16.12.1"
table persist file "/etc/telecom_ip"

# nat outgoing connections on each internet interface
### nat on $ext_if1 from $lan_net to any -> ($ext_if1)
### nat on $ext_if2 from $lan_net to any -> ($ext_if2)

nat on $ext_if2 from $lan_net to -> ($ext_if2)
nat on $ext_if1 from $lan_net to any -> ($ext_if1)

# default deny
block in from any to any
block out from any to any

# pass all outgoing packets on internal interface
pass out on $int_if from any to $lan_net

# pass in quick any packets destined for the gateway itself
pass in quick on $int_if from $lan_net to $int_if

# load balance outgoing tcp traffic from internal network.
#### pass in on $int_if route-to \
{ ($ext_if1 $ext_gw1), ($ext_if2 $ext_gw2) } round-robin \
proto tcp from $lan_net to any flags S/SA modulate state
############

pass in quick on $int_if route-to ($ext_if2 $ext_gw2) \
proto tcp from $lan_net to flags S/SA modulate state

pass in quick on $int_if route-to ($ext_if1 $ext_gw1) \
proto tcp from $lan_net to any flags S/SA modulate state

# load balance outgoing udp and icmp traffic from internal network
#### pass in on $int_if route-to \
{ ($ext_if1 $ext_gw1), ($ext_if2 $ext_gw2) } round-robin \
proto { udp, icmp } from $lan_net to any keep state
############

pass in on $int_if route-to ($ext_if1 $ext_gw1) \
proto udp from $lan_net to any port 53 keep state

pass in quick on $int_if route-to ($ext_if2 $ext_gw2) \
proto icmp from $lan_net to keep state

pass in quick on $int_if route-to ($ext_if1 $ext_gw1) \
proto icmp from $lan_net to any keep state


# general "pass out" rules for external interfaces
pass out on $ext_if1 proto tcp from any to any flags S/SA modulate state
pass out on $ext_if1 proto { udp, icmp } from any to any keep state
pass out on $ext_if2 proto tcp from any to any flags S/SA modulate state
pass out on $ext_if2 proto { udp, icmp } from any to any keep state

# route packets from any IPs on $ext_if1 to $ext_gw1 and the same for
# $ext_if2 and $ext_gw2
pass out on $ext_if1 route-to ($ext_if2 $ext_gw2) from $ext_if2 to 《表名》
pass out on $ext_if2 route-to ($ext_if1 $ext_gw1) from $ext_if1 to any

昨天晚上还看了一篇在 Route OS 上做双线的文章，它的优势是可以检测线路状态，当一条线路断掉了，就把双线并成一条线路来走。这个我想也可以写成一个脚本来实现。但是今天没时间了。

发表在这里的文字中的尖括号会被过滤，所以请以理解的方式阅读。Sorry。

Policy Routing

Policy Routing

Policy routing is the process of forcing packets to take a certain route, often different from the default route, based on certain packet attributes (source, type of packets, interface, etc.).

The need for policy routing arises for multi-homed hosts (hosts having two network connections to different ISPs), that cannot use a single default route. Having a central Internet connection for multiple private networks can require policy routing. It is most useful in case you have multiple networks connected behind a FreeBSD host with more than one connection to the Internet, when each network needs to use a different outgoing route.

Also, policy routing should be employed when you need to separate traffic by certain criteria, such as source, destination or protocol

When Not to Use Policy Routing
Policy routing can be extremely useful, but it is not always needed. A frequent case of misuse is when configuring a tunnel as a default route. Instead of using the first gateway as the default, then injecting all the other traffic in the tunnel, use more specific routes. For example, if the tunnel ends at 10.10.10.10, the ip of the other end inside the tunnel is 1.2.3.4, and the gateway your provider gave you (and don't actually want to use) is at 2.2.2.2, you can use these routes in /etc/rc.conf:

static_routes=“tunnel_end”
route_tunnel_end=“-host 10.10.10.10 2.2.2.2”
defaultrouter=“1.2.3.4”Since policy routing does involve a certain overhead on the system, do think of a different solution first. Specific routes can be very useful, but policy routing is the only solution for certain cases.

Configuring Policy Routing
The easiest way of doing policy routing with FreeBSD is by using IPFilter. It comes by default with all recent versions of FreeBSD, and it can be easily enabled by adding this line to /etc/rc.conf:

ipfilter_enable=“YES”Be advised that the IP addresses used in this article are fictional, and I have mixed public IP addresses with private ones (see RFC1918). You might find those in private networks, but they will almost never be assigned to you by an ISP.

Example 1 - Multi-homed Host

Suppose you have a host that is connected to two networks, via two interfaces: fxp0, with an IP address of 192.168.100.100, and fxp1, with an IP address of 192.168.200.200, both with the default classful netmask of 255.255.255.0. On both those networks, the gateways are located at 192.168.*.1. I'll assume that the default route is out fxp0.


Now suppose the host receives a ping (an ICMP echo request packet) from host 1.1.1.1, to its IP address 192.168.200.200. The reply will be generated with a source address of 192.168.200.200, and sent to 1.1.1.1. But if the host has no specific route to 1.1.1.1, it will use the default route out fxp0, and try to send the reply via the gateway at 192.168.100.1. If a stateful firewall is in place on the gateway, the packet will most probably be dropped, as the request came via the other gateway (in the 200 network). Also, if egress filtering (filtering of outbound traffic) is configured on the 100 network gateway, the reply will be blocked because it is coming from a different network (and possibly logged as a spoofing attempt). This will also affect other types of packets, like TCP and UDP.

The solution for this problem is rather simple: use policy routing to force packets from the 192.168.200.200 address to leave via the network they belong to. This is line in /etc/ipf.rules will do just as I have described:

pass out quick on fxp0 to fxp1:192.168.200.1 from 192.168.200.200 to any This rule says “force packets from my fxp1 address to exit via the gateway on fxp1 (not my default gateway on fxp0)”.

Example 2 - Asymmetric Traffic

This is similar to Example 1, but with a different purpose. Let's assume that one of the links, fxp0, is an ADSL link with a static IP address. The fxp1 link is a cable modem, with a dynamic IP. You want to have a web server, so you put it on the static IP. But the cable link has better bandwidth, so you want to take advantage of that.

Web traffic in inherently asymmetric, with a small request from the client to the server, and a larger reply from the server to the client. So receiving the request on the ADSL link will not hurt, and it has the added benefit of having a static IP address. So you decide to send the replies via the bigger bandwidth link.

What you need to do is allow requests to the web server on your static IP address to come in via the ADSL link, but policy route the reply, diverting it to the cable link. Here is a configuration that does that:

pass in quick on fxp0 proto tcp from any to 192.168.100.100
pass out quick on fxp0 to fxp1:192.168.200.1 proto tcp from 192.168.100.100 port = 80 to any The gateway on fxp1 (192.168.200.1 in this example) is assigned by DHCP, and you can find it out by running this command:

# grep routers /var/db/dhclient.leasesThis will not work if your ISP on the cable link implements egress filtering. For a web request to complete successfully, the reply must come from the host the request went to. Since you will be sending the reply via another link (possibly belonging to a different ISP than the one who owns the fixed IP address), the cable ISP might choose not to forward your packets, and make this setup inoperable.

Example 3 - Routing for Multiple Networks

Another scenario is a network that has access to two different ISPs, and has assigned addresses from each of those. The interfaces connecting to those ISPs are fxp0 at 10.0.0.2, using the gateway at 10.0.0.1, and fxp1 at 20.0.0.2, using the gateway at 20.0.0.1. The networks assigned by those two ISPs are 192.168.100.0/24, and 192.168.200.0/24. Traffic originating from the 100 network belongs to the 10.0.0.1 ISP, and the other network to the 20.0.0.1 ISP. The interfaces connecting those two nets are de0 and de1.


This needs a simple configuration in /etc/ipf.rules. But we do have to take care and not shoot packets out to the ISP that are intended for the FreeBSD machine (which we'll suppose is at .254 on both those networks).

# accept packets to us, don't forward them
pass in quick on de0 from 192.168.100.0/24 to 192.168.100.254
pass in quick on de1 from 192.168.200.0/24 to 192.168.200.254

# other packets should be handled by their respective ISP
# ISP1
pass in quick on de0 to fxp0:10.0.0.1 from 192.168.100.0/24 to any
# ISP2
pass in quick on de1 to fxp1:20.0.0.1 from 192.168.200.0/24 to anyAdditional Resources
Egress filtering - Keeping the Internet Safe from Your Systems
Anatomy of a Stateful Firewall
- Alex

离开方正

前天，我选择了离开方正。

五一期间

　　五一期间，放假七天，中间有两天上午去值班。其余的时间被我一次性挥霍了，包括去德惠给冯海东修一次电脑，给刘小祎装系统一次。玩游戏到后半夜两次。弄得我体力透支，用两个晚上的狂睡才得以恢复。而五一之前策划将在五一期间完成的事项和计划几乎没一个完成。我还是我。

　　明天就要上班了，高波来电话说要明天去，我也答应他了。只是之后的工作不知道怎样去做。我没有一点想法。什么也没有。

2006-08-19 | 无悔

原来写在我的 Sohu 博客里了。

昨天跟妈妈通了电话，听妈妈的意思，似乎没有办法支援我更多的钱了。

我想，这些都是我的事情，是不应该让妈妈来操心的。我告诉妈妈，可以和亲戚们说一声，因为她们以前都答应过帮助我的，不说反而不好。至于能不能借给我和能借给我多少的事就不用管了。反正，以目前的情况，也是可以结婚的。千万不要底三下四厚着脸皮去如何如何...。

然后，带着我的女朋友去重庆路 -- 我和海伦约好了吃烧烤，海伦要认识一下我的女朋友。

回来的路上，我想象着这会是怎样的一个婚礼呢？婚后在空旷如野的屋子里的每一天又是怎样的一种生活呢？看着我身边非常开心的女朋友，她想象的婚礼是个什么样子的呢?她想象的婚后的生活是什么样子的呢？到时候会不会后悔选择我这样的一个废物呢？我开始有点后悔如此草率的和女朋友这么早谈论结婚的事情。听她闲聊了一会后，我小心翼翼的说：我们还没有登记，离婚礼举办的时间还有一个月，你可以仔细的考虑一下我们的事 …。

我女朋友开始用一种奇怪的眼神看着我，接着，她对我说了她的想法,最后,她哭了 ……

看来，她中毒已深，已经听不进别人的逆耳忠言了。我们的事，也已经如箭离弦无法回头了。我不能带给她美好的现在，希望我能够带给她她想要的未来。

如果你不后悔，我就娶你。