Couple of days ago, I had really interesting discussion about using BGP communities to influent inbound traffic by modifying ISP’s LOCAL_PREF on advertised prefixes in multi homed environments. There are many Internet Service Providers that support this, including, for example, Level3 (as visible on this link).
Taking in consideration that BGP communities are basically prefix tags, one can ask how can this be done. I was wondering about it as well, so I have found out that this particular behavior is described in RFC 1998, and it needs to be supported by your ISP. Here is the example:
AS 100 is under our administrative control, and AS 200 is our ISP. Addressing is as follows:
R1:
FastEthernet0/0 10.10.10.1/24 (100Mbps)
FastEthernet0/1 20.20.20.1/24 (10Mbps)
Loopback77 1.1.1.1/24
Loopback10 150.0.10.1/24
Loopback20 150.0.20.1/24
Loopback30 150.0.30.1/24
R2:
FastEthernet0/0 10.10.10.2/24 (100 Mbps)
FastEthernet0/1 192.168.1.1/24 (100 Mbps)
Loopback77 2.2.2.2/24
R3
FastEthernet0/0 20.20.20.2/24 (10Mbps)
FastEthernet0/1 192.168.1.2/24 (100 Mbps)
Loopback77 3.3.3.3/24
Please note that in this particular example, I am using static routing to ensure network reachability between above listed loopbacks (loopbacks that are used for update-source, in this particular example loopback77).
Once we have network reachability working as expected, we need to configure our eBGP peers. In order to configure eBGP peers on R1 with R2 and R3, we need to execute following:
R1(config)# router bgp 100
R1(config-router)#neighbor 2.2.2.2 remote-as 200
R1(config-router)#neighbor 2.2.2.2 update-source loopback77
R1(config-router)#neighbor 2.2.2.2 ebgp-multihop
R1(config-router)#neighbor 3.3.3.3 remote-as 200
R1(config-router)#neighbor 3.3.3.3 update-source loopback77
R1(config-router)#neighbor 3.3.3.3 ebgp-multihop
Taking in consideration that we are going to use BGP communities new-format, we need to configure following:
R1(config)#ip bgp-community new-format
R1(config)#router bgp 100
R1(config-router)#neighbor 2.2.2.2 send-community
R1(config-router)#neighbor 3.3.3.3 send-community
On provider side, in order to establish eBGP session from R2 to R1 we need to execute following:
R2(config)#router bgp 200
R2(config-router)#neighbor 1.1.1.1 remote-as 100
R2(config-router)#neighbor 1.1.1.1 update-source loopback77
R2(config-router)#neighbor 1.1.1.1 ebgp-multihop
Additionally, R2 is connected to R3 using their Fa0/1 interfaces within the same AS 200, therefore, ISP needs to configure iBGP, as follows:
R2(config-router)#neighbor 192.168.1.2 remote-as 200
R2(config-router)#neighbor 192.168.1.2 next-hop-self
Additionally, we will be using BGP Communities new-format, so we need to configure following:
R2(config)#ip bgp-community new-format
R2(config)#router bgp 200
R2(config-router)#neighbor 1.1.1.1 send-community
R2(config-router)#neighbor 192.168.1.2 send-community
On provider side, there is R3 that needs to be configured. In order to configure eBGP peering with R1, we need to do following:
R3(config)#router bgp 200
R3(config-router)#neighbor 1.1.1.1 remote-as 100
R3(config-router)#neighbor 1.1.1.1 update-source loopback77
R3(config-router)#neighbor 1.1.1.1 ebgp-multihop
Also, R3 FastEthernet 0/1 is connected to R2 FastEthernet 0/1 so we need to configure iBGP as follows:
R3(config-router)#neighbor 192.168.1.1 remote-as 200
R3(config-router)#neighbor 192.168.1.1 next-hop-self
and we need to apply BGP communities new-format related configuration:
R3(config)#ip bgp-community new-format
R3(config)#router bgp 200
R3(config-router)#neighbor 1.1.1.1 send-community
R3(config-router)#neighbor 192.168.1.1 send-community
Last step that we need to preform is to verify that our peers are up and running as expected:
R1#show ip bgp summary | begin Neig
Neighbor V AS MsgRcvd MsgSent TblVer InQ OutQ Up/Down State/PfxRcd
2.2.2.2 4 200 156 158 4 0 0 00:41:15 0
3.3.3.3 4 200 143 144 4 0 0 00:41:15 0
R1#
R2#show ip bgp summary | begin Neig
Neighbor V AS MsgRcvd MsgSent TblVer InQ OutQ Up/Down State/PfxRcd
1.1.1.1 4 100 158 156 20 0 0 00:41:23 0
192.168.1.2 4 200 128 127 20 0 0 01:57:31 0
R2#
R3#show ip bgp summary | begin Neig
Neighbor V AS MsgRcvd MsgSent TblVer InQ OutQ Up/Down State/PfxRcd
1.1.1.1 4 100 144 143 23 0 0 00:41:26 0
192.168.1.1 4 200 127 128 23 0 0 01:57:35 0
R3#
As visible above, everything seems to be working fine. We have peering between R1 that is under our administrative control and ISP routers R2 and R3. ISP routers have iBGP peering over their FastEthernet 0/1 interfaces.
Now that we have everything working as expected, we will advertise directly connected networks from R1 (in our particular example lo10, lo20 and lo30) by using redistribute connected BGP command.
We will create prefix-list first, and route-map that matches that prefix-list in order to control what are we advertising:
R1(config)#ip prefix-list REDIST permit 150.0.10.0/24
R1(config)#ip prefix-list REDIST permit 150.0.20.0/24
R1(config)#ip prefix-list REDIST permit 150.0.30.0/24
R1(config)#route-map BGP-REDIST permit 10
1(config-route-map)#match ip address prefix-list REDIST
and then we need to redistribute it as follows:
R1(config-router)#redistribute connected route-map BGP-REDIST
We can verify results by checking R2/R3 BGP table:
R2#show ip bgp | begin Net
Network Next Hop Metric LocPrf Weight Path
* i150.0.10.0/24 192.168.1.2 0 100 0 100 ?
*> 1.1.1.1 0 0 100 ?
* i150.0.20.0/24 192.168.1.2 0 100 0 100 ?
*> 1.1.1.1 0 0 100 ?
* i150.0.30.0/24 192.168.1.2 0 100 0 100 ?
*> 1.1.1.1 0 0 100 ?
R2#
R3#show ip bgp | begin Net
Network Next Hop Metric LocPrf Weight Path
* i150.0.10.0/24 192.168.1.1 0 100 0 100 ?
*> 1.1.1.1 0 0 100 ?
* i150.0.20.0/24 192.168.1.1 0 100 0 100 ?
*> 1.1.1.1 0 0 100 ?
* i150.0.30.0/24 192.168.1.1 0 100 0 100 ?
*> 1.1.1.1 0 0 100 ?
R3#
So, we have expected BGP results. Now, taking in consideration that our link between R1 FastEthernet 0/0 and R2 FastEthernet 0/0 is 100bps we would like to prefer that link for inbound routing, as it is better than the other available R1 FastEthernet 0/1 R3 FastEthernet 0/0 link which is 10 mbps.
Usual way to achieve this would be to use MED or AS Prepanding, but if supported by your ISP you could possibly use BGP communities.
In case that your provider supports that, first thing that we would need to do is to set communities:
R1(config)#route-map SET-BGP-COMMUNITY-200
R1(config-route-map)#set community 200:200
R1(config-route-map)#exit
R1(config)#route-map SET-BGP-COMMUNITY-150
R1(config-route-map)#set community 200:150
As visible from configuration above, we have two different route-maps. First one is SET-BGP-COMMUNITY-200 and it is setting community to 200:200. Notice that 200 is our ISP ASN and it is followed by :200 which should, in our particular example, set our ISP’s LOCAL_PREF to 200 on our advertised routes. By doing that, we are directly influencing our inbound traffic. Following the same logic, route map SET-BGP-COMMUNITY-150 is going to set community to 200:150, and it should influence ISP’s LOCAL_PREF to be 150.
Once we configure those route-maps we need to apply them. We will apply SET-BGP-COMMUNITY-200 to our eBGP peer with R2 as it is 100Mbps link. SET-BGP-COMMUNITY-150 will be applied to our BGP peer with R3:
R1(config)#router bgp 100
R1(config-router)#neighbor 2.2.2.2 route-map SET-BGP-COMMUNITY-200 out
R1(config-router)#neighbor 3.3.3.3 route-map SET-BGP-COMMUNITY-150 out
In order to verify our results we need to check BGP table on our ISP routers, R2 and R3:
R2#show ip bgp | begin Net
Network Next Hop Metric LocPrf Weight Path
*> 150.0.10.0/24 1.1.1.1 0 200 0 100 ?
*> 150.0.20.0/24 1.1.1.1 0 200 0 100 ?
*> 150.0.30.0/24 1.1.1.1 0 200 0 100 ?
R2#
As visible from R2 output, LOCAL_PREF is set to 200 and all of the prefixes that are learned from R1 on 100Mbps link. Therefore, this means that our setup is working as expected. Just to make sure, lets verify one of the routes and it’s set community:
R2#show ip bgp 150.0.10.0
BGP routing table entry for 150.0.10.0/24, version 33
Paths: (1 available, best #1, table Default-IP-Routing-Table)
Advertised to update-groups:
2
100
1.1.1.1 from 1.1.1.1 (150.0.30.1)
Origin incomplete, metric 0, localpref 200, valid, external, best
Community: 200:200
R2#
As visible from the output above, community is set to 200:200. Therefore, our configuration is working as expected.
Far more interesting output should be visible on R3:
R3#show ip bgp | begin Net
Network Next Hop Metric LocPrf Weight Path
*>i150.0.10.0/24 192.168.1.1 0 200 0 100 ?
* 1.1.1.1 0 150 0 100 ?
*>i150.0.20.0/24 192.168.1.1 0 200 0 100 ?
* 1.1.1.1 0 150 0 100 ?
*>i150.0.30.0/24 192.168.1.1 0 200 0 100 ?
* 1.1.1.1 0 150 0 100 ?
R3#
As expected, we can see that all of the prefixes have two possible paths, but in order to reach prefixes advertised by R1 traffic will go over R2 (iBGP peer) because LOCAL_PREF is better than from routes learned from R1 (1.1.1.1).
As visible from the above output, we successfully influenced inbound routing.
But what needs to be configured on our ISP routers in order to support this?
On both ISP routers we need to configure community lists that are going to match communities that are sent by R1:
R2(config)#ip community-list 20 permit 200:150
R2(config)#ip community-list 10 permit 200:200
R3(config)#ip community-list 20 permit 200:150
R3(config)#ip community-list 10 permit 200:200
Once that is done, ISP needs to create route-map that is expecting above community lists, and then based on the match it sets required LOCAL_PREF:
R2(config)#route-map BGP-COMMUNITY-LOCAL-PREF permit 10
R2(config-route-map)# match community 10
R2(config-route-map)# set local-preference 200
R2(config-route-map)#route-map BGP-COMMUNITY-LOCAL-PREF permit 20
R2(config-route-map)# match community 20
R2(config-route-map)# set local-preference 150
R2(config-route-map)# route-map BGP-COMMUNITY-LOCAL-PREF permit 30
R3(config)#route-map BGP-COMMUNITY-LOCAL-PREF permit 10
R3(config-route-map)# match community 10
R3(config-route-map)# set local-preference 200
R3(config-route-map)#route-map BGP-COMMUNITY-LOCAL-PREF permit 20
R3(config-route-map)# match community 20
R3(config-route-map)# set local-preference 150
R3(config-route-map)# route-map BGP-COMMUNITY-LOCAL-PREF permit 30
and last step that ISP needs to do is to apply that on all BGP peers, for example:
R2(config)#router bgp 200
R2(config-router)#neighbor 1.1.1.1 route-map BGP-COMMUNITY-LOCAL-PREF in
R3(config)#router bgp 200
R3(config-router)#neighbor 1.1.1.1 route-map BGP-COMMUNITY-LOCAL-PREF in
And that is all. Everything is working as expected.
In case that you have some questions don’t hesitate to leave comments.
- RFC1998