CloudFlare recently posted a blog post about how they do load balancing inside the data center.
It makes sense to use BGP for server fail-over, since you can configure a server to stop publishing it’s routes when it dies. But I fail to see how it’s effective at load balancing. Let’s say you have each external IP/range associated with a set of servers. Since your primary router is going to choose the “shortest path”, that means that all traffic to 184.108.40.206 will always go to Server A, until another server has a higher weight/priority or if Server A stops publishing its route. If 220.127.116.11 starts to get a lot of traffic that particular server will get hammered with traffic and it won’t be “load balanced”. When this happens there are a few things that could happen. Server A could realize that it has a large load and lower it’s weight, which would cause another server to start getting ALL the traffic for 18.104.22.168. If Server B could realize that Server A is getting hammered it could raise it’s weight because it has no traffic, but once again, it will start to receive ALL traffic for 22.214.171.124. Finally, if the DNS server knew that Server A is getting hammered at 126.96.36.199 and specifically, domain xyz.com, it could change the address for xyz.com to 188.8.131.52, effectively moving the traffic to a better server. If that traffic is too insane to be handled by any single server/endpoint, then you’re out of luck.
However, there are possible solutions. You could use BGP to point IPs at different hardware/software load balancers to distribute the load further for an IP, which seems like the most desirable solution. Or you could rely on DNS round-robin, which can be used, but notably isn’t as effective, on domain-based setups. See updates below for other solutions brought to my attention.
If you have a lot of domains/IPs and you’re running a proxy service like CloudFlare, then you might be able to pull off internal BGP-only balancing. But for the rest of the Internet, it doesn’t make a lot of sense. I’d suggest using load balancers, which are easier to setup, anyways.
(Disclosure: I have been using CloudFlare for over a year now.)
Update: It’s been brought up that you could use ECMP but there can be issues with connection-based protocols, like TCP, if you’re not using flows. If you’re just serving UDP content, then that would be a good solution for you.