λ Tony's blog λ
The weblog of Tony Morris

MSN/Yahoo! web spiders blocked

Posted on January 24, 2010, in General

I have blocked the msnbot and yahoo! web spiders from indexing this site (and all others at this IP address). They choke up the network with distributed multiple connections. Pretty silly/primitive technique of indexing a web site.