前几天我的 bluehost 账号总有 CPU 使用超量的问题，今天甚至因此被暂时停掉。察看了下访问记录，发现有个叫 twiceler 的 robot 很奇怪，每秒种访问同一个页面多次（例子附在帖子最后，懂行的人帮我看看）。猜它就是问题的来源，不过并不确定。似乎有人在做一个叫 Cuill 的新搜索引擎，放出这个叫 Twiceler 的机器人。我现在只好阻止它了。在 robots.txt 里加上

User-agent: twiceler
Disallow: /

略微学习一下这个 Cuill，发现还被人认为是 Google 的有力挑战者呢。他们自己声称 Cuill 检索网页的速度比 Google 快，成本低十倍。

……
38.117.64.101 – - [05/Oct/2007:21:59:38 -0600] “GET /111/ HTTP/1.0″ 302 0 “http://vonye.com/111″ “Mozilla/5.0 (Twiceler-0.9 http://www.cuill.com/twiceler/robot.html)”
38.117.64.101 – - [05/Oct/2007:21:59:39 -0600] “GET /111/ HTTP/1.0″ 302 0 “http://vonye.com/111″ “Mozilla/5.0 (Twiceler-0.9 http://www.cuill.com/twiceler/robot.html)”
38.117.64.101 – - [05/Oct/2007:21:59:39 -0600] “GET /111/ HTTP/1.0″ 302 0 “http://vonye.com/111″ “Mozilla/5.0 (Twiceler-0.9 http://www.cuill.com/twiceler/robot.html)”
38.117.64.101 – - [05/Oct/2007:21:59:39 -0600] “GET /111/ HTTP/1.0″ 302 0 “http://vonye.com/111″ “Mozilla/5.0 (Twiceler-0.9 http://www.cuill.com/twiceler/robot.html)”
38.117.64.101 – - [05/Oct/2007:21:59:40 -0600] “GET /111/ HTTP/1.0″ 302 0 “http://vonye.com/111″ “Mozilla/5.0 (Twiceler-0.9 http://www.cuill.com/twiceler/robot.html)”
38.117.64.101 – - [05/Oct/2007:21:59:40 -0600] “GET /111/ HTTP/1.0″ 302 0 “http://vonye.com/111″ “Mozilla/5.0 (Twiceler-0.9 http://www.cuill.com/twiceler/robot.html)”
38.117.64.101 – - [05/Oct/2007:21:59:40 -0600] “GET /111/ HTTP/1.0″ 302 0 “http://vonye.com/111″ “Mozilla/5.0 (Twiceler-0.9 http://www.cuill.com/twiceler/robot.html)”
38.117.64.101 – - [05/Oct/2007:21:59:41 -0600] “GET /111/ HTTP/1.0″ 302 0 “http://vonye.com/111″ “Mozilla/5.0 (Twiceler-0.9 http://www.cuill.com/twiceler/robot.html)”
38.117.64.101 – - [05/Oct/2007:21:59:41 -0600] “GET /111/ HTTP/1.0″ 302 0 “http://vonye.com/111″ “Mozilla/5.0 (Twiceler-0.9 http://www.cuill.com/twiceler/robot.html)”
38.117.64.101 – - [05/Oct/2007:21:59:42 -0600] “GET /111/ HTTP/1.0″ 302 0 “http://vonye.com/111″ “Mozilla/5.0 (Twiceler-0.9 http://www.cuill.com/twiceler/robot.html)”
38.117.64.101 – - [05/Oct/2007:21:59:42 -0600] “GET /111/ HTTP/1.0″ 302 0 “http://vonye.com/111″ “Mozilla/5.0 (Twiceler-0.9 http://www.cuill.com/twiceler/robot.html)”
38.117.64.101 – - [05/Oct/2007:21:59:42 -0600] “GET /111/ HTTP/1.0″ 302 0 “http://vonye.com/111″ “Mozilla/5.0 (Twiceler-0.9 http://www.cuill.com/twiceler/robot.html)”
38.117.64.101 – - [05/Oct/2007:21:59:43 -0600] “GET /111/ HTTP/1.0″ 302 0 “http://vonye.com/111″ “Mozilla/5.0 (Twiceler-0.9 http://www.cuill.com/twiceler/robot.html)”
38.117.64.101 – - [05/Oct/2007:21:59:43 -0600] “GET /111/ HTTP/1.0″ 302 0 “http://vonye.com/111″ “Mozilla/5.0 (Twiceler-0.9 http://www.cuill.com/twiceler/robot.html)”
38.117.64.101 – - [05/Oct/2007:21:59:43 -0600] “GET /111/ HTTP/1.0″ 302 0 “http://vonye.com/111″ “Mozilla/5.0 (Twiceler-0.9 http://www.cuill.com/twiceler/robot.html)”
……

关键词: Cuill, Twiceler

10:00 pm Saturday, October 6th, 2007 贴于 因特网. 引用通告地址：http://yan.me/dia/1219/trackback