I found that the website was running slow and that a computer in Milan university's network (159.149.133.232) was slurping up pages at an alarming rate. So I banned the catzo.
I'm just posting in case anyone knows of a valid reason why someone in Italy would want to mechanically harvest the site, in which case they can be unbanned.
Banning some Italian scumbag
On the first admin page after logging in, you see a list of people who are on the site and where they are in the site and when their IP addresses are. Sometimes you will see an IP address that is mentioned more often. At the moment 66.249.66.203 is getting through 3 pages a minute. This is Googlebot visiting the site for the search engine. But yesterday this computer in Milan was doing 10 pages a minute and the site was noticeably slower.
There is a code of conduct for web spiders which says that you shouldn't be aggressive in harvesting pages, hence Googlebot only doing 3 per minute.
There is a code of conduct for web spiders which says that you shouldn't be aggressive in harvesting pages, hence Googlebot only doing 3 per minute.
It's a bit like making burritos really...
Imagine the forest's website server is the forest kitchen. Everytime a customer comes and asks for a burrito a volunteer goes away and makes it then serves it up to them. That's the same thing the forest's website server does, except it's serving up delicious webpages.
Now, if it's busy you might find yourself serving up many burritos very fast, and that probably means each person is gonna wait longer than normal for their burrito, but that's okay because it's good for business.
But now you're assuming that every burrito order comes from a different person. But what if a robot came into the cafe and started ordering burrito after burrito after burrito, just churning its way through them all?
If it was the Google bot it would not be so bad, because it is programmed to politely request a burrito only 3 times a minute. But this Milan robot wants 10 burritos per minute. What do you do?
I'd ban the catzo.
Sorry.
Imagine the forest's website server is the forest kitchen. Everytime a customer comes and asks for a burrito a volunteer goes away and makes it then serves it up to them. That's the same thing the forest's website server does, except it's serving up delicious webpages.
Now, if it's busy you might find yourself serving up many burritos very fast, and that probably means each person is gonna wait longer than normal for their burrito, but that's okay because it's good for business.
But now you're assuming that every burrito order comes from a different person. But what if a robot came into the cafe and started ordering burrito after burrito after burrito, just churning its way through them all?
If it was the Google bot it would not be so bad, because it is programmed to politely request a burrito only 3 times a minute. But this Milan robot wants 10 burritos per minute. What do you do?
I'd ban the catzo.
Sorry.
I've had it with you. If I had an image of a laser gun I would absolutely position it right here in my hand...
Ha! I have a real laser absolutely positioned in my hand!
Ha! I have a real laser absolutely positioned in my hand!
