A few days ago I received email alerts notifying me of abnormally high web server traffic. Naturally my first thought was that this might be a Denial of Service attack (DoS): My sites have been attacked before, including by botnets consisting of several thousand rogue computers. This time, during a one hour period, traffic exceeded the same period one week earlier by 800 MB, which works out as 20 GB per day if sustained.
A search of the server logs showed numerous requests like the following, requesting up to 50-60 documents per second:
91.205.124.4 – – [09/Jan/2009:19:20:39 +0000]
“GET /emails/2008-06/30/00209262.117.htm HTTP/1.1” 200 12928 “-”
“Yanga WorldSearch Bot v1.1/beta (http://www.yanga.co.uk/)”
All originated from the same IP address in Russia (91.205.124.4) owned by Gigabase Ltd in Moscow. I found it very odd that this search bot listed the URL of a commercial UK website as its reference, but the company that operates the service does so from Russia. A visit to yanga.co.uk yielded little information – the UK website turned out to be little more than a placeholder page, with no UK street address and only a non-geographic phone number. I grew very suspicious at this stage.
A Google search found a thread on Webmasterworld.com in which “Alexey”, who introduced himself as the CEO of Yanga, responded to criticism. He didn’t give his last name, but perhaps he is Alexey Tarasov who is listed in the Gigabase Ltd WHOIS record.
While it is good to see that Yanga WorldSearch / Gigabase are concerned enough about their reputation to respond to public criticism, they would do well to take steps to raise fewer suspicions in the first place. They could definitely be more open about their identity and their purpose, as well as trying be a good citizen when visiting other people’s websites (e.g. complying with robots.txt, sticking to reasonable traffic levels). Trust is social capital that is hard to earn but quick to burn. No business can succeed on the web without it.
Yes, we also recently got attacked by the same hack…Does that mean our password had been compromised and the attacker used that password to plant those files?
Hello,
what files were planted on your site? And how exactly was this connected to the “Yanga WorldSearch Bot “?
In my case they “only” downloaded tens of thousands of documents in quick succession. Luckily my server could cope and I stayed within my bandwidth allowance.
Its searched my site, and so far has been reasonably relaxed about it – has only read something like one page every 20 mins or so.
Can anyone explain in plain english what this bot is all about?
I found a whole lot of pages by searching for xiando at http://www.yanga.co.uk/ so I will allow it on my servers for now. It seems like a small but legit search engine to me. The crawler seems very stupid and aggressive, which is what made me search and find this webpage in the first place. It seems legit, but not harmless and does not obey crawl-delay in robots.txt.
Pingback: Yanga WorldSearch Bot v1.1/beta - legit and misbehaved