On spiders and search engines
On the last few days when analyzing the logs of a site I have seen 2 interesting spiders:
First is Google AdSense and authentificates itself as a spider. Usually after it comes Googlebot. What is more curious is that the site didn’t had AdSense
The second authentificates itself as a user and it’s eating a lot of pages. Apparently it’s origin is http://beta.exalead.com/search/C=0/?2u=3. This seem to be a new search engine with some very good features.
- results clustering
- audio/video search
- clean interface apparently ajax based (can anyone figure for wahat are the boxes below the serach box on the main page?Personalization?)
- multiple ways to see the results
- filtering results based on GeoIP(?)
It seems that they started digging from DMOZ like Google. From the amount the pages I have seen on that site they seem prepared to ingest a large amount of data.
While writing this post I found this on their site:
Bred from the prestigious Ecole des Mines in Paris, the founding members of Exalead cut their teeth on the first generation of Web search engines. Since the creation of Exalead in 2000, they have concentrated their efforts on facilitating access to their client’s information bases for all users: Employees, customers, suppliers, and the general public.
It has also been mentioned on Search Engine Watch
You can spot the spiders if you look in visitors after: ng21.exabot.com
Pretty good european job. Waiting for the Desktop Beta
Yes indeed it’s a preety neat search engine. Altho all the bling-bling gadjets are cluttering the page, also it’s not very easy to use, but the search filtering/twicking options are amazing… great job guys!
But that ain’t ajax