Search Engine nightmare

The search engines such as the google search engine use extensive resources to index the various sites in the internet. They make use of advanced algorithms to provide the most suitable results for the users.Just imagine if the results offered by these powerful search engines is very similar to those offered by a simple javascript orPHP script.

I disscus one such javascript which tries to provide such results.And later look into some of the causes,solutions etc..

The Search engine nightmare is a simple javascript that shows a list of results which link to the top sites on the internet. It does not check for the exsistence of pages.It should be possible to do that using PHP and other server side scripts.You can compare the results with google using the last result.I plan to make a personalised version of Search engine nightmare as a google gadget later.

The increasing pouplarity of sites such as wikipedia, delicious, digg, urban dictionary,simpy etc.. has resulted in people linking to the same site for multiple words. The web 2.0, which is driven by user submitted or generated content has resulted in tremendous popularity of these sites which thrive on user provided content. The users who submit content also refer to this content and link to them whenever neccessary. So this effect leads to unintentional google bombing of these sites.

Moreover since these sites have implemented the no follow attribute, the sites that are tagged dont get indexed. The effect can be seen very easily in Google. Large number of search's give Wikipedia as the first result.Similarly Technorati, delicious,simpy,urban dictionary,other bookmarking services etc..also come up in search results.

I could recognise two types of words that get into this effect (which i like to call search engine klogging). The first type of words are those that are being used for the first time.For example "googleoop" is a word that has not been used. So a search for this gives a result that has links to Simpy,Rojo (a feed aggregator)..etc..Generally these words get results that are more relevant as time passes with more popular use.

The other type of words that get into this klogging of the search engine are "tag" words.These are very commonly used words that are used for tagging something.This may seem veryimproper results. But actually they are not. In fact when you search for "blog" it gives the blogs of a few people and companies. This is actually a 'wrong result'. For example the http://dilbertblog.typepad.com/ appears as top results but has nothing to do about a blog except that it is a blog.

Google seems to have realised this(not very sure of this) and has implemented a algorithm to overcome the effects of google bombing. The custom search engine also is a step in this direction. So basically when you want to search the web, you search the web using the expertise of somebody or your own expertise.The google custom search engine can be customised to search the sites that you have tagged using the google marker.The coop search provides a search tool to find the most suitable CSE(customized search engine) for a particualr query.

So does it mean that it is the end of good old fashioned web search engines?In most probability it is an end to the search engines that index the entire internet.The huge size of the internet is one good reason for this. But the same algorithms are now being used to index the selected number of sites.

A question that arises with these developments is--"will people pay to get tagged?
"If this catches up and spamming of bookmarking sites continues, it wil become a mess. Content can be stolen and tagged excessively to get traffic. Since the tagging sites dont check which content is original.