Dissecting the search box code

The google guideline clearly mentions"The code for the search box will be generated for you to cut and paste into your site". But Plenty of examples can be seen which dissect the google code. One of the most controversial is the removal of the "Google Custom Search" line which is present in the search box. It surely is violating the google guidelines.

Its fairly simple to do. Just add Value="" in the line Input name="q". If google is really serious they may modify the script or penalise those who are removing the google line. The other things that are done involve changing the word that is displayed in the search button. I have seen varaints such as "Find", "Locate"...etc. This is done by replacing the value="search" with value="Find".

If you have seen the google homepage more carefully, you would have noticied the "Feeling Lucky" button beside the "Google search" button. If you hit this button, you are directly taken to the first search result site. This is not still implemented in CSE's. If you try changing the name="sa" for the button it does not seem to affect the search results in any way.Hopefully google will provide these options in the long run.

And if you feel the search box is very long for you liking, you can decrease its length by changing the size="40" to a more appropriate size. This can be useful if your search box is not fitting on your side bar. Generally the side bar is long enough to fit the Google CSE. However, it can become a little long in some side bars. You can reduce to sya 30 or 35 to fit it in. If the side bar is still smaller than this, you better increase the size of your side bar.

CSE networks-advanced blog rolling

A very interesting new development with Google's CSE's is the formation of networks. A network search is very similar to blog rolling. Instead of providing a link to another blog, CSE networks include only one or two sites or blogs on a particualr topic. When a user searches for a particular topic he will be driven to that topic, but only to the site which has been included.

Each of the sites in the CSE network will have the search box and drive traffic towards each other. It can be compared to the ClikZ network. However, it is different in that one of the person generally owns the network.

This will surely lead to user monopoly. Each person will try to limit the search network to his own benefit. The same concept can be extended to create klog networks. To understand what a klog is visit http://kloggenerator.googlepages.com

Take a look at the network of Daniel Malewitz. It has as many as 9 volunteers with 34 sites being searched. Try the keywords such as Google in this CSE and most of the results will be from within 2 or 3 blogs. Similarly, other networks are in the making at present. If you have created such Networks do let me know.

CSE networks may well increase adsense click fraud drastically. The bigger the network, the more is the chances of missuse. The network badges for these networks may soon become popular. Each of these sites will also display CSE network badges and encourage others to join the networks.

Death of the search box?

Prologue (this is hopefully a technical article and not a story)

The times are changing as they always do…People are too lazy to use the search box, be it the search box in a search engine (no search engines have a long life ahead of them) or a specific site. Thus, the need for alternatives have driven people to make lot of unique and interesting ways of linking up data. In this post I hope to review all these methods. --end of prologue

Problems with the Search box.The search box became popular due to the flexibility it offered, the user had the capacity to give a query he liked. But this seems to be a problem as well. How—

• The user has to know what to type into the search box---for example if you were searching for the “pictures” of a palace (to use as a background-so any palace would do). The user could try words as diverse as “Palace pictures”, “Palace images”, “Palace photos” and the search results you would get would be as diverse as these -Palace pictures Palace images Palace photos Although all these queries mean nearly the same thing for a layman like me, they do have a subtle difference. This difference is based on context.
• Auto fill(google toolbar) –this seems to be a good problem solver. If you type a particular query once you can use it again without having to remember it. But again the results are not related to context. So the problem is lack of context sensitive results. What i mean is--you get a result about computer viruses when you search for "virus". This should seem rather annoying for a biologist.Not that any biologist would search for 'virus' alone. He would give a more apporpriate query like "XYZ disease causing virus". Yes, so, what is "appropriate"? It is relative to context. Context seems to be the problem. Enough of defining the problem--lets see some results.

Google Earth

"Offers maps and satellite images for complex or pinpointed regional searches."--So the context here seems to be specific to a region. If you were trying to find a shop near you that sells "high sucrose content milk", you just go to your region in the map and find one. You can personalise your search to a particular region.

Smart Tags and Auto link

Although both these methods of providing context specific links at the browser were proclaimed evil, they seem to be the first attempt at providing Context specifc linking.Smart tags was introduced by Microsoft in Internet explorer. Auot link comes with the google toolbar. These tools will provide links to sites of their choice from the words that appear in a particular page. The main problem was that the publisher of that particular site would not in any way get any benefits, and he was inadverdently driving traffic to other sites.

Grease Monkey

The "grease monkey" is firefox extension that allows its users to choose a module which can be used to link words appearing on its site to other sites. The advantage with this tool is that the user can decide which kind of sites get linked.The main drawback is that if you need to share your list of sites, you generally have to transfer a file.

Monkey Grease

This is a server side alternative, in which the publisher of the site gets to choose which sites to link to. The program will then link words and links "appropriately". But most of these auto linking methods seem to be one sided-either the publisher has full control or the user has full control.

Klogging (using the kloggenerator)

Its simply linking each word to a list of links. Each word appearing on the website is linked to a list of websites about that topic. Both the publisher and the users of that web site can add or remove web sites from that list.

So the use of the search box is on the decline. You generally use a serch box when you find a new word in a site and it does not have a link. If all the words on all the web sites and blogs were linked to appropriate sites or lists of sites, the chances of your using a search box are very small. The klog tool also provides a method to link to multiple words together as well using NHTML. However, every time you come across something new, like a new car--how would you find it??--no you would not use the search box--You would log into your account (tagging service account) and click on the tag "Car". This brings up a list of sites about cars--in these sites, you would try to find your new car brand. Your tagged sites are more likely to provide you with content related to your context than a search box.

The future

The virtual worlds seems to hold the key to the internet of tommorow. A world linked together, we move from one virtual world house(site) to another.We talk to people through virtual characters, send e-mails through virtual post boxes, buy goods in virtual money... the possibilities seem limitless. The Death of the search box is the death of the 2 dimensional web. The idea of having to view the entire web through a browser window is loosing ground. The search engines need to provide a virtual world alternative to the search box. The most likely alternative i can think of is a "wise old man". A bot that can answer the questions of all those who want answers. How well the AI grows around us is the quesion.

buffalobird-game search CSE

Search engine Topics:Games,game reviews, downloads, demos, cheats..

Contribution:No explicit declaration

Results hosted at:Google pages

What's Interesting: You get result refinements with the following headings Game Cheats, Game Reviews, Mac Games, Internet Games, Game Demos, Game Downloads

See the difference in results for the query "cheats" with google and

buffalobird-game search

Google

Try it:http://buffalobird.googlepages.com/gamer

Google CSE wisdom of the crowd or expert tagging

Google Custom Search Engines (CSEs) can be maintained by a one person working alone, a defined team or be open to all. If the CSE is being maintained by the 'crowd'(-defined as any person who may be or may not be technically qualified in that domain.) the wisdom of the crowd is at work. One of the best examples of such an experiment is the "PUTCH". I hope to interview the person who started it one day.

The google CSE is different from general 'crowd' wisdom in that experts can provide highly selected content to the users of their CSE's. I came across few CSE's on very specific subjects. These can be very helpful as most of these sites are popular only within a small community of users. Now coming to the idea of teams of experts, this is a still more effective stratergy in that the experience of all those experts is made use of.

Most of the CSE's are encouraging others to suggest sites that need to be included in a CSE, rather than asking them to volunteer. The concept of volunteers is in some controversy mainly due to the following reasons

1. Sharing of Adsense income if any
2. Resolution of conflicts regarding the inclusion/exclusion of sites
3. If one volunteer is removed all the changes made by the volunteers is removed

However, the overall effect of CSE's seems to be one of user specific search. Very much like tagging--i say like and not the same. In tagging, each user tags different sites with tags of his choice, without bothering to see what others have tagged it with. In a google CSE on the other hand, the users of a single CSE, can see for themselves, what tags other users use and have a chance to improve or modify those tags.

As the CSE gains popularity, more of these interesting features of social bookmarking based search will come to light.

Find Metal- search engine

Search engine Topics:Heavy metal news, interviews, bands, videos, and metal history

Search range:All sites indexed by google (with preference to list of sites)

Contribution:You can suggest which sites need to be included by mailing webmaster@find-metal.com

Results hosted at:Own site

What's Interesting: You get result refinements with the following headings History of Metal,
News, Blogs, Video, Lyrics, Christian Metal, Speed Metal.

See the difference in results for the query "bands" with google and Find-metal

Find-Metal

Google

Try it:http://find-metal.com/

Google CSE- increase annotation limit- comment campaign

Google did a very good job by introducing the CSE's. But why did they have to put a limit to the number of annotations. May be there are many reasons. But it seems that most of us have reached the 5000 limit. Many methods have been used to overcome this limit, such as listing out sites on a particular page, or different pages of a domain and annotating that url. But these cannot be annotated.Thus,

I am here by requesting "Google Customized search" to increase the number of annotations one can add to a CSE. If you also want them to increase the limit, Support my requestby commenting below. Thank you...

Google Boat Building and Cruising Search Engine

Search engine Topic:Boat Building and Cruising

Search range:All sites indexed by google (with preference to list of sites)

Contribution:You can suggest which sites need to be included

Results hosted at:Own site

What's Interesting:The CSE has options to show different number of results (one result at a time or more), it shows all the results in the side-bar. So no need to leave the main page to see the search results. As you can see in the picture the search results are listed one below the other in the side bar. No need to refresh the page as results can be cleared using the clear results button.




Try it: http://www.craftacraft.com/GoogleCustomSearch

Klog and NHTML beta

It looks like Klog generator development will keep me busy for some time. Moreover NHTML seems to be better in its working when combined with CSS, in the mean time i was planning to feature a few customized search engines on this blog, be my guest if you have a good search engine.

The kloggenerator is still available at http://kloggenerator.googlepages.com/, if you feel it needs more features or better explanation or lesser features and morre explanation let me know, i get the feeling that documentation for the klog generator is somewhat patchy.

Klog your site-Connect it to your CSE

The method to klog your site can be found at

http://kloggenerator.googlepages.com/

Just enter the first page of the results of your search engine for any query into the GOOGLE CSE klog tool and generate the KLog code for any text you input into the kloggenerator. It does not do any magic, it simply links your words to your Google CSE. If you want to allow user participation, allow the users of your site to add or remove sites from your google CSE. So the klog method has the flexibility to allow both the publisher as well as the user to control the linking.

It would be highly impossible to have grease monkey scripts for each and every site on the internet. However, each publisher can klog his site and allow for user modification of links only or the entire website. It does seem to be very much CHMOD777 or web 2.0. The klog is different from multiple tags in that numerous links or NHTML is adopted.You can lok at

http://kloggenerator.googlepages.com/nhmtl

to understand the working of NHTML.

Klogging compared

Criteria Klogging Auto link Smart tag Muliple tagging Grease Monkey

Manufacturer You Google Microsoft You You

Browser support Not needed Requires toolbar Only IE Not needed Only firefox

at present


Who can modify Both the user Mostly Google Mostly Microsoft Publisher only User only

where to link to and the publisher


Disabling Possible Possible Possible Not possible Possible

(Requires browser support)

Downloading Not required Tool bar needs only for IE Not required Required

to be downloaded

I do doubt if i am allowed to do such a comparison, as i am invlolved in the development of klogging. However, feel free to contradict any of these claims. Also it would be acceptable if further changes are needed to the concept of klogging, as it is still in BEta.

subscribed links-similar to RSS feed

With google customized searche engines you can have subscribed links. It is somewhat similar to getting RSS feeds from a site, only results from your search engine are the feed into google. You can ask people with similar intrests to subscribe to your search engine. When their search queries match those for your search engine, they will be taken to your results. This is very similar to Networks in Delicious.

You can create a subscribed link by going here.

http://www.google.com/coop/subscribedlinks/

So the more number of subscribers you have, the better. People can also subscribe to individual sites, however, the idea of subscribing to a CSE is more appealing as you can restrict your search field.Moreover, as their seems to be a limit on the number of subscribed links, it would be more beneficial to have a CSE, which in turn has a list of those needed sites.

Irrelevant content and lower ranking

As many new Google customised search engines are being created. These are getting included in my search engine at http://cse.search.googlepages.com/

However, many of these sites have a tendency to include irrelevant content on the same page as the search engine box. Not that it is wrong..It leads to the COOP search providing lower ranking to the sites with time. As more and more people will get these sites in their search results and find it irrelevant.Ok I will take an example to explain what i mean by irrelevant content.Consider the search engine at http://www.indigoguide.com/spain/ski-resorts.htm

It is a page about ski resorts. However, to the right corner they have put a custom search engine. When somebody searching for a search engine to find a "restaurant" or "entrratainment" enters the query this page will come in the results as it contains those words. However, since that person is not going to find it useful in most of the cases he will not use the page and go to the next result. This way that page gets a lower ranking. So if you include words or sentences irrelevant to what your search engine finds, it will lead to an overall decrease in your ranking in the long run. And since COOP search is a human parsed search engine, A lower starting score is given to those pages which contain irrelevant words and sentences.

Coop search engines directory

The google team has created a search engine directory at

http://www.google.com/coop/cse/examples/GooglePicks

It lists different types such as cool tricks, bloggers, educators,enthusiasts, Government, Industry experts, Locals, Media publishers, researchers and techies. Each list is not exclusive. Therer are many search engines that are not included in that list as such. Something like your google gadgets content directory.

My mysore search engine has made it to the Locals list in that directory. Take a look and see how well you can search mysore related sites. I have not included any mysore tourist spam sites in my search. That gives way better results than google results which is filled with tourist sites.You can take a look at it in

http://mysorechronicles.blogspot.com

Enough of blowing my own trumpet. Let us see what other goodies that this directory contains as we move along.

Annotations-grade sites in your search engine

You can grade your sites by using annotations. To do so download the XML or TSV file from the "advanced" tab of your site. If you have not loaded a list yet, no file will be present. Just type in the following manner a list of your sites with their scores.

The labelname "your search engine code" is specific to different search engines.You can know what yours is from the advanced tab of that specific search engine.Once you have typed it as shown above, save the file as .xml and upload it.And you are ready to go.

Make a google module for your search engine

After having suffeciently tested your search engine, you can go ahead and make your google module. You can either create a "Custom Search Gadget" using the facility offered by google or make your own by pasting your CSE's code into the content section. To add the one created for you by google you can visit your search engine homepage and click on the add to google page. You can then use your search engine from your google homepage itself.

CSE list-find what others have done

Before you start out and build your own CSE- customized search engine it would be wise if you saw what others have done. To get a list of CSE's regarding any topic you can use the search engine in this page. Just type the name of the topic and hit the return button. You will be presented with a list of search engines dealing with that topic. However, don,t be disheartened if many such options are present. You can always do something that is different.

Take a look at the search results given by these search engines. If you feel you can do a better job, go ahead and do it. Most of the search engines that are seriously into the CSE work would have got themselves tagged in this search engine. However, it contains more than the 90 odd sites that have been tagged as of now. It uses a special consturct to search the entire list of Google CSE's

Once you have decided what topic you want to base your search engine, collect a list of sites that provide quality information about that topic. Preferably visit each of those sites to determine the type of content.Make a list of these site with a score of 0.5 or 0.75 or 1.0 for the different sites depending on thier content. You can then convert it into XML format or other formats supported by google such as TSV.

You can also make a list of sites you want to exclude from your search results if you are allowing the search enines to search the entire web. If you can exclude some of the big sites which occupy most of the search results you could get some really fascinating results.More about what else you can do in the next post.

Browser requirements-Customized search engine

If you are having problems accessing your customized search engine control panel or adding or removing sites. you better upgrade your browser. If you are using internet explorer the IE7 or above is preferable. If you are using firefox use a version after 1.5. The opera browser is also fine if you use a version above 9.02. The page may just load and show erroe if you use a browser that is lower than these.

If you want to use the advanced features of the control panel, it would be best to upload the files with the annotations in XML or other permitted formats. It provides you the chance to provide a preliminary score to your site. This feature is yet to be implemented for the users of the interface.

You can download the latest version of firefox for free. It is supposed to provide better security then IE. However, it would be best suited for linux environment. If you are a windows user you can use it just like your IE, but it may be a little slowere than IE in windows XP home edition.

creating a customized search engine part II

Fill out the form as described in the last post. After pressing the next button you will be taken to the screen shown here. You can try out a few search queries in your own search engine.After pressing the Finsish button you are done, Your customized search engine is ready to run. You will be taken to the control panel of the search engine, where you can do all the tweaking to make your search engine look and work like how you want it to.As such you can add as many as 200 annotations per file and a maximum of 5000 annotations per gmail account. Let us see what other things we can do with this search engine in the next post,

How to make a customised search engine?

Go to the following site and you need to login with your gmail account to get access

http://google.com/coop/manage/cse/create/

Once you have logged in you will have to go through two steps to create your own search engine.

In the first step you will be asked to give information about

Search engine name-Two or more search engines can have the same name, so dont worry, you need not have to check for availabilit etc,But make sure you select a name that explains what your search engine is going to search for. And since this appears on your google homepage(yes, google provides each of the search engine a specific homepage ), make sure it is a descriptive name, as search engines will give a better relation to your word and its meaning.

Search engine description-This is to describe what your search engine is about and the same applies to this as for the name. But don,t include unrelated words, as it will lead to ranking in for those unrelated words.Although if you want to attract unwanted traffic, go ahead, but i would not reccomend you to do that.

Search engine keywords-This seems to be the most important part of the entire form, select two or a maximum of three keywords. keep the range of your search engine in mind while selecting the keywords. Since the search engine will be searching for these keywords choose them wisely. You dont have to worry about choosing them wrong the first time, as they can be changed latter on.

The option to include only sites you mention, or giving prominence to these sites is also provided. Choose the option of searching for only included sites if you want to narrow down your search drastically, however, if you plan to make it more broad, choose search the entire web. But choosing to search the entire web will take a little time to be of much use.

To start a COOP(Customized search engine) you need to include atleast one site while starting the venture. So find the best site to include first on your engine, as the first few sites that are included will appear on your google homepage.

You can choose to collaborate with as many as 100 volunteers. It would be wise to collaborate if you are covering a wide topic. However, be careful as to whom you collaborate, as they too get to change the sites to include and exclude.

More about the next step in the next post..

Harness the power of Google search

"Create a highly specialized Custom Search Engine that reflects your knowledge and interests. Place it on your website and, using our AdSense for Search program, make money from the resulting traffic."

Google describes its Customised search engine thus. Let us see what each of those lines mean in a little detail

--Make topic specific search engines

--human parsed search engine in contrast to google's search engine which is mainly robot generated

--possibly a result of competition from delicious

--allow members to make money from that-generates more interest

--a high degree of customization to help people who keep searching for very obscure topics--like searching needle in a hay stack..

Searching search engines

With the release of google's customised search engine, the race to build custom search engines has heated up. But it would be such a waste if all these COOP's could not be used. So COOP search is a way to find the most used COOP's and using them. Since I have been AJAXED, it would be easy to use these search tools with ease.