How Search Engines Index Your Web Site
Join 4Shared Now!
Send me One
Million FREE Guaranteed Visitors
Your Ad Here
Notes on helping search engines index your Web site
This section provides some simple suggestions that will make your documents more accessible to search engines.
Define the document language
In the global context of the Web it is important to know which human language a page was written in. This is discussed in the section on language information.Specify language variants of this documentIf you have prepared translations of this document into other languages, you should use the LINK element to reference these. This allows an indexing engine to offer users search results in the user's preferred language, regardless of how the query was written. For instance, the following links offer French and German alternatives to a search engine:link lang="fr" title="La vie souterraine" hreflang="fr" href="mydoc-fr.html" type="text/html" rel="alternate"link lang="de" title="Das Leben im Untergrund" hreflang="de" href="mydoc-de.html" type="text/html" rel="alternate"Provide keywords and descriptions
Some indexing engines look for META elements that define a comma-separated list of keywords/phrases, or that give a short description. Search engines may present these keywords as the result of a search. The value of the name attribute sought by a search engine is not defined by this specification. Consider these examples,meta content="vacation,Greece,sunshine" name="keywords"meta content="Idyllic European vacations" name="description"Indicate the beginning of a collectionCollections of word processing documents or presentations are frequently translated into collections of HTML documents. It is helpful for search results to reference the beginning of the collection in addition to the page hit by the search. You may help search engines by using the LINK element with rel="start" along with the title attribute, as in:
link title="General Theory of Relativity" href="page1.html" type="text/html" rel="start"
Provide robots with indexing instructionsPeople may be surprised to find that their site has been indexed by an indexing robot and that the robot should not have been permitted to visit a sensitive part of the site. Many Web robots offer facilities for Web site administrators and content providers to limit what the robot does. This is achieved through two mechanisms: a "robots.txt" file and the META element in HTML documents, described below.
B.4.1 Search robots
The robots.txt file
When a Robot visits a Web site, say http://www.foobar.com/, it firsts checks for http://www.foobar.com/robots.txt. If it can find this document, it will analyze its contents to see if it is allowed to retrieve the document. You can customize the robots.txt file to apply only to specific robots, and to disallow access to specific directories or files.Here is a sample robots.txt file that prevents all robots from visiting the entire site
User-agent: * # applies to all robotsDisallow: / # disallow indexing of all pagesThe Robot will simply look for a "/robots.txt" URI on your site, where a site is defined as a HTTP server running on a particular host and port number. Here are
some sample locations for robots.txt:Site URIURI for robots.txt
http://www.w3.org:80/robots.txt
http://www.w3.org:1234/robots.txt
There can only be a single "/robots.txt" on a site. Specifically, you should not put "robots.txt" files in user directories, because a robot will never look at them. If you want your users to be able to create their own "robots.txt", you will need to merge them all into a single "/robots.txt". If you don't want to do this your users might want to use the Robots META Tag instead.Some tips: URI's are case-sensitive, and "/robots.txt" string must be all lower-case. Blank lines are not permitted within a single record in the "robots.txt" file.There must be exactly one "User-agent" field per record. The robot should be liberal in interpreting this field. A case-insensitive substring match of the name without version information is recommended.If the value is "*", the record describes the default access policy for any robot that has not matched any of the other records. It is not allowed to have multiple such records in the "/robots.txt" file.The "Disallow" field specifies a partial URI that is not to be visited. This can be a full path, or a partial path; any URI that starts with this value will not be retrieved.
For example,Disallow: /help disallows both /help.html and /help/index.html, whereasDisallow: /help/ would disallow /help/index.html but allow /help.html.An empty value for "Disallow", indicates that all URIs can be retrieved. At least one "Disallow" field must be present in the robots.txt file.Robots and the META element
The META element allows HTML authors to tell visiting robots whether a document may be indexed, or used to harvest more links. No server administrator action is required.
In the following example a robot should neither index this document, nor analyze it for links.
(In order to have the right figures and index your site or not ,please, visit the site which has this article on the following link)
http://www.w3.org/TR/REC-html40/appendix/notes.html
Note. In early 1997 only a few robots implement this, but this is expected to change as more public attention is given to controlling indexing robots.
Related Key words for your search:-
Tags for SEOseo,seok,seong,seoul,senseo,seo uk,seo web,seorang,web seo,blog seo,closeout,firm seo,museo di,seo firm,seo site,seo tool,site seo,closeouts,elite seo,mouseover,pop seoul,seo eliteTags for Metameta,metal,metabo,metals,cemetary,meta tag,metacafe,metairie,metal cd,metallic, metatags, html meta, http meta,meta head, meta html, metaname,metatags,metabolic,metallica, metallics, metastock,timetable,metatitle,metabolife,metabolism,metabolite, metasearch, metastasis, metastatic,metatrader, meta engine, meta search,metabolites,metacrawler,metaphysics,search engine optimization
Labels: cemetary, meta, meta tag, metabo, metacafe, metal, metals, search engine optimization, senseo, SEO, seo elite, seo site, seo tool, seo uk, seo web, seok, seong, seoulsite seo