Novell NETWARE 6-DOCUMENTATION Manual page 2099

Using the Robots META Tag
170 Getting Results with Novell Web Services
Another effective way of controlling what Web Search indexes is using the
Robots META tag, a tag inserted into the content that is being indexed by Web
Search.
When a Web-based search engine encounters a document containing the
Robots META tag, the search engine will do as the META tag instructs.
There are four values you can specify in the Robots META tag:
NOINDEX: Indicates that the document is not to be indexed.
NOFOLLOW: Indicates that hypertext links in the document are not to be
crawled.
ALL: Indicates that the document can be indexed and all links can be
crawled.
NONE: Indicates that the document is not to be indexed and that
hypertext links are not to be crawled.
To include the Robots META tag, use this syntax:
<META name="Robots" content="INDEX, value">
You can also use the Robots META tag to exclude specific sections of HTML
documents from your search results. For example, you might not want such
sections as repetitive headers, footers, navigation bars, and server-side
includes to be indexed.
HINT:
You could also place these tags in Include files so they never get indexed
at all.
To direct Web Search where to begin skipping content while indexing, do the
following:
1
At the point in your HTML document where you want Web Search to
begin skipping content while indexing, enter the following tag:
<!--*Robots NoIndex-->
2
Just after the content you want skipped, enter the following tag:
<!--*Robots Index-->
3
Save your changes and index (or reindex) the content.