Excluding Web Pages From Being Indexed by Web Crawlers

It is recommended to exclude the home pages that allow access to the Enterprise Services Repository, the Integration Directory, and the PI monitoring tools from being indexed by web crawlers (for example, as used in Internet search engines). To do this, the corresponding URL strings have to be provided in the robots exclusion standard file ( robots.txt file).

More information: icm/HTTP/file_access_<xx>

Provide the following URL patterns in the corresponding robots.txt file:

• /rep/*

• /dir/*

• /rwb/*

• /mdt/*

• /pimon/*

• /nwa/*