robots.txt advice

Qualidator respects the instructions of the robots.txt-File.

The content could look like this:

User-agent: *
Disallow: /directory1
Disallow: /directory2

User-agent: QualidatorSiteAnalyzer
Disallow: /directory3
Allow: /directory1

User-agent: QualidatorWebSiteAnalyzer
Disallow: /directory3
Allow: /directory1

User-agent: QualidatorBot
Allow: /directory1
Allow: /directory2

Explanation

Basically the bots should exclude directory1 and 2.

The QualidatorSiteAnalyzer resp. the QualidatorWebSiteAnalyzer (earlier version) should exclude directory3 but include directory1.

The QualidatorBot, which works by MyRating should, contrary to other bots, include directory1 and 2.

Avoid Websites being ranked

Registred users can decide if a website is ranked or not. You find the according function on the site with the results on the bottom of the page.

Also the Meta Robots Tags are respected. Following line provokes that a website will not be listed in the Qualidator-Rankings:

<meta name="Robots" content="noindex, nofollow" />

Qualidator News

october 2009

Do you already know the Qualidator® SiteAnalyzer?
The Qualidator® SiteAnalyzer is a Windows application, developed to allow a one step, page by page, automated validation of an entire web project (XHTML & CSS), and to review the quality by carrying out over 30 additional tests.
october 2009

Qualidator.com with Alexa Traffic Rank 16'605!
august 2009

Follow us on Twitter
august 2009

Free Tools for you on qualidator.com
august 2009

Qualidator.com in english ready! After two very successfull years in the german speaking area we are proud to welcome you in english.

Recommendation: Uptime Monitoring

Recommendation: HTML2PDF Rendering Service

Key Benefits
  • Renders URLs of (dynamic)(x)html pages including CSS to PDF on the fly
  • html2pdf is a Service NOT a class library or EXE
  • Very easy to use

Tell me more about the HTML2PDF Rendering Service

Partner

  • SwissMadeSoftware
  • ZTPrentner IT
Copyright © 2009 seven49.net GmbH
Quelle: http://www.qualidator.com/Web/en/Support/robotstxt_advice.htm