How to Include and Exclude Pages in Sitepager

Sitepager lets you customize Checks to focus on the most important parts of your website. Whether testing the entire site, specific sections, or key pages, the Starting URL, Include List, and Exclude Patterns give you full control.

Understanding Include and Exclude

1. Starting URL: Defines where the crawl begins.

   • Use the homepage to test your entire site.

   • Use a section-specific URL to limit the scope to one area.

     
     

2. Include List: Adds specific URLs to test in addition to the Starting URL.

   • By default, only the Starting URL and Include URLs are tested. Child pages of Include URLs are not crawled.

   • To include child pages of the Include URLs in the Check, enable the Crawl Included URLs option. This setting allows Sitepager to follow internal links on the Include URLs.

     
     

3. Exclude Patterns: Skips pages or sections matching a pattern.

   • Exclude Patterns are applied after the Include URLs. Any pages matching an Exclude Pattern will be skipped, even if they are listed in the Include URLs.

   • Use wildcards in Exclude Patterns to filter multiple pages at once.

Important Note on Subdomains: Subdomains are excluded from the primary domain’s crawl by default. If you need to test a subdomain, set its URL as the Starting URL in a separate Check.

To make these concepts easier to understand, we’ll use the example of sitepager.io. Here is a test structure:

Choosing the Right Starting URL

The Starting URL defines the entry point of the crawl and determines which pages are included in your test. Here's how to pick the right Starting URL based on your goals.

| <strong>What You Want to Test</strong>               | <strong>Starting URL</strong>               | <strong>Additional Configuration</strong>      |
|-------------------------------------------------------|---------------------------------------------|------------------------------------------------|
| <strong>The entire website</strong>                  | <strong>Homepage</strong> (https://sitepager.io) | Use <strong>Exclude patterns</strong> to refine the crawl. |
| <strong>A specific section</strong>                  | <strong>Section URL</strong> (https://sitepager.io/features) | Use <strong>Exclude patterns</strong> for finer control. |
| <strong>Specific key pages only</strong>                | <strong>Homepage</strong> (https://sitepager.io)        | Use the <strong>Include List</strong> for exact URLs.<br><strong>Crawl Included URLs</strong> enabled or disabled based on scope.|
| <strong>A specific subdomain</strong>                   | <strong>Subdomain URL</strong> (https://admin.sitepager.io) | Subdomains must be tested separately. Set each subdomain as a <strong>Starting URL</strong>.|                                 

Real-World Use Cases

Now that you understand Sitepager’s Include and Exclude functionality, here are some real-world examples of how these configurations can be applied to the sample site, sitepager.io.

1. Test the Entire Website While Skipping Specific Sections

Goal: Check all public-facing pages while skipping blog posts and internal admin areas.

Configuration:

• Starting URL: https://sitepager.io (homepage).

• Exclude Patterns:

  • /blog (skip blog section).

Result:

• Sitepager crawls and tests all pages linked from the homepage except for the admin dashboard and blog section. admin.sitepager.io is skipped by default, as subdomains require separate Checks.

2. Test All Pages Under a Specific Section While Skipping a Subsection

Goal: Focus on all pages under /features while excluding specific subsections like "beta features."

Configuration:

• Starting URL: https://sitepager.io/features.

• Exclude Patterns:

  • /features/beta (skip beta features).

Result:

• Sitepager crawls and tests all pages under /features, excluding the "beta features" subsection.

3. Test Specific Key Pages Only

Goal: Test critical pages such as the homepage, pricing page, and features pages.

Configuration:

• Starting URL: https://sitepager.io.

• Include List:

  • https://sitepager.io/pricing

  • https://sitepager.io/features

• Crawl Included URLs: [Enabled/Disabled]

Result:

• Crawl Included URLs [Disabled]: The default option. Sitepager tests only the homepage, pricing page, and features page, without testing child pages of /features.

• Crawl Included URLs [Enabled]: Sitepager tests the homepage, pricing page, features page, and all child pages linked from these included URLs.

Best Practices

1. Test Configurations First:

   • Validate your configurations with a small Check before running site-wide tests.

2. Start Broad, Then Refine:

   • Start with the homepage to test your entire site, and use Exclude Patterns to filter out irrelevant sections.

3. Use Section URLs for Focused Checks:

   • For focused testing of specific sections, set the Starting URL to the section URL (e.g., /features).

4. Be Precise with Includes:

   • Use exact URLs in the Include list when you only need to test key pages.

Next Steps

1. Learn More About Visual Comparison Thresholds:

   • Explore how to fine-tune sensitivity in the How to Configure Visual Comparison Thresholds Guide.

2. Manage Configurations and Baselines:

   • Save and reuse configurations in the How to Manage Multiple Baselines Guide.