---
sourceDocument: Australia ServiceNow AI Platform Administration
sourceDocumentLink: https://servicenow-prod.fluidtopics.net/r/platform-administration

 Release :

    - australia

ft:locale :

    - en-US

ft:publication_title :

    - Australia ServiceNow AI Platform Administration

ft:clusterId :

    - platadm

bundleId :

    - platadm

workflow :

    - Platform


---

# Create connector

# Create a Webcrawler external content connector {#ariaid-title1}

Release version: Australia  
Updated March 12, 2026  
![](https://www.servicenow.com/docs/portal-asset/ico-clock) 5 minutes to read  
Create an external content connector to retrieve searchable content from pages and subdomains in a public web source system. Select from a list of predefined web sources or specify your own web source.

## Before you begin

Role required: sn_ext_conn.xcc_admin

## About this task

Follow this procedure to create a new Webcrawler connector to retrieve searchable content and metadata from URLs found in a predefined or custom web source.

By default, you can configure up to three Webcrawler connectors for custom web sources. If you need to retrieve items from more than three custom web sources, you can create a Customer Service and Support case at <https://support.servicenow.com/now> to request a limit increase for the Webcrawler connector.

## Procedure

1. Navigate to AllExternal Content ConnectorsExternal Content Admin Home.
2. If prompted, select Switch scope to switch to the External Content Connectors Admin scope.  
   You must be in this scope to create or edit external content connectors.
3. In the Connectors section, select New.
4. On the Choose source page, select the Webcrawler tile, then select Next.
5. On the Connection settings page, specify the web source that you want to retrieve searchable content from.
   * To retrieve searchable content from a web source in the list of predefined web sources, select the Predefined web source tile, then select your desired web source from the list.
   * To retrieve searchable content from a public web source not included in the list, select the Custom web source tile. Enter a name for the connector and the URL for your selected web source, then select the legal disclaimer option.  
     Note:  
     If you select this option, content crawls for the Webcrawler external content connector retrieve content and links from URLs found in the source system's sitemaps that include the specified start point URL. The connector reads matching URLs from all sitemaps referenced in the source system's robots.txt file and all sitemaps located in common sitemap locations.

     As an
     example, suppose you select the Use sitemap option and then specify https://example.com/mysite as the start point URL for a content crawl. When you run the content
     crawl, the Webcrawler connector retrieves content and links from sitemap URLs that include https://example.com/mysite. In this case, the connector retrieves content and links from sitemap
     URLs https://example.com/mysite/a and https://example.com/mysite/b but ignores sitemap URLs https://example.com/othersite/c and
     https://example.com/yoursite/d because they don't include the start point URL.
   {#create-ext-cont-connector-webcrawler__choices_yjl_ptq_4fc} {#create-ext-cont-connector-webcrawler__specify-web-source-step}
{#create-ext-cont-connector-webcrawler__specify-web-source-step}
6. Save and validate your connection settings by selecting Validate Connection.  
   Note:  
   If validation of your connection settings fails, the system shows an error message. Verify your connection settings to ensure they're correct. If permissions required by the connector are missing or incorrectly configured in the source system, a warning message appears showing the permissions that must be corrected. Provide the information from this message to your source system administrator. {#create-ext-cont-connector-webcrawler__new-connector-save-connection-settings-cmd}
7. On the Crawl settings page, modify any default crawl settings that you want to override for this connector, then select Next.  
   If you want to skip this step for now, select Skip instead of Next. You can modify the crawl settings for this connector from the External Content Admin Home page. For details on this procedure and the available crawl settings, see [Configure crawl settings for an Atlassian Confluence Cloud external content connector](https://servicenow-prod.fluidtopics.net/X~Zy3wKKLqVAFQsA0L2olw "Specify the spaces you want your Atlassian Confluence Cloud external content connector to crawl. Define inclusion or exclusion filters to dictate the types of content the crawl retrieves and feeds to AI Search for indexing.").
8. On the Create crawl page, create a content crawl for this connector by selecting a crawl scope (if supported) and any desired options, then select Next.  
   If you want to skip this step for now, select Skip instead of Next. You can create and run crawls for this connector from the External Content Admin Home page. For details on creating content crawls, see [Create a content crawl for an external content connector](https://servicenow-prod.fluidtopics.net/fFJ92ZYif96fkc2JA4HtyQ "Retrieve searchable content and metadata from your source system with a content crawl. Run the crawl as a one-time task or schedule it to run on a recurring basis.").
9. On the Connect search profile page, use the Connect to search profile field and Add button to add any search profiles that you want to connect this external content connector's default search source to, then select Save.  
   If you want to skip this step for now, select Skip instead of Next. You can connect search sources for this connector to search profiles from the External Content Admin Home page. For details on connecting an external content connector to search profiles, see [Connect an external content connector to a search profile](https://servicenow-prod.fluidtopics.net/O1_d0dPqWANwyTUPT43BrA "Make results from an external content connector searchable in your portals and search applications by connecting its search sources to your existing search profiles.").

   When you link a connector search source to a search profile in this step, the system automatically publishes the search profile to make the new link take
   effect.

## Result

Your new external content connector appears in the Connectors list on the External Content Admin Home page.

## What to do next

To retrieve searchable content with your new connector, you must configure and run content crawls for it. You can modify your new connector's crawl settings and create crawls for it from the External Content Admin Home page even if you skipped these steps during connector creation.

* To learn how to configure your new connector's crawl settings, see [Configure crawl settings for a Webcrawler external content connector](https://servicenow-prod.fluidtopics.net/2InfYbiEPnW2iLnGbpKtDg "Specify the pages and subdomains you want your Webcrawler external content connector to retrieve from your specified web source.").
* For details on creating crawls for your new connector, see [Create a content crawl for an external content connector](https://servicenow-prod.fluidtopics.net/fFJ92ZYif96fkc2JA4HtyQ "Retrieve searchable content and metadata from your source system with a content crawl. Run the crawl as a one-time task or schedule it to run on a recurring basis.").
{#create-ext-cont-connector-webcrawler__ul_zkt_3qj_4fc}  
To make content crawled by your new connector searchable in portals and search applications, you need to link one of its search sources to the search profile used by each portal or search application. You can use the connector's default search source or create your own custom search sources.

Search sources for predefined web sources
:   The Webcrawler external content connector includes a dedicated search source for each predefined web source. Each dedicated search source includes content from the specified predefined web source that's stored in the connector's indexed source. Use these search sources to access searchable content and metadata retrieved from predefined web sources. As an example, searchable content and metadata retrieved from the Google Drive predefined web source is accessible in a search source named Google Drive.

Default search source
:   By default, the system creates a search source that includes all searchable content from your Webcrawler connectors.

Custom search sources
:   You can create your own search sources with filters to specify which content from the connector's indexed source is searchable. To view the connector's indexed source, navigate to AllAI SearchAI Search IndexIndexed Sources. For information about creating search sources, see [Search sources in AI Search](https://servicenow-prod.fluidtopics.net/BzYQ0BGXWlEpJrmmvXanyw "Define search sources to make your searchable content available in search experiences. Choose an indexed source and configure optional filter conditions to limit the content that users can search.").
You can link connector search sources to search profiles from the External Content Admin Home page. For details on this procedure, see [Connect an external content connector to a search profile](https://servicenow-prod.fluidtopics.net/O1_d0dPqWANwyTUPT43BrA "Make results from an external content connector searchable in your portals and search applications by connecting its search sources to your existing search profiles.").

*[\>]: and then