Skip to content

crawler.check_alias_exists

Check if aliased URLs exists - if not, revert back to original URL.

Key: crawler.check_alias_exists
Type: Boolean
Can be set in: collection.cfg

Description

This parameter controls whether to check if an aliased URL exists. During the course of a crawl we may detect a server as an alias of another e.g. home.example.com is an alias of www.example.com.

If this parameter is set to true then we will use a HTTP HEAD request to check if the aliased URL is there before making a full request. If the aliased URL does not exist we will then revert to the original URL.

This results in extra HTTP requests so the default setting is false i.e. we do not normally run this check as it is usually not required.

Default Value

crawler.check_alias_exists=false

See Also

top

Funnelback logo
v15.24.0