Skip to content

crawler.request_header_url_prefix

Optional URL prefix to be applied when processing the crawler.request_header parameter

Key: crawler.request_header_url_prefix
Type: String
Can be set in: collection.cfg

Description

This parameter can be used to specify an optional URL prefix to be applied to the crawler.request_header parameter.

If defined and non-empty then only URLs which start with this prefix will be requested with the additional request header. Otherwise the request header will be sent with all outgoing requests if defined.

Default Value

(Empty)

Examples

Specify that the defined crawler.request_header should only be sent in requests to URLs which start with http://example.com/forums/

crawler.request_header_url_prefix=http://example.com/forums/

See Also

top

Funnelback logo
v15.22.0