Skip to content

crawler.request_timeout

Timeout for HTTP page GETs (milliseconds)

Key: crawler.request_timeout
Type: Integer
Can be set in: collection.cfg

Description

This is the maximum number of milliseconds that the crawler will wait for a HTTP GET request to return. If the web server does not start to return the page in question before this timeout, the crawler assumes the page was unavailable.

Note: This value is multiplied by 4 to give the parser (link extraction) timeout.

Default Value

crawler.request_timeout=15000

See Also

top

Funnelback logo
v15.24.0