Skip to content

crawler.revisit.num_times_unchanged_threshold

Threshold for the number of times a page has been unchanged when deciding whether to revisit it.

Key: crawler.revisit.num_times_unchanged_threshold
Type: Integer
Can be set in: collection.cfg

Description

This parameter specifies a threshold to use when looking at how many times the content of a given URL has been unchanged over the course of multiple crawls.

A particular revisit policy might decide that if the number of times a URL has been unchanged is higher than the threshold then a revisit would not be required.

Increasing this threshold would result in more revisits (HTTP HEAD and/or GET requests) being performed.

Default Value

crawler.revisit.num_times_unchanged_threshold=5

See Also

top

Funnelback logo
v15.24.0