Skip to content

crawler.revisit.num_times_unchanged_threshold (collection.cfg setting)

Description

This parameter specifies a threshold to use when looking at how many times the content of a given URL has been unchanged over the course of multiple crawls.

A particular revisit policy might decide that if the number of times a URL has been unchanged is higher than the threshold then a revisit would not be required.

Increasing this threshold would result in more revisits (HTTP HEAD and/or GET requests) being performed.

Default value

crawler.revisit.num_times_unchanged_threshold=5

See also

top

Funnelback logo
v15.18.0