Skip to content

crawler.max_individual_frontier_size

Specifies the maximum size of an individual frontier.

Key: crawler.max_individual_frontier_size
Type: Integer
Can be set in: collection.cfg

Description

This option limits the number of URLs stored in a frontier. If new URLs are discovered during a crawl but can not be queued because the frontier is full, they will be dropped (though they may be found again later in the crawl).

Default Value

If not defined, it is unlimited.

Examples

crawler.max_individual_frontier_size=1000

See Also

top

Funnelback logo
v15.24.0