Skip to content

crawler.frontier_hosts (collection.cfg setting)

Description

This option allows you to specify which hosts crawlers are running on when using a DistributedFrontier (distributed crawling). Each host will be responsible for its own subset of the web servers being crawled.

Default value

(empty)

Examples

Specify the IP addresses of the hosts that the distributed crawl is running on.

crawler.frontier_hosts=12.67.89.8,12,67.89.9

See also

top

Funnelback logo
v15.18.0