crawler.monitor_delay_type
Type of delay to use during crawl (dynamic or fixed).
Key: crawler.monitor_delay_type
Type: MonitorDelayTypes allowed values: dynamic or fixed
Can be set in: collection.cfg
Description
This parameter specifies what type of delay the webcrawler will use between each request to a webserver. There are two types of delay supported:
- dynamic: Calculate delay based on time taken for previous request to this server
- fixed: Use the same fixed delay after each requestThe default and more efficient method is to use a dynamic delay.
Default Value
crawler.monitor_delay_type=dynamic
Examples
Change to a fixed delay:
crawler.monitor_delay_type=fixed