crawler.monitor_delay_type (collection.cfg setting)
This parameter specifies what type of delay the webcrawler will use between each request to a webserver. There are two types of delay supported:
- Dynamic: Calculate delay based on time taken for previous request to this server
- Fixed: Use the same fixed delay after each requestThe default and more efficient method is to use a dynamic delay.
Change to a fixed delay: