crawler.form_interaction_in_crawl (collection.cfg setting)
This parameter controls whether the crawler will submit form login details during a crawl (in-crawl authentication) instead of using a pre-crawl authentication step.
The login details are specified in a form_interaction.cfg file.
To turn on in-crawl interaction:
- This setting changes the semantics of the fields in the configuration file - please read the documentation on the form_interaction.cfg file for details.
- It will also mean that cookies will not be explicitly renewed during the course of the crawl. Instead the crawler will submit the required form details when an appropriate form is encountered during the course of the crawl.