crawler.accept_cookies
This option enables, or disables, the crawler's use of cookies.
Key: crawler.accept_cookies
Type: Boolean
Can be set in: collection.cfg
Description
Cookies are small text files which are normally stored by a user's web browser to save their preferences and other information when browsing a web site.
In some cases e.g. crawling dynamic websites having cookie support turned on may be useful.
Default Value
Cookie policy. Default is false i.e. do not accept cookies. Requires HTTPClient if true.
crawler.accept_cookies=true
Examples
To disable this feature set it to "false" e.g.
crawler.accept_cookies=false