crawler.request_header (collection.cfg setting)
Send a cookie string:
crawler.request_header=Cookie: phpbb2mysql_data=xyx; phpbb2mysql_sid=123
This cookie information could be got by loading up the relevant website in a web browser and then examining the cookies it tries to set and store.
- If sending cookie strings you should set crawler.accept_cookies to "false", to avoid the cookie strings you are trying to send being overridden.
- You will probably want to use the crawler.request_header_url_prefix parameter as well to limit what URLs the crawler sends these request headers to.