Skip to content

crawler.send-http-basic-credentials-without-challenge

Specifies whether HTTP basic credentials should be sent without the web server sending a challenge.

Key: crawler.send-http-basic-credentials-without-challenge
Type: Boolean
Can be set in: collection.cfg

Description

Historically, the Funnelback web crawler has sent configured HTTP Basic credentials to web sites without requiring a 401 challenge from the website first. This behavior is higher performance, and avoids issues with sites that do not issue challenges as expected, but risks unnecessarily sending HTTP Basic credentials to URLs which do not require them.

This option allows this behavior to be switched between waiting for a challenge or blindly sending credentials.

Default Value

crawler.send-http-basic-credentials-without-challenge=true

Examples

crawler.send-http-basic-credentials-without-challenge=false

Will cause credentials to be sent only after a 401 response is received for a request to the same URL without credentials.

See Also

top

Funnelback logo
v15.24.0