Skip to content

crawler.allow_concurrent_in_crawl_form_interaction

Enable/disable concurrent processing of in-crawl form interaction.

Key: crawler.allow_concurrent_in_crawl_form_interaction
Type: Boolean
Can be set in: collection.cfg

Description

This experimental option allows concurrent form interaction processes to be prevented, which may work around problems in some web single sign-on environments.

When using in-crawl form interaction, the web crawler normally processes required form interaction while fetching the page which requires it, which may mean that multiple in-crawl form interaction processes occur concurrently.

In some web authentication environments, this may pose a problem because the web crawler shares cookies between all crawler threads. For example, if a separate SSO authentication server relies on cookies to control the post-authentication return page, two parallel logins may not behave as expected.

Please note that the locking necessary to prevent concurrent form interaction may have a detrimental effect on web crawler scalability/performance.

Note: this option is experimental it may be subject to major changes without backwards compatibility in future versions of Funnelback.

Default Value

crawler.allow_concurrent_in_crawl_form_interaction=true

Examples

Prevent concurrent form interaction processing

crawler.allow_concurrent_in_crawl_form_interaction=false

See Also

top

Funnelback logo
v15.24.0