Skip to content

crawler.max_dir_depth (collection.cfg setting)

Description

This option sets the limit for the number of directories in a valid URL. The crawler will ignore all URLs that have more than this number of directories.

Note: this limit is not checked for dynamic URLs, e.g. ones containing a '?'.

Default value

crawler.max_dir_depth=15

Examples

crawler.max_dir_depth=2

Will have the following effect:

http://host/one/two/ok
http://host/one/two/three/fails

See also

top

Funnelback logo
v15.18.0