Skip to content

crawler.max_parse_size (collection.cfg setting)

Description

The crawler will stop parsing documents larger than the specified value (in megabytes), and their content will be truncated. This only applies to MIME types listed in the crawler.parser.mimeTypes parameter (e.g. HTML, text, XML). Here parsing refers to link extraction from these file types.

Default value

crawler.max_parse_size=10

Examples

Increase the limit to fifteen megabytes.

crawler.max_parse_size=15

See also

top

Funnelback logo
v15.16.0