crawler.robotAgent (collection.cfg setting)
This is a string that is used to examine any
robots.txt files that are found to see if there are Funnelback specific restrictions.
The string is case-insensitive, so it will match
User-Agent: funnelback User-Agent: FunnelBACK
The user-agent check is equals, not a substring so it will not match
User-Agent: funnelback crawler
Note: The webcrawler will still obey all rules in the robots.txt file which are meant to apply to all webcrawlers regardless of name.