Debugging failed updates

Funnelback updates can fail for a number of reasons. These tips should assist in doing some preliminary debugging on what caused an update to fail.

The first thing to do is look at the update log for the collection that failed and determine at what point the update failed. Look for error lines.

You can access this file from the administration console by browsing the log files from the Administer tab, or from the command line from the $SEARCH_HOME/data/<collection name>/log folder. The main update log is located in the collection log files section and is called update-<collection name>.log.

Once you have found the source of the error you may need to check the relevant log file for the particular phase of the update where the failure occurred. These fill be located in the offline log files section.

Common errors include:

• Failed to access seed page: For some reason Funnelback was unable to access the seed page so the whole update failed (as there is nothing to crawl). Look at the offline crawl logs and url error log for more information. Cause could be a timeout, or a password expiring if you are crawling with authentication.
• Failed changeover conditions: After building the index a check is done comparing with the previous index. If the index shrinks below a threshold then the update will fail. This can occur if one of the sites was down when the crawl occurred, or if there were excessive timeouts, or if the site has shrunk (eg. because it has been redeveloped or part of it archived). If a shrink in size is expected you can run an advanced update and swap the views.
• Failures during filtering: Occasionally the filtering process crashes causing an update to fail. The offline filter log may provide further information to the cause.
• Swap views – No such file or directory. Is <offline folder> being used by another program?: Under Windows swap views renames the offline and live folders so if something (eg. explorer in a remote desktop session) has a lock on the folder when it attempts to change then the update will fail. Solution – make sure all folders/files in the data folders are closed then select the collection, run an advanced update and swap the views.
• Lock file exists: The update could not start because a lock file was preventing the update. This could be because another update on the collection was running; or a previous update crashed leaving the lock files in place. Check to see if the lock files exist (at D:\funnelback\data\\log). If there is a .lock and .state file and no currently running update then these files can be safely removed.
• Open data folder: A data\ folder was left open (eg. in a remote desktop session) so the swap process was unable to rename the folders to complete the update. This is a common error when running Funnelback under Windows.
• Failures during indexing: Have a look at the offline index log for more details.