Skip to content

Hook scripts

Introduction

The Modern UI allows you to insert hook scripts at certain stages of the search process. These hook scripts allow you to perform actions over the input parameters before the query is submitted to the query processor or transform the results after the query has been run but before the results are displayed to the user.

The hook scripts must be written in the Groovy programming language. Groovy runs on top of the Java Virtual Machine and offers the same power of Java classes but with more flexibility. The hook scripts will be automatically re-compiled when a change is detected in them.

Search process / lifecycle

Search-process-lifecycle.png

The Modern UI provides various hook points where scripts can be inserted in the search lifecycle. The search lifecycle runs the following phases:

  1. A query is submitted on a specific collection and passes an input processing phase. This phase will configure any relevant parameter depending on the collection configuration, apply faceted navigation constraints, transform some meta_* or query_* parameters into query expressions, etc.
  2. Extra searches (if any) are prepared, cloning the environment from the main search.
  3. Once the input parameters have been prepared the search results are fetched, both for the main search and the extra searches. The query processor is run with the input parameters and return results.
  4. The results then passes an output processing phase. This phase will transform the result data according to the configuration, apply click tracking URLs, generate faceted navigation data from the metadata counts, etc.
  5. Once the results have been transformed and the Data Model is ready, it's passed to FreeMarker which will render the search results page.

The hook scripts can be placed:

  • At the very beginning of the lifecycle, before the input phase. This hook is called pre_process
  • At the end of the input phase when extra searches have been setup, but not yet submitted. This hook is called extra_searches and allows you to modify each extra search before it's run.
  • Just before the query processor is run, but after the input phase: This hook is called pre_datafetch
  • Just after the query processor is run for the main search, but before the output phase: This hook is called post_datafetch
  • At the very end of the process, just before the search results page is generated: This hook is called post_process

Additionally, a pre_cache hook will be called prior to displaying any cached copies.

Creating hook scripts

The hook scripts for a collection must be created in the collection configuration directory (they can be created using the file-manager). The name of the files starts with hook_ and uses the name of the hooks previously mentioned. It's suffixed with .groovy.

You can create either 1, 2, ... or the all the possible hooks scripts for a collection:

  • $SEARCH_HOME/conf/[collection]/hook_pre_process.groovy
  • $SEARCH_HOME/conf/[collection]/hook_extra_searches.groovy
  • $SEARCH_HOME/conf/[collection]/hook_pre_datafetch.groovy
  • $SEARCH_HOME/conf/[collection]/hook_post_datafetch.groovy
  • $SEARCH_HOME/conf/[collection]/hook_post_process.groovy
  • $SEARCH_HOME/conf/[collection]/hook_pre_cache.groovy

Hook scripts immediately take effect as soon as they're present in a collection's configuration directory. Any changes in them will be immediately taken into account.

Writing hook scripts

Search lifecycle hook scripts

The hook scripts have access to the same data model as the form files. If you're already comfortable with writing search forms for the Modern UI using FreeMarker, writing hook scripts shouldn't be difficult.

The only difference reside in accessing the root data model objects like question and response. While they're directly available as is from a FreeMarker template, in hook scripts they're grouped under a transaction root object. So for example if you need to access the ID of the collection being searched you must use:

transaction.question.collection.id

The hook scripts doesn't need to implement a specific interface or to have a specific header. You can start writing the body of your script within the first line of the .groovy file. For example the following 1-line hook script appends "cat" to the user submitted query:

transaction.question.query += " cat"

Cache hook scripts

Because the pre_cache hook script doesn't run in the context of a search transaction, the data model is slightly different. Please consult the cached copies documentation for more details.

Debugging hook scripts

Compilation problems

If your script doesn't compile for any reason the error will be logged in the Modern UI log file: $SEARCH_HOME/web/logs/modernui.(Public/Admin).log.

The most common compilation problems are:

  • Syntax errors.
  • Typos when accessing the data model, such as using transaction.question.colection instead of transaction.question.collection.

Message logging

If you need to log messages from within your hook script you can do so by using an logger object. A logger object has a name and can log messages with various severity levels (debug, warn, info, error or fatal). To use a logger object from within your script use the following code:

def logger = org.apache.logging.log4j.LogManager.getLogger("com.funnelback.MyHookScript")
...
logger.info("The query is: " + transaction.question.query);
...
logger.fatal("No results were found")

The log messages will be written in one the Modern UI log files:

  • $SEARCH_HOME/web/logs/modernui.(Public/Admin).log.
  • $SEARCH_HOME/data/<collection>/log/modernui.(Public/Admin).log (Linux)
  • $SEARCH_HOME/web/logs/modernui.<collection>.(Public/Admin).log (Windows)

Note: The default configuration of log messages is set to output the error level of above, except for the loggers belonging to the com.funnelback namespace which output info messages. That mean that your messages won't appear unless:

  • You use logger.error() or logger.fatal()
  • You use logger.info() and your logger belongs to the Funnelback namespace: def logger = org.apache.logging.log4j.LogManager.getLogger("com.funnelback.MyHookScript")

The logging configuration and levels for the Modern UI can be edited in $SEARCH_HOME/web/conf/modernui/log4j.properties.

Examples

Please consult the data model documentation if you're not familiar with the data model contents (question, response, resultPacket, etc.).

Transforming each result

This example iterates over each result and changes the host name on each live URL using a regular expression. This is a post_datafetch hook because it should happen before the URLs are updated with click tracking information during the output phase.

transaction?.response?.resultPacket?.results.each() {
  // In Groovy, "it" represents the item being iterated
  it.liveUrl = (it.liveUrl =~ /www.badhost.com/).replaceAll("www.correcthost.com")
}

Processing additional input parameters

This example takes a country query string parameter and applies a query constraint on the p metadata class if it exists. Calling http://server.com/s/search?collection=test&query=travel&country=Australia will result in the query travel |p:"Australia" being run.

It's a pre_process hook because it should be run before the meta_* parameters are transformed to query expressions during the input phase.

def logger = org.apache.logging.log4j.LogManager.getLogger("com.funnelback.hooks.CountryParamHook")
def q = transaction.question  
// Set the input parameter 'meta_p_phrase_sand' to the value
// of the 'country' query string parameter, if it exist
if (q.inputParameterMap["country"] != null) {
  q.inputParameterMap["meta_p_phrase_sand"] = q.inputParameterMap["country"]
  logger.info("Applied country constraint: " + q.inputParameterMap["country"])
}

Alternatively it could be done as a pre_datafetch hook by providing directly a query expression:

if (q.inputParameterMap["country"] != null) {
  q.metaParameters.add("|p:\"" + q.inputParameterMap["country"] + "\"")
}

Transforming results metadata

This script takes the value of the d and t metadata, concatenates them and put them in the x metadata, for each result. It can be either a post_datafetch or a post_process hook since it doesn't depend on any transformation done in the output phase.

transaction?.response?.resultPacket?.results.each() {
      // In Groovy, "it" represents the item being iterated
      if (it.metaData["d"] != null && it.metaData["t"] != null) {
        it.metaData["x"] = "Document " + it.metaData["t"] + " created on: " + it.metaData["t"]
      }
   }

Modifying an extra search query

This script modify an extra search to update its query with a metadata constraint. The resulting query expression for this extra search will be a:shakespeare. This example needs to be in a extra_searches hook since it needs to be run just before the extra search is actually submitted.

if ( transaction.extraSearchesQuestions["myExtraSearch"] != null ) {
  def searchQuestion = transaction.extraSearchesQuestions["myExtraSearch"]
  searchQuestion.query += " a:shakespeare"
}

This hook makes use of the extraSearchesQuestion data model node which contains a SearchQuestion object per configured extra search.

Transforming CGI parameters

Hook scripts can be used to perform CGI parameter transformations. The following example will change the gscope1 parameter depending on the selected form:

if (transaction.question.inputParameterMap["form"] == "scoped") {
  transaction.question.inputParameterMap["gscope1"] = "12"
}

This is equivalent to the following CGI Transform:

form=scoped => gscope1=12

Pre-selecting a facet

Pre-selecting a facet is done by injecting the relevant query string parameter in the question. This needs to be done in the pre_process hook script so that the injected parameters are transformed in the relevant query constraints before the query processor is called:

transaction.question.inputParameterMap["f.Location|X"] = "Canberra"

If you want to pre-select two categories simultaneously for a single facet you must use rawInputParameters which allow multiple values to be set (See the data model documentation for more details about inputParameterMap and rawInputParameters:

transaction.question.rawInputParameters["f.Location|X"] = ["Canberra", "Sydney" ]

The syntax of the parameter is f.<FacetName>|<constraint>=<value> where FacetName and constraint are defined in your faceted navigation configuration. The constraint is either a metadata class name such as X, Y, ... or a gscope number, for example f.Color|1=red.

Query another collection and process the transaction

To perform extra searches using the built in feature is recommended, however you might want to perform an additional query manually for various reasons, such as running an extra query for each result of the main search. To do so you need to fire an HTTP request to the Modern UI and transform the resulting XML into Groovy objects.

import com.funnelback.publicui.xml.SearchXStreamMarshaller
def logger = org.apache.logging.log4j.LogManager.getLogger("com.funnelback.RequestModernUIHookScript")

// Create instance of utility class that will transform our XML back into Java/Groovy objects
def marshaller = new SearchXStreamMarshaller()
// Initialise the marshaller
marshaller.afterPropertiesSet()

// Request the XML for the additional search
// Beware of *not* requesting the same collection or it will get stuck in an infinite loop
def extraTransaction
new URL("http://localhost:8080/s/search.xml?collection=funnelback_documentation&query=best").withInputStream {
  // Unmarshal the XML into a transaction object
  stream -> extraTransaction = marshaller.unmarshalInputStream(stream)
}

// Put the extra transaction in the customData map
transaction.response.customData["mySearch"] = extraTransaction

Read values from a custom configuration file

This example makes use of the Java .properties format. First define your configuration in SEARCH_HOME/conf/<collection>/myConfig.properties:

results.titleSuffix=Suffix
postcode.2000=Sydney
postcode.3000=Melbourne

Then use the following hook script:

def props = new Properties()
new File(transaction.question.collection.configuration.configDirectory, "myConfig.properties").withInputStream {
  stream -> props.load(stream)
}

transaction?.response?.resultPacket?.results.each() {
  // Append suffix to title
  it.title = it.title + props["results.titleSuffix"]
  // Put city name in Z from postcode metadata in X
  it.metaData["Z"] = props["postcode."+it.metaData["X"]]
}

top

Funnelback logo
v15.12.0