Skip to content

Knowledge Graph

Introduction

Funnelback Knowledge Graph is a framework for exploring entities and their associated relationships. Instead of working with pages and hyperlinks, it operates on data about real life things, and real life relationships, to create a centralised knowledge roadmap. Knowledge Graph will allow users to make decisions based on all available information by supplementing enterprise systems with additional situational awareness and context.

Example of Funnelback Knowledge Graph

Architecture

Knowledge Graph extends the existing collection architecture to generate entities and relationships. A Knowledge Graph can be created for every $COLLECTION and $PROFILE combination.

Technical architecture

The following describes the elements of the Knowledge Graph architecture.

Example of Funnelback Knowledge Graph

  1. Collections - Contains gathered documents which have been indexed and made available for search. Knowledge Graph can enabled on all collections types including web, push and meta collections.
  2. Knowledge Graph database - A graph database which stores entities are relationships.
  3. RESTful APIs - Standard mechanism to query content from the Knowledge Graph database.
  4. Knowledge Graph Widget - A javascript based widget which uses the RESTful APIs to provide an intuitive user interface for navigating complex Knowledge Graphs

Update workflow

The following describes the process which is used by Funnelback to generate the entities and relationships.

  1. Data is gathered using Funnelback collections.
  2. Entities are created for all searchable documents based on special metadata; FUNfkgNodeNames and FUNfkgNodeLabel.
  3. @mention relationships are created where one entity has a reference to another entity.
  4. Custom relationships are created based on the supplied configurations.

Configuration

Funnelback Knowledge Graph can be configured on any new or existing collection using the following steps:

  1. Gathering content
  2. Defining entities
  3. Running updates
  4. Defining relationships
  5. Customising the widget
  6. Integrating the widget

Gathering content

Funnelback provides an array of collections types to gather content from various repositories. For more information, please see collection overview.

Defining entities

There are two key metadata classes which must be configured to control how entities and relationships are created:

  • FUNfkgNodeNames - Specifies the unique names which are used to identify an instance of an entity. Metadata fields mapped to this class can have multiple values which are separated by delimiters. e.g. "Michael Smith;Mike Smith;msmith"

    Note: The delimiter which is used to split values can be specified using the -facet_item_sepchars=<string> indexer option.

  • FUNfkgNodeLabel - Defines the label (also know as the entity type) and provides a means to group related entities e.g. People, Courses, Documents etc.

Both FUNfkgNodeNames and FUNfkgNodeLabel can be defined using the metadata mapping user interface.

Example of metadata mappings for Funnelback Knowledge Graph

For more information please see metadata classes.

Tip: During the metadata mapping process, it may be useful to prefix metadata classes with the entity type. This will make it easier to configure relationships, labels and templates.

Modifying and supplementing the data source

In some cases, the target data source may be less that ideal such as having data in an invalid format or missing key values. In situations like these, external metadata and custom filters implemented in groovy can be used.

External metadata

External metadata can be used to inject FUNfkgNodeLabel values to quickly group related web documents without needing to modify the document source.

e.g.

www.example.com/books FUNfkgNodeLabel:"product"
www.example.com/cds FUNfkgNodeLabel:"product"
www.example.com/writers FUNfkgNodeLabel:"author"

All documents under the books and cds paths will be created as product entities. Similarly, all documents under the writers paths will be created as author entities.

Custom filters

Custom filters implemented in groovy can be used if more complicated actions are required where data needs to modified or santised so that it is in the correct format.

More information can be found on the writing filters page.

Running updates

To start an update of Knowledge Graph:

  • Open the administration user interface, navigate to the Graph tab and press the Update Knowledge Graph button.
  • Select the profile which is to be updated (either the preview or live view).

The progress of the update can be found in the collection's log folder. Steps to view the logs can be found below:

  • Using the administration user interface, navigate to the Administer tab.
  • Select Browse Log Files.
  • Underneath the $COLLECTION / Collection logs files section, click on knowledge_graph.$PROFILE.log to view the Knowledge Graph logs.

The following is an example of the logs:

2018-11-10 21:17:19,193 [main] INFO  integration.Main - Graph statistics before the update
2018-11-10 21:17:19,429 [main] INFO  integration.PrintStatistics - Number of [superhero] nodes: 7
2018-11-10 21:17:19,430 [main] INFO  integration.PrintStatistics - Number of [document] nodes: 6
2018-11-10 21:17:19,432 [main] INFO  integration.PrintStatistics - Number of [movie] nodes: 5
2018-11-10 21:17:19,448 [main] INFO  integration.PrintStatistics - Number of mentions relationships: 19
2018-11-10 21:17:24,157 [main] INFO  integration.Main - Processing of FKG on super-heroes-web is complete!
2018-11-10 21:17:24,170 [main] INFO  integration.Main - Graph statistics after the update
2018-11-10 21:17:24,175 [main] INFO  integration.PrintStatistics - Number of [superhero] nodes: 7
2018-11-10 21:17:24,175 [main] INFO  integration.PrintStatistics - Number of [document] nodes: 6
2018-11-10 21:17:24,175 [main] INFO  integration.PrintStatistics - Number of [movie] nodes: 5
2018-11-10 21:17:24,179 [main] INFO  integration.PrintStatistics - Number of mentions relationships: 19

The log files can also be found on the server in the following location:

$SEARCH_HOME/data/$COLLECTION/logs/knowledge_graph.$PROFILE.log

where

  • $SEARCH_HOME - The location where Funnelback is installed
  • $COLLECTION - The ID of the collection
  • $PROFILE - The ID of the profile

Scheduling automatic updates

Scheduling regular updates can be done by adding the following to the server's native task scheduler (e.g. cron or Windows task scheduler):

curl --user $USERNAME:$PASSWORD -X POST --header 'Content-Type: application/json' -d '{ "collection": "$COLLECTION_ID", "profile": "$PROFILE_ID", "profileView": "$PROFILE_VIEW"}' 'https://$FUNNELBACK_SERVER_HOSTNAME:8443/admin-api/task/v1/queue/UPDATE_KNOWLEDGE_GRAPH'

where

  • $USERNAME - the name of an admin user who can run the knowledge graph update

  • $PASSWORD - the password of the admin user above

    Note: You may prefer to instead create an access token which can be separately revoked.

  • $COLLECTION_ID - The collection ID for the knowledge graph collection

  • $PROFILE_ID - The profile ID for which knowledge graph should be updated

  • $PROFILE_VIEW - Either 'live' or 'preview' depending on which profile view should be updated.

Defining relationships

Knowledge Graph has two types of relationships; @mentions relationships which are automatically created by Funnelback and "custom relationships" which are defined by configuration within knowledge graph.

@Mentions Relationships

Funnelback will automatically create @mentions relationships whenever one entity makes a reference to another entity. It does this by scanning the contents of every searchable document and looks for references to other entities based on the unique names as defined by FUNfkgNodeNames. Note: This process will ignore case treating "MSMITH" equivalent to "msmith".

Tip: The -all_summary_text=<boolean> query processor option can be used to review the content of any document.

e.g. Given the following XMLs which represents entities:

Superhero entity - S1

<document>
    <id>S1</id>
    <FUNfkgNodeLabel>Superhero</FUNfkgNodeLabel>
    <FUNfkgNodeNames>Bruce Wayne;Batman</FUNfkgNodeNames>
    <Name>Bruce Wayne</Name>
    <Alias>Batman</Alias>
</document>

Movie entity - M5

<document>
    <id>M5</id>
    <FUNfkgNodeLabel>Movie</FUNfkgNodeLabel>
    <FUNfkgNodeNames>Dawn of Justice</FUNfkgNodeNames>
    <Title>Batman v Superman: Dawn of Justice</Title>
    <Description><![CDATA[Batman v Superman: Dawn of Justice is a 2016 American superhero film featuring the DC Comics characters Batman and Superman.]]></Description>
</document>

Knowledge Graph will automatically create a @mentions relationship between S1 and M5 as one of the names of S1 (Batman) appears in the "Description" of M5.

Only one @mentions relationship is ever created between any two entities. i.e. In the scenario that one entity has multiple references to another, only one @mentions relationship will be created.

Tip: When working with XML documents, knowledge graph will only create @mentions relationships based on fields mapped as Indexable document content

Custom Relationships

Custom relationships provide a connection between two entities referred to as the source and target.

To add a custom relationship, open the administration user interface, navigate to the Graph tab and press the Edit Relationships button.

Note: A service needs to be created on the profile before various controls for Knowledge Graph are enabled.

Every relationship is directional:

  • Outgoing relationships are created from a source entity to a target entity
  • Incoming relationships are created from a target entity to a source entity

This is important when it comes to customising the knowledge graph widget presentation.

Example

Given entities of type person and document, we can configure a created relationship between them:

outgoing

person -> created -> document

incoming

document -> created (by) -> person

where person is the source entity and document is the target entity.

Customising the widget

Funnelback Knowledge Graph ships with a widget which can be easily customised using dedicated configuration screens; UI Labels and UI Templates.

UI Labels

Funnelback Knowledge Graph allows the user to control the display name of entities (and their associated properties/metadata) and relationships of the widget without needing to modify HTML templates directly. This is achieved using the UI Labels configuration screen.

To change the display name of entities and relationships, open the administration user interface, navigate to the Graph tab and press the Edit UI Labels button.

The UI Labels configuration screen provides the ability to configure the display labels of three main components of the Knowledge Graph widget.

CategoryDescriptionExample
MetadataThe display labels of each metadata of an entityMetadata class a is to be displayed as Author.
RelationshipsThe display label of the custom relationshipsIf a ActedIn relationship was created between an actor (source) and a movie (target), the outgoing label is Acted in and the incoming label is Cast member.
TypeThe display name for each entity typewordDocument is to be displayed as Microsoft office word document.

UI Templates

Similarly to UI Labels, Funnelback Knowledge Graph provides the ability to control how each entity is displayed without needing to modify HTML templates directly. This is achieved using the UI Templates configuration screen.

To control how each entity type is presented, open the administration user interface, navigate to the Graph tab and press the Edit UI Templates button.

The UI Template configuration screen provides the ability to configure the different presentational aspects of Knowledge Graph widget which includes:

  • Main display - How each entity type is to be present when it is in focus.
  • List View - How each entity type is presented when it appears as a related entity.

More information can be found within the inline help of the UI Template configuration screen.

Integrating the widget

Knowledge Graph can be included directly into any website or web application through the inclusion of javascript and style sheets (css). Detailed steps can be found on the Knowledge Graph Frontend page.

Previewing the Knowledge Graph widget

To preview knowledge graph, open the administration user interface, navigate to the Graph tab and press the Browse Knowledge Graph button. Alternatively, it can be previewed directly using the following url:

https://<FUNNELBACK-SERVER>/s/knowledge-graph/index.html?collection=<COLLECTION>&profile=<PROFILE>

Caveats

  • Please be aware that IP based restrictions and document level security are not supported by funnelback knowledge graph.

top

Funnelback logo
v15.18.0