Skip to content

YouTube collections

Introduction

YouTube is a social media site focused on video sharing.

Please note that your usage of Funnelback to gather content from YouTube must comply with YouTube's terms of service.

Getting API key

To be able to gather content from YouTube you must first use your Google account to create an API key. Google provides instructions on creating an API key which will provide the most up-to-date instructions for creating the key.

Configuration options

YouTube collections support the following settings:

  • youtube.api-key: API key retrieved from the Google API console.
  • youtube.channel-ids: Comma delimited list of YouTube channel IDs to crawl.
  • youtube.liked-videos: Boolean flag to enable fetching of videos liked by a channel ID. It'll apply to all channels provided in youtube.channel-ids.
  • youtube.playlist-ids: Comma delimited list of YouTube playlist IDs to crawl.

Metadata mappings

YouTube collections includes a number of default YouTube specific metadata mappings:

Class IDTypeBehaviourExplanationMetadata fields included
authortextcontent/com.funnelback.socialmedia.youtube.v3.YouTubeXmlRecord/channelTitle
authorUrltextdisplay/com.funnelback.socialmedia.youtube.v3.YouTubeXmlRecord/channelUrl
ctextcontentDescription/com.funnelback.socialmedia.youtube.v3.YouTubeXmlRecord/description
categorytextcontent/com.funnelback.socialmedia.youtube.v3.YouTubeXmlRecord/category
commentCountnumberN/A/com.funnelback.socialmedia.youtube.v3.YouTubeXmlRecord/commentCount
ddatedateDate/com.funnelback.socialmedia.youtube.v3.YouTubeXmlRecord/publishedDate
dislikeCountnumberN/A/com.funnelback.socialmedia.youtube.v3.YouTubeXmlRecord/dislikes
durationtextdisplay/com.funnelback.socialmedia.youtube.v3.YouTubeXmlRecord/durationPretty
durationInSecondsnumberN/A/com.funnelback.socialmedia.youtube.v3.YouTubeXmlRecord/durationInSeconds
favoriteCountnumberN/A/com.funnelback.socialmedia.youtube.v3.YouTubeXmlRecord/favoriteCount
identifiertextdisplay/com.funnelback.socialmedia.youtube.v3.YouTubeXmlRecord/videoId
imageSmalltextdisplayImage URL - small/com.funnelback.socialmedia.youtube.v3.YouTubeXmlRecord/thumbnails/veryDefault/thumbNailUrl
imageMediumtextdisplayImage URL - medium/com.funnelback.socialmedia.youtube.v3.YouTubeXmlRecord/thumbnails/medium/thumbNailUrl
imageLargetextdisplayImage URL - large/com.funnelback.socialmedia.youtube.v3.YouTubeXmlRecord/thumbnails/high/thumbNailUrl
latLonggeospatial x/y co-ordinateN/A/com.funnelback.socialmedia.youtube.v3.YouTubeXmlRecord/latLong
likeCountnumberN/A/com.funnelback.socialmedia.youtube.v3.YouTubeXmlRecord/likes
ttextcontentTitle/com.funnelback.socialmedia.youtube.v3.YouTubeXmlRecord/title
viewCountnumberN/A/com.funnelback.socialmedia.youtube.v3.YouTubeXmlRecord/viewCount

Use the -SF query processor option to access these metadata fields on the search response and in the templates (i.e. -SF=[author,viewCount]).

Limits

Please note that YouTube applies limits to the volume of content which can be retrieved from their APIs, and so in the case of large channels Funnelback may be unable to gather all historical content.

Working with the fetched data

Funnelback will crawl YouTube and convert responses into XML. You can use the metadata customisation tool to map elements to metadata classes.

Note: To preview the crawled records please enable debug mode by setting youtube.debug=true in collection.cfg file.

The XML that Funnelback generates for a YouTube collection is as follows:

<com.funnelback.socialmedia.youtube.v3.YouTubeXmlRecord>
  <url>http://www.youtube.com/watch?v=video_id&amp;feature=youtube_gdata_player</url>
  <title>some title</title>
  <description>some description</description>
  <videoId>video_id</videoId>
  <viewCount>1</viewCount>
  <commentCount>0</commentCount>
  <thumbnails>
    <veryDefault>
      <thumbNailUrl>https://i.ytimg.com/vi/video_id/default.jpg</thumbNailUrl>
      <thumbNailUrlWidth>120</thumbNailUrlWidth>
      <thumbNailUrlHeight>90</thumbNailUrlHeight>
      <thumbNailUrlExtras class="linked-list"/>
    </veryDefault>
    <medium>
      <thumbNailUrl>https://i.ytimg.com/vi/video_id/mqdefault.jpg</thumbNailUrl>
      <thumbNailUrlWidth>320</thumbNailUrlWidth>
      <thumbNailUrlHeight>180</thumbNailUrlHeight>
      <thumbNailUrlExtras class="linked-list"/>
    </medium>
    <high>
      <thumbNailUrl>https://i.ytimg.com/vi/video_id/hqdefault.jpg</thumbNailUrl>
      <thumbNailUrlWidth>480</thumbNailUrlWidth>
      <thumbNailUrlHeight>360</thumbNailUrlHeight>
      <thumbNailUrlExtras class="linked-list"/>
    </high>
    <stndard>
      <thumbNailUrl>https://i.ytimg.com/vi/video_id/sddefault.jpg</thumbNailUrl>
      <thumbNailUrlWidth>640</thumbNailUrlWidth>
      <thumbNailUrlHeight>480</thumbNailUrlHeight>
      <thumbNailUrlExtras class="linked-list"/>
    </stndard>
    <maxres>
      <thumbNailUrl>https://i.ytimg.com/vi/video_id/maxresdefault.jpg</thumbNailUrl>
      <thumbNailUrlWidth>1280</thumbNailUrlWidth>
      <thumbNailUrlHeight>720</thumbNailUrlHeight>
      <thumbNailUrlExtras class="linked-list"/>
    </maxres>
  </thumbnails>
  <publishedDate>2016-08-16T23:14:04.000Z</publishedDate>
  <category/>
  <durationInSeconds>378</durationInSeconds>
  <durationPretty>00:06:18</durationPretty>
  <likes>1</likes>
  <dislikes>0</dislikes>
  <favoriteCount>0</favoriteCount>
  <channelTitle>channel_name</channelTitle>
  <channeld>channel_id</channeld>
  <channelUrl>https://www.youtube.com/channel/channel_id</channelUrl>
  <embedHtml>&lt;iframe width=&quot;480&quot; height=&quot;270&quot; src=&quot;//www.youtube.com/embed/video_id&quot; frameborder=&quot;0&quot; allow=&quot;autoplay; encrypted-media&quot; allowfullscreen&gt;&lt;/iframe&gt;</embedHtml>
  <extras class="linked-list"/>
</com.funnelback.socialmedia.youtube.v3.YouTubeXmlRecord>

top

Funnelback logo
v15.18.0