Grabbers posted here must not violate the source site's TOS (Terms of Service)

Grabbers do not have to be based on an API and can be a screen scraper as long as they do not violate TOS. Just be prepared for more frequent support requests as site changes will often break a screen scraping grabber.

Grabber Script Standards

All grabbers must at least support a search and/or a tree view option. Supporting both is optional.

Grabbers expect that the environment is set to UTF-8.

A search grabber by default should return a maximum of 20 video items per page. This usually keeps the response time to a reasonable duration.

A search grabber must support paging with a specific page number specified through the "-p" or "--pagenumber" command line option. The grabber must interpret the page number into what the source api requires. Sometimes it is a page number and sometimes a video index (e.g. 21st video of the search results).

A grabber's file name must have a matching image file name for an icon to be displayed by the Netvision plug-in.

Required Command line options

Command Line Option

Arguments

Example

Description

-v

None

/usr/share/mythtv/mythnetvision/scripts/youtube.py -v

This option is used by Netvision to identify the grabber's name and its capabilities, which can include search and/or tree view

The format is grabber title then a "|" character followed by the grabber's supported options search and/or tree view

Tag identifies the start of a channel. The channel must eventually be closed with a "</channel>"

title

The title of the source of the meta data which will be displayed by the Netvision plug-in

link

URL of the home page for the source of the meta data

description

Descriptive text relevant for the source of the meta data

numresults

The number of results returned from a search. Usually this is total items or total pages of items. This value is not used in a tree view.

returned

This number reflects the maximum number of items per page or the number returned when a search does not find a full page worth of video items. This value is not used in a tree view.

startindex

The start index of the next page or next video item. If this number is less than or equal to the number of results the Netvision plug-in will not allow the user to request a next page. This value is not used in a tree view.

Directory tag format

Directories include a label and icon image used to organise the tree view.

Directories can be nested

This is the method for the grabber to telling Netvision how to display/organize the video items.

Identifies the start of a video item. The video item must eventually be closed with a "</item>"

title

Descriptive label for the video

author

The author who submitted the video to the source web site. Often this is submitter's user id.

pubDate

The date that the video was submitted to the site. The format of "Thu, 17 Dec 2009 13:51:34 GMT" must always be used. If you cannot retrieve the timezone, simply specify GMT. If you do not know the time then set it to all zeros "00:00:00".

description

The text description of the video. A description should not contain any HTML or line-feeds.

link

A URL to the web page containing the video or directly to the video player. If this link is not identical to the "media:content url" tag then that indicates to Netvision that the "media:content url" file can be downloaded. Unless the source site TOS allows a video file to be downloaded then the "link" and "media:content url" URLs should be identical.

media:group

Indicates the start of a video's thumbnail and meta data. The item must eventually be closed with a "</media:group>"

media:thumbnail url

A URL to the thumbnail image for the video file. The source site does not always have a thumbnail for a video. When there are multiple thumbnails choose the first one over 200 pixels wide as it will display well but keep download times reasonable.

media:content

The main tag for the video link and video meta data

url

A URL to the video file/player/web page. This link should only differ from the URL "link" tag when the URL points to a downloadable video file.

duration

The duration of the video in seconds. Some sources use a format of "MM:SS". This value must be converted to seconds.

Return code is zero for any processing that completed without a script abort. This includes a search or treeview that did not return any results.

Return code of one if an error forced the premature termination of the grabber

If a grabber script skips bad data during process an error about the bad data should be sent to stderr but a return code of zero would still be returned indicating the grabber completed its processing successfully.

Python grabber development

Although a grabber can be developed in any computer language that can support the grabber standards a framework for grabber development is available for python.
If the framework is adopted by your python grabber you be able to focus on source site specific processing without having to recode generic grabber functions.
See this wiki link for details Netvision python grabber development