The Ardour Canvas

The main area of the editor in Ardour is normally referred (by
developers) to as "the canvas". It displays tracks and busses as
arrangements of data of various kinds along a horizontal
timeline. The term "canvas" comes from several GUI toolkits,
which use this word to describe a GUI element on which arbitrary
objects can be placed, stacked on top of each other and moved
around. Another phrase for the same thing is a "scene graph".

Historical Background

Ardour2 and versions of Ardour3 up to and including 3.5.x used a
canvas called "GnomeCanvas" which actually had nothing really to
do with the GNOME project. It had all the features one would
really want from a canvas/scene graph but all drawing was done
using the CPU and the appearance of the canvas was first drawn
into an RGBA buffer in memory before being rendered to some
surface/element known to the native window system. The
"libart_lgpl" library was used for drawing, along with the
option to use direct pixel manipulations.

The performance of the canvas was not bad on Linux, but worse on
OS X where the primitive used to render the RGBA buffer to the
window was not very efficient. The main problem for Ardour with the
GnomeCanvas was that it could not utilize graphics hardware to
do drawing under any circumstances, which meant that drawing
many common "modern" object renderings (notably, gradients)
were not available as basic operations and relied on the CPU,
even if a GPU was available that could the job faster and better.

This made it hard to really move forward with GnomeCanvas as the
basis for any future changes to Ardour's appearance. We did not
want to write our own primitives for gradient rendering, or for
rendering rectangles with rounded corners, amongst other
things. If a GPU was available, we wanted the canvas to use it
when rendering. In addition, nobody was maintaining GnomeCanvas
anymore. In fact that final few releases of GnomeCanvas were all
based on patches that came from the Ardour project. The decision
was clear: we should switch to a new canvas implementation.

The GTK project has steadfastly refused to identify any other
canvas implementation as "blessed", meaning that there have been
several efforts to improve on GnomeCanvas. In fact, the GNOME
project used to have a useful page on this
(https://wiki.gnome.org/ProjectRidley/CanvasOverview) that has
now been deleted. Of the candidate canvases listed there, the
only clearly appropriate for Ardour was "goocanvas". It was
written in C++, retained most of the GnomeCanvas semantics,
added a few useful items, and used Cairo for
drawing/rendering. It seemed close to ideal.

However, a more detailed inspection of GooCanvas didn't leave us
overwhelmed by the feeling that we had found the right solution
for our purposes. So in about the middle of 2012, Carl
Hetherington set about implementing a brand new canvas. It was
almost entirely disconnected from GTK itself, used Cairo for all
rendering, and was carefully designed to tackle some of the
scaling issues that we were aware of (it is entirely possible to
have thousands or even tens of thousands of items in the editor
canvas). Carl's work is the basis of the canvas that was
introduced in Ardour 3.6 although several basic aspects of its
design and implementation have been changed since his first version.

Requirements

For a canvas-type GUI widget/object to be useful for Ardour, it
has to have the following properties:

No limit to the number of items present on the canvas

A coordinate space that extends along both X and Y axes at
least as far as Ardour's 64 bit timeline

Full use of a powerful drawing API (Cairo is the reference
drawing API, and given its likely adoption as part of a future
C language specification, a very desirable API to have
available)

Ability to utilize GPU rendering operations when
appropriate.

Common operations must scale to conditions involving on
the order of 10k items. This means that they must avoid
recursion wherever possible

Ability to independently scroll different sections of the
canvas. For example, Ardour's rulers scroll left and right
but not move in response to vertical scroll requests. In
contrast, the main track display scrolls along both axes. At
some time in the future, when track headers are a part of
the canvas, the headers will scroll only up and down, not
left and right.

Ability to place items on the canvas that span
independently scrolling regions (e.g. the playhead, which
covers both the rulers/timebars and the tracks, even though
those two areas scroll independently)

There are also several features that we do not require
or use:

Scaling and rotational transforms

3D

It may seem puzzling that the various geometric transforms are
not of interest. This is is because Ardour's use of the canvas
is really nothing like the presentation of an image combined
with user-driven operations to zoom or rotate the image. Ardour
draws very specific images in which single pixels have semantic
content inherent in their existence and placement. When we
"zoom", we are not zooming text, or the outlines around boxes,
but the way waveform data is presented. And even the waveform
drawing itself is not just rescaled, but redrawn potentially
with different (more precise data). As for 3D, perhaps we will
one day find a use for this aspect of visual data presentation,
but so far, we have no use for it.

The Implementation

Concepts

Canvas

A Canvas is an object on which zero or
more Items can be drawn and displayed to the
user. The Canvas can also receive events notifying it of
interactions with the user (such as a button press, a motion
event, a key press, etc). There is no limit to the number or
type of Items that can appear on a Canvas, and the full extent
of a Canvas spans the maximal size of a double precision
floating point value. All coordinates are double precision
floating point.

The base class for the Canvas has no relationship with GTK at
all. But it also is an abstract class that requires a few methods
to be implemented in order to be instantiable. The GtkCanvas is a
concrete class that causes the Canvas to be rendered in a
GtkEventBox, and to receive events via the GtkEventBox.

The Canvas implementation also provides a utility class,
GtkCanvasViewport, which packages a GtkCanvas and adds two
GtkAdjustments which can be used to scroll the contents of the
canvas. If scrolling is not required, then it is possible to use
GtkCanvas directly.

Item

An Item is a discrete entity that can be placed on
the canvas. Items can be drawn, hidden, moved in either
direction, and raised or lowered in the stacking order on the
Canvas (sometimes called Z order or Z-axis stacking). Item
implementations are completely responsible for drawing the
Item's appearance under all conditions. Items can receive events
notifying it of interactions with the user. All Items inherit
from a simple base class (ArdourCanvas::Item). The
key method of an item implementation (the one that
differentiates it from any other type of item) is
Item::render(), a virtual method called to draw all or part of
the Item as necessary. Each Item has a pointer to a parent,
and to the top level canvas. Each Item has a position specified
in the coordinates of its parent. If this position is (0,0),
then the Item appears at the origin of its parent. If the
position is (10,10) then the Item appears 10 pixels right and 10
pixels below the origin of its parent. The Item's origin is
always (0,0) in its own coordinate space.

Note that the actual drawn area of an Item may have little to do
with the position of the Item. The position may just act as
the origin for the coordinate system of the Item, with rendered
pixels being arbitrary distances away from this origin.

The rectangular area that the Item will draw to is given by
its bounding box, returned as
a boost::optional<Rect>. It is an optional
type because the bounding box may be undefined if (for example)
a Group has no children or a Rectangle has zero area. The
bounding box of a Group is the union of the bounding boxes of
all of its children.

Item children

All Items may have zero or more child Items added to them,
created a recursive nesting that defines the structure of a
particular Canvas. In general, Items that draw particular forms
(e.g. rectangles or waveviews) will not generally ever have
children added to them, because their drawing code ignores the
existence of children entirely. However, a Container is a
derived type of Item which does pay attention to the presence of
children when it draws itself, specifically by
calling Item::render_children().

Container

A Container is a special kind of item that draws
nothing by itself, but does draw its children. It allows the
easy relative placement of a set of Items as well as a way to
move a set of Items as a single entity. When the Container is
moved, all of its children will draw to a new position on the
Canvas.

Note that the position of a Container's children is defined by
each child's position() method. A Container does not
arrange its children in any automatic way, so it is the
responsibility of other code to place Items correctly within the
Container.

In the future there may be derived classes based on Container
which do set their children's position (e.g a "Box" type which
arranges them along an axis, or a "Table" type which packs its
children into a tabular arrangement).

Root Group

A Canvas has a single top level Container, called "root", as its
only Item. It has one special function: to track its own
bounding box size and notify the canvas of changes that may
affect the display of the canvas using the underlying windowing
system.

ScrollGroup

A ScrollGroup is a special kind of Container which
responds to instructions from the Canvas to "scroll" - that is,
to alter what is visible within the window in which the Canvas
is viewed. A ScrollGroup can respond differently to horizontal
and vertical scroll commands (it can even not respond to them at
all, if necessary). A ScrollGroup that responds to both scrolls
will move its children within the window whenever any scroll
command is received.

If you want items on the Canvas to move in response to
scrolling, you need to add them to a ScrollGroup with the
appropriate sensitivity to scroll commands. Items without
a ScrollGroup in their ancestors will not move when the Canvas
is scrolled.

ScrollGroups should occur immediately below the "root" group on
a Canvas, and should not overlap. Their placement is achieved by
setting their position. Nothing will prevent overlaps, but the
result is ambiguity in which ScrollGroup covers a given
area. They may be moved on demand just like any other Item.

Coordinate Systems

In a given Canvas, there are 3 coordinate systems in use, all
with coordinates specified as double precision floating point. A
single axis value is stored in a type called Coord,
and a pair of axes values used to define a point in a coordinate
space is stored in a type called Duple. These names
were chosen partly to avoid with the primitives found in various
native Window system APIs.

All 3 coordinate systems use the same convention as Cairo and X
Window, with the origin at the upper left, increasing x-axis
values indicate rightward movement along the axis, and
increasing y-axis values indicate downward movement along the
axis.

Window Coordinate Space

(0,0) refers to the pixel at the upper left of the window
in which the Canvas is displayed. Coordinates can be negative
but never for drawing purposes. Coordinates can be of any
positive size, but for drawing purposes should never exceed
the height or width of the window in which the Canvas
displayed. Thus if the window is W pixels wide and H
pixels high, then (W-1, H-1) refers to the pixel at the
lower right corner of the window. All coordinates should be
non-fractional because they should refer to actual pixel
positions in the displayed window.

Events are delivered to the
Canvas in Window coordinate space.

Canvas Coordinate Space

A vast coordinate space. (0,0) is the center, and it
extends by approximately 1e307 in both directions along each
axis.

The relationship to Window coordinate space depends on
the scrolling commands that have been issued to the canvas
and the arrangement of ScrollGroups within the
canvas. Canvas::window_to_canvas()
and Canvas::canvas_to_window() can be used to
convert between the two spaces. If the canvas has not been
scrolled, then (0,0) in Window Coordinate space
corresponds to (0,0) in Canvas Coordinate space.

Note that if the Canvas contains more than 1 ScrollGroup,
there is no single mapping between a Window Coordinate
space axis value and a Canvas Coordinate Space axis
value. The transform depends on precisely which
ScrollGroup covers an (x,y) coordinate, which in turns
defines the offsets between Window and Canvas Coordinate
spaces.

Events are delivered to canvas items in Canvas
Coordinate Space.

Item Coordinate Space

A translated version of canvas coordinate space. (0,0)
refers to the origin of the item (even though it may never
draw anything there). (10,10) is 10 pixels right of and
below the origin.
Item::item_to_window(), Item::window_to_item(),
Item::item_to_canvas()
and Item::canvas_to_item() exist to convert
between Item Coordinate space and the other two.

Available Item Types

Many of the items listed below are derived from one or both of
ArdourCanvas::Fill and ArdourCanvas::Outline. These base classes
provide a few methods to control how the item is drawn,
specifically how/whether it is filled and how/whether an outline
is drawn. Common methods
include Fill::set_fill_color, Fill::set_gradient, Outline::set_outline_width
and Outline::set_outline_color.

Because the canvas uses Cairo for drawing, and because
single-pixel width lines are a common requirement, it may be
important to read and carefully understand this Cairo
FAQ answer. The
short version is that you can't optimize the design of a drawing
API for both trivially obvious semantics when filling and when
drawing lines. Cairo's API was optimized to make fill semantics
trivially obvious and line drawing slightly more complex. Perhaps you
would have done it differently.

In general, the Canvas items that draw lines or outlines default
to rendering single-pixel wide lines, and do so using the 0.5
coordinate offset "trick" mentioned in the Cairo FAQ cited
above.

Rectangle

Draws a rectangle. Each edge can be
specified as outlined or not. Note: the edges of a rectangle
are on its boundaries, not outside them. The fill and
outline colors can be specified independently.

Arc

Draws some part of a circle, with a specified angle. The arc
can be outlined and filled, with two separately specified colors.

Circle

Derived from Arc, but always draws the full 360 or
2*PI radians.

Line

Draws a straight line between two points. Width of the line
can be controlled using ::set_outline_width().

PolyLine

Draws a series of straight line segments between any number
of points. Width of the line can be controlled using ::set_outline_width().

Curve

Draws a smooth curve through an arbitrary series of data
points. Internal implementation uses Catmull-Rom splines, with
the option to use either Centripetal, Chordal or Uniform curves
(Centripetal is generally the best for drawing smooth
curves). This curve differs from those offered by Cairo, which
are Bezier splines and are not guaranteed to avoids "knots" when
interpolating between points, which is critical when
representing various kinds of data in Ardour.

Polygon

Draws a series of straight line segments between any number
of points, but always closes the path to link the first and last
points. Separate outline and fill colors may be specified.

Arrow

Draws a straight line between two points, and offers the
option to draw an arrow of various sizes and shapes at each end.

LineSet

Maintains information about a set of all-horizontal or
all-vertical lines. Each line has its own startpoint and width,
but they all have the same origin along one axis and the same
extent. We use this in Ardour for the grid/measure lines, and
also for the ruled background of MIDI tracks. It is
significantly more efficient than using the equivalent number of
Line items. Note that the drawn lines never receive Event
notifications of any kind, neither does the LineSet as a whole.

Text

Draws a short piece of text. The text will be drawn using
Pango, and can have its color, font and alignment characteristics fully
specified (no outline option is available, however). The text to
be drawn can also be limited by a pixel width, so that the item
never displays "too much" text regardless of what text it is
asked to display.

Image

Draws an image. The image is given to the Item as a raw data
buffer with a specified width, height, stride and data format.

Pixbuf

Draws an image. The image is given to the Item a Gdk::Pixbuf.

WaveView

A specialized item for Ardour which draws a representation of an
audio signal.

XFadeCurve

A specialized item for Ardour which draws a representation
of the pair of curves that fade material in and out near the
near the ends of a region.

Ruler

A specialized item which draws a ruler with a series of 3
levels of tick marks each with optional annotation. Methods to
determine where the ticks should be placed and what annotations
(if any) should be drawn near them are handed to the Ruler
constructor, and called whenever the visible part of the ruler
changes. A Ruler does not know about or care about the nature of
the time units it is displaying.

The Coordinate Space problem

In Carl's original implementation, the Canvas Coordinate Space
and the Window Coordinate Space were unified. We took advantage
of Cairo's use of double for its own coordinate
space, assuming that to accomplish scrolling we could simply use
cairo_translate() to move objects as needed when
rendering.

Unfortunately, Cairo has a deep and longstanding bug (feature?)
which means that although it uses doubles for coordinates, the
actual size of the coordinate space it can manage correctly is
limited to roughly 32767 pixels x 32767 pixels. Any attempt to
use cairo_translate() to move outside this area, or
any use of coordinates outside of this area causes drawing
glitches and outright errors. Although the Cairo developers are
aware of this bug, no fix for it appears in sight, and so we
were forced to work around it. The Ardour canvas is required to
span a much larger coordinate space than Cairo can handle.

The adopted solution is not as clean as Carl's original
equivalence relationship, but is still not too hard to
comprehend. Items maintain their own coordinates using the full
range of a double precision float. However, rendering is done by
converting all coordinates used for drawing into window
coordinate space. If the item's bounding box doesn't intersect
with the area visible in the Window, then the item will not be
asked to Render itself. If it is asked then the transformed
coordinates passed to various Cairo calls. Since the limits on
window dimensions more or less match Cairo's internal
limitations, these coordinates are guaranteed to be in range for
Cairo's API.

Rendering Model

The concrete base class (e.g. GtkCanvas) receives "expose" or
"draw" notifications from the underlying window system, telling
it that part or all of the window in which the canvas is
displayed needs to be redrawn. These notifications arise from
two sources:

Window-system level changes involving the visibility of
the canvas' window (e.g. inital display, becoming visible
after another window is moved away from the canvas' window,
etc)

Redraw requests queued by the canvas itself in response to
changes in its Items. For example, when an Item is added,
moved or resized, some or all of the part of the Canvas
visible in the window may need to be redrawn. Of course, the
Item could be completely off-screen, in which cases such
changes will not cause a redraw request.

Both sources for expose/draw notifications result in the same
GdkEvent being sent to the GtkCanvas, which includes a
specification of the area of the window that needs to be redrawn
(this area is specified in Window Coordinate space). The
GtkCanvas obtains the Cairo context being used for drawing,
repackages the area as a Canvas-native type, and passes both
into the Canvas::render() method.

The rest of the rendering implementation has no relationship to
Gtk or the native window system, and consists of zero or more
calls to the Cairo API to alter the appearance of a Cairo
surface (assumed to be the window in which the Canvas is
displayed). The Canvas calls ::render() on its root group, and
this finds all children that are fully or partially visible
within the redraw area and then (recursively) calls their
::render() method. Note that every item is passed the same
original area to be redrawn as was given the Canvas itself. Each
item converts its bounding box into Window Coordinate space, and
then decides precisely what and how it will draw. Some items
(e.g. the Image item) may simply render using a pre-drawn,
memory-cached Cairo surface. Others will draw directly using
some programmatic logic and common Cairo drawing operations (for
example cairo_rectangle(), cairo_line_to(), cairo_arc(),
cairo_stroke(), cairo_fill()).

Event Propagation

The concrete canvas object (e.g. GtkCanvas) receives events from
the native window system (e.g. X Window on Linux, Quartz on OS
X) via the GTK toolkit. The following event types are explicitly
handled by the canvas:

draw/expose (described above)

button press and release

motion

enter and leave

Note that at present, key press and release events are not
handled by the Canvas, but left for a parent widget to handle
(which suits Ardour's general interaction model).

Current Item

By default, button press and motion events will be delivered to
a designated item termed the "current item". If a drag is in
process, there is an alternate item, the "grab item" to which
events will be delivered. Maintaining the correct current item
for event delivery is one of the central tasks of the canvas implementation.

This is implemented in Canvas::pick_current_item (Duple const&
point, int state). The design of the canvas allows for the use
of "smart" methods of determining which items cover a given
point, but at this time (June 2014) we use a naive, O(N) linear
search through each group. This is done via the LookupTable
object table associated with each Item. The code contains an
attempted "smart" but unfinished and untested implementation
(OptimizingLookupTable). For now, DumbLookupTable is used to
find items covering a point, and that just iterates over a list
of children and calls Item::covers() on each one.

Enter/Leave

For every enter/leave event and for every motion event, as well
as whenever there are any changes to an existing Item or when an
Item is added or removed from the canvas, the canvas takes the
current pointer position and determines what item covering that
position should be considered the "current item". Recall that
there may be many items covering the pointer position - the
canvas will use the upper-most item as "current item".

Whenever the "current item" changes, a series of events must be
sent to various items in the canvas to notify them of the change
so that they may, if necessary, modify their appearance. The
existing current item is sent a "leave" event, with the "detail"
field of the event set to indicate whether the new current item
is a child, parent or unrelated item. Then the parent chain for
the old current item is traversed, with each of them being sent
a leave event (again with the "detail" field set appropriately).

After the leave events have been sent, the parent chain for
the new current item are sent an enter event (with the
"detail" field correctly set), and finally the new current item
is sent a enter event also.

Once this set of events has been delivered, all Items have been
appropriately notified of a change in the current item, and
"current item" is finally reset. Subsequent button and motion
events will be delivered to that item (unless a grab is in
effect).

Note that enter/leave events do not cascade to parents as
described in the next section. They are explicitly delivered by
the canvas, without regard for whether or not a particular item
returns true or false to indicate that it handled the event.

Parent/Child chain

Each canvas item has a member:
sigc::signal<bool,GdkEvent*> Event; to which
arbitrary functors can be attached as handlers. Each handler
returns a boolean value to indicate whether or not it handled
the signal/event or not. This signal will be "emitted" as part
of the event delivery mechanism for that event.

If an event is delivered to an Item, zero or more handlers for
the event will be invoked. If any of them return true, the event
is considered to have been handled, and no further processing
will be done relating directly to the event. Note that as in
GTK, the connection order of the handlers can have important
implications. If there are two functors/handlers connected to an
item's Event signal, and the first one always returns true, then
the second one will never be
invoked. sigc::signal::connect() has arguments to
control the ordering of handlers as they are connected.

If there are no handlers, or none of the handlers returns true,
then the event is considered unhandled, and is redelivered to
the item's parent. This cascade-to-parent continues until the
event is delivered to the root group. If it is still unhandled
at that point, we notify GTK that the canvas has not handled the
event, allowing it to continue with its own propagation strategy
(e.g. to the top-level window in which the canvas occurs).

We use sigc::signal rather
than PBD::Signal here because the signal is
connected to and delivered ("emitted") in a GUI context that is
always serialized and always single threaded. PBD::Signal
(unlike sigc::signal) is thread-safe, but we don't need those
semantics for GUI-related signals.

How Scrolling Works

Canvas::scroll_to (Duple const& point) is invoked to tell
the canvas to redisplay its contents so that where appropriate,
they are offset by point.x and point.y
along each axis.

ScrollGroups are the currently implemented solution for this
problem. When a ScrollGroup is created, its constructor requires
a specifier of which axes (X, Y, X&Y or none) it will
scroll. When Canvas::scroll_to is called, the
canvas iterates over all children of its "root" group. Each
ScrollGroup that it finds at this level has
its ScrollGroup::scroll_to method invoked. This
alters the _scroll_offset member of the
ScrollGroup.

As described above, when items are rendered they first convert
from their own coordinate space to the window space. Every item
has a pointer to its "scroll parent" (a Scrollgroup; the
ScrollGroup may actually be several "generations" from the Item,
but every item has only 1 scroll parent). The
implementation of Item::item_to_window uses the
ScrollGroup parent's scroll_offset to determine
what (if any) offset in effect due to scrolling, and
incorporates this into the returned value.

As a concrete example: an Item has a scroll parent which
responds to both vertical and horizontal scrolling. The canvas
has been scrolled by (-100, 100). The Item sits at (40,40) in
Canvas Coordinate space. It calls item_to_window on
a point (x,y) within its coordinate space. It first converts to
canvas coordinate space, generating (x+40,y+40). Then it applies
the scroll offset of its scroll parent (-100,100) to give
(x+40-100,y+40+100). The item coordinate (x,y)
is thus (x-60,y+140) in window coordinates.

A similar item with a scroll parent that responds only to
horizontal scrolling would do the same calculation and end up
with (x-60,y+40). The two items would thus appear at different
positions along the y axis, as appropriate given their scroll
parents' sensitivity to scrolling.

Items that are have no scroll parent will never scroll. Although
it is possible to also use a scroll group with no scroll
sensitivity, scroll groups may not overlap, so this
no-scroll-parent => no-scroll-behaviour feature has some
potential use for global canvas items that should remain in the
window at all times but that span different sections of the canvas.

Note that because items have direct pointers to their scroll
parent, this design adds a very lightweight O(1) operation to
compute the effect of scroll on the
item_to_window/window_to_item transforms. We do not have to
traverse parent/child chains, which would make the computation
O(N) where N represents the depth of a given item from the
canvas root.