October 2009 - Posts

Introduction

Today, a colleague and I were playing with new C# 4.0 and BCL 4.0 features, me trying (and succeeding I think) to convince my co-worker about the merits of LINQ and its peripheral technologies. Users of Visual Studio 2010 Beta 2 may have noticed the new IObservable<T> and IObserver<T> interfaces in the System namespace:

namespace System
{
// Summary:
// Defines a provider for push-based notification.
//
// Type parameters:
// T:
// The object that provides notification information.This type parameter is
// covariant. That is, you can use either the type you specified or any type
// that is more derived. For more information about covariance and contravariance,
// see Covariance and Contravariance in Generics.
public interface IObservable<out T>
{
// Summary:
// Notifies the provider that an observer is to receive notifications.
//
// Parameters:
// observer:
// The object that is to receive notifications.
//
// Returns:
// The observer's interface that enables resources to be disposed.
IDisposable Subscribe(IObserver<T> observer);
}

// Summary:
// Provides a mechanism for receiving push-based notifications.
//
// Type parameters:
// T:
// The object that provides notification information.This type parameter is
// contravariant. That is, you can use either the type you specified or any
// type that is less derived. For more information about covariance and contravariance,
// see Covariance and Contravariance in Generics.
public interface IObserver<in T>
{
// Summary:
// Notifies the observer that the provider has finished sending push-based notifications.
void OnCompleted();
//
// Summary:
// Notifies the observer that the provider has experienced an error condition.
//
// Parameters:
// error:
// An object that provides additional information about the error.
void OnError(Exception error);
//
// Summary:
// Provides the observer with new data.
//
// Parameters:
// value:
// The current notification information.
void OnNext(T value);
}
}

Those interfaces deserve whole blog series of their own (I promise to do so, some day) and have to do with the Reactive Framework discussed on various occasions on Channel 9:

More info on this will appear on Channel 9 in the foreseeable future (stay tuned; I’ll be there!) but all that matters in the scope of this post is the use of two interesting keywords in the interface definitions above:

Did you see them? The reuse of keywords out and in as modifiers on generic type parameters is what’s referred to as the new C# 4.0 feature called definition-site generic co- and contravariance for interface and delegate types, or generic co- and contravariance for short. I’ve written about this in the past, explaining the concept in terms of apples of tomatoes:

If you don’t know about this stuff just yet, take a break and visit the link above before returning back here. The nice thing about all of this is that as a user of generic types declared to be co- and/or contravariant in certain type parameters, you don’t need to worry about it at all. In fact, this feature allows you to write things you couldn’t write before:

But in .NET 4.0 with C# 4.0 and VB 10.0, you can. The point of variance annotations on generic type parameters is to make such operations provably, at compile-time, safe. Before this new language-level feature, generics were always invariant.

An observed problem?

Loaded with my knowledge of generic co- and contravariance, I was surprised to see the following code compile fine. To be honest, the real scenario was more contrived than the simplified one below, with many intermediate types in the picture:

//
// When apples are thrown at us, we'll write them to the screen.
// We don't expect to see other fruits here!
//
// Note: This uses an extension method on IObservable<T> that
// hides the declaration of an IObserver<T>. C# doesn't
// have anonymous inner types unfortunately...
//
IObservable<Apple> apples = new AngryAppleThrower();
apples.Subscribe(Console.WriteLine);
//
// Fine, since IObservable<T> is covariant in T.
//
IObservable<Fruit> fruits = apples;
//
// What? A Banana is injected into an observable of Fruits?
// Though you may think this is fine, how will the observer
// above, expecting Apple objects, react to seeing a Banana?
// This is not type-safe; covariance should prevent this!
//
fruits.Inject(new Banana());

It just couldn’t be true that you could somehow sneak a Banana in an observable of Apple objects. That’s the whole point of type safety guarantees provided by co- and contravariance on generic types in C# 4.0. I tried a few things to verify we could really pass subtypes of Fruit into the IObservable<T>. The following didn’t compile while the Banana-case did:

fruits.Inject(new string());

Just looking at the sample above you can already smell what’s going on, maybe? Tip: look at the types of the variables. What letter do they start with? Then, where does Inject come from? If you don’t see it yet, don’t worry as I’ll reveal the answers in a second. Either way, I started to repro the above with simpler fictional types:

In fact, the two interfaces shown here sort of add an additional layer of complexity as was the case in the application I was staring at. The above should fail to compile: since IMyIO<T> declares T as a covariant (output-only) type parameter, it should not be possible to inject an object of type T, or any of its subtypes, into it. So Inject should not work. And indeed it didn’t compile:

Invalid variance: The type parameter 'T' must be contravariantly valid on 'IMyIO<T>.Inject<S>(S)'. 'T' is covariant.

What this says is that Inject requires T to be contravariant (available for input positions) but it’s declared to be covariant. Good job dear compiler (and its developers), just what I expected to see here! Still, that was what I was observing in the concrete fruity flavored sample shown to me. To make matters worse, we were dealing with concrete types implementing the interfaces, so things were far less obvious than they are in the sample shown above. Going back to the questions I asked to the reader earlier, it’s clear the Inject method in the Fruit-sample is coming from an extension method definition, as the left-hand side is an interface that doesn’t have the method by itself:

Well, clearly this is fine. Inject is just a method sitting on the outside of the type it extends (in this case IObservable<T>), hence it cannot reach into its internals in any way. The syntactical characteristic of extension method invocation as if it were an instance method simply threw me off:

fruits.Inject(new Banana());

Even though it looks like the covariant Fruit container accepts an object of subtype Banana, were not doing so. The three words “covariant”, “accepts” and “subtype” are a contradictio in terminis. Valid triplets are:

All that’s happening is here is that a “peer method” (with no more “privileges” than the caller in terms of reaching out to the fruits object’s state and internals) is being called:

EvilLooking.Inject(fruits, new Banana());

Here it’s clear that fruits is not accepting a Banana directly. And obviously the “Implementation doesn’t matter yet…” comment in the Inject method implementation shown above is wishful thinking. It won’t be able to contaminate the IObservable<T> source object with an S object (where S is a subtype of T) no matter how hard it tries (short of reflection-based techniques that will fail at runtime).

Case solved. Remaining work for the night was to come up with a good illustration (recall the real problem was not with Fruit, Banana or Apple objects, nor with IObservable<T> directly, but with less intuitive types) of how:

The extension method doesn’t break safety at all, though it looks as if it does (to the trained covariant eye);

Co- and contravariance are a great feature;

And maybe throwing in some of the dangers of dynamic typing when used inappropriately (a tangent to the original topic).

Extension methods ain't Trojan horses

Here’s what I came up with on the bus heading home: the story of how the Trojan War could have been gone differently if the city of Troy protected itself against the Trojan horse adequately... Lots of references to this story appear in the code below, so keep a copy of the linked pages on the side (using Windows 7 Snap if you have installed the brand new wonderful OS already). I hope you have as much fun reading it (and working your way through it as bedside lecture) as I wrote it. I know, it’s geeky humor at best. And analogies are just that: analogies. I don’t guarantee it to be perfect.

/* Extension methods ain't Trojan horses
*
* Illustration of safety guarantees provided by generic covariance in C# 4.0
* and how extension methods can merely provide a fata morgana making you
* believe they can defeat it.
*
* bartde - 10/22/2009
*/
using System;
using System.Collections.Generic;
using System.Dynamic;
using System.Threading;
using Microsoft.CSharp.RuntimeBinder;
// How the Trojan war could have been...
namespace NewTrojanWar
{
/// <summary>
/// Running the attack vector!
/// </summary>
class Program
{
/// <summary>
/// Proves the attack DOESN'T work.
/// </summary>
static void Main()
{
try
{
GreekArmy.TheOneAndOnly.Attack();
}
catch (EntryPointNotFoundException ex)
{
// Ouch. "The gates are closed it says." Maybe they are covariant after all?
// They got smarter than we thought! Did they hire Anderius?
Console.WriteLine(ex);
// What happened to Epeius is left to your own imagination...
}
}
}
/// <summary>
/// City protected by covariance.
/// </summary>
/// <typeparam name="NotAllowedToComeIn">Persona or objects of this type are not allowed to come in.</typeparam>
interface ISecureCity<out NotAllowedToComeIn>
{
/// <summary>
/// What can't come in, can go out.
/// </summary>
/// <returns>You're fine to leave, but beware of the point of no return!</returns>
NotAllowedToComeIn Escape();
/* Thanks to Laocoon and Cassandra, the following is not possible anymore due the covariantly
* protected city. The Trojans are a little unhappy as the annual horse meeting can't come in
* anymore though. It was quite an invasion every year; too bad we lost it since Laocoon and
* Cassandra put their Troy Sharp 4.0 plan into action. The fact they convinced us that the
* city has support for covariant protection in place since the Troy City Council 2.0 came out
* almost five years ago, we - the Trojans - are convinced that cancelling the annual horse
* meeting for additional protection is a good thing. Out with those foreign horses!
*/
//void Invade(NotAllowedToComeIn evil);
}
/// <summary>
/// Base class for every horse.
/// </summary>
class Horse
{
}
/// <summary>
/// Troy doesn't want horses to come in.
/// </summary>
sealed /* we're an incredibly safe city, no-one should override us! */ class Troy :
DynamicObject /* Another protection put in place by Laocoon and Cassandra, see further... */,
ISecureCity<Horse /* Laocoon and Cassandra were right */>
{
/// <summary>
/// We constructed it ourselves and are proud of it.
/// </summary>
private Troy()
{
}
/// <summary>
/// There's only one real Troy!
/// </summary>
private static Troy s_city;
/// <summary>
/// For publication in the Easy Jet brochure.
/// </summary>
/// <returns>Attractive city in the sun; and safe against incoming horses!</returns>
public static ISecureCity<Horse> City
{
get
{
if (s_city == null)
s_city = new Troy();
return s_city;
}
}
/// <summary>
/// Troy breeds horses and people can ask for them from the outside.
/// </summary>
/// <returns>You want a horse? Have one!</returns>
public Horse Escape()
{
return new Horse();
}
/* Sigh. This used to be fun. To be a good covariant citizen, we have to drop it for
* better or for worse. Now we implement the new ISecureCity interface!
/// <summary>
/// Horse parade.
/// </summary>
/// <param name="parade">External horses.</param>
[Obsolete]
public void Invade(Horse parade) // implemented the invariant old ICity interface
{
// Hang out in the streets and watch the horse parade coming in through the gates.
}
*/
/// <summary>
/// We'll pretend to accept dynamic calls.
/// </summary>
/// <param name="binder">Greek language invocation.</param>
/// <param name="args">What are they saying?</param>
/// <param name="result">We won't every talk to them though...</param>
/// <returns>Hard cheese.</returns>
public override bool TryInvokeMember(InvokeMemberBinder binder, object[] args, out object result)
{
throw new EntryPointNotFoundException("The gates are closed!");
}
}
/// <summary>
/// Greeks trying to attack will need to sit in the horse for quite a while till
/// the Trojans wheel the horse in.
/// </summary>
interface ICanSitStill
{
void SitStill();
}
/// <summary>
/// Greeks trying to attack by creating a special horse.
/// </summary>
/// <typeparam name="T">Objects to put inside the horse.</typeparam>
abstract class FilledHorse<T> : Horse where T : ICanSitStill
{
/// <summary>
/// The Trojans will be curious to see what's inside the horse and will subscribe
/// to the present to receive whatever is in there. Greeks will invade through the
/// well-known OnNext method that fires the attack inside the city walls.
/// </summary>
public abstract IObservable<object> Present { get; }
}
/// <summary>
/// Non-HR-compliant base class for the type below, but according to the story all soldiers were men.
/// </summary>
class Male
{
}
/// <summary>
/// Soldiers need to be tough men.
/// </summary>
sealed /* inheritance is not allowed in the army */ class GreekSoldier : Male, ICanSitStill
{
/// <summary>
/// Event to launch the attack.
/// </summary>
internal ManualResetEvent GoGoGo { get; private set; }
/// <summary>
/// Wait for the attack to start.
/// </summary>
public void SitStill()
{
GoGoGo.WaitOne();
// Really they looked for the city's gates and let in their friends. But let's assume they were
// particularly eager to attack straight away. No matter what they do, thanks to the new city // security plan, they won’t even get here: Troy won’t burn!throw new DestroyTroyException();
}
}
/// <summary>
/// Destroy Troy!
/// </summary>
class DestroyTroyException : Exception
{
}
/// <summary>
/// A present for the Trojans.
/// </summary>
sealed /* don't mess with us! */ class InterestingHorse
: FilledHorse<GreekSoldier /* Let's hope they don't look at the type label at the "base" of the horse. */>
{
/// <summary>
/// Invaders.
/// </summary>
private IEnumerable<GreekSoldier> _soldiers;
/// <summary>
/// Fill the horse with invaders. Make internal not to arouse suspicion.
/// </summary>
/// <param name="soldiers">Invaders.</param>
internal void FillWith(IEnumerable<GreekSoldier> soldiers)
{
_soldiers = soldiers;
}
/// <summary>
/// You got a present! Get it and listen to it :-).
/// </summary>
/// <remarks>Typed as a mysterious observable of objects. Little do they know the objects are soldiers.</remarks>
public override IObservable<object> Present
{
get { return new WaitingInvaders(_soldiers); }
}
/// <summary>
/// Other than a regular horse, it can be wheeled in.
/// </summary>
public void Wheel()
{
// This should really override the Horse's feet.
}
/// <summary>
/// But the horse is full of invaders waiting till the attack has to happen at night.
/// </summary>
sealed class WaitingInvaders : IObservable<GreekSoldier>
{
/// <summary>
/// Invaders.
/// </summary>
private IEnumerable<GreekSoldier> _soldiers;
/// <summary>
/// Creating new waiting invaders.
/// </summary>
/// <param name="soldiers">Invaders sitting still.</param>
public WaitingInvaders(IEnumerable<GreekSoldier> soldiers)
{
_soldiers = soldiers;
foreach (var invader in _soldiers)
invader.SitStill();
}
/// <summary>
/// Curious Trojans will definitely call this.
/// </summary>
/// <param name="observer">The curious Trojans will observe.</param>
/// <returns>It doesn't give you back something to unsubscribe. In a disappointment, the Trojans go to bed.</returns>
public IDisposable Subscribe(IObserver<GreekSoldier> observer)
{
new Thread(() =>
{
// Wait till the night falls. We anticipate there will be at most 12 hours of celebration
// from the time they wheel in the horse. Then all Trojans will be drunk and asleep.
Thread.Sleep(12 * 60 * 60 * 1000);
// If coast is clear...
foreach (var invader in _soldiers)
{
observer.OnNext(invader);
// Wake-up call! Don't ask me what alarm mechanism they used back then... For one thing,
// it wasn't the Windows 7 kernel's thread scheduler.
invader.GoGoGo.Set();
}
}).Start();
return null; // if you try to dispose the attackers, you'll null-ref yourself :P
}
}
}
/// <summary>
/// There's this evil invader that believes in static typing, hence he declared himself as a classy
/// static man. He can't have anything but static things. In a evil mood, he decided to help out the
/// Greek army by building the horse and providing an invasion plan for them to enter Troy.
/// </summary>
static class Epeius
{
/// <summary>
/// Build the horse.
/// </summary>
/// <returns>The invading horse.</returns>
public static InterestingHorse BuildHorse()
{
return new InterestingHorse();
}
/// <summary>
/// How evil. Putting a Gift method before the city door. How could you resist
/// calling it? But Epeius is faking it, as we shall see. The city is protected
/// quite well thanks to the Troy Sharp 4.0 feature.
/// </summary>
/// <param name="city">The city to invade.</param>
/// <param name="present">Hmm, horse beef (assuming the Trojans are not vegetarian)!</param>
public static void Gift(this ISecureCity<Horse> city, InterestingHorse present)
{
// Sometimes Epeius silently surrenders to dynamic typing because he's lazy.
// No-one will suspect him to describe Troy as a dynamic vibrant city. After
// all, they eagerly want to get into it and Epeius was pretty excited about
// his job for the Greek army helping them to invade the city. No reason for
// suspicion at all. We believe Epeius' use of dynamic is fine...
dynamic vibrantTroy = city;
// But Epeius didn't find (statically) a method or property on city that accepts
// the present. He makes the Greek army believe this method works by putting
// dynamic fairy dust in their eyes though. In fact, Epeius fails here and will
// let down the Greek army: there is no way to invade Troy. So, the invasion
// plan (the compiled IL) contains some hidden decision logic on finding out how
// to carry out the invasion (the DLR machinery behind the scenes). When the
// soldiers are in the horse, they'll eventually fail right here and their horse // will explode (see DynamicObject::TryInvokeMember method implementation!).
vibrantTroy.Invade(present);
// Unfortunately, Epeius made himself so static he even doesn't have a this
// reference to call .Escape() on himself. The rest of the story of what happened
// with Epeius after the Greek army discovered he cheated on them is not for the
// sensitive reader.
}
}
/// <summary>
/// The Greek army is cruel.
/// </summary>
class GreekArmy
{
/// <summary>
/// Army.
/// </summary>
private List<GreekSoldier> _soldiers;
/// <summary>
/// Only one!
/// </summary>
private GreekArmy()
{
_soldiers = new List<GreekSoldier> { /* recruit good fighters */ };
}
/// <summary>
/// Only one!
/// </summary>
private static GreekArmy s_army;
/// <summary>
/// Only one!
/// </summary>
/// <returns>Only one!</returns>
public static GreekArmy TheOneAndOnly
{
get
{
if (s_army == null)
s_army = new GreekArmy();
return s_army;
}
}
/// <summary>
/// Written by Odysseus who believes Epeius' poisonous Gift call on the City of Troy will work.
/// </summary>
public void Attack()
{
var horse = Epeius.BuildHorse();
horse.FillWith(_soldiers);
Troy.City.Gift(horse); // Sweet, Odysseus sees a Gift command he can shout to the City of Troy...
}
}
}