The Microsoft Security Development Lifecycle (SDL) process guidance illustrates the way Microsoft applies the SDL to its products and technologies. It includes security and privacy requirements and recommendations for secure software development at Microsoft.

SDL 5 addresses SDL guidance for Waterfall and Spiral development, Agile development, web applications and Line of Business applications. IT policy makers and software development organizations can leverage this content to enhance and inform their own software security and privacy assurance programs.

What's New in Version 5

SDL for Agile included: The largest change in SDLv5 is the inclusion of SDL for Agile Development as an Addendum at the end. The SDL-Agile guidance that was published in November 2009 is included in the parent SDL document to make it a one-stop resource.

New and updated security requirements and recommendations - refer to Jeremy's article for details.

This is a welcome consolidation of pre-existing resources, advice, guidelines, best practices and tools. The SDL integration of Agile methodologies is particularly well executed (previously, previously).

Sunday, 25 April 2010

As you might have guessed now, this is the second in a series of articles attempting gently to introduce C# users to a few functional programming concepts (note: Part 1 has been updated & extended since the original post).

One great motivation for functional techniques is that they facilitate stateless programming, which in turn is good for parallelisation - for example, sharing out the work among many processor cores. That's because the removal of state solves the synchronisation problem beautifully. If there's no state, then there's no state to become inconsistent while you're waiting for all your threads to complete!

Myriad patterns and other techniques exist to handle the awkward manipulations and side effects imposed on your code by a seriously stateful world. The Monad is such a pattern. Prominent in the established functional languages such as Haskell and Scheme, it is by now almost a first-class citizen of the C# world too - it's just a little hidden.

Service Tunnels

Monads are great whenever the work involves concatenating pure functions into a pipeline, then feeding your raw ingredients in at one end, and finally collecting your perfectly prepared hot meals as they cascade out of the other. Among their many other uses, one thing at which monads truly excel is: everything else except your basic vanilla process. "Exceptions", if you will, though in a much more general sense than "error conditions". They work by attaching secondary pipeline systems, one per monad, to divert the impurities from the main flow and manage them independently.

This means that each atomic function, each section in the main pipeline, has to be capable of returning slightly more information than the succeeding one strictly requires. In general therefore, a monad is associated with functions accepting inputs of certain types, and returning so-called amplified types.

The Maybe monad is the archetypal Haskell example. Its job is to handle data values that might be "missing".

Maybe?

Suppose that we want to compose functions accepting and returning value type data. Occasionally a function might encounter a problem, and be unable to return a value. In C# 2.0 and up, we can use a Nullable value type such as int? or double? as the return type. So, this will be our first example of an amplified type. Can we compose such functions?

The monad trick is to combine or "bind" the two functions in a slightly different way, so that all our atomic fs and gs don't have to be rewritten to accept nullable inputs, and to worry about propagating these correctly. Instead, that worry is delegated to the new Bind operation:

Presto! We can once more compose functions, using After. This is despite each function expecting some value type, of which the preceding function actually returns rather the "amplified", Nullable variant.

The example below contains three ad hoc functions from int to int?. Function up returns its input plus one, if such is in int range; otherwise null. Similarly, down returns either its input minus one, or null. Composite function climb calls up once, then down twice. If any constituent call fails (returns null), then so does the composite. But see how such failures are actually routed through the Bind operation, and not propagated through the rest of the main pipeline, which is therefore free not to accept null inputs:

Obviously each atomic function is still responsible for detecting and reporting its own exceptional conditions, but no longer for relaying those of its pipeline predecessors. This decoupling pattern is absolutely key to containment or efficient management of emergent complexity.

Complexity Containment

Statelessness and parallelism aside, monads are the key to the management of complexity, because they offer the maximal generalisation of functional composition. The best example available to us C# programmers is LINQ itself; because IEnumerable is a monad, and its Bind operation is SelectMany. Check its signature and you'll see! Ever wondered how LINQ managed to deliver such incredible power over a range of domains including queries, objects, XML etc? Because it was designed to the strict algebraic constraints of a monadic architecture!

Friday, 23 April 2010

Update: John Lewis's reply letter appended below; my reply, subsequent correspondence, and the dénouement, are all now in the Comments. At the time of writing, there are three American million dollar class action lawsuits (update: now four) pending against Sony (all at the US District Court of Northern California, oddly enough).

Best of luck also to Michael Trebilcock from Modbury, Adelaide, who is on disability pension, and is suing Sony for $800.

Update [July 22] Seven (!) class actions so far filed against Sony in San Francisco federal court have now been consolidated into a single class-action complaint, the request of "all counsel." The three law firms involved will act as co-lead counsel against Sony.

The Story So Far

Big surprise! Sony Corporation screwed a bunch of their legal, fee-paying, law abiding customers, including me - as described in this earlier article:

I took up the matter with my retailer, John Lewis, as documented by the email exchange recorded in the Comments section of that article. Now we've got to the big standoff, and while we wait to see who blinks first, I've sent to the retailer this PSA, which was a little too big, and a little too RTF, for a comment...

As I said to you on the telephone, I am currently keeping an open mind with respect to your recent offer of a £75 payment to me, which I recognise is after all 25% of the purchase price, and as such represents an improvement on previous refunds in the history of this issue. However, I do feel that I must insist on a clear statement of the terms and conditions under which this offer is being made, for if they involve my agreeing that John Lewis have not contravened the relevant consumer protection legislation, clearly I should be quite unable to comply with that.

We acknowledge your intention to reply more fully next week. As you know, we are now more than halfway through the 5th week since I filed my initial complaint with John Lewis, by email, on Tuesday 30th March at 10am. I am sorry to hear that in the interim, your legal department appears to have misinformed you on this matter. Perhaps while awaiting your final response, I might take the opportunity to clarify the applicable law, and to reiterate our position.

[1] EU Directive 1999/44/EC

The Directive states the following:

“The goods must:

comply with the description given by the seller and posses the same qualities and characteristics as other similar goods

be fit for the purpose which the consumer requires them and which was made known to the seller at the time of purchase.”

The Directive applies to the contract of sale between the retailer (not the manufacturer) and the customer. It also mandates a minimum two-year warranty on all new consumer goods (in Scotland, the Sale and Supply of Goods Act 1994 extends this to five years; in England, six).

[2] The "Purpose"

The Purpose, for which we required the goods, was: as a general purpose computer, Blu Ray player, and online games console. This Purpose was made known to the seller, viz. John Lewis personnel, at the time of purchase.

[3] How the Purpose was made known:

There were three distinct ways in which this occurred, any one of which alone suffices to establish liability.

[3.1] We explicitly made this Purpose known.

We made our requirements perfectly clear to your sales personnel at the time of purchase. And that, essentially, is all that is required under consumer protection law.

We were shopping that day for either a computer (for use as [my] "home office" machine, and also to let my wife edit & process photographs), or a games console (for my upcoming birthday), or both. Your salesman pointed out to us that the Sony Playstation [3] could do both; and that although it could not run Microsoft Windows, there was a lot of free software available for its "Other OS" feature.

[3.3] Sony advertised the Playstation's suitability for our Purpose.

At the time of purchase, I was in fact already aware of the "Other OS" capability of the Playstation 3; since launch, it had been widely advertised by Sony personnel on their websites, their public forum websites, and elsewhere. Here are four such published examples, all of which predate our purchase:

There is more to the PLAYSTATION®3 (PS3™) computer entertainment system than you may have assumed. In addition to playing games, watching movies, listening to music, and viewing photos, you can use the PS3™ system to run the Linux operating system.

By installing the Linux operating system, you can use the PS3™ system not only as an entry-level personal computer with hundreds of familiar applications for home and office use, but also as a complete development environment for the Cell Broadband Engine™ (Cell/B.E.).

[...]

To use the Linux operating system, you must update the PS3™ system software to version 1.60 or later.

Very recently, a red warning has been added to the top of that page, saying: "On PS3™ system models sold earlier than the CECH-2000 series models, the Open Platform feature will not be available if the system software is updated to version 3.21 or later". This condition was of course not present at the time of our purchase.

"The Linux Distributor's Starter Kit provides information, binary and source codes to Linux Distribution developers who wants to make their distro support PS3.""Because we have plans for having Linux on board [the PS3], we also recognize Linux programming activities... Other than game studios tied to official developer licenses, we'd like to see various individuals participate in content creation for the PS3."

"The Playstation 3 is a computer. We do not need the PC."[Note: German text]

In the past several weeks, Sony moved urgently to remove some such evidence from certain promotions similar to those detailed above, to alter the wording of agreements, and so on. But in fact their websites had by then already been cached by search engines such as Google, and many further "incriminating" backup copies have also been secured since then.

I have mentioned earlier in this ongoing email exchange, the console games that we are presently unable to play online, and the Blu Ray discs whose BD-Live content we are currently unable to access. What I did not mention, because it wasn't obvious until we received the relevant papers and began preparing our sheriff court claim, is that these purchases themselves amount to literally thousands of pounds. Should this case go to court, this is indicative of the level of damages that we would claim.

In view of your message today, we shall make our final decision on how to proceed, based on all available information at that time, no later than Monday 3rd May 2010.

Thank you once more for your efforts in trying to negotiate a happy resolution for us.

I'm especially fond of the part where they assert my PS3 can't be a computer, because it is a games console; designated by the numbers 826 against the product on your receipt [...] If it had been sold as a computer the number would be 827.

But then reading on to the part John Lewis remain not liable [sic] for what has happened to your product... I realised that I hadn't won after all. What I want is neither a partial £75 compensation, nor a return for a full refund - both of which I've now been offered - what I want is just my retailer's acknowldegement that giant corporations like Sony cannot simply EULA their blustering way past any and all consumer protection legislation, to steal from me in my living room, or at my place of business.

John Lewis are indeed liable. They should accept their responsibilities under EU Law. Yes, they are also victims; but they should be kicking Sony's arse, not their own customers.

I'm about to decline this latest offer, just as soon as my language cools down to the optimum temperature for discursive efficiency. Meanwhile, as we prepare for war both in the media and in the Sheriff Court, we should perhaps pause to see what happens next in America. For it is extremely gratifying at last to see Anthony Ventura, a user from California, launch a class-action suit against Sony, alleging deceptive business practices:

Thursday, 22 April 2010

Disclosure: The Black Rats' drummer is a colleague of mine. But I would have said good things about them anyway...

In those far-off and fondly remembered days, for example last month, when our PS3 could connect to the Playstation Network without permanently nuking our OpenOffice, we'd pass many a happy virtual hour swigging the Pinot (noir for me, grigio for the moll) while driving like methylated crackheads round Paradise City or Big Surf Island. Custom soundtrack pounding from the dust storm buggy. Best times of all were when The Black Rats came on the radio.

They're "a hard-rocking four piece with a load of attitude and swagger", according to Bryan Quinn on the Daily Record's Music Blog. His 2008 article Rise of the Rats? described their sound as "[Black Rebel Motorcycle Club] covering The Music in a plane falling from the sky", before concluding "perhaps most importantly, they don't want to be the Kooks".

Yeah, what he said! And yeah, we have the bootleg of their entire, as yet unreleased, Mill Recordings album. Which makes it all the worse to realise we'll be missing them again this Sunday night, when they play Glasgow's Pivo Pivo.

My personal favourite: Sorry About the Mess. Anthony from SceneonSound wrote, reviewing their Glasgow appearance last May, The four piece looked comfortable and at home on the Classic Grand stage and playing songs like ‘Dirty Little Girls’ you could tell the welcoming crowd were enjoying the sing-a-long. Following this the tunes mellowed out with the ballad ‘Sorry About The Mess’ which has a great running drum beat leading up to a powerful chorus.

Earlier in the year Rebecca Hogg had written at whatsurmusic.com, reviewing their March gig at ABC2, “Sorry about the Mess” was probably the most memorable of their set with the repetitive cynical line and angsty accompaniment. This band may be from a small part of the country, but they sure do know how to make big noise!

Let's hope they continue to get even more attention this year. So far they're confirmed only for Hangerfest at Strathaven Airfield in June, but they're still tensely awaiting the response to their T-Break application...

Wednesday, 14 April 2010

Our flagship product is SQL Server backed, and initially used a text-based SQL builder. It was robust, sturdy and reliable, though sometimes quite difficult to extend when a new query building requirement materialised. Other times, it seemed virtually impossible!

So we developed another, object-based SQL builder. This could manipulate queries at the level of individual terms, expressions, clauses, or entire composite superqueries. Essentially, we anticipated many features of LINQ to objects, and a small subset of LINQ to SQL.

The nontrivial overheads of maintaining such a code base, in the face of finite development resources, as well as new and unforeseen requirements, were soon apparent. More seriously, as it was in a sense retrofitted to an already extensive application architecture, we didn't quite derive all the anticipated benefits of composable queries, closure, and the other great things that a well formed algebra brings.

Those benefits are considerable; the move to so-called functional programming technique is comparable to the jump from a simple imperative procedural language to one using objects. Working in C#, the hero of functional programming style is the humble Lambda Expression.

Composition of Functions

As C# 3.0 developers we are by now familiar with lambda expressions; fairly arbitrary fragments of code that takes any number of input parameters (including none), and may return a value. They are related to C# 2.0 anonymous methods and C# 1.0 delegates in obvious ways.

Lambdas allow us to represent the notion of functional composition succinctly and generally. For example, suppose we wish to work out the Root Mean Square, or RMS, value of a sequence. This involves three steps:

square each value in the sequence;

compute the mean of those squares;

extract the square root of that mean.

These three "atomic" functions can easily be composed to obtain the answer, the root of the mean of the squares:

In functional programming, we generalise the construction of composite expressions, like Root(Mean(Square(values))), using an operation that takes functions pairwise, and returns their composition as a new function. This lets us build new functions like RMS at run time, invoke them directly, pass them around to other functions, and so on.

With type safety in mind, we must accommodate functions with arbitrary parameter signatures and return types. Notice that my "atomic" functions Square, Mean and Root above all differ in this regard: Square projects a sequence to a sequence; Mean aggregates a sequence to a double; Root converts double to double.

After takes two delegates f and g representing functions (i.e., value-returning methods accepting one input parameter), and combines these into a single delegate, representing the function whose effect is the same as applying the original functions in the sequence "f after g" (i.e., first apply g to the input parameter, and then apply f to that result). It is used twice in the above code; first to compose square with mean, then to compose that result with root.

What if I want to vary the order of composition? Suppose I need to know the square of the mean of the roots of my sequence. For simplicity - literally, to avoid complex numbers! - let's assume the input sequence comprises only nonnegative values. Can I use the After method to compose my atomic functions in the new order, generating the Square Mean Root (or "SMR") function?

Sadly no; all the joints are mismatched. Root expects a single input value, not a sequence. It also delivers a single output value, but Mean expects a sequence. And so on.

Maybe it would help if all the inputs and outputs were of the same type. We're only dealing with double precision floating point values, and sequences of these. So, let's replace all single value contexts with sequences containing just a single value:

That's one possible approach. Note the simplification achieved in the After method. Now that all the atomic functions have the same input and output type, so will all compositions of these; we have a closed algebra.

Actually it's a bit too restricting to insist that all composable functions have the same input and output type. We would still be able to compose our atomic functional operations if their result type was different from the input, but we knew how to convert from one to the other. Next time we'll take a look at the Monads, and see how very surprisingly far this chaining idea can take us.

Future Builder

For any (purely hypothetical) future projects, we are unlikely to need a SQL builder design at all. Whatever our architectural object basis might be, let's say it uses SkinnyXmlObjects, we can derive SQL using some combination of LINQ to XML, LINQ to objects, and LINQ to SQL, with perhaps just a custom Skinny provider to handle Skinny specific details. But in fact we will probably bypass custom code entirely, and use a higher level, third party, persistence framework. Either way, the inherent complexity of the underlying mapping operations is certain to be managed by functional techniques.

Thursday, 8 April 2010

This started out as a reasoned argument about the benefits of good copyright law, the nature of infringement, the relation of preventative measures to criminal penalties and civil reparations, proportionality, and human rights. Then the words started moving around the screen, crossing lines and joining up in ways I never intended. Rhymes and rhythms appeared from nowhere. Now all that remains of my rational critique is the title. Everything else has somehow morphed into an old fashioned protest song. It's irrational, unreasonable and untidy. Worse, it's no longer even aimed at the correct target. I'm just glad I didn't write it.

Horse theft was once a hanging offence, for the arguably excellent reason that in pioneering days, taking a man's method of transport was precisely equivalent to taking away his right to life. Today's inner cities throughout the world harbour rings of corruption and vice, with similarly protective and extreme codes of retribution. And then there are those in the entertainment industry, who view copyright infringement as on a par with horse theft. Their livelihood is threatened; they smell blood; no penalty is too severe. Internet access might by now be a basic human right, but no matter. They'll continue to press for ultimate power over their victims, to demand disconnection from all aspects of life, for an act precisely as innocuous as lending a friend a book.

Anyway, here's the song, clean version. The original as mentioned above more or less writes itself; you should have no trouble reconstituting it with a packet of F-bombs. It's dedicated to all the music and other entertainment industry lobbyists out there, who lobbied hard and lobbied in secret, paid in gold and silver, to get the dreadful Digital Economy Bill forced through its crucial third reading in the House of Commons. You are a parcel of rogues.

Entertainers!

You're entertainers!You're an actor or a singer or you maybe play guitarYou're entertainers!But you've got some big ideas of who the hell you think you are

For you would kick us off internet if ever we're accusedOf taking copies of your work from sites we've never even usedAnd you would love to see our innocents and families deprivedOf their resources and of services that are their human rights

You are not teachers!Not a surgeon nor a doctor nor a fireman nor a copYou're entertainers!But your sense of self importance has no notion where to stop

And the power you demand's like disconnecting our supplyOf electricity or air or making water taps run dryYou'd prevent us online shopping you'd disrupt our educationAnd our private mail and messages and all communication

You're entertainers!Or you push the pens and count the beans of those who make that claimYou're entertainers!And you're a little bit deranged if you believe you're not to blame

Just as voters are responsible for all their countries' warsSo you're every bit as guilty as the suits who fight your causeDid you ever stop to think the prosecutions you pursueAre all against the very people who contribute most to you

You're entertainers!You're a dancer or a writer or you make computer gamesYou're entertainers!But it looks like you forgot the very streets from which you came

Did you seriously think we'd all lie down while you unfurledOur basic fundamental rights to interact with all the worldWell if copyright infringement disconnects us through the courtsI'll tell you I can dance and I can sing and you can eat my shorts

Rolled out of bed at 7 this morning and checked first thing, as you do, for any Porcupine Tree news. Turns out they've opened the presale, overnight, of their new, deluxe, special edition, 130 minute concert film Anesthetize on both standard definition DVD and high definition Blu-Ray disc.

Yippee!And just as abruptly, it's sold out.

Boo!

Actually it's only the red cover, 1,000 copies limited edition that's gone in the past few hours. The grey cover, 4,000 copies limited edition is still available. And apart from the fact that you have to paint it red yourself, it has exactly the same content:

The concert film on DVD and Blu-Ray, featuring a complete performance of the Fear of a Blank Planet album and 11 other tracks.

Taken from two concerts at Tilburg, The Netherlands, at the end of the Fear of a Blank Planet tour, October 2008.

Directed and edited by Lasse Hoile on high definition cameras, with the soundtrack mixed into stereo and 5.1 sound.

Two audio CDs of the entire live film soundtrack, with a bonus track not featured in the film.

All in a beautiful cloth bound hardback book, featuring concert photography of the band taken over the last few years.

Okay, it costs half a ton plus postage, but look, I can easily point my wife to Steven Wilson's previous special editions, his Insurgentes solo album and PT's last audio release The Incident, both of which are making good returns on ebay today.

And look - there's already a copy of the red version being offered on ebay. Hell's bells and buckets o'blood, there isn't even a release date for it yet. Give us a break, man!

Friday, 2 April 2010

In a week when the BBC's iPhone apps for news and sport are put on hold, due to complaints by the Newspaper Publishers Association, whose director David Newell has demanded they be blocked...

In a week when the Rupert Murdoch minion army announces its intention to have the Times and Sunday Times websites disappear behind a paywall...

In a week appropriately containing an April Fools' Day, this woman has been busying herself with the paste and the scissors, composing an ensemble piece all about the hands being wrung, and souls searched, today throughout one particular Church.

Ruth Gledhill is The Times Religion Correspondent. In this blog she offers her views on the issues of the day. So runs the blurb beside her mugshot. Her views? Would that it were true! And it continues: Your responses are invited.

Ah, good.

The subject of today's lesson shall be: the twin evils of plagiarism and copyright infringement. Ruth, from a total of 827 words in the forementioned article, fully 166, or 20%, are your own, and these are mostly of the type "She says...", "He writes...", "She said...". Particularly irksome, however, is your block transcription of the highly platitudinous 458 word Thought for the Day piece by Tablet editor Catherine Pepinster.

Normally of course, I would have time for neither your writing, nor Pepinster's, and still less for the worldwide organisation of kiddie fiddlers you write about. However, you really must get someone to explain copyright law to you, or else perhaps find a job that you can do. It is scarcely enough merely to note that it "...was so profoundly moving that I hope the BBC won't mind if I reproduce it..." and then pinching your nose, pull down the big fat Ctrl-V.

Did you also find the new Lily Allen album so profoundly moving, that you hope she won't mind if you tear off a few free copies to share with sell to your friends?

Is this the standard of journalism that we are to expect from The Times Online, when it starts charging daily for access to recycled content, blagged from the BBC where we have already paid for it once (and even then, not without a great deal of protest at the Thought for the Day show's exclusion of secular humanists)? In a publication which calls publicly for the dismantling of the BBC, the blocking of its technological development, the scrapping of its license fee "to create a level playing field", where its journalists can get down in the mud, rutting beside the likes of you?

I was very sorely tempted to repurpose your own headline, actually an India Knight quote, You can't take lessons in morality from people who disgust you, for this item; but that would have gone too far. Please, just try to remember that you're paid to write, not to compile scrapbooks of stolen trinkets.

Homage to Science Fiction's grandmasters.John And Linda's Big French AdventuresNotes from our 2010 & 2011 Brittany holidays.So Long PCW, and belatedly, Sub SetThanks for my (rewarding, but brief) writingcareer.Sony FB: Part One : Part TwoEvil Corporation in Bait And Switch!Wee MacThe story of our Border Collie (1993-2009).What's in a Gristleizer?Life as a solder jockey; recycling Golden Virginia tobacco tins.