TAGS

Today is a big day, I’m pleased to announce that “Design for Software: A Playbook for Developers” is finally available to the public! It was a long journey taking nearly 14 months to complete, but it’s here and I couldn’t be more excited. Check it out at designforsoftware.com

A little background

Over the past ten years I’ve worked closely with designers and developers in the emerging technology industry. During that time I’ve realized that designing software isn’t like any other type of design. Designing software is unique. It combines many fields of design into a single medium – Motion, typography, color theory, interaction design, information architecture, ethnography, and more. And for many reasons it deserves its own design process.

That is the premiss for this book – a process for designing software. My goal is to make this unique discipline more accessible for those that don’t have expertise in UI/UX design. Throughout the book I attempt to break down design theory concepts and present them in a way that makes sense for application design.

This book is written for non-designers and tech-savvy artists looking to create best in class software. I believe anyone can design great software, regardless of background. It doesn’t matter if you have a design degree (although it might help) or a history of being a hard core developer. Many design principles can be broken down into frameworks that help take the guesswork out of an otherwise abstract field.

By the end of the book you’ll have effectively learned all my design “secrets”. I hope that the design process I’ve written about is as useful to you as it has been for me.

Book Extras

It’s pretty difficult to write about interactive design in a static medium..go figure. During the writing process there were many times that static images didn’t do the content justice, especially when trying to describe motion. So I created a bunch of complementary web content.

You don’t have to buy the book to reap the benefits of the extras, but you should. They will make more sense…and it will make my dog happy (not my actual dog).

Where can I buy it?

The publisher (Wiley) has done an amazing job making the book available from a variety of retailers and in a ton formats. Of course I would encourage you to pick up the physical book, but the digital versions are good too. As of now, you can purchase the book from the following retailers:

We produce a ton of artwork for Windows 8 and when we present ideas to our clients I like the artwork to look as good as possible.

One way we do this is by placing the designs within a device frame. This gives the artwork some context, and a more premium positioning.

Mocking up artwork in a device frame is pretty commonplace for iOS and Android design, so I figured the Win8 community would benefit from these. Unfortunately, many of the MS Surface templates floating around the interwebs appear to be slightly inaccurate. They look as if a Samsung Series 7 and an iPad had an awkward kid. So I was compelled to have a legitimate go at making an accurate template. I’d like to think mine are at least 98.9% accurate.

Get the Goods

If you have Photoshop or Illustrator installed these should come in handy. And if you don’t…you can hand them off to your neighborhood friendly designer and I’m sure they’ll be pleased as punch…

Following up on my last post, I mentioned that we are creating a fun installation that involves tracking small fast-moving objects. So without further ado I give you Rally Keeper:

Overview

Before I get into how it works, I want to mention some of the inspiration for the project. A while back I saw this cool product concept for an interactive ping pong table. Inspired by the idea, the team and I spent some time pondering how we could take this “future” concept and turn it now concept.

Researching similar projects we examined a variety of approaches including audio triggers, triangulating sound waves, and color tracking. Then the ‘aHa’ moment came – we only needed to track the ball at a certain depth to capture it’s coordinates. Knowing that we could easily track depth with the Kinect, we quickly put together a prototype.

Aside from a being purely a cool visualization, we also wanted to add some utility. After a typical battle session and a brutal defeat, we often exclaim “I can’t believe you beat me! How’d you do it!?” Convinced, that if we could see the ball patterns we could calculate a better strategy for the next bout. Additionally, analyzing data is cool. With the ball data we can calculate percentages, make predications, and see trends in our games.

What’s Next?

After messing around with this for a week we came up with dozens of additional ideas we’d love to explore…time permitting. Some of the highlights include:

Authenticating purely with Kinect gestures

Leverage the Kinect’s microphone array for automated scoring

Pushing game results up to a website for comparison and analysis

Use projection mapping to draw the visualization onto the table in real-time.

With the latest SDK release, the Kinect platform has become pretty robust – skeleton tracking, gesture support, and access to the raw infrared stream all come “out-of-the-box”. Which is great, because if you’ve ever used any of the opensource equivalents, things can get pretty hairy.

Hands and skeletons are cool and all, but unfortunately there isn’t an *easy* way to do blob or generic object detection.. Right now, we’re working on a fun project and ultra fast blob detection is one of the major tech hurdles we’re working through.

After googlin’ around for blob-tracking with Kinect I came up empty handed. So I decided to look into OpenCV for doing blob detection. Many curse words later, I finally got things working the way I wanted. Along the way I noticed a handful of forums with people looking to do similar object tracking with Kinect, so I figured I would share how I cobbled things together.

Demo Overview & Source Code

The demo below is showing a couple of key things:

Consuming the raw Depth image from the Kinect sdk

Slicing the Depth image to a particular range

Using OpenCV to perform object/blob tracking

Rendering OpenCV output with WPF

Before We Get Started

Kinect SDK

Be sure to download the latest Kinect SDK. There are also a bunch of handy samples that come with the toolkit. You can get it here

EMGU (.Net Wrapper for OpencV)

Emgu is a .net wrapper for OpenCV. It is very complete and does a good job of providing .net hooks into the common OpenCV functions. Unfortunately, getting it to work with WPF is a little tricky (More below ). Also, I wasn’t able to get the lastest (2.4) version of the code to work with .net 4.5 and Visual Studio 2012, but the 2.3 version works well. Get it here

Kiblob (optional)

This codeplex project is similar, but it’s using another computer vision library called AForge and only compiles with a beta version of the Kinect sdk. Regardless, there are some handy tid-bits of code in there. You won’t need it to build my sample but its a good resource to look through.

Setup

Ok, now that we have the dependencies installed, its time to start setting up our WPF project. There are similar instructions on the Emgu site, but I found them a little confusing, so I’ll give you the cliffnotes version here.

Add References

After you’ve installed Emgu you need to add some .dlls to your project. This is the standard Solution->right-click->Add Reference and then browse to the Emgu installation path to add the following .dlls

Emgu.CV.dl

Emgu.UI.dll

Emgu.Util.dll

The next step is a little odd. You need to add 3 .dlls to the root of your project. Right-click the Project fill and select “add existing item”, then navigate to the Emgu installation folder and add the following:

opencv_core231.dll

opencv_highgui231.dll

opencv_imgproc231.dll

The last thing, and this is important, is to set the build action to “copy always”. This will dump those dlls into your build folder when you build the project. When you’ve got all that done you’re solution should look something like this:

Getting OpenCV to Play Nice with WPF

Ok, now that we have all the references in place we need to make openCV (Emgu) play nice with WPF. The biggest thing to note is that OpenCV uses it’s own type of image objects (IImage) which don’t work by default with the native WPF image controls.

Luckily, there is a little helper class that comes with Emgu that will convert the IIMage to a BimapSource, which works nicely with the WPF image control. By adding this helper method, we now have a convenient way to plug an OpenCV image into a WPF image. See the helper class here

The Fun Part

Now we can get to the good stuff. When you do blob tracking it is very important to have a “clean” grayscale image that has a high contrast between the object you are tracking and the background. This is one of the best parts about Kinect – it provides a grayscale depth image that we can analyze and extract out the regions of interest relatively easily. Let’s looking at the key pieces of the solution.

Depth Image Slicing

Like I mentioned above, when you are doing generic object detection you want to have high contrast between your object and the background. In other words, you want to isolate the object the best you can and ignore everything else. By doing so, you can ensure the vision library, like OpenCV, does a reliable job of finding the blobs.

The beauty of using the Kinect for blob tracking is that it gives us a depth image that contains all of the information we need to “slice” the image into the parts that we’re interested in. While I was trying to figure out the best way to slice the depth image I came upon a Kinect depth data video that really opened my eyes to working directly with the depth byte stream coming from Kinect. Check it out here

By simply ignoring (coloring pixels black) data that is out of the defined depth range, I was able to get a nice high-contrast image of the objects the depth camera was picking up. You can imagine this being something like an invisible plane that excludes everything in front and in back of it, which coincidentally creates a nice interaction space where we can track objects. You can see the slicing code here. This is the output:

OpenCV Contour Finding

The last thing to do is to get the blob tracking working. At this point we have a very clean image that we can feed into OpenCV’s “Find Contours” method. The FindContours method analyzes a grayscale image and returns the number of blobs as well as their bounding rectangles. It was a little tricky getting this to work in WPF, you can see the main piece of the code here.

Once I was able to figure out some of the cryptic tracking mechanisms in OpenCV everything came together. The cool thing about OpenCV is that not only can you get blobs, but you can blobs within blobs..in a tree. Crazy..I know. And lastly, we use our handy helper method from above (ToBitmapSource(OpenCVImg)) and plug it into the Image control in the main window xaml. Which ends up looking something like this:

Wrap up

The last thing I added were a few sliders to the UI to adjust the slicing parameters and the max/min Blob size. The image below shows the finished application.

All said and done, I am happy with the performance and excited about the opportunities to use this slew of technologies in some upcoming installation work.

Ok, so what?

I know what you’re thinking…so you have some blobs, big deal. And you might be wondering what else you can do with blobs. Well, let me tell you. By using generic blob detection you can not only detect when objects are present, you can also track their size, shape and rotation. For example, its possible to track an open hand transitioning to a closed fist for a grab gesture, or a user holding a wand or prop. Or in our case, we want to very efficiently capture the position of small, fast moving objects that penetrate an “invisible” IR plane. There are a ton of uses for blob detection including surveillance, card counting, and even real-time projection mapping.

What’s Next

While I’m pretty happy with the performance of this solution, I’m curious if it would be better/faster if I rewrote everything with the C++ flavor of the Kinect sdk and coupled it with DirectWrite or XAML to render the UI. Anybody out there do anything like this yet? If so, shoot me a tweet or email.

We’re currently rolling this stuff into a fun side project that I’ll post about shortly…stay tuned.

Thanks to everyone who attended my design talk at That Conference. It’s always great to see the dev community genuinely interested in learning about design. That Conference was awesome, the support of the attendees and overall energy at the conference was some of the best I’ve ever seen. Water park after party?! Nuff’ said.

New Content Highlights

The slides from the talk are available below, but before you go jumping in face-first, there is some new hotness worth noting. I posted this deck about a year ago, when I originally gave the talk at DevSum. And since then I’ve spent about a year turning the talk into a BOOK. And over the course of the year I’ve generated quite a bit of new content. Along the way I’ve created a few design tools that help make some mundane design/dev tasks a little easier.

Grid Helper

The first of the two tools I showed during the talk is called GridHelper. If time allows, my plan is to write up a more detailed post on how to use it. But here is the gist:

If you’re a designer or a creative coder you’ve probably needed a grid at some point. Whether you are aligning elements on a page, or creating a layout from scratch – grids are the layout power-tool.

If you’re like me, you probably hate the meticulous process of adding a bunch of guides to your photoshop or illustrator files. Or you hate writing code to generate a grid to overlay in your projects.

Last week Clarity was invited to the Windows 8 Developer Launch event in here in Chicago. Matt Hidinger and I were asked by some good friends at Microsoft to give our 2-cents on why we’re excited about Window 8. It was a packed event, and we had a great time. Below are some pictures, slides and a short recap on our talk. Pictures courtesy of Bob Laskey

Win8 : A Business Perspective

Here at Clarity we’re excited about Win8 for a number of reasons. One of the wins we see from a business perspective is reusability in three main areas: Design, Code, and Skillset.

Design – We can now share design assets and create a consistent brand experience across Microsoft’s three major screens: Xbox, Windows Phone, and now Windows 8.

Code – We primarily use C# and XAML to build our Windows Phone apps. Xbox, and Windows 8 also give us a framework to write C# and XAML. While the code isn’t necessarily copy & paste, there is a ton of overlap between the platforms.

Skillset – This is sort of a nice side-effect of having similar coding platforms. We are now sharing talent across our dev projects. In other words, our mobile guys are building Win8 apps and our XBOX guys are able to make the transition to Win8 or Win Phone just as easily.

Win8 : A Design Perspective

I’ve been building and designing software for a long time. And as a ‘design guy’ I tend to focus primarily on the UX aspects of software. With Windows 8, UX and aesthetics have finally been promoted to a first class citizen. This is great news for both designers and developers. This is especially exciting from a development perspective because it makes good design a hellava lot more accessible to the non-designer.

Metro makes Design More Accessible. . I like to think of Metro as a ‘developers kind of design’. It gets to the point, and promotes functionality over flash. Its about solving problems, visually, which is something we can all get on board with.

That last statement might seem a little broad. Specifically, the Windows 8 platform makes a handful of tedious design scenarios very easy for developers :

Touch First Controls – MS has finessed the crap out of these touch controls. They are accurate, and have a ton of user research driving their functionality. This is something that takes a lot of precision and therefore time to nail. Now you get it for ‘free’.

Animation – I’ve spend a good portion of my career figuring out the ‘perfect’ easing functions and timings to create snappy elegant motion in applications. MS has spent a bunch of time making the animation framework super easy to use for everyone. Its really good. Use it.

Built-in Styles – Margins, spacing, alignment, font-sizes, type-casing, font-weight. These are things that make most developers cringe. Well guess what, MS has been awesome enough to provide you with templates that have all of the base styling built-it so you don’t need to fret about those teeny-tiny pixels.

Win8 : A Development Perspective

Our own Matt Hindinger did an awesome job talking about the great new development opportunities that come with the Win8 Metro platform. I’ll make a feeble attempt to recap the highlights of his portion of the talk.

In Metro you have a choice to use 3 different development platforms : HTML5/Javascript, XAML/C++, XAML / C#. And the question we get asked the most is ‘when do I use what?‘. This is the when & where breakdown that we’ve come up with:

C++ XAML / DirectX – We think this platform is best used when you have a need for performance and precision. C++ and DirectX give you full control of every pixel on the screen and its super fast.

C# / XAML – This is our platform of choice. It offers a very reliable strongly typed coding environment that couples speed of development and a robust control set. If you come from a .net background, this is the one for you.

HTML5 / Javascript – We think this platform offers great flexibility and an awesome set of tools for loose and fast prototyping. If you are coming from a web background you will be pleasantly surprised at the modern CSS, js support. We’ve had really good success with this framework so far.

Our Slides

Below is our slides from the event. Check em’ out and share them with your friends. We had a lot of fun embracing the ‘trenches’ theme .

If you have any questions or thoughts you’d like to share with us don’t hesitate to hit us up on twitter.

Windows 8 is coming in hot! And here at Clarity we just couldn’t wait for MS to publish the official design guidelines. Our clients want to see their brand on Windows 8 now. So we meticulously picked apart screen shots and some XAML to create our own Photoshop templates. Get the unofficial Windows 8 photoshop templates below to kickstart your next Win8 project. Free of charge from your friends at Clarity Consulting

Whoa..8? I just got 7…

Yeah dude, the Windows 8 beta is right around the corner and the excitement is almost palpable. Over the past 6 months Microsoft has been slowly revealing the next evolution of Windows. The first glimpses of the platform came at BUILD where the developer preview was released along with a ton of sample apps and code snippets. Shortly after, MS publicly announced their plans for the Windows 8 Store. The Store announcement boasts some great opportunities for app makers and showcased some very nice apps.

The new Windows 8 platform has adopted the “Metro” design language that can be seen in Zune, Xbox, and Windows Phone products. The Metro styling has its roots in traditional Bauhaus design principles which, for this designer, is a breath of fresh air. Yet, for a platform driven by rich design concepts, guidelines and templates are sparse. Right now, the best documentation is scattered throughout the MSDN library and doesn’t give much in the way of practical design resources.

Why do I need this?

A few times now I have gone through the rigamarole of designing a Windows 8 app. And every time I go through the same process – take screenshots of my Window 8 tablet; creating guides and font sizes; start adding in content. In efforts to speed up our design process we made some template time-savers that I’d like to share.

Thanks to our senior photoshop template maker Brittany Travitz we now have some basic templates that should help you:

Set up proper margins and spacing

Find consistent Font Sizes & Weights

Create a basic application flow

What do I get?

The templates are primarly based on the Expression Blend windows 8 default project template. But we’ve also baked in some of the fine print found in these articles

There are four files included in the download. These files represent a bare-bones Metro application flow (see below). The four files include:

Home – This is the basic landing page or Hub of an application where you surface featured and curated content.

Category – One level below the home page is the category view. This is where you would present the full list of items for a given category.

About 5 months ago I gave a talk on Design for Software for DevSum11. It was a great event and I was pleasantly surprised at positive attitude the community had towards design. Shortly there after I was contacted by Wiley to see if I would be interested in elaborating the talk into a book. After a many fist pumps, I calmly proceeded to pursue the engagement.

Design For Software: A Playbook for Developers

As a formally educated ‘design guy’ I’ve had the unique opportunity to work along-side some rather brainy engineers here at Clarity. Even though I’ve sort of been the ‘odd man out’ I’ve gained some unique insights into the deep dark trenches of building software.

And over the course of my career I’ve hand-picked a creative process that won’t break the precious budget and cause timelines to get bogged down. That is the basis of the book, and what I’d like to share with both the tech-savvy artists and the creative-minded technologists of the world.

SO WHAT’S IT ALL ABOUT?

In my opinion well-designed software consists of three key ingredients:

Cognitive Psychology – Understanding and anticipating how users will use the software

Aesthetics – The fit n’ finish, bells & whistles, glitz & glam

Solid Code – The way it works and how it ‘feels’

Techy people, Arsty people, you are one in the same, with different priorities, but one great ability in common – Solving Problems.

However, designers and developers are typically viewed as opposing forces, left-brain vs. right-brain, if you will. This implies there is nothing ‘creative’ about developing software, which, most developers would tell is complete bull dooky. Perhaps its because we designers tend to gravitate towards the gray-area and developers have a natural propensity towards the black & white.

Regardless, some design principles in the context of software can be made black & white. And that is precisely what I’m talking about. Design is no more than solving problems, visually. And developers are already great at solving problems, so just because you didn’t go to design school doesn’t mean you don’t have the ability to make your software shine.

WHO IS IT FOR?

‘Design for Software’ is meant to connect the dots between software and design. Its targeted at tech-minded folks that have a taste for design, but aren’t sure what it takes to get there. Or the tech-savvy artists looking to fill the gap between art theory and cold hard code.

WHEN CAN I GET IT?
Unfortunately, I juuuust started the writing process and it will be a few months before anything is available to buy. My plan is to create a blog series that coincides with the high-level concepts that will be elaborated in each chapter of the book. I’ll key you posted as I make progress!

The Original Talk

Below is the original talk I gave at DevSum. It is more-or-less a high-level outline of what I’ll be writing about in the book.

Overview

It seems like everybody is getting bitten by the HTML5 bug lately. And as more folks jump on the band-wagon its going to become harder and harder to one-up the competition. Interestingly, the modern browsers are starting to offer some browser-specific functionality that can help make your shiny new HTML5 site stand out.

The Sticky Browser

It seems like the big trend in the browsers is to stick, pin, save, your sites somewhere convenient. I guess you can think of it as bookmarks 2.0. The biggest difference is how you access the saved site. This is important for three reasons:

It keeps the site in the user’s view

Offers app-like access with one click convenience

Increases the likelihood of visitor return

IE9 PINNED SITES
I think IE9 does the best job with this. It has concept of pinned sites which lets people attach a site to their taskbar or start menu. This is especially useful for html5 sites that are app-like in nature. The new Pandora site, grooveshark, and other utility apps like rulers, clipboard copiers, and dropboxes examples of apps that work perfectly with this concept.

Pinned sites are unique because they:

Attach to the taskbar/start menu in Windows

Give jumplist functionality for easy navigation

Allow notifications to get pushed to the OS level

OSX WIDGETS
OSX widgets aren’t anything new, but they do support html5 and can be a great companion to any HTML5 site. As a mac user I love widgets, I use them mostly for quick task-based interactions. Things like calculators, generators, quick-glance info, or notifications are all great uses for widgets.

Widgets are unique because they:

Live in the users OS for maximum convenience

Can access OS-level functionality like pasteboard, quartz composer, and run applescript

CHROME WEBSTORE
The chrome webstore has been filling out nicely with new and exciting web-based experiences. All the cool kids are putting their ‘apps’ in the webstore and you should too. I found it to be as easy as adding some basic javascript to your site and uploading it. With a nice apple-esque interface, the webstore should give your site some street-cred making it feel a little more official. Its also a great way to get feedback, rating, and track the success of your stuff.

Web-apps that exist in the webstore have some advantages:

Usage and rating analytics

Full screen or “Pinned Tab” Mode

Discoverability and exposure from the web store

Nice launch surface in the browser

FIREFOX ADDON
In my opinion, the firefox addon model is a little old-school and techy for the new immediate-gratifyin’ breed of users out there. But it is still a great way to give your site that little extra bump of usability. I haven’t actually built anything as a firefox addon but it seems like it works best when used to augment a core feature of a site. Evernote and StumbleUpon are two good implementations I’ve seen.

Firefox Addons are unique because they:

Integrate with Firefox’s native functionality

Offer quick and easy access to common tasks

Summary

Long story short, the HTML5 party is getting crowded….fast. Exploiting some the browser-specific functionality can be a great way to increase the accessibility and stickiness factor of your site. And if anything, it may be a way to squeeze out a few extra delight-points from your audience.

Overview

I recently gave the Design for Software talk to my colleges here at Clarity, and not surprisingly, the typography portion raised the most questions. Unfortunately there really isn’t a definitive source for learning how to set type “the right” way. So I decided to drill into some key principles of digital typography.

There are a few typography questions that seem to bubble up more than the rest:

What sizes should I pick for headers and body text?

What are some typefaces look good together?

Should I use serif or san-serif?

In this first post I’ll try to shed some light on the topic of choosing a suitable scale for sizing your screen elements. In the following posts I’ll attempt to tackle the fine art of pairing typefaces and give some general rules of thumb for using serif and sans fonts. For the record, I am no expert on the subject, just enthusiastic about it and hopeful that other tech-minded creatives will find this post helpful.

Rhythm and Scale (What sizes should I pick?)

Choosing the right font sizes for your application involves three key components: Scale, Leading and consistent Vertical Rhythm. Once you wrap you mind around these little nuggets making space and size decisions should be a breeze.

Scale

Scale refers to the intervals of size and the relationship between them. There are many different scales available for many scenarios- these two are well suited for most typographic tasks

Scale in typography is akin to scale in music. Choosing notes in the correct scale result in harmonious chords and tones. The same is true for choosing a scale for typographic elements, arbitrarily picking sizes can result in a discorded visual rhythm.

When you’re deciding what size to make your headers a good rule of thumb – multiply your body size text by 2 and then find the nearest value in your scale.

Leading

Leading is the space between the baseline of the type and the next. In CSS and most programming languages this is the Line-Height property. Most classic typography books recommend a general leading value of 1.5em or:

Anything less than 1.5 is typically too cramped for comfortable readability.

Vertical Rhythm

The Vertical Rhythm is the spacing and positioning of text blocks as they descend down the page. The primary characteristic of perfect vertical rhythm is when the spacing between paragraphs and headers are equal or related to the leading. In other words, equal spacing between headings and paragraph text is ideal for keeping a consistent vertical rhythm.

Rhythm & Scale Widget

So far so good, BUT, programmatically positioning elements on a page can be a real downer. Adding margins and spacing manually to all your textblocks could get real cumbersome real fast.

The good news is that with some simple math inspired from this site we can calculate the CSS values for keeping good vertical rhythm. I created this html5/OSX widget to generate the base CSS values for scale and baseline aligning text in html.

Using the widget

First, choose a comfortable reading line height value. The amount will vary depending on your typeface…sans-serif faces typically need more than serif faces. Depending on my base font size I typically choose a value between 18 and 24.

Second, pick between one of the two common size scales. The Diatonic Scale is based on traditional typography sizes and the Golden Ratio Scale is based on the Fibonacci sequence (for natural-looking size values).

Last, preview the generated css on the backside of the widget and copy it to the clipboard. The values generated should be a good starting point for establishing vertical rhythm and font sizes.

*Bonus* If you are a mac user, download the dashboard widget. If you are an ie9 user, pin the site to your taskbar. And if you are a chrome fanatic get the widget HERE from the chrome web store

Generated Sample

Check out this sample page to see all this theory stuff working together. Taking the generated css from the widget I added a couple of css rules for columns and the bingo-bango…everything lines up nice.

Resource Roundup

During my exploration of this topic I found some great resources that give a much deeper dive into the typic of rhythm and scale. If you are interested in learning more these are key: