Kinect for Windows SDK

I read the other day that we released a new version of the Kinect for Windows SDK that is supported by Windows 8. With as busy as I’ve been working on IE10, I’ve been a bit of a slacker when it comes to trying new things so I decided that it was time to dive in and take a quick peak at what I could do with Kinect.

In a matter of a few minutes, I managed to play around a bit with the depth image sensors and the color image sensors. I also mucked around a bit with the elevation and accelerometer data. I’m not going to get fancy with this post so hang onto your hat and follow along. If you’re the one random guy that stumbles on this post and actually reads it and has questions, feel free to ask. I mean, I won’t know how to answer it, but you can always ask!

Here goes.

Purchase

There isn’t much sense in doing anything if you haven’t purchased a Kinect for Windows device already. I am very specific in stating Kinect for Windows. While there are some folks out there that provide you with the ability to use your Kinect for Xbox 360, I wouldn’t recommend it. Go get yourself one of these devices and we’ll all wait patiently till you come back.

[tapping foot]

Download/Install/Setup

Download and Install the Kinect for Windows SDK. Make sure you get the latest one from October 8th, 2012.

Plug your device into your computer and make sure to plug the power into the device. Plug and play is your friend here as there is nothing to install to make the device available.

Start Coding


Setting up your Form

  1. Open up Visual Studio 2012
  2. Go to File | New Project
  3. In the dialog, select C# | Windows | Windows Forms Application (obviously you can do this in other languages, but I’m using C# here)
  4. Right-click the References tree node in Solution Explorer and select Add Reference
  5. Press Ctrl+E, type “Kinect“, check the “Microsoft.Kinect” assembly and click OK
  6. Add two PictureBox controls and a TrackBar control to Form1
  7. Set the TrackBar Maximum to 27 and the Minimum to -27. These are the range values for the camera elevation angle.
  8. Add a Timer control to the Form and set the interval to 1000ms (1 second)
  9. Double click Form1 to go to Code view

Coding

Add a couple of using statements at the top of the form:

using System.Drawing.Imaging;
using Microsoft.Kinect;
using System.Runtime.InteropServices;

Define a variable to reference the Kinect sensor inside the Form class. I also added some variables to hold my sensor data:

public partial class Form1 : Form
{
    KinectSensor sensor; 

    // Depth sensor readings
    int depthStreamHeight = 0;
    int depthStreamWidth = 0;
    int depthStreamFramePixelDataLength = 0;

    // Color sensor readings
    int colorStreamHeight = 0;
    int colorStreamWidth = 0;
    int colorStreamFrameBytesPerPixel = 0;
    int colorStreamFramePixelDataLength = 0;
    ...

Set the reference to your sensor. We’ll be lazy here and just reference the first in the array. Then make sure to enable the depth stream and color stream. Here’s the code inside my form load handler:

private void Form1_Load(object sender, EventArgs e)
{
    sensor = KinectSensor.KinectSensors[0];
    sensor.DepthStream.Enable();
    sensor.ColorStream.Enable();
    sensor.DepthFrameReady += sensor_DepthFrameReady;
    sensor.ColorFrameReady += sensor_ColorFrameReady;
    sensor.Start();
}

You’ll notice that I also hooked up two event handlers here. These handlers fire when a frame is ready for rendering or otherwise processing. In our case, we’re just going to muck around with the data and dump it into the picture boxes. After we’ve set up our handlers, we go ahead and call start() on the sensor.

So now we need to create our handlers. In our handlers, we are going to:

  1. Attempt to open the frame
  2. Read the stream data for height, width, pixel length and bytes per pixel
  3. Convert the raw data to a bitmap
  4. Assign that bitmap to the picture control

Here’s my code for the sensor_DepthFrameReadyhandler:

private void sensor_DepthFrameReady(object sender, 
    DepthImageFrameReadyEventArgs e)
{
    DepthImageFrame imageFrame = e.OpenDepthImageFrame();
    if (imageFrame != null)
    {
        depthStreamHeight = sensor.DepthStream.FrameHeight;
        depthStreamWidth = sensor.DepthStream.FrameWidth;
        depthStreamFramePixelDataLength = 
            sensor.DepthStream.FramePixelDataLength;
        pictureBox2.Image = GetDepthImage(imageFrame);
    }
}

The GetDepthImage function is doing the conversion work for us. This is the code that does the conversion from the raw image frame data to a bitmap (btw, if you know an easier way to do this, please chime in. I’m not a graphics dev).

Bitmap GetDepthImage(DepthImageFrame frame)
{
    // create a buffer and copy the frame to it
    short[] pixels = new short[sensor.DepthStream.FramePixelDataLength];
    frame.CopyPixelDataTo(pixels);

    // create a bitmap as big as the stream data. Note the pixel format.
    Bitmap bmp = new Bitmap(depthStreamWidth, depthStreamHeight, 
            PixelFormat.Format16bppRgb555
        );

    // Create a bitmap data, size appropriately, and lock the data
    BitmapData bmpdata = bmp.LockBits(
            new Rectangle(0, 0, depthStreamWidth, depthStreamHeight),
            ImageLockMode.WriteOnly,bmp.PixelFormat
        );
    // Wave your magic wand and unlock the bitmap
    IntPtr ptr = bmpdata.Scan0;
    Marshal.Copy(pixels, 0, ptr, depthStreamWidth * depthStreamHeight);
    bmp.UnlockBits(bmpdata);

    // return the bitmap
    return bmp;
}

It’s important to note the pixel format in this function because, when we create a similar function for the color frame, we will use a different format. You’ll want to make sure you use the right format depending on what data you are getting.

Here’s my code for the sensor_ColorFrameReady handler:

void sensor_ColorFrameReady(object sender, ColorImageFrameReadyEventArgs e)
{
    ColorImageFrame imageFrame = e.OpenColorImageFrame();
    if (imageFrame != null)
    {
        colorStreamHeight = sensor.ColorStream.FrameHeight;
        colorStreamWidth = sensor.ColorStream.FrameWidth;
        colorStreamFramePixelDataLength = 
            sensor.ColorStream.FramePixelDataLength;
        colorStreamFrameBytesPerPixel =
            sensor.ColorStream.FrameBytesPerPixel;
        pictureBox1.Image = GetColorImage(imageFrame);
    }
}

Here’s the code for the GetColorImage function:

Bitmap GetColorImage(ColorImageFrame frame)
{
    // Create a buffer and stuff the frame pixels into it
    byte[] pixels = new byte[colorStreamFramePixelDataLength];
    frame.CopyPixelDataTo(pixels);
    
    // Create your bitmap. Notice the PixelFormat change!
    Bitmap bmp = new Bitmap(
            sensor.ColorStream.FrameWidth, colorStreamHeight,
            PixelFormat.Format32bppRgb
        );

    // Create a BitmapData objet and lock
    BitmapData bmpdata = bmp.LockBits(
            new Rectangle(0, 0, colorStreamWidth, colorStreamHeight),
            ImageLockMode.WriteOnly, bmp.PixelFormat
        );

    // Wave your magic wand again and copy the data. Notice this time
    // we multiply by the bytes per pixel? This is important to get the right
    // data size copied.
    IntPtr ptr = bmpdata.Scan0;
    Marshal.Copy(pixels, 0, ptr, 
            colorStreamWidth * colorStreamHeight * 
            colorStreamFrameBytesPerPixel
    );
    bmp.UnlockBits(bmpdata);

    // Return the bitmap
    return bmp;
}

All done well, you should now be able to run your application and get something like the following screen shot:

Kinect For Windows Sample - ScreenShot

Kinect For Windows Sample – ScreenShot

Next, we want to add some control for the elevation of the Kinect. Of course, the Kinect does have limits so it can only move so far. In this post, we’ll just move the X component of control. We will use the TrackBar control that we added to the form at the beginning of this post.

First, a little caveat. We don’t want to move the elevation of the Kinect very often. In fact, the API documentation for Kinect tells us that it is very bad to move it often as you’ll wear out the controls. For that reason, we don’t want to move the Kinect on every event received from changing the TrackBar. That would cause problems. Instead, we will update a variable when the TrackBar is changed, and use a timer that fires every second (added earlier in the project) to do the actual validation of the data and updating of the elevation.

So, add the following declarations to your form:
"]public partial class Form1 : Form
{
KinectSensor sensor;
int lastElevation = 0;
int elevation = 0;

At the very bottom of your Form_Load event tell the timer to start:

timer1.Start();

Now, create an event handler for your trackBar’s ValueChanged event as follows:

private void trackBar1_ValueChanged(object sender, EventArgs e) {
    var value = trackBar1.Value;
    var reading = sensor.AccelerometerGetCurrentReading().X;
    if (value > (sensor.MaxElevationAngle + reading)) {
        value = sensor.MaxElevationAngle;
    }
    else if (value < (sensor.MinElevationAngle + reading)) {
        value = sensor.MinElevationAngle;
    }
    elevation = value;
}

What we are doing here is first reading what the current reading is on the Accelerometer’s X component. We do this because that can have an affect on how far the sensor can move. Next, we will do some validation to make sure that the elevation angle added to the reading isn’t greater or less than the maximum or minimum values allowed (currently 27 and -27 respectively). When we change the trackBar, it updates our elevation variable. Now we need to do something with that variable.

Create a Tick event handler for the timer control you added. Add the following code:

private void timer1_Tick(object sender, EventArgs e)
{
    if (lastElevation != elevation)
    {
        sensor.ElevationAngle = elevation;
        lastElevation = elevation;
    }
}

Again, we are just doing validation to see if the last elevation is different from the current elevation. If it isn’t, then we go ahead and move the sensor and update the current position.

All this should get you to a state where you can change the sensor elevation.

There’s a lot more fun to be had, but it took me longer to write this post than it did to write the code. I suspect you can take it from here faster than you could read my post. Happy coding!

Cross-browser event hookup made easy

I seem to be on a roll. I’m coding and might as well be sharing some of what I’m working on while I do it. In that sense, here goes another small bit of code that I’ll be adding to jsSimple on github.

We all know how to hook up events cross-browser. We know that we have both attachEvent and addEventListener. Both have varying syntax and we’ve all written this code a million times. Let’s just make this simple. Here’s how to call this:

Event.add(element, 'click', callback, false);
// .. 
Event.remove(element, 'click', callback, false);

The remove looks like you’re passing around the same data, so I added a return value to “add” that allows you to capture the event and pass it in as a single object to the remove function as follows:

var evt = Event.add(element, 'click', callback, false);
// .. 
Event.remove(evt);

So, without further delay, here is the object definition:

/*!
 * jsSimple v0.0.1
 * www.jssimple.com
 *
 * Copyright (c) Tobin Titus
 * Available under the BSD and MIT licenses: www.jssimple.com/license/
 */
var EventObj = function (target, type, listener, capture) {
  return {
    'target' : target,
    'type' : type,
    'listener' : listener,
    'capture' : capture
  };
};

var Event = (function() {
  'use strict';
  function add (target, type, listener, capture) {
    capture = capture || false;
    if (target.addEventListener) {
      target.addEventListener(type, listener, capture);
    } else if (target.attachEvent) {
      target.attachEvent("on" + type, listener);
    }
    return new EventObj(target, type, listener, capture);
  }
  
  function remove(object, type, listener, capture) {
    var target;
    if (object && !type && !listener && !capture) {
      type = object.type;
      listener = object.listener;
      capture = object.capture;
      target = object.target;
    } else {
      capture = capture || false;
      target = object;
    }
    
    if (target.removeEventListener) {
      target.removeEventListener(type, listener, capture);
    } else if (target.detachEvent) {
      target.detachEvent("on" + type, listener);
    }
  }
  
  return {
    'add' : add,
    'remove' : remove
  };
}());

You can see this all in action on jsFiddle.

JavaScript animation timing made easy (requestAnimationFrame, cancelRequestAnimationFrame)

I often find myself writing a lot of the same code from project to project. One of those pieces of code is around using requestAnimationFrame and cancelRequestAnimationFrame. I wrote a quick and dirty animation object that allows me to register all of my rendering functions, start the animation and cancel it. Here’s an example of how to use this object:

function render () {
   // Do your rendering work here
}
Animation.add(render);
Animation.start();

It can also be used to add several animations before and even after Animation has started, you can add new rendering routines on the fly. This is useful in a number of instances where objects have their own render routines and you want to add rendering routines at different times.

I put in a few workarounds. The typical hack to allow setTimeout/cancelTimeout fallbacks are in there. There is also a work around to deal with the missing mozCancelRequestAnimationFrame and the fact that mozRequestAnimationFrame doesn’t return an integer per spec.

So here’s the code for the Animation object:

/*!
 * jsSimple v0.0.1
 * www.jssimple.com
 *
 * Copyright (c) Tobin Titus
 * Available under the BSD and MIT licenses: www.jssimple.com/license/
 */
var Animation = (function() {
  "use strict";
  var interval, moz, renderers = [];

// PRIVATE
  function __mozFix (val) {
    /* Mozilla fails to return interger per specification */
    if (!val) { 
      moz = true;
      val = Math.ceil(Math.random() * 10000);
    } 
    return val;
  }

  function __renderAll() {
    var r, length;

    // execute all renderers
    length = renderers.length;
    for (r = 0; r < length; ++r) {
       renderers[r]();
     }
     if (interval) {
       interval = __mozFix(window.requestAnimationFrame(__renderAll));
     }
   }
   function __registerAnimationEventHandlers() {
     var pre, prefixes = ['webkit', 'moz', 'o', 'ms'];
     // check vendor prefixes
     for (pre = prefixes.length - 1; 
          pre >= 0 && !window.requestAnimationFrame; --pre) {
      window.requestAnimationFrame = window[prefixes[pre] 
            + 'RequestAnimationFrame'];
      window.cancelAnimationFrame = window[prefixes[pre] 
            + 'CancelAnimationFrame'] 
            || window[prefixes[pre] + 'CancelRequestAnimationFrame'];
    }

    // downlevel support via setTimeout / clearTimeout
    if (!window.requestAnimationFrame) {
      window.requestAnimationFrame = function(callback) {
        // could likely be an adaptive set timeout
        return window.setTimeout(callback, 16.7);
      };
    }

    if (!window.cancelAnimationFrame) {
      window.cancelAnimationFrame = function(id) {
        clearTimeout(id);
      };
    }
  }
  __registerAnimationEventHandlers(); 

// PUBLIC
  function add(renderer) {
    if (renderer) {
      renderers.push(renderer);
    }
  }

  function isRunning() {
    return !!interval;
  }

  function start() {
    interval = __mozFix(window.requestAnimationFrame(__renderAll));
  }

  function stop() {
    if (!moz) {
      window.cancelAnimationFrame(interval);
    }
    interval = null;
  }

// OBJECT DEFINITION
  return {
    "add": add,
    "isRunning": isRunning,
    "start": start,
    "stop": stop
  };
}());

I’ve put this code on jsFiddle to test out.

I’ve also put this on github as jsSimple. I’ll likely add a few other bits of code to this as I move along.

CSS: Opacity vs Alpha (via rgba and hsla)

Once again, I was working on my new blog theme again and ran into the need to set the opacity of a background div without affecting the children. The support for opacity is spotty and slightly unpredictable. The other downside is that setting opacity via CSS affects the children of the transparent object. You can get around this by using an rgba background colors and specifying the alpha channel.

background-color: rgba(24, 117, 187, .5)

This specifies a background color of red: 24, blue: 117, green: 187 and an alpha of 50% transparency. I put up an example of this on jsFiddle here: http://jsfiddle.net/tobint/gazuB/

Here’s a side-by-side example of the differences between opacity and rgba.

Figure demonstrates opacity vs alpha

The div on the left has set an opacity of .5 on the div container. On the div on the right, I’ve set the background color with an alpha of .5 using rgba. Notice that the sample on the left has affected the transparency of the child div, but the sample on the right has not.

You can also set colors using hsla (hue, saturation, and lightness).

background-color: hsla(206, 77%, 41%, .5)

Of course, this only works in browsers released in the last few years:

  • Safari 3.1 in March 2008
  • Firefox 3 in June 2008
  • Chrome 4.0 in January 2010
  • Internet Explorer 9 in June 2011

Detecting CSS3 Transition Support

I’m in the process of developing a new blog theme for myself. I was trying to use CSS3 transitions but I wanted to fall back to JavaScript if transitions weren’t supported. I really want to avoid Modernizr if I can so I needed a quick and dirty test for detecting them. Here’s what I came up with:

var Detect = (function () { 
  function cssTransitions () { 
    var div = document.createElement("div"); 
    var p, ext, pre = ["ms", "O", "Webkit", "Moz"]; 
    for (p in pre) {
      if (div.style[ pre[p] + "Transition" ] !== undefined) {
        ext = pre[p]; 
        break; 
      } 
    } 
    delete div; 
    return ext; 
  }; 
  return { 
    "cssTransitions" : cssTransitions 
  }; 
}());

It’s simple and does the trick. Simply check the return of Detect.cssTransitions().

if (!Detect.cssTransitions()) {
   // Do JavaScript fallback here
}

You can also get the extension used in the current user-agent back from this method (ms, O, Moz, or Webkit).

Powershell: Selecting files that don’t contain specified content

I had a quick task today that required that I search a couple hundred directories and look in specific files to see if they contained a certain piece of code. All I wanted was the list of files that did not. PowerShell to the rescue!

I’ll break down each of the calls I used and provide the full command-line at the end for anyone interested.

First, we need to recurse through all the files in a specific directory. This is simple enough using Get-ChildItem.

Get-ChildItem -include [paths,and,filenames,go,here] -recurse

Next, I needed to loop through each of those files using ForEach-Object :

ForEach-Object { … }

Next, I need to get the content of each of those files. That’s done using Get-Content:

Get-Content [file]

I then need to be able to determine if the contents of the file contains the string I’m looking for. This is easy enough with Select-String:

Select-String -pattern [pattern]

That’s pretty much all the commands you need to know — we just need to put them together now.

PS> Get-ChildItem -include *.html,*.xhtml -recurse |
    ForEach-Object { if( !( $(Get-Content $_) | 
                            select-string -pattern "pattern") ) 
                     { $_.PsPath }}

Community Question: Synching Remote Locations

I’ve been thinking about a problem that I’ve solved before. I’ve solved it a million ways but I’ve never really thought about which approach is best. I’ve decided to request some input on the subject. I’d appreciate your thoughts.

The Scenario

You’re a business with multiple locations and a single website. You want be able to allow customers to log into your site, and order something from one of your locations, but you want to give the customer immediate confirmation that their order was received by the store location — not just that the website received the order.

That means you only want to let the user know they can rest at ease knowing their order has been received by the people directly responsible for holding/delivering/shipping your product.

Application

The scenario above could describe any number of business. This could be a retail business. You may want to make sure that the store knows to hold a piece of inventory for a customer who is on their way to get it (think Best Buy’s “in store pickup”). It could also be something much more loosely connected by even legal boundaries like OpenTable does when it allows you to make reservations for a number of restaurants for open times.

Solution Approaches

There are obviously a number of ways to solve this problem and the application may very well dictate the solution you pick. There are three main approaches, and within those approaches, there are several technical ways to solve the problem.

Centralized Approach

In the OpenTable application, you could, for instance, require all reservations be made through OpenTable’s website. In that case, it means that even if someone called a restaurant, they would need to use the website.

The downside to this is what happens when the internet connection between the restaurant and OpenTable.com is interrupted. People calling the restaurant wouldn’t be able to make reservations but people on the web would. Alternatively, the restaurant and the web would be out of sync and more reservations would be made than are available.

Distributed/Brokered Approach

This approach says that each location is responsible for it’s own allocation of inventory/product/time slots. The website simply acts as a broker and sends orders to the location. This means that the website must collect the intended details and get them to the store location. There are two problems with this approach.

First, the communication from the centralized approach is still problematic. In the reservation scenario, you can’t simply store the request at the server and send it when communications return. The reservation may not be available. Ditto that with the inventory issue. If that elusive Tickle-me Elmo doll is in stock, you want the customer to be able to buy it and secure the inventory for in-store pick-up.

The second issue to determine is whether you want a push mechanism from the website to the remote location, or if you want a pull mechanism that polls for details on a regular basis. Because the inventories are disbursed, there is also some synchronization that needs to occur. You need to make sure that if you say there is an item in inventory, that there is. If it’s purchased at the store, you need the website to reflect that. If it’s purchased/reserved on the website, you need the item/reservation to be updated at the location.

Delegated Approach

There is another approach which is a bit less risky. This approach delegates a certain amount of guaranteed inventory from a remote store location to the web. This allows you to treat the web as it’s own store, handle it’s own inventory, and simply sync with the remote locations as you can. This is a problem for retail locations that do this, however, because you have to maintain two different inventory statuses — in inventory, but allocated for in-store pickup. It’s not as big of a deal for things like restaurant reservations because they aren’t “real” entities that require inventory shuffling.  There are definitely benefits to this approach, but it wouldn’t work for something like, for instance, grocery delivery.

My Solution

The solution I’m inclined toward is to use the distributed approach. When someone submits a request, it will be queued up, and an attempt will be made to submit it to the remote location, when the remote location receives the order, it will return a confirmation which will then be returned asynchronously to the user (either through polled HTTP requests or other means — Twilio, for instance). If the location cannot be reached, it will simply return an error to the user with instructions on how to contact the store directly. This is a fairly common approach.

Questions to the Community

So now that I’ve laid out my thinking, I’d like to ask some questions to you all. If you feel so inclined, please feel free to post your responses in the comments. Thanks very much in advance.

  1. Am I missing any approaches above? Is there another approach I haven’t considered?
  2. Are there any downsides that I’m missing?

I’m looking forward to hearing from you.

Taking on New Challenges: Joining the IE Team

How I Started

I’ve been into computers since I was a kid. The story is long, but I essentially started writing software in the 5th grade at North Elementary School in East Liverpool, Ohio. We had a quick introduction to computers. Our teacher discussed high-level and low-level languages and then introduced us to BASIC programming. I wrote my first program:

10PRINT “TOBY”
20GOTO 10
RUN

(Aside: For those that don’t know, I begrudgingly admit I went by “Toby” back in those days).

800xl[1]My eyes filled with wonder as my name scrolled down the screen. I immediately looked at the next instructions, and the next, and the next, and the next. My dad picked up an ATARI 800XL for the house. I spent many a full day at home working on that computer — painstakingly putting Microsoft ATARI BASIC and ATARI Assembler instructions from programming books and watching with wonder as the screen flashed exactly what I told it to do and the four voice, 3.5 octave sound screamed at me. I’ll spare you the details, but those were the beginnings of my fascination with computers. Some years later I had, for a number of reasons, stopped programming on the ATARI. When it was time to go to college, my parents got me a Packard Bell 486 DX2 66 computer. I went to college for Graphic Design – not computer science. However, when I found BASIC on the computer, I suddenly found myself hanging out with the C.S. folks more than my own classmates.

The Wonder of the Web

packardbell[1]After three semesters of college, I ran out of cash and my choice of Graphic Design over computer science made it hard for me to want to go back to college. I moved to Greenville, SC in 1994 and started playing with my Packard Bell. I mostly used it to talk on BBS’s and occasionally AOL when I could afford the time (for those of you that don’t know/remember back that far, you used to get charged by the minute for AOL use). I spent a bit of time writing small desktop applications with my copy of Visual Basic 3, patching software to get it to do what I wanted, or adding photos of myself inside my favorite games to my friends amazement. But I also started playing with HTML – something that had not yet standardized, but that I wanted to be a part of. I’ll never forget my first webpage. It had a picture of Alicia Silverstone on it – the “chick from that Aerosmith video”. The thing I liked about the web is that it combined my love for graphic design with my need for technical understanding.

I’ll again spare you the details and skip forward a few years. I was working for a consulting company called Metro Information Services as a contractor for AIMCO – at the time it was the largest property management REIT in the country. I worked on several web-based intranet applications in the 1999-2000 timeframe getting use something new called XML Data Islands, DHTML behaviors and something new called XMLHttpRequest. This allowed me to create applications that appeared more responsive and did work behind the scenes on users behalf. I fell in love with this technology with Internet Explorer and probably overused it. With just HTML and JavaScript, I felt back then that eventually most applications would become web-based with very little need for ActiveX or Java fillers. I knew that the server was still an important part too so I continued to work on enterprise middleware and database development. But I knew that the web – both server and client — was where I was meant to work.

Moving to Microsoft

Skipping forward to 2006, I had worked contract after contract, making a lot of money but starting to get bored and burned out. I was going to take a year off work and “play” with the technologies I loved most. Then Microsoft called. They had an opportunity to work with the teams that built ASP.NET and IIS. This seemed like the best opportunity so I took it. I joined as a Programming Writer for IIS. You could summarize this as simply a “documentation writer” position if you didn’t know any better. That said, I came to realize that this job required skills as both a great communicator and a great programmer. You needed to learn how to analyze API code that was written, and describe how to use it without any help. I loved my job when I first started it. However, the continued pressure to document every exposed method no matter how trivial it was also began to weigh heavily on me. I loved the web and this position seemed like the perfect position for me – till you find yourself writing documentation such as “The IAppHostSectionDefinition::Name Property Gets the name of the current configuration section definition.”

loginI had a growing desire to fix MSDN. As a developer, there were so many things that I felt I could help MSDN with, I thought I couldn’t go wrong. I loved working with the MSDN and TechNet teams. I helped redesign the home page based off of customer feedback, metrics we collected and the overall direction the company was heading. I started the MSDN News program which allowed us to aggregate content from various places inside and outside of the company and disseminate it across multiple locations. I designed and MSDN Insiders program. At every turn, however, I just felt like I was building someone else’s vision rather than making the impact I wanted to. I also, strangely enough, wasn’t getting to use the latest, greatest technology to build MSDN. I had a heavy desire to build HTML5 sites, mobile sites, and incorporate features like pinning. It’s not that I didn’t believe in what was being built, but my passions for it were not there.

Introduction to IE

UntitledI started diving into HTML5, CSS and JavaScript pretty heavily again. Inspired by the type of work that Rey Bango was doing, I wanted to reestablish my passion for the web. I began talking to people about this passion and paying more attention to what the community was saying about standards, development challenges, and of course, browsers. As I read more books I noticed that IE was rarely mentioned and if it was, it was in a bad light. Some of this was perception but some of it was reputation earned long-ago on the backs of still-lingering down-level versions of the product. I became angry when I saw a cartoon being passed around twitter. The cartoon, which many of you may have seen, was of three boys dressed in browser costumes. A Chrome-costumed boy had a Firefox-costumed boy in a headlock and they were clearly battling it out – each shouting the name of their product while holding a determined look on their face. In the corner was a baby-blue pajamad boy with a baby-blue helmet on. The helmet, of course, had an “e” on it to indicate Internet Explorer. This boy wasn’t battling at all. He was eating paste! This made me mad because I knew that the IE team was actually listening for feedback and had made considerable investment in battling back since IE6. I knew we still had our problems in IE, but certainly they didn’t deserve the “touched boy eating paste” image.

I sent a well-thought out email to the VP of IE and expressed my frustration with not only the cartoon, but the idea that we weren’t even considered a player for many people. I expected to get a pink slip for this “career-limiting email”. How often have you heard about someone telling a VP that their product needed work to be rewarded for it? Not often, in my experience. However, I was surprised to see an email come back from the VP fairly quickly. At the end of my email I asked “what can we, as employees, do to help with these problems?” The answer for this VP was pretty simple – “come and fix it.” At the VP’s request, I met with a man from IE. He listened to me and what I was passionate about and what I felt needed fixed. He asked me if I had worked with HTML5 canvas yet to which I told him I did not. He asked me to go play with it for a couple weeks or a month and let me know if I liked it and what I thought about working with it for a while. I wrote a few quick[1] demos[2] in a few days with HTML5 and shot them over to him. They weren’t great so I expected him to hate them. I hadn’t optimized my code. I wasn’t using standards. I wasn’t doing all the things someone interested in performance should do.

darkbookWe met again for coffee and he listed some of these things on the board as we chatted. At the end of the conversation he said “these could be your commitments.” Long story short, after a few more discussions I was given an offer to come join the IE performance team. I’ve got my work cut out for me. The people I met with in IE are top notch guys. They know their stuff inside and out. They know not only the code base and how it works, but how it should work. I’m told it’s going to take me about a year just to ramp up and be slightly effective. These guys have been working on the product for several years on average. I’m humbled to be able to work with them. I doubt I’ll ever be as smart as them, but I’m hoping to make my small impact on the product.

I was thankful to work with amazing people in MSDN. I’m thankful for this opportunity to work with great people again in IE. I’m excited about this move and the work ahead – something I desperately need to feel again about the work that I do. I’m excited about not only making IE better, but making the web better along the way for all of us.

My Request to Each of You

I’m going to be looking for feedback on IE. I don’t mind brutally honest feedback. If you’ve got it, I can take it. I really don’t know what work will look like in IE entirely or exactly what I’ll be able to do with most of the feedback – particularly so soon after joining the team. However, I’ll always be looking for feedback and will do my best to feed that back into the appropriate people inside the team.

Thanks for indulging me as I reminisced.

[Update] Just for clarity sake, I’m not sure how transparent I can be about what I’m doing or what the team is doing. You’ll never hear a new announcement about features or release dates or anything else coming from this blog. That’s really up to the team to communicate on official blogs. Just want to set expectations correctly.

Building a Better Web Application: Part 3

Note:  This is part 3 of a series of posts on building better web applications. If you haven’t done so already, please read parts 1 and 2:

 

Where We Left Off:

In part one of this blog post series, I explained a four-step process to shape the direction of your solution:

  • Define the problem you solving
  • Define the purpose your application will play in solving the problem
  • Identify the audience will you be a hero to
  • Identify the functionality your application will have

In part two, I explained how to gain a significant amount of advantage for your applications by working on a loose methodology involving:

  • Innovation – brainstorming on every idea we could think of to solve for each piece of functionality.
  • Emulation – analyzing competitors and innovative web sites to determine how they solved for various pieces of functionality.
  • Integration – Combine your innovation and emulation findings to create a set of workflows that will be needed to support your functionality.
  • Authentication – Verify the validity of your workflow by documenting your proposed workflows

In this part, I’m going to help you define your success. No seriously. In doing this, you will help refine your problem statement, and make smart investment decisions for your startup as it matures. Failure to take the time to work on step 8 WILL result in a failed startup – either because your customers aren’t successful and you won’t know how to correct it, or because your investors who don’t understand the problem as well as you will define a metric that doesn’t make sense for your business model, and then will judge you by it! Stick with me here. I know I’m long-winded.

Step 8: Define Success

 

“What gets measured gets managed” — Peter Drucker

It’s true that you can do the work I’ve presented to you in the previous posts, set yourself up beautifully for a project, and then fail miserably. Time and time again, we’ve seen new companies or products make a big splash with marketing and an influx of curiosity from customers, only to quickly fade away. Any of you still using Friendster?

What is it that causes a company with, what seems like a great idea to fall apart so quickly? In my view, it’s a failure to define for yourself what success means, and then do everything you can to become successful by your own standards.

Now, many of you may be ready to throw the towel in on this post saying “Oh no, here comes the mission statement and the meaningless metrics.” We ALL have reason to hate metrics. Time after time again, we’ve been given a not-so-clear goal of improving a random metric because someone decided that it would determine whether a project was successful or not. To your dismay, the metric didn’t improve, so the stakeholder of your project decided to dump money into a campaign that would help improve that metric. Your metric improved, but your project failed or was dumped. No wonder developers hate measuring things and loath the topic any time it comes up. Can’t we just follow the same mysterious voice Kevin Costner did in “Field of Dreams” – ‘if you build it, they will come’, right? Well, sure. Honestly, if you’ve done everything right to this point, you’re going to attract attention and people WILL come. However, even if you do solve a problem for someone, your ability to track how well you are doing at solving that problem and then learn from each success or failure your customers have in using your product to solve that problem is what will keep your project on track.

Let me ask it to you this way:

Would you give up time with your friends and loved ones and invest your life savings into a project that was just going to attract people who looked at your project then walked away? There are very rare circumstances where this is a good idea and a startup usually isn’t in that position to meander that long. I think many of you would agree that would just be foolish. Yet, what is it that many of you want to track the minute your web application goes live? Say it with me: page views!

“Page Views" and Other Roads to Ruin

I’m sure that somewhere in the silicon valley is a homeless shelter filled with failed entrepreneurs who will shiver any time you mention the phrase “page view”. The page view metric is today’s equivalent of window shoppers. Window shoppers don’t make your business successful. Hell, people walking in your store don’t make your brick-and-mortar a success by almost any standard. you won’t stay in business long if you just try to improve the traffic in your store. Yet marketing organizations in big companies are judged successful if they can just drive up page views on your site.

If you’ve set up your project using the method I’ve described in the previous post, and you decided that you’re going to stop at nothing to drive up page views, you might as well just start handing me your money. No seriously, there’s a donate link right … here. By the time you are done reading this post, you’ll understand that page views, and other related metrics, are really just symptoms of a well or poor performing application or web project.

Refining your “problem” by defining the metric

So if not page views, what should you measure? To answer that, let me step away from software and give you another example and then we will come back to your application – weight loss.

According to BusinessWeek, Americans spent $40 billion a year on weight-loss programs in 2008. Today, AOL Small Businesses claims that number is now $60 billion spent on products, and services of various types. It’s all a gimmick. Jenny Craig plasters images of dieters like Carrie Fisher who lost 30 pounds in 18 weeks. Weight Watchers, a company with the absolutely wrong metric in their company’s name, boasts that it is also rated the #1 best plan for weight loss. These companies are set up to help customers solve a problem – their weight. Seems noble. It makes a good business industry, obviously. Yet it also leaves many people stuck in yo-yo diets because they set their expectation and focus on the wrong measure of success – weight. So what’s wrong with measuring weight? That’s the goal for most of us, right? Not really. I challenged myself to 21 days of exercise and proper diet just to kick start my own personal fitness plan (blog post forthcoming). The first couple of days, my weight just shed off. After that, I had some days where I weighed more than I did the days prior. It would have been frustrating if my metric for success was weight. Those on those diets from Jenny Craig and Weight Watchers would have said that the program just wasn’t working. However, I also had the sense to track several other metrics such as:

Sure, I do track weight. But weight is a symptom of solving the problem. The real problem is that I was doing all of the things above wrong. I wasn’t moving enough in the day. I wasn’t eating right and wasn’t getting exercise. An amazing thing started happening the day that I started tracking those metrics instead of just measuring my weight – I felt successful with my every act, not just when I stepped on a scale to see if my weight dropped. By valuing the right metrics, I get instant gratification when I make the right food choice, go for a short walk just to keep my steps up for the day or burn calories on a bike ride. The thing that Weight Watchers and Jenny Craig are missing here is that they are solving a problem for customers – they are just solving the wrong one. Their metric only let’s their customers see success once a week when they step on a scale – and often too slowly to keep the customer interested. My metric allows me to see success early and often. You can see that by defining what success means, I actually have a way to refine and focus my problem statement, improve my business model, and make customers more successful on a regular basis.

So What’s Being Measured ?

So now let’s back to the world of web applications. Page Views are not a metric to concentrate on improving. Increased PVs are just a symptom of a web application that is solving a problem correctly for customers.

Many businesses incorrectly assume that they can game the system by spending their money to gain more views. This is a trick to try and get investors fooled into putting their money in your company. They can see your upward tick of page views and determine that you’re app must be doing well. You’d never catch Dave McClure or Garry Tan making that mistake.  A good investor can tell if a business is wisely spending their money by asking what metrics they are tracking. If page views are it, then they know the business is just gaming the system. The business might even try to throw in a “conversion rate” or other traffic-centric metric. Those too are easy to spot as useless because, as stated above, what get’s measured, gets managed. You can tell a lot about how a company is managed based on what they are measuring. If the metrics are centered around customer success, that company is destined for good things. If the metrics are centered around a one-sided win for the company, you can be sure that company is hoping for an early windfall and an exit strategy. If nothing is being measured, that company is in deep need of help!

If you want to be successful, you MUST DEFINE SUCCESS — for the customer, for yourself, and for the potential investor. You have to dig deeper into your problem and determine what metrics will give you more insight into determining if your customers are successful at solving their problem with your solution.

Let’s pull our example from the previous two blog posts into play here. My fake application is a content curation system. The problem , if you recall, is as follows: “The content that is most relevant to a developer’s evolving needs is buried in a pile of useless material and is never discoverable when they need it.” My purpose is: “to curate the best developer content targeted at an individual’s needs and to deliver it to the customer how they want and when they want it.” My audience consists of professional and experienced developers who build standards-compliant web applications and are passionate about software development. The functionality that I determined was important was: low-barrier community contribution, rewarding community interaction, compulsive browsability, personalizable content, and diverse delivery.

To me, the metrics for this application seem pretty clear, but this is an acquired skill that didn’t come easily for me as a developer. What would you choose to measure to determine success? Go ahead and jot some ideas down. I’ll wait.

Seriously go ahead.

OK, so what’s being measured? If what you jotted down had anything to do with how successfully the customers used the functionality that you determined was important to successfully solve the problem, then you can give yourself a pat on the back. All of your metrics should indicate if customers are successful with your functionality. The best place to start would be to go back to your Evernote (or other planning system of your choosing) and look at your functionality notes. Remember that we speced out “Low-barrier community contribution” and we determined one workflow is:

  • User posts a link to twitter using a cell phone and uses hash tag #CR8ME (curate me).
  • User receives a response tweet instructing them to connect their twitter to the app
  • User clicks on link, authorizes app, and new post is added to curation queue

Your metrics should then include things like

  • “Number of customers using #cr8me” tag to initiate sign-up process
    • % who are sent an authorize link but do not click
    • % who click on authorize link
    • % who click on authorize link by do not authorize
  • “Number of customers using #cr8me” tag to add new items to queue

Why would you do this for every feature? Because, if you truly believe that “low-barrier community contribution” is vital to the success of your project, and a month after launch you determine that people aren’t using your solution to that problem, you can manage that however you feel best. You could either supplement or substitute your functionality with additional functionality. You could determine that people aren’t aware of the feature and market it better. The point is, you how can manage the things that are relevant to the solution you have defined.

 

What Doesn’t Get Measured

What doesn’t get measured is a question very frequently not asked. More data is always good, right? Wrong. There are times in life when it’s better to not allow data to be collected. Collecting data that means nothing allows someone to later look at data that has no relevance to your project and start rat-holing on it. For instance, many web analytics packages allow you to track things like “cost per end action”. This is great if you are actually running a marketing campaign, have defined end-actions, and want to determine the effectiveness of your campaign. If you haven’t done those things, but are just collecting and reporting that metric because you can, an investor, stakeholder, or customer can look at it and say “well, they can’t be that good because their CPA is really bad”. Additionally, going back to the adage, “what gets measured, gets managed”, you’ll find yourself trying to improve a metric that has no meaning to you. It will waste your time and resources. If you don’t have a need to track it, and you do anyway, you or your investors will say “well, maybe I should be running a marketing campaign so we can track this better.” Then you’ll spend countless hours working on a plan and process and marketing campaign so you can improve your numbers. All the while, those resources could have been better spent tracking and improving metrics that are relevant to the problem you are trying to solve and the functionality you determined solves that problem.

If it ain’t planned, don’t track it!

Document your Metrics

Document all of the metrics and reasoning behind metrics in your project document system (again, I’m using Evernote, but you might just have hand-written paper and a file folder or two). For me, in my Evernote, I added the metrics under each workflow, such as in the following screenshot:

image

 

 

Summary

Measuring success of a web application is not a cookie-cutter process. If you haven’t defined success for your application before you started coding, you’ll likely find yourself trying to get meaningful data when it’s too late – like when your business is failing or an investor comes calling and wants more information. With any initiative in life, it’s always good to know what the end game is to keep you focused on your vision and to help refine your problem statement.

  • track workflow metrics – Your workflows are tried to functionality which in turn is tied to your problem statement. So it makes sense to start by looking at the workflows you created for each piece of functionality and determining what data is needed to track your success or failure.
  • set your limits – Determine in advance what values or trends you’ll accept as “success” and what values mean you need to regroup.
  • avoid mindless collection – Don’t track what can’t be tied back to a problem-solving functional workflow, or to a process or marketing program that supports your current business plan. Collecting “just in case” will waste resources, time and may make you lose out in the future.
  • document your metrics – Keep track of what metrics you intend to collect and document them with the rest of your project. Once again, showing an investor or other stakeholder what you are doing to solve the problem, and how you are tracking success with your approach gives you credibility and your stakeholder gains confidence.

I’m “counting” on your feedback (ba-dum dum). I intend to revise this post and keep it updated based on community feedback so please give some! Thanks for reading.

I would appreciate if you would tweet, “like” or otherwise recommend this post if you feel it has been helpful. I’m not tracking these as metrics for success, but it’s always nice to know when someone finds your work helpful. Thanks again for reading!

Building a Better Web Application: Part 2

Note:  This is part 2 of a series of posts on building better web applications. If you haven’t done so already, please read part 1:


Where we Left Off:

In part one of this blog post series, I explained how to frame the basic questions around your web application:

  • What problem are you solving?
  • What purpose does your application play in solving the problem?
  • What audience will you be a hero to?
  • What functionality will your application have?
    By this point, I’ve compiled all of my current findings into a document and will call it the “Draft Project Definition.” You can use whatever system you like: A hand-written notebook, napkin, or something more sophisticated like Microsoft OneNote. I love to use Evernote because I can easily access my notes from the majority of devices that have Evernote applications available, or directly through the web interface. Once again, just use whatever you care to.

Using Evernote to document your progress

The point is that you now have something to show someone should they ever ask you what you’re working on. It’s a pretty quick projection of your plans and should only take you an hour or two to put together. You could spend more if you are building this with someone else and/or are iterating with a client.

Can We Code Yet?

This document is something that we developers are usually opposed to. Documenting our process is about as painful as anything we can imagine. We just want to hurry up and get to the coding part. I know it is unintuitive for some of you, but the coding part should be much more enjoyable once you’ve got all this behind you. It means you’ll have fewer interruptions explaining what you’re working on once you do start coding.

Ready for The Next Step?

The feedback I received on the first post was overwhelmingly positive. I’ve got several contacts from individuals saying they are ready to walk through this process. My question to them has been “What will you do once you’ve defined your problem, purpose, audience and functionality?” Most of these individuals have said they will continue to define the finer feature set. One person said they’d just wait for my next blog post. For that person, and the others who have asked me to post part 2 quickly, I’ve decided to expedite this post. I’ll pick up where I left off and introduce “Step Five – fine-tuning your functionality through innovation and emulation.”

Step 5: Innovation and Emulation

In the previous post we defined the following five broad strokes of functionality:

  • Low-Barrier Community Contribution
  • Rewarding Community Interaction
  • Compulsive Browsability
  • Personalization
  • Diverse Delivery
    We need to get some clarity on our functionality, but we still aren’t ready to define implementation yet. We are going to do two things, however, that will make all the difference in the world: brainstorm on proposed functionality (innovate), investigate your competition or other applications that have (emulate).

I opt to do the brainstorming (innovation) part first. After all, if you are just going to copy someone’s work, you aren’t solving a problem! If you are copying someone else’s work, you aren’t building something that fits your defined problem and defined purpose. If your problem and proposed solution match up enough to someone’s site you haven’t been paying attention and you’re doing it wrong! If you’re doing it wrong, please go back to phase one and start again. This time, remember that step two is about solving a problem in a unique way!

You should opt to do your brainstorming first so that you aren’t “encumbered” into coming up with the same sandboxed solutions that others have invented. Many a website is simply a shuffling of the deck chairs and they don’t last long. These “carbon copy” sites don’t distinguish themselves from their competition.

I’ll use the example of Facebook and MySpace again. Remember that MySpace and Facebook had similar functionality. Someone might try to argue that Facebook copied MySpace’s problem and purpose. That’s not true. Arguably, MySpace was solving a problem of allowing hosted, heavily-customizable home pages. Facebook was solving the problem of building an easy-to-contribute, easy-to-extend platform that connected you with your friends and topics of interest. They had a similar problem, but had VERY different approaches to solving the problem. Don’t copy, shift your thinking on the solution to the problem and go from there.

That said, both innovation and emulation play distinct and important roles in developing your idea.

Solution Brainstorming (Innovate)

For each of your functional areas, think of everything from the most rube goldbergian implementations of your functionality that your imagination can muster to the most simplistic. Write it all down or whiteboard it out with others to get an optimal effect. Having good people to bounce ideas off of is key. I will argue that you should not make your brainstorming group too large. One or two people is actually plenty to get the intellectual banter working (see my previous post on Actualization and Collaboration).

Don’t constrain yourself in this step, to the most practical and probable ideas. Even if an idea comes to mind and seems completely useless, impractical, unwise or otherwise undoable, write it down anyway. You aren’t going to include all of your ideas in the final solution. You’re just capturing the ideas for now. Write down some practical solutions too. Write down as many ideas as you can and as quickly as you can do it. Just let the information flow straight to your pen. Don’t think about what everyone else has done or will do. Just free-flow information no matter how ridiculous it may seem. The point with writing down the improbable or impractical solutions is to get you thinking outside of the solutions that everyone else has thought of. If you constrain your thoughts to what you know you can do, you’ll come up with some of the same tired solutions.

In my example of a content curation system, I’m going to write the following:

Low-barrier community contribution

  • Hand-written notes added to system
  • Devices given to people which will automatically contribute content
  • Toolbars / Browser plugins that allow for submission and voting
  • WordPress plugins that allow automatic contribution
  • Forum plugins that allow submission and voting
  • Stream content from existing community sites
  • Anonymous posting w/ability to link to a real profile later
  • Poll for content from the community

Rewarding community interaction

  • Gifts delivered to the home of users for extreme participation
  • Sponsored software or services for participation in product-related discussions
  • Points / Reputation system
  • Most active content reviewers rewarded with weekly column
  • Ad revenue sharing based on quality of content submitted
  • Get paid for milestones reached (combination of quality and quantity gates)

Compulsive Browsability

  • Additional content presented based on content you’ve viewed or contributed in the past
  • System predicts what to present to you based on linked profiles/interests
  • System predicts what to present to you based on the content that your friends you interact with most are looking at

Personalization

  • Allow filtering based on tags
  • Allow filtering based on words found in text
  • Allow filtering based on what my friends have read
  • Allow filtering based on what my friends have recommended to read
  • Designate a delegate to curate my content for me
  • Follow company recommended reading
  • Follow product group recommended reading

Diverse delivery of content

  • Carrier pigeon delivery
  • Existing Standardized Aggregation
  • Hand delivered each morning (like a newspaper subscription)
  • Monthly personalized magazines – Physical and Digital
  • Deliver headlines, summaries and “key quotes” from articles of importance
  • Delivered to phone through notification – one article at a time
  • Summary of all curated content delivered to phone through notification
  • Filtered content delivered to phone, unfiltered delivered in summary (split importance)
    When you’ve completed this process, you’ll likely notice that you are weak on a few functional areas and have a LOT of ideas for others. This is OK for now, but if you finish your next step and are still light on ideas, you’ll want to start iterating specifically on the light areas until you have a large list of items to chose from.
    Once again, I write down my progress into a system so I can keep track of it. I won’t go into too many details of my personal process because it doesn’t matter how you do it so much as that you do it. I create a new note in Evernote under my project for each bit of functionality and list the findings of my brainstorming there. Once I’m done with the due-diligence part, I’ll add the findings from that process to each note. I’ll show you a screen shot of my system at the end as I don’t want to get too caught up in those details just yet.

Competitive Due Diligence (Emulate)

For each of your function areas, identify who might provide this functionality already. It doesn’t have to be someone who is working on the same area. It can just be a tiny piece of functionality from an unrelated site that you want to emulate. For instance, the website “Ravelry” is a well-known knit and crochet site. Surely I don’t want to solve the same problem as them. However, they have solved some problems along community interaction that would be fantastic to model after.

Don’t limit your learning to just the good site examples. Make sure to document the poorly executed solutions to your problem too – you don’t want to repeat other’s mistakes! For me, I take a screen shot of the functionality from various sites and then list pros and cons for that implementation. For instance, look at my first piece of functionality, “Low-barrier community contribution” and see how I broke out some functionality to emulate:

Low-barrier community contribution


Facebook Wall Posting

image

Pros:

  • After signing up contributing is very easy
  • Link posting fills in the data after simply posting a URL
  • Photo submission can be done through attached devices or through file upload
  • Upload from any device

Cons:

  • Requires sign-up before contribution
  • Privacy concerns raised through the years limit what some people will share

Facebook Like Button

image

Pros:

  • One-click contribution makes for very limited contribution AND sharing across social networks

Cons:

  • Privacy hard to get right

Stack Overflow Ask a Question

image

Pros:

  • No log-in/sign-up required. Simply a name/email will suffice
  • Optional log-in allows use of OpenID – registration therefor is low-barrier
  • No need to select the right “forum” to post your question. Multiple tags allow for your question to catch the attention of anyone watching those tags

Cons:

  • May feel intimidating to respond to questions
  • Community can be harsh at times – driving off would-be contributors

Posterous New Entry

(No Interface Required)

Pros:

  • Post by emailing
  • No sign-up required – just email your first entry to post@posterous.com
  • Easy submission to your personal <name>@posterous.com address allows easy update of your Posterous page.

Cons:

  • “Feels” like a high-barrier to submission if I don’t want to open my email

For each functional area, I would continue to do this for as many sites as I felt necessary to start seeing patterns in the pros and cons. I’d probably list several others. Off the top of my head, I’d list Wikipedia, Reddit, Twitter and Hacker News (which has a similar “purpose” as my fictitious business, so it’s good this is a hypothetical business for me).

If you can’t immediately think of a pro or a con of a specific item, try vetting it with others. Do mini-usability studies by asking someone else that you know to use the product or service and give their feedback while they are using it. This should give you some off-the-cuff responses that will help you really dig down into what is good and bad with a site’s implementation.

Remember, you are scouring these sites for what is good and bad about that SPECIFIC FUNCTIONALITY  that you are currently investigating. You don’t want to get side-tracked into saying “Oh, they also have <x> functionality and I really think I should do that too!” This is where scope creep happens a lot. Many of us just want matching features because they feel that’s the only way to compete. If you look at solving your problem the way everyone else does, you’ll never be a competitive player. Additionally, all the work you did in steps one through four to keep your scope manageable so you can love your customers will be for not. Stay disciplined here and focus only on the functionality of a site that matches your functional areas. For instance, I LOVE that Posterous allows me to easily theme my site. However, that isn’t in scope for THIS functional area so I’m not going to list it as a “pro” under “Low-barrier community interaction”. The theming feature will, however, fit into my “personalization” functional area. So don’t include everything about the site that you like. Include only the pieces from the site that refer to the given functional area.

I’m going to state again that you aren’t just looking at existing applications who are solving your same problem. You’re looking at anything that has the functional area you are looking to provide. Posterous isn’t solving the same problem that my “curation” problem is. Facebook isn’t anywhere near my same problem area. However, both of those sites offer insight in the functionality that people have come to expect. If I start copying their features exactly, I’m going to be solving their problem, not mine. My application will fail miserably!

Step 6: Putting the Chocolate in The Peanut Butter (Integrate)

During the 80’s, Hershey’s produced a commercial for Reece’s Peanut Butter Cups. In the commercial, a man is eating a chocolate bar and a woman is, for reasons we may never know, eating peanut butter directly from the jar. They are minding their own business, walking their own separate paths when they come to a cross-roads, bump into each other and exclaim “Hey, you dipped your chocolate in my peanut butter. Hey you got peanut butter on my chocolate.” For some reason the two strangers feel they both have good enough hygiene to eat after one another. The result is that they both liked their new peanut-butter/chocolate concoction.

We haven’t talked you into producing a mission statement, THANK GOD! We haven’t produced a functional specification for our software. We have, however defined our problem and approach to a solution for a specific audience. We’ve defined feature areas and now we have brainstormed some of our own ideas and have looked at the way that others have solved the same problem. It’s time to put the two sides together to form the best possible solution you can.

As mentioned before, for each functional area, I have a “function sheet” in Evernote under my project folder. Here’s a screen shot:

image

You can use whatever software, or organizational technique you wish (hand-written notes, folders and envelopes for all I care). This works for me. Find what works for you. Admittedly, this is where I miss some of the organizational capabilities of OneNote so you may want to consider using that.

Notice that I keep my brainstorming and competitive analysis of a functional area in one sheet. Now it’s time to go through each one and hold another brainstorming meeting. Yes, I know. You just want to code. You don’t want to sit in these meetings about these functional areas. I agree that mindless meetings are useless. However, if you keep your meetings small (you and two or three other people – maximum), and the scope of the meeting limited to one piece of functionality, you can usually knock out some clarification work in an hour.

Look through the brainstorming section and try to imagine what unique and exciting way you can solve this part of the problem. In this phase, you are taking the ideas you have captured and are trying to create a workflow or multiple workflows for the feature. Don’t be afraid to incorporate some of your crazy ideas if it can make sense.

Here’s what I came up with in my own 2-minute brainstorming session on “Low-barrier community contribution”:

“Low-barrier Community Contribution” Workflow 1:

  1. User posts a link to twitter using a cell phone and uses hash tag #CR8ME (curate me).
  2. User receives a response tweet instructing them to connect their twitter account to the app
  3. User clicks on link, authorizes app, and new post is added to curation queue

In this workflow, I have slightly modified the crazy idea of handing out devices to people that will automatically contribute content to using a device the user already has.

Rabbit Trail: This was a lesson learned many years ago when I did some work for a company called ZapCode GoCode in the late 90’s. We were a competitor to the Cue:Cat. RadioShack gave away Cue:Cat reader devices (at a cost to them) which allowed you to scan a “Cue:Code” out of a magazine or other surface. This barcode would effectively link you directly to a site with more information. At the time, cell phones weren’t taking pictures. The scanner had to be connected to a computer. So you literally couldn’t use this anywhere but home where typing a URL might very well have been easier. These days, we can just use our cell phones to scan DataMatrix codes or Microsoft Tags and avoid the proprietary device all together.

I’ve also incorporated Posterous’ idea of “no sign-up required”. You could keep on going here and think up several more workflows in your hour-long meeting. The point here isn’t to solve the entire problem. You are simply thinking of how you can provide the functionality that is required in your overall approach to the problem.

Add your findings to the bottom of your “function sheet”.

Step 7: Experience-Driven Development (authenticate)

Now that you have an idea of what the workflow looks like for your function, you’re going to want to define the end-to-end experience of your functionality. This is your chance to be a bit more specific about the User eXperience!

But wait, shouldn’t we be defining the protocols and functions here? Shouldn’t we be writing code? Nope. This is something I like to call “Experience-Driven Development” (E.D.D. — not to be confused with E.D., because that’s a whole different problem altogether). Once you have an idea of the problem you want to solve and the basic functionality you want the user to have, you want to design the experience first and then let the technology support the decisions you make here.

You DO NOT want to pick a technology and then be constrained about the experience you can provide to your customer because of it.

Note: There are rare times that you will design your experience only to find out that no technology can solve your problem. These are usually due to decisions external to your control. For instance, I can design an experience that automatically authorizes your twitter account to connect to my application if I use the #CR8ME hash tag. However, twitter requires that I use OAuth and allow customers to manually allow that connection. In this hypothetical case my designed experience will not match the reality of the final solution.

Let’s design the experience. First, create a diagram of the workflow so you have a visual of the process and include some detail in the responsibilities.

image

You should also create some screenshots or create some mock-ups of UI that match your designed workflow. Use your due-diligence/competitive analysis to help you design UI that others are used to using. Remember that you aren’t trying to copy experience so much as reduce the barrier to entry by using familiar concepts for interface. Depending on your problem and intended solution, you may not have a UI at all. Everything you do may be driven by back-end services connected to someone elses UI (such as our twitter example). Notice that in this example, the user never had to actually interact with our web interface – only twitter’s OAuth screens?

You’re going to paste this into another new sheet in your notebook. I call mine “Experience Definition”  and use the format: “Experience: <function name>” as the subject. THis helps me connect my experience to the intended functionality.

Just for posterity sake, you’ll want to describe this experience step by step in as much detail as possible. You may want help developing the functionality and having the details will help anyone who picks up after you. To do this, I’m going to add a number next to each part of my diagram above.

image

Notice the numbers 1 through 11 on the individual boxes of the diagram? I’ll then create a document with an outline that matches these numbers. In each number section, I’ll describe, in full, what that part of the workflow is doing. Do this for each piece of functionality. You’ll likely have several diagrams and several documents for EACH piece of basic functionality we defined. It’s a little time-consuming but you’ll be happy you did it later.

In case you haven’t figured this out, the document and this diagram come together to form a pretty close facsimile of a functional specification – and we haven’t had to work very hard to get here.

What does all of this documentation gain you, as a developer?

Early Problem Detection:

Documenting the process here helps you better determine the problems you’ll run into while developing a product. It’s amazing how many problems in logical flow I’ve found just by creating a diagram of the workflow. Just like spotting an off-kilter cornerstone of a building at the outset will prevent a large number of problems in the future, early detection of program logic can prevent a large number of hazards and risks when you start developing your software.

Credibility:

Showing your stakeholders, investors, or partners how you came to the conclusions you did shows that you didn’t just happen upon your feature set. It gives those people confidence that you are, indeed, running the show and have things under control. When an investor asks you what your plan is, and you immediately send him an email copy of your planning docs, he is going to have a lot more confidence in your ability to deliver consistency in the future. Don’t get me wrong, I’ve seen plenty of plans with documentation who weren’t worth the paper their project plans were printed on. An investor is certainly going to care about a lot more than your planning docs. But it’s going to be a helper if you haven’t built anything for an investor in the past.

Meaningful Buy-in:

We’ve documented the problem we are trying to solve, the approach we are taking to solve the problem, and audience we are trying to reach. We’ve defined our basic needed functionality. We then brainstormed on solutions to our functional problem areas and done some due diligence on what others are doing to solve here. We’ve defined some workflow and perhaps even mocked up some UI to show the experience we are building. At this point, we should submit these documents to your stakeholders and perhaps even some customers for review. Document the responses and get everyone to sign off. You don’t need to actually have them sign anything (unless they are a contracted client and you need the agreement on terms documented for legal purposes). You just need them to see your proposal before you start coding.

Purposeful Pushback:

Since you gained buy-in from your stakeholders and they saw exactly what problem you were solving, who you were solving it for and exactly how you planned to solve it, any changes the stakeholder brings (read: scope creep) can be countered with a discussion about how the change will impact the overall project. You can have a real discussion about how the change may change your approach and how that will affect your functionality. Even if you don’t have stakeholders, this is valuable. If this is just you and some buddies building something in your spare time, you can still use these documents to determine the impact of decisions you make during development.

A Basis for A Test Plan

If you plan to do test-driven development, how would you know how to write your tests if you didn’t know how an operation was suppose to occur? Having your workflow documented allows your testers to create functional tests that iterate on each workflow.

There are a lot more benefits to the documentation you just built, but these reasons are good enough for anyone building a solution to a problem for ANY reason to do the work up front.

Summary

In our previous post we described a four-step process to shape the direction of our solution:

  • Define the problem you solving
  • Define the purpose your application will play in solving the problem
  • Identify the audience will you be a hero to
  • Identify the functionality your application will have

In this post, we actually defined and planned our product using a loose methodology involving:

  • Innovation – brainstorming on every idea we could think of to solve for each piece of functionality.
  • Emulation – analyzing competitors and innovative web sites to determine how they solved for various pieces of functionality.
  • Integrate – Combine your innovation and emulation findings to create a set of workflows that will be needed to support your functionality.
  • Authentication – Verify the validity of your workflow by documenting your proposed workflows

I am not following a specific methodology here. You won’t pas a PMP certification by following my advice. However, hopefully these posts can help the struggling technical developer to improve his or her web application, stay focused during the development process, allow him or her to pitch it to investors or stakeholders at any point in the process, and show progress to anyone he cares to (or needs to) show it to.

Feedback is appreciated. I intend to revise this post and keep it updated based on community feedback. Thanks for reading.

I would appreciate if you would tweet, “like” or otherwise recommend this post if you feel it has been helpful. Thanks again!

Post Navigation