Looking Backwards

Recently, my director published a new "capacity sheet". It's an Excel sheet with a high-level summary of all the projects scheduled for the upcoming year, with breakdowns in hours. The hours themselves are based on some simple formulae. A salesperson assigns the project a t-shirt size (S, M, L, or XL) and this translates to a predefined number of hours.

The intention of the sheet is not to plan capacity of our systems (i.e. transactions per second, etc), but rather see if we have sufficient resources to do the work. Let me start off by saying, I like the idea...in principle. In fact, at my old company it seemed no thought was given to actually staffing for the work coming in, which meant all of us ended up working ridiculous hours. So, the idea is a good one.

The problem is the implementation. It’s fine to assign t-shirt sizes to projects that are 9-10 months out and haven't even been defined (you can't really do anything else), but we use these boiler-plate numbers even for the projects that are nearly started. Further, as you move further out in time, the exercise becomes more and more meaningless because invariably the client changes his wishes and prioritization of work. Now, I made these points last time my group went through the exercise and that's not really the point of this blog.

The point of this blog is that no one is that I've not heard one person mention the need to look at the exercise we went through last year. Having gone through the exercise before and having voiced a number of concerns about the value of the exercise, I decided to go back and look at the sheet from last year. What I found there was very telling and hopefully will persuade others to reevaluate the way in which we're handling things this year.

Almost 70% of the projects on our plan never even happened. And for those that we did do, the boiler plate estimates we used were not even close to the actuals. Now, this didn't specifically cause us problems insofar as we delivered on our commitments. That is primarily because in general the formulae grossly overestimate the amount of resources needed for any given project. Of course, I'm not sure the actuals in our time-tracking system will prove this, because as we know from Parkinson's Law, work expands to fill time.

The problem is going through the exercise this time around using these numbers uncritically, tells us that we're short staffed. Nothing could be further from the truth. I have many more developers than I need and many of them end up billing huge amounts of time to "sustainment" activities or training. Since these guys are offshore, I really couldn't say definitely what the hell they're doing half the time. I'd say we're probably 25-40% overstaffed.

But again, that's somewhat incidental to my point (I do have one!) for the blog and that is when doing something. George Santayana made the famous statement that ‘Those who do not remember the past are doomed to repeat it’. In general, I prefer Kurt Vonnegut’s rebuttal ‘I’ve got news for Mr. Santayana, we’re doomed to repeat the past no matter what. That’s what it is to be human’.

In the case of software engineering (and most sciences), you absolutely *have* to go back and look at what went right and what went wrong to improve. That’s what of the things I think Scrum and Agile absolutely nailed, the idea of iteration/sprint retrospectives. Unfortunately, things like that seem to be the exception rather than the rule. More often than not people don’t do this, so we continue to tread a sea of mediocrity in which nothing ever really improves.

TFS Migration Or: There's Still No Silver Bullet

Recently my company started a migration project to move all of our internals tools over to Microsoft Team Foundation Server (TFS). If you're not familiar with TFS, Microsoft bills it as an Application Lifecycle Management (ALM) suite which handles everything from source code control, to bug tracking, to project planning.

Currently, tools and processes vary across teams. My company has been the result of mergers, of growth and shrinkage, and the toolset reflects this. Many tools are shared across teams, but there are differences. For example some teams use HP's QualityCenter for bug tracking, while others use Bugzilla, while my group uses an in-house bug tracking system written years ago and no longer actively maintained.

The promise of TFS is to standardize our processes and tools across all teams and to make life simpler by having a single toolset that can track any given item of work from inception through completion and after.

TFS has been the pet project of one of my fellow development manager's within the organization. In fact, he was my boss when I first joined the company. He's the one that has pushed hard for TFS, including bringing a consulting agency onsite to evaluate our tools and see how TFS might help us (funded by Microsoft it should be noted), putting together the proposal for the company’s operating committee, and serving as evangelist for the tools.

He’s also made repeated (and somewhat forced) attempts to show how TFS will revolutionize our lives.

Don’t get me wrong, some aspects of TFS seem intriguing. It has some very powerful features. One of the most interesting is its ability to recreate the environment within which a bug occurred, so that, say a tester encounters a bug while testing, he can save the state of the servers on which it occurred, and the developer can at some point later, actually spin that exact environment back up and step through the code as if it was executing at that moment. That’s pretty cool.

As a source control system, TFS seems to be fairly decent. I think it misses the boat in its reliance on a centralized repository (versus the more recent developments in distributed revision control systems), but its notion of discrete changesets, and its management of branches seems to be significantly better than CVS.

But at its core, it’s a Microsoft product. And that has some very serious implications. Joel Spolsky’s keynote address at the recent Stack Overflow conference was about the dynamic between keeping things simple and giving people the power and flexibility they need to do what they want. He came down to the conclusion that the problem isn’t necessarily in giving people too many choices, it’s in giving them choices that are meaningful to them.

To me, the TFS interface is typical Microsoft. Incredibly powerful and flexible, but for you base cases, the stuff that you want to be able to do intuitively and easily, it makes something that should be a click or two into a 10-step process, that is needlessly complex.

One example is creating a new task in TFS. So, you have a story in the system and want to break it down into tasks. You’d think that’d be simple. So, you click ‘Create Link’, then you are taken to another menu, where you select the ‘Link type’, of which ‘Task’ is one choice. Then you make a selection of ‘affected by’, ‘affects’, etc.

I don’t know about you, but to me this should be a one click operation. All the abstraction and flexibility it just annoying for 95% of the time I’m using it.

But I suspect TFS will be an improvement on what we have currently. The thing is, I don’t think it will make that much of a difference. A good toolset (and one can argue over whether TFS is a “good” toolset) is all well and good. But the real problems in software engineering are not the tools, but how they're used. And without a significant investment into making our processes simpler, having people who apply those processes with critical thought and not blindly following some set number of steps, is what makes things more efficient.

And even if TFS is the greatest toolset is the world, go back to Fred Brooks' seminal ‘No Silver Bullet’ article. Such systems only address the accidental complexity of software engineering, not the inherent complexity. As such, they cannot result in order of magnitude increases in productivity.

To me, the money which has been spent on TFS (which is not insignificant) would have been far better spent in bringing in experts to look at our processes, our team organization, at reducing the bureaucratic processes that waste so much money and time, not some shiny new toy.

Business Viability Or: Should I Open Source ZPlanner?

I've now been working on my little project tracking application, ZPlanner, for nearly a year now. It's certainly not been a full time effort, rather I work on it here and there when time allows and when my attention doesn't drift elsewhere (to blogging, the book I've been working on, and other random stuff), but I've made reasonable progress on it. It's actually getting reasonably close to being a *uable*. Mind you it's nothing revolutionary. It's just a simple web application, but nonetheless I'm fairly happy with where things are at.

I started working on ZPlanner because I thought I was going to be laid off from my job as a development manager around last March. Due to a reorg, my job was promised to someone they'd relocated from the East coast offices which left me without a position. The job market at that time was so horrible, I felt like I'd be lucky to find a position as a developer somewhere, so I figured I should brush up a bit on my programming skills, seeing as how I do so little for my job these days. Luckily, however, I was able to snag another position within the company--it was a better one to boot. But I kept working on ZPlanner anyway as I think there's a need for a simple, project tracking application.

I've toyed with the idea of actually trying to sell it--at an affordable price--as the only options for project tracking are either free (and kind of crappy) or fairly expensive. But to be honest, I'm not sure how viable it is as a product. There are a lot of entries in the market, even more if you consider that many software stacks that label themselves as "bug trackers", such as Jira, have also incorporated project planning features.

But the idea of starting up my own company, even if it barely made any money is incredibly alluring. Like many developers, the idea of being in charge of things and running a company how *I* think it should be run is an exciting prospect. And in lieu of any better product, I figured, why not try selling ZPlanner.

But the thing is, over the winter break, I had what I think is a really good idea for a piece of software. I won't go much into the details, but from what I can tell from some brief Googling, it's a niche that has not yet been solved, has the potential to be sold to larger companies and deliver measurable increases in profit and efficiency, and is a problem which would not be trivial to solve. In fact, the reason I thought of the idea was because the lack of it caused me considerable frustration. I was dealing with a company I would have assumed to have software as I describe, but they did not and as a result I ended up being one very pissed off customer. I've also found some theoretical whitepapers on the subject, but very little in the way of concrete implementations. I think the potential is pretty large.

Which brings me back to ZPlanner? The product I'm describing above will take quite some effort to implement and I'm anxious to get started. But I really want to polish off ZPlanner. I've decided I will. But then the question becomes do I try to sell it still? Or should I just open source the thing? If there really is no market for "yet another project tracking tool", maybe it's better to open source it. ZPlanner had it's genesis due to the failings of XPlanner, another open source project tracking tool. Despite it's failings, however, XPlanner has seen wide usage and even though it seems to be fairly moribund, with the last release almost 3 years ago there are still a ton of people using it...because it's free.

Even if it didn't profit me directly, the thought that Zplanner might replace XPlanner as the open-source, free tool for choice for tracking is pretty cool and might indirectly profit me in other areas. It's hard, given that I've sunk somewhere between 300-400 hours into it at this point, to just give it away. Though I certainly make use of a lot of open source stuff and maybe it's time to give back.

I don't know. I guess we'll see what I decide.