Monday, January 4, 2010

Business Viability Or: Should I Open Source ZPlanner?


I've now been working on my little project tracking application, ZPlanner, for nearly a year now. It's certainly not been a full time effort, rather I work on it here and there when time allows and when my attention doesn't drift elsewhere (to blogging, the book I've been working on, and other random stuff), but I've made reasonable progress on it. It's actually getting reasonably close to being a *uable*. Mind you it's nothing revolutionary. It's just a simple web application, but nonetheless I'm fairly happy with where things are at.


I started working on ZPlanner because I thought I was going to be laid off from my job as a development manager around last March. Due to a reorg, my job was promised to someone they'd relocated from the East coast offices which left me without a position. The job market at that time was so horrible, I felt like I'd be lucky to find a position as a developer somewhere, so I figured I should brush up a bit on my programming skills, seeing as how I do so little for my job these days. Luckily, however, I was able to snag another position within the company--it was a better one to boot. But I kept working on ZPlanner anyway as I think there's a need for a simple, project tracking application.

I've toyed with the idea of actually trying to sell it--at an affordable price--as the only options for project tracking are either free (and kind of crappy) or fairly expensive. But to be honest, I'm not sure how viable it is as a product. There are a lot of entries in the market, even more if you consider that many software stacks that label themselves as "bug trackers", such as Jira, have also incorporated project planning features.

But the idea of starting up my own company, even if it barely made any money is incredibly alluring. Like many developers, the idea of being in charge of things and running a company how *I* think it should be run is an exciting prospect. And in lieu of any better product, I figured, why not try selling ZPlanner.

But the thing is, over the winter break, I had what I think is a really good idea for a piece of software. I won't go much into the details, but from what I can tell from some brief Googling, it's a niche that has not yet been solved, has the potential to be sold to larger companies and deliver measurable increases in profit and efficiency, and is a problem which would not be trivial to solve. In fact, the reason I thought of the idea was because the lack of it caused me considerable frustration. I was dealing with a company I would have assumed to have software as I describe, but they did not and as a result I ended up being one very pissed off customer. I've also found some theoretical whitepapers on the subject, but very little in the way of concrete implementations. I think the potential is pretty large.

Which brings me back to ZPlanner? The product I'm describing above will take quite some effort to implement and I'm anxious to get started. But I really want to polish off ZPlanner. I've decided I will. But then the question becomes do I try to sell it still? Or should I just open source the thing? If there really is no market for "yet another project tracking tool", maybe it's better to open source it. ZPlanner had it's genesis due to the failings of XPlanner, another open source project tracking tool. Despite it's failings, however, XPlanner has seen wide usage and even though it seems to be fairly moribund, with the last release almost 3 years ago there are still a ton of people using it...because it's free.

Even if it didn't profit me directly, the thought that Zplanner might replace XPlanner as the open-source, free tool for choice for tracking is pretty cool and might indirectly profit me in other areas. It's hard, given that I've sunk somewhere between 300-400 hours into it at this point, to just give it away. Though I certainly make use of a lot of open source stuff and maybe it's time to give back.

I don't know. I guess we'll see what I decide.

Thursday, December 10, 2009

ZPlanner Technology Brief: Maven2

A few weeks ago, I devoted a blog to talking about Mercurial, the distributed revision control system I decided to use for ZPlanner, my Agile project tracking tool. This week, I figured I'd briefly talk about one of the other choices I made, namely to use Maven2.

For most of the career as a hand-on developer, I was trapped in the world of Perl and Apache. Our proprietary system didn't require doing builds. You just created a new file, put it in an appropriate directory, and voila, it worked. All of the libraries we used were global to our servers and installation of new packages (from CPAN) was a rare thing. When they did occur, it usually just mean shooting a request to our of our admins and asking him to install it. So, for all intents and purposes we never really thought much about dependency management or builds or any of that stuff.

A bit later, I was assigned as technical lead to a new, large scale (for our company) project and we decided, for various reasons, to use Java. Immediately, we had to start thinking about how we'd do our builds. Given that I was the first person to start coding and was relatively inexperienced with Java at that time, I didn't think about it very much. I started coding in Eclipse, created a directory structure that I thought was reasonable and did builds solely via Eclipse. Dependencies were managed by plopping a jar in the lib directory, then clicking 'Add to classpath'. I didn't think about it much after that. It seemed to work.

When we brought other more experienced Java developers on board, they immediately starting talking about Ant. My boss (whom I still regard as the most brilliant programmer I've ever worked with), steadfastly maintained that dependencies and builds should be managed via Eclipse. That to use Ant (or Maven) was a duplication of what we'd be doing in Eclipse anyway. Why do we have the need to do builds via command line? We should be able to build our artifacts directly in Eclipse which would produce an artifact (in the case of the service I was writing, a jar), deploy it any which way we wanted, and create simple scripts to set the classpath when we ran it. Why muck around with byzantine Ant files, when Eclipse managed all of this for us? And I really do think he had a good point.

Eventually, though we did end up with an Ant file. I can’t recall if it was for any great reason. Thankfully, due primarily to the efforts of my boss, who whittled down the initial file one of our contractors created to about 1/3 its size, it wasn't too hard to understand. Most Ant files are not so nice. I've seen more than my share of horribly long, incomprehensible Ant files that copy files arbitrarily from one location to another, rename directories, and all sorts of other crazy, random stuff.

And the thing is, when you start creating an Ant file, you almost invariably end up reinventing the wheel. Deploying a jar or a war generally consists of pretty much the same steps every time. But because someone decided to structure the project a little differently than the last one, or made some other random decision, there are all these one-off, unique things that the Ant file has to do.

And ant *only* really manages builds--and maybe starting up your app. If you're doing a build in Eclipse, you'll still have to go into the lib and make sure all the jars you need are on the classpath. Some may be assumed to be provided by your web container at runtime, but Eclipse doesn't konw about those, so make sure you go and add those to the classpath too. It just ends up being a pain in the ass.

And that's where Maven comes in. Instead of recreating the wheel every time you start a project, you agree to some conventions. You'll always structure your directories in a certain, canonical way. This is facilitated by the notion of archetypes in Maven. Archetypes are basically just templates for how projects are structured. There available for the most of the common (and even uncommon) types of applications you’re like to build.

In return for this compliance with convention (with which you may or may not totally agree), Maven provides the basic things you always need to do: build, deploy, run tests without having to hand code them. So, in my mind, it’s already got Ant beat at this point. You don’t have to tell it to copy all the jars in your lib into some other dir, or copy some random config files you shoved in the ‘etc’ directory, into the other directories. But then Maven adds something much, much more useful: Dependency management.

Too often I’ve checked out some Java project, add all the jars in the lib path to the classpath, then try to do a build…and stacktrace.

Crap, okay, so this lib I added, has a dependency on another lib. Time to go out on Google and search around. Then you download that, add it to the classpath, then blam.

Dammit, there’s some other dependency. And through trial and error (and a bunch of time), you eventually figure out what you were missing to start with. Often, these are libraries that are only needed for compilation, or they’re assumed to be provided by your web container, so they haven’t been included in lib. But you need them to do anything. And then you have the problem that it’s quite likely that none of your jars are versions

Okay, I have hibernate.jar from two years ago that has no version information. What the hell version is it? Can I upgrade it or will that break everything? These are all lovely questions to wrestle with.

Instead, Maven manages this stuff for you. The POM (Project Object Model) file that you write, instead of a build.xml as Ant uses, is not a procedural set of steps. Maven knows about doing builds via Plugins (which I won’t go into here), and so really all you end up needing to represent are which plugins (i.e. what types of activities you’re performing, rather than the explicit steps that you need to represent in Ant) and what the dependencies are that your project relies on. You can even notate that a certain dependency is only necessary for the build step or the test step. If one of the libraries you’re using requires some other jars of which you were unaware, Maven will figure this out and download them without you ever having to muck around.

And you can specific that *only* version 1.1.2 of a library should be used. You explicit tell it what version to use, so later on down the line, you can easily make the decision of whether it’s safe to move to a new version. If you use the Maven2 Eclipse plugin, you can even reuse the POM to set your classpath, so that whole laborious process of adding jar after jar to your classpath…you don’t need to do that. Just check out the code and bam, ready to run.

Of course, a lot of this happens by magic. But magic, to paraphrase Arthur C Clarke, just means that something is going on that is more advanced than you understand. And recently I’ve seen some complaints that when you have to delve into the magic, things can get messy.

But while using it on ZPlanner, I’ve not encountered any real problems. A few little hiccups here and there, but honestly I imagine going back to futzing around with 200 line long build.xmls for Ant. Or not having the dependency information readily available so I can make intelligent decisions about what my project should or shouldn't be relying on.

If you haven't played with Maven before, I highly recommend taking a look at the excellent online Maven book, available freely from Sonatype. It clearly explains the basic concepts of Maven and let me get my project up and running using Maven in a matter of hours.

Thursday, December 3, 2009

Timetracking, Funny Math, and the Accountants

I joined my current company about 2.5 years ago. And since then the one constant has been my frustration with how inefficient the company is at times. These have included things such as an overreliance on obscenely expensive contractors (The first team of five developers I managed, cost somewhere around one million a year and contained no full-time employees), to endless meetings in which nothing is resolved, to various hopeless complicated processes that amount to nothing more than busy work.

Of course, when the company was bought out a year ago and a new executive team was brought onboard, they immediately began trying ot reduce cost. This jettisoned the 70% contractor work force (something a few of us had been complaining about for years), moved most of the real development offshore, and made cuts in various departments across the board.

But they're not done. And now, apparently, some of them have turned their gaze on our time tracking system.

Now, I've always seen corporate time tracking a curious thing. Time tracking is almost always something mandated by the accountants and managers, so they can get a handle on cost. Like many things, it's an abstraction, and in every implementation I've seen it's an abstraction meant to give the financial people the information they want.

For the people doing the real work, meaning development and QA, the abstraction is generally far less meanginful. They don't see it as something valuable because financials and project codes aren't how they think about their lives. They think about the project or task they're doing. Sometimes these overlap, but just as often they don't.

Of course, I firmly believe in tracking estimates, how close actuals are to estimates, and so on. That's a big part of what I'm trying to tackle with ZPlanner. The difference though, is that in tools like ZPlanner, the numbers come from the actual tasks and work that people have to do. Entering time in such a system gives them value, becuase they can look at a burndown and see how much work remains for their team, they can look at how many hours and tasks are assigned to them and see what's left.

Time tracking systems don't give any of this valuable feedback to a developer. So, people enter their time because they have to. As such it's validity is very questionable. I have only to open up my manager's interface in our time tracking system, and see timecard after timecard filled in with an exact 8 hours every day to know this is not "real" data. It's constructed for my benefit and the benefit of the accountants. My reports fill it out because if they don't, I send them an email chiding them for having not entered their time not because they really *care* about it.

When people are doing something just because they have to, more often than not they do a half-assed job. That's just human nature. If you have a problem with that, take it up with evolution.

And the more complicated you make the rules and more difficult you make it, the more carelessly the task will be performed. Which brings us back to the the attempts of our executives to make our company more efficient.

For 2010, we now have a series of new rules for entering time. The two most impactful changes being:
  • Every time card must have 8 hours entered per day
  • Buckets for management will be eliminated, which means all time must be charged against a specific project

Currently, when I have to file a timecard, I really try my utmost to charge the time I spent to the appropriate project. But frequently, there will be a big chunk of time on any given day, for which I can't really account. The time was usually spent in random conversation about our technology or other ongoing projects, answering email, and so on. It's in such small, discrete portions, though, it's imposisble to recollect exactly how it was spent. Currently, I log this to the "management" bucket. Now I will have to charge it against a project. Most likely this will mean, I spread it evenly across all the projects for which I have oversight, which is everything in my division. Trying to count it accurately would take 50% of my overall time and would be an exercise if futility.

Even the time I spend with my reports in one-on-ones needs to be charged to a project I'm told. I no longer have a 'management' code against which to charge it which will likely be whatever project that particular person is assigned to. The fact that I probably spent the time discussing stuff having nothing to with the project is immaterial. As flawed and inaccurate as the data is now, I'm pretty convinced these rules will cause it to be even more skewed.

My initial reaction to this was that it was a terrible thing. I kind of thought of it in an absolute sense as in, time logged should correspond to reality, and that's the only way the data can led to sane decisions. I thought the intention was to log time as accurately as possible, but I now understand that's not really the case.

Last night, I brought up my concerns about the new time tracking rules to a VP. He explained that the intention was to either pass along the cost or use use these tools to find inefficiency. I told him I thought it was dumb to mandate that everyone has to log a minimum of 8 hours per day. There are probably days they work less than that. And I guarantee you, no one is "productive" eight hours out of the day at this company. If they work 8 hours, they take a coffee break here, discuss some random thing with their colleagues that's completely non-work related and so on.

He countered that the new system would serve two goals. To make sure the cost is passed along to the customer and to make sure people are being efficient. He said that right now the buckets of things like "Management" allow people to hide things. If they try to just amortize this time across their projects, the people who are in charge of those projects will see through it.

But that presumes people know exactly (or even remotely) how long "managerial" stuff takes. I don't think the problem in our company is (primarily) inefficiency on the part of the developers or QA or anyone doing real work. It's the amount of managerial overhead. I can't imagine what requres my team to have 6 project managers for the amount of projects we have. Often a project manager has only 1-2 assigned projects. How can their be *that* much to manage? I just don't buy it.

The VPs contention is that will come out. I suspect something different, however. People will learn other ways to game the system and because now the data will be ever less accurate (if that's possible) than it is now, the decisions the executives make based upon it will be every more divorced from reality.

But maybe it all works out. I don't know. This VP did bring up some good points insofar as "reality" doesn't matter, if the cost is passed along appropriately to the customer and the margins rare. I guess my incredulity is because I don't have an MBA. Reality doesn't matter.

But I like to think it does. And the *real* way to answer these questions, to make sure that costs are calculated appropriately is to start with how the estimates are generated. To track them in a real tool that caputure how much real work is done and how much "management" is logged. To ask hard questions based on empirical observation of why so much *management* time is needed, and to cut the cord in some cases and see what happens.

Managers will always justifty their existence. They create work for eachother. It's not out of ill will, but it's just their nature. There's a great quote from Northcote Parkinson, after whom Parkinson's law is named which goes:

"But the day came when the air vice marshal went on leave. Shortly afterwards, as it happened, the colonel fell sick. The wing commander was attending a course, and I found I was the group. And I also found that, while the work had lessened as each of my superiors had disappeared, by the time it came to me, there was nothing to do at all. There never had been anything to do. We'd been making work for each other."

So to me using a tool that most people don't care about, with an abstraction meaningful only to a few who aren't actually doing any of the "real" work, and which the smart people will find simple ways to game is not the way to fix these problems.

But what the hell do I know, I don't have an MBA.

Tuesday, November 17, 2009

ReviewBoard Woes Or: Build vs. Buy

A couple weeks ago I wrote a bit about ReviewBoard, an open-source web application for doing code reviews. I now have it up and running and have introduced it to my team. Hopefully, over the next weeks and months we'll find it a useful tool for reviewing code.

But it certainly took more than a little work to get it up and running on Windows. I had to install Apache, MySQL, PHP, Django, and any number of PHP packages. Add to that the time I spent fiddling with Apache. Of course, I got it working on my own desktop fairly easily, but for some reason when I put it on a dedicated box and started trying to run Apache as a service, the program was having none of it.

My first issue was that even though I'd configured the thing nearly identically to how I had on my successful installation on my own desktop, it wasn't quite working. The first problem I encountered was that even though it said I'd successfully added my CVS repository (which is required to submit reviews) the dropdown displaying the repositories against which it would check submitted reviews would not populate.

Of course, I did the requisite Google-ing but apparently no one else had had the problem.

Soon enough, I was looking at Apache logs, had started opening up the Python code (and mind you I don't know Python), and eventually descended into putting in print outs in the source to the Apache log.

Ultimately, I traced the problem to an included library. For each possible SCM (CVS, SVN, etc), there is a file in ReviewBoard to manage the interactions. Before populating the dropdown, the appropriate library makes sure it can run the commands it needs to interact with the repository. To do this it uses an including Djblets library which has a function called is_exe_on_path. This function is passed a string literal representing the command to be invoked, in my case 'cvs'. The function then appends '.exe' if the app is running on windows and checks that the file is on the path. In my case, this resulted in a check for a file named 'cvs.exe'.

The problem is the newest version of Tortoise NO LONGER HAS A FILE CALLED cvs.exe. Oh, it used to...and that's why I didn't have the problem on my desktop where I'd used an older version. But on my new computer I'd installed the latest version of TortoiseCVS which has replaced cvs.exe with a file called TortoiseAct.exe. How it's invoked on the command line when one types 'cvs' I don't know (I think via a registry entry and DLL), but ReviewBoard was having none of it.

So, I made a copy of the stupid TortoiseAct.exe, renamed it 'Cvs.exe' and hey, presto, the dropdown was working. I left off for awhile and started playing with some other applications for my new tools box assuming everything had been resolved. In fact, subsequently I ended up installing an older version of Tortoise as the newer version didn't work with Hudson, the Java-based continuous integration server, I also put on the box. So ultimately all of my effort would have been unnecessary had I installed Hudson first.

After I’d finished setting up Hudson and prior to rolling out ReviewBoard to the team, I decided I should do a quick run through. So, I made a file diff (which is how one submits a review) and clicked 'Upload' and waited.

And waited. The thing just sat there hanging frozen on the submission page.

I quickly descended back into opening Python source, adding print outs in various ReviewBoard libraries until I found that the line that it was hanging on. It was where ReviewBoard tried to invoke a 'cvs up' command.

Okay, it's CVS again. Great.

But here there was no apparent problem. I spent a few hours, looking at environment variables, making sure all my batch files were readable by the Default Windows user (which is who services on Windows run as default) was accessible. But nothing worked. I could run the same cvs command the program was running on the command line myself just fine, though. It made no sense.

Finally, I set Apache to run not as the Default User in Windows but as *me*. Hey, presto, it worked.

I tried copying over my ssh.bat and ssh keys into the Default User home directory, tried all sorts of random stuff, but nothing worked. Ultimately, I created a new local user on Windows, gave him administrator rights, and logged into the box to start Apache while actually logged in as the user.

It still hanged.

So, I tried running the cvs command myself while logged in as my new user. And a prompt popped up saying the host needed to be added to the 'known hosts' file. I clicked 'Yes' and suddenly the thing was working. Goddammit! It was because the known hosts file was local to my user account and the Default User’s hadn’t had the IP of the CVS server added yet.

Of course, you may be asking what the point of all this blathering is. Well, I got into a conversation with my co-worker, Jim, during the middle of all of this about the age old 'build vs. buy' question.

There was a very definite cost to all this mucking around. I probably spent the better part of two days mucking around trying to get the goddamn thing to work. I don't blame the ReviewBoard people for this. They used an existing library and the people that wrote that library weren't expecting the application used for CVS access to not have a fill called ‘cvs.exe’ (I still think it's dumb code). Why did TortoiseCVS get rid of the cvs.exe file? Seems stupid to me. But again, it's all random decisions by unrelated parties that make installing an piece of software a potential pain.

The argument could be, I could have bought SmartBear code review and been spared all this pain. I don't know that that's true. It probably is. Of course, I see value in the time I spent. Coming out on the other end of all this, I'd learned quite a bit about setting up Apache2, I'd learned a little tiny bit of Python and Django, and I knew exactly how the software was set up. But again, maybe it was easier to have just bought something that *worked*.

Jim mentioned that he'd really have a reversal of the opinion he had years ago, that often it's better to just shell out the money for something that works, so you don't have to waste time or end up with using mediocre software...just because it's free. And he also mentioned that, as someone who writes software for a living, he believes in the value of software. That good software is worth paying for. I think that's a noble sentiment. Jim is one of the most principled people I know, meaning he really thinks about his beliefs and he acts in accordance with them at all times.

I'm much more of a hypocrite, contradictory, and while I write software that I hope maybe to charge people for one day, I mutter 'No goddamn way I'm paying for this!'. But there is a cost to being cheap.

I think I do agree with Jim in that if at the end, you're using shitty software just because it's free, you've made a mistake. By the same token, I think it's a huge mistake to assume that because you've been charged for something that it has value.

Is JBoss that much better than Jetty? MyEclipse vs Eclipse? SmartBear CodeReview better than ReviewBoard? Maybe.

But I think you have to put some time in and choose where you want to spend money. Look at the cost/benefit carefully. And do realize that in cases such as my ReviewBoard installation, sure I did spend a bunch of my time. But I also learned a hell of a lot. And I see definite value in that. That said, if I tried CodeReview and it was miles and miles better than anything open source, I'd like to think I'd pony up the cash.

In my case, I'm working for a company and having to get a VP's approval on any purchase, just means such things will languish in purgatory forever, whereas I already have ReviewBoard up and running and am ready to roll it out to my team.

Of course, my company has no problem in spending 20,000 on an installation of TFS (Team Foundation Server), which has been presented as the second coming, yet I wonder if people have really examined all the inefficiencies in process we have, rather than assuming Microsoft will solve all our problems. It's all a very interesting subject, but that's all I have time for...for now.

Thursday, November 12, 2009

What is the role of a manager?

I take a pretty dim view of 'management'. Hell, I wrote a whole blog about the existential angst I feel being a manager myself. If you're a developer, your contributions are concrete. You're writing the code and in the end the code is all that matters. In a very real sense, everything else, management, qa, client management, etc is ancillary to what you do.

Once you step into a managerial role, however, things take a decidedly different turn, It's quite likely that nothing you do has such a concrete value.. When I get to the end of each day, I'm often left wondering 'What the hell did I even do today?' And if I'm not sure what I did, how can I even assess whether I added value, whether I contributed something to the team or whether, in fact, I'm just along for the ride, contributing nothing of real worth.

That's not true in all situations, of course. If you're a manager a company where you rose through the ranks--and I suspect, like me, this is how most people make it into management--you probably have a very good understanding of the core code. It's likely you were the one who originally wrote the stuff who others are maintaining. I think that's in those cases it's easy to fall into the trap of ignoring what a manager *should* do so you can continue to contribute in the same way you always have, jumping in to write code, troubleshooting issues no one else can figure out, at that point you're not really a manager, you're a senior developer. Which is fine...as long as you don't have manager in your title.

While labels are often just that, with no real concrete meaning, I think there's an important distinctoin to draw between a "manager" and a "dev".

And if you've take a managerial position at a company where you're *not* already familiar with the technolgy, where you haven't spent years traversing the chaos, so you can navigate it with ease, you have to ask yourself where you can make the biggest value contribution. You can spend your time learning everything, going deep into the details of a small number of items, or you can trust your coworkers, delegate where necessary and focus on the bigger picture stuff that no one else has much time for.

In a company where they've "figured everything out", where the team is cohesive and happy, where client expectations are managed well, and where there are seldom fire drills, there isn't a need for a manager. To paraphrase Groucho Marx, any company I'd really want to work for, doesn't need someone like me. The developers do all the things a manager does. They manage themselves, they introduce change themselves, and collectively they make sure all the important things happen.

But at most places this isn't the case. And while I often have a lot of self-doubt about *not* being involved at everything at the lowest level, I've decided the most important contributions I can make are to make sure things move forward, that the organization improves, that the team is constantly looking for better ways to do things. And that the developers have an advocate, someone who truly understand their concerns and can speak on their behalf. There are many times when I did not have that and I would have been thrilled had my manager done these things.

And when there is a moment free here or there, I do try to run "cvs co" and get into things. Becaues despite all of what I've said, in the end, the code is all that matters.

Tuesday, November 10, 2009

ZPlanner Technology Brief: Mercurial

If you've been following this blog for any period of time at all, you're probably aware of my little personal project called ZPlanner, an Agile project tracking tool. I've listed some of the technologies I chose for the project in past entries, and figured it might be interesting to devote a blog entry to one of them now and again.

Today, I'll give a very brief view of the revision control system I chose, Mercurial. This is by no means a comprehensive overview, there are plenty already floating around on the web and Mercurial's official documentation is great. I see no need to tread the same ground, so I'm more just going to give a very high level overview, why I think it's worth your time to look at Mercurial in more depth, and my own subjective opinions.

In the last few years there has been a dramatic uptick in interest around distributed version control systems. There are a number of competitors in the space, many of which have very similar feature sets. Some of the earlier entries in the field, such as Darcs, seem to be waning in popularity, wherease a few have clearly emerged as leaders in the space. The two biggest names in distributed version control are now GIT and Mercurial.

I'll be upfront, I've not used GIT myself, so I'll stear clear of too much in the way of comparison. What I do mention comes from Mercurial proponents, the validity and objectivity of which I leave to you to assess. The main reason I mention GIT at all is when I was deciding what system to use, it basically came down to a choice between the two.

That said, one of the big reasons I've made the progress I have on ZPlanner is that I try not to obsess too much about technological decisions. One should always do a little research, but at some point it's much better to just pick something and move on than to obsess endlessly about such things. I'm sure if I'd picked GIT instead, I'd be just as far along and it likely would have made little difference in the grand scheme of things.

Pretty much all of my career has been spent using CVS--unless I count an early and unfortunate encounter with Visual SourceSafe at my first job--and most of that time was spent on the command line in Linux. I've certainly heard my share of gripes around it, but to be honest, I never hated it. Part of that, of course, is that most of the time I spent with it it was being used on relatively small, self containted projects. There was little need to branch (and consequently merge), there was usually only a few people doing stuff on head at any one time, and it just pretty much worked. When something went horribly awry, I could usually Google the commands to fix it. In my current environment (where I'm only managing) there is tons of branches and merging and CVS is not really very well suited since commits are only performed on files. In contrast, other systems (and they are not necessarily distributed) have a notion of changesets which show you *everything* that was done as part of change in aggregate. This can be a powerful tool when you have lots of concurrent development.

But if you have a small team, I think CVS is just fine. And when I first started setting up to do work on Zplanner, my thought process was pretty much: I'd like to not have to rename files and move them around when I want to try something new. So, my first impulse was to install CVS. Then, after a moment or two, I thought, everyone says Subversion is better, so maybe I'll do that.

I started installing a Subversion server on my laptop, but about 2 minutes in, I suddenly had an epiphany "What the hell am I doing?" I only need to be able to roll back to older versions. If the server is on my laptop there's nothing special that's making the code any more "backed up" so why go to all this effort? For me, a DVCS would work just as well.

So, I did a bit of reading, and most of what I found talked either about GIT or Mercurial. And enough of it was in favor of Mercurial, that I said 'Hell, Mercurial it is'. The biggest thing for me in Mercurial's favor was the nearly universal opinion that it had better documentation and had an easier learning curve. That was important to me. In contrast to some of my other technology choices, I wasn't using Mercurial to learn Mercurial--I was using it because I wanted to manage change safely within my project.

The biggest difference between Mercurial and something like CVS is the notion that everyone has their own repository and all are equally valid. Like many I suspect, this gave me an oddly unsettled sense that there was no centralized control. But if you think about it, even in CVS there's nothing to stop people from committing bad code in most places. And both GIT and Mercurial have the notion of a hub repository to allay such concerns. This repository is not special, it's simply agreed upon by convention and up to the developers to make sure that the code there is what it should be. If it still sounds disconcerting, it shouldn't. It's really not different.

The biggest difference between Mercurial and CVS, however, is that both Bob and Alice can clone (this is the operation performed rather than a checkout) all the code from a centralized repository and then make any changes they want locally. They don't worry about collisions, or having to update code as they do work. There repository is whole and complete. It contains the complete history of the cloned respository plus anything they've done subsequently.

One they want to integrate their work back into the hub, they simply clone the repository again, merge their changes in and push it back to the hub.

There are a number of advantages to this model of course. One of the biggest is that for geographically dispersed teams (think offshoring), commit operations are much faster. There is no network latency and one need not even have access to do commits, merges, whatever.

Additionally, every contributor has a copy of the repository (though in differing states) on every his PC. If one is lost, it's probalby not the end of the world. Contrast this with a usual CVS setup, where if cvsroot gets hosed you're screwed. One could argue that in such a scenario, people will have a copy of the checked out code, which is just as good. The difference, however, is that in such a situation they *only* have a code. They don't have the complete history of checkins, old version so the files, and so on. They only have the last revision. In Mercurial, you have *everything*.

A big plus for Mercurial is its ease of use. Particularly, if you've used CVS, many of the commands are nearly identical. Additionally, the documentation around Mercurial is excellent and although some of the tools lag a bit, they are quickly catching up. There is already a good plugin for Eclipse, which is what I've been using. For local development, it behaves almost identically to using CVS via Eclipse. There is also a standalone client from Tortoise, TortoiseHg, (though from what I know it is a bit buggy on Windows) among others.

Additionally, Mercurial comes with an extension named 'convert' which allows one to import revision history from any of Subversion, CVS, Git, or Darcs.

When compared to its primary competitor in the distributed RCS arena, GIT, it is often noted that Mercurial is easier to learn due to its focus on simplicity. While most of them are not needed, GIT has over 139 commands. Mercurial about half that. Additionally, due to the how the data is stored GIT repositories need frequent manual maintenance (called "repacks") to maintain performance and prevent rapid increases in disk usage. Mercurial, in contrast, requires no active maintenance.

Mercurial is currently in use by projects having hundredes of contributors, including such high-profile projects such as Mozilla, NetBeans, Python, Symbian, and XEmacs.

The core, however, is what is it like to use Mercurial on a day-to-day basis? And for me, it's nearly invisible. Admittedly, I've been using it in a fairly limited context, but to give some sense of its power, here a quick example:

I installed Mercurial locally to track changes on my laptop. Recently, I decided I'd like to share my code with a few people. Using BitBucket.org (what GitHub is to Git, BitBucket is to Mercurial), I imported my repository--AND ALL OF ITS HISTORY in about 5 minutes. Now, I can give a URL to anyone to checkout the code and they can see every commit I ever made (with associate comments) since I started. I thought that was pretty damn cool.

If you're curious to see the ZPlanner source or what a Mercurial repository looks like, sign up for an account on BitBucket.org and shoot me your username and I'll add you (this is assuming I know you as ZPlanner isn't open source).

Tuesday, November 3, 2009

Using software to do code reviews (ReviewBoard)

It's fairly universally accepted that all code should undergo some type of review. The problem is, while it's easy in theory, in practice it tends to be kinda tough. I've had various experiences participating in code reviews ranging from realtively decent to horribly painful and I still don't know that I have a perfect answer as to "right way to do things" (tm).

If I grab my "Software Engineering" book back from my college days, I can read about doing print outs of the code, having an official reader, while others follow along to make comments, and so on. No one I know has ever done anything quite so formal.

More often, it's grabbing a bunch of developers, and having the guy who wrote the code put it up on the projector, while people make various comments about his variable naming, or why he wrote

if(!boolOne && !boolTwo)

rather than

if(!(boolOne boolTwo))

or things equally silly.

In one particularly painful code review, I remember an "architect" giving a discourse on why magic numbers were bad. Fine, fair point. But after the first fifteen minutes, the point was made...yet we kept going for another 20 minutes. Yay!

The other problem, of course, is that while one should leave his ego at the door, it's kinda tough. While I think there's a lot of value in having devs *know* someone else will look at their code (The "Crap, if do this hacky thing, someone will see" effect), sometimes code reviews can be incredibly demoralizing if it just turns into a 'beat up on the developer' exercise.

Even with these caveats, though, I think code reviews are absolutely essential. But as challenging as code reviews are even with a bunch of motivated, full-time employees, all located onsite, it becomes even more challenging when your resources are located remotely.

What do you do then? Have everyone call in from India at 11:20pm their time using a web-ex and a conference bridge that's intermittantly goes in and out, while they walk through the code?

I've tried this and it's never worked out very well. It still had value, but man, it was *not* fun.

Given the fact that 80% of the devs writing software for my team are remotely located, I've struggled a lot with this question over the past few months. In fact, I've let things kind of just go on as they have, which entails the onsite dev leads waiting for CVS check-in emails, or notes from the offshore guys, then passing back comments via emial.

The problem with this, of course, is that it's ad hoc. Even putting some type of process around it (a checklist?), maybe an item on a Wiki page for the project to be struck-through, it still doesn't say anything qualitatively about the review. And there's no real audit trail. It's all via emial or the phone.

So, recently I started to look around for software that might be able to help facilitate this process. I've known about tools like SmartBear CodeReview for some time, but it's always tough to get approval for software that costs money. Mind you, I'm not saying it's impossible, but if you want to put something in place quickly at my company, it's not the way to go. Add to that the fact, that the company is looking at putting a company-wide installation of Microsoft TFS (Team Foundation Server) which includes some code review facilities and I have little hope of getting approval.

So, instead, I started looking at open source projects. There are a few options, but I ended up installing ReviewBoard.

ReviewBoard was created internally by the guys at VmWare so it's definitely more aimed at the Linux crowd. But with a little effort, I was able to (mostly successfully) get it up on an running on a Windows box. It's not without it's warts, but I think it might be quite useful in my current situation.

I can post all sorts of screenshots and so on, but instead I'll just post up what I did to get it running (in case you want to try it out). Hopefully this will be useful to someone out there (Keep in mind I'm not a systems admin, so please forgive any retardation in what follows):


  1. Download Xampp
    1. Xampp is an easy-to-install Apache distribution with MySQL, PHP, and Perl already included
    2. To install, simply unzip the file to your directory (I use C:\opt) and your good to go
  2. Download Python v2.5.4
    1. Subsequent releases are not currently supported in mod_python unless you want to build your own binary (And I didn't)
    2. Add a new system variable name PYTHON_HOME and set it to your install dir location, which should be something like: C:\python25
      1. Add the following to your PATH:
        1. %PYTHON_HOME%;%PYTHON_HOME%\Scripts
  3. Download mod_python-3.3.1
    1. Make sure the version you download corresponds to your version of Python (2.5.4 if you followed the instructions above), to your processor type (most people it'll be the 32-bit binary), adn to the version of Apache you'll be using.
      1. The version I used (32-bit, Python 2.5, Apache2.2) is here
    2. Edit the Apache conf (which should be located somewhere like C:/xampp/apache/conf/httpd.conf, find where the other LoadModule lines are and add this line

    LoadModule python_module modules/mod_python.so


    1. Create a simple test directory within XAMPP htdocs directory (which is where any deployed apps go by default) to test your install of mod_python. If you have any difficult with the steps as detailed below, see the instructions on the mod_python site here:
      1. In my case I created a directoy C:\xampp\htdocs\test
      2. Update your XAMPP install so it can recognize Python code, by updating httpd.conf. You'll want somethlng like this (note this is specific to the test directoy I created in the previous step)

        <Directory "C:\xampp\htdocs\test">
        AddHandler mod_python .py
        PythonHandler mptest
        PythonDebug On
        </Directory>

      3. Add a file containing the following to the newly create 'test' dir (Keep in mind this is Python, so spacing matters and you probably can't just copy and paste what I have below here)
        from mod_python import apache

        def handler(req):
        req.content_type = 'text/plain'
        req.write("Hello World!")

        return apache.OK
  4. Download the tarball of Django v1.1.1
    1. Extract the tarball to your directory of choice
      1. In my case this was C:\opt

    2. Goto the directory to which you installed Django and run the following command:
      1. Run setup.py install

  5. Download GNU patch.exe
    1. Run install (by clicking on .exe)
    2. Add the bin directory to your PATH system variable (i.e. C:\Program Files\GnuWin32\bin)

  6. Download the Python imaging library (Make sure you get the version appropriate to your version of Python. In my case, this was Python 2.5)
  7. Download PyCrypto
  8. Install SetupTools for Python.

    1. This includes EasyInstall which will make installing additionally needed Python components for ReviewBoard easy

  9. Use EasyIntall (installed via step 8) to install memcache from the command prompt:
    1. easy_install python-memcache

  10. Install ReviewBoard using EasyInstall
    1. easy_install ReviewBoard

  11. Install MySQL connector using EasyInstall
    1. easy_install mysql-python
    2. While ReviewBoard comes with an embedded SQLLite install, you probably want to use MySQL.
    3. This step actually failed for me as it requires VisualStudio 2003 (I have VS 2005, which doesn't work for this), so I'm using SQLLite for the moment

  12. Create your 'site' for your reviewboard installation, naming it as you'd like the site to be accessed. I used reviews.com and updated my localhost file so this would work (i.e. I access ReviewBoard on *my* computer via reviews.com)
    1. rb-site install C:\xampp\htdocs\reviews.com
    2. Source the ReviewBoard specific conf file in your global httpd.conf file
      1. The ReviewBoard conf file should be be here: C:\xampp\htdocs\reviews.com\conf\
      2. The Apache conf will be here C:\xampp\apache\conf, if you used the default install location)
      3. Here's the line you'll want to add:
        1. Include <install dir for ReviewBoard site>/apache-modpython.conf
  13. Create a .reviewboardrc file in C:\Documents and Settings\<username>\Local Settings\Application Data with the same name as your site (in my case "reviews.com")

    REVIEWBOARD_URL = http://reviews.com/

  14. Use TortoiseCVS to configure access to CVS via the command line (I'm going to leave this out unless someone complains about it, at which point I can provide more details)
    1. Download TortoiseCVS
    2. Generate SSH keys using PuttyGen
    3. Install your SSH public key on your CVS server
    4. Create the following file and name it "ssh.bat"
    5. @echo off

      call "C:\Program Files\TortoiseCVS\TortoisePlink.exe" -2 -i "C:\Documents and Settings\yourusername\sshidentity.key" %*

  15. Create a new Environemnt variable named CVS_RSH with a value of the full path to the "ssh.bat" file
  16. Add the CVS_RSH environment variable as well as the path to the TortoiseCVS install to your PATH variable

Pheww! Now, you're ready to start using ReviewBoard, which maybe I'll cover later.