Thoughts on Offshoring, Pt III

Last time I wrote about my experience with offshoring on a large migration project. Interestingly, as soon as we moved our code into production I was moved over to manage a completely different group. As a result of a company reorg my job had been promised to someone else. For awhile, it looked like I was going to be laid off, but after checking around a bit I was able to snag a development manager position within a different group . Ironically this new position was within a bigger group, the rising star of my company in fact. So what had at first seemed like a terrible disaster was, in fact, a blessing in disguise.

My new team had been formed around the same time my old team had started implementing its offshoring stategy. While in my former group we'd charted our own path (to mixed success), my new group had followed the path laid by the executives within our company, both in terms of their choice of vendor (which was located in India), as well as the way they interacted with the offshore team members.

In fact, when the topic of using offshore as our primary development force (it had been use in a minor capacity for quite some time), started to be discussed in earnest, the decision of who would be our official "offshore partner" was one of the first topics to be discussed. I was told at that time that one vendor stood out because of the contract terms we had with them. Our contract with them contained clauses such that for we would levy "SLA" penalties against them based on the number of bugs in the code they produced. Additionally, if the code they wrote resulted in downtime of our system (which would result in SLA penalties for us) we would also charge them.

I thought this punitive model of development one was a terrible way to begin the relationship and said as much to my director. I really hold with the Agile folks about the absolute criticality of trust and collaboration. Of course, Agile doesn't hold any kind of monopoly on these things, but they do tend to put particular emphasis on these qualities. They seemed to disregard the importance of these things entirely.
"You wrote bad code, I'm going to smack your hand!"

But then the whole idea that "SLAs" in the contract were the reason we preferred this company was a a bit of smoke and mirrors. In fact, it was offhandedly mentioned in another meeting that our new offshoring company had a number of the same people on its board as our company did. When I learned that, I stopped asking questions. As with most business decisions (at least in my experience) it seemed to be based around how those at the top can put or keep th e most money in their own pockets.

The interaction model with this vendor was roughly as follows: we would maintain a staff of "development leads" on site. The leads along with our higher-level solution architects would be our only onshore development resources. The solution architect would interact with our internal program managers and customers to create a high-level solution. This solution would then be further fleshed out by an onsite development lead. The onsite development lead would then interact with a corresponding development lead on the offshore team, who would divy out this work to the developers on his side. This offshore team would also have their own managers to oversee the work who would interact with their counterparts on this side of the world.

I was extremely dubious about this model with its emphasis on "hand offs" and overlapping team members. More personally I was concerned this would mean I'd be on the phone with the guys in India at 3am every morning, and so on. This was not a pleasant prospect.

Interestingly, however, as part of the partnership we decided to bring over a number of offshore team members (mostly managers, a few devs) who bridged the gap between our offices. It is mostly these poor souls who now end up on the phone late every night with India, or having to convey information across to the team overseas.

What this has meant for me os that my interations with offshore have been relatively painless. The pain has largely been bourne by the onsite representatives of the offshore team. There have been minor issues I've had to deal with, but keeping track of resourcing, communicating, etc has been facilitated by the onsite members of the offshore team. Of course, the devs themselves still have to interact, but when we need to have calls between the teams they are done early in the morning our time. Sucks for the dudes in India, but relatively okay for us.

Also, surprisingly the model seems to be working. We've successfully launched a number of very large projects using it. That said, there is one huge caveat, and it's this:

I don't think it's saving us any money.

While the work churned out by the offshore team is reasonably adequate, it's very evident to me that there's a tremendous amount of overhead. I have almost 25 offshore resources reporting to me, logging thousands of hours. I guarantee you were not doing anywhere near that much real *work*. Of course, one example is worth a thousand words.

Recently, we had a fairly small project come in. We had one of the onsite dev leads provide an estimate. It ended up coming in around 4-5 days of effort. When we reached our formal kick-off, it was apparent we'd allocated 2 offshore devs full time for almost two weeks (so roughly 4 times the amount of time) to do the work. We also had hours and hours of time for multiple onshore and offshore managers. If you looked at the totals for hte project, it was absurd. All for four of five days of work. I've seen this now several times. The offshore resources may be a third as expensive, but they seem to take two times as many hours (if not more) to do the same work as our onsite resources. Then you have to factor in the fact that the onsite dev leads are spending 50-60% of their time simply overseeing this work, that you have to allocated hours and hours of offshore manager time, and it and pretty soon you've just blown way past the cost of the doing it onshore.

But the weird things is the company *is* saving money, so it's hard to argue. The problem is that there is no reasonable state prior to the offshoring with which one can compare costs. In fact, prior to the move to offshore the company was using almost 70% contractors, who were *extremely* expensive. Many of them weren't even good, some were atrocious. Even a crappy dev in such cases could cost as much as $210,000 a year. Of course they personally only saw a fraction of that. I managed a team of about 5 developers at one point, all contractors. My team cost over $1 million a year. If these guys had been FTEs, even with benefits the team would have maybe cost $600,000/yr Of course, the offshore stategy--even with it's tremendous inefficiency--is saving money over this insane policy.

Think of it this way, your nutty uncle who thinks he's the reincarnation of Elvis seems quite tame next to Charles Manson. Whether or not our offshoring strategy is saving money over a *rational* approach using great FTE developers is quite a bit more questionable to me.

Next article I'll wrap up with a summary of each of my encounters with offshore and the conclusions I've drawn from all of this.