Monday, October 29, 2012

Technical Debt: Every Company Needs a Technical Debt Remediation Program

The Gartner Application Development Advisory for 2012 strongly mandates abandoning waterfall, going Agile, and implementing a technical debt remediation program. This needs to be driven by the Product Owner and supported by management. But what is technical debt, and how can we manage it?

IEEE Software (Nov 2012) devotes a whole issue to this problem with many enlightening analyses and viewpoints. The point-counterpoint pages above show how to put a financial number on technical data. On the other hand the case is made that technical debt is more far reaching than any financial number and can lead to success or failure of products and companies.

Point: Technical Debt as Meaningful Metaphor for Code Quality by Israel Gat
     Unlike financial debt, technical debt can't be calculated from history alone. Growth of financial debt as a function of time is determined by computing interest and adding dollars owed on older debts to newly taken ones. In contrast, the technical debt accrued by accepting a high level of complexity nine months ago is quite different from the technical debt accrued by quickly duplicating some blocks of code six months ago. when various kinds of technical debt are taken at different points in time, the grand total at a later point is hard to assess. The development team might still recall that nine months ago, it borrowed a week's worth of time on complexity, and more recently, a day through code duplication. However, between these two "loans" and sofware decay, will it take 6, 10, 20, or 50 days to pay back the debt? That;s the real question.
     To find your current level of technical debt, you need to start with the current state of your code and dig deep: start with code analysis, look at quality deficits, determine the time to fix per deficit, determine the time to fix per deficit, aggregate the time to fix, and then aggregate the cost to fix. Once you monetize technical debt, any stakeholder can understand and appreciate its operational, financial, and business implications. A CFO will have no problem relating to a statement such as, "the technical debt on the project amounts to $500,000." Likewise, his or her peers in marketing, professional services, or customer support will easily assess what this level of technical debt means to their departments.

Counterpoint: A Useful Metaphor for Risk--Poorly Practiced by Christof Ebert
     My own definition (of technical debt) is pretty simple: technical debt is the eventual consequence of poor engineering of a software product for short-term benefits that make the same work cost more to do later. Let's look to any industry case study.
     In 1996, Netscape Navigator ruled the search engine market with a share of 80 percent; Microsoft's Internet Explorer held the remaining 20 percent. This should have been enough to make it a fat cash cow for decades to come, yet in 2002, Microsoft's market share jumped to 96 percent and Netscape's was just 2 percent. What happened? When founded in 1994, Netscape owned a wonderful and unique product for accessing the Internet. Engineers evolved it quickly but lost control--speed dominated engineering and quality, the code got worse with each new function, and nobody realized that they had accuulated a debt that couldn't be repaid. This brings us to our first lesson: make technical debt visible. Explorer's initial code also got out of control, but Microsoft took another direction--it opted to fully redesign Explorer, which delayed the product but eventually helped it dominate the market. Our second lesson: evaluate tradeoffs. Microsoft decided that a core architecture team, product management, and maintainability and portability where its top goals. In contrast, Netscape pushed Java into a new product but without a clear architecture and product design. Eventually the company disappeared. Our third lesson: systematically control technical debt.

Thursday, October 25, 2012

Getting Lean with Scrum

By Christine Hegarty

In working for the creator of Scrum, I have gained an appreciation for the influence of Lean thinking on the development of Scrum principles.  The concepts of continuous improvement, eliminating waste, limiting work in progress, etc. are intrinsic to culture at Scrum Inc., but at times I forget that the influence of Lean, specifically on the Sprint Retrospective, is not entirely intuitive. 

The purpose of the Retrospective is to a) inspect how the Sprint went, b) identify and order what went well and what didn’t and c) create a plan for implementing improvements.  The vision of this meeting was clearly influenced by a core Lean principle: continuous improvement.  However, like most aspects of Scrum, the process for implementing these improvements is not defined, and thus, sometimes is difficult for teams to conceptionalize.  And, once teams are up to speed on the Scrum process, fighting the urge to become comfortable becomes a challenge.  Asking questions and looking at things differently is hard and uncomfortable, especially when an organization or team feels that everything is going well.

One particular example comes to mind from a Capability Assessment we recently led at First Line Software in St. Petersburg, Russia.  From top to bottom, First Line has proven robust Scrum competency.  They have maintained a culture that embraces Scrum best practices, supports high quality development and fosters collaboration with clients to relevant increments each Sprint, a practice that earned them a Scrum Capability Rating this September.  They have proven themselves to be mature, disciplined Scrum teams.  In our experience, teams at this level fall into a rhythm of addressing the more topical impediments since they aren’t dealing with a major pain point.  The First Line teams were no different.

Through the Assessment, the Scrum Master recognized an opportunity to incorporate continuous improvement.  I worked with her to reinvigorate the Retrospective by using the Happiness Metric and the Lean concept of kaizen (Japanese for “improvement”).   Introducing this somewhat different angle helped her realize the power of putting a process around their Retrospective.

Regular readers of the blog will recall previous posts on the Happiness Metric.  From a 1-5 rating by each team member of “happiness” about the previous sprint and discussion about likes and dislikes, the team agrees on a single kaizen to be added to the Sprint backlog as a task, with a clear goal and criteria for “done.”  During the week, the team keeps focused on the agreed improvement while burning down backlog.

As a result, we saw that by applying this process and Lean principal, the team was able to facilitate a more rigorous retrospective, while connecting the impediment backlog with the Sprint Backlog.  The combination brings process improvement to the forefront, sharpening the focus on increased productivity and velocity.  Here’s to finding another Scrum team, ready to incorporate the Lean edge.

Friday, October 19, 2012

How Scrum Manages Risk

There are 457,889 Scrum job openings today in the United States, up from only 20,000 earlier in the year. The top companies hiring are:
As a result, I am working with a lot of traditional project managers and they keep on asking me, "How does Scrum manage risk."

A lot of what is in Scrum is based on my experience as a fighter pilot, particularly my 100 missions over North Vietnam. Everything about Scrum is risk management.

When I flew into Udorn airbase in northeast Thailand in 1967, my RF4-C squadron was replacing two RF-101 squadrons which had 45 aircraft shot down during the last year. Of the five that remained, all had so many bullet holes they were not flyable.

My strategy was to enter into an evasive maneuver crossing the North Vietnamese border and never stop evading until returning back into Laos. You could not see the tracer bullets in the daylight.

When I joined a large banking company in 1983, I notice that every day there were bad things happening to projects and the failure rate of larger projects was over 80%. Not quite as bad as the RF-101 squadrons but almost.

The solution was to implement short sprints, inspect and adapt every day, and do a major retrospective at the end of every sprint. Failing fast and learning for the next sprint provides huge risk avoidance benefit and it lays the groundwork for innovation. Proper execution causes incremental improvement with increased quality that accelerates velocity.

This gives the Scrum team the ability to operate like a small entrepreneurial company even when embedded in a large enterprise. The right attitude for the team is exempified in this video on entrepreneurship. Start small and execute one sprint at a time. You will fail early and often which will, counterintuitively, radically reduce the risk of the total effort. The ability to dodge bullets will surprise you!

Friday, October 05, 2012

Making the Daily Standup Work

As the author of the paper Scrum in Church, I’ve been invited to give a talk about using Scrum in non-IT settings.  It’s hard to find a setting more removed from software development than a church!

For the past two years I have been working at Scrum Inc. where we do no software development and use Scrum to run everything.  It’s not much like a church either.  So, I was really surprised, as I reread Scrum in Church, to realize that the congregations I served and Scrum Inc. have many of the same challenges.

An issue I encountered in each congregation I served was that because the staff included part-timers and people who worked very different schedules, it was impossible to have daily meetings that included everyone.  Staff who worked on Sundays, took days off during the week.  Others worked 9-5 Monday thru Friday.  Part-timers by definition aren’t there all the time.

Scrum in Church.key resized 600In church, we decided to hold daily stand-ups every morning regardless of who was there.  So every morning, Monday through Friday, all staff who were in the building gathered.  After a while, some of the full-time staff who took days off during the week started dropping in for the Stand-up on their day off. 

At Scrum Inc. we also wrestle with having everyone at our Daily Standup.  We have two part-time employees.  Jeff in particular travels extensively, mostly in Europe.  One of us lives and works in D.C.   The rest of us are in the Boston area.  

Being able to use Google Hangout and Skype makes it easier for those of us who either work different schedules or are geographically dispersed to attend the daily Stand-Ups.  But, when one or more of us is leading a course or consulting, it’s not possible to get to a Daily Meeting, even for 15 minutes.

Scrum works best when the Team is collocated and everyone works the same schedule.  The real world rarely works like that.   There are no rules about how to adapt to those shifting circumstances, but here are a few techniques we find helpful:

1. If a team member can't make the daily standup, ask them to email their report to the Scrum Master so it can be included in the update, and then email the key points from the standup out to the whole team.  (Just the key points, not meeting minutes.)

2. Schedule regular face-to-face meetings.  There is no substitute for being in the same space at the same time, for the conversations at the water cooler, for hearing what people did over the weekend, for sharing a meal, and acknowledging milestones like birthdays and tenure at the company.  Design a travel budget so that people who live and work at a distance can get together on a regular basis.

3. Use a virtual Scrum board.  While we know that sticky notes on a wall work best, using the simplest most lightweight tool you can is the next best solution.  Every member of the team needs to be able to access the board all day, every day. (We like Pivotal Tracker.)

4. Focus on communication saturation.  When in doubt, send that email to everyone on the Team.  Make sure that all work is captured in a backlog story so that everyone knows what is happening.  If a document was the result of a story, attach it so all can read it.  If there is a meeting circulate a high-level summary.

At its heart, Scrum is about putting people first, about creating a Team, about excelling, about having fun!  We human beings can be an irritable cantankerous lot; we can also be a generous joyous people working to make our lives and this world a better place.

By Arline Sutherland