Monday, October 29, 2012

Technical Debt: Every Company Needs a Technical Debt Remediation Program















The Gartner Application Development Advisory for 2012 strongly mandates abandoning waterfall, going Agile, and implementing a technical debt remediation program. This needs to be driven by the Product Owner and supported by management. But what is technical debt, and how can we manage it?

IEEE Software (Nov 2012) devotes a whole issue to this problem with many enlightening analyses and viewpoints. The point-counterpoint pages above show how to put a financial number on technical data. On the other hand the case is made that technical debt is more far reaching than any financial number and can lead to success or failure of products and companies.

Point: Technical Debt as Meaningful Metaphor for Code Quality by Israel Gat
     Unlike financial debt, technical debt can't be calculated from history alone. Growth of financial debt as a function of time is determined by computing interest and adding dollars owed on older debts to newly taken ones. In contrast, the technical debt accrued by accepting a high level of complexity nine months ago is quite different from the technical debt accrued by quickly duplicating some blocks of code six months ago. when various kinds of technical debt are taken at different points in time, the grand total at a later point is hard to assess. The development team might still recall that nine months ago, it borrowed a week's worth of time on complexity, and more recently, a day through code duplication. However, between these two "loans" and sofware decay, will it take 6, 10, 20, or 50 days to pay back the debt? That;s the real question.
     To find your current level of technical debt, you need to start with the current state of your code and dig deep: start with code analysis, look at quality deficits, determine the time to fix per deficit, determine the time to fix per deficit, aggregate the time to fix, and then aggregate the cost to fix. Once you monetize technical debt, any stakeholder can understand and appreciate its operational, financial, and business implications. A CFO will have no problem relating to a statement such as, "the technical debt on the project amounts to $500,000." Likewise, his or her peers in marketing, professional services, or customer support will easily assess what this level of technical debt means to their departments.

Counterpoint: A Useful Metaphor for Risk--Poorly Practiced by Christof Ebert
     My own definition (of technical debt) is pretty simple: technical debt is the eventual consequence of poor engineering of a software product for short-term benefits that make the same work cost more to do later. Let's look to any industry case study.
     In 1996, Netscape Navigator ruled the search engine market with a share of 80 percent; Microsoft's Internet Explorer held the remaining 20 percent. This should have been enough to make it a fat cash cow for decades to come, yet in 2002, Microsoft's market share jumped to 96 percent and Netscape's was just 2 percent. What happened? When founded in 1994, Netscape owned a wonderful and unique product for accessing the Internet. Engineers evolved it quickly but lost control--speed dominated engineering and quality, the code got worse with each new function, and nobody realized that they had accuulated a debt that couldn't be repaid. This brings us to our first lesson: make technical debt visible. Explorer's initial code also got out of control, but Microsoft took another direction--it opted to fully redesign Explorer, which delayed the product but eventually helped it dominate the market. Our second lesson: evaluate tradeoffs. Microsoft decided that a core architecture team, product management, and maintainability and portability where its top goals. In contrast, Netscape pushed Java into a new product but without a clear architecture and product design. Eventually the company disappeared. Our third lesson: systematically control technical debt.

5 comments:

Sebastian Radics said...

Hi Jeff,
thanks for sharing your thougths on technical dept. Especially the hints for measuring from the beginning and what to measure are helpful - also to show where problems are coming from (and not to blame to wrong side e.g. Scrum for making things transparent).



I already had long discussions regarding technical dept and why refactorings and redesigns are necessary. Maybe it's often a problem that it's not clear what's technical dept - thanks for your definition - and why it's really necessary to work on it - thanks for the Netscape, IE example.



From my experience it's also really important to give objective arguments, as sometimes developers tend to play with new technologies and this could drive a wish for refactorings. It helps to involve the product owner and business side and collect convincing arguments - this way all can commit to necessary changes and the risk for purely "new" technology driven changes gets lower.



Do you have concrete suggestions for visualizations of technical defects?

Sebastian Radics said...

Hi Jeff,
thanks for sharing your thougths on technical dept. Especially the hints for measuring from the beginning and what to measure are helpful - also to show where problems are coming from (and not to blame to wrong side e.g. Scrum for making things transparent).



I already had long discussions regarding technical dept and why refactorings and redesigns are necessary. Maybe it's often a problem that it's not clear what's technical dept - thanks for your definition - and why it's really necessary to work on it - thanks for the Netscape, IE example.



From my experience it's also really important to give objective arguments, as sometimes developers tend to play with new technologies and this could drive a wish for refactorings. It helps to involve the product owner and business side and collect convincing arguments - this way all can commit to necessary changes and the risk for purely "new" technology driven changes gets lower.



Do you have concrete suggestions for visualizations of technical defects?

Cyril Landriot said...

Thanks a lot for this post. We start in my company a training program regarding legacy product management and technical debt and it will definitely give me more data to push it.
BTW, I haven't found any reference to this 'Gartner Application Development Advisory' out of your post. Could you please help me find this report or guide to the place where I can buy it?

To Sebastian: These concept are used by some maven plugins, such as Sonar. It has been used by at least one project in my company to convince product management to prioritize technical debt related user stories.

Cyril Landriot said...

Thanks a lot for this post. We start in my company a training program regarding legacy product management and technical debt and it will definitely give me more data to push it.
BTW, I haven't found any reference to this 'Gartner Application Development Advisory' out of your post. Could you please help me find this report or guide to the place where I can buy it?

To Sebastian: These concept are used by some maven plugins, such as Sonar. It has been used by at least one project in my company to convince product management to prioritize technical debt related user stories.

Jeff Sutherland said...

Gartner is the leading analyst group on the planet for CIOs and their advisory services publishes an annual report on application development strategy. http://www.gartner.com/technology/research/industry_advisory_services.jsp