I ran across this article on ZDNet (Wanted: ROI for internal app development) that really worried me. My friends at PreEmptive Solutions ran a survey across a wide number of developers that included people from 21 different industry segments in 33 countries asking about how the company measure the ROI of an application that they are building. The terrifying part is that the survey found that 58% of companies don’t bother measuring ROI on their internal applications and the majority of the ones that do measure don’t do so in a consistent and proven way.
This means that you are almost positively throwing away time and effort and therefore money. For some crazy reason, companies don’t like that.
Of all of the companies surveyed, only 6% have a consistent and reliable approach to measuring ROI. That terrifies me.
What are your Goals?
For a long time I’ve been preaching that IT needs to become a strategic partner to the business.
The issue is that most IT departments are considered cost centers. This means that they are the first to be cut when times get tough, they are the least invested in, they are the last at the table to be heard and they are pushed around by all of the other departments. This is not a good position to be in. This is why a lot of companies look to outsource IT. Since they don’t see it as a strategic asset and partner it doesn’t matter if they keep it close.
The way to move from cost center to strategic partner is to start showing value and to start pushing the edge on ideas that will drive a great ROI for the business.
But, if you’re not measuring the ROI, how can you talk to the business about the possible ROI?
And every time that someone asks me how to sell their management on usability studies or spending any time on a UI, I tell them to go to the numbers.
And every time someone asks me how to sell their management on buying a given tool, I tell them to go to the numbers.
And every time someone asks me how to sell their management on training, I tell them to go to the numbers.
And every time someone asks me how to sell their management on anything at all, I tell them to go to the numbers.
ROI really it comes down to a simple question. How much money did the business make or save based on their investment in the software that they just paid to have developed?
There are two ways businesses think about investments. ROI, the first, is the percentage of return over a given period of time. Payback, the second, very closely related to ROI but very importantly different, is the length of time that it takes to recoup the investment. The reason that payback is important is because once you hit the payback point everything else is profit.
I borrowed the graphic to the left from Paul Sheriff’s article called The Business Value of .NET.
An overly simplistic example is as follows. Let’s say a company wants to build a new online e-commerce site. If it takes 4 developers whose salaries are each $100,000.00 working for 3 months on a project with tools that they already own and so on. That means that it cost $100,000.00 to build. If the new site generates an average of an additional $15,000.00 a month for the first year, the gross profit is $180,000.00 with a net of $80,000.00.
We look at that as $80,000.00. That’s a good thing. However, there are two ways that the business is going to look at this example. The ROI is 180% and the payback is 6.6 months. Now, if the business sees an investment in another department that could yield a higher percentage or lower payback, that department is more likely to get the investment.
This is not a new problem. Back in 1999, Bill Gates wrote
“When I sit down with developers to review product specifications, or with Microsoft’s product divisions to review their three-year business plans, or with our sales groups to review their financial performance, we work through the difficult issues. We discuss feature tradeoffs vs. time to market, marketing spend vs. revenue, head count vs. return and so on. Through human intelligence and collaboration, we transform static sales, customer, and demographic data into the design of a product or a program.”
Business @ the Speed of Thought : Using a Digital Nervous System
By: Bill Gates
Whether you are selling software or building internal applications, you need to go through the same process.
There are two basic ways to increase ROI, produce things cheaper or provide more value.
How did it cost to develop the software?
The problem is that this includes more than just the developer’s salary. This includes how much were the machines, rent for the building they were in, telecom costs, training costs, their secretaries, management, consultants, tools, components that they bought and more.
You can keep costs down in a tremendous number of different ways.
Understanding what you’re building – There are many different types of processes that are out there that are more or less efficient depending on the team, project and a whole lot of different requirements. There are a handful of absolutes.
The first is that you need to know that you’re building. You can accomplish this through solid requirements analysis.
Martin Shoemaker makes An Argument for Requirements Analysts where he points out research done by Boehm and published in Software Cost Estimation with COCOMO II found that “Excellent requirements analysts can reduce a project’s schedule by almost 30%, while inadequate analysis can increase the schedule by over 40%.”.
David Platt has been talking about this same problem for years as “Why Software Sucks”. He’s been on ArCast twice, once with Ron Jacobs and once again with Bob Familiar. And he’s got a book on the topic called Why Software Sucks…and What You Can Do About It
Before you freak out, this does not mean Waterfall. This means that you understand what you’re going to be building and have a good grasp on the requirements. You’re going to make different web framework decisions if you have to scale to 10 users verses scaling to a million users. You’re going to make different input type decisions if your users are working outside and likely to be wearing gloves. You’re going to make different data locale choices if your user is going to be sitting in the corporate office next to the data center verses sitting in India or somewhere.
With the Unified Modeling Language (UML), which can be leveraged with almost any process, requirements are captured in Use Case Diagrams. Check out Martin Shoemaker‘s UML Applied for a light weight process he calls 5 step UML.
In Extreme Programming, one of the Agile methodologies, these requirements are captured in User Stories. Regardless of your process, there is a way to do requirements analysis and it’s absolute that you have to do it.
Another absolute when it comes to process is that you need to have some form of user acceptance. A huge issue with most waterfall methodologies is that they push this user acceptance to the end of the project. At this point in time, the cost of fixing issues that the user finds with the software is prohibitively expensive. This is why Agile methodologies have the user involved in the project the whole way through and get smaller bits of functionality in front of the user earlier and more often. Course corrections along the way are a lot less costly than reversing and undoing months of work.
Buying a component or application – Building software is expensive. 
Most of the time, it’s cheaper to acquire something than to write it yourself. The question is, how close is the component or application to what you needed and how much customization do you need? Many people have heard me talk about “Buy, Rent or Generate”. If it’s not adding to your business’ bottom line, why are you spending the time and energy to build up the intellectual property yourself?
Think about a control suite such as NetAdvantage for WPF from Infragistics. It’s a $995.00 price tag. At first glance that’s a lot of money but how long would it take for you to write a control such as their xamChart that does 2D and 3D charting? A month? Two? Let’s be overly optimistic and confident like all of us technical folk are and say a week. If you’re making $50,000.00 a year, you just broke even if you actually hit the ridiculously tight deadline (technically you still lost because of all of the other factors that go into the cost of supporting you as a developer but for the moment, let’s pretend that you broke even…). What about support or feature enhancements?
The question you have to answer is does building the control, component, application yourself worth more to the company than buying it? What’s the (you should have seen this coming) Return on Investment?
With any of type of components or applications, you have to look at the full cost however. Often you’ll find that the licensing cost is a small part of the overall acquisition. How long does it take to integrate with the rest of your application? How much modification does it require? How much maintenance does it require? Whether it’s free (as in beer – i.e. no licensing fees) or not, there’s a cost associated. People who charge money for their software, open source or not, are betting on the fact that the cost of acquisition is going to be lower than you building it yourself. Sometimes they’re right, sometimes they’re wrong.
More efficient tools – The more efficient you are with your tools, the less time it will take for you to get features and functionality in front of the business.
For example, if you start leveraging CodeRush and/Or Resharper on top of Visual Studio, it will save you a lot of time. Or if you are leveraging tools that cut down on the amount of time you spend reporting or anything else you are saving time.
Again, pull out the numbers. If you can prove that you’ll have a 5 minute a day savings in time by leveraging CodeRush for example you might have a good case. At $50,000.00 a year / 2080 paid hours per year = about $24 an hour and a savings of 5 minutes * work days (with two weeks of vacation) is a savings of 900 minutes or 15 hour which equals $360.00. This means that at a price tag of $249.00, the ROI is 144% with a payback of 8.3 months.
The same type exercise applies if you are looking at buying a source control management system, bug tracking system or any other set of tools such as VSTS and Team System. In fact, there’s a whole lot of case studies that lay out the ROI of VSTS and Team Suite.
Not building a feature – The cheapest feature is the one that you don’t build in the first place. The 
question is, how do you know which features you need to build in the first place? First and foremost, you need to understand the user. This is accomplished by doing user research. Next you need to, going back to my first point, do solid requirements analysis so that you know what you are supposed to build. And most importantly, you need to look at partnering with the business to build out the application. This is a core tenet all of the Agile methodologies. This means that you’re going to be building only the features that the folks on the business side want, not the features that you think that they need.
The second question here is after you build a feature, do you measure to see if the users are actually using a given feature? In other words, if you spent X amount of time and energy building out a specific set of functionality, how do you know that the business is actually realizing the potential return? It’s a simple matter of logging when a given menu item or button is clicked or code path is executed. There are tools that will automate setting all of this up. PreEmptive Solutions, for example, can build this functionality into your application without you having to write any code with their 2010 Dotfuscator application.
How much does it cost to run the software?
There’s a lot that goes into calculating the cost of running the software. It includes not only the cost of running the data center, hardware that the users use, the electricity to run the machines and the like but it also includes training to user to use the software, the user’s salary while they wait on latency issues and a whole lot more.
Again, there’s a tremendous amount of ways that you can save money in this arena.
Deploy in the cloud – The idea of utility computing is that you only pay for the service that you use just like the electric bill. In a recent conversation with a friend whose company maintains MASSIVE data centers, he said that the servers were averaging less than 15% utilization. This is nuts but smart at the same time. On the one hand, you’ve got 6-7 times the data center that you need. On the other hand, you’re ready to handle spike traffic. The issue is that you’re still paying for that unused 85%.
This is where Windows Azure and other cloud computing platforms comes in. By deploying to the cloud you are cutting the costs of the data center down to what you are utilizing rather than what you anticipate as the possible high water marks.
This comes back to the question, however, what’s more valuable to the company? Keeping the data and processing in-house or saving the cost of the data center. A key architectural decision that you need to make here is what’s business critical to keep in house and how can you architect to keep that in house while rolling other parts out to rented data centers?
Build a better UX – If done correctly, this can make you user more efficient, cut down on the training costs and reduce support costs.
A great book that talks about UX is The Inmates Are Running the Asylum: Why High Tech Products Drive Us Crazy and How to Restore the Sanity by Alan Cooper. I can recommend reading not only any book that Alan Cooper has written but also can recommend reading his blog. In this book, Alan makes the analogy that most technology is like the dancing bear at the circus. The reality is that the bear is not a great dancer but people still flock to the circus to see the bear dance because, well, it’s a dancing bear. Many (maybe even most) of our technology solutions are really not that great for the user but the fact that it does anything at all is a novelty.
In Principles Of Software Engineering Management, Tom Gilb wrote that “The rule of thumb in many usability-aware organizations is that the c
ost-benefit ratio for usability is $1:$10-$100.”
This cost benefit ratio is realized in several ways. Users will often call support to figure out how accomplish a given task. These calls can be eliminated by careful building of the user experience so that it’s “intuitive”. To figure out what’s “intuitive” to your users, you need to understand who they are and how they work. You can attract new users to your application. As mentioned before, you cut down on the number of features that you build that don’t get used and more.
Reduce the cost to maintain the software – This includes support calls, time and effort finding and fixing bugs, adding new features and the like. Dealing with legacy systems is what a very high percentage of developers do in their day to day jobs. The cost of maintenance can be mitigated on the front end in a number of ways.
There are a ton of books out there that cover this topic in great detail.
First, you have to have sound architectural principles and rigor. The larger the application is, the more important this is. If you can separate out your business logic from your data tier from your UI cleanly and absolutely, you have a much better chance of being able to do maintenance on one piece without devastating the rest of the pile of spaghetti.
Second, you have to write maintainable code. The definition of “Maintainable” varies but my go to book in this topic is Code Complete: A Practical Handbook of Software Construction. I had the great fortune to have the first edition of Code Complete. It taught me things from properly named variables to building high performance data structures. I highly recommend anything that McConnell writes.
Third, find a good way to do a given task and make it the standard. For example, if you’ve got 3 different logging mechanisms, that’s probably 2 too many. If you’re doing one off security mechanisms in every application or even different parts of the same application, you’ve got huge headaches. If you have many different mechanisms for hitting the database you need to simplify and standardize.
There are lots of other techniques that will cut down on the amount of time and effort that it takes to maintain the software. However, there is a tipping point where it makes a lot more sense to do a rewrite than it does to keep maintaining the existing code base. This tipping point depends on the function of the application, state of the code, state of the architecture, practices used to build the application in the first place, availability of tools, availability of talent who understand the technologies used to write it in the first place and the requirements for new features. Determine whether it’s going to be more cost effective, given all of those variables, to do a rewrite or to keep maintaining.
This is not an easy decision. You have to fully understand the risks. Legacy code contains often vast amounts of implicit requirements: lessons learned that were never documented anywhere but the code. They should be documented elsewhere, but reality says otherwise. This is especially common when the original developers are no longer around: the new developers, in their hubris, assume all the old code is junk and can be “easily” replaced. Along the way, they lose implicit requirements that they’ll have to relearn the hard way. Additionally, rewrites from the ground up invariably take longer than adding a single small feature. Joel Spolsky is talks about these risks in an article called Things You Should Never Do, Part I.
Even with Joel’s wise words of warning, there comes a time when it’s cost effective to do a rewrite. In 2004, Rutherford and Associates took their 15 year product and rewrite 80-90% of their existing functionality and enabled a tremendous amount of new functionality with a 6-9 month investment of time from 3 developers. They had gotten to the point where they were not able to respond in an efficient manner to new customer requirements with the existing C code base. Of course, part of that code base was their own database engine because when they started 15 years prior, there was not a database engine for mobile devices.
But again, that’s going back to the numbers and understanding the needs of the business.
What’s the return?
This is the really difficult part to measure. Most of the time there’s a way to measure how much your application costs to develop in the first place but there are many ways to measure return.
Returns come in three basic forms, money saved, time saved or new business enabled which brings in more revenue.
To really bring and measure a great return, you have to understand the current state of the business. This involves studying how the company earns money today and how they spend that money. When you understand this, you can start looking for ways to improve things.
Money Saved is not hard to explain but you can look for it in a lot of different places. One contract that I did before joining Microsoft was building out a specialized survey engine specific to the medical field to replace the paper one that they were currently using. This saved on printing costs, typists to type in the returned results into a spreadsheet but the real savings came in shipping costs. That alone paid for my contract within the first year.
Are you cutting back the costs in the data center or even cutting portions of it by rolling out to the cloud?
Are you cutting the power requirements of the client software by employing green methodologies?
Time Saved is a little more nebulous to measure but possible. Are you cutting down on the amount of time that it takes to process an order? One CTO of a call center told me that if he could save an average of 3 seconds a call, he would save 2 million a year. That’s a lot of money. Because of that level of savings, he was willing to spend a lot of time and money researching where users were spending time and what we cut. Latency issues were a killer. Near side caching, GEO location of data and many other relatively simple things shaved seconds and therefore saved time and money.
Are you automating currently manual processes? If so, how long do those tasks take and what’s that cost?
Enabling New Business is where you want to be. This is where you start to become that strategic partner to the business and get a first class seat at the table w
ith the rest of the business leadership team.
Are you increasing capacity? This has to be carefully examined. If you are increasing the capacity to take orders but not the capacity to fulfill those orders, this is wasted effort. For benefit from the increased capacity, capacity for ordering, fulfillment and marketing all have to be working in concert. This comes back to understanding the business and asking people “where are the bottlenecks?”.
Are you attracting new customers to the business? Is your new online presence ranks higher in the search engines or taping into social media to leverage the “word of mouth” marketing that happens in that arena?
Are you making it simpler for people to buy your products? I recently blogged about Jarod Spool and how he changed the text on a button and created The $300 Million Button. The quick moral of the story was that the old wording was confusing customers and driving them away. The new button brought in $300 Million dollars.
Are you enabling new lines of business? One example of enabling new business is when a wine distributor bought EO Star‘s distribution management application and realized that they could start managing and distributing many different types of foods with their wine and manage those as easily sold and packaged deals. A counter example is where one restaurant chain couldn’t offer a new type of food because they had to rewrite their menu application to handle it.
Conclusion
As technology folk, we have to understand business or we will be at the mercy of the business. The primary motivation for all businesses is making money. This is measured via ROI and payback.
In order to become a strategic partner to the business you have to prove that you are providing a great ROI for the business’s investments in you.
You can improve your projects ROI by reducing development costs, runtime costs, maintenance costs and by saving the company time, money and enabling new business.
While this measurement can be a lot of work and a little scary sometimes, it will prove to be absolutely essential because once you have that data, you will be in a great position with the business.