Flexiblility: How much is enough?

I went back to my master's thesis (written 13 years ago), to try to get some insights on the following question: how do market innovators and engineering architects collaborate to design a platform with the right amount of flexibility? Too little flexibility, and the investment in the system will either be obsolete and need replacement or will handcuff the innovator, preventing future opportunities. Too much flexibility, and the extra investment goes wasted -- or possibly the initial opportunity is lost because of the additional time and money needed to build in flexibility.

In my thesis, I studied the value of investing in a biotech license (which is the right to develop sell a particular family of biotech formulations, but not the formulations themselves). The license cost about $500K, which would be lost investment if the company never chose to "exercise their option" and market a biotech product that used the license (unless the value was in prventing someone else from having that option). The question was, is $500K too much to pay for the license?

The answer depended on a series of risks and how they played out. They were:

• Scientific risk - could the product be engineered? Was it feasible at all?
• Regulatory risk - could the product be legally manufactured and sold? To what markets?
• Manufacturing risk - how much would it cost to manufacturing the product? How long would it take?
• Industry risk - how much competition will there be for the opportunity we are trying to seize? Will only a few entrants duke it out, or will there be a lot of players
• Market risk - how much demand will there be for the product? What price can we get for it?

I look at software innovation, and the risks are similar, but their significance and costs are sometimes different. For example:

• Scientific risk - This is pretty much the same. People who are experts in either field will be able to tell you quickly whether there's any risk here at all. And, if there is, it means an R&D effort just to play in the game.
• Regulatory risk - In biotech and pharma, the costly clinical trials process process (4-5 phases) determines this risk. In software, the risks usually are answerable by lawyers: patent infringement and patentability, privacy and other information legality concerns, etc.
• Manufacturing risk - In software, this is where the game is usually won or lost. Historically, more than 90% of projects go over budget by as much as 50% (this is a stat from 10 years ago; things may be a little better now, but probably not much, based on my observations). Embedded in the manufacturing risk for software, I believe, is a much more complicated set of issues regarding architectures, which are probably resolved much sooner in the process when dealing with biotech / pharma.
• Industry risk - This is another area where understanding the possible scenarios and how they will play out has much different effects in software. For example, if everyone is rushing to build something, that may create a new opportunity in itself (sell shovels to gold diggers). Another possibility is to collaborate on standards or open source, allowing for lower overall development costs across the board and competition on a basis other than exclusivity (which, by the nature of pharma/biotech today, is somewhat inherent in the product).
• Market risk - This risk is consistent between biotech/pharma and software, though it looks very different. Both ultimately worry about things like: who will pay and how much? Will someone subsidize payment (insurance for pharma, advertising for software)? Both worry about different abilities to pay in different regions of the world, and how to price discriminate.

In software, the market innovator really holds the cards on industry and market risks; engineers know best on scientific and manufacturing risks. It depends on regulatory risk. But, in software, all of these risks are best resolved in the product conceptualization process. In that process, one can get a handle on feasibility issues, regulatory/privacy issues, degree of difficulty and cost to build as well as uniqueness in the industry and market opportunities.

To answer "how much is enough" flexibility, one needs to put together a product conceptualization process that facilitates the kind of dialogue that clarifies and resolves a lot of these issues and risks. Once a software product is conceived, it can be tested against all of these risk areas, and engineers can better understand what options the market innovators will want to exercise through flexibility over time.

The "over time" part is important, too. Usually, all flexibility is not needed immediately. The market risk needs to resolve -- who is going to use this product and how -- before one feels comfortable investing more. So the architecture of the product system is only one part of the design. The other is the architecture of the product roadmap. The clever sequencing of the roadmap can allow questions of risk to be resolved over time, reducing the number of options one needs to worry about investing in. The product roadmap and the product system together define how the product can evolve over time. I'm sure Darwin has something to say about what kinds of products will withstand evolutionary processes better than others...