QUESTION: How to Measure Innovation

[b] Once a company has taken the initiative to embark into the use of open innovation which indicators can a firm apply to measure progress on open innovation?

I think innovation, open or not, is measured the same way. There are three components that determine the wealth of a company: the human capital, machinery capital and technical progress. If the company gets richer even if the first two components remain unchanged, it has to be a growth from the third component.

As an important reason to adopt open innovation is to benefit financially from a more "collaborative" approach, I think traditional indicators such as number patents/R&D budget, or R&D intensity (R&D budget/sales) remain appropriate for inside out and alliances.

Yeah, but this works for "closed" companies which use their own patents ideas know how etc. what happens when outsider, using our data, technology, know how and so on creates an idea/technology which is better or more competitive than ours? Is such a situation possible at all? Do we have the right to it? How does it work?

One way is to look at the return on invested capital. Eventhough, by using open innovation, other companies get access to the innovation the cost advantage of, for example, letting another company to develop a former internal project further, might be bigger than developing it internally. By calculating the value of these open innovations and benchmarking the company against rest of the market is a good way to see if the decision of using open innovation was successful.

If i'm not mistaken isn't the point of the having open innovation to allow other companies to work from the idea we started and if things were t0 develop to a better and more unique product or service, the whole product or service was first derived from our idea. The rights to the new service or product should be discussed in the contract of agreement.

However, going back to the measurement of open innovation, there is no exact answer, however, i believe we can measure by the time and results between previous methods and open innovation methods. Also the percentage of sales that comes from external licensed technology. These are just two of some suggestive ways of measuring open innovation.

I agree with uroi.salii, there has to be always a contract or agreement between the company using an open innovation moves and the one benefiting from it, and this is surely the case. What would be the advantage to do open inovation if there wasn't?
The first company should either be able and allowed to use all new technologies created from / thanks to its tehcnology, and/or should be rewarded with a percentage of sales on those products. It is one of the attracting point of using open innovation.

Then, for the measure of innovation, i think it is hard to really measure it, but we know when there is some. I mean, we can't quantify innovation, but we can know when a firm is good at innovatin, because it is in advance on its competitors, or is the only one using some technologies. It is also hard i think to measure it.

I agree with you florent, it is hard to measure it since there is not a standard way of measuring the different methods of open innovtion. However, Im sure that the progress of inside out innovation can show by the changes in figure.

which indicators can a firm apply to measure progress on open innovation? I think that we can use the time to market, the new product sales and the return on investement in innovation to measure the open innovation. We can also used a ratio like: the revenue achieved / revenue forecating, just an idea.

How to measure innovation?

I guess there is a few methods that can be applied, but of course non of them is the ultimate measure. Regarding the balance sheet, you could us goodwill as an indicator, which of course includes more than just innovation, but is still a good indicator of your value created. So just take total assets - tangible assets and you get your goodwill. More concrete, there could be a numbers (in relative terms ) such as:

Revenue in licensing fees
Share price (if listed)
ROI
Number of patents

Of course, they all just account for a small part, since I agree with you guys that innovation is hard to quantify. In the long term, however, a company could check the progress of these (and/or different) measures within a Balanced Scorecard framework.

I think the above measurements are quite interesting, but measuring the gain in value of
using open innovation is hard. It is not like you can put an exact value to it,
but one should rather think of it as a strategic move for gaining a competitive
advantage. It is not often you will hear the question: What is the value of
your competitive advantage?” Though, it surely
will take your company in a different direction if you are not already applying
Open Innovation.


I would like to add, if you choose to benchmark against previous performance, I think
it is important to add a natural growth factor, or else it is easy to
overestimate effect of Open Innovation. Innovative companies often grow very
fast with or without the use of Open Innovation. I would perhaps rather say
that you should benchmark against your potential.


Keeping track of income from license fees and royalties is also a good and probably one
of the best quantitative measures, but they are backward looking. If you want
to nurse your IP portfolio you will also need to look forward, and that is the
hard part. Valuation of early-stage technology is not easy, as it requires
looking into the future. One could start out with a qualitative approach and
move gradually towards a quantitative approach as more market and product information
is revealed through time.


/Anders

Well said Anders. I agree that it would be difficult to monitor the progress given that innovative companies are growing quickly regardless whether they use Open Innovation. In the long run, quantitative analysis may help to measure progress.

However, in the short run, what qualitative analysis can help analyze the value of Open innovation?

Brodie wrote:

However, in the short run, what qualitative analysis can help analyze the value of Open innovation?

It is a very tough question. I think the best way is comparing with other companies. But this kind of datas are often private.

My other point is : is it trully useful to analyze the value of Open innovation in the short run ? It demands a huge amount of time to be 100% effective and I'm not sure a rushed analyzis would be accurate. But this is only my opinion

I agree that it is fairly hard to analyze the value of Open innovation in the short run. But, in a recession, when companies are short of resources, perhaps those who taking a open innovation move should watch out the changes of competitive position among their investors, allies, partners and suppliers carefully. cooperation/partnerships should be parallel to competition.
Another straight foward qualitative analysis is the market. Whether or not the new initiative appeals to the customers?

With regards to short run models you can apply. I can think of a few, but there a
many out there, each with it benefits and limitations.


Scoring Models could be one type. Setting up parameters that you and your competitors
benchmark against, and see how you are positioned according to them. Even a
simple BCG-matrix could be helpful. It is important to create a dynamic model
that can be adjusted over time, as parameters changes during the product /
portfolio lifecycle


Balanced Scorecards as mentioned by Alexandra are also good tools to quantify intangible
goals. They often serve as a good “translators” moving from a qualitative to
quantitative approach.


Sensitivity Analysis could also be a good tool, but is more related to risk management. However
it can provide useful insight of where to concentrate efforts.


The difficult part is often related to estimating the market potential. Real Option Analysis could be a good tool, but has many practical complications, as combining multiple types of options is often
difficult.


It could be interesting to hear if some of you have experience in forecasting sales or
profits for early-stage technologies or companies, and which methodologies you
have used? Perhaps even companies working with E-commerce?


/Anders

Anders_Schmidt wrote:With regards to short run models you can apply. I can think of a few, but there a
many out there, each with it benefits and limitations.


Scoring Models could be one type. Setting up parameters that you and your competitors
benchmark against, and see how you are positioned according to them. Even a
simple BCG-matrix could be helpful. It is important to create a dynamic model
that can be adjusted over time, as parameters changes during the product /
portfolio lifecycle


Balanced Scorecards as mentioned by Alexandra are also good tools to quantify intangible
goals. They often serve as a good “translators” moving from a qualitative to
quantitative approach.


Sensitivity Analysis could also be a good tool, but is more related to risk management. However
it can provide useful insight of where to concentrate efforts.


The difficult part is often related to estimating the market potential. Real Option Analysis could be a good tool, but has many practical complications, as combining multiple types of options is often
difficult.


It could be interesting to hear if some of you have experience in forecasting sales or
profits for early-stage technologies or companies, and which methodologies you
have used? Perhaps even companies working with E-commerce?


/Anders

Think of when you have a problem which requires a creative solution. You can never solve a problem if you remain at the same level of conscience you were when you created it; innovation is always key!. In my experience with case solving problems my boss always weighted our ideas on a two axis graph: x> How well it fits the problem (quality) y> how unique and different is the idea (innovation). The idea which scored higher on goth always won, and was assigned weights in a sensitivity analysis to compare with the others depending on the different situations.

Hi JesusMiguel, thanks for sharing. I like the graph of weighting ideas a lot.

I second shirley! great great!

I think an important factor is to evaluate the expected lifecycle of the new innovation. Once again it also depends on the industry. E.g. selling new prodcuts in the PC market gives you a very short timeframe, because the innovation cycles are generally short. Try to estimate how long your innovation will be a valuable asset in the market. If it keeps selling beyond the industry's usual cycles, it must be considered a success.