Gauging a Firm’s Innovative Performance Using an Integrated Structural Index for Patents

Hirsch (2005) proposed the h-index as an author-level indicator combining productivity (published articles) and impact (received citations). Soon his idea was applied to other source-items relations such as journal publications and citations (Braun, Glänzel, & Schubert, 2005), a company’s patent assignments and their citations in other patents (Guan & Gao, 2009), publications and citations of topics, restricted to recent years (Banks, 2006) or availability of books and their loans according to a library classification (Liu & Rousseau, 2009). We first recall the basic mechanism for calculating the h-index of an actor (author, company, or a journal). One considers a two-dimensional table of sources and items, where sources, e.g. publications or patents, are ranked according to items, e.g. received citations. Sources with the same number of items are given different rankings, but the exact order does not matter. Then actor A’s h-index is equal to the number h if the first h sources have each at least h items, while the source ranked h+1 has strictly less than h+1 items.

The relation between the breadth and depth of its patents on the one hand, and the health of a firm on the other, has been studied for several decades (Denicolò, 1996; O’Donoghue, Scotchmer, & Thisse, 1998; Palokangas, 2011; Prencipe, 2000; Wang & von Tunzelmann, 2000). Yet, no final answer about the optimal breadth and depth of patents has been found (Ozman, 2007; Zhang, Chen, & Niu, 2012; Lodh & Battaggion, 2014; Breschi, Lissoni, & Malerba, 2003). When using diversity indexes to measure the technological breadth and depth of a firm, it may happen that results are biased downwards for small and medium-sized firms for which the scale of technological activities is small (Chen, Jang, & Wen, 2010; Hu & Rousseau, 2015; Miller, 2006; Palokangas, 2011). Moreover, diversity indices such as the Rao-Stirling index may show cyclical patterns that are not related to a company’s profits but are rather related to the number of inventors (Leydesdorff, 2015). This suggests that if one wants to understand the optimal breadth and depth of patents, an approach different from the “complexity and diversity” might be worth investigating (Lodh & Battaggion, 2014; Wang & von Tunzelmann, 2000).

Traditionally, the breadth and depth of patents of a firm and their citations are considered separately. This approach, however, does not provide an integrated insight in the major characteristics of a firm’s patents. It has been observed that return on investment of a patent depends largely on a firm’s market value and its technological value, while the competitive strength of a firm’s patents bears a close relation to market value, technological value, social value of patents, and healthy management styles (Guan & Gao, 2009; Hu & Rousseau, 2015; Lee, 2009; Palokangas, 2011).

We claim that one of the most important elements affecting the potential applications of a patent is its breadth, operationalized by codes, such as the IPC, the U.S. Patent Classification System (USPC), Cooperative Patent Classification (CPC) or the European Patent Office (EPO) codes assigned to it. This set of codes forms a basic aspect to grant its owner either a very limited right to exclusive use or a more general right covering a variety of different realizations of the invention (Reitzig, 2003). This fact implies that patents can differ with respect to the degree of protection afforded to an invention (Gilbert & Shapiro, 1990; Klemperer, 1990). In this context we note that accrediting codes to a patent is an arena in which patent examiners exercise wide discretion. In general, the broader the patent, the higher the chance to be applied in different practical fields and the larger the potential profits to the firm or a purchaser of the firm’s patent (Palokangas, 2011). This leads to the claim that the optimal breadth of patents should focus on a firm’s performance. Excessively broad patent claims increase the patentees’ non-market related risks from rivals and provide them with little flexibility to face unexpected situations (Merges & Nelson, 1990). However, the narrower a patent’s claims, the more the patentee may be victim of imitation as very similar products may lie outside the original patent’s claims (Denicolò, 1996; Kanniainen & Stenbacka, 2000).

A firm which focuses on excessively broad patents would overspend its research and development (R&D) capital by developing or buying an overly large number of patents. And, vice versa, if most of the firm’s patents are of narrow breadth, the firm reduces its chance to earn larger profits than competitors. Obviously, these two extreme cases do not lead to healthy management styles in a competitive industry. Therefore, it is very important to measure the competitive strength of patents and hence the “weight” of a firm’s patent portfolio. Such an investigation must include the number of patents, their impact and their breadth.

It is well known that the received number of patent citations is an important indicator to measure the influence of a patent. Moreover, patent citations have a positive relation with the profits of the patent owner (Hu, Rousseau, & Chen, 2012; Trajtenberg, 1990).

Many investigations point out that, compared to the breadth of a patent (the primary dimension), it is less meaningful to focus on the depth of a patent because the determination of a patent’s depth is just approximate and no positive relation between a patent’s performance and its depth has been found (Gilbert & Shapiro, 1990; Kanniainen & Stenbacka, 2000; Klemperer, 1990; Lodh & Battaggion, 2014; Ozman, 2007; Palokangas, 2011; Reitzig, 2003; Zhang, Chen, & Niu, 2012).

Grönqvist (2009) argues that broader patents are not necessarily more valuable than narrower ones. Concretely, patents described with many codes do not necessarily lead to a larger profit for the firm. Therefore, neither the breath of patents nor the number of received citations on their own are clear-cut indicators for the value of a company’s patent portfolio. If we want to understand the competitive strength of a firm from the perspective of patent performance, the primary structure of patents (patent breadth), the secondary structure (patent depth), and their influence should jointly be taken into account in a multi-layered approach (Denicolò, 1996; Hu, Rousseau, & Chen, 2012; Palokangas, 2011). Abstractly, their relationships can be described with Equation (1): SP=f(p,bp,dp,cp), $$ SP = f (p,bp,dp,cp), $$ (1) where SP denotes the competitive strength of patent-related performance of a firm, and p is the number of patents; bp denotes their breadth, dp their depth and cp the number of received citations.

To reveal the relation between the essential structure of patents and their competitive strength, e.g. profit performance, in the real world, and clarify the controversy on the influence of depth on a patent’s profit, we propose two types of structural h-indices for patents: (1) the structural h-index, a primary one, combining the number of patents with the primary structure (breadth of patent) and with forward, i.e. received, citations; (2) the synthetic structural h-index, using the number of patents, the breadth and depth of these patents, and the number of forward citations.

Hence, we hypothesize that the primary structure of patents (patent breadth) and their influence on a firm can be measured by a structural h-type index, combining different aspects in a dynamic way.

A firm’s innovation activities are operationalized as the number of patents, while their technological breadth is operationalized by the number of 3- or 4-digit IPC codes. Consider a set of patents granted to a firm in a certain year Y, ranked in decreasing order of the number of 3- or 4-digit IPC codes assigned to them. Then the IPC h-index of this firm in the year Y is equal to q if q is the highest rank such that the first q patents are assigned to at least q IPC codes (Hu & Rousseau, 2015). The resulting indicator is denoted as IPCh₃ or IPCh₄ depending on the number of digits that have been used.

Next, we define a yearly h-index slightly modified from the original meaning of Hirsch (2005) to map a firm’s innovation activities and influence in the year Y. The yearly h-index of a firm in the year Y, denoted as h_Y, is equal to h if h is the largest rank such that the first h patents receive each at least h citations within a given citation window. In the examples investigated below the citation windows always end on May 20, 2014.

Next, we define the yearly h-index of patent depth in the year Y, denoted as Dh_Y as follows. Consider the set of patents granted to a firm in the year Y, ranked in decreasing order of their technological depth index, d(p). The Dh_Y index of this firm in the year Y is equal to k if k is the highest rank such that the first k patents have at least a technological depth equal to k.

We define the structural h-index for patents granted in the year Y, denoted as S_h(Y), as a combination, actually a multiplication, of the IPCh and the yearly patent h-index. Hence S_h(Y) can be calculated with Equation (2): Sh(Y)=IPChs×<!-- ? -->hY, $$ S_h(Y) = IPCh_s \times h_Y, $$ (2)

where s = 3 or 4. Moreover, although not indicated in the notation, S_h(Y) is time dependent, i.e. depends on the citation window, which in our examples ends on May 20, 2014. The S_h(Y) sequence shows a firm’s innovation activities and their technological breadth, as well as the influence of patents (by citations) in each year. As such we claim that it can be used to gauge the “primary weight” of a firm’s patents. This claim is investigated in the next section.

Finally, we define the synthetic structural h-index for patents granted in the year Y, denoted as SS_h(Y), as a summary indicator constructed from the IPCh, the yearly patent h-index (h_y), and the patent depth yearly h-index and it can be calculated with Equation (3): SSh(Y)=w1IPChs+w2hy+w3Dhy, $$ SS_h(Y) = w_1IPCh_s + w_2h_y + w_3Dh_y , $$ (3)

where w₁, w₂ and w₃ are positive weights such that w₁ + w₂ + w₃ = 1.

The general range of firms acceptable for our purposes contains those pharmaceutical companies listed in Fortune 500 2006–2010 issued by the CNNMoney website

http://money.cnn.com/magazines/fortune/fortune500/

As there are many invisible factors affecting the performance of patents, we try to control for external variables by considering the following criteria for inclusion in our case study.

Firm location: Different countries have different regulations for patents which may influence realized profits (Chen, Shih, & Chang, 2013). For this reason only US companies were selected.

Taking all these requirements into account resulted in eight US-based multinational pharmaceutical companies meeting all the criteria, namely Johnson & Johnson, Pfizer, Merck, Bristol-Myers Squibb, Amgen, Genzyme, Allergan and Biogen Idec.

We extracted from the Derwent Innovations Index (DII) all patents granted to these eight companies during the period 1990 – 2005. For each record we downloaded all fields, including IPC-codes and citations received (so-called forward citations). Data were extracted on 20/05/2014. This led to a total of 19,080 patents for the eight firms. Next, we collected the yearly profits for each company as reported by Fortune 500 2006–2010.

For the dataset of a company’s patents, we first counted the number of 4-digit IPC codes for each record via a simple program written by ourselves, and determined the yearly IPCh and yearly h-index during 1990–2005 for each company (Appendix Tables A1–A3). Then, we calculated the yearly S_h(Y) and yearly SS_h(Y) for each firm according to Equations (2) and (3). As the breadth of a patent is a primary structure while its depth is a secondary one, and because research suggests that both breadth and number of citations have positive relations with the profits of the patent owner, we take all these factors into account. Moreover, as previous research pointed out that 4-digit codes and citation-weighted counts can be taken as “patent-equivalents” (Miller, 2006), we – tentatively – weighted them higher than Dh_Y according to a weight of 0.4 for IPCh and for h_Y, and a weight of 0.2 for Dh_y in Equation (3) (Appendix Table A4).

To compare results based on 3-digit IPC codes with those based on 4-digit codes, we also collected the number of 3-digit codes for each patent (Appendix Tables A2 and A3), and calculated the corresponding Sh index.

To observe the relationship between the S_h(Y) and a firm’s profits, we use two different statistical methods:

We calculated the Spearman rank correlation coefficient between the eight companies, mean S_h(Y) and mean SS_h(Y) values and total profits over the period from 2006 to 2010.

To apply the NCC method, the eight companies are grouped according to their profits: Group H (high profit) consists of the four companies with the highest profit; Group L (low profit) consists of the four companies with the lowest profits. For each group, we re-rank companies by their profits in a descending way and denote them GHR1, GHR2, GHR3, GHR4, GLR1, GLR2, GLR3, and GLR4 (Table 1). In this way, case-control is performed between four control-pairs of companies with the same rank order in the respective groups (such as GHR1 vs GLR1), and the nested control is designed by a sequence of time points, that is, yearly S_h and yearly SS_h among controlled cases between two groups during the period 1990–2005. Hence, 16 time points in total are used as observations. We recall that the S_h(Y) indicator is time dependent. For example, in our case, the S_h(Y) of the year 1990 has a citation window from the year 1990 to May 20, 2014, and the S_h(Y) in the year 1991 has a citation window from the year 1991 to May 20, 2014, and so on. As pointed out above, such a stratified sampling of controls can lead to an efficient result.

Table 1

Controlled cases design for companies included in NCC study.

Then, we compare the yearly S_h and yearly SS_h for each company during the period 1990–2005 between two groups using a Paired Samples Test, where pairs consist of a company from GH and a corresponding company from GL, as a so-called ‘control.’

In this section, we present the results obtained from our analysis of the 19,080 patents. We will show that the two types of structural h-indices S_h(Y) and SS_h(Y) have significant correlations with a firm’s profits as given by Fortune 500 2006–2010. Moreover, the S_h(Y) index has more significance than SS_h(Y).

4.4.4

Differences of Yearly S_h and SS_h Indices of Firms between Two Different Profit Groups

Tables 6 and 7 present the results of a longitudinal observation combined with a nested case-control design. Obviously, the yearly S_h and SS_h indices of firms in Group HP are much bigger than those in Group LP during the period 1990–2005; these differences are significant. We note that the same statistical significances of paired differences are valid for results of S_h indices as well as for SS_h indices.

Table 6

Results of paired differences tests of firms’ yearly S_h between Group H and Group L (based on IPCh₄).

	Paired differences		95% confidence interval of the difference
Pairs-Sh	Mean	Std. Deviation	Lower	Upper	t-value	df	Sig. (2-tailed)
GHR1 - GLR1	144.250	45.918	119.782	168.718	12.566	15	0.000
GHR2 - GLR2	126.938	34.555	108.524	145.351	14.694	15	0.000
GHR3 - GLR3	158.000	46.286	133.336	182.664	13.654	15	0.000
GHR4 - GLP4	105.688	26.630	91.497	119.878	15.875	15	0.000

Table 7

Results of paired differences tests of firms’ yearly SS_h between Group H and Group L (based on IPCh₄).

	Paired differences		95% confidence interval of the difference
Pairs-SSh	Mean	Std. Deviation	Lower	Upper	t-value	df	Sig. (2-tailed)
GHR1 - GLR1	8.188	2.331	6.94560	9.429	14.052	15	0.000
GHR2 - GLR2	7.422	2.700	5.98329	8.861	10.996	15	0.000
GHR3 - GLR3	6.975	2.213	5.79602	8.154	12.610	15	0.000
GHR4 - GLR4	6.963	1.570	6.12586	7.799	17.738	15	0.000

Figure 1 shows average profit values as a function of average S_h(Y) values (using IPCh₄). As the Pearson correlation R is about 0.83, the rank correlation of Table 2 as well as the results shown in Table 5 are logical consequences of this relation. Note that, although this figure consists of just eight points, each of them is the result of thousands of values.

Figure 1

Compared to the case of IPCh₃, S_h based on IPCh₄ can better indicate a firm’s innovative activities, measured through patents, as well as their technological breadth, and map the potential market value of patents. Instead of the yearly h-indices which may represent a firm’s innovation activities and their influence, the S_h(Y) index, proposed in our investigation, can reflect a firm’s innovation activities, its technological breadth, and its influence in an integrated way. As such the new index reflects the primary structure of a firm’s patents and their influence and is an indicator for the “weight” related to primary competitive strength of a firm’s patent portfolio (with significant correlation to a firm’s profits).

Although SS_h(Y) is a comprehensive indicator for the “weight” of the essential, competitive strength of a firm’s patent portfolio (including the depth of patents), and although the relation between SS_h(Y) and a firm’s profits is also significant, it does not have the same “strong” correlation as the S_h(Y) index does, which suggests that the breadth of a patent is the primary structure affecting a patent performance. The depth of a patent plays a smaller role in a firm’s profit. The Spearman rank correlation coefficient between the yearly Dh and a firm’s profits is 0.690 (p = 0.058), while this correlation between the yearly average depth of patents and a firm’s profits is -0.024, and hence is not significant (Appendix Tables A5 and A7).

The first h items in a firm’s patent list, known as its h-core, reflect market value and technological value. These core patents are closely related to the competitive strength of a company. Although there are multiple dimensions involved in the innovative performance of a firm, the core competitive strength of a company is highly dependent on the performance of patents (Hagedoorn & Cloodt, 2003), one aspect being that patents are transferable, so that the patent assignee benefits in monetary terms from their purchase (Lee, 2009; Palokangas, 2011).

Our work further leads to the suggestion to different sized firms to include policymaking on technological innovation in its management. This is because there is always a limited R&D capital in a company. Indeed, we also found out that the Spearman correlation coefficient between the yearly average number of 4-digit codes of patents and a firm’s profits is even negative (namely –0.310, Appendix Tables A6 and A7), suggesting that a firm’s profits are highly dependent on the first h items of a firm’s patents rather than the “average patent” (Palokangas, 2011; Reitzig, 2003). The fact that a small group of patents essentially determines the competitive strength of a company is yet another example of the law of the vital few, also known as the 80–20 rule. In this sense, we claim that the structural h-index proposed in this study will be beneficial for modelling an optimal patent system.

Patent evaluation is a complicated issue which requires taking a full picture from different perspectives. This preliminary study proposes a new and simple indicator for gauging a company’s patent portfolio. Positive results are backed by evidence based on a large dataset from the pharmaceutical industry. Of course, we are aware that this is just a case study and, moreover, that any R&D indicator is at best PAC, as put forward in the case of citation indicators by Rousseau (2016). We are convinced though that the structural h-index is a useful addition to the field of patentometrics.