And these investments have paid off. Since the 1980s, the top four firms in each industry have increased their market share by 4% to 5% in most sectors. My research shows that investments in proprietary software caused most of this increase.
This greater industry dominance by top firms is accompanied by a corresponding decline in the risk that they will be disrupted, a prospect that has obsessed corporate managers ever since Clayton Christensen’s The Innovator’s Dilemma came out in 1997. At the time Christensen wrote his book, disruption was on the rise. But since about 2000—when top firms started their investment spree in proprietary systems—this trend has declined sharply. In a given industry, the chance that a high-ranking firm (as measured by sales) will drop out of one of the top four spots within four years has fallen from over 20% to around 10%. Here, too, investments by dominant firms in their internal systems largely account for the change. While some new technologies disrupt entire industries—think of what the internet did to newspapers or DVDs—others are now suppressing the disruption of dominant firms.
How does this happen, and why does it apparently affect so much of the economy? It is because these business systems address a major shortcoming of modern capitalism. Beginning in the late 19th century, innovative firms found that they could often achieve dramatic cost savings by producing at a large scale. The shift dramatically reduced consumer prices, but there was a trade-off: in order for companies to achieve those large volumes, products and services needed to be standardized. Henry Ford famously declared that car buyers could have “any color so long as it is black.” Retail chains achieved their efficiencies by providing a limited set of products to their thousands of stores. Finance companies offered standard mortgages and loans. As a result, products had limited feature sets; stores had limited selection and were slow to respond to changing demand; and many consumers could not get credit or obtained it only on terms that were costly and not suited for their needs.
Software changes the equation, partly overcoming these limitations. That’s because it reduces the costs of managing complexity. With the right data and the right organization, software allows businesses to tailor products and services to individual needs, offering greater variety or more product features. And this allows them to best rivals, dominating their markets. Walmart stores offer far greater selection than Sears or Kmart stores, and they respond faster to changing customer needs. Sears was long the king of retail; now Walmart is, and Sears is in bankruptcy. Toyota quickly produces new models when it detects new consumer trends; smaller car companies cannot afford the billions of dollars it takes to do that. Similarly, only Boeing and Airbus can manage to build highly complex new jumbo jets. The top four credit card companies have the data and the systems to effectively target offers to individual consumers, gaining maximum profit and market share; they dominate the market.
These software-enabled platforms have allowed top firms to cement their dominance. They have also slowed the growth of rivals, including innovative startups.
A variety of evidence supports the idea that startup growth has slowed down substantially. One sign is how long it takes for venture-backed startups to receive funding: from 2006 to 2020, the median age of a startup in the seed-round funding stage increased from 0.9 years to 2.5 years. The median age of a late-stage startup rose from 6.8 years to 8.1 years in that same period. Among firms that were acquired, the average time from first financing to acquisition tripled, from a little over two years in 2000 to 6.1 years in 2021. The story was similar for firms that went public. But the clearest evidence of a slowdown is what happens when firms become more productive.
Big firms are employing large-scale technologies that make it harder for startups to grow.
The key feature of dynamic economies, what economist Joseph Schumpeter called “creative destruction,” is that more productive firms—those with better products or lower costs or better business models—grow faster than less productive incumbents, eventually displacing them. But after 2000, on average, firms with a given level of productivity grew only half as fast as firms with that same level of productivity grew in the 1980s and 1990s. In other words, productivity has less effect on growth than it used to. And when productive firms grow more slowly, they are less likely to “leapfrog” industry leaders and displace them—the hallmark of disruption. Last year, research I conducted with my colleague Erich Denk directly linked the waning impact of productivity improvement to the greater industry dominance of large firms and their investments in software and other intangibles.
Another view, expressed forcefully by congressional investigators in hearings and in a staff report published in 2020, attributes the decline in economic dynamism to a different source: the weakening of government antitrust policy since the 1980s. In this account, large firms have been permitted to acquire their rivals, reducing competition. Acquisitions have made these firms more dominant, especially in Big Tech, leading to a decline both in the emergence of new tech firms and in venture capital funding for early-stage firms. But in fact, the rate at which new tech firms enter the market is down only modestly from the exceptional surge of the dot-com boom, and early-stage venture capital financing is at record levels, with twice as many financings today as in 2006 and four times the amount invested. The problem isn’t that large firms are preventing startups from entering markets or getting funding; the problem is that big firms are employing large-scale technologies that make it harder for startups to grow. Moreover, big firms like Walmart and Amazon have grown mainly by adopting superior business models, not by buying rivals. Indeed, the rate of acquisitions by dominant firms has declined since 2000.
Of course, such acquisitions do sometimes affect the startup landscape. Some researchers have identified so-called “kill zones,” where Big Tech makes acquisitions to shut down competition, and venture capital becomes hard to find. But other researchers find that startups often respond by moving their innovative activity to a different application. Moreover, the prospect of acquisition by a large firm often incentivizes people to found startups. Indeed, despite what happened to Nuance, the number of speech-recognition and natural-language-processing startups entering the market has quadrupled since 2005, and 55% of these startups have received venture capital investments.
The slowdown in the growth of innovative startups is not just a problem for a few thousand firms in the tech sector; the headwinds blowing against companies like Nuance are responsible for problems that affect the health of the entire economy. Researchers at the US Census Bureau have shown that the slower growth of productive firms accounts for much of the slowdown in growth of aggregate productivity, a figure that measures the amount of output the economy produces per person and serves as a rough index of economic well-being. My own work has also shown that it plays a role in growing economic inequality, greater social division, and the declining effectiveness of government.
What will it take to reverse the trend? Stronger antitrust enforcement might help, but the changes in economic dynamism are driven more by new technology than by mergers and acquisitions. A more basic problem is that the most important new technologies are proprietary, accessible only to a small number of huge corporations. In the past, new technologies have spread widely, either through licensing or as firms independently developed alternatives; this enabled greater competition and innovation. Government sometimes helped this process. Bell Labs developed the transistor but was forced by antitrust authorities to license the technology broadly, creating the semiconductor industry. Similarly, IBM created the modern software industry when, in response to antitrust pressure, it began to sell software separately from computer hardware.
Today we are seeing some similar developments even without government action. Amazon, for example, opened up its proprietary IT infrastructure to create the cloud industry, which has strongly improved the prospects of many small startup firms. But antitrust policy can be used to encourage or compel more large firms to open their proprietary platforms. Loosening the restrictions that noncompete agreements and intellectual-property rights place on employee mobility can also encourage a greater diffusion of technology.
Coming up with the right balance of policies will be difficult, and it will take time—we don’t want to undercut incentives to innovation. But the starting point is to recognize that in today’s economy, technology has taken on a new role. Once a force driving disruption and competition, it is now being used to suppress them.
James Bessen is a lecturer at the Boston University School of Law and the author of the upcoming book The New Goliaths: How Corporations Use Software to Dominate Industries, Kill Innovation, and Undermine Regulation, from which this essay is adapted.
