Academic Study Says Open Source Has Peaked: But Why?

from the share,-and-share-alike dept

Open source runs the world. That's for supercomputers, where Linux powers all of the top 500 machines in the world, for smartphones, where Android has a global market share of around 75%, and for everything in between, as Wired points out:

When you stream the latest Netflix show, you fire up servers on Amazon Web Services, most of which run on Linux. When an F-16 fighter takes off, three Kubernetes clusters run to keep the jet's software running. When you visit a website, any website, chances are it's run on Node.js. These foundational technologies -- Linux, Kubernetes, Node.js -- and many others that silently permeate our lives have one thing in common: open source.

Ubiquity can engender complacency: because open source is indispensable for modern digital life, the assumption is that it will always be there, always supported, always developed. That makes new research looking at what the longer-term trends are in open source development welcome. It builds on work carried out by three earlier studies, in 2003, 2007 and 2007, but using a much larger data set:

This study replicates the measurements of project-specific quantities suggested by the three prior studies (lines of code, lifecycle state), but also reproduce the measurements by new measurands (contributors, commits) on an enlarged and updated data set of 180 million commits contributed to more than 224,000 open source projects in the last 25 years. In this way, we evaluate existing growth models and help to mature the knowledge of open source by addressing both internal and external validity.

The new work uses data from Open Hub, which enables the researchers to collect commit information across different source code hosts like GitHub, Gitlab, BitBucket, and SourceForge. Some impressive figures emerge. For example, at the end of 2018, open source projects contained 17,586,490,655 lines of code, made up of 14,588,351,457 lines of source code and 2,998,139,198 lines of comments. In the last 25 years, 224,342 open source projects received 180,937,525 commits in total. Not bad for what began as a ragtag bunch of coders sharing stuff for fun. But there are also some more troubling results. The researchers found that most open source projects are inactive, and that most inactive projects never receive a contribution again.

Looking at the longer-term trends, an initial, transient exponential growth was found until 2009 for commits and contributors, until 2011 for the number of available projects, and until 2013 for available lines of code. Thereafter, all those metrics reached a plateau, or declined. In one sense, that's hardly a surprise. In the real world, exponential growth has to stop at some point. The real question is whether open source has peaked purely because it has reached its natural limits, or whether they are other problems that could have been avoided.

For example, a widespread concern in the open source community is that companies may have deployed free code in their products with great enthusiasm, but they have worried less about giving back and supporting all the people who write it. Such an approach may work in the short term, but ultimately destroys the software commons they depend on. That's just as foolish as over-exploiting the environmental commons with no thought for long-term sustainability. As the Wired article mentioned above points out, it's not just bad for companies and the digital ecosystem, it's bad for the US too. In the context of the current trade war with China, "the core values of open source -- transparency, openness, and collaboration -- play to America's strengths". The new research might be an indication that the open source community, which has selflessly given so much for decades, is showing signs of altruism fatigue. Now would be a good time for companies to start giving back by supporting open source projects to a much greater degree than they have so far.

Follow me @glynmoody on Twitter, Diaspora, or Mastodon.

Hide this

Thank you for reading this Techdirt post. With so many things competing for everyone’s attention these days, we really appreciate you giving us your time. We work hard every day to put quality content out there for our community.

Techdirt is one of the few remaining truly independent media outlets. We do not have a giant corporation behind us, and we rely heavily on our community to support us, in an age when advertisers are increasingly uninterested in sponsoring small, independent sites — especially a site like ours that is unwilling to pull punches in its reporting and analysis.

While other websites have resorted to paywalls, registration requirements, and increasingly annoying/intrusive advertising, we have always kept Techdirt open and available to anyone. But in order to continue doing so, we need your support. We offer a variety of ways for our readers to support us, from direct donations to special subscriptions and cool merchandise — and every little bit helps. Thank you.

–The Techdirt Team

Filed Under: open source, software development

Reader Comments

Subscribe: RSS

View by: Time | Thread

  1. identicon
    SirWired, 3 Sep 2020 @ 5:49am

    Hard to get higher than the top of the mountain.

    At this point, there are feature-complete Open Source products that cover pretty much the entire breadth of IT, with the exception (and this is a pretty big exception) of end-user applications beyond the Web browser. In many areas (as the article pointed out), the Open Source product is the dominant one.

    There will always be new projects, but all the catch-up work is pretty much over.

    Except for end-user apps, where the catch-up work has been largely abandoned. I guess it's not interesting enough for the people that would be good at it to do in their spare time, and little incentive for corporations to sponsor enough professionals to make products that are superior to their commercial counterparts. Yes, you can get useful work accomplished with, say, OpenOffice, or desktop Linux, but few could argue that they are actually better (in a getting $hit done sense) for end-user tasks than the commercial equivalents. (In my circle of friends, all of which are IT professionals, half of them veteran sysadmins (so no lack of knowledge here), precisely zero of us run Linux on the desktop; it's all Mac, Windows, and Chromebooks.)

Add Your Comment

Have a Techdirt Account? Sign in now. Want one? Register here

Subscribe to the Techdirt Daily newsletter

Comment Options:

  • Use markdown. Use plain text.
  • Remember name/email/url (set a cookie)

Follow Techdirt
Insider Shop - Show Your Support!

Essential Reading
Techdirt Insider Chat
Recent Stories

This site, like most other sites on the web, uses cookies. For more information, see our privacy policy. Got it

Email This

This feature is only available to registered users. Register or sign in to use it.