Many of today’s hottest new enterprise technologies are centered around free, “open-source” technology. As a result, many big companies—from financial giants to retailers to services firms—are building their businesses around new, community-based technology that represents a sea change from the IT practices of the past.
But how can corporate customers—and investors—evaluate all these new open-source offerings? How can they tell which projects (often strangely named—like Ansible, Vagrant or Gradle) are generating the most customer traction? Which ones have the biggest followings among software developers, and the most potential to capture market share? These questions are especially tough to answer because most open-source companies are still private, and don’t have to disclose key user and financial metrics. (Though that’s changing–just last week, open-source giant Cloudera finally announced plans to go public, increasing the market’s focus on open-source technology.)
That’s why we decided to create a new, detailed index to track popular open-source software projects, and gain some insights into the new companies powered by these technologies. It is the Battery Open-Source Software Index (BOSS Index), which we’ve spent months putting together with publicly available information and are introducing here. We hope to update it quarterly—and the index should gain more relevance as more open-source companies using some of these projects grow and go public.
The index contains 40 open-source projects, gleaned from an initial scouring of projects listed on the GitHub source-code repository site, as well as Datamation, an enterprise-IT publication that also tracks open-source projects. The top 25 are listed below and the full list can be found on our website. We focused on projects in enterprise-IT related areas like IT operations, including technology powering operating and provisioning systems; data and analytics, including tools for artificial intelligence and machine learning as well as databases; and DevOps, which includes projects focused on the hot new trend of “containers”, which help people develop software quickly in a sort of self-contained environment.
Companies ranked according to four factors. Overall project rating represents the geometric mean of two of the four individual scores, which reflect online discussion activity; search activity; jobs impact; and GitHub activity. All data is as of February 9, 2017.
*Denotes a Battery portfolio company. For a full list of all Battery investments and exits, please click here.
There are some very well-known names on the list, including projects that have spawned big companies. They include Linux, which underlies Red Hat; MySQL, which powers the company of the same name and was bought by Sun Microsystems (now part of Oracle) for $1 billion in 2008; and Hadoop, which brought us Cloudera and Hortonworks. But some more-obscure names, like Selenium, also rank highly, indicating there is plenty of grassroots innovation happening in the open-source sector—and many new projects out there that are spawning valuable companies. Still, our research also found that having lots of users for your open-source project does not automatically translate into creating a commercially viable company.
We ranked the projects according to four factors, which included:
- Public interest in the project, as measured by Google search activity;
- User activity, gauged by mentions of the projects on the popular tech-discussion board Stack Overflow;
- Jobs impact, measured by the number of job postings citing each open-source project listed on the job boards Indeed and Simply Hired; and
- Impact in the open-source community, tracked by measuring a project’s influence on GitHub. Specifically we tracked the number of “forks”, or extensions, built on each project; the number of GitHub “stars” a company received, indicating its popularity; and the number of “watches”, another popularity indicator, all as of Feb. 9.
Since some projects may have done extremely well, or poorly, on certain criteria—perhaps one had an off-the-charts Google search number, but a sub-par job-postings score–we threw out the top and bottom individual-criteria scores for each project. This is a methodology called “trimmed mean”, and it’s similar to what happens in Olympic gymnastics, where officials throw out an athlete’s highest and lowest scores from each judge and average the remaining ones. (We had no East German judges in our competition, but are being careful.)
Even so, there’s always room for improvement. Some adoption and popularity criteria, such as download metrics, are obviously somewhat difficult to measure, and surely we haven’t captured all the hottest, new emerging tools. Though with our planned quarterly updates, we should be able to track new leaders as they emerge. So we’d love feedback from the community to improve our numbers, and the index, over time. Please let us know at email@example.com if you have further insights into any of these metrics.
Some other key takeaways from our research:
Linux, Git, MySQL lead the pack
Perhaps not surprisingly, the open-source project leading our index is Linux, technology that was first released in 1991 and is one of the most widely adopted open-source projects in the world. It is commercialized by companies including Red Hat, one of the few open-source companies to trade publicly, as well as Ubuntu and SUSE.
Git, which came in at #2 on the list, inspired GitHub and GitLab. The wildly popular open-source project serves as a “version-control system” for tracking changes and coordinating work between software developers.
Also high on the list, at #3, was MySQL, the database technology first developed in 1995. MySQL currently helps run huge, Web-scale companies like Google, Facebook and Twitter. But it’s also worth noting that several “NoSQL” database technologies—which are non-relational databases, unlike MySQL, and are often better suited for parsing the unstructured data being thrown off by many companies today—also ranked highly.
These NoSQL technologies include MongoDB, which came in at #9 in our index; Redis, which is being commercialized by company Redis Labs, at #12; Cassandra, which came in at #25 and is behind the database company DataStax; and Elasticsearch at #7, which is being commercialized by Elastic. MongoDB raised a new round of financing in late 2015—the company is estimated to be valued at around $1.5 billion–and now competes against established database players like Oracle, IBM and Microsoft. Overall, several of these NoSQL vendors are growing independently of one another—quite quickly in some cases—rather than converging into one giant system. This points to further fragmentation in the broader data-infrastructure sector, and could lead to the creation of several strong NoSQL players that could be public companies in the future.
Big data fuels open source
As many organizations struggle to manage huge volumes of structured and unstructured data today—generated by everything from security software to tweets to Web-enabled sensors in manufacturing plants—they are increasingly looking for new data-management and storage solutions. That trend is reflected in our index, as more than a dozen (15) of the 40 projects included are open-source technologies powering databases and data processing.
Hadoop, mentioned previously, is one such technology. But Spark, which is commercialized by companies like Databricks, is another, and ranked eighth on our list.