A Checklist for Using Open Source Software in Production

A great majority of the web is built on open source software. Approximately two-thirds of public servers on the internet run a *nix operating system, and over half of those are Linux. The most popular server-side programming languages also tend to be open source (including my favorite, Ruby). This post is about adding a new open source library to an existing code base. What questions should you ask before adding such a dependency to a production application?

The first set of questions are the most basic. A “no” to any of these should prompt you to look elsewhere.
  • Is the project written in a language you support? Is it in a language you support? If not, is it compatible (e.g. through stdin/stdout or by compiling to your language of choice)?
  • Is the project in a version of of the language you support? If it’s written in Python 3 and you only support Python 2, for example, using this library could lead to headaches.
  • Can you use the project in your framework of choice (e.g. Rails or Django)?
  • Are there conflicts with other libraries or packages you’re currently using? (This is probably the hardest question to answer, and you might not know until you try it.)
Assuming there are no immediate technical barriers, the next questions to ask are of the legal variety. Open source licenses come in many flavors. In the absence of a license, traditional copyright rules apply. Be especially careful if the project you are investigating uses the GPL license–even basing the code you write off of a GPL open source project can have serious legal ramifications. There’s a great guide to OSS licenses on Github. If you’re the author or maintainer of an open source project checkout choosealicense.com.
The next thing to consider is whether and how the project is tested. If there is not an automated test suite, consider starting one as your first contribution to the project and be very reluctant to add the project to your application. Other related questions include:
  • Are there unit tests?
  • Are there integration tests?
  • What is the test coverage like?
  • Do the tests run quickly?
  • Are the tests clearly written?
Finally, by using an open source project you are also joining a community of developers. None of these questions are necessarily show-stoppers but knowing the size of the community and the tone of its discourse can save you pain down the road.
  • Is the project actively maintained? When was the last commit?
  • Does the community have a civil, professional style of debate and discussion?
  • Is there only one developer/maintainer who knows everything? This doesn’t have to be a deal breaker. However, if there is a single gatekeeper you should make sure you understand the basics of the code and could fork the project if necessary.

This is by no means an exhaustive list but these questions can serve as a useful checklist before adding an open source as a dependency for your project.

Tirole on Open Source

Jean Tirole is the latest recipient of the Nobel prize in economics, as was announced Monday. For more background on his work, see NPR and the New Yorker. My favorite portion of Tirole’s work (and, admittedly, pretty much the only part I’ve read) is his work on open source software communities. Much of this is joint work with Josh Lerner. Below I share a few selections from his work that indicate the general theme.

open_sourceThere are two main economic puzzles to open source software. First, why would highly skilled workers who earn a substantial hourly wage contribute their time to developing a product they won’t directly sell (and how do they convince their employers, in some cases, to support this)? Second, given the scale of these projects, how do they self-govern to set priorities and direct effort?

The answer to the first question is a combination of personal reputation and the ability to develop complementary software (Lerner and Tirole, 2002, p. 215-217). Most software work is “closed source,” meaning others can see the finished product but not the underlying code. For software developers, having your code out in the open gives others (especially potential collaborators or employers) the chance to assess your abilities. This is important to ensure career mobility. Open source software is also a complement to personal or professional projects. When there are components that are common across many projects, such as an operating system (Linux) or web framework (Rails), it makes sense for many programmers to contribute their effort to build a better mousetrap. This shared component can then improve everyone’s future projects by saving them time or effort. The collaboration of many developers also helps to identify bugs that may not have been caught by any single individual. Some of Tirole’s earlier work on collective reputations is closely related, as their appears to be an “alumni effect” for developers who participated in successful projects.

Tirole and Lerner’s answer to the second question revolves around leadership. Leaders are often the founders of or early participants in the open software project. Their skills and early membership status instill trust. As the authors put it, other programmers “must believe that the leader’s objectives are sufficiently congruent with theirs and not polluted by ego-driven, commercial, or political biases. In the end, the leader’s recommendations are only meant to convey her information to the community of participants.” (Lerner and Tirole, 2002, p. 222) This relates to some of Tirole’s other work, with Roland Benabou, on informal laws and social norms.

Again, this is only a small portion of Tirole’s work, but I find it fascinating. There’s more on open source governance in the archives. This post on reputation in hacker culture or this one on the Ruby community are good places to start.

Github for Government

What happens when you combine open source software, open data, and open government? For the city of Munich, the switch to open source software has been a big success:

In one of the premier open source software deployments in Europe, the city migrated from Windows NT to LiMux, its own Linux distribution. LiMux incorporates a fully open source desktop infrastructure. The city also decided to use the Open Document Format (ODF) as a standard, instead of proprietary options.

As of November last year, the city saved more than €11.7 million because of the switch. More recent figures were not immediately available, but cost savings were not the only goal of the operation. It was also done to be less dependent on manufacturers, product cycles and proprietary OSes, the council said.

We’ve talked before about how more city governments could follow the open data, open government initiatives of NYC, using tech to benefit citizens rather than (only) creating initiatives to attract tech companies to the area. This shift in emphasis, toward harnessing the power of technology for widespread gains in happiness, is likely to become even more important following recent protests against tech employees in the Bay Area.

Open data and open government will take the principles of open source and use them to make an even bigger social and political impact. One tool from open source that can be adapted for use by these newer movements is Github. We will continue to follow these trends here, and if you are interested in this trend you can also check out Github and Government for more success stories.

How is Government Like the Internet?

Reid Hoffman thinks we should view government as an internet platform upon which citizens build lives. He isn’t speaking literally, but he isn’t far off the mark either. Hoffman’s background is in philosophy, so it is not surprising that his professional work (he is the founder of LinkedIn) and his political opinions intersect. Taking the view that Hoffman suggests means thinking of government as a service and citizens as customers who have a choice over which services to use and which not to use. I also like that it inverts the usual view by placing government at the bottom, as a foundation, and citizens can rise as high as they want.

Another philosopher/sociologist, Kieran Healy, has an interesting analogy of internet and government. He says

The U.S. system of employer-sponsored healthcare provision is iTunes. It’s complicated and overburdened; it wasn’t originally designed to do most of the things it now does; in fact, at the outset its design wasn’t really thought through at all (there wasn’t time); many of those involved backed it as a distant second-best solution—better than nothing, but not nearly good enough. Over the years, new features were shoehorned into the basic structure. New problems and inconsistencies emerged and were partially patched. And, inevitably, groups who did pretty well out of the system emerged and entrenched themselves, too. In situations like this, some reforms are possible around the edges, but it’s clear to most people that real structural reform is needed.

In development terms, that means it is time for a refactoring or even a complete overhaul. This debugging of government could also incorporate what Eric Raymond calls Linus’s Law (“Given enough eyeballs, all bugs are shallow.”) by responding to the feedback of informed citizens.

How far can we take this metaphor? Is it too utopian?

Wednesday Nerd Fun: The Setup

The Setup is a cool new site with interviews from all kinds of nerd heroes: designers, photographers, engineers, programmers, and so on. The interviews are all structured basically the same. First, the interviewee is asked about what they do, and they usually answer by describing both their professional and personal lives (note how many of them mention their home office). Second, what hardware do they use? In the interviews I’ve read, this often means a list of 3-4 key devices. Finally, the interviewee mentions the software they use most often. This is where I’ve seen the most variation between interviews.

XKCD on Computer Problems

Here are some fun ones:

Kellan O’Connor, SpaceX

Drew Conway, NYU

James Freeman, Blue Bottle Coffee

Paul Graham, essayist and YCombinator chief

Benjamin Mako Hill, MIT

Paula Pell, NBC (she includes coffee as hardware)

Eric Raymond, open source developer

why, Internet rodent (and author of the Poignant Guide to Ruby)

Stephen Wolfram

The full list is here. Enjoy!