Podcast: OpenShift Origin and the community with Matt Hicks

OpenShift Origin is the open source upstream for Red Hat's OpenShift Platform-as-a-Service offerings. Red Hat engineering director Matt Hicks talks about the tools and practices that let developers engage with OpenShift and how people can get involved.

The OpenShift community page also has a lot of great links to get you started, including to a Google+ community list and IRC channels.

I'd also like to plug the upcoming OpenShift Origin community day, which will be held in Portland, OR on April 14, the Sunday before the OpenStack Summit. If you're going to be in Portland, I encourage you to check it out.

You can also read more about this topic here.

Listen to MP3 (0:12:07)
Listen to OGG (0:12:07)

Transcript:

Gordon Haff:  Hi, everyone. This is Gordon Haff, Cloud Evangelist with Red Hat. I'm sitting here with Matt Hicks, who's the director of engineering for OpenShift, which is Red Hat's platform‑as‑a‑service offering. Matt's done lots of great talks and written a lot of great things about some of the engineering details around OpenShift, such as its multi‑tenancy approach, but today is going to be a little bit different. Matt's also been very involved with the open source contribution side of OpenShift ‑‑ how the community is built, how we accept contributions. I thought this would be great opportunity to sit down with Matt and talk a little bit about those things. Welcome, Matt.

Matt Hicks:  Hi, Gordon.

Gordon:  You've had an opportunity to write up some of the things that we've been doing with OpenShift over the last couple years or so. Maybe you could share a little bit about how we got from where we started to where we are today.

Matt:  If we were to roll back the clock a couple years, being Red Hat, we know what we want the end goal to be with our open source communities. But where we started on OpenShift in the initial days was, we were open in the code aspect but not necessarily in the community aspect of people being able to get involved at a low level in the project.

Gordon:  What is the contribution model today?

Matt:  Today our goal has been to allow users to contribute at any level of the project where they're comfortable. We have a bunch of different components from making contributions to the client tools or the Web tools or the server-side pieces. We wanted all of the information from planning to involvement with our team, to communications, to being able to submit code changes, and understand how they're tested and evaluated ‑‑ for that all to be transparent to let people get engaged at whatever level they're comfortable with.

Gordon:  How does someone get code into OpenShift?

Matt:  On the mechanical level, we built a lot of our processes around GitHub's pull request model. You fork our project, whichever project is where you're going to base your contributions. You make your changes. We try very hard to make sure that we have a lot of automated tests that to help guide you down a path where the expectation is that when you do your pull request, we're going to have a bot that picks up those changes and runs it against all the tests that are in the project. If those tests pass, we're going to have somebody do a review and then merge your changes in.

We try to keep that process consistent whether you're making changes to something simple on the client side like our client tools or something on the server side that could be a little bit more bound to the OS like our node component.

Gordon:  You mentioned this has been a couple‑of‑year process. I think a lot of the time with open source, a lot of people are like, "Why hasn't this been done yet?" Whatever aspect of open sourcing things are. I've been on both sides of this. It certainly is a bit harder than it sometimes looks from the outside.

Matt:  Yeah. The challenge that we have is, as much as I wish that in a single day we could have the community where we want that end goal to be, my experience in open source is that every community is a little bit different, whether you're working with Fedora more on the distribution side, or whether you're working with a software development project. The people are different. The expectations are different. Our approach has been we're in this for the long haul and we are going to progressively make improvements and see what users’ feedback was and keep building on that, instead of laying out our grandiose plan that was going to make everybody be happy. Then running the risk that we put a lot of effort into it and it doesn't make anybody happy.

Gordon:  I think, particularly for a company like Red Hat that has so much experience around open source and open source governance, I think there's sometimes this assumption that, "Well, there must just be a template for this kind of thing and you just turn it on and do it." But it's not like that.

Matt:  I wish it was. I think the challenge is the technologies are changing, processes are changing. Being able to use something like GitHub wasn't even an ability five years ago. Then if you add cloud computing to that, a lot of how we use our continuous integration environment to run tests, it's fundamentally changing so fast that that template is different every time you turn around and try something new.

Gordon:  Let's dive into that technology a little bit, around the communications, around the integration environment. What are some of the specific things that we're using and we're doing there?

Matt:  Communication is probably the one area that has changed the least for us. That should be comfortable with people. We're on IRC. We're on mailing lists. We have tried to pull things together in a Google+ community; it’s like an aggregation community. But our first goal is to make sure that our communications were visible and the engineers on our team are accessible. But whereas that area has stayed the same, if you look at how code contributions are tested or changed, it's fundamentally different to where we have a continuous integration environment that's running in open shift and every single change that's submitted to our project is tested by itself against thousands of tests on VMs that are running in Amazon.

Our cost, cost wise it costs us about a quarter to do that. The ability for you to do that at such massive scale, a decade ago or five years ago, was an impossibility. Yet now, it's a great process for us, because it's a very objective process to tell whether changes worked or didn't work, instead of just immediately going to a more subjective human review.

Gordon:  Is this technology we mostly developed at Red Hat or what open source projects we leveraged?

Matt:  For continuous integration we leveraged Jenkins heavily. We have made customizations to Jenkins in terms of how Jenkins master/slave environment works. Which those are open source as well, but really we've built on a lot of components that are there. Amazon at the VM level for providing infrastructure as a service, using RHEL and Fedora images within Amazon, which are already there. Using Jenkins for the continuous integration project itself, using GitHub for the pull requests. Our goal has been to weave these all together in a very transparent and efficient process for how changes can get into the abstract projects.

Gordon:  We're talking about the OpenShift Origin here, which is the open source version of OpenShift. How does that relate to OpenShift online, which is the hosted service and OpenShift enterprise, which is our on premise enterprise PaaS?

Matt:  OpenShift Origin is the upstream project for both of those offerings. Everything that she'll see in online or enterprise is going to be a derivative of what you see in OpenShift Origin. Our goals with OpenShift Origin are really, it's a collaboration environment. There's not going to be a lot of stability or ABI compatibility from release to release, but we want it moving fast to make sure that we can pull in changes and get other users’ ideas into the platform.

Now, if that's the goal of OpenShift Origin, when you move down to OpenShift Online, our goal is to provide a stable environment that tracks, pretty closely, to OpenShift Origin. So you're seeing new features, but you're also not breaking week to week. So we put in a lot of effort to consume those updates from Origin, but at the same time not break user applications.

Then when you move to OpenShift Enterprise, it's the same relationship, but now you can actually install that project on-premise. We go to an even greater extent of testing to make sure that the install is polished and that we know how to do highly available environments and the packages and all the components can be supported by Red Hat.

That's the distillation process you see from Origin all the way down to OpenShift Enterprise.

Gordon:  How does somebody get involved?

Matt:  There are a lot of great places to start. In general, I tell people, "If you're just cruising by, start with the Google+ community," because you'll see a lot of the activity that's occurring on OpenShift. Whether that is building a new DNS plug-in that's based on Route 53 or DNS mask at the internal side, all the way down to building a... We had a guy that built a drag and drop project based on our REST APIs, to let you drag and drop the cartridges and pieces in OpenShift.

Google+ will give you great exposure to what's going on, help you make that decision as to where do your interests lie and what might be a good area to dig in more. After that, my preference is usually the development lists in IRC. We have a bunch of different IRC channels that are available. #openshift is great. #openshift‑dev are the two high level ones.

Our development mailing list really has got a lot of eyeballs both from Red Hat and externally and people's responsiveness in that list has been phenomenal in terms of supporting users and helping get them up to speed and vet ideas and new features for them.

Gordon:  We have a community day coming up in advance of the OpenStack Design Summit in Portland.

Matt:  Yeah, that's going to be neat, because that's the culmination for us. We finally think we're at the point where our decision making process is transparent. Our architecture, guides, our enhancement proposals, they're out in the open. People can see why we're making decisions. Our sprint-level details of user stories are out there. Our communications are open. Our contribution process is open. The Origin community...It was really the next step for us, is to get people face to face and say, "All the mechanics are in place and they seem to work. How do we engage users at a deeper level now to make sure that their ideas, they have direct access to our guys to push bigger changes and bigger ideas in OpenShift?"

Gordon:  It's a lot more complicated than just slapping an Apache license on something, isn't it?

Matt:  [laughs] Yeah, that's right. I wish it all stopped with pick ASLv2 and then be done with it. But unfortunately it's a lot of things and it's a constantly evolving process. That's my other request for users, if they try it, we know we're going to have rough edges, especially depending on the area they come in. The most important thing for us is, tell us what those rough edges are. We'll knock them down. That's our goal. We are committed to this in the long term. We'll make it better and better and better as we go on until it really is a pleasant engagement environment for everybody.

Gordon:  I think that's something maybe that a lot of folks even at this late date don't necessarily understand. Open source isn't about viewing code. Well, it is about that, but certainly not just about that. It's really about a development methodology and a community model.

Matt:  Yeah, exactly. It's more of a living thing than it is just accessibility to code.

Gordon:  Thank you, Matt. Anything else you'd like to add?

Matt:  I'd recommend people dig in. Let us know what you think. Let us know what interests you and what doesn't. Tell us how to get better, and we'll do it.

Gordon:  Great. Thanks, Matt.

Matt:  Thanks, Gordon.