Tuesday, February 21, 2017

Podcast: Cloud Native Computing Foundation with Dan Kohn

Dan Kohn is the Executive Director of the Cloud Native Computing Foundation. In this podcast, he discusses the goals of the CNCF and the reason why Kubernetes is under the CNCF's umbrella--plus his take on serverless computing. In addition to Kubernetes, the CNCF also hosts Prometheus, Fluentd, OpenTracing, and Linkerd.

In the links below, check out the cloud-native landscape in particular, which catalogs the broad set of projects playing within this technology area. As Dan puts it: "Kubernetes is the cornerstone of a containerization and orchestration solution but is not a complete solution."

Link to MP3 (0:22:25)
Link to OGG (0:22:25)


Gordon Haff:  Hello everyone. Welcome to another edition of the "Cloudy Chat" podcast. This is Gordon Haff, technology evangelists with Red Hat, and I'm sitting here, at the Open Source Leadership Summit in lovely Lake Tahoe, with Dan Kohn, who is the Executive Director of the Cloud Native Computing Foundation, which is under the Linux Foundation. Welcome Dan.
Dan Kohn:  Thank you very much. Glad to be here.
Gordon:  Dan, first of all, could you give us a little bit of background about yourself?
Dan:  Sure. I actually used to be the Chief Operating Officer of the Linux Foundation a decade ago, when it was a much smaller organization, back when there was just a few of us. I helped Jim, the Executive Director merge together the two predecessor organizations. I then went off and worked on a few startups, one of my own, one of another.
Then as the Linux Foundation has grown and brought in new organizations under it, and it has become a foundation of foundations. Jim has been recruiting in different folks to run the different sub‑foundations and pulled me back into run the Cloud Native Computing Foundation.
Gordon:  Cloud Native Computing Foundation is probably best known as the home of Kubernetes. That’s a very well known container orchestration platform. Maybe we'll start off talking about Kubernetes, and how you see the role of the CNCF with respect to Kubernetes is? How you see things are going? Then maybe about what are some the next steps you see happening are?
Dan:  Sure. We definitely are incredibly proud to be the host for Kubernetes. It's one of the most exciting software projects on the Internet today. It's also one of the highest velocity projects by almost any metric of number of commits per day, number of companies participating, number of developers participating, total volume of issues, pull requests. It's actually, probably just second or third behind the Linux itself in terms of velocity that it's been able to keep up.
Then even more than that, it's just the fact that it's out there solving real problems for users, for enterprises, for startups, all kinds of companies today, both in the public cloud and bare metal and private clouds where a containerization is this trend that's taking over the world to allow people to run all kinds of different applications in a variety of different environments.
When they do that they need an orchestration solution in order to keep track of all of those containers and schedule them and orchestrate them. Kubernetes is an increasingly popular way to do that.
Gordon:  Before it joined the CNCF or really formed the core of the CNCF, Kubernetes was already becoming pretty popular. Although there was contributions from a number of companies including Red Hat, certainly Google, which contributed in the first place. What was the genesis of the CNCF in the context of Kubernetes? Why was it really needed?
Dan:  The origin of Kubernetes was three folks at Google, and it was really built on the intellectual foundations of Borg, which comes from 15 years of Google experience with containerization.
As you said, they built that out and then they recruited folks from Red Hat, from Huawei, from a number of different places in the community and they said, "Hey, this project has a huge amount of potential. What will it really take for it to reach that potential?"
One of the things they realized very early on is that a project with a neutral home is always going to be able to achieve a higher level of collaboration. They really wanted to find a home for it where a number of different companies could participate.
A huge piece of that is the intellectual property framework where the idea is that Kubernetes operates under the Apache license what we think of as an intellectual property, no‑fly zone. Everyone contributes, there's no patents that the companies will file against each other.
Generally that the trademark, the rules are neutral between all of the different participants, all of the different users that, there's a trusted neutral body that that they can look at. Those early users went to the Linux Foundation and said, "You're one of the best‑known folks in open source. Can you help us work with this?" That's why they set up the Cloud Native Computing Foundation.
I think that's a good segue to say that, they also said that, "They were not just interested in creating a Kubernetes Foundation." They saw Kubernetes as a cornerstone of containerization and orchestration solution. As a critical piece of it but not as a complete solution and then seeing that there should be a number of other projects that were really very important to their stack.
Since I've joined over the last nine months, we've begun the process of bringing in new projects into CNCF. Prometheus is a very popular, well‑respected monitoring application that interestingly originally came out of Soundcloud.
Not a services company like Google is, but just they were initially scratching an internal itch, but it's since used by hundreds of different companies around the world and commercialized and are very popular. Now there's three new projects behind that one, Fluentd, OpenTracing and the newest one, just as of a couple of weeks ago is Linkerd.
We have several new ones in the pipeline. What we're trying to do is, over time, build a comprehensive open source stack of software that provides all the solutions that companies and enterprises and start‑ups and individuals need in order to deploy Cloud Native solutions.
Gordon:  Now, there's a lot of activity, obviously, going on in the broadly speaking cloud native space, there could probably be hundreds of projects in the CNCF, it casts a wide‑enough net. What are your criteria? What do you see the walls being around what you want to accept? What would cause you not to be interested in a project? Give us a little color about all that.
Dan:  One piece that I would recommend ‑‑ and I'm sure you can link to this from the show notes ‑‑ is that we're publishing a Cloud Native landscape, which is an open‑source document. It's on GitHub. It's trying to track basically all of the projects in the space. As you would imagine, there's a ton of them.
We're also tracking closed‑source startups and companies' offerings. It's a project, so if you see something on there that we're missing ‑‑ your company or your startup ‑‑ please open an issue, and we'll try and get it in there in the next version. That's a good way of tracking the progress and how we think of the space and the landscape. It's available at github.com/cncf/landscape, but we can include a link to it.
One of the interesting things about the way that CNCF is set up is that I can't actually bring in any new project. No one from the CNCF staff can and no one from the governing board, which are the vendors who provide most of the funding for our foundation, can't bring in projects, either.
Instead, we have a group of technical architects, experts in the field, folks like Bryan Cantrill from Joyent, and Alexis Richardson from Weaveworks, Brian Grant from Google, Solomon Hykes from Docker, who are our technical oversight committee, nine folks. It takes a supermajority vote of that group to bring in any new project. That's really a technical gatekeeping function, that they have very high standards for the kind of projects that can come in.
Now interestingly, when we first got started, with Kubernetes being such a successful, high‑velocity, exciting project, and then Prometheus as well setting very a high standard, we had a little bit of anxiety that the hurdle was going to be so high, say, "Hey, you know, there's not that many projects out there that already have hundreds of users, or, or thousands of developers."
The TOC recently just approved a new graduation guidelines that include a lower tier of project that we call, "Inception level." This is a little bit more of an experimental level to say, "Hey, this level of project isn't quite as mature as the other one, but it's very promising. We think it's really worth taking a look at, and we're optimistic that it will get there."
Then, it requires the TOC every 12 months to come back and essentially renew its status, either at inception level, or move it up to incubating, or just to have it exit the foundation.
Gordon:  What do you provide in terms of resources for those projects which are not up to the highest level yet?
Dan:  We actually try to provide the same resources for all of our projects, and so we love all of our children equally, or do our best to. There's a whole set of resources that by far the most important one is what I said before, which is that a neutral home for a project increases collaboration. That's the biggest piece of it.
If you say, "Hey, you're providing a CLA bot," which is a little robot hooked into GitHub which keeps track of whether the user has signed a contributor licensing agreement. Yes, we provide that, but are our engineers really better at Google than Google's at doing that? Probably not.
It's not that we're able to do something uniquely that Google couldn't, or Sound Cloud couldn't, or some of the other homes couldn't, but it's the neutrality is the huge value. That's the most important one, but we do have a whole set of foundation services, starting with the fact that I'm a full‑time employee, and we have several others.
We're all dedicated to promoting our projects and trying to help them succeed. We have a press and an analyst relations team, we have a really fantastic events team. We do two big events per year. CloudNativeCon, KubeCon, coming up in Berlin at the end of March and then in Austin's at the beginning of December.
We also help our projects with smaller events if they want to do it. For instance, Prometheus is going to be running PromCon in Europe this summer, that we are helping them organize. They're going to still do it as their own developer event, and not that's going to be a smaller scale one.
Then one of the really amazing, extraordinary resources that we have, it was made available to us by Intel, is a $20 million, 1,000 server cluster, which is housed by another one of our members SUPERNAP, at their switch facility in Las Vegas.
This is available for priority access to our five projects, but it is actually available to any open source project that's interested in demonstrating or experimenting with or working with a Cloud Native technology. Anyone can go to github.com/cncf/cluster, open an issue and file a request for an allocation from that cluster for as little as 20 machines and soon up to 1,000.
Gordon:  You obviously have a lot of experience in foundations, the Linux Foundation. There's a whole lot of experience, but things are always changing, you're always learning, environments change. What have you learned?
Dan:  That's an excellent one. I'd say by far the biggest part for me has really come down to a respect for the developer. I've been involved in open source in different ways in 20 years, but I really see the developer as ‑‑ I'm not quite sure, calling them the plankton of the ecosystem is the most pleasant metaphor, but I actually mean it in a very nice way ‑‑ in that every other aspect of what we do, absolutely depends on them.
When you look at a healthy project it has a ton of developers that are incredibly excited about it, that are contributing. Certainly the core maintainers that are actively involved in and hopefully they're being paid by companies to work on it or as consultants or such, and then probably a number of others who are doing it more part‑time or as a hobby or making specific bugs fixes or filing issues.
When you have those developers that feel like their contributions are valued and taken seriously, then there's a whole ecosystem that forms around them, of companies that are interested in offering services to them, employing them, that want to make these services available to other folks. Then a foundation like ours can come up and help make those services available. I really think that, that developer focus is the key thing to keep in mind.
Gordon:  At the risk of being a little inside baseball. I want to ask you about relationship between a couple of the projects under Linux Foundation. OCI, the Open Container Initiative is separate from the Cloud Native Computing Foundation. What is the reason for that? What do you see is the relationship there being?
Dan:  It's a very close relationship because the head of the OCI, Chris Aniszczyk is also the COO of CNCF. Whatever you want to say like the Chinese wall, we don't share it. It's definitely not that. There's a incredibly high overlap of 80 or 90 percent of our membership and otherwise. The other reality of helping foundations work is that we're pragmatic.
When we set this up, a lot of the core folks wanted to have standards organization, that was standardizing the technology behind containers, and so the Linux Foundation was willing and eager to help make that happen. At the same time there was an interest in having a foundation that was hosting some of the core technologies here, and we have that as well.
It's a little bit confusing. Sometimes it seems that we have too many panels of, what's the relationship between OCI and CNCF, but hopefully it's becoming clearer over time.
Gordon:  You've talked a little bit about the wide range of projects that do or can fit under the CNCF. What some your criteria are for membership or for becoming a project under the CNCF. If we look forward, let's say 12 months ‑‑ I'm not sure we can look any forward than 12 months in this industry, particularly in the Cloud Native computing space ‑‑ what changes might we expect to see?
Dan:  It's a great question. Even 12 months is a little challenging for me. My hope is that you'll see just more projects in CNCF that people are excited about, they see as complementary to the ones that we already have. I do want to emphasize that when we add in a new project, we're never saying to existing users of Kubernetes or Prometheus. "Oh, you must use OpenTracing, Oh, you must use Linkerd."
We're saying, "Hey, these are projects that we think are a complementary. We're investing continuous integration resources to try and make sure that these projects work well together. We're setting a signal that we think that these different projects attain the same level of quality."
My hope is that you would see another half dozen or more projects along the same lines that have been added into CNCF. I also hope that you would see all of our existing projects have continued to grow and thrive.
Then particularly these projects are happy with their home at CNCF that they feel like they're getting value from us, that our members feel were providing value to them. The bigger picture is, this is my comment to Jim Zemlin about the brilliance of the Linux Foundation where he said, go back 10 years.
You say, "Oh, you know, what was it about how it was set up? Was it that the technical advisory board of Kernel Developers was separate from the Linux Foundation Board?' Which it was and was very valuable and we replicated that model and I don't mean to downplay that was useful. Fundamentally, the reason the Linux Foundation was successful is because Linux was successful.
Similarly, I think one of the reasons that CNCF has a very good shot at being successful is because Cloud Native computing is an incredibly exciting trend that just has a huge amount of momentum behind it, in the public cloud, in private and bare metal computing and hybrid cloud computing. Kubernetes is one of the most exciting projects around there as is our number of our other projects. The future is definitely looking very bright.
Gordon:  What are seeing about serverless computing on the horizon? Certainly what you have now ‑‑ I think it's pretty fair to say its container‑centric. Do you see serverless computing as under CNCF purview and where do you see that going?
Dan:  I definitely do. We're talking to a number of serverless options that work with Kubernetes in different ways and there's different folks who are solving different pieces of that. I think AWS Lambda is incredibly exciting piece of technology.
There's a blog post that I just love of a YCombinator startup up called Benchling and they had an intern rewrite their application from running on a bunch of EC2 machines to working on Lambda and it's dropped the average time for all their users dramatically because of they didn't have to spin up the same number of machine that didn't have contention. They dropped their hosting bill from thousands of dollars a month to $60 a month.
It just been a compelling enough story. I mean it was a uniquely good used case for it, to say, "Hey, there's really something here." It's definitely an area that we're looking at, but what we would love to do is find some open source solutions for it that don't lock you into a single cloud provider, that can work across different cloud providers.
Gordon:  I think you touched on this may be a couple minutes ago but obviously the OCI for example in terms of container standardization has been a pretty significant benefit in keeping away a level of fragmentation, which might very well probably would have slowed adoption.
In Kubernetes being under the CNCF there's obviously other orchestration projects, but there does seem to be a certain role in terms of not picking winners necessarily but maybe, showing a preference for stronger projects and projects with larger communities. How strong is the CNCF in terms of showing preferences for certain technology choices?
Dan:  Our vision is that we're trying to promote Cloud Native computing, and we think that, that entails an open source stack for each of the different functions. We think that, as an example Mesos, Docker Swarm and Nomad are all perfectly valid orchestration platforms that are alternatives to Kubernetes.
Now, we host Kubernetes and so we obviously love that and promote it and are excited about it, but we also think that companies choose different projects. For example, Prometheus works great with all of those other orchestration platforms, as do a number of our other projects.
Certainly our expectation is that most of our or all of our projects should continue to work with all of their competitors. There's not any sort of and just in general terms of how the open source world works. That serve our hope of how things will continue.
Gordon:  Great. Thank you there's obviously been a vast amount of interest in projects under the CNCF. Your last cloud day of KubeCon, sold out. I don't think I could get in there in Seattle and you have another event, Berlin coming up, which as I understand you've allocated a whole lot more space for.

Dan:  To be honest with you we tripled the capacity from London up to 1,500 but we are on track to sell out in Berlin as well. Folks who are thinking about coming, we would really encourage you to sign up now. It's March 29 and 30th. We think it's really going to be a fantastic event. The schedule is up there now. This would also be a great time to begin thinking about Austin, this December 5th and 6th.

No comments: