Friday, February 24, 2017

Podcast: Open source and cloud trends with IDC's Al Gillen

Al Gillen is responsible for open source research and oversees developer and DevOps research at IDC. Al gave a keynote at the Open Source Leadership Summit at which he provided some historical context for what's happening today in open source and presented recent research on digital transformation, commercial open source support requirements, and how organizations are thinking about cloud-native architecture adoption and deployment.

Listen to the podcast for the whole conversation but a few specific points that Al made were:

Digital transformation can be thought of as taking physically connected systems and logically connecting them, i.e. connecting the processes, the data, and the decision-making.

It's important to bridge new cloud-native systems to existing functionality. Organizations are not going to be rewriting old applications for the most part and those "legacy" systems still have a great deal of value.

Enterprises are asking for open source DevOps tools, but most are specifically asking for commercially-supported open source tools.

Audio:
Link to MP3 (00:15:46)
Link to OGG (00:15:46)

Transcript:

Gordon Haff:  Hi, everyone. Welcome to another edition of the "Cloudy Chat" podcast. I'm here at the Open Source Leadership Summit with Al Gillen of IDC, who gave one of the keynotes this morning. Welcome, Al. How about giving a little background about yourself?
Al Gillen:  Hey, Gordon, thanks a lot. Thanks, everybody for listening. This is Al Gillen. I'm a group vice president at IDC. I'm responsible for our open source research, and oversee our developer and DevOps research.
Gordon:  One of the things you went through in your keynote this morning was the historical perspective of how Linux is developed. Both of us have pretty much been following Linux from the beginning, certainly from its beginnings as something that was interesting commercially. Maybe you could recap that in a couple of minutes or so.
Al:  I actually went back to a presentation I delivered to the Linux Foundation, an event at the Collaboration Summit that was back in 2008. I pulled those slides up, because I was curious. "What can I learn from what we talked about back then, and how does that relate to what's going on in the industry today?"
I went back, and I pulled up the deck. I was looking at some of the things that I thought were really interesting. For example, I was looking at one of the first pieces of data, which compared perceptions of Linux from 1999 and 2001.
Remember what the time frame was there. Linux had only just begun to be commercially accepted in the '99‑2000/2001 time frame. One of the things that served as a significant accelerator for Linux at that time frame was the dot‑com bust.
What happened then is we had a big contraction in the stock market. Most large companies, what they did is they went and they started to cut costs. We all know that one of the places they first cut costs is IT.
Suddenly, the IT departments were charged with standing up new Web servers and new network‑infrastructure servers and so forth, and they had no budget to do it. What did they do?
They went and they got a free copy of Linux. They recycled a well‑equipped PC or x86 server that had been taken out of service, and they turned it into a Linux server.
When we look back at the data that we saw then, really, one of the big drivers for Linux was initial price. People said, "Yeah, it was great. The cost was really low." One of the things that was also amazing was the users back then rated the reliability of Linux as very, very high.
In fact, when you compare it to other operating systems, it compared very favorably to much more mature operating systems. That context was really fascinating, but when you think about it, that was just the beginning of a long gestation period for Linux.
Over the next, what, seven, eight, nine years, as Linux became a truly mature and a truly robust commercial operating system that had both the features, had the application portfolio, and had the customer base to use it, it took, basically a decade to get there.
Gordon:  You've been doing some more research recently. What are your numbers showing today?
Al:  A couple of things that I showed in the presentation today. One is we presented data on Linux operating system shipments. One of the things that's happened over the last few years is that Linux has continued to accelerate, in part because of the build‑out of cloud.
Most of the public cloud infrastructure, with the exception of the Microsoft Azure cloud, is almost all Linux. To the extent that Google continues to build out and Amazon continues to build out, and companies like that ‑‑ Facebook, Twitter, and so forth, it's primarily Linux being stood up.
That has driven the growth of non‑commercial Linux, meaning distributions that are not supported by a commercial company you might think of, rather are either they're CentOS, or they're Debian, or potentially Ubuntu, that's not supported, things like that, as well as Amazon Linux, and Google's own Linux and so forth.
That's been really where a lot of the growth is, but that's not to say that there hasn't been growth in the commercial side of the market. There's been growth there as well.
Gordon:  What are some of the drivers that you hear? I know you did some research for us. You also have some research here around commercially‑supported environments and maybe some of the reasons why people buy those.
Al:  That has been something which has been really consistent through the years. We find that large enterprise organizations have a tendency to prefer commercially‑supported software.
That has always been the case with Linux and yes, we find that there is [also] non‑commercial Linux. You'll talk to any enterprise ‑‑ and you could talk to a really big Red Hat shop or a big SUSE shop, and you ask them, "What is in your infrastructure?" They'll typically tell you that, "Yeah, we're 95 percent or 98 percent Red Hat, but we've also got some CentOS," or, "We've got some Debian," or, "We've got SUSE for this one application."
They generally have a mix of other things in there. The same thing, if you talk to a SUSE shop, where they'll say, "Yeah, we're mostly SUSE, but we've got some openSUSE," or again, "We've got some Ubuntu or CentOS, or something else in the mix."
The reason why is that these things get stood up for work loads that are considered not critical. Maybe they might be something simple like a DNS server, maybe something that's a print file server, or maybe a Web server which is providing some internal Web serving capabilities, something that's not critical if it disappears off the network suddenly. There's not going to be customers that are left hanging.
Gordon:  Let's switch gears and talk about digital transformation. This is one of those terms that I think is almost a cliche at this point, at least when you go to these types of events, because we hear it at every one of these events.
As somebody I was talking to recently said, just because it's cliche doesn't mean it's not an important trend. What are some of the things that IDC is seeing about digital transformation?
Al:  If we go out to, say 2025, and we look back, I believe that we're going to look back and say, "Yeah, the mid‑teen years were important years from a digital transformation perspective."
When we at IDC talk about digital transformation, what we're really talking about is we're talking about the interconnection of all of the systems that are in our environments.
When I say interconnection, we're not talking about getting them all on the same network. We've done that. It's been done for 20 years, already. What we're talking about is interconnecting the processes, interconnecting the data, interconnecting the decision‑making. In many cases, that's not done.
We've got systems that are physically connected, but are not connected from a logical sense. That's what's happening with digital transformation. I might add that the way we expect that that's going to happen is it's going to be a model where we're going to be building new applications that are going to essentially bridge the existing functionality that's on these servers.
We're not going to be rewriting those old applications in a cloud‑native format, for example. We're going to keep those applications. We may wrap them with some consistent API so we can get access to the logic and the data.
But at the end of the day, the business value that's in those applications that are in place today remains valuable, and frankly, it's going to mean that the applications themselves and the servers that they run on are workloads that are going to be around for the long term.
Gordon:  I think that's a really important point, because one of the things that I hear a lot when I talk to customers is the importance of this bridging of the older systems that may be modernizing, but as you say, you're not turning them into cloud‑native systems.
On the other hand, you have these cloud‑native infrastructures. I think probably in the industry, there's too much thinking of those as two disconnected islands, and not enough thinking of the bridges, the integrations, and so forth, between them.
Al:  There's a really good parallel here, and I like to bring this story up, especially when I'm in a room full of end users. I like to say, "You guys remember what you were doing in 1999?" You get a little bit of a quizzical look, and I say, "Were you remediating any applications that had ywo digit date codes?"
People start nodding their heads. The next question I say is, "What did you do? Did you fix them?" and the heads keep nodding. I say, "OK, can I see a show of hands? How many of you have gotten rid of those systems, or do you still have them in use?"
All those same people, they put their hands up, reluctantly, I might add, and say, "Yeah, we still have those systems in use." The point being is that the value of the systems does not go away. The value of the systems is in the data. It's in the processes and the business logic that are coded in those applications.
Going forward, we think the same thing's going to be true for the distributed computing environment. All of the Linux servers that you have in place, all of the Windows servers you have in place, have real important business value. The logic and the data there is really valuable to your business, which means that you're going to want to use that going forward.
I do agree with you, when we build that cloud‑native applications, they're going to help bridge these systems, but don't for a minute assume that those old systems have no value left.
Gordon:  As we talk about the new applications that are going to be required for this digital transformation, what are some of the...I think you even used the term, they were the "pivot point," this morning. Tell us a little bit more about that.
Al:  Again, taking a long view, so if we look out, say, if you go out to 2025, and you look back, I think that we'll be able to draw a line in history and say, "Somewhere between 2015 and 2017 or 2018, there's going to be this line where everything before that line will be considered a legacy application, and everything after that line is probably going to be a cloud‑native and a modern application."
Again, let's not associate the term legacy application with something that has no value. Let's assume that that's an architectural statement more than anything else.
When I think about it, I believe we're right in the midst of this transition where we begin to build all of our applications using a cloud‑native format, which means that our applications are built to run on‑prem in private cloud, or off‑prem in public cloud, which means that we have flexibility on where they run, how we want to run them, how we want to scale them.
The other thing I might add is remember that cloud‑native and cloud‑scale are not necessarily the same thing. There's lots and lots of applications that should be cloud‑native, but not all applications have to have cloud scale.
Take for example your average enterprise. You've got ‑‑ pick your number ‑‑ it's 1 thousand, 10 thousand, 100 thousand users, whatever the number is, that access your business applications. That number does not scale up to 1 million or 10 million overnight.
By comparison, somebody who's doing, say, business to consumer, where there could potentially be a consumer event that causes everybody to come in and access that application. There you'd need to have the ability to do cloud‑level scaling.
Gordon:  Al, going back to the commercial support still being important, you certainly see in the cloud an awful lot of this consumption of free software, but you mentioned earlier that enterprises by and large do want commercialized tools. We're not talking just operating systems here.
Al:  No, in fact, operating systems are probably one of the best understood pieces of open‑source software today. As we go up the stack, customers still see value associated with commercialization, so a company that will take your project and make it something that is consumable will provide the support.
The reason why that's so valuable is that then the company does not have to have the expertise on staff. You don't have to have a kernel guy, you don't have to have a guy that knows how to patch Xen, for example, or a KVM if there's a problem.
It's really important to have that taken care of by somebody if you're a commercial organization, which is in the business of selling widgets or manufacturing things, or providing health care. That's your primary business. Your primary business is not being in the business of IT.
We ran a survey earlier this year. I guess it was actually late in 2016. We were talking to people about their consumption of DevOps products. We asked a question which I think was really interesting.
The question was if they have a chance to buy a product, are they going to look at a product which is open‑source, or are they going to look at a product which is likely to be a closed‑source and/or a proprietary‑type product?
What we found is we asked people to rank their preference on these things. The answer came back as 45 percent of the people we talked to ranked their preference for an open‑source base product as their first choice over anything else.
If you asked them what their preferences were, for example, for a proprietary product, only 15 percent of the people said that was their first choice.
The reason why this is really interesting is that when we look at that, these companies are telling us that they want an open‑source base product, but they also told us that they wanted it to be commercially supported.
They could get the bits and run it as a project themselves. We asked that question as well. That's not what they're asking for. They're asking for it to be commercially supported. The reason why is if it breaks, you pick up the phone or you get on your computer and send an email, and you say, "Fix it."
Gordon:  That was true going back with Linux in the early days as well. I think one of the things that's happening today when you look at DevOps tools is there is an incredible amount of innovation and number of products out there. That's good news.
The bad news is there's an incredible amount of innovation, rapidly changing products, and a need to integrate all of those together.
Al:  You know what, Gordon? It's one of the challenges that we've had with fast‑moving markets like this. When I say markets, I'm referring to open‑source, collectively.
The problem we have is that the technology changes so fast that the people, the end‑user organizations, are not able to gain the skills fast enough to keep up with these technology changes.
Frankly, when we asked questions about Linux in the early 2000s, people said one of their top challenges was having the skill set to support Linux. Today, we find the same questions about things like container technology and using DevOps tools.
Gordon:  To net it all out and close this out, what are some of the recommendations, the guidance, that you give on probably pretty much a daily basis to your clients?
Al:  There's a few things. Number one, recognize that cloud‑native applications are going to be architected very differently than classic applications. That's pretty much a given, but when you think about it, it affects your choice of tools, it affects your choice of deployment scenarios, and it affects your skills that you need to have on staff.
Another thing is recognize that we've moved to an era of platform independence far beyond anything we ever had before. We always like to talk about platform independence, but we've never really had it.
Now, with container technology and the ability to produce a true cloud‑native application that's running on some kind of a framework which happens to be available on‑prem or in cloud, you suddenly have the ability to move that application on‑prem or off‑prem, or both ‑‑ run in both places at the same time if so you choose ‑‑ and be able to do that in a way that's been unprecedented in our industry.
Finally, just to reiterate the other point is recognize that the existing applications don't lose their value. They still have value. Yes, they may get bundled up in a VM, or maybe packaged up in container and dropped into somebody's IS cloud, but they're going to be around for the long term, and recognize that that's something you have to support.

Again, driving home that point I made earlier, recognize that all the new applications we build today are going to have to bridge the classic applications we've had, and the data that those applications support together with the modern things that we're going to be doing with our new applications.




Podcast: Open Container Initiative with Chris Aniszczyk

Chris Aniszczyk is the Executive Director of the Open Container Initiative (OCI). I sat down with Chris at the Open Source Leadership Summit where we discussed the role of the OCI, how it relates to other efforts such as the Cloud Native Computing Foundation, and future directions for the organization.

Among the topics that we covered were:
  • How the OCI primarily focuses on areas where standardization is critical
  • The OCI's relationship to the Cloud Native Computing Foundation
  • Future directions, especially in distribution
  • How the industry has changed to encourage this sort of organization
Some links:
Audio:
  • MP3 (00:12:51)
  • OGG (00:12:51)
Transcript:

Gordon Haff:  First of all, probably a lot of our listeners know what the OCI [Open Container Initiative] is, but maybe a little background for those who don't.
Chris Aniszczyk:  The OCI came together a little over a year ago when there was a desire to come and standardize on container runtime and format, instead of having different competing implementations out there, which would ruin the original promise of containers being truly portable, vendor neutral, and all that good stuff. The OCI was created to build the standard around the container format and runtime.
Gordon:  Has it been successful?
Chris:  I think it's been going great. We originally started out the project with...Docker donated something at the time called Libcontainer, which morphed into runC. runC has grown and seen lot of success. It's embedded in Cloud Foundry.
Of course it exists in Docker's containerd, Engine, and all that stuff. Kubernetes is picking it up in its container runtime interface, and runC and the runtime specification is becoming the de facto way to execute containers, which is great.
Another aspect of the OCI...when the OCI started, it focused just on the runtime spec. About six months after the project was started, the OCI technical oversight board said, "Hey, we should consider standardizing on the image format," so the image specification project was created at OCI. There's been a lot of good work in that area.
We've seen just recently, like a week ago, Amazon announced in their ECR product, the Elastic Container Registry, that they support the draft of the OCI specifications. Pretty much, most other projects are adopting the image specs. I think in general we've made a ton of progress in many fronts. Mostly any system that runs a container is probably using some of form of OCI technology.
Gordon:  When do you see that spec being finalized, getting to a v 1.0 level?
Chris:  Last week, for both specifications ‑‑ the runtime spec and the image spec ‑‑ we released v1.0 RC4, release candidate 4. In OCI parlance, what that means is there's been four release candidates, but in order for them to declare a 1.0, they need at least three.
We have the minimum requirements set. It's just a matter of when developers are happy with the stability, and when they want to bless it as a 1.0. My personal feeling is we'll see RC5 or RC6 will be the 1.0 final. That's a month or two away.
Gordon:  Obviously, the Cloud Native Computing Foundation is also a Linux Foundation. Another podcast, which I did, which will be linked to from this one, was with Dan Kohn, who's the executive director there. In your copious spare time, you work with the CNCF, as well. How do you see, the groups relating?
Chris:  I served as an executive director for CNCF, since the beginning, until Dan came on board to help out. I transitioned to a more COO‑, CTO‑type thing, so my role in CNCF is mostly focusing on attracting the projects, working with the TOC and all that stuff, and trying to build out the Cloud Native Landscape, technology‑wise, within CNCF.
In terms of how these organizations relate, there's a lot of membership overlap. There's probably 80 percent membership overlap between organizations, but OCI was defined very narrow in scope. It's mostly focused on runtime and the image format. There may be some other things that we consider in the future, but things like networking, storage, these are all out of scope. These are things that will exist in the CNCF in some fashion.
There is this awesome thing that we developed called the Cloud Native Landscape. If you go to github.com/cncf/landscape, you'll see this picture. OCI is featured in there, and that's in the aspects of the container runtime bit. That's just one piece of technology that we consider important in CNCF land, but as a whole, there's a slew of stuff out there.
The foundations work closely, lots of membership overlap. It's something we try to do, make sure people talk to each other and all that good stuff.
Gordon:  I think you've just answered this question, but I'll ask it again in slightly different way. Where do you see, from an OCI‑specific perspective, where you want to standardize and where you want to innovate?
Chris:  Like I mentioned before, we've initially focused on the runtime specification, then the image specification. That area is super important to standardize, because people want to make sure their containers are portable and will run on different things.
Other aspects that have come up within what we call technical developer community within OCI, distribution is one of those topics that people feel like there's a lot of different registries out there. Maybe it's time to come together and standardize on how containers are fetched and distributed.
That discussion's happening in the OCI TDC. But in order for them to expand the scope of the OCI, they're going to have a meeting with the Technical Oversight Board, do a vote, and all that typical, wonderful governance process that you see in open source foundations.
That may be something else that will end up in the OCI in the future, but there's no guarantees. It's whatever the TOB decides. But things like networking and storage, those will most likely be permanently out of scope for OCI. There's too much innovation going on in that space. There's lots of ways to do container networking. There's tons of ways to do storage. Those are areas we expect CNCF to take over and shepherd.
Gordon:  In terms of some of the specific work that's going on in terms of the interfaces between Kubernetes and the container stuff, CRI-O and so forth, where does that fit in here?
Chris:  CRI-O's an interesting project. Essentially, Kubernetes is doing the leg work to extract itself from the underlining container engine, which is a smart move. The Mesos community actually did this a long time ago with their external containerizer.
CRI-O is a similar effort in a similar vein. The goal is to support all the different container engines out there. OCI will be supported by default, but Docker, rkt, and all the other specific engines will be supported too with their specific flavors of additional functionality.
CRI-O will use the runtime format and the image format to support the basics, but storage, how you fetch your stuff, that's not standardized yet. They'll use other respective other technologies out there to make that happen.
Gordon:  To try and summarize here, you really see the OCI as focusing on standardization to avoid fragmentation.
Chris:  Portability. I think if you are an end user, if you're a a developer and you want to build your container and run it somewhere, you'd be really pissed if you have to rebuild it to run it on a different cloud provider. I view this like all the different browsers out there. There's Safari, Firefox, and so on, but HTML CSS, if you write a page using the latest specs from W3C, it's going to run on the different browsers.
But the different browsers compete on performance and other types of little features, developer tools. I think you'll see similar things in container tools. The container format and execution of it will be standard across everywhere, but the different engine or orchestrators will find different ways of compete on performance, security, and so on.
Gordon:  But you don't really want to have two different web pages for Internet Explorer, Firefox.
Chris:  As we've been saying, absolutely. [laughs]
Gordon:  You mentioned distribution, specifically. What are some of the other areas that you think, at least potentially, might be interesting for the OCI in the future?
Chris:  It's hard to say. I think networking could be interesting, but there's too much. It's fertile ground. There's a lot of innovation going on. There's different approaches. I don't know. It's like browsers. Over time, certain things got standardized in CSS and HTML that may have been crazy at that time, but down the road, who knows.
The next year or so, I think distribution will be a hot topic within OCI. Anything after that, it's really hard for me to say. It's up for the OCI technical development community and the Oversight Board to make these decisions. But we'll see tons of these projects in these spaces.
Over time, I think developers will get together, "OK, it's really stupid for us to like compete on maybe all these different registries. At least let's standardize on some of the basics, and compete on, you know, the other 20 percent."
Gordon:  Chris, a slightly more meta question. Historically, companies haven't wanted to standardize in most areas. It strikes me that, in the container space, the virtualization format battles that went on for years and years, there's been a lot more willingness to go, "OK, this area doesn't differentiate. We're just going to..." you know, an HD DVD, Blu‑ray thing.
We really all standardize here, and it's really happened fairly quickly.
Chris:  I think the industry has changed over the years. Open source is more prevalent. People have learned a lot of lessons around lock‑in, and they don't want to repeat the mistakes. The visualization fiasco with the format, VM, all that, that's a painful memory in a lot of people. People are worried about paying the "VMWare tax." Lots of lessons have been learned.
If you remember from the Java EE days, the whole notion of you having a WAR file, and you'd be able to deploy it on WebLogic versus WebSphere, that just worked. That was good. There was lots of "gotchas" around that, but the idea, in theory, was good.
People have learned their lessons, and I think they want to standardize on the thing that will allow the market to grow. Everyone wants containers to be super‑successful, run everywhere, build out the business, and then compete on the actual higher levels, sell services and products around that, and not try to fragment the market in a way where people won't adopt containers, because they're scared that it's not ready, it's a technology that's still [laughs] being developed.
Gordon:  I'm not sure if it's still a maturing industry or taking the desires of their customers into account. Maybe a mature industry does that.
Chris:  I think customers are smarter these days, too. I think a lot of them are making the demands, especially from our end‑users at least. I won't name specific companies, but some of the higher‑end financial companies, this is definitely 100 percent something that they ask for.
It helps them for price pressure on different providers, like, "Hey, I can move my stuff easily here." People are learning the lessons. It took a generation for us to get there, but this is something people now ask for from their vendors.
Gordon:  With containers, specifically, as you mentioned, portability is one of the promises of containers. So if you don't get portability...
Chris:  Exactly! You're locked in, and no one wants that. It's great to see the Windows folks contributing, Intel, all the OSs, architectures. Everyone is contributing to make sure that this actually will work on different platforms, so people will chose their platform, potentially, down the road.
I think it's great. Definitely the industry has changed over the years, especially if you look back a decade ago. A long time ago, it was mostly like, "Let's lock people in and secure all the profits." That's not going to [laughs] work anymore. You have to collaborate and build the business on the actual thing you want to differentiate on.
Gordon:  Great. Thanks Chris. Any closing comments?
Chris:  OCI's definitely a growing project, and I look forward to seeing the 1.0 release in a couple of months.