Eclipse IoT with Ian Skerrett of the Eclipse Foundation

For many people, Eclipse may not be the first open source organization that pops to mind when thinking about Internet-of-Things (IoT) projects. But, in fact, Eclipse hosts 28 projects that touch on a wide range of needs for organizations doing IoT projects. In September, I was attending the Eclipse IoT day and RedMonk's ThingMonk conference in London and had a chance to sit down with Ian Skerrett. Ian heads marketing for Eclipse and we spoke about Eclipse's IoT projects and how Eclipse thinks about IoT more broadly.

(Apologies for the audio quality not being quite up to my usual standards. We had to record this outside, it was a windy day, and I didn’t have any recording gear to mitigate the wind noise.)

Listen to podcast:

• MP3 [17:15]
• OGG [17:15]

Links:

• iot.eclipse.org

Transcript:

Gordon Haff:  Hi everyone. This is Gordon Haff, Technology Evangelist with Red Hat. You're in the Cloudy Chat podcast. I'm at London at ThingMonk, RedMonk's annual event on IoT. I'm here at Shoreditch Studios, and I'm pleased to be here with Ian Skerrett who runs marketing for the Eclipse Foundation. Welcome Ian.

Ian Skerrett:  Great to be here, Gordon. Thanks for having me.

Gordon:  Ian, could you start off by giving a little bit of background for yourself, how you came to be at Eclipse, and what your role is at Eclipse?

Ian:  As you said, I'm working at the Eclipse Foundation, and my official title is the Vice President of Marketing. I help bring together the community and talk about what the community is doing around the different open source projects. Lots of people don't know, but Eclipse is a huge community of well over 300 projects.

My specific role is on marketing, but I also deal a lot with IoT or IoT community which we're going to talk a bit more about. I probably spend half of my time on IoT right now.

Gordon:  Eclipse has come a long way. I'm sure everybody listening to this has heard of Eclipse, but they probably think in terms of the IDE or some other specific development things. As you say, you have a very large presence in IoT today.

Before we get into the details of, specifically, what Eclipse is doing in IoT, you gave a talk yesterday where you discussed things like Industry 4.0. That might be a useful context in order to talk about what Eclipse is doing, specifically, in IoT.

Ian:  Industry 4.0 is a term that probably started in Germany. It's reflective of how things get made, if you think about a factory floor. Lots of people know about the Industrial Revolution. The first Industrial Revolution, that was the start of steam power, steam powered machines.

The second Industrial Revolution was mass production. You think about car manufacturing, mass production around that.

The third Industrial Revolution is credited with automation and robotics that have gone into the factory plants. Where Industry 4.0 comes from is what people talk about, the fourth Industrial Revolution, is how do you start connecting all those factory floors, the machinery and automation machinery on the factory floors to the enterprise IoT system.

It's a term that comes out of Germany. Germany is the hub of industrial automation. That's where the machines that can make other machines that go into factory floors start off often. It started there. It's become an industry term that's been adopted globally and to talk about how do you start connecting up what a lot of people call the operational technology. The technology that's on the factory floor, to the enterprise IoT technology.

That's the context and one industry that IoT plays into. IoT is a general term that plays in pretty well every industry out there, be it automotive, be it wearables, be it healthcare, be it industrial automation and manufacturing.

Gordon:  Before we go further, you touched on something, which our listeners will be interested in. One of the questions I hear a lot is we've been connecting things up in factories forever. We've had various types of industrial control systems. Many of these systems have been connected within modern factories. From your personal perspective, from Eclipse's perspective, how is IoT different besides being a cool new term?

Ian:  You're right. A lot of factories are connected. A lot aren't though. There's a term called SCADA, Supervisory Control and Data Analysis. SCADA system would often be how you use IoT technology to monitor a factory. Often, SCADA systems and even the factory floor technology, is very proprietary, very siloed. It's hard to change it. It's hard to adapt to it.

One of the drivers of Industry 4.0 is that the manufacturing process is trying to be more flexible. Right now, when you set up a manufacturing run, you need to manufacture hundreds of thousands of that piece, of that unit. What they want to do is to meet customer demand, to have manufacturing processes that are very flexible, that you can actually do a lot size one.

You do an entire manufacturing process of just one unit and then change it as quickly as possible. To do that, it has to be much more flexible. Software has to be much more flexible. It has to be distributed, where the actual machines have the intelligence of what to do. That's a very new way of doing the software that's being put out on the factory floor. That's where the industry is going.

Gordon:  It's a little bit like cloud has “been there since time‑sharing,” but obviously, it's qualitatively different today.

Ian:  IoT and terms like embedded system development, a lot of this is being done. It's taking it to the next step where you can actually interoperate, where that information can run, having multiple factories talking to each other, doing data analysis across multiple factories, and just having a lot more flexibility that go to those systems.

Gordon:  Let's talk about Eclipse. As we said in the beginning, there's activity across the whole IoT spectrum. A lot of people's attention is focused on more consumer‑type stuff, SmartHome, Roombas, what have you. Obviously, there's factories, there's transportation, logistics work. Out of all that, how is Eclipse thinking about where you want to put wood in your arrow, where you want focus?

Ian:  Our goal is to be, when developers are building IoT solutions, they have set building blocks that they can draw on. An analogy I like to make is that, in early days of the web, I used to look at IBM. IBM used to have four different HTTP servers that they're trying to commercialize. They wanted to do e‑business, they wanted to have e‑commerce, and you need a web server to do that.

If you wanted a website, you needed a web server. They were trying to commercialize that. It turned out having HTTP server to sell had no value to any customer so they shut them all down, which are Apache.

What we can see in IoT is that there's core fundamental technology that every IoT solution needs, that want an open source. Everyone can use it so they can get broader adoption of it. The way we think first is that an IoT solution, there's three stacks of software.

What we see is that you need three different software stacks, building block technology for IoT. You need a stack of software for constrained devices, the MCU [microcontroller unit] level sensor type of hardware.

There's usually some type of a gateway that aggregates information and data from the different sensors and sends it to the network. You need a software stack for there.

And then a software stack for the IoT platform on the backend, on the cloud. Our goal is to be the provider of the underlying technology for those three stacks of software.

Gordon:  One of the things about IoT that seems to be a good fit with open source is this idea of modularity and gluing things together. Without going into details here, we've seen a number of things over the past year [suggesting that] a monolithic software stack that handles everything isn't the best answer.

Ian:  IoT is so broad. When you go to getting a solution done, there's very specific things that need to be built, but there's a lot of underlying technology that can be used like messaging protocol, like gateway services. It needs to be a modular approach to scale up to the different use cases that are up there.

It isn't just one big stack of software behind this. Certainly, the microservices were obviously constrained. The IoT needs to be working and moving in that direction too.

Gordon:  Now, I know you love all your children and we don't want this to be a two‑hour podcast. We're going to bore our listeners, but what are some highlights of some of the projects on your Eclipse?

Ian:  They're all amazing.

[laughter]

Ian:  No, I'm just kidding. In reality, there's a maturity of IoT business projects. Let's start with what I would consider our more mature projects that are being used in production today. Certainly around MQTT, the messaging protocol for IoT. We have two projects, Eclipse Mosquitto and Eclipse Paho. Mosquitto is the broker, Paho is the client for MQTT. Those are widely used, widely successful.

If you're doing MQQT, you probably want to look at Paho and Mosquitto. MQTT has been a great success in terms of being a standard that's being widely adopted in IoT and an open source application.

Gordon:  For our listeners, what is MQTT?

Ian:  It's a pub/sub, publish‑subscribe messaging protocol that was designed specifically for oil and gas pipeline monitoring where power management network latency is really important. You can't have an HTTP client that's always pinging home. It's got to be a pub/sub.

Another project that's very mature and well‑used is Eclipse Kura, which is an IoT gateway. Essentially, it provides northbound and southbound connectivity. There's a lot of different protocols. There are Bluetooth, Modbus, CAN bus, OPC UA. We just keep on growing the list.

Instead, you writing your own connectivity. Kura provides that and then connect you to the network via satellite, via an ethernet or anything. It handles all that and things like firewall configuration. It handles network latency. If the network goes down, it will store messages until it ever comes back up. Kura is another well‑used project from Eclipse.

We have a project in home automation area called Eclipse SmartHome. In the maker community, there is a project called openHAB. OpenHAB is based on Eclipse SmartHome. It's very well‑used and successful community for that.

Where we've been working in, probably, the last 18 months, is on cloud platform. We have a new project called Eclipse Kapua, which is taking a microservices approach to providing different services for an IoT cloud platform. That's up and coming. It's not being deployed yet, but Eurotech and Red Hat are very active in that.

One of my more intriguing projects is Eclipse hawkBit, which is for software updates. From a security perspective, if you can't update your device, you've got a huge security hole. Most of the IoT security disaster reports that you see is the fact that they couldn't just update it too. That's what hawkBit does.

HawkBit, basically, manages the backend of how you do scalable updates across your IoT system. That's interesting.

We've got 28 different projects. Do you want me to keep going, or we will stop there?

[laughter]

Gordon:  That's probably good for right now. I'm going to cap this off with a pretty typical question when I do these podcasts around community run, open source projects. How do people find out? How do people get involved? If they're not coders and they are still interested, how can they get involved?

Ian:  How to find out? We've got a website, iot.eclipse.org. Go there. That's our developer portal. We've written a white paper called the "Three Software Stacks for IoT." I'd recommend reading that to get a sense of what our view of IoT is from a software perspective.

I'd start there before getting started. We have some good getting started documentation to help people try some of the software out. We have some sandbox servers for a lot of our backend server projects.

If you want to, for instance, try out MQTT, you don't have to install Mosquitto. We have it in the system Mosquitto that's open that anyone can use. For device management, we have a device management server running called Eclipse Leshan. They're that, they're there.

As with any open source, you try it out, you give feedback, you open bugs. If you got a bug and have a fix for it, do a pull request. It's very typical open source, and I'm encouraging that. Certainly, if there's people that want to join the community, we have a working group. Organizations come together and collaborate on bringing together these projects for IoT solution developers.

If you want to start a project, if you have some technology that you think is relevant to IoT, come talk to us. We're certainly an open community and welcome other people to join us.

Gordon:  Thank you. Anything you'd like to add?

Ian:  No. It's great to see you again at ThingMonk. I'm going to put in a plug for ThingMonk in here because, I don't know about you, but I think it's an amazing show. Pretty well every talk, I learn something. I go a lot of IoT shows, and I usually don't learn much at an IoT show, but ThingMonk, I always do.

Gordon:  I'll put in a plug for RedMonk's other events as well. Great analyst firm, they do a lot of work with developers. Good guys. Used to work with a couple of them. Definitely check them out.

Red Hat's Mark Wagner on Hyperledger performance work

Mark Wagner is a performance engineer at Red Hat. He heads the Hyperledger Performance and Scalability Working Group. In this podcast, he discusses how he approaches distributed ledger performance and what we should expect to see as this technology evolves.

Podcast:

Listen to MP3 [13:45]

Listen to OGG [13:45]

Links:

Podcast with Brian Behlendorf

Hyperledger Announces Performance and Scalability Working Group

MIT Tech Review Business of Blockchain event

MIT Sloan CIO Symposium: AI and blockchain's long games

Transcript:

Gordon Haff:   I'm sitting here with Senior Principal Performance Engineer, Mark Wagner. What we're going to talk about today is blockchain, Hyperledger, and some of the performance work that Mark's been doing around there. Mark, first introduce yourself.

Mark Wagner:  My name is Mark Wagner. I'm in my 10th year here at Red Hat. My degree, from when I started many years ago, was hardware. I switched to software. I got the bug to do performance work when I saw the performance improvements I could make in software, in how things ran.

Here at Red Hat, I've worked on everything from the kernel up through OpenShift and OpenStack at all the layers. My most recent assignment is in the blockchain area.

Gordon: A lot of people probably associate blockchain with Bitcoin. What is blockchain, really?

Mark: Blockchain itself is a technology where things are distributed. I like to think of it more as a distributed database at a really high level. Bitcoin is a particular implementation of it, but in general, blockchain ‑‑ and there's also a thing called distributed ledgers ‑‑ they're fairly similar in concept, but the blockchain itself is more for straight financial things like Bitcoin.

Distributed ledgers are coming up a lot more in their uses across many different vertical markets, such as healthcare, asset tracking, IoT, and of course the financial markets, commodity trading, things like that.

Gordon: As we've really seen over the last, I don't know, year or two years, there's still a lot of shaking out going on in terms of exactly what the use case is here, which of course makes the job for people like you harder when you don't know what the ultimate objectives necessarily are.

Mark: Yes. It's shaking out in terms of both new verticals are being added, as well as there's multiple implementations going on right now, in a sense competing, but they're designed at different verticals in many cases, so that, in a true sense, not really competing, per se.

Gordon: Now you're working in Hyperledger. Introduce Hyperledger.

Mark: Hyperledger is a project in the Linux Foundation to bring open source distributed ledgers out into the world. I've been involved in it since December of 2016. Red Hat's been a member for two years.

One of the things in Hyperledger, there are multiple projects within Hyperledger. The two main ones that people know are Fabric from IBM, Sawtooth from Intel. There's a bunch of smaller projects as well to complement these technologies.

Both Fabric and Sawtooth are distributed ledger implementations with different consensus models and things like that, and getting to the point where they can do pluggable consensus models.

One of the things that no one was doing at Hyperledger, and where I felt I could help across all the projects, is performance and scalability. People see out in the world that the Bitcoin and Ethereum stuff is not scaling. When it hits scale issues, things go poorly.

I proposed in April that we have a Performance and Scale Working Group to go off, investigate this, and come up with some tests and ways to measure. It passed unanimously, but the scope was actually expanded from what I proposed, and they don't want it to just focus on Hyperledger but to focus industry‑wide.

Since that time, I've been in touch with the Enterprise Ethereum Association, with the person leading their performance and scale work. In principle, we've agreed to work together.

Gordon: I'm interested in some of the specific things that you've found in this performance and scale work. Maybe before we go into detail there, at a high level, where do you see the scalability and performance challenges with blockchain and distributed ledgers?

It's obviously early days. You've done performance work with the Linux kernel, which is about tweaking for very small increments of performance, where distributed ledgers are obviously in a very different place today.

Mark: The design of the original Bitcoin, and those technologies, is what was called proof of work. They gave you a large cryptographic hash you needed to go solve in order to prove that you actually did the work.

There were consensus algorithms based on that, and who got first and who got to build the chain and add to the chain. It quickly became people started using GPU offload or going off and fabricating FPGAs directly to give them an advantage doing this. There's a quick example of performance and scalability.

The other issue is, because it's consensus, everything gets shared. Everyone has to agree on it, or some large percentage has to agree on it. As the network grows, more and more nodes are involved in this, and it becomes a big scalability problem.

Gordon: Let's talk about the work that you've done so far. What have you been focusing on?

Mark: The Performance and Scale Working Group is really just getting started. Right now, we're trying to go through and identify three or four different vertical use cases. We're focusing more on distributed ledgers and their smart contracts, things like that.

We're trying to right now go through and identify use cases at Hyperledger. Another working group within Hyperledger has already defined. We can take those, and then say, "These are the key characteristics of those," because some of these vertical markets may not need the most transactions per second. It may be more how much you can scale.

The other interesting thing is there's two types of implementations, or deployments I should say. One is permissioned, where you need permission. That's called a private. The other is permissionless, which is public. Bitcoin is public. Anyone can join.

In the permission, you need to be invited so you can control the scale that way.

Gordon: Also, there's at least some discussion that in private distributed ledgers or blockchains, it's even possible you may not need proof of work.

Mark: Yes, a lot of it is working now towards proof of stake, where you prove that you're a stakeholder. It's less computation involved.

Gordon: Now, you mentioned it in the beginning of this podcast that you can almost think of a distributed ledger as almost a form of ‑‑ not to put words in your mouth ‑‑ distributed database. There's obviously very different performance characteristics, at least as things stand now.

How do you see that interplay of distributed databases substituting for, or instead of, or what do you see the relationship between distributed ledgers, blockchain, and distributed databases?

Mark: Distributed databases are more focused on sharing data, spreading it out. With blockchain and distributed ledgers, everyone has the same copy. People are looking at sharding now. You can go off and do just the specific set of transactions, or something like that with sharding.

It's also referred to as collections. Certain sets of nodes can go off and be involved in some transactions, others in different ones. That's one way to go around the performance and scalability.

Gordon: If you're looking back from, I don't know, five years from now or whatever, what do you think have been some of your toughest challenges that you've had to overcome in terms of improving the performance, usability, and so forth of distributed ledgers?

Mark: Five years from now, we'll look back, and we'll think how naive we were, in trying to solve some of these issues. Again, there will a big difference between public and private, but trying to come up with consensus algorithms, I think they'll keep evolving. The amount of work needed will change.

The other thing people will need to start thinking about is storage. How are you going to store all this data over time?

Gordon: What's Red Hat's interest in this?

Mark: Red Hat, right now, we have customers coming to us saying, "We like blockchain, but we'd like it to run on your enterprise‑class software."

One of the things I'm trying to do with Hyperledger is get things running on our OpenShift platform with Kubernetes with a RHEL base underneath it, looking at being able to contribute software so that it can become part of a CI environment once we get further along.

In general, right now our goal is to offer multiple blockchain solutions. Internally, we're figuring out what that means and how to do that. Right now, we're working with several.

Gordon: To your earlier "how naive we were" comment, that's one of the things we absolutely see today around blockchain, around distributed ledger, is really everyone's trying to figure out, "Where is this going to be a great fit?" Conversely, "We really thought we could use it for that? What were we thinking?"

I was at an event about a month ago, and Irving Wladawsky‑Berger, who basically ran Linux strategy for IBM when they were first developing a Linux strategy, was up in the panel on blockchain at the MIT Sloan CIO Symposium.

I think he's fairly representative of a lot of people who think that blockchain can very possibly be a very big deal, but also recognizing, Irving said we were probably in the equivalent of the 1980s Internet. It takes a long time to build out these kind of infrastructures.

Mark: That sums it up pretty well. One of the other things I heard when I first started with Hyperledger back in December at a conference in New York, was everyone agreed we're at the peak of the hype cycle, but also that it's still going to be very big.

Gordon: Actually, somebody made a very similar comment to me. It might have been the same event. They asked me where did I think it was in the hype cycle.

I actually looked up a Gartner "Emerging Technologies Hype Cycle" report and guess where blockchain was in that report? [At the peak of the hype cycle.] It scares me a little bit, but I agree with Gartner, to tell you the truth, but that was certainly their opinion.

Mark: Through my interactions here at Red Hat, I'm seeing lots of interest from healthcare, insurance. You can use this to cut down on paperwork for insurance companies, things like that.

"Here's the list of treatments that you're eligible for." The doctor goes in, says, "I did these," and he just gets paid. There's no going back through the review process, things like that.

Gordon: There certainly seem at least a lot of potential use cases out there. You have to believe that some of those are going to pan out at least.

Mark: Right.

From Pots and Vats to Programs and Apps: Coming Soon!

Monkigras in London this past January had packaging as its theme, both in the software and the more meta sense. James Governor graciously extended me an invitation to speak. The resulting talk, A Short History of Packaging (video here), described how packaging in both retail and software has evolved from the functional to something that improves the buying and using experience.

I’d been looking to write a new book for a while. I knew I wanted it to relate to the containers, microservices, cloud broadly, etc. space, but I didn’t really have an angle. I considered just rewriting my three-year old Computing Next but so much had changed and so much remained in flux that the timing didn’t feel right.

But packaging! Now there was an angle and one that I could work on together with my Red Hat colleague William Henry (who is a senior consulting engineer and works on DevOps strategy). 

So that’s what we did. We set a target to do a book signing at Red Hat Summit in early June. We mostly made it. We signed a pre-release version and have spent the past month or so working off-and-on to polish up the contents, give colleagues the opportunity to review, and update a few things based on various industry announcements and events. 

We’re finally just about ready to go. I expect to have the paperback version orderable through Amazon by about mid-July. We’ll also be making a free PDF version available at around the same time; distribution details TBD. Given the free PDF I don’t expect to release a Kindle version. The layout of the book (sidebars, footnotes, some amount of graphics) doesn’t really lend itself to this format and it would be extra work.

The thesis of the book is that if you think about packaging broadly, it highlights critical tradeoffs.

Unpackaged and unbundled components offer ultimate flexibility, control, and customization. Packaging and bundling can simplify and improve usability—but potentially at the cost of constraining choice and future options.

Bundling can also create products that are interesting, useful, and economically viable in a way the fully disaggregated individual components may not be. Think newspapers, financial instruments, and numerous telecommunications services examples.

Open source software, composed of the ultimately malleable bits that can be modified and redistributed, offers near-infinite choice.

Yet, many software users and consumers desire a more opinionated, bundled, and yes, packaged experience—trading off choice for convenience.

This last point is a critical tension around open source software and, for lack of a better umbrella term, “the cloud” in the current era. Which makes understanding the role that packaging may play not just important, but a necessity. Ultimately, packaging enables open source to create the convenience and the ease of use that users want without giving up on innovation, community-driven development, and user control.

Links for 06-13-2017

• HTML Cleaner - Word To HTML Converter
• Who invented e-mail? Inside Shiva Ayyadurai’s legal war against his critics | Ars Technica UK
• Is blockchain a security topic? – Alice, Eve and Bob – a security blog
• Steven Pinker on sentence construction, language, and today's grammar police. | New Republic
• Untitled (http://www.bizjournals.com/boston/news/2017/06/08/alphabet-to-sell-off-waltham-based-boston-dynamics.html?ana=e_bost_bn&u=9wR29cYS61Bx5Mzzgb%2FS%2BQ076de72c&t=1496978218&j=78348701) - RT @ddkirsch: Google's sale of @BostonDynamics and the failure to build #nest ecosystem reminds us that #Google is an ad company
• Red Hat and Apache OpenWhisk – RHD Blog - Red Hat refocusing FaaS/serverless efforts on Apache OpenWhisk.
• Practical Deep Learning For Coders—18 hours of lessons for free

MIT Sloan CIO Symposium: AI and blockchain's long games

I wrote earlier about the broad transformation themes at the MIT Sloan CIO Symposium last month. Today, I’m going to wrap up by taking a look at a few of the specific panels over the course of the day.

Artificial Intelligence

Andrew McAfee and Erik Brynjolfsson are regulars at this event. Their bestselling Second Machine Age focuses on the impact of automation and artificial intelligence on the future of work and technological, societal, and economic progress. Their new book Machine, Platform, Crowd: Harnessing Our Digital Future will be available later this month. Another panel, moderated by the MIT Media Lab’s Job Ito, featured discussions on the theme “Putting AI to Work.” 

Like blockchain, which I’ll get to in a bit, a common thread seemed to be something along the lines of AI and machine learning being supremely important but with much still to do. In general, panelists avoided getting too specific about timelines. Ryan Gariepy, CTO & Co-Founder, Clearpath & OTTO Motors put the timing on the majority of truck driving jobs going away as a “generation.” My overall takeaway is that AI is probably be one of those things where many people are predicting greater short-term effects than is warranted while underestimating the effects over the longer term.

For example, Prof. Josh Tenenbaum, Professor, Department of Brain and Cognitive Sciences at MIT highlighted the difference between pattern recognition and modeling. He noted that "most of how children learn is not driven by pattern recognition” but it’s mostly pattern recognition where AI is having an impact on the market today.  He went on to say that "other parts like common sense understanding we are quite far from. We’re quite a way from a conversation.The narrative that expert systems are a thing of the past is wrong. You can't build a system that beats the world's best Go players without thinking about Go. You can't build a self-driving car without driving."

Users of common “personal assistants” like Alexa have probably experienced something similar. Like a call center reading from a script, these assistants can recognize voices and act on simple command quite well. But get off script, especially in any way that requires an understanding of human behaviors, and their limitations quickly become clear.

McAfee also pointed to the confluence of AI with communications technology as a major factor driving rapid change. As he puts it “two huge things are happening simultaneously: the spurt of AI and machine learning systems and, it’s easy to forget about this, but over a decade have connected humanity for the first time. Put the two together and are in very very new territory."

As they do in their books, McAfee and Brynjolfsson also touched on the economic changes that these technological shifts could drive. For example, Brynjolfsson highlighted how “the underlying dynamics when you can produce things at near-zero marginal cost does tend to lead to winner takes all. The great decoupling of median wages is because a lot of the benefits have become much more concentrated."

Both suggested that government policy will eventually have to play a part. As McAfee put it "times of great change are not calm times. There’s a concentration of wealth and economic activity. Concentration has some nice benefits but it leaves a lot behind.” With respect to Universal Basic Income, however, McAfee added that "a check from the government doesn't magically knit communities back together. There's a role for smart policies and smart government."

Blockchain

The tone of the Trusted Data: The Role of Blockchain, Secure Identity, and Encryption panel was similar to that at Technology Review’s all-day blockchain event the prior month that I wrote about here. I’d sum it up in three bullets:

• It’s potentially very important
• Cryptocurrency existence proofs notwithstanding, as a foundational technology it’s still very early days
• Use cases and architectures are still fluid

Sandy Pentland, who moderated the panel, laid out some of the reasons why blockchain may be both useful and challenging. For example, he noted that "Data sharing is really difficult. You need to combine data from different sources that you may not own” On the other hand, "auditability is increasingly important. Are you being fair? You need to show decisions made. Existing architectures are just not up to it. Probably need consensus mechanisms like blockchain."

Hu Liang, Senior Managing Director Head of Emerging Technologies Center, State Street pointed out how some of the basic architectural elements of blockchain are still being debated. He went so far as to say that blockchain is just a fairly vague concept.” For example, he wondered whether "some things that made bitcoin popular may not be needed in an institutional world. Banks exist and regulators exist. Still get eencryption, auditability, but do you need proof of work?"

Finally Irving Wladawsky-Berger, Fellow, MIT Initiative on the Digital Economy (and long-time IBMer), framed blockchain as a transactional mechanism. He noted that "the internet never dealt with directly was transactions. Transactions are things that when they go wrong people get really really really upset. When transactions are part of interactions between different institutions it is a pain. The promise of blockchain over time is to be a record of transactions. benefits are gigantic.It  could do for transactional systems what the internet does for connections."

But it will be a slow process. “The internet of the early to mid 90s was really crappy. The internet we are really happy with today took another 15 years to get there. We're at the toddlers stage. Foundational technologies take a long time."

----

Photos:

Top. Jason Pontin, Andrew McAfee, and Erik Brynjolfsson [Gordon Haff]

Prof. Josh Tenenbaum, Professor, Department of Brain and Cognitive Sciences, MIT [Gordon Haff]

Irving Wladawsky-Berger [Gordon Haff]

Transformation at MIT Sloan CIO Symposium 2017

When I attend an event such as the MIT Sloan CIO Symposium, as I did in Cambridge yesterday, I find myself thinking about common threads and touch points. A naive perusal of the agenda might have suggested a somewhat disparate set of emerging strategies and technologies. Digital transformation. AI. Man and Machine. Blockchain. IoT. Cloud.

However, patterns emerged. We’re in such an interesting period of technology adoption and company transformation precisely because things that may at first seem loosely coupled turn out to reinforce each other. Thereby leading to powerful (and possibly unpredictable) outcomes. IoT is about data. AI is, at least in part, about taking interesting actions based on data. Cloud is about infrastructure that can support new applications for better customer experiences and more efficient operations. Blockchain may turn into a mechanism for better connecting organizations and the data they want to share. And so forth. 

We observe similar patters at many levels of technology stacks and throughout technology processes these days. New business imperatives require new types of applications. Delivering and operating these applications require DevOps. Their deployment demands new open hybrid infrastructures based on software-defined infrastructures and container platforms. (Which is why I spend much of my day job at Red Hat involved with platforms like OpenStack and OpenShift.)

That it’s all connected is perhaps the primary theme the event reinforced. In this post, I focus on the “big picture” discussions around digital transformation. I’ll cover specific technologies such as AI in a future piece.

Digital transformation on two dimensions

Peter Weill, Chairman, MIT Sloan Center for Information Systems Research (CISR) led off the day with some research that will be made public over the next few months. This research identified change associated with digital transformation as taking place on two different dimensions: customer experience (e.g. NPS) and operational efficiency (e.g. cost to income). Companies that transform on both dimensions ("future ready firms”) have a net margin 16 points higher than the industry average.

Weill emphasized that these transformations are not just about technology. “Every one in room is struggling with cultural change question,” he said. As Jeanne Ross, also of CISR put it later in the day “Digital transformation is not about technology. It’s about redesigning your value prop and that means redesigning your company."

Finally, it’s worth noting that these two dimensions mirror the two aspects of IT transformation that we see more broadly. The “bimodal IT” or two-speed IT model has somewhat fallen out of fashion; it’s often seen as an overly rigid model that de-emphasizes the modernization of legacy systems. I don’t really agree although I get the argument.

Nonetheless, the CISR research highlights a key point: Both IT optimization and next-generation infrastructures and applications are important. However, they require different approaches. They both need to be part of an overall strategy connecting the business and the business’ technology. But the specific tactics needed to optimize and to transform are different and can’t be treated as part of a single go-forward motion. 

Four decisions

Ross broke down designing for digital transformation into four decisions.

The first is defining a "vision for improving the lives of customers” because this affects what innovations will pursue.

The second decision is defining  whether you’ll be primarily focused on customer engagement (market driven) or digitized solutions (product driven).

The third decision is defining the digital capabilities will you'll pursue. Ross said that "the operational backbone is the baseline. But you also need a digital services platform that relies on cloud, mobility, and analytics.” Such a platform emphasizes "developing components rapidly and stitching them together.” (The evolution award microservices, DevOps, and container platforms is very much in response to these sorts of requirements.)

Finally, digital transformation is fundamentally about how the business is architected. "Pre-digital we architected for efficiency. In a digital economy, we architect for spped and innovations. This requires empowering and partnering.” (From the vendor side, this also mirrors the shift we see from a historical emphasis on individual products to an emphasis on ecosystems and communities. These are perhaps especially important within open source software but it’s a broader observation.)

Stay tuned for future posts about some of the more technology-oriented discussions at the event.

Podcast: Dr. André Baumgart of EasiER AG on jumpstarting app dev with Red Hat's Open Innovation Labs

EasiER AG used Red Hat's Open Innovation Labs to create a new category of healthcare product to improve the emergency room experience. Dr. André Baumgart is one of the founders of EasiER AG and he sat down at Red Hat Summit with myself and my Red Hat colleague Jeremy Brown to talk about his experiences with the process. (Spoiler Alert: He’s a big fan.)

Among the key points he makes is that the program focused on business outcomes and problems that need to be solved rather than technology stacks.

Also in this podcast, Jeremy Brown shares some highlights about what’s different about the Open Innovation Labs from a more traditional consulting engagement. 

Link to MP3 (12:43)

Link to OGG (12:43)

Links:

• Video: Dr. André Baumgart speaking at OpenShift Commons Gathering in Berlin (Healthcare Goes Mobile on OpenShift at EasierAG)
• Video: easiER AG jumpstarts modern application development with Red Hat Open Innovation Labs
• Open Innovation Labs
• EasiER AG (in German)

 

Cautiously optimistic on blockchain at MIT

Blockchain has certain similarities to a number of other emerging technologies like IoT and cloud-native broadly. There’s a lot of hype and there’s conflation of different facets or use cases that aren’t necessarily all that related to each other. I won’t say that MIT Technology Review’s Business of Blockchain event at the Media Lab on April 18 avoided those traps entirely. But overall it did far better than average in providing a lucid and balanced perspective. In this post, I share some of the more interesting themes, discussion points, and statements from the day.

It’s very early

Joi Ito, the Director of the MIT Media Lab, captured what was probably the best description of the overall sentiment about blockchain adoption when he said that we "should have a cautious but optimistic view.” He went on to say that “it's a long game” and that we should also "be prepared for quite of bit of change.” 

In spite of this, he observed that there was a huge amount of investment going on. Asked why, he essentially shrugged and suggested that it was like the Internet boom where VCs and others felt they had to be part of the gold rush.  “It’s about the money." He summed up by saying "we're investing like it's 1998 but it's more like 1989."

The role of standards

In Ito’s view standards will play an important role and open standards are one of the things that we should pay attention to. However, Ito also drew further on the analogues between blockchain and the Internet when he went on to say that "where we standardize isn't necessarily a foregone conclusion” and once you lock in on a layer (such as IP in the case of the Internet), it’s harder to innovate in that space. 

As an example of the ongoing architectural discussion, he noted that there are "huge arguments if contracts should be a separate layer” yet we "can't really be interoperable until agree on what goes in which layer."

Use cases

Most of the discussion revolved around payment systems and, to a somewhat lesser degree, supply chain (e.g. provenance tracking).

In addition to cryptocurrencies (with greater or lesser degrees of anonymity), payment systems also encompass using blockchains to reduce the cost of intermediaries or eliminating them entirely. This could in principle better enable micropayment or payment systems for individuals who are currently unbanked. Robleh Ali, a research scientist in MIT’s Digital Currency Initiative notes that there’s “very little competition in the financial sector. It’s hard to enter for regulatory and other reasons." In his opinion, even if blockchain-based payment systems didn’t eliminate the role of banks, moving money outside the financial system would put pressure on them to reduce fees.

A couple of other well-worn blockchain examples involve supply chains. Everledger uses blockchain to track features such as diamond cut and quality, as well as monitoring diamonds from war zones. Another recent example comes from IBM and Maersk who say that they are using blockchain to "manage transactions among network of shippers, freight forwarders, ocean carriers, ports and customs authorities.” 

(IBM has been very involved with the Hyperledger Project, which my employer Red Hat is also a member of. For more background on Hyperledger, check out my podcast and discussion with Brian Behlendorf—who also spoke at this event—from a couple months back.)

It’s at least plausible that supply chain could be a good fit for blockchain. There’s a lot of interest in better tracking assets as they flow through a web of disconnected entities. And it’s an area that doesn’t have much in the way of well-established governing entities or standardized practices and systems. 

Identity

This topic kept coming up in various forms. Amber Baldet of JP Morgan went so far as to say “If we get identity wrong, it will undermine everything else. Who owns our identity? You or the government? How do you transfer identity?"

In a lunchtime discussion Michael Casey of MIT noted that “knowing that we can trust whoever is going to transact is going to be a fundamental question.” But he went on to ask “how do we bring back in privacy given that with big data we can start to connect, say, bitcoin identities."

The other big identity tradeoff familiar to anyone who deals with security was also front and center. Namely, how do we balance ease-of-use and security/anonymity/privacy? In the  words of one speaker “the harsh tradeoff between making it easy and making it self-sovereign."

Chris Ferris of IBM asked “how do you secure and protect private keys? Maybe there’s some third-party custodian but then you're getting back to the idea of trusted third parties. Regulatory regimes and governments will have to figure out how to accommodate anonymity."

Tradeoffs and the real world

Which is as good a point as any to connect blockchain to the world that we live in.

As Dan Elitzer, IDEO coLAB, commented "if we move to a system where the easiest thing is to do things completely anonymously, regulators and law enforcement will lose the ability to track financial transactions and they'll turn to other methods like mass surveillance.” Furthermore, many of the problems that exist with title registries, provenance tracking, the unbanked poor, etc. etc. aren’t clearly the result of technology failure. Given the will and the money to address them in a systematic way that avoids corruption, monopolistic behaviors, and legal/regulatory disputes, there’s a lot that could be done in the absence of blockchains.

To take one fairly simple example that I was discussing with a colleague at the event, a lot of the information associated with deeds and titles in the US isn’t stored in the dusty file cabinets of county clerks because we lack the technology to digitize and centralize. They’re there for some combination of inertia, lack of a compelling need to do things differently, and perhaps a generalized fear of centralizing data. In other situations, “inefficiencies” (perhaps involving bribes) and lack of transparency are even more likely to be seen as features and not bugs by at least some of the participants.  Furthermore, just because something is entered into an immutable blockchain doesn’t mean it’s true.

Summing up

A few speakers alluded to how bitcoin has served as something of an existence proof for the blockchain concept. As Neha Narula, Director of Research of DCI at the MIT Media Lab, put it, bitcoin has "been out there for eight years and it hasn't been cracked” even though “novel cryptographic protocols are usually fragile and hard to get right."

At the same time, there’s a lot of work still required around issues like scalability, identity, how to govern consensus, and adjudicating differences between code and the spec. (If the code is “supposed” to do one thing and it actually does another, which one governs?) And there are broader questions. Some I’ve covered above. There are also fundamental questions like: Are permissioned and permission-less (i.e. public) blockchains really different or are they variations of the same thing? What are the escape hatches for smart contracts in the event of the inevitable bugs? What alternatives are there to proof of work? Where does monetary policy and cryptocurrency intersect?

I come back to Joi Ito’s cautious but optimistic.

-----

Photos: 

Top: Joi Ito, Director MIT Media Lab

Bottom: Amber Baldet, Executive Director, Blockchain Program Lead, J.P. Morgan

by Gordon Haff

DevOps Culture: continuous improvement for Digital Transformation

In contrast to even tightly-run enterprise software practices, the speed at which big Internet businesses such as Amazon and Netflix can enhance, update, and tune their customer-facing services can be eye opening. Yet a miniscule number of these deployments cause any kind of outage. These companies are different from more traditional businesses in many ways. Nonetheless they set benchmarks for what is possible. 

Enterprise IT organizations must do likewise if they’re to rapidly create and iterate on the new types of digital services needed to succeed in the marketplace today. Customers demand anywhere/anywhen self-service transactions and winning businesses meet those demands better than their competition. Operational decisions within organizations also must increasingly be informed by data and analytics, requiring another whole set of applications and data sets.

Amazon and Netflix got to where they are using DevOps. DevOps touches many different aspects of the software development, delivery, and operations process. But, at a high level, it can be thought of as applying open source principles and practices to automation, platform design, and culture. The goal is to make the overall process associated with software faster, more flexible, and incremental. Ideas like the continuous improvement based on metrics and data that have transformed manufacturing in many industries are at the heart of the DevOps concept.

Development tools and other technologies are certainly part of DevOps. 

Pervasive and consistent automation is often used as a way to jumpstart DevOps in an organization. Playbooks that encode complex multi-part tasks improve both speed and consistency. It can also improve security by reducing the number of error-prone manual processes. Even narrowly targeted uses of automation are a highly effective way for organizations to gain immediate value from DevOps.

Modern application platforms, such as those based on containers, can also enable more modular software architectures and provide a flexible foundation for implementing DevOps. At the organizational level, a container platform allows for appropriate ownership of the technology stack and processes, reducing hand-offs and the costly change coordination that comes with them. 

However, even with the best tools and platforms in place, DevOps initiatives will fail unless an organization develops the right kind of culture. One of the key transformational elements is developing trust among developers, operations, IT management, and business owners through openness and accountability. In addition to being a source of innovative tooling, open source serves as a great model for the iterative development, open collaboration, and transparent communities that DevOps requires to succeed.

Ultimately, DevOps becomes most effective when its principles pervade an organization rather than being limited to developer and IT operations roles. This includes putting the incentives in place to encourage experimentation and (fast) failure, transparency in decision-making, and reward systems that encourage trust and cooperation. The rich communication flows that characterize many distributed open source projects are likewise important to both DevOps initiatives and modern organizations more broadly.

Shifting culture is always challenging and often needs to be an evolution. For example, Target CIO Mike McNamara noted in a recent interview that “What you come up against is: ‘My area can’t be agile because…’ It’s a natural resistance to change – and in some mission-critical areas, the concerns are warranted. So in those areas, we started developing releases in an agile manner but still released in a controlled environment. As teams got more comfortable with the process and the tools that support continuous integration and continuous deployment, they just naturally started becoming more and more agile.”

At the same time, there’s an increasingly widespread recognition that IT must respond to the needs of and partner with the lines of business--and that DevOps is an integral part of that redefined IT role. As Robert Reeves, the CTO of Datical, puts it: “With DevOps, we now have proof that IT can and does impact market capitalization of the company. We should staff accordingly.”

------------------

Photo credit: http://marshmallowchallenge.com/Welcome.html

DevSecOps at Red Hat Summit 2017

We’re starting to hear “DevSecOps" mentioned a lot. The term causes some DevOps purists to roll their eyes and insist that security has always been part of DevOps. If you press hard enough, they may even pull out a well-thumbed copy of The Phoenix Project by Gene Kim et al. [1] and point to the many passages which discuss making security part of the process from the beginning rather than a big barrier at the end.

But the reality is that security is often something apart from DevOps even today. Even if DevOps should include continuously integrating and automating security at scale. It’s at least in part because security and compliance operated largely in their own world historically. At a DevOpsDays event last year, one senior security professional even told me that this was the first IT event that was not security-specific that he had ever attended.

With that context, I’d like to point you to a session that my colleague William Henry and I will be giving at Red Hat Summit on May 3. In DevSecOps the open source way we’ll discuss how the IT environment has changed across both development and operations. Think characteristics and technologies like microservices, component reuse, automation, pervasive access, immutability, flexible deploys, rapid tech churn, software-defined everything, a much faster pace, and containers.

Risk has to be managed across all of these. (Which is also a change. Historically, we tended to talk in terms of eliminating risk while today it’s more about managing risk in a business context.)

Doing so requires securing the software assets that get built and well as the machinery doing the building. It requires securing the development process from the source code through the rest of the software supply chain. It requires securing deployments and ongoing operations continuously and not just at a point in time. And it requires securing both the application and the container platform APIs.

We hope to see you at our talk. But whether or not you can make it to see us specifically, we hope that you can make it to Red Hat Summit in Boston from May 2-4. I’m also going to put in a plug for the OpenShift Commons Gathering on the day before (Monday, May 1).

-------------- 

[1] If you’re reading this, you’ve almost certainly heard of The Phoenix Project. But, if not, it’s a fable of sorts about making IT more flexible, effective, and agile. It’s widely cited as one of the source texts for the DevOps movement.

Links for 04-13-2017

• 3 open source web design templates | Opensource.com
• Fitbit’s smartwatch struggles are real and not just in hardware - The Verge
• Threads on Vimeo
• Let's Tell a Story Together
• Twitter - RT @asheshbadani: The actual list of @openshift customers speaking @RedHatSummit is even larger. Get to Boston in early April!!
• Bruce Conner's restored mushroom clouds: You can't look away - LA Times
• The Data Visualisation Catalogue
• The IoT Testing Atlas | ThoughtWorks
• Twitter - RT @dberkholz: Still waiting for the non-laptop experience to stop sucking for creators rather than consumers
• What makes us Red Hat - What Makes Us Red Hat. Really great piece. #redhat
• Twitter - RT @apanouss: “Digital transformation — brought to you by open hybrid cloud” by @ghaff
• Untitled (https://www.youtube.com/watch?v=fBZplQKKbLs&feature=youtu.be) - SNEAK PEEK: The latest #opensourcestories trailer is here. See the full film at #RHSummit.
• Speakers | Red Hat Summit 2017 - Great speaker lineup at #RHSummit I’ll also be talking on DevSecOps with @ipbabble
• Flickr - My photos from #openshift Commons Gathering in Berlin
• Connections: Upcoming: MIT Sloan CIO Symposium - Was able to wrangle schedule so I can attend MIT Sloan CIO Symposium again on 5/24. I’m hosting lunch discussion.
• OpenJDK and Containers – RHD Blog - RT @RealGeneKim: Ah: turning the JVM runtime. Still necessary, esp inside socket: Red Hat blog: OpenJDK and Containers
• Twitter - RT @kernelcdub: Amadeus describing processing 40-70B transactions per day with thousands of microservices #ONS2017
• Intel Vigorously Defends Chip Innovation Progress - "Intel Vigorously Defends Chip Innovation Progress" Good read as always from TPM.
• Thoughts and Perspectives from OpenShift Commons Berlin – OpenShift Blog - RT @joefern1: "Volvo migrated their car buying system to containers and onto OpenShift" >>Websphere on #Kubernetes in @Openshift
• Release v0.1 · kubernetes-incubator/cri-o · GitHub - RT @kelseyhightower: The cri-o 0.1 release is a huge milestone and demonstrates the value of the Kubernetes container runtime interface.

Podcasts: Talking cloud native projects at CloudNativeCon in Berlin

Eduardo Silva, Fluentd/Treasure Data

A project within the Cloud Native Computing Foundation, Fluentd is focused on logging, pulling together data from a variety of sources and sending it to a back-end. Eduardo Silva spoke with me at CloudNativeCon in Berlin about Fluentd and its flexible architecture for plug-ins. Fluentd is widely used for tasks like aggregating mobile stats and to understand how games are behaving.

Listen to MP3 (15:10)

Listen to OGG (15:10)

Miek Gieben, CoreDNS

CoreDNS, which provides cloud-native DNS server and service discovery, recently joined the CNCF. In this podcast Miek provides  context about DNS and explains how today’s more dynamic environments aren’t always a good match with traditional approaches to DNS. Miek takes us through how CoreDNS came to be and discusses some possible future paths that it might take.

Listen to MP3 (12:24)

Listen to OGG (12:24)

Björn Rabenstein, Prometheus/SoundCloud

Bjorn Rabenstein of SoundCloud sat down with me at CloudNativeCon in Berlin to discuss Prometheus, the first project to be brought into the Cloud Native Computing Foundation after Kubernetes. Prometheus is a popular open-source monitoring system with a dimensional data model, flexible query language, efficient time series database and modern alerting approach. In this podcast, we get into the background behind Prometheus, why new monitoring tools are needed for cloud-native, and when you should wake people up with an alert--and when you shouldn't.

Listen to MP3 (16:38)

Listen to OGG (16:38)

Sarah Novotny, Kubernetes/Google

Sarah Novotny does open source community for Google Cloud and is also the program manager of the Kubernetes community. She has years of experience in open source communities including MySQL and NGINX. In the podcast we cover the challenges inherent in shifting from a company-led project to a community-led one, principles that can lead to more successful communities, and how to structure decision-making.

I’ve written an article with excerpts from this podcast which will appear on opensource.com. I’ll link to it from here when it’s available.

Listen to MP3 (20:54)

Listen to OGG (20:54)

Upcoming: MIT Sloan CIO Symposium

My May schedule has been something of a train wreck given a Red Hat Summit in Boston (use code SOC17 for a discount) that’s earlier than usual and generally lots of events in flight. As a result, I didn’t know until a couple of days ago whether I would be able to attend this year’s MIT Sloan CIO Symposium on May 24. I always look forward to going. This is admittedly in part because I get to hop on a train for an hour ride into Cambridge rather than a metal sky tube for many hours.

But it’s also because the event brings together executives who spend a lot of time focusing on the business aspects of technology change. As you’d expect from an MIT event, there’s also a heavy academic component from MIT and elsewhere. Erik Brynjolfsson, Andrew McAfee, and Sandy Pentland are regulars. As I have for the past few years, I’ll be hosting a lunchtime discussion table on a topic TBD as well as covering the event in this blog afterwards. 

Data, security, and IoT at MIT Sloan CIO Symposium 2016

MIT Sloan CIO Symposium 2015: Dealing with Disruption

This year the Symposium will focus on the theme, “The CIO Adventure: Now, Next and… Beyond,” and will provide attendees with a roadmap for the changing digital landscape ahead. Among the associated topics are challenges of digital transformation, talent shortages, executive advancement to the C-suite, and leading-edge research.

Here’s some additional information from the event organizers:

The full agenda is available at www.mitcio.com/agenda. Highlights include:

Kickoff Panel: “Pathways to Future Ready: The Digital Playbook” will discuss a framework for digital transformation and facilitate a conversation on lessons learned from executives leading these transformations. Virtually every company is working on transforming their business for the digital era and this panel will provide a playbook for digital. Featuring Peter Weill, Chairman, MIT Sloan Center for Information Systems Research (CISR); Jim Fowler, Vice President & Chief Information Officer, General Electric; David Gledhill, Group Chief Information Officer and Head of Group Technology & Operations, DBS; and Lucille Mayer, Head of Client Experience Delivery and Global Innovation, BNY Mellon.

Fireside Chat: “Machine | Platform | Crowd: Harnessing Our Digital Future” will be moderated by Jason Pontin, Editor-in-Chief and Publisher of MIT Technology Review and feature Erik Brynjolfsson, Director, and Andy McAfee, Co-Director, of the MIT Initiative on the Digital Economy (IDE), discussing what they call "the second phase of the second machine age." This phase has a greater sense of urgency, as technologies are demonstrating that they can do much more than just the type of work we have thought of as routine. The last time new technologies had such a huge impact on the business world was about a century ago, when electricity took over from steam power and transformed manufacturing. Many successful incumbent companies, in fact most of them, did not survive this transition. This panel will enable CIOs to rethink the balance between minds and machines, between products and platforms, and between the core and the crowd.

Other panel sessions driven by key IT leaders, practitioners, and MIT researchers will include:

“The Cognitive Company: Incremental Present, Transformational Future”; “Cloud Strategies: The Next Level of Digital Transformation”; “The CIO Adventure: Insights from the Leadership Award Finalists”; “Preparing for the Future of Work”; “Expanding the Reach of Digital Innovation”; “Running IT Like a Factory”; “Navigating the Clouds”; “Winning with the Internet of Things”; “Talent Wars in the Digital Age”; “Who’s Really Responsible for Technology?”; “You Were Hacked—Now What?”; “Measuring ROI for Cybersecurity: Is It Real or a Mirage?”; “Putting AI to Work”; “Trusted Data: The Role of Blockchain, Secure Identity, and Encryption”; and “Designing for Digital.”

Links for 03-17-2017

• GDC Vault
• A Guide to Raleigh, N.C.'s Rooftop, Hidden and Underground Bars
• How to Succeed in Business by Bundling – and Unbundling - Much of the business story of the digital age so far has been about taking products and institutions apart — unbundling them. Music CDs were unbundled into MP3s that were sold (and illicitly downloaded) individually. Newspapers have been unbundled by blogs and classified ad sites.
• Publishing in the Age of the Assembled Web | Open Parenthesis
• Harold "Doc" Edgerton
• Physicist declassifies rescued nuclear test films | Lawrence Livermore National Laboratory
• Twitter - Thanks! @monkigras talks in general also further cement long-held belief that many conferences would be better with…
• The Class | PrecisionStory
• Why do we have blood types? | Mosaic
• Frozen Alive | Outside Online
• Boston Smart City Playbook — from the Mayor's Office of New Urban Mechanics
• Housing insights | Jed Kolko
• On snot and fonts / Luc Devroye

Video: A Short History of Packaging

From Monkigras London 2017

We’re in the middle of big changes in how we bundle up software and deliver it. But packaging didn’t start with software. I take you on a tour of how we’ve packaged up goods for consumption over time and--more importantly--why we did so and what different approaches we’ve taken then and now. The goal of this talk is to take the packaging discussion up a level so to better focus on the fundamental objectives and some of the broad approaches and trade-offs associated with achieving them.

Final Open Source Leadership Summit podcast roundup

I recorded a number of podcasts at the Open Source Leadership Summit in Lake Tahoe last month. Most of them are with heads of the various foundations under the Linux Foundation. They’re each about 15 minutes long. In addition to the podcasts themselves linked to from the blog posts, five of them have transcripts and two have stories on opensource.com. 

opensource.com

Heather Kirksey, Open Platform for Network Functions Virtualization (OPNFV) 

"Telecom operators are looking to rethink, reimagine, and transform their networks from things being built on proprietary boxes to dynamic cloud applications with a lot more being in software. [This lets them] provision services more quickly, allocate bandwidth more dynamically, and scale out and scale in more effectively."

Mikeal Rogers, node.js

"The shift that we made was to create a support system and an education system to take a user and turn them into a contributor, first at a very low level and educate them to bring them into the committer pool and eventually into the maintainer pool. The end result of this is that we have a wide range of skillsets. Rather than trying to attract phenomenal developers, we're creating new phenomenal developers."

Connections with transcripts

Brian Behlendorf, Hyperledger

 "That's what gets me excited is these positive social impacts that at the same time, are also potentially helping solve structural problems for the business sector. I haven't seen that kind of synergy, that kind of combination of value from these two different things since the early days of the Internet."

Dan Kohn, Cloud Native Computing Foundation (CNCF)

"When you have those developers that feel like their contributions are valued and taken seriously, then there's a whole ecosystem that forms around them, of companies that are interested in offering services to them, employing them, that want to make these services available to other folks. Then a foundation like ours can come up and help make those services available. I really think that, that developer focus is the key thing to keep in mind."

Nicko Van Someren, Core Infrastructure Initiative (CII)

  "Going forwards, we're trying to move to probably put more into the strategic stuff, because we feel like we can get better leverage, more magnification of the effect, if we put money into a tool and the capabilities to use that tool. I think one of the things we're looking at for 2017 is work to improve the usability of a lot of security tools.There's no shortage of great tools for doing static analysis or fuzz testing, but there is often a difficulty in making it easy for you to integrate those into a continuous test process for an open‑source project. Trying to build things to make it easier to deploy the existing open‑source tools is an area in the strategic spin that we want to put a lot into in 2017."

Chris Aniszczyk, Open Container Initiative (OCI)

 "People have learned their lessons, and I think they want to standardize on the thing that will allow the market to grow. Everyone wants containers to be super‑successful, run everywhere, build out the business, and then compete on the actual higher levels, sell services and products around that, and not try to fragment the market in a way where people won't adopt containers, because they're scared that it's not ready, it's a technology that's still [laughs] being developed."

Al Gillen, IDC

 “With container technology and the ability to produce a true cloud‑native application that's running on some kind of a framework which happens to be available on‑prem or in cloud, you suddenly have the ability to move that application on‑prem or off‑prem, or both ‑‑ run in both places at the same time if so you choose ‑‑ and be able to do that in a way that's been unprecedented in our industry."

Connections

In addition to the above podcasts with foundation directors and analysts, I also sat down with Josh Bernstein, VP of Technology, and Clint Kitson, Technical Director for {code} by Dell EMC to talk about open source and communities.

 

 

Links for 02-28-2017

• rclone - rsync for cloud storage
• Practical Deep Learning For Coders—18 hours of lessons for free
• 6 R's of a Cloud Migration - SPR
• Standage on VR
• Monki Gras 2017: Claire Giordano - Packaging - and the Gone Girl with the Dragon Tattoo on the Train - YouTube - This talk on packaging by @clairegiordano at @monkigras was great! (Loved the title too>)

Podcast: Talking open source and communities with {code} by Dell EMC

Josh Bernstein, VP of Technology, and Clint Kitson, Technical Director for {code} by Dell EMC sat down with me at the Open Source Leadership Summit to talk about their plans for this strategic initiative.

{code} by Dell EMC

Audio:
Link to MP3 (00:13:22)
Link to OGG (00:13:22)

Podcast: Security and Core Infrastructure Initiative with Nicko Van Someren

As the CTO of the Linux Foundation, Nicko Van Someren also heads the Cloud Infrastructure Initiative. The CII was created in the wake of high visibility issues with widely-used but poorly funded open source infrastructure projects. (Most notably, the Heartbleed vulnerability with OpenSSL.) In this podcast, Nicko discusses how the CII works, his strategy moving forward, and how consumers of open source software can improve their security outcomes.

In addition to discussing the CII directly, Nicko also talked about encouraging open source developers to think about security as a high priority throughout the development process--as well as the need to cultivate this sort of thinking, and to get buy-in, across the entire community.

Nicko also offered advice about keeping yourself safe as a consumer of open source. His first point was that you need to know what code you have in your product. His second was to get involved with open source projects that are important to your product because "open source projects fail when the community around them fails."

Core Infrastructure Initiative, which includes links to a variety of resources created by the CII

Audio:
Link to MP3 (00:15:01)
Link to OGG (00:15:01)

Transcript:

Gordon Haff:   I'm sitting here with Nicko van Someren, who's the CTO of the Linux Foundation, and he heads the Core Infrastructure Initiative. Nicko, give a bit of your background, and explain what the CII is?

Nicko van Someren:  Sure. My background's in security. I've been in the industry‑side of security for 20 plus years, but I joined the Linux Foundation a year ago to head up the Core Infrastructure Initiative, which is a program to try and drive improvement in the security outcomes in open‑source projects. In particular, in the projects that underpin an awful lot of the Internet and the businesses that we run on it. The infrastructural components, those bits of open source that we all depend on, even if we don't see them on a day‑to‑day basis.

Gordon:  Around the time that you came in, you've been in the job, what, a little over a year, is that right? There were some pretty high visibility issues with some of that infrastructure.

Nicko:  Yeah, and I think it goes back a couple of years further. Around three years ago, the Core Infrastructure Initiative ‑‑ we call it the CII ‑‑ was set up, largely in the wake of the Heartbleed bug, which impacted nearly 70 percent of the web servers on the planet.

We saw a vulnerability in a major open‑source project, which had very profound impact on people across the board, whether they were in the open‑source community, or whether they were running commercial systems, or whether they were building products on top of open source. All of these people were impacted by this very significant bug.

While the community moved swiftly to fix the bug and get the patch out there, it became very apparent that as the world becomes more dependent on open‑source software, it becomes more and more critical that those who are dependent on it support the development of those projects and support improving the security outcomes of those projects.

Gordon:  Many of the projects that we're talking about there, was a tragedy of the commons sort of situation, where you had a few volunteers ‑‑ not being paid by anyone, asking for donations on their PayPal accounts-- who, in many cases, were responsible for these very critical systems.

Nicko:  Absolutely. Probably trillions of dollars of business were being done in 2014 on Open SSL, and yet in 2013, they received 3,000 bucks worth of donations from industry to support the development of the project. This is quite common for the projects that are under the hood, not the glossy projects that everybody sees.

The flagship projects get a lot of traction with a big community around them, but there's all of this plumbing underneath that is often maintained by very small communities ‑‑ often one or two people ‑‑ without the financial support that comes with having big businesses putting big weight behind them.

Gordon:  What exactly does the CII do? You don't really code, as I understand it.

Nicko:  Well, I code in my spare time, but the CII doesn't develop code itself, for the most part. What we do is, we work to identify at‑risk projects that are high‑impact but low‑engagement.

We try to support those projects with things like doing security audits where appropriate, by occasionally putting engineers directly on coding, often putting resources in architecture and security process to try to help them help themselves by giving them the tools they need to improve security outcomes.

We're funding the development of new security testing tools. We're providing tools to help projects assess themselves against well‑understood security practices that'll help give better outcomes. Then, when they don't meet all the criteria, help them achieve those criteria so that they can get better security outcomes.

Gordon:  In terms of the projects under the CII, how do you think about that? What's the criteria?

Nicko:  We try to take a fairly holistic approach. Sometimes we're investing directly in pieces of infrastructure that we all rely on, things like OpenSSL, Bouncy Castle, GnuPG, or OpenSSH, other security‑centric projects.

But also things like last year, we were funding a couple of initiatives in network time, those components that we're all working with, but we don't necessarily see at the top layer. We're also funding tooling and task framework, so we have been putting money into a project called Frama‑C, which is a framework for C testing.

We've been funding The Fuzzing Project, which is an initiative to do fuzz testing on open‑source projects and find vulnerabilities and report them and get them fixed.

We've been working with the Reproducible Build project to get binary reproducibility of build processes, so the people can be sure that when they download a binary, they know that it matches what would have been built if they downloaded the source.

We're also funding some more educational programs, for instance, the Badging Program allows people to assess themselves against a set of practices which are known good security practices, and they get a little badge for their GitHub project or for their website if they meet those criteria.

We have a Census Project, where we've been pooling different sets of data about the engagement in projects and the level of bug reporting and the quickness of turn‑around of bug fixes, and the impact of those projects in terms of who's dependent on it, and try to synthesize some information about how much risk there is.

Then, publish those risk scores and encourage fixes. We're trying to take a mixture of some fairly tactical approaches, but also have investment in some strategic approaches, which are going to lead to all open‑source projects getting better security outcomes in the long run.

Gordon:  How do you split those? Certainly, some of the projects, particularly early on, it was very tactical, "There's frankly a house fire going on here, and it needs to be put out."

Then, some of the things that you're doing in terms of the assessment checklists and things like that, that feels much more strategic and forward‑looking. How do you balance those two, or if you could put a percentage, even, "Oh, I spend 30 percent of my time doing this?"

Nicko:  That's, of course, the perennial question. We have finite resources and huge need for this. Resource allocation is what I ask input from my board members for how they think. We, historically, have had a fairly even split between the tactical and the strategic.

Going forwards, we're trying to move to probably put more into the strategic stuff, because we feel like we can get better leverage, more magnification of the effect, if we put money into a tool and the capabilities to use that tool. I think one of the things we're looking at for 2017 is work to improve the usability of a lot of security tools.

There's no shortage of great tools for doing static analysis or fuzz testing, but there is often a difficulty in making it easy for you to integrate those into a continuous test process for an open‑source project. Trying to build things to make it easier to deploy the existing open‑source tools is an area in the strategic spin that we want to put a lot into in 2017.

Gordon:  As we also look forward at some of the areas that are developing in this point, Automotive Grade Linux, for example, AirNav's things, there's new vectors of threats coming in, and areas of infrastructure that maybe historically weren't that important from a security perspective are becoming much more so. What's on your radar in that regard?

Nicko:  I think, obviously, one of the biggest issues that we're facing going forwards is with Internet of Things. I think we have been seeing a lot of people forgetting all the things that we've learned in desktop and server security over the years, as they rush into getting things out there, Internet‑connected.

Often, it's easy to have a good idea about Internet‑connecting something and building a service around it. It's less easy to think about the security implications of doing that in a hasty manner.

We've been talking with a number of players in this space about, "How do we adapt some of the programs we've already built for improving the security process in open‑source projects to apply those to the development of IoT devices?" I think that we can do quite a lot in that space, just with the tools we've already got, tuning them to the appropriate community.

Gordon:  Anything else that you'd like to talk about?

Nicko:  One of the biggest issues that we face is improving the security outcomes in open source is to encourage open‑source developers to think about security as a high priority, as high a priority as performance or scalability or usability.

We've got to put security up there as one of the top priority list items. We also have to make sure that, because most open‑source projects get developed in a very collaborative way with a community around them, that you get buy‑in to that taking it as a priority across the whole community.

That's the best first step to getting good security outcomes, is to have people think about security early, have them think about it often, and have them keep it as a top‑of‑mind priority as they go through the development process. If they do that, then you can get very good security outcomes just by using the same practices we use everywhere else in software engineering.

Gordon:  In one of the areas I work around DevOps and continuous integration and application platforms, like one of the terms that's starting to go off currency is a DevSecOps term, and the push‑back of that is, "Oh, we know security needs to be in DevOps." Well, if you know it, it doesn't happen a lot of the time.

Nicko:  I think that's true. I think it's a question of making sure that you have it as a priority. At my last company, I was actively involved in doing high‑security software, but we were using an agile development process.

We managed to square those two by making sure the security was there in the documentation as the definition of done. You couldn't get through the iterative process without making sure that you were keeping the threat models up to date and going through the security reviews.

Code review ought to involve security review as well as just making sure that the tabs are replaced by four spaces. We need to integrate security into the whole process of being a community of developers.

Gordon:  One other final area, and it's probably less under the purview of something like the CII, but as we've been much talking about in this conference, open source has become pervasive, and that's obviously a great thing.

It also means that people are in the position of grabbing a lot of code ‑‑ perfectly legally ‑‑ from all kinds of different repositories and sticking it into their own code, and it may not be the latest version, it may have vulnerabilities.

Nicko:  Absolutely, and I think, key to keeping yourself safe as a consumer of open source...

Well, there are probably two things there. One is you need to know what you've got in your products, whether you built them yourself or whether you brought them in, there's going to be open source in there.

You need to know what packages are in there, you need to know what versions of packages are in there. You need to know how those are going to get updated as the original projects get updated. That whole dependency tracking needs to be something that you think about as part of your security operations process.

The other bit is, get involved. Open‑source projects fail when the community around them fails. If you want a good security outcome from the open‑source projects that you use, get involved. Don't just complain that it doesn't work, come up with a good diagnose bug report and file it.

Maybe produce a patch, and even if you don't produce the patch that gets accepted, you've given them the idea for how to fix it, and they'll go and recode it in their own style. If you're going to be dependent on the security of this project, put an engineer on it.

Get involved in these projects, and that's the way to make sure that you get really good security outcomes, is for people who care about the security of these products to get involved.

Gordon:  Well, I think that's as good a finish as any! Thank you.