What I'm keeping my eye on for computing's future

A bit over a year ago, I gave a presentation at the Linux Foundation Member Summit, where I took a look at some of the technologies catching a lot of attention and how the might develop. At the time, a lot of this was based on various work around Red Hat, including Red Hat Research, where I was working at the time. Think of this as both a fleshing out of the presentation, for those who weren't there, and a minor updating.

Here's the list. It mostly has to do with software and hardware infrastructure with an emphasis on open source.  https://bitmason.blogspot.com/2020/04/podcast-if-linux-didnt-exist-would-we.htmlI'll be diving more deeply into some of the individual technologies over time. I don't want to turn this into a 10,000 word+ essay but mostly want to lay out some of the things I currently see as notable.

First, let me say that tech development has frequently surprised us. Here are a few items I came up with:

• Microsoft Windows on servers (as seen from ~1990) and, really, Microsoft was going to nominate everything, right?

It looked poised to dominate. But it "merely" became important. Colorful language from Bryan Cantrill on this particular topic. And, really, it transitioned to Azure over time.

• Linux and open source (as seen from mid-90s)

Linux was a curiosity and, really, so were things by BSD and the Apache Web Server until the first big Internet buildout got going in earnest. (And probably even then as far as enterprises were concerned until IBM made its very public Linux endorsement.

• Server virtualization (as seen from 1999)

Server virtualization had been around for a long time. VMware made it mainstream. But it didn't become a mainstream production server tech until the economy soured and tech spending dried up to a significant degree at the beginning of the aughts.

• Public cloud (as seen mostly incorrectly from mid-2000s)

Public clouds more or less came out of nowhere even if they had antecedents like timesharing. 

• Back-end containers/orchestration/cloud-native (from ~2010)

This came out of nowhere too from the perspective of most of the industry. Containers had been around a long time too. But they mostly settled into a largely failed lightweight alternative to virtualization role. Then Docker came along and made them a really useful tool for developers. Then Kubernetes came along and made them essentially the next generation alternative to enterprise virtualization—and a whole ecosystem developed around them.

• Smartphones (as seen from ~2005)

I won't belabor the point here but the smartphones of 2005 were nothing like, and much less interesting, than the Apple and Android smartphones that came to dominate most of the industry.

The heavy hitters

These are the technology and related areas I'm going to hit on today. A few others for a future time. I also wrote about artificial intelligence recently although it impinges on a couple other topics (and, indeed, maybe all of them):

• Data
• Automation
• Heterogeneous infrastructure

Data

Data touches on so many other things. The enormous training datasets of AI, observability for intelligent automation, the legal and regulatory frameworks that govern what data can be used, what data can be opened up, and even what open source really means in a world where data is at least as important as code. Security and data sovereignty play in too.

Let's break some of this down starting with security and data sovereignty.

There are a ton of subtopics here. What am I specifically focused on? Zero trust, confidential computing, digital sovereignty, software supply chain… ? And what are the tradeoffs?

For example, with confidential computing, though early, can we project that—in addition to protecting data in rest and in transit—it will increasingly be considered important to protect data in use in a way that makes it very hard to learn anything about a running workload.

As digital sovereignty  gains momentum, what will the effects on hyperscalers and local partnering requirements be—and what requirements will regulations impose?

There are major open legal questions around the sources of training data. After all, most of the contents of the web—even where public—are copyrighted non-permissively. Not a few people consider the use of such data as theft though most of the IP lawyers I have spoken with are skeptical. There are ongoing lawsuits however, especially around the use of open source software for tools like GitHub's CoPilot.

There is also a whole category of license geek concerns around what "open" means in a data context, both for AI models and the data itself. Some of these concerns also play into releasing anonymized data (which is probably less anonymous than you think) under open licenses.

Automation

Although some principles have remained fairly constant, automation has changed a lot since it was often lumped into configuration management within the domain of IT. (I'm reminded of this on the eve of post-FOSDEM CfgMgmtCamp in Ghent, Belgium which I attended for a number of years when Ansible was often viewed as another flavor of configuration management in the vein of Puppet and Chef, the latter two being mostly focused on managing standard operating environments.)

The bottom line is that complexity is increasingly overwhelming even just partially manual operations. 

This is one of the many areas where AI is playing into. While early days—with the AI term being applied to a lot of fairly traditional filtering and scripting tools—nonetheless AIOps is an area of active research and development.

But there are many questions:

Where are the big wins in dynamic automated system tuning that will most improve IT infrastructure efficiency? What’s the scope of the automated environment? How much autonomy will we be prepared to give to the automation and what circuit breakers and fallbacks will be considered best practice? 

Maybe another relevant question is: What shouldn’t we automate? (Or where can we differentiate ourselves by not automating?) Think, for example, overly automated systems to serve customers either in-person or over the phone. Costs matter of course but every exec should be aware of systems causing customer frustration (which are very common today). Many aspects of automation can be good and effective but, today, there's a ton of sub-par automation determined to keep enabled humans from getting involved.

Heterogeneous infrastructure

I was maybe 75% a processor/server analyst for close to 10 years so some of the revitalized interest in hardware as something other than a commodity warms my heart. 

Some of this comes about because simply cranking the process on x86 no longer works well—or at least works more slowly than over the past few decades. There are still some levers in interconnects and potentially techniques like 3D stacking of chips but it's not the predictable tick-tock cadence (to use Intel's term) that it was in years past.

GPU's, especially from NVIDIA and especially as applied to AI workloads, have demonstrated that alternative more-or-less workload-specific hardware architectures are often worth the trouble. And it's helped generate enormous value for the company.

Cloud APIs and open source play into this dynamic as well in a way that helps allow a transition away from a single architecture that evolves in lockstep with what independent software vendors (ISVs) are willing to support. Not that software doesn't still matter. NVIDIA's CUDA Toolkit has arguably been an important ingredient of its success.

But the twilight of the CMOS process scaling that has helped cement x86 as the architecture almost certainly goes beyond CPUs. Google's Tensor Processing Units (TPUs) and the various x86 Single Instruction Multiple Data (SIMD) extensions have not had the impact of x86 but there are interesting options out there. 

RISC-V is an open-standard instruction set architecture (ISA) being explored and adopted by major silicon vendors and cloud providers. This article from Red Hat Research Quarterly discusses RISC-V more deeply particularly in a Field Programmable Gate Array context.

More broadly, although RISC-V, so far, has been deployed primarily in relatively small, inexpensive systems, the architecture is fully capable of both server and desktop/laptop deployments. Though a few years old—so don't pay too much attention to the specifics—this interview I did with RISC-V's CTO gets into a lot of the approach that RISC-V is taking that applies today.

While longer term and more speculative, quantum computing is another important area of hardware innovation. A quantum computer replaces bits with qubits—controllable units of computing that display quantum properties. Qubits are typically made out of either an engineered superconducting component or a naturally occurring quantum object such as an electron. Qubits can be placed into a "superposition" state that is a complex combination of the 0 and 1 states. You sometimes hear that qubits are both 0 and 1, but that's not really accurate. What is true is that, when a measurement is made, the qubit state will collapse into a 0 or 1. Mathematically, the (unmeasured) quantum state of the qubit is a point on a geometric representation called the Bloch sphere.

While superposition is a novel property for anyone used to classical computing, one qubit by itself isn't very interesting. The next unique quantum computational property is "interference." Real quantum computers are essentially statistical in nature. Quantum algorithms encode an interference pattern that increases the probability of measuring a state encoding the solution.

While novel, superposition and interference do have some analogs in the physical world. The quantum mechanical property "entanglement" doesn't, and it's the real key to exponential quantum speedups. With entanglement, measurements on one particle can affect the outcome of subsequent measurements on any entangled particles—even ones not physically connected.

While a lot of attention has focused on the potential impact of quantum on cryptography, it's more broadly imagined (assuming various advances) to potentially increase efficiencies or even to solve problems that are just not practically solvable in many fields.

Conclusion

Broadly, there's this idea that we'll go beyond x86 to what goes by various names but amount to aggregating various types of resources dynamically to meet workload needs. Those resources will be increasingly diverse: To optimize for different workloads, to work with large amounts of data, and to automate wherever it makes sense to do so. 

The Sinking of the Itanic: free ebook

Throughout my stint as an IT industry analyst during the 2000s, one of my significant interests was Intel's Itanium processor, a 64-bit design intended to succeed the ubiquitous 32-bit x86 family. I wrote my first big research report on Itanium and it seemed like something of an inevitability given Intel's dominance at the time.

But there were storm clouds. Intel took an approach to Itanium's design that was not wholly novel but it had never been commercially successful. The dot-com era was also drawing to a close even as Itanium's schedule slipped out. Furthermore, the initial implementation was not ready for primetime for a variety of reasons.

Especially with the benefit of hindsight, there were other problems with the way Intel and its partner, Hewlett-Packard, approached the market with Itanium as well. Itanium would ultimately fail, replaced by a much more straightforward extension to the existing x86 architecture.

This short book draws six lessons from Itanium's demise:

• Lesson #1: It’s all about the timing, kid 
• Lesson #2: Don’t unnecessarily proliferate risk
• Lesson #3: Don’t fight the last war
• Lesson #4: The road to hell is paved with critical dependencies
• Lesson #5: Your brand may not give you a pass
• Lesson #6: Some animals can’t be more equal than others

While Itanium is the study point for this book, many of the lessons are applicable to many other projects. 

Download your free PDF ebook today.

RISC-V with CTO Mark Himelstein

RISC-V is an open instruction set architecture that's growing rapidly in popularity. (An estimated two billion RISC-V cores have shipped for profit to date.) In this podcast, I sat down with Mark Himelstein, the CTO of RISC-V International, to talk about all things RISC-V including its adoption, how it's different from past open hardware projects, how to think about extensibility and compatibility, and what comes next.

• RISC-V International

Listen to the podcast [MP3 - 22:54]

[TRANSCRIPT]

Gordon Haff: I'm very pleased to have with me today Mark Himelstein, who's the CTO of RISC-V, who just got off having a summit in San Francisco that I was pleased to be able to attend in person.

Welcome Mark. Maybe you could just introduce yourself and maybe give a brief overview of what RISC-V is.

Mark Himelstein: I'm Mark Himelstein. I'm the CTO. I've been in the industry for a bit. I was an early employee of MIPS, I ran Solaris for Sun. I've done a lot of high-tech stuff, and I've been with RISC-V for about a year and a half. Very excited. This was an incredible year for us, a very big change for us.

First of all, we believe that there's been well over 2 billion RISC-V cores deployed for profit this year which is an important thing. Success begets success and adoption begets adoption.

A lot of people joined us early on and they're early adopters, and now, you're seeing people say, "Oh, they're successful now. I can be successful."

RISC-V is an instruction set architecture kind of halfway between a standard and open source Linux, kind of right in between there. We don't do implementations. We're totally implementation-independent. We work with other sister organizations that are nonprofit like lowRISC, and CHIPS Alliance, and Open Hardware who do specific things in hardware with RISC-V.

We just really work on the ISA -- the instruction set architecture -- and we work on fostering the software ecosystem. All compilers, runtimes, operating systems, hypervisors, boot loaders, etc., all the things that are necessary for members to be successful with RISC-V.

It's a community of probably about 300 institutions and corporations. There's probably over 2,000 individual members, somewhere around 55 groups, doing technical work, about 300 active members in those groups and about 50 leaders.

They just did an incredible job this year ratifying 16 specifications. In 2020, we did one, so a very big growth for us. A lot of things that have been hanging out there for some time, four to six years, things like Vector, Scalar Crypto, very innovative things as well as some some basic stuff like hypervisor and bit manipulation.

We finally got the standard out, so everybody's grateful for that.

Gordon: I want to talk about standards a little bit more in a moment. You mentioned this open ISA. What was the thinking behind taking this approach? Because obviously, there have been earlier, open hardware or semi-open hardware types of projects, which haven't necessarily had a big impact, or at least not as big an impact as maybe some people had hoped they would have at the time.

How is RISC-V different?

Mark: Yeah, it's a really good question. One of the problems when you hand something whole cloth as open source, is it's hard for people to really feel ownership around it. The one thing that Linux did was everybody felt a pride of ownership. That was really hard to do.

We are the biggest open source ISA that was born in open source. Unlike the other ones, we were actually born in open source. People are afraid that if one of these big corporations goes away, that's behind them, then the open source will go away, the actual standard will go away. Rightfully so, we've seen that occur before in the past.

RISC-V comes along, and it's different. Krste Asanović at Berkeley wanted to do some stuff. The story was, he was wanting to do some vector work and Dave Patterson had done RISC I, II, III, IV. They came up with this RISC-V, and with a V

V doubles as RISC-V and vector, and start off doing this. All of a sudden, there's this groundswell of people who are interested in it. It got so exciting for folks that in 2015, they started plotting how to make it an open source organization, and they did in 2016. It's just taken off from there. People have been dying for this.

It's very clear. There's flexibility with respect to pricing. It's free. More importantly, it's also flexibility with respect to customize. You can do anything you want with it, nobody's standing over your shoulder.

We provide places for people to do custom opcodes and codings and stuff like that. It's set up for extensibility. We believe that it will last for a long time because you can extend it over and over and over again, as we did this year, we added vector, we added these other things.

It's extensible. It's free. It's flexible to use any way you want to. We've also had a renaissance in EDA over the last 15 years.

It's a lot easier to pump down a bit of logic to go off and do, hey, some security module using a RISC-V core, where it may have been harder to do that around the year 2000. That's gotten easier. This combination of things has been incredible.

You see adoption and you see deployment of products more in the IoT embedded space because the runway is shorter. It's not a general-purpose computer. You're running one application, you get it working.

Wearables and industrial controllers and disk drives and accelerators that go into data center service for AI and ML graphics. All those things, you're seeing them first. Then, the general-purpose computers come out a little bit later.

Accepting there's always exceptions, Alibaba announced at the summit last year that they have a cloud server based on they have their next-generation coming out.

You see RISC-V in every single part of computer science, from embedded to IoT to Edge to desktop to data center to HPC. I even have a soldering iron made by pine64.org that has a RISC-V processor.

Gordon: To this point about extensibility, there was a fair bit of discussion at the RISC-V Summit over, centrally, fragmentation versus diversity. This idea that you have all these extensions out there, but if people use them willy-nilly, then you're breaking compatibility.

I know there are some specific things like profiles and platforms that are intended to address that potential issue to some degree. Could you discuss this whole thing?

Mark: Yeah. I have a bumper sticker statement that says, "Innovate. Don't duplicate." That's the only thing that keeps us together as a community. Why do you want to go ahead and implement addition and subtraction for the thousandth time? Why do you want to implement the optimizers for addition and subtraction the thousandth time? You don't.

The reason why so many people are coming to the table as part of the community with a contributor culture that was built by Linux.

Why are they showing up? Why are they doing work? They're doing it because they realize they don't want to do it all. It's too expensive to do it all. There are many, either countries or companies or whatever, that were doing unique processors themselves because the licenses or the flexibility were available in other architectures.

They don't want to do their own stuff. The same reason why people didn't want to get hooked into Solaris or AIX. All those things that are going to Linux have gone to Linux.

Is the same reason why the coders in RISC-V, they don't want to be beholden to our company, they want the flexibility and the freedom to prosecute their business the best way that they see fit, and we allow them to do that.

Now. they want to share, how are we going to have them share? We have the same thing that shows up with something like Linux, in that we have to make sure that there are versions that work together.

We've done the same thing, on the same way that you have generational sets of instructions that work together, either by a version number or a new product name or a year.

We have the same thing with us, with profiles. RVA is the application profile, RVM is the microcontroller bare-metal profile. They'll both be coming out almost every year, initially, and probably slower as time goes on.

RVA 20 is the stuff that was ratified in 2019. RVA 22 is the stuff that was ratified in 2021. It works for all applications. We can tell the distros, we can tell the upstream projects like the compilers, GCC, LLVM, this is what you go after.

Everybody knows, all the members know. If they're going to do something unique and different, they have to support that themselves. If they want to negotiate with the upstream projects, we don't get in the way, they can go ahead and do that.

The upstream projects know the profiles that are most important. The platforms are very similar, but for operating systems. We want to show it to be able to create a single distro, a single set of bits, people download and configure and work. Things like ABI's, things like Discovery, things like ACPI, all those things are found in the platform.

The same thing will happen, it will come out on a yearly basis. There's, again, an application layer platform, and there's a microcontroller for real-time OSs and bare-bones things. As you might imagine, the bare bones both in the profiles and platform, very sparse.

There's not much in there, because people don't want you to do a whole lot to the point where we had the M extension previously, and that M extension had multiply and divide. They don't divide. It's too expensive in IoT, so we're breaking it down.

We're going to have a separate multiply extension that people can go ahead and use. Both of them are optional down the bottom. We've provided a way that all the upstream things can go ahead and deal with it, all the distros can deal with it. Then, people can jump on board and use those things.

Ultimately, the goal is simple, be able to take the application that was compiled for one implementation and have it run on another implementation, have them produce the same results within the bounds of things like timing and other things like that.

Same thing as operating systems, one set of bits will be able to download multiple implementations, configure it, and have it work. That's how we're working on constricting fragmentation and giving you a tool to be able to do it. Again, the only reason for people who want to fragment is so that they can share.

Gordon: It was Dave Patterson who made a comment in "Meet the board" before the RISC-V Summit for a lot of uses. You alluded to this with IoT devices. The certain microprocessor compatibility like you've had with that x86, is often not the right lens through which to look at RISC-V. It can be, of course, but it isn't, necessarily.

Mark: Even those guys want to share things. They're not going to want to do their compiler from scratch, but they're using the base 47 instructions, instead of all the rest of the extensions. They don't care about those because of exactly what you said.

Again, the thing that brings people together are common things that they have to do over and over again. I'll give you one very simple example, working on something called Fast Interrupts right now, what does it mean? It's shortening the pathway to get into an interrupt handler, not having to save and restore all the registers for embedded.

That's what it's for. Very simple. All the embedded guys are in there, even though they're doing their own thing. They want to agree on one set of calling conventions and make it easy for them to do that.

That's not something that they're using for interoperability between their parts. That's something they're using, so they don't have to duplicate the work between the companies.

Gordon: Let me ask you a couple of related questions. The first of them is, where were the initial wins for RISC-V? A related question is, have there been wins with RISC-V that you didn't expect?

Mark: First of all, remember, we don't collect any reporting information. We don't require that somebody tell us how many cores, what they're used for, or anything like that. Anything we get is anecdotal.

The other thing is we don't announce for anybody. It's not our job to do that. We'll help amplify. We have a place on the RISC-V website for everybody to advertise for free? Whether I remember or not, called the RISC-V Exchange. All that's wonderful.

The stuff we hear is when we have side meetings at conferences, like the summit and stuff like that. We know that there's more design wins and deployments that we know of in the IoT embedded space, again, because of the runway. It's not a general-purpose computer.

One that's exciting that people may not realize, is that a lot of the earbud manufacturers, especially out of China, are using RISC-V as their core. One is called Bluetrum, now remember, probably tens of millions of units per month with RISC-V cores. That's exciting to me.

I think that again, it's one of those things where it shows off the ability to take a RISC-V core, do something with it quickly, and get it out there. I have in my house 85 WiFi connected devices with switches and outlets and doorbells and gates and garages and all that stuff. 10 percent of them are Espressif.

Espressif, again, a member. They have gone ahead and gone and produced the RISC-V. You can see the RISC-V module, home automation stuff. There's a lot of things that are showing up and a lot of places that we may not hear about right away.

We hear about secondarily, that are A, a surprise, but B, exciting, and C, what it does is it engender success. When people see other people being successful doing this, they go and say, "Hey, I can do this, too." I think that that's amazing.

Again, you're going to see this continue up the chain. There are exceptions like Alibaba doing their cloud server, the servers are a little bit further out. The HPC guys are actively working in European processor initiative, Barcelona Supercomputer Center. All those guys are working on stuff. We know that the United States government in various places is working on things.

The gentleman who runs our technology sector committee, this guy named John Liddell from Tactical Labs in Texas. He works with various government organizations, and has simple things like Jenkins rigs to do test for RISC-V and stuff like that.

There's a lot of work that goes in various areas, but I don't think there's a single part of computer science that isn't looking at RISC-V for something or another, whether it be a specialized processor to help them do security or processing for ETL, or something like that, or something that's a general-purpose thing. It's everywhere.

You're going to see more and more products come out over time. We're not the only ones who are taking a look at how much it's coming out. All the state or analysts have patent numbers, and they're predicting 50 billion to 150 billion range of cores out there in a very short period of time. It's going to grow as people see that it's an easy thing to do.

Gordon: What is your role at RISC-V? What do you see your primary mission as being?

Mark: I like to make things simple. The most important thing for me is the proliferation of RISC-V cores for profit. That has to be the thing that stays in your mind. In the short term, my goal is to get people over the goal line with the pieces they need to get over the goal line with.

In 2020, we produced one spec, in 2021 we did 16. That's through the effort of me and everybody else in the team in order to prioritize, put governance in place, get them help where they needed help, and try to push things over the goal line. Get those specs out there that the members care about in order to make their customers successful.

Then, finally, the ecosystem. Look, without compilers, without optimizers, without libraries, without hypervisors, without operating, it's just, it doesn't matter. It doesn't matter how good your ISA is. Having all those pieces there is really important.

I'm a software guy, and they hired a software guy to do this job because of that. I've worked in the NSA, but I understand software everywhere from boot loaders up to applications.

I've worked all those pieces. It's really critical, and you're going to see us provide even more emphasis over that. That's been the greatest growth area in our groups over the last year, and you're going to see continued effort by the community.

Gordon: I think you've maybe just kind of answered this, but if you look out in a year, two years, what does success look like or conversely, what would you consider to be flashing alarm lights or bells going off?

Mark: One of the things that we haven't done up until now is really put a concerted effort after industries. A lot of it has been really bottoms up, "Hey, we need an adder, right? We need multiply. We need vector, right?" Those are things we go, "Hey, other architectures have this."

Now, we're really starting to take a look from the board, to the steering committee, down through the groups at things like automotive, at things like data center, at finance, at oil and gas, at industries and trying to take a look holistically at what they need to succeed.

Some of it's going to be ISA. Some of its going to be partnering with some of these other entities out there. Some of it's going to be software ecosystem. The goal is to not peanut butter spread our efforts to a point where nobody can be successful in any industry, right?

It's important we say, "OK, you're doing automotive." All of a sudden, you have to look at ASIL and all these ISO standards, functional safety, blah, blah, blah, and we have to make sure that stuff occurs. We have a functional safety SIG by the way.

Success, to me, looks like continued deployment of cores that are sold for profit, and then starting to attack some of these industries holistically that need these pieces and make sure that all the pieces they need inside of RISC-V are there and working and completed.

Gordon: Well, thank you very much. Is there anything else you'd like to add?

Mark: Well, again, I think the biggest thing is just a big thank you to you and the rest of the community of being inquisitive and participating and joining the contributor culture, and helping make RISC-V a success. We're always looking for people to help us and join us, so look at riscv.org. If you have any questions, send mail to help@riscv.org. Thank you very much.

Gordon: Other than just going to riscv.org, are there any particular resources that somebody listening to this podcast might want to consider looking at?

Mark: If they're very tech, under the riscv.org, there's a tech tab. Underneath there, there's a tech wiki. That sends pointers to GitHub with all the specs, to the upstream projects, GCC, LLVM, our governance, all those things. It gives you a really good jumping-off point. There's a getting started guide there as well for tech guys.

In general, if you're not a member, become a member. It's really easy. If you're an individual, you can become a member for free. If you're a small corporation just starting out, we have some breaks. There's different levels of membership, strategic, premier TSC, premier. Come join us. Help us change the world. This is really different.

I had no clue what this was when I joined it. I'm very grateful, and I'm very happy to see it really is making a very big difference in the world.