Tuesday, August 25, 2020

Red Hat's William Henry on grid, containers, and orchestration

William Henry is a senior distinguished engineer at Red Hat who has been involved with open source since Slackware. Part of this involvement included the development of open source grid software such as Condor.

In this podcast, we discuss:

  • How supercomputing has evolved
  • What grid software was designed to do
  • How container orchestration and grid did and do relate
  • Why containers for HPC

[Transcript]

Gordon Haff:  This is Gordon Haff, technology evangelist with Red Hat, for another episode of the "Innovate @Open" podcast. Today, we're going to talk about containers, grid, high performance computing, container orchestration and more.

I have with me my colleague, William Henry, who has a whole lot of experience in this area.

William, why don't you introduce yourself?

William Henry:  I am William Henry. I'm a distinguished engineer at Red Hat. I had multiple roles with Red Hat over my last 12 years here. One of the exciting times that I had was as Red Hat were transitioning from essentially a Linux platform company to an expanded role within enterprises and adding value on top of Linux with some other open source projects in the area of messaging and real time, Linux, and grid.

Gordon:  Let me start this off with a little bit of context setting because the term grid has been used throughout computing history for a number of different things, including, for instance, some of the early peer to peer computing.

To be clear, when I'm talking and we're talking about grid here, we're really referring to it in the sense of high performance computing, resource management, and scheduling.

William, can you take us through some of the history of this technology?

William:  From my perspective, it was a breakaway from the whole large supercomputer based approaches. As very large organizations either in the defense field or medical field or whatever, required huge amounts of computing.

As they started looking at the costs of supercomputing and noticing the pricing on the cheaper versions of this around distributed computing and they saw the growth of distributed computing along with networking technologies, etc., there was a shift from...there are certain problems we can try to solve by putting a lot of different processors to work across a network to solve problems.

That proved to be a very effective approach particularly within the areas of things like seismic analysis in the oil business or scientific analysis in the medical field, weather related stuff, and also in other areas of the military, and research areas. Then we saw move into things like the movie animation and CGI world, as well.

It was trying to solve a number of different problems. One was that distributed supercomputing problem. Then there was more about high throughput computing, on demand computing, which was more similar to what we're seeing now in some of the container world. Data intensive computing, and then also collaborative computing as well, working with scientific teams.

That's where I see it coming from. It's an evolution out of supercomputing to more cost effective at larger scale initiatives, and then moving from the scientific world into a more commercial based environment.

Gordon:  If you look at the list of the large supercomputers over the years, one of the things you saw, as we went from a world where these vector supercomputers dominated by the likes of Cray to network computers. Initially, these were often proprietary interconnects, or at least there was a real mass of different interconnects.

Large RISC systems and increasingly, though, you still do see RISC systems there as well, these are "commodity x86 systems" connected by a couple of standard interconnects, specifically Ethernet and InfiniBand.

William:  It was funny. I remember on a visit, it was 10 years ago, down to New Mexico to see a DreamWorks based rack system that they were using for doing animation. Seeing it in the same ex Intel prefab room, massive room in the corner beside the New Mexico supercomputer. The DreamWorks' one was much larger.

Gordon:  We fast forward to, I guess, maybe five years or so ago, containers are coming in    now containers, of course, being a technology that's been around for a long time but containers really becoming this very important technology for computing broadly.

Docker at the time was useful for developers on their laptops and small systems, but if you were going to have applications made up of hundreds, thousands of containers, there needed to be some job scheduling system, resource management system, what have you. As we all know, Kubernetes has become the dominant orchestration technology.

When this whole thing was jumping off, in fact, it wasn't clear that there was just one layer there because you had this orchestration layer as we tended to call it, that was Kubernetes. Then there was also this idea of resource management, which wasn't really quite the same thing. Can you talk a bit about that?

William:  This was one of the things I noticed early on. Although there was a lot of parallels to grid, and we're seeing those parallels moving much more closely together now that we're several years into the container technology based stuff, originally, you could see some differences.

With the large grid computing efforts, they were focused more on having a well defined cluster of resources that were quite often hardware based. Later on, we saw some virtualization and stuff coming into that, but for the most part, I have a large cluster of hardware, I can add it to that to cluster, etc.
Although that might be difficult depending on the grid technology I'm using, but really, I want to try to share those resources at the start within one organization, and then more broadly to multiple organizations.
You had the idea of trying to give a certain amount of resources, let's say, 20 percent of the resources to this particular research field, and another 10 percent of the resources to that field. Of course, originally, it would be just one group. I'm running this huge big job, I'm going to take over the supercomputer today, and do it.

Then it was like, hold on, how do we divvy it out, and provide fair shares between each of the different groups, while at the same time, building the different organizations as well. It started getting very complicated.

Most of the jobs, when you think of it, were very batch oriented. They were like, "I need to take over the grid today to run this particular large workload of batch computational stuff, or data intensive analysis work." Then, they evolved even more to, "I could set up a workflow of this work."

It's getting very complex with resource management, fair shares, costing back to different departments, because these were expensive resources. The idea of, data versus high computation, and also how do we get workflow into this.

That contrasted with what we saw originally in the container world which was essentially using a cluster of hardware completely abstracting that away into abstract concepts of CPU percentages, or whatever, or memory percentages, and trying to keep the workload up.

It was often about, "I want to be able to run this application which might have been a website, etc. I want to run it for forever. By the way, can you scale it up a bit on high demand and scale it back down again?" It was a different kind of workload on a similar concept of a cluster.

Back then, originally, it would be a very homogeneous set of machines. Then as different university resources were saying, "I could add our departments' computers over onto this grid technology for use." Then it started getting complex, in terms of, describing, "This is a particular type of chipset base. It's got this operating system running on it."

It also became very complex into, what type of workloads, where are you going to schedule. In other words, "I have this huge pool of resources, but my workload needs to run on Linux, or it needs to run on Solaris, or it needs this particular type of hardware base."

It became very complex in terms of the matchmaking efforts that were going on to get things up. This machine has GPUs. This machine has a desktop. Are we adding our desktops at night to the grid?
During the day, the animation artists are working on these machines, but at night, we don't want them idle. Are these Windows machines versus Linux machines? There was a huge amount of effort going into doing this crazy amount of matchmaking that the resource manager would have to do.
There was matchmaking. Then there was this idea of giving out and saying, "I'm going to give you a claim on that particular machine at this sort of time," or there was booking machines in advance kind of concepts.

It became very complex. Originally, there was very expensive applications that were doing this Platform LSF or things like that. Then there was efforts to try to do open source with taking an academic project like Condor from up in University of Wisconsin.

They had a Condor project. Red Hat helped open source that. The more we looked into it, we saw just, "This was fantastic stuff, hugely complex matchmaking capabilities." It was pretty amazing. We learned a lot on that. Then, of course, you had the container revolution.

Gordon:  Yeah. I love that. There was this huge vision around grid computing around 2000 or so where people imagined this kind of computing everywhere type of thing, as you say, peer to peer computing. Intel was very big on this. The Intel CTO at the time referred to ppeer-to-peer possibly going to be bigger than the Internet.

However, to come back to our earlier discussion, a lot of this ended up more consolidating into job scheduling across a large, fairly fixed cluster of computing resources. Fast forward, back to containers.

William:  Probably 2013.

Gordon:  Containers are getting big. They're becoming more widespread. You had Kubernetes coming onto the scene. You had a variety of other open source resource management projects out there, some of which were basically competitors to Kubernetes, others of which were probably closer to next generation HPC resource managers of various sorts.

Yet, today, we seem to have largely decided we're just going to do Kubernetes.

William:  Yeah. I was in the middle of this transition and caught a little bit on the wrong side of it at one point. Let me explain.

Having come from this grid technology background with Condor and working with some very large clusters    we're talking 25 30,000 nodes or whatever, doing massive amounts of work across multiple different movies and stuff in the animation world, for example    the container technology starts coming on. We already had our own scheduler.

Remember, when we started this with OpenShift and Red Hat, we had already started using containers, but they weren't the OSI model or the Docker model back then. They were just using raw Linux containers our way. We call the ability to start those cartridges. Then we had a technology called GearD, which was to schedule them.

It was all open source stuff, because that's all we do at Red Hat. It was our open source projects that we were hoping to get some backing on.

Very quickly, we saw the Kubernetes community jump on board with the Docker format at the time. Then Google introduced Kubernetes. We saw it. We saw similarities between its approach and the GearD approach.

We saw some advantages that it has concepts like the pods in Kubernetes that seem to make sense. We decided to jump on that.

Now, that technology was based on the use case that I mentioned earlier, which was around, how do we get developers building apps that are very consumer focused applications, web based applications? How do we get those up and running and keep them running in clustered environments of Linux resources?
That was our approach. When we saw Kubernetes, we jumped on board with that.

There was another technology around at the time that was popular, another project    an Apache project, it's still around    called Mesos, which had a bunch of frameworks on top of it that were looking at the ability to schedule workloads.

Also, at the same time, you had things like the Hadoop environment which were doing data intensive work.

There was a struggle at the start around which workloads are we going to be supporting. Can we support within the Kubernetes world this concept of very much, almost a batch oriented work, the workloads we were talking about earlier on the grid? There was a lot of tension there.

The tension was, where do we want to put our resources? Remember too, at the same time as Kubernetes is going, Docker themselves are introducing this concept of Swarm. Its project called Swarm, that it's some other juggling around as well. That would confuse people, didn't help them. I don't think a lot.
Kubernetes has started. Mesos is taking off. At that time, a Mesos world or Mesos fest or whatever it was called was much bigger than anything that was going on in the Kubernetes world. People were super excited about it and all of the different frameworks on top of that for doing the data intensive workloads.
Mesos is also looking at what Kubernetes is doing around clustering. There was a, "Where do we want to put these resources? Where do we want to put our engineering resources?"

Kubernetes is also looking at batch work. Remember, these batch workloads we talked about earlier are very complex. They're in what I would call workflows.

In other words, let's do this amount of analysis, thousand CPUs doing this analysis. If it was all successful, then scheduling the number of workloads needed to do to work on the output of that in the next step to doing the next step of a workflow, etc., then consolidating some results at the end. That whole workload stuff didn't have a place in Kubernetes.

They had the concept of getting an application up and keeping it up and scaling it up and down as needed, but essentially always having it running. This whole idea of the replication controller to make sure that we have this resource available.

Whereas if you think of it, the Kubernetes concept of a job was essentially saying, "Hey, let's run this, but it's OK if it dies. You don't have to keep it up." In other words, let it run to the end. If it dies, don't try to restart it because it's a job and you expect it to die. That's about as complex as it was getting on that.
At that time the Kubernetes community, and certainly Red Hat, decided we needed to not focus on the batch state analytics world. We really needed to win the battle within the Kubernetes world. Kubernetes was going to be the future.

At the time, there were certain large customers that we were looking at. We were looking at the Mesos world of numbers, trying to propose that we try to look at both of those. This is where I got caught on the wrong side of it. There was a certain amount of consensus to maybe we should support both Mesos and Kubernetes.

Thankfully, the folks on the OpenShift side of our house decided, "Nope. Kubernetes is it. Kubernetes is the future."

Kubernetes won at Red Hat and as we've seen, Kubernetes has won in the marketplace as well.
There's still a lot of workload that we can talk about that needs to be supported or is beginning to be supported from that grid side of the house. That old high performance computing side or our data intensive computing side, or whatever you want to call it. Still a lot of work to be done on that, but Kubernetes is going to happen there.

Gordon:  Why might you want to use containers in HPC?

William:  This is why I think you're going to see a much more increase, or you are seeing a larger increase of this in the HPC world. You can look around it from the AI machine learning perspective, which is where a lot of this is falling right now. But when you think of it, that's really sometimes complex algorithms on data. It's AI ML stuff is where those workloads are falling right now.
What happens is, with the container side of this, I get a much more flexible...All the matchmaking that we have to do before becomes a lot more simpler. It's all Linux for a start. It's all going to be running on an abstracted way, very homogeneous world of compute.

Let's look at this from a container perspective now, similar to what we did before. Remember, these scientists before were writing an application that they knew their environment, what their environment was. It was the job of the complex matchmaker to figure out how to schedule the jobs on the right machines. The application developer didn't care too much, because they were writing to their chosen platform.

In this container world, everyone's writing to the same platform, they are writing into Linux containers. There's a much greater pool of developers who are out there working on this. You can containerize your application and, therefore, provide more amount of agility, in terms of, updates around that.
The scheduling becomes very simpler, because really, all you're looking for now is the CPU memory type of slices that we're looking at. Maybe GPUs, depending on the type of workload. There's been a lot of work with the Kubernetes community around GPUs.

The idea of containerizing it allows the developer to have a lot of freedom, and it provides a much more homogeneous world for the scheduling of that workload. You're seeing then things like Open Data Hub, which are putting their machine learning and AI algorithms inside of the container world to run on this Kubernetes cluster.

Gordon:  One of the things we're seeing here is really part of a long ongoing story which is enterprise computing and HPC, which used to be totally different worlds, have been coming closer and closer together really for the last two three decades or so.

William:  It's fair to say, the world we're living in now where you bring the hybrid cloud infrastructure, along with containers with AI ML open source projects and all that good...It's like bringing the supercomputer to the masses.

Before, this was in the realm of large government, certain massive well funded research labs, or biotechnology labs, or whatever it might be. Now, if I'm a small company somewhere, I can bring up a number of resources on a public or private cloud, and run a workload on an open source AI ML. Suddenly, I have a supercomputer, just consumable on an hourly basis or whatever, at my fingertips.

Now, any department in an enterprise can quickly spin it up.

What I think is amazing is the speed, the type of effort and work that was done around this to get your workload up years ago on a grid computer. It was a large project and a huge effort. Now you're hearing about the campaign website approach to supercomputing problems.

I've been a little bit facetious there, but the idea that a small funded project can spin up an AI ML learning project on something. Then we're seeing it right now with the COVID 19 crisis, where everyone seems to be coming bringing to bear the grid or the Kubernetes concepts to bear on this problem at very low cost.

Gordon:  Maybe to close out, what do you see as next steps for Kubernetes orchestration in order to continue to accommodate these other kind of workloads which came from a different direction.

William:  All of the pieces are there, and certainly the knowledge is there across the communities that are involved in this thing. You're seeing the AI ML space. We know about distributed computing, we know about all of the different parts of this. The computer science part of this is already there in various areas.

What I'm seeing is the next step is streamlining. We're not at the automatic workflow from beginning to end with this. We're still pulling together some pieces.

Now, some enterprises have done this already, but, in terms of the mass model for this problem, out on the IoT devices connecting into your edge devices. Being consumed on the enterprise platform across hybrid cloud infrastructure, when all the scheduling, and the billing, and all that good stuff. All the pieces are there, but we still are doing these as projects that are different in every deployment at the moment, but it's becoming more similar.

Open source initiative that brings it all, obviously, I personally think it'll be in the Kubernetes community, but that brings that whole streamlining of this together in a cohesive manner as opposed to, let's pick this project here, and that project there, and then let's all work together and trying to connect them all.
It's great that it's open source, and it's great that they're open API. We can do it and it's going to be low cost. Great, but wouldn't it be nice to have that tie down for everybody right now.

Is that where you're seeing it too, maybe, Gordon?

Gordon:  Yeah, and you mentioned Open Data Hub earlier, and that's a good example of pulling all of this open source goodness really together, and trying to make it more consumable.

William:  I still think that the workflow tools, it's probably another area of the framework that needs to become more mainstream as well. Not just running AI ML workload, but the idea of being able to plan out the different layers in the workflow. I'm sure there's projects that I've lost track of in that area too. Probably in Data Hub.

That, plus the seamless console that allows you to see where data is coming from into the process. Instead of the gathering and the analytics being done on a, you know, "Let's gather, and now let's bring it over to this." If that workflow is streamlined too, where the gathering or the data is almost as CI/CD or DevOps y as the backend enterprise stuff is at the moment.

We have the applications in CI/CD but it's almost like we need to connect it all together from the data gathering. It's scary, but it's also that higher level view of it all into one pattern would be really useful.








Sunday, July 26, 2020

50 authors, 50 science fiction stories shorter than novels

I've been playing around with re-reading favorite science fiction short stories while also seeing if there's anything that has managed to elude me over the years. I've also tried to catch up with more recent authors given that I don't read as many books as I used to. (The market for science fiction short stories, in general, is in something of a decline but there are still some fabulous recent stories out there.)

A few rules I set for myself:
  • One story per author. Certainly there are many on this list for whom I could effortlessly reel off multiple deserving entries. A side effect of this rule is that it brought a fair bit of organic diversity to the list which would otherwise have been more heavily populated by a handful of favorite authors.
  • Nothing is novel length but you'll find plenty of novelettes and novellas alongside the short stories. (But I've tried to favor shorter works all other things being equal.)
  • I also didn't include authors whose novels I enjoy but just don't have any short stories I'm aware of that wowed me. 
  • Everything here can be plausibly categorized as science fiction although some hew far closer to traditional sf tropes than others that are at least adjacent to horror, fantasy, and philosophical fiction.
I used various lists and recommendations to tickle my memory and to ferret out works I might not be familiar with. But I've read everything here and don't think I've included anything just because it's "expected." These are personal favorites, or at least a coin flip among favorite stories by a given author. I did aim for some overall variety of time period, style, and subject matter but I think anything here can be enjoyed by a modern reader without making too many allowances.

Many of these stories can be found online and others in popular "best of" collections.

Anyway, my initial list (sorted chronologically) is below. While it's one story per author I do include pointers to other works in the comments. I've been careful to err on the side of avoiding spoilers but if you want to go into the stories completely cold, by all means, just work from this list of titles and authors.

"A Dream of Armageddon" H. G. Wells (1901)

This story comes from prolific Victorian writer Herbert George Wells. Wells wrote in a variety of genres including social commentary and history. However, he's most remembered for his science fiction novels like War of the Worlds (which was famously made into a radio play as well as films of varying levels of quality). This story used the fictional device of a man recounting dreams of a future time in which he is a political figure who has withdrawn from life to live with a younger woman on the island of Capri, even though this withdrawal is enabling opponents to start a great war with futuristic aircraft. Wells would later build on the concept for his novel The Shape of Things to Come which would itself serve as the loose basis for a film of the same name in 1936.

"With the Night Mail" Rudyard Kipling (1905)

Rudyard Kipling would win the Nobel Prize in Literature a couple years after this story was published. Unlike Wells, Kipling is not remembered for science fiction; rather he's mostly associated with stories and poetry often set in colonial India, where he was born and lived much of his early life. "With the Night Mail" is one of only two science fiction stories that Kipling wrote. Both concern the workings and activities of The Aerial Board of Control, a 21st century fictional organization that manages the dirigible and other air traffic for the whole world and acts as a de facto world government. 

"The Machine Stops" E. M. Forster (1909)

Rounding out this trio of Victorian English authors, Forster's reputation also comes in part from a novel, A Passage to India, that concerns the relationship between East and West as seen through the lens of India in the latter days of the British Raj--as well as other novels that primarily focused on class differences in England. However, Forster wrote that "'The Machine Stops' is a reaction to one of the earlier heavens of H. G. Wells." [The Eloi of The Time Machine.] In this novella, humanity lives underground and relies on a giant machine to provide its needs. There are technologies similar to video conferencing and the Internet. But the machine begins to fail and no one remembers how to fix it. This story is widely considered to be one of the most prescient short works of early science fiction.

"At the Mountains of Madness" H.P. Lovecraft (1936)

H.P. Lovecraft's primary reputation is as a horror writer. However, his stories that were part of the Cthulhu Mythos (such as this one) borrow premises often associated with science fiction such as alien invasion, other dimensions, and interference with human cultural and physiological evolution. For example, this novella popularized ancient astronaut theories as it details the events of a disastrous expedition to the Antarctic continent and what was found there by a group of explorers led by the narrator, Dr. William Dyer of Miskatonic University. 

"Microcosmic God" Theodore Sturgeon (1941)

Originally published in the magazine Astounding Science Fiction, this story is an early example of the use of the 'pocket universe' concept in science fiction. A biochemist develops a synthetic life form which lives (and invents) far more quickly than humans can. This biochemist, Kidder, is their "god" who exerts his power over them and profits from them. Sturgeon is not generally very well known but this particular story regularly and deservedly appears in collections and best of lists.

"The Library of Babel" Jorge Luis Borges (1941)

This short story, originally written in Spanish by Argentine author and librarian Jorge Luis Borges, is  more of a philosophical exploration than a science fiction story in the usual sense. It asks you to imagine a library whose books contain every possible combination of letters and numbers. Every great work is there, the account of your death is there, every possible scientific discovery that can be written down exists in the library. But any true information would be buried in, and rendered indistinguishable from, all possible forms of false information and complete gibberish. 

"The Weapon Shop" A. E. van Vogt (1942)

This story was developed from a much shorter 1941 story, "The Seesaw." It was, in turn, used as the basis for a portion of the 1951 fix-up novel The Weapon Shops of Isher. Another closely related novel published during the same period was The Weapon Makers. These books revolve around the immortal Robert Hedrock who once created the Weapon Shops as a force to counteract the imperial world government long dominant on earth. These two novels bring together van Vogt's style of hard science fiction and transcendent superheroes more than his other books. However, this short story may do an even better job on its own.

"Mimsy were the Borogoves" Lewis Padgett (1943)

Padgett was the pseudonym of spouses Henry Kuttner and C. L. Moore when they were writing together, often to humorous effect. This story title was inspired by a verse from “Jabberwocky,” a poem found in the classic novel Through the Looking-Glass by Lewis Carroll. It imagines mysterious educative toys timeslipped in from the future, which leads to the children discovering them to start thinking in new patterns and communicating in strange ways. "Twonky" and "The Time Locker" are other Padgett stories for which they're often remembered.

"Arena" Frederic Brown (1944)

Brown, who also wrote detective fiction, was noted for stories that were written with, what The Encyclopedia of Science Fiction calls "professional economies of effect." This short story is one of those. It tells of the settling of an interstellar war through single combat between a human and an alien. Either the story name or the plot summary may seem familiar to Star Trek fans; it was a popular episode during the first season of the classic series. Brown experimented with narrative styles. "The Sentry" is a micro-short which may seem more predictable today than when it was written. 

"A Logic Named Joe" Murray Leinster (1946)

This story, originally published under Leinster's real name (William F. Jenkins) contains an early description of a computer (called a "logic") in fiction. In the story, he imagined the Internet in many respects. He envisioned logics in every home, linked through a distributed system of servers (called "tanks"), to provide communications, entertainment, data access, and commerce; one character says that "logics are civilization."

"The Lottery" Shirley Jackson (1948)

Jackson was known for writing mystery and horror and this story strays as far as any on this list from what's usually considered science fiction. (This story was published in The New Yorker and few of her short works were in traditional science fiction magazines.) "Lottery in June, corn be heavy soon." To say much more is to probably give too much about this powerful short story away. To quote Martin Cahill: "If you asked anyone about a American short story that stuck with them for their entire lives, it would not shock me if they were to think for a moment, and then say, “that one story, ‘The Lottery,’” followed up with some form of, “that shit is fucked up.” Many of her best stories appear in the collection The Lottery and Other Stories.

"The Little Black Bag" C. M. Kornbluth (1950)

One might say that this story shares a similar premise with "Mimsy were the Borogoves" in that an object, in this case, a doctor's little black bag, gets sent back in time. However, suffice it to say that this is not an equally playful story. Kornbluth's classic works are this story and its sequel "The Marching Morons." However, he was a prolific author and his other works include "Two Dooms" which is considered to be one of the better What if the Nazis Won WWII treatments.

"There Will Come Soft Rains" Ray Bradbury (1950)

Bradbury's work has a poetic and evocative style that can tend towards fantasy and horror as much as science fiction. Charlie Jane Anders, who appears later on this list, had this to say about this particular story: "But this one, in which basically a smart house keeps going after its human inhabitants are all dead, is in a special league in the knife-twisting sweepstakes. Like a lot of stories in the years following World War II, it's concerned with the threat of nuclear annihilation, but also with how our technology might outlive us." This story appears in The Martian Chronicles, which is Bradbury's greatest work and you should just read the whole thing even if you're not a particular Bradbury enthusiast. But if you're looking for a single more Martian-themed story in that collection of linked stories, give "The Third Expedition" a try.

"Surface Tension" James Blish (1952)

During the 1950s, Blish wrote his Okie stories which envision a time when earthbound cities take to migrant wandering among the stars. Epic in scope, these stories were eventually wrangled into a single volume, Cities in Flight. During the same period, he was writing short stories of which "Surface Tension" is my favorite. The story concerns the events that follow after a spaceship crew genetically engineering their descendants into something that can survive after the ship crashlanded on a planet with only puddles of water. The miniaturized descendants must overcome surface tension if they're to get beyond their single puddle. Other well-regarded Blish short stories include "Common Time."

"It's a Good Life" Jerome Bixby (1953)

This story, more horror than science fiction, was the basis of a particularly good episode of the original Twilight Zone; it also has led to various remakes and riffs that played off of the idea including an episode of The Simpsons. At the center of the story is Anthony Fremont, a three-year-old boy with near-godlike powers; he can transform other people or objects into anything he wishes, think new things into being, teleport himself and others where he wants, read the minds of people and animals, and even revive the dead. As Anthony can read minds, the town's population must not only act content with the situation when near him, but also think they are happy at all times. 

"Fondly Fahrenheit" Alfred Bester (1954)

This is a breathless story of a man and his android whose personalities intermesh to become two aspects of a single insane murderous personality. Author Robert Silverberg has described it as a "paragon of story construction and exuberant style" which is what really makes it stand out on this exalted list. Other Bester short stories of particular note are "Adam and No Eve" and "The Men Who Murdered Mohammed."

"The Star" Arthur C. Clarke (1955)

Now we get into tough and at least somewhat arbitrary choices. Clarke is one of the authors on this list--two others, Asimov and Heinlein, are coming up shortly--who produced such an astonishing volume of quality short stories that picking just one is really impossible. But here we are. I give "The Star" the nod because there's deeper emotion than in much of Clarke's work. It's hard to say too much about this story without giving things away. Let's just say it concerns an expedition that has discovered the remnants of an advanced civilization destroyed when its star went supernova. Other favorite Clarke shorts include "The Nine Billion Names of God," "The Wall of Darkness," "Rescue Party," and "Meeting with Medusa." I might also throw in "The Sentinel" but mostly because it was the inspiration for the film 2001: A Space Odyssey.

"Fiat Homo" Walter M. Miller Jr. (1955)

This one is a bit of a cheat. In the early 1950s, Miller published a series of short stories which would shortly be combined and reworked into A Canticle for Leibowitiz, his only novel and the work for which he is known. A Canticle for Leibowitz is arguably the best post-apocalyptic work of science fiction and differs significantly from the more common "Mad Max" model. The novel is set in a sort of Dark Ages hundreds of years after the holocaust and tells its story from the point of view of monks preserving knowledge as in the previous Dark Ages. The reworked "Fiat Homo" makes up the first part of Canticle and is arguably the strongest but, in practice, it's the full novel that you'll read. 

"Time in Advance" William Tenn (1956)

While not especially well-known (he never won a science fiction writing award), Tenn, the pseudonym of Philip Klass, wrote sharp-witted stories, often with a darkly comic side. This story, written fairly near the end of his active writing career, is set in a far future where a law has been passed enabling citizens to serve out sentences for crimes they intend to commit, serving the full term, but with a 50% pre-criminal discount. When two such pre-criminals return to earth intending to carry out the crimes they've earned, things don't go as planned. "The Brooklyn Project" is another story well-worth your time and was pretty much a toss-up with this one.

"The Last Question" Isaac Asimov (1956)

Asimov himself suggested that this was one of his favorite short stories. "I got the idea all at once and didn't have to fiddle with it; and I wrote it in white-heat and scarcely had to change a word. This sort of thing endears any story to any writer," he once wrote. It really is almost a perfect idea-based science fiction short story. It deals with the development of a series of computers called Multivac and their relationships with humanity through the courses of seven historic settings, beginning on the day in 2061 when Earth becomes a planetary civilization. In each of the first six scenes a different character presents the computer with the same question; namely, how the threat to human existence posed by the heat death of the universe can be averted. it's tight, it's funny, and it's got a killer ending. His original novella "Nightfall" is also wonderful but I think this short story gets the nod. One or more of his Robot stories could make their way onto this list as well but I think of them as more of a group achievement award than individual standouts.

"All You Zombies" Robert Heinlein (1959)

This story involves time travel paradoxes and further explores themes introduced in Heinlein's earlier "By His Bootstraps." In his analysis of the story, Davi Ramos writes that "Individualism and the free expression of love and sexuality are among its main themes. The work analyzed here pushes the boundaries of causality to mind-bending extremes, in order to discuss the boundaries of sexuality and the logic of identity." The narrative flow is complex and you probably won't totally get it on your own the first time around. The film Predestination is based on this story. Some other short stories I'd put on the list are "The Unpleasant Profession of Jonathan Hoag" and many of the stories collected in The Past Through Tomorrow future history including "Requiem" and the prequel to his Lazarus Long immortality saga Time Enough for Love, "Methuselah's Children."

"Flowers for Algernon" Daniel Keyes (1959)

Algernon is a laboratory mouse who has undergone surgery to increase his intelligence. The story is told by a series of progress reports written by Charlie Gordon, the first human subject for the surgery, and it touches on ethical and moral themes such as the treatment of the mentally disabled. The short story was later expanded into a novel and made into the film Charly (for which Cliff Robertson won the Oscar for Best Actor) and many other adaptations but I think the original story is still the most effective. 

"Chronopolis" J. G. Ballard (1960)

Ballard probably became best known (whether people knew his name or not) for his war novel, Empire of the Sun, a semi-autobiographical account of a young British boy's experiences in Shanghai during Japanese occupation--which Stephen Spielberg later made into a film. But he also had a career as a science fiction author which spanned the "world destroying" genre (The Drowned World, The Wind from Nowhere) like other British authors such as John Wyndham to experimental efforts some of which were adapted into films like  David Cronenberg's Crash. However, he also wrote some enticing short stories. One of them is "Chronopolis" which begins with a man in prison, Newman, and proceeds to examine his fascination with the concept of time in a world where clocks have been prohibited and are regulated by time police. Other Ballard shorts that aren't too experimental include "The Drowned Giant, The Overloaded Man," and especially "Thirteen for Centaurus."


"The Lady Who Sailed the Soul" Cordwainer Smith (1960)

The nom de plume of Paul Myron Anthony Linebarger, Smith (not to be confused with E. E. Doc Smith of Lensman fame) wrote relatively little, in part because he held jobs that required extensive foreign travel. Most of his science fiction was in the Instrumentality Universe and author Frederik Pohl described his stories as "a wonderful and inimitable blend of a strange, raucous poetry and a detailed technological scene, we begin to read of human beings in worlds so far from our own in space in time that they were no longer quite Earth." This story is one of his more lyrical works. It's adjacent in timeline to "Scanners Live in Vain" which is another Smith story written ten years earlier near the beginning of his science fiction writing. While Scanners is often cited as Smith's best work, I find its older provenance gives it a more archaic feel.  

"Harrison Bergeron" Kurt Vonnegut Jr. (1961)

Although Vonnegut was a mainstream author, whatever that means exactly, many of his novels contained fantastical or science fictional elements whether the Ice Nine of Cat's Cradle, the time displacement of Slaughterhouse Five, or just about anything related to The Sirens of Titan. The premise of this story is that amendments to the Constitution dictate that all Americans are fully equal and not allowed to be smarter, better-looking, or more physically able than anyone else. The Handicapper General's agents enforce the equality laws, forcing citizens to wear "handicaps" such as masks for those who are too beautiful, loud radios that disrupt thoughts inside the ears of intelligent people, and heavy weights for the strong or athletic.

"A Rose for Ecclesiastes" Roger Zelazny (1963)

This is another one of those stories that the reader is probably best discovering on their own. It involves the Biblical book of Ecclesiastes, language, Mars, and prophesy and we should probably leave it at that. This story was an early career work for Zelazny who would go on to write many novels, most notably Lord of Light and his Amber series.

"'Repent, Harlequin!' Said the Ticktockman" Harlan Ellison (1965)

I had the pleasure of seeing Ellison speak a couple of times when I was an undergrad and he was simultaneously both entertaining and thought-provoking. If I'm being honest though, I've tended to admire rather than love many of his short stories. That said, "'Repent, Harlequin!' Said the Ticktockman" opens with a passage from Civil Disobedience by Henry David Thoreau. The story is a satirical look at a dystopian future in which time is strictly regulated and everyone must do everything according to an extremely precise time schedule. And breaking the law has severe consequences. Another perpetual Ellison favorite is "I Have No Mouth, and I Must Scream."

"Light of Other Days" Bob Shaw (1966)

With Shaw we return to a writer, who in spite of a body of work, is mostly known for a single story and, to a lesser degree, its sequel "Burden of Proof" and an associated novel, Other Days, Other Eyes. The melancholy "Light of Other Days" builds on the idea of "slow glass," glass through which light takes years to pass, to explore the preservation of the past into the present.

"We Can Remember it for you Wholesale" Philip K. Dick (1966)

Dick's fiction explored varied philosophical and social themes, and featured recurrent elements such as alternate realities, simulacra, monopolistic corporations, drug abuse, authoritarian governments, and altered states of consciousness. He was prolific but a number of his works were arguably more successful as screen adaptations than in their original print form. This story about memory implants and the nature of reality stands well by itself but also was the basis for the 1990 film Total Recall directed by Paul Verhoeven and starring Arnold Schwarzenegger--which is probably best interpreted in light of its original source material. 

"Neutron Star" Larry Niven 1(966)

Niven is best known for his hard-science Known Space series populated by imaginatively-drawn strange aliens and exotic technologies in a far-future. This story is the first to feature Beowulf Shaeffer, the ex-pilot and reluctant hero of many stories. It also marked the first appearance of the nearly indestructible General Products starship hull, as well as its creators, the Pierson's Puppeteers, a cowardly but advanced race that have an appearance somewhat in common with centaurs. Shaeffer's mission is to explore a neutron star and things go wrong; it turns out the science in the story in the story isn't quite right but it's still a great example of Niven's work. Niven's shorts tend to be stronger than his solo-authored novels. Many of the stories in the known space series are good reads, including those in the collection named after this story, Neutron Star. Outside the Known Space universe, "Inconstant Moon" is a great read and arguably his best if just picking stories in isolation.

"Aye, and Gomorrah" Samuel Delany (1967)

This first appeared as the final story in Harlan Ellison's seminal 1967 anthology, Dangerous Visions. It was controversial because of its disturbing sexual subject matter although, like many of the stories in that anthology, stories that pushed the envelope in 1967 don't necessarily do so today. The narrative involves a world where astronauts, known as Spacers, are neutered before puberty to avoid the effects of space radiation on gametes. Aside from making them sterile, the neutering also prevents puberty from occurring and results in androgynous adults whose birth-sex is unclear to others. Spacers are fetishized by a subculture of "frelks" attracted by the Spacers' lack of arrousability.

"The Ones Who Walk Away from Omelas" Ursula Le Guin (1973)

Le Guin wrote over a course of almost 60 years to a variety of audiences in a variety of genres. This story is essentially philosophical fiction that explores a summer festival in the utopian city of Omelas, whose prosperity depends on the perpetual misery of a single child. Another, longer, Le Guin story "Buffalo Gals, Won't You Come Out Tonight" recounts a human girl's meeting with incarnations of Native American spirit animals and appears in an eponymous collection.

"Where Late the Sweet Birds Sang" Kate Wilhelm (1974)

This story is the first of three linked stories later published as a novel of the same name in 1976. It takes  place as civilization is collapsing; an isolated group prepares for this largely infertile post-collapse world by undertaking a cloning program. Wilhelm wrote novels and stories in the science fiction, mystery, and suspense genres over the course of a long career. One other story especially worth checking out is "The Girl Who Fell into the Sky," which is more supernatural than SF but did win a Nebula for best novelette.

"Houston, Houston, Do You Read" James Tiptree Jr. (1976)

This novella is another one of those stories where just about any description would shortchange readers wanting to experience it without knowing what it's really about and where it's headed. Tiptree was the pen name used by Alice Bradley Sheldon from 1967; her real identity wasn't widely-known but was generally assumed to be that of a man until it leaked out shortly after this story was written. However, there was speculation, based partially on the themes in her stories (including this one), that Tiptree might be female. In any case, this is an interesting story that won both Hugo and Nebula prizes, even if the gender and social stereotypes seem dated for the time it was written.

"Ender's Game" Orson Scott Card (1977)

This story gave birth to a series of novels, starting with a novelization of "Ender's Game" and leading to a series of increasingly undistinguished novels. While the first sequel, Speaker for the Dead, was quite excellent in its own right the initial short story shines brightest in my opinion. This story begins as Ender is made the commander of Dragon Army at Battle School, an institution designed to make young children into military commanders against an unspecified enemy. Armies are groups of students that fight mock battles in the null gravity Battle Room. Due to Ender's genius in leadership, Dragon Army dominates the competition. Go ahead and read the story.

"The Guy with the Eyes" Spider Robinson (1977)

In general, I've tried to pick standalone stories for this list. This one works fine in that vein but it's really here as the first entry in the linked stories that make up Callahan's Crosstime Saloon stories; they've been collected into a number of books beginning with Callahan's Crosstime Saloon. The stories are filled with strange or unusual events and visitors and are an obvious homage to Fletcher Pratt and L. Sprague de Camp's Tales from Gavagan's Bar and Arthur C. Clarke's Tales from the White Hart.

"Sandkings" George R. R. Martin (1979)

Martin is, of course, best known for his series of epic fantasy novels A Song of Ice and Fire, which were adapted into the HBO series Game of Thrones. However, before this breakout, he had earlier written a variety of stories that mixed fantasy, horror, and science fiction of which "Sandkings" is a clear example. It certainly includes sensibilities that will be familiar to Game of Thrones viewers. Suffice it to say that Martin was supposedly inspired by a college friend at Northwestern University who had a piranha tank and would sometimes throw goldfish into it between horror film screenings. (This explains a great deal about Martin.)

"The Gernsback Continuum" William Gibson (1981)

This is probably an unconventional choice. "Burning Chrome" or "Johnny Mnemonic," associated with his Sprawl trilogy, would probably be more more conventional picks for Gibson short stories. But I think this story is actually more central to cyberpunk. As Bruce Sterling writes "'The Gernsback Continuum' shows [Gibson] consciously drawing a bead on the shambling figure of the SF tradition. It's a devastating refutation of 'scientifiction' in its guise as narrow technolatry." It takes down the glimmering science fictional cities in which zeppelins rule the airways.

"True Names" Vernor Vinge (1981)

This novella doesn't have the gritty ambiance but it's still a seminal work of the cyberpunk genre. In 2001, The New York Times declared that Vinge's depiction of "a world rife with pseudonymous characters and other elements of online life that now seem almost ho-hum" had been "prophetic." I'd add that the idea of a true name, which is to say the tying of someone's online identity to their identity in the real world, doesn't utilize Vinge's term as much as it should. He subsequently wrote novels that explored a future libertarian society and the impact of a technology which can create impenetrable force fields called 'bobbles' but is probably best known for his 1992 novel A Fire Upon the Deep and, to a lesser degree, its sequels. A newer story that deals with a similar topic updated is "Fast Times at Fairmont High," which can be viewed as a sort of pilot for his Rainbows End.

"The River Styx Runs Upstream" Dan Simmons (1982)

Although his Hyperion Cantos series of (primarily) novels are science fiction, much of Simmons' output is horror and this story certainly bridges both genres. This was the story Simmons published after attracting the attention of Harlan Ellison at a writing workshop. The story imagines a world in which "Resurrectionists" can technologically revive family members but, while they can then function somewhat autonomously, they no longer have higher brain functions. Many of Simmons' short stories are in the collection Prayers to Broken Stones, which also includes "Remembering Siri," originally a part of the aforementioned Hyperion

"Bloodchild" Octavia Butler (1984)

Set on an alien planet, this story depicts the complex relationship between human refugees and the non-humanoid aliens who keep them in a preserve to protect them. Themes of co-existence and breeding appear throughout much of Butler's writing, including the Xenogenesis trilogy that came out during the five years following this story.

"The Crystal Spheres" David Brin (1984)

The Fermi paradox, named after physicist Enrico Fermi, refers to the apparent contradiction between the lack of evidence for extraterrestrial civilizations and the many estimates that suggest such civilizations should be commonplace in a very large and old galaxy and universe. This story postulates a solution, the starting point of which is that every habitable solar system is surrounded by a "crystal sphere" which can only be broken from the inside--once a civilization has the technology to do so. Another good Brin short story is "The Giving Plague" but I figured there were already enough stories with a plague component on this list given the times.

"Schrödinger's Kitten" George Alec Effinger (1988)

Partially sharing the same setting as a number of his novels, this novelette follows a Middle-Eastern woman, Jehan Fatima Ashûfi, through various realities. She perceives these realities as "visions" and assumes they might come to her from Allah. The realities correspond to a form of the many worlds interpretation of quantum mechanics, and the story's title comes from the Schrödinger's cat thought experiment.

"Axiomatic" Greg Egan (1990)

In this story, which appears in a collection of the same name, the protagonist enters a store selling mods not only for every variety of psychedelic experiences, but for altering one's personality traits, sexual orientation, and even religion. The protagonist seeks a custom-made mod that will suspend his moral convictions long enough for him to murder his wife's killer. All the stories in the collection delve into different aspects of self and identity.

"Even the Queen" Connie Willis (1992)

This short story falls squarely on the comic side of Willis' production. The Village Voice called it "a comedy of identity politics and mother-daughter relations" and it's as good an introduction as any to Willis' light-hearted and sometimes madcap writing. She also has a more more somber, even tragic, side with stories such as "The Last of the Winnebagos" and "A Letter from the Clearys." "Fire Watch" is a short story introduction to her works that feature time travel by history students and faculty of a future University of Oxford; noteworthy novels in that series are Doomsday Book and the To Say Nothing of the Dog.

"Story of your Life" Ted Chiang (1998)

Although the decline of science fiction magazines has arguably helped lead to a decline of shorter science fiction overall, the well-awarded Chiang has never written a full-length novel. Moviegoers may recognize this novella as the basis for the 2016 film Arrival. (The film keeps the basic structure of the story but adds what are probably best described as Hollywood elements.) The story is narrated by linguist Dr. Louise Banks who is called in to communicate with alien visitors. It brings in questions of free will and the nature of time but it's another story that rewards discovering its structure on your own. Chiang has many fine works but another particularly noteworthy novella is "The Lifecycle of Software Objects."

"The Janitor on Mars" Martin Amis (1998)

Amis is a British writer who has published very little science fiction; indeed this story, which was originally published in The New Yorker, is the only science fiction in the Heavy Water and Other Stories collection where it appears. In this story, a robot makes contact from Mars and reveals the  truth of mankind's place in the Universe. It's funny and sad at the same time. 

"The Fermi Paradox is our Business Model" Charlie Jane Anders (2010)

This story is a fun take on why the previously mentioned Fermi Paradox might be the result of deliberate outside actions. Aliens may be involved. And they may not be benevolent. Anders has also won a Hugo for her novelette "Six Months, Three Days."

"Sinners, Saints, Dragons, and Haints in the City Beneath the Still Waters" N. K. Jemisin (2010)

Much of Jemisin's writing concerns cultural conflict and oppression. This story is set in the Ninth Ward of New Orleans in the aftermath of Hurricane Katrina. Heather Rose Jones writes that "What makes this story great is the immersive voice and language, the way descriptions of everyday surroundings slide easily into the fantastic, and the way the folklore of cities and peoples is woven into a new mythos."

"Inventory" Carmen Maria Machado (2013)

This is yet another story about which I hesitate to say much of anything. It's more literary than genre and to even say why it's connected to science fiction would be to cheat the reader.

Some of the text in this post is adapted from Wikipedia and The Encyclopedia of Science Fiction by John Chute and Peter Nicholls.

Hacker News thread: https://news.ycombinator.com/item?id=23959648

Monday, June 01, 2020

Podcast: Was open source inevitable? (Part 4)

EPISODE 4: HOW OPEN SOURCE WON—OR DID IT?

In this four-part miniseries within the Innovate @Open podcast, we explore the alternative histories of open source software through the voices of many of the people who lived through its rise.

In this fourth and final episode, we consider our last counterfactual: Was open source mainstream commercialization inevitable and what role did the IBM turn-of-the-century investment in Linux make to the ultimate trajectory of Linux and open source? Was that investment even a sure thing? We then wrap up the series by thinking about where open source is today. Has it really won? What are the challenges ahead? What should we be cautious about taking for granted?

On this episode, we're joined by the following guests:
  • Irving Wladawsky-Berger, formerly led IBM internet then Linux strategy
  • Chris Aniszczyk, Linux Foundation
  • Matt Asay, AWS
  • Dave Neary, Red Hat
  • Steven Vaughn-Nichols, CBS Interactive
  • Diane Mueller, Red Hat
  • Bryan Cantrill, Oxide Computer
  • Mike Bursell, Red Hat
  • Rob Hirschfeld, RackN
Related links:

Tuesday, May 26, 2020

Podcast: Was open source inevitable? (Part 3)

EPISODE 3: OPERATING SYSTEMS FOR A HORIZONTAL STACK

In this four-part miniseries within the Innovate @Open podcast, we explore the alternative histories of open source software through the voices of many of the people who lived through its rise.

In this third and penultimate episode, we play out the great Linux vs. Windows rivalry of the 1990s and 2000s. What might have happened had Linux Torvalds decided to take up ice sculpture instead of writing Linux? Or what could a kinder, gentler, and more politic Microsoft have done to steer history further in its favor to the detriment of the Unix and Unix-like operating systems?

On this episode, we're joined by the following guests:
  • Steven Vaughn-Nichols, CBS Interactive
  • Bryan Cantrill, Oxide Computer
  • Brian Proffitt, Red Hat
  • Rob Hirschfeld, RackN
Related links:

Monday, May 18, 2020

Podcast: Was open source inevitable (Part 2)

EPISODE 2: THE FOUNDATIONS OF A MOVEMENT

In this four-part miniseries within the Innovate @Open podcast, we explore the alternative histories of open source software through the voices of many of the people who lived through its rise. The central question is “Was open source inevitable?” Not necessarily in the particulars but in the macro.

In this second episode, we consider whether some of the foundational elements of modern open source were inevitable. Unix is intertwined with the history of open source, including but not limited to Linux. Did something like it have to come about? And what about Richard Stallman and the Free Software Foundation? Would things have played out differently absent this political dimension of free software? Finally, what of the role of copyright law and licenses more broadly.

On this episode, we're joined by the following guests:
  • William Henry, Red Hat
  • Mike Bursell, Red Hat
  • Dave Neary, Red Hat
  • Richard Fontana, Red Hat
  • Luis Villa, Tidelift
Related links:
Note: This podcast references the University of Washington license for Sendmail. Sendmail uses the Sendmail license written at the University of Washington.

Listen to the podcast (MP3 - 31:40)

Monday, May 11, 2020

Podcast: Was open source inevitable? (Part 1)

EPISODE 1: SETTING THE STAGE

In this four-part miniseries within the Innovate @Open podcast, we explore the alternative histories of open source software through the voices of many of the people who lived through its rise. The central question is “Was open source inevitable?” Not necessarily in the particulars but in the macro.

In this first episode, we consider those factors that probably were inevitable in more or less the form that we find them today. We then take the listener through the history of open source software by way of background for the upcoming episodes.

On this episode, we're joined by the following guests:
  • Andy Grove, former CEO of Intel, by way of a clip from a 1996 talk at MIT
  • Irving Wladawsky-Berger, who ran internet and then Linux strategy at IBM
  • Chris Aniszczyk, Linux Foundation
  • Harish Pillay, Red Hat
  • Jan Wildeboer, Red Hat
Related links:



Thursday, April 02, 2020

Podcast: If Linux didn't exist, would we have had to invent it?

I've been working on a multi-part podcast series that considers the question "Was open source inevitable?" through interviews with many industry experts. The series will be produced/edited as opposed to just straight-through interviews. However, in the course of talking with people and doing research, I got into a bit of a debate with Bryan Cantrill and Steven Vaughan-Nichols on Twitter.

I've known both of them for ages. Bryan Cantrill is a co-founder of Oxide Computer and a longtime employee of Sun Microsystems where he was, among other things, responsible for creating Dtrace in Solaris. Steven J. Vaughan-Nichols, aka sjvn, has been writing about technology and the business of technology since CP/M-80 was the cutting edge and is a contributing editor at CBS Interactive. They agreed to hop on a call together and debate the question posed in the title. Consider this a teaser for the upcoming series.

Bryan's take is that the general arc of open source operating systems was more or less inevitable. Steven's not so sure and sees Linux as a very important ingredient in the software mix associated with the early days of the commercial Internet—without which things would have played out differently.

I mostly stayed out of their way. Listen to their spirited debate about Linux, Perl, the LAMP stack, and more.

Listen to the podcast [MP3 - 36:30]

Tuesday, March 17, 2020

A new project: Was open source inevitable?

Last fall, I listened to the episode Was the Protestant Reformation Inevitable? on the Tides of History podcast. It turns out to be a fascinating question both because of the importance of the event and the difficulty of giving a definitive answer. I encourage a listen.

However, for me, it turned out to be fascinating for another reason as well. It got me thinking that a similar question could be asked about open source. And this got me excited because I'm a big believer in how history and counterfactuals can shed a lot of light on current and future processes—such as those around how open source software (and other types of openness) might develop in the future.

My plan is to turn this into a miniseries within my existing Innovate @Open podcast. Although my podcasts are normally fairly straightforward one-on-one interviews, this three(?) parter will be more produced and feature edited interviews from a variety of guests interspersed with commentary and perhaps historical material.

I had originally planned to do most of the interviews at events and I had started to do so. But with essentially all travel shutdown at the moment I'm taking advantage of the relative lull to reach out to people.

If you think you have something to add, feel free to get in touch. I'm mostly interested in ways things could have played out in a materially different way. Perhaps not without open source at all but a materially different landscape.

Monday, March 16, 2020

Podcast: Hyperledger's Arnaud Le Hors on best practices for Technical Steering Committees

Arnaud Le Hors is the chair of the Hyperledger Project's technical steering committee (TSC). Earlier this month, I sat down with Arnaud at the Hyperledger Global Forum to talk about the role of technical steering committees and some of the things that they've learned with Hyperledger over the past few years.

The Hyperledger Project is a group of related enterprise blockchain projects under the umbrella of the Linux Foundation. However, in this discussion, we didn't focus so much on the technology but, rather, on how best to manage a project from a technical perspective. Perhaps the most interesting part of this discussion related to how managing a project like this one is at least as much about process as it is about the core technology. Example. What could the TSC have done better? Document everything!

Some related links:
Hyperledger Project
Hyperledger TSC Home
Open governance insights from Chris Aniszczyk, VP of Developer Relations at the Linux Foundation
Blockchain reality check 2020: Challenges and winning applications (write-up from Hyperledger Global Forum 2020)

Listen to podcast [MP3 - 24:11]

Saturday, March 14, 2020

Favorite Science Fiction Short Stories: First Draft


I've been playing around with re-reading favorite science fiction short stories and seeing if there's anything that has managed to elude me over the years. (Or at least I've totally forgotten if I did read it.) It's not always as easy as it seems as if it should be to track down individual short stories--at least legally. But that's a story for another day.

Anyway, my initial list (sorted chronologically) is below. A few rules I set for myself:
  • One story per author. Certainly there are many on this list for which I could effortlessly reel off multiple deserving entries.
  • I didn't worry about exact definitions. There are stories here that are longer than the science fiction awards definition of a short story.
  • I didn't cut a lot of slack for popular older stories that require a lot of historical perspective to appreciate today.
  • I also didn't include authors who I like for their novels but don't have any short stories I'm aware of that really wowed me. 
  • I tried to pick stories that work well in isolation. For example, while I really like Larry Niven's Known Space stories, I think "Inconstant Moon" is probably his best standalone story.
Who am I missing? Are there any of my picks that you think are really off?

The Machine StopsE. M. Forster1909

A Martian OdysseyStanley G. Weinbaum1934

Microcosmic GodTheodore Sturgeon1941

The Weapon ShopA. E. van Vogt1942

Mimsy were the BorgovesLewis Padgett (pseudonym)1943

A Logic Named JoeMurray Leinster1946

The LotteryShirley Jackson1948

Scanners Live in VainCordwainer Smith1950

There Will Come Soft RainsRay Bradbury1950

Surface TensionJames Blish1952

It's a Good LifeJerome Bixby1953

Fondly FahrenheitAlfred Bester 1954
The StarArthur C. Clarke1955
The Last QuestionIsaac Asimov1956
All You ZombiesRobert Heinlein1959

Flowers for Algernon (short story)Daniel Keyes1959

A Rose for EcclesiastesRoger Zelazny1963
Repent, Harlequin!' Said the TicktockmanHarlan Ellison1965
Light of Other DaysBob Shaw1966
We Can Remember it for you WholesalePhilip K. Dick1966

Aye, and GomorrahSamuel Delany1967

Inconstant MoonLarry Niven1973

Ender's Game (short story)Orson Scott Card1977
SandkingsGeorge R. R. Martin1979
The Gernsback ContinuumWilliam Gibson1981
True NamesVernor Vinge1981
Buffalo Gals, Won't You Come Out TonightUrsula Le Guin1987

Bears Discover FireTerry Bisson1990

Even the QueenConnie Willis1992
Story of your LifeTed Chiang1998

Wednesday, March 11, 2020

Podcast: Open hardware and firmware with Bryan Knouse of Project OWL

Project OWL (Organization, Whereabouts, and Logistics) has developed a mesh network of Internet of Things (IoT) devices called “DuckLinks” that can be deployed or activated in disaster areas to quickly reestablish connectivity and improve communication between first responders and civilians in need. On March 10, the Linux Foundation announced that Project OWL’s IoT device firmware effort will be hosted at the Foundation.

In 2018, Project OWL was the global winner in the inaugural Call for Code Global Challenge, competing with more than 100,000 participants from 156 nations. The Call for Code Global Challenge encourages and fosters the creation of practical applications built on open source software, with a focus on immediate and lasting humanitarian impact in communities around the world. “Project OWL was our first Call for Code winner that went through the Code and Response incubation process, and we’re excited to see this solution grow closer to reality,” said Daniel Krook, IBM Chief Technology Officer for Call for Code and Code and Response.

Listen to the podcast [MP3 - 16:15]

Tuesday, February 18, 2020

Podcast: Nextcloud founder Frank Karlitschek on breaking away from the cloud

Frank founded the ownCloud project in 2010 to put home users and enterprises back in control of their data. To bring file sync and share technology to the next level and better align to the needs of users and customers he founded Nextcloud in 2016. We caught up at FOSDEM in Brussels earlier this year to talk open source.

Among the topics we covered were:

  • Nextcloud
  • How users/organizations have become more sophisticated about hybrid cloud and multi-cloud and have generally rejected making a binary choice
  • Reasons why some of his customers have chosen to self-host storage and collaboration
  • Why we should talk more about the freedoms that come from choosing open source

Listen to the podcast [MP3 - 11:50]