Tuesday, May 05, 2015

Podcast: The Go language with Red Hat's Mark Lamourine

In this podcast we discuss what features of Go make it so interesting, why compiled languages like Go are a topic of discussion today, and how they fit with containers, microservices, and hyper-scale computing. (And why Mark is porting some perfectly good Python code to Go.)

Listen to MP3 (0:18:40)
Listen to OGG (0:18:40)


Gordon Haff:  Hi, everyone. This is Gordon Haff with Red Hat with another edition of Cloudy Chat podcast. I'm joined once again by my colleague, Mark Lamourine.
Today, we're going to talk about the Go Language. This may seem a little different from our other topics around containers, but as you'll see, it's going to come back around to containers just like a lot of things seem to do these days.
Mark, to start off with, one of the interesting trends that we've seen, the RedMonk boys have more or less convinced me with some of their data that we really are seeing an increase in the number of languages that are getting significant deployment these days.
As always, there are a lot of science experiments, but there also seems to be an increase in real, live languages for production use, of which Go is probably one of the most visible candidates.
Mark Lamourine:  Yep and Go, if you look at the origin of Go, it was actually designed with that in mind. In fact, if you look at some of the origin stories, one of the really distinctive characteristics about Go, in contrast to some of the other languages, is that Go is actually designed to make things easier for the computers.
Most computer languages, and it's been my philosophy too, is that computer languages should exist to make programming easy for humans. But, there are a couple of things that they did in the design of Go which were there specifically to make life easier for the computers.
Gordon:  Before we go into that, let's maybe pop up a level for our listeners and talk about where Go fits into the landscape of computer languages which, I think is fair to say, have been dominated by scripting languages over the last decade or so.
Mark:  Yep. That's fairly clear. Most of the big advantages have been in scripting languages. We've certainly got enough CPU power that the arguments against scripting languages for speed no longer apply. Go is in some senses is a step, not backwards, but back towards the machine.
It's intended to replace not scripting languages but C. One of the authors is Rob Pike. He was instrumental in the beginnings of the C language, not the very beginning, but in some of the evolution immediately after and in the early evolution of Unix.
One of his notes was that C is the primary language for system‑level programming, but it hadn't gained any benefit from new understandings of networking, new understandings of distributed systems. The Go language is an attempt to take some of those constructs which didn't exist at all in the 1970s and '80s and build them into a programming language so that they're essentially invisible.
Gordon:  C had also been dragged into doing a lot of tasks almost by default. It was not necessarily designed to originally, and obviously, C++ and a lot of features that came with that where partial attempts to move it beyond being purely a system programming language.
But still, essentially by default given that no other compiled language really took off in the same time frame, C and C++ ended up being used for almost every programming tasks.
Mark:  Pretty much if you weren't scripting it, you were writing it in C or C++. Java is the exception to that. Again, if you're talking about system‑level languages, it's C and C++. Even C++, if you look at the additions that Stroustrup made to C that created C++, he actually started out mostly doing academic experiments on object‑oriented programming.
It turned out later that some of them were useful in wide areas, but they didn't really advance system programming in any way.
In some senses, C++ still is a bag on the side of C. The language constructs are derived from C in ways that other object‑oriented programming languages look cleaner to work with and cleaner to write because they don't have C's baggage with respect to namespaces, and things like that.
Gordon:  Before we get into the container connection, what are some of the innovations from your perspective in Go relative to C and C++?
Mark:  The big ones, a couple of them are controversial...one of them is garbage collection. The single, largest programming error in C programming is memory leaks, is failure to manage memory well.
While garbage collection in compiled languages is controversial, there's definitely something to be said for designing a language that makes it harder to do the most common errors. Java was one of the first attempts to do that.
Java removed pointers completely and said you'll only get references purely because the other really common error was seg faulting due to incorrect memory references or incorrect pointer arithmetic. So, adding language features to counteract user problems, coding difficulties really with the language, has an old history. That's one of them.
The other one which is pretty significant is the library and inclusion mechanism. This was where I was talking about Go being written for computers. One of the characteristics of the C preprocessor is that it re‑reads all those include files. It doesn't know you've read one until you look at an ifdef and say, "Have I already read this?"
You can get situations with nested include files where, I think, in one instance they said they had re‑read the same include files something like 4,000 times to write a fairly simple C program. Go places a restriction on includes that each file can only be included in one place so that when you're building a binary, it knows it only reads that library source code once.
Another innovation which is controversial is that it eliminates the separation of code and headers. In C, you have the code which defines the behavior of some interfaces, and then you have the header files which declare the headers. The point there was to avoid having to reload and recompile all that code.
Go has gone the other direction explicitly. Go has said, "Well, you're going to reload and recompile that code every time, but we're only going to include the things you actually call." The compiler is a bit smarter about what it reads and so they would argue, eliminates the need for the preprocessor.
Gordon:  I said we're going to get to the connection between Go and containers. What is the connection?
Mark:  If you look back at what Google has been doing for their work for the last, at least, a decade. We can't see inside to be sure. They've been using containers, or some kind of container model, for certainly years and possibly as much as a decade.
What they've found in programming their containers among other things is that if you write using scripting languages, you end up having to pull along all of this infrastructure for the scripting language.
You might only have a script that's a few hundred lines long to do a small, tightly‑bound task, but then you have to carry along all of this infrastructure, the binary, the libraries, any special libraries. I'm just talking about the language libraries here, not about the shared libraries that go with the binary.
What they saw in Go, or one of the reasons they designed Go, was to make it so that all those bindings happen at compile time instead of at run time, rather than carrying all of that stuff along to the runtime of the container, if you use a Go binary, those are all resolved at build time.
Your programs, your containers can be much smaller and much tighter. You spend less time pushing around files that you don't actually use. In their mind, Go is a really good language for actually writing container binaries.
Gordon:  One of the interesting things that seemed to have happened here is computers have obviously gotten a lot faster and there's almost been this philosophy, as you said earlier, of it's more important to save the programmers' time than it is to save computer's time, but as we've gone to this hyper‑scale systems, small differences in resource efficiency multiplied by a hundred thousand, multiplied by a million, multiplied by 10 million, starts to look like real resources.
Mark:  What containers are starting to do, is they're starting to expose some entirely understandable laziness on the part of people who are writing scripting languages, writing packaging for traditional operating systems, for conventional operating systems.
When you knew that you were going to be installing this software, you're going to be installing these files, and you knew that once the package was installed on a host, that there were no boundaries between the software that you had installed and whatever application that the user is going to write.
You didn't worry about those boundaries anymore. In fact, we've built up some fairly complex systems that might have usages that run two, or three, or four layers deep. With shared libraries, with scripting libraries, with placement of user‑level libraries, the imposition of containers is starting to expose the fact that there are boundaries there that are being crossed.
I think...over time, as those get exposed, we're going to start looking at things where people are going to go, "We need to be more rigorous, more careful about understanding the boundaries so that we can build tighter, smaller, faster, and more resource‑efficient containers."
Gordon:  This gets to some of the prior discussion we've had in this podcast, around microservices and really understanding what the interactions between different elements are, and that takes some work, but it's really a big win if you do it.
Mark:  And I think that while it's difficult now...because we've understandably neglected it, because it didn't matter on a host, it was a fair assumption that you could reach these things.
I think that as the work happens, it looks hard now...it looks like, "Oh, we've got to do all this extra work." But we got the same kind of response when we first started telling developers, "You can't run that processes as root, or you can't do your development as root." They started going, "How am I ever going to do that?"
Then people modified the Apache stuff so that it would run in its own user space. They modified the packaging itself, so that it would run in its own user space. We've gotten to the point now where people run web servers in their home space and just buy into a high‑numbered port.
This is considered, "What do you need root to do that for?" I think the same kind of evolution is going to happen in containers. As we start understanding these things, they're going to become commonplace and people are going to stop thinking it's all that hard.
Gordon:  One of the things that seems to be happening right now, as computing became more mainstream, as more people were involved, it seems to bring a lot of inertia to certain elements of the infrastructure and ecosystems.
We've talked about languages. C was just too hard to replace because too many people knew it. The virtual machine metaphor in technology which really was what I call the skeuomorphic version of physical servers.
That so many people had invested so much sysadmin learning time in that model, that was hard to displace. Elements of storage and networking infrastructure models, those were hard to displace. But it seems that right now, the whole thing has become so creaky at some level that we're finally getting the energy to replace many of those pieces.
Mark:  I think there are people who were putting their energy into it, because they see a problem that needs to be solved and they've got spare cycles. There are still people who...I saw someone today who said, "What I want is a container image that will run in a container, it will run in a VM, and it will run on bare metal."
And, my response...this is somebody I work with and respect, and he's a really good guy. I looked at him and said, "Why would you ever want that? Because it seems to me you're tossing out the advantages of each of the runtime environments by trying to make all of them one."
I think the characteristics of containers make them suitable for some really cool things. The characteristics of VM make them suitable for slightly different things. The characteristics of bare metal, yet again, we're still going to fight with that inertia, but there is a reason why they're doing it. They need to get their job done quickly and they're following the easiest path.
Luckily, there are other people who see the problem, who see that this is going to be difficult going forward and that we're not taking advantage of the best characteristics of each environment if we continue that way.
And so, we're getting people who are throwing into question the idea that you should treat everything as a VM, or you should treat everything as a container. People are asking the question, which is the most important thing, and they're exploring.
We're going to have some cases where somebody goes, "That looked like a great idea yesterday, but it really bombed." Then, we'll get innovation. We'll get things that are "aha" moments through this...and I think it's exciting that way.
Gordon:  Bryan Cantrill as a side note had quite the rant about running containers within VM at ContainerCamp a couple of weeks back. I'll put the link in the show notes if it's available by then.
Coming back to languages, it's interesting. We're talking about Go here, but Apple has also replaced Objective C in their development tool kit with a new language, Swift.
So it seems that there is this somewhat of a shift away from C in general for at least certain types of new applications.
Mark:  Yes. I think there is a recognition that we had gotten used to the idea that machines were so powerful that you only use C when you absolutely had to be down at the machine level. And that they were tasks that scripting language might not be the best option, but it was certainly the easiest.
There were no resource constraints that would make you turn around and go, "No. I need a system‑level language." Unless there were, in which case you would, but there's a recognition that something between scripting languages and the traditional C system language is beneficial.
That there are places where the overhead of the complete scripting environment is more than you really need and it's worthwhile to try coding something in a compiled language.
Gordon:  Before we close, since we're talking about languages any way, any of the other new language projects out there, anything that catches your eye particularly?
Mark:  You hit on what are the interesting ones to me. They're certainly functional languages that are interesting and there are people who use them in production. It's not something that I follow strongly except as a point of interest.
I think the big thing is going to be the compiled languages. The return to compiled languages, especially in the context of containers, is going to be an interesting phenomenon. It will be interesting to see if people actually do backports.
In one case, I'm working on one where there...there's something I'm working on that requires a full scripting environment, and I'm looking at it, and it requires that because it's intended to run on a complex full host with a completely unknown starting point.
When I try to do the same task if I'm running on a CoreOS box or an Atomic box, I have a much more constrained starting environment. I can make much stronger assumptions about where I'm going to begin.
I'm working now in an environment where pulling in lots of static files just for scripting, it doesn't smell right. It doesn't feel right. I'm actually backporting some well‑written Python code to Go, to see if writing a small, tuned, targeted binary will win over using a more complex script.
Gordon:  In closing, what do you personally really like about programming in Go, just from a purely personal perspective?
Mark:  I hate to admit this but I'm old. I grew up programming C and Fortran. It's refreshing to go back. There are people who I know of are annoyed with the idea that you suddenly have to be aware of INT16, INT32, and typecasting.

I'm remembering my youth. It may not be something that everybody else enjoys, but it's actually fun to get back down and get my hands dirty back at the lower level again.

No comments: