This blog comments on a variety of technology news, trends, and products and how they connect. I'm in Red Hat's cloud product strategy group in my day job although I cover a broader set of topics here. This is a personal blog; the opinions are mine alone.
Increasingly, software teams aren't all working in the same office. Or maybe some are and some aren't. Red Hat's Chief Agile Architect Jen Krieger discusses her experience and offers advice for how to make distributed teams work effectively from appropriate tools, to practices, to behaviors.
Haff: Thanks for joining us today. I'm Gordon Haff, technology
evangelist with Red Hat. Today, I'm joined by Jen Krieger, who's the Chief
Agile Architect at Red Hat. We're going to start off this session talking about
these distributed teams.
not going to talk about whether teams should be distributed or not and rehash
all the debate about individual productivity versus team productivity and so
forth. Because frankly, our experience at Red Hat is that distributed teams are
the reality. Not every software engineer in the planet can live in Silicon
Valley, or indeed wants to live at Silicon Valley.
introduce yourself please.
Krieger: Hi. Jen Krieger, as Gordon mentioned, Chief Agile Architect
at Red Hat. Basically, what that means on a daily basis for me is I just roam
the halls here at Red Hat and hope people understand what that word, Agile,
talking about distributed teams. It's an interesting topic here at Red Hat
mostly because we actually have been distributed since day one, work with our
open‑source communities and we can't expect everybody to be in the same room
all the time. We're very familiar of this idea of having a distributed working
of the things that I think about a lot when I'm thinking about the word
"Distributed" is, going back to that whole concept of what forms of
communication are what we call high bandwidth or the most valuable forms of
communication. A lot of people in the Agile space would say something to the
tune of, "Face‑to‑face communication is the best way to have a
conversation with each other."
today, Gordon and I are sitting across a table from each other where normally
we would get over some form of distributed video conferencing to do a podcast.
We're actually uniquely in the same room, which is not standard for us at Red
I like to tell people is generally, when you're talking about face‑to‑face
communication, video conferencing, email, IRC, all these different methods that
you can use to communicate, it doesn't mean that face‑to‑face is the only way
to communicate nor does it mean it's the perfect way to communicate.
people actually prefer not to have a face‑to‑face communication because there
might be language barriers, personality barriers. Maybe it's easier for them to
think about what they want to say before they actually are meant to say it.
are other concerns and other considerations, especially in the world of
software engineering today. it's not that it's bad to be distributed, we just
have to recognize as an industry what it means. It's the same with face to face
communication. That can't be your "Mode one" of communication, if you
would. You have to think about the impact that other forms of communication may
have on the ability for you to communicate the ideas that you're having.
We're going to get to talking about some of the tools and techniques for
using those tools and some of the best practices, but one thing I'd like to
start off with is we had talked about distributed teams. We talked about non‑distributed
teams. The reality is that there's usually going to be some sort of mix of
those things, and that mixture can be challenging itself.
In my experience, I've had several different situations. One of the Agile
teams I've had most recently, they were all in the same office. There were six
of us, all the cubicles centralized together. We had the glass pulled out
between them, so we could have that truly collaborative, truly just face‑to‑face
type of environment.
actually worked really well except there were some challenges, because we
didn't necessarily want to be in everybody's face all the time. If you are an
engineer, it's really sometimes hard to be in a situation where you're
constantly pulled out of what you're trying to think about. We would actually
pick a day every week where everybody would be working from home, so that folks
could focus on just getting their work done.
also had situations where I would say 90 percent of all the people on the team
are distributed, so they're working in their home offices, or one of them is in
one of our main offices. There's not really a challenge of trying to share
communication, because everybody has to take that second step to communicate.
Everybody is distributed. Fundamentally, no one is in the same location.
third challenge, or the third type of team that I've worked with, is the team
where a couple of people are distributed but the central portion of that team
is located in a main office and located very close to one another.
all have their different types of challenges and different ways to address
them, but I would say the one situation that is harder to deal with is that
third situation, where I'd say maybe half your team is all in one place and
half of the team is distributed, so some people are getting the benefit of that
face‑to‑face interaction and the hallway conversation, and half the team is
Let's talk about that third situation, because I think that's very
common, maybe it's even the most common today, in the open source world, very
distributed teams are often a norm.
maybe not quite a norm at a lot of other organizations but I've certainly seen
that myself, whether we're talking software development or something else where
decisions get made informally around the water cooler or over pizzas, over
beer, whatever, and two or three guys or gals who are in London, in Hawaii, in wherever
they're working from, they find out after the fact, "That's right, we've
made that decision over lunch last week. Sorry. We'll try and remember to tell
you next time."
are some of the best practices that you've found for dealing with that kind of
The best situation that I can recall in my own career is, I would say
it's about 12, 13 years ago now. I was the sole distributed team member of a
team trying Scrum for the first time. Everybody else was in our Miami office and
they were all, on a daily basis, participating in live and active vocal
was their Scrum master, sitting in my remote office in Raleigh, North Carolina,
trying to somehow manage impediments and manage the team from a very remote
place. There was no technology at the time, at least at that office that we
could rely on because the company didn't have Slack. It was nothing back then.
we could've signed up for WebEx but it was too expensive. We didn't really have
anything other than a phone. Get that. I was a remote team member, being a
Scrum master, and all I had was the phone. A lot of times, what would happen is
they would have team meetings and they would forget to call me, because the
phone exchange didn't allow direct call in, if they didn't call me, I just
didn't go to the meetings.
became very familiar with the angst of being that remote team member and came
up with a long list of things. It wasn't that long. it was top three things
that I needed that team to do, which is not forget that I was up there.
played fun games, we printed out a picture of me. We put it on the conference
machine so they would bring it around the meetings so they will remember that I
was to be called on the phone. Simple things like that. Now that we have all
this technology out there, there are a lot of other things that e can do to
integrate that team into a central office.
of the best things I've seen a team do is have one of those gigantic monitors
in the center of a team area and actually having a running live feed of folks
to actually join in video conference. Whenever they are thinking to say hi to
people, they can just dial in and their face shows up on the monitor.
a little awkward if you're in a large office where lots of people walk through
all the time. If you are in a small situation where you can actually dial in,
you can see people sitting around. It's fun to see that.
other thing that is really important is, those water cooler conversations are
not going to stop happening. They are always going to happen. That's human
nature to have a conversation with somebody. If you are sitting right next to
them, to not have the conversation is a little weird. You're going to wind up
having a conversation.
things to understand though is that when you are making decisions, the first
thing I always like to tell people is, "Who you actually bring to the
decision making room is actually a pretty important topic." If you are
going to randomly make decisions in the hallway without all the people who need
to be in that decision, it's probably not going to be a very good way for you
to actually get consensus or drive change to your organization.
you're going to make a pretty impactful decision, you probably should make sure
that you have all the decision makers in the room when you're doing that. Yes,
it does slow the decision making down, but in some cases it probably is a good
you're making an architectural decision about your product line. Something
needs to change. You're going to switch one technology in for another
technology. Is it appropriate for you to have that decision made over a water
cooler or should you bring in the extended team who is going to be impacted by
this decision to get their feedback on that.
number two is probably better than option number one, but it also doesn't mean
that every single decision has to be in that central decision making body. I
would also say that there are probably some decisions that aren't going to
really matter like, "What time do you have your stand‑up meeting?” If you
guys want to make a decision and simply say something to the tune of,
"Five out of six team members think we want to change it to this time, is
it OK with you sixth team member? Would you mind if we do that?"
to delay that conversation until everybody is on the phone may not make sense.
It's a matter of determining the decision that need to be made or the question
you're asking and how critical is the question, how much impact is it going to
have? What's the risk level of the decision you make? Is it really risky? If
so, you probably want to talk to more people to reduce the risk of what you
might be deciding to do.
Maybe related to that, one of the things that I've often found challenging
in my career when I've been dealing with folks who are maybe not right next to
me, maybe just in another building in the same location, there are a lot of
decisions and a lot of useful conversations just to flesh out thinking, that
maybe don't fall into the formal decision making bucket, but they're just part
of the day to day interaction.
find a lot of time, when people are remote from each other, it's harder to have
those conversations because you feel like, "It's not really worth bothering
that person with a phone call to talk about this half‑formed thought I
have." What are some ideas you have about dealing with that kind of thing?
It's interesting because a lot of Red Hat doesn't have...maybe they do,
but my observation is, a lot of the engineers I work with don't fall into bad
habits because they generally use IRC to communicate.
watch this every morning, when I get into work, people say, "Good
morning." Some folks chat about what they did the previous evening. There
is a connection that people make during the day. They have general conversation
during the day that helps with that cross‑communication.
interesting part about that is to layer that team dynamic on there where you've
got most of the folks located to one office and a few people distributed, I
would imagine that a lot of the conversation and the general chit chat happens
around cubicles, happens around whatever space that team occupies and rarely,
if ever makes it back to IRC, or whatever communication channel you choose to
use, whether it be email or maybe Skype, whatever you're using.
important for teams to understand, especially if you're committed to doing
something together as a team, that you have a social responsibility to the
folks that you're actually interacting with. It's not just, get on your team
meeting and immediately start talking about work but it's asking somebody about
their day, or finding out what they did over the weekend, or trying to make
some sort of social connection so that it's easier for you to have casual
your own case, you might feel like you can't reach out to somebody and share
that half‑baked idea because you don't know them too well. If it was somebody
that you knew, you wouldn't hesitate to pick up the phone and have that
conversation with them.
When was an industry analyst for a number of years, one of the things I
found was that I can certainly connect with people over the phone and have good
personal connection with them, but it was much easier if I had actually met
that person in real life and had dinner with them, had some beers with them or
comes back to, there is some value in these high‑bandwidth, in‑person
communications because often, once you've established those things face to
face, that's probably an argument. For example, having a commitment to things
like team off‑sites, if you are possibly in a position to afford them.
Companies probably make out a little bit by having distributed work
force, especially if they're not paying for co‑working space or Internet or a
phone, they're putting that burden on their employees, they probably are making
a little bit of money by not having to pay for office space or maybe not making
money but having it reduced overhead for having to pay for something...a
gigantic office building in downtown Raleigh.
critical point is that it's not as easy to just say, "If I am not paying
for a desk, that means I have X dollars free to do whatever else with that I
need to," because there actually is the bottom line impact to the way that
people are able to work together. Often, because it's not a quantifiable one,
as in I can't establish that if I don't get Team A in the same room at least
once a year, I can say for sure that they'll have a decrease 10 percent of
production. I can't say things like that.
can't quantify that to a finance person and say, "If you give me $5,000 so
that they can all travel to a team bonding exercise or whatever you want to
call it, I can ensure that they will have a steady line production for the rest
of the year." I can't say stuff like that.
evidence indicates to me that it actually is a critical and important part of
everything that we do as an industry. One of the things that I have had happen
recently is that I did work with a team that was very distributed. They were
from all over the world. They were not really brand new together.
were having a little bit of trouble actually working well together and coming
up with just a general backlog. Everybody was new to whatever it was going on
on that space. We brought everybody together so that they can meet each other,
have food together. People bond while eating. It's just a thing. It's a
universal thing, whether it be beer in the Czech Republic or Italian food in
Little Italy in New York City, people bond over eating and drinking. It's just
a thing that we do as humans.
giving them the opportunity to do that bonding, they just worked better
together afterwards. That face‑to‑face interaction, it's critical for me,
especially being a distributed employee to have that bond with people on a
I still remember a few years ago. It must have been a LinuxCon, it was
one of the Linux Foundation Events. There was a kernel panel talking about how
the development process worked with Linux, kernel and the like.
of the kernel developers, one of the core maintainers, I don't remember which
one it was, but he made a comment about one of the reasons that Linux kernel
development has historically worked as well as it has, is that fortunately a
lot of people involved...certainly, there's a lot of individual contributors as
well but a lot of the core people involved... work for pretty well resourced
companies, IBM, Red Hat, Intel, Hewlett Packard, and so forth.
provides the opportunity and the budget, frankly, that people can get together
in Linux Plumbers events, LinuxCons and the like. From their experience, this
has been a very important part of Linux kernel development historically even
though on a day to day basis, Linux kernel developers are all over the world.
They cite the social interaction they're having as their ability to long‑term
work well together.
switch gears a little bit and talk a little more about some of the benefits you
see with specific tools. You raised a great point about IRC and Slack...call
them messaging but in a sense, they're almost ambient distributed conversation.
add to your point, when I had this feeling in my product management days that I
had to go around and have in‑person conversations because things weren't
important enough to bother some of them with an email or with a telephone call.
One thing just those kind of tools provide is they do allow for this
asynchronous, non‑interrupting type of conversation to take place.
It's interesting too because a lot of people think that they need to read
everything that comes across IRC. I choose to read when I am available. If I've
got a gigantic back load of things that I've not read yet, I just say,
"Forget it," and start wherever I am at that given moment.
is ambient conversation. It's not conversation that should be...if we're
talking also too about that decision making, IRC is probably not a great place
to do decision making because unless you're paying attention in the moment,
it's likely that you won't even notice that a decision was made, or if you're
trying to make a decision at IRC and somebody makes a decision, make sure it
gets logged somewhere else so that you can actually go back and look at it
is better for that because if you've got Enterprise Slack, you can watch and
you can read the entire history. You can see some of that stuff. I know there
are ways in IRC that you can also do that. Obviously, I've got a proxy set up
so I can see the entire backlog. Again, it's like the social existence around
those messaging systems where it's not necessary or mandatory for you to read
everything. It is really just general chatter.
like whatever you would be experiencing while you're in the office, you'll just
listen to people talking. I was just sitting over in the breakout room and I
could see people just having random conversations, I could hear them doing it.
I didn't necessarily need to listen for content. I just knew that they were
having a conversation.
a way to pass information back and forth pretty quickly but it's also a way to
feel maybe a little bit bonded to the people that you're working with even
though you're not participating in that given moment.
We've talked about documenting decisions and we need to make certain
decisions, make sure everybody in the team knows. What are some of the
practices that you use for documenting decisions that everyone has participated
The easiest way to say that is that if you are working at a product line
and you have some sort of roadmap of what you're trying to develop for the year
or maybe even a given release. The easiest way to track those decisions is to
associate the decisions directly with the work that you're actually doing.
the space of Scrum and Agile, we call them user stories. We tag them back to
problem statements in which we're trying to solve something for our customers.
They go through this, not massive process, but a pretty well‑defined process of
how we get from problem statement to actual user story where our team is going
to do something, but if you are making a decision about, "I'm going to use
technology A versus B to solve this problem," and you decide to go with
technology B, the easiest place to put that decision is on that user story.
it's harder to capture that information for product line. If you make decision,
we're going to go with technology B, how does your customer know? 10 years from
now, how do you remember that you actually made that decision? The best way to
do that is just to document your product.
don't really know any better way to do it. Maybe you could have a system to
track decisions and you can go back and you can say, "On October 1st made
a decision and this is what the outcome was,"
don't really know how that's valuable if it's not tangentially related to the
product that you're working on. Say you're doing a CICD system and you choose
Jenkins as your technology, I don't think it matters who made the decision or
when you made the decision but rather, was it implemented, is it in the product
line, can you document how to use it or what the implementation details are?
Those are the things that matter.
when the decision was made and who made it feels maybe a little bit like you're
feeling untrustworthy about who made the decision or why was the decision made.
Associating back to the actual work that you're doing is probably the best
place for you to be.
Continuing on the theme of tools, one of the things that's been percolating
for a long time in business is video conferencing. It would be fair to say that
historically a heck of a lot of money has been wasted for not much effect. How
do you feel about the current generation of tools and what their value is in
terms of teams working together?
I don't know anybody who doesn't complain about video conferencing. I was
legitimately just at Agile 2016 and listened to some hallway conversations
about people complaining about video conferencing. I just took a casual poll
where, what are we using? All the big names came up.
goes through this 5‑ to 10‑minute pre‑conference...there's Internet memes about
this. Pre‑conference, who's on the phone? Dialed in, interrupting. All these
things have happened.
are definitely improvement from what I had when I was working and that sole
employee, remote, because I had nothing. It's 2016, it should be far better than
it is. It just doesn't seem to be any better in the last five years than it was
when I first joined Red Hat, really. It just seems to be not going anywhere.
We can all agree that some of the messaging tools, for example, are
really quite good now. Without starting a traditional Internet versus software
as a service knockdown, you'll find certainly Slack or potentially other tools
in that general bane. It has legitimately moved to promote collaboration
can remember having conversations with Novell maybe 10 years ago when they were
doing a lot of work on collaboration. Wouldn't it be nice to be able to do the
equivalent of standing up at whiteboard in a room with a bunch of people
sitting there and have that kind of interactive, very rich visual experience?
The fact is, it's still pretty bad.
That's actually a pretty interesting conversation because I've been
looking for whiteboarding programs for a while. Just so that we're all clear,
especially for folks listening to this podcast who might be creating a for‑pay
tool that's really fantastic, I'm looking for free things, things that my Linux
users will feel comfortable using.
space here is not that broad. There are some really great things out there but
they're not necessarily perfect. They're still worlds away to go.
was actually talking to one of my friends a couple of months ago about...I
guess they were using these smart whiteboards that they purchased for one of
the school systems. I was asking her how it was going. She said it was a
complete disaster because no one knows how to use it and so they were
immediately broken. They've got these $25000 whiteboards that no one knows how
was a complete waste of money and time because they don't understand the
technology. You can get these really fancy things that unless you really know
what you're doing, you don't know what to do. That's sad, right?
Yeah. I'd hate to admit this but probably 20 years ago, having things
that take pictures of a whiteboard or that you could use different colors and
write and it would supposedly put this onto a computer screen, none of that
stuff really worked. I'm sure we spent at least a few tens of thousands of
dollars in this stuff at a former employer that never really worked. The really
sad thing is, I'm not sure we're an awful lot better today.
No. A lot of the video conferencing stuff is always going to be
challenged by networking and bandwidth caps. Unless you are somebody who wants
to pay a significant amount of money for your home office Internet connection,
you're going to be challenged by that. That's not something that I
think...technology obviously will make that better as time goes on. Right now,
given the state of things in general, it's a goal that's far out in the future
to be able to stream video back and forth like that, and not have problems.
I've heard some of the very high end Cisco systems. You have a special
room and you have someone directing the cameras. It can be very good. Of course,
at that point, pretty much everybody says, "Let's just fly together to a
nice place." [laughs] To get there.
I have a lot of friends who work at Cisco and they claim that these
systems are fantastic. I believe them.
I'm in the cloud product strategy group at Red Hat. Prior to Red Hat, I wrote hundreds of research notes, was frequently quoted in publications like The New York Times on a wide range of IT topics, and advised clients on product and marketing strategies. Earlier in my career, I was responsible for bringing a wide range of computer systems, from minicomputers to large UNIX servers, to market while at Data General. Among other hobbies, I do a lot of photography and enjoy the outdoors.