My Principles for Hiring Software Engineers

Hiring is difficult. Hiring reliable, talented software engineers is even more difficult. In fact, hiring is probably the most arduous task that managers must handle, no matter what type of business you are conducting. And, based on the topic of Marc Andreessen's famous article Why Software is Eating the World, every conceivable industry is now being dominated by companies that are adept at developing software. So, the competition for good software engineers is pretty steep right now and will probably remain this way for the foreseeable future.

TL;DR (too long didn't read):

1. The candidate must have a demonstrated hunger to continually learn 
2. The candidate must be a culture fit
3. The candidate must demonstrate humility

On with the background...

Throughout my years of hiring experience that began 20+ years ago, I have learned a thing or two. I've read many articles and books about hiring in the software industry and heard many opinions on the topic. The result of my years of experience is a set of principles that I follow when hiring. But before I get to these principles, let me explain what lead me to them.

Because I'm hiring from a management standpoint, I pay attention to aspects beyond just the candidate with the best technical skills and/or qualifications. I focus on team building which encompasses a myriad of aspects far beyond just technical aptitude. For me, team building requires the consideration of many different cross-cutting aspects beyond technical abilities. Below are just a few of the skills that candidates must possess:

• Exhibit the skills necessary for software development
• Demonstrate the ability to constantly learn and do it proficiently
• Want to work as a team and collaborate closely with teammates
• Communicate well
• Behave like a professional 
• Manage their own work schedule
• Fit the culture 
• Demonstrate empathy and humility
• Not be an asshole
• Etc.

Many of these skills can be taught and even bad habits can be broken with the right coaching, but the candidate must come through the door with some base skills intact. Let me walk through what I mean here. 

Consider trying to teach someone a complex task. First, if the complex task requires a large amount of prerequisite knowledge (like software development), then the candidate must demonstrate a certain level of technical aptitude. Let's assume they can successfully demonstrate their technical chops. Second, trying to teach someone is pointless if they are not interested in having others help them to learn or assume that they can figure it out themselves. In other words, sitting in the corner and coding away with zero communication with others is not allowed. Not only must the candidate be open to coaching and learning from others, they need to know how to learn and they must be proficient at learning. If the candidate possesses some of these base skills with the ability to learn and the willingness to be taught, then they can be brought up to speed over time. So, what I'm really looking for are some indicators of these characteristics in the candidates. These indicators are the principles that I have identified.

When I was asked some years ago to summarize these principles, I referred to them as culture-building. When building a culture, leaders must focus on the behaviors, expectations, norms and values that are most important for the environment and the work being done. Upon much consideration and over a long period of time, I was able to distill my hiring to the following three principles:

1. The candidate must have a demonstrated ability and hunger to continually learn 
2. The candidate must be a culture fit for the company, the environment and the people
3. The candidate must demonstrate humility and empathy

In my experience, these three principles serve as indicators for someone who might be a fit for our culture and the position. Neither are these principles generic nor are they fool proof. You must adjust for different hiring needs. Also, we have had a couple duds over the years, but our turnover rate is less than 10% which is extremely good. Such a low turnover rate is partially due to hiring well and partially due to the benefits, compensation and perks from working for a large, global company that takes deliberate steps to care for its employees. Without these benefits, all the good hiring practices in the world won't stop your turnover rate from increasing over time. I also believe that you should pay people enough that they won't go elsewhere only for more money.

These three principles serve as a litmus test for a broad set of aspects as I noted further above. Here is some explanation around each of these principles:

• The candidate must have a demonstrated ability and hunger to continually learn - I have met some engineers through my career who chose to go very deep with a very narrow skill set and no interest in spending time on things outside this narrow track. Without the interest to learn new topics and, the desire to do so all the time on an on-going basis, I have watched these people never move their skill set, never stretch themselves. For some positions, this approach is necessary. But for many/most positions, given the changing landscape in technology, I have found that the best candidates are always willing to make themselves uncomfortable to learn new things. Given the willingness to learn, a candidate must also be proficient at learning. If they find themselves pining away and making little progress, then they need consult someone they trust who can help them identify and conquer the hurdles they are facing. If you're willing to do this, it demonstrates a deeper sense of self which lends itself to things like helping others, being part of the collective, etc.
  
  
• The candidate must be a culture fit for the environment and the people - Culture fit can mean a many different things. But, as I mentioned above, for me this term embodies the behaviors, expectations, norms, thinking, values (what is acceptable/unacceptable), working time, work ethic, etc. During interviews with a candidate, I have found it to be invaluable to include folks from the team where the open position exists. Everyone should be coached to take note of both verbal and non-verbal cues from candidates as there are many signals to be picked out that inform you of their true nature. Then, assessing these signals after the interview with the entire panel of interviewers oftentimes winds up telling a much different story than any one interviewer can tell about a candidate. Also, the questions you ask candidates can help to reveal many things, so be conscious of what you're asking and why you're asking it.
  
  
• The candidate must demonstrate humility and empathy - This one is a bit odd for some people because it delves deeper into the psychology of a candidate, but it has proven to be very important measure over time. Anyone who has worked with people who have big egos will be able to recognize this -- the rock star developer who ignores input from others, the developer who sits in the corner coding away with zero communication, the developer who has high standards for themselves but does not want to take the time coach younger or newer team members, etc. When hiring members of a team, you are doing team building and this needs to be taken seriously. Teams of developers should operate like a team, not like a ship full of pirates who can't come together to deliver. This requires the ability to empathize with teammates, to assist one another, to exhibit a compassion for others, to be respectful, to be kind and to be humble. Without these characteristics, there may be serious disagreements on a team, even if those disagreements are not vocalized. Without an eye toward how one affects their teammates and how their teammates affect them, as a manager, you are just missing the boat. 

While these principles have worked for me, they may not work for others for various reasons. In the past, I have read that people don't leave a company, they leave a manager. While this can certainly be true (I've had my fair share of bad managers), I have left numerous companies not because of my manager, but because of the culture or the processes within the company or the lack of career growth opportunities. The three principles that I have outlined above assume that your company culture is healthy, useful and not problematic. If they are problematic, then as a manager you should be working toward improving them so that they are not problematic. When the culture and/or the processes are standing in the way of good hiring, all the principles in the world will not help you.

Installing PostgreSQL 9.6 on macOS 10.13.6 via MacPorts for Development Purposes

After finally receiving a new MacBook Pro with 2.9 Ghz, 6-core i9 processor and 32gb of memory, I am reinstalling all of my development tools. Not only is the big increase in resources really useful from the development point of view, but the computer is lighter and thinner as well. That being said, there are still things I do not like, such as the keyboard, the touchbar (I miss the tactile feedback from the escape key which, as a Vim user, I use constantly) and some other minor things. For my own purposes, I usually document the steps to make it easier in the future. But I also figured it might help other folks out there to achieve the same ends.


Having documented this procedure once or twice before, I'm going to keep this brief.






BTW, below are the start/stop scripts that I created for PostgreSQL:

GopherCon Denver 2018

Recently I attended GopherCon 2018 in Denver and really enjoyed the whole event. There were about 1500 people there and I really learned a lot. One of my teams writes Kubernetes operators using Golang which got me into writing some Golang myself.

On the first day, we attended a workshop titled Advanced Ultimate Go that was taught by Bill Kennedy from Ardan Labs and it was excellent. Not only is Bill incredibly knowledgeable with Golang (he has years of experience not only in Golang but also in C++) but he is also an extremely good teacher. I learned a lot in a single day and I would have loved to have this class extend for another day or even two.

Coming from a background of mainly Java for 20+ years, the Java community has an amazing piece of engineering in the JVM that handles most performance related concerns for us. We design Java classes however we please and we make use of any data structures we please and never really give it a second thought simply because the JVM handles all the real mechanics for us. With Golang, this is not the case. Although Golang has garbage collection (which is really nice, I hated using malloc() and free() in C) it's very different than what JVM does. Also, in many ways, the Go language is very succinct compared to Java. Both in terms of the amount of code you must write (Golang requires a lot less boilerplate code) to the ease of deployment with Golang (you have a single binary to deploy, no dependencies or CLASSPATH to manage). That being said, I'm still not sure that I would completely switch all web development away from Java to Golang. While the Golang learning curve is a lot easier than Scala, the issue I see are the implications on performance with the code you write. Whereas with Java, while there are performance concerns, as I said above the JVM handles a tremendous amount of things under the covers so that we don't have to care nearly as much.

Having spoken at many, many conferences over the years, I was very pleasantly surprised by the number of female attendees and speakers! I saw more talks delivered by women than I saw delivered by men which was excellent! The community seemed very open and engaging to everyone which I really liked. Being in the software industry for so long, it's still shocking to me how much of it is dominated by males. Anyway, I really enjoyed the cultural and gender diversity at Golang.

There is one minor change that I would like to suggest to the organizers of the Golang conference. Many years ago, I spoke at a conference in Denmark where I first saw this. On tables next to the doors at the back of a room where talks are taking place, there are big glass bowls with three piles of Post-It size paper, each pile a different color -- red, yellow and green. As attendees exit the room, they are asked to grab a single piece of paper in the color that represents how you felt about the presentation/talk. When the talk completes and the room is empty, the conference organizers gather the papers from the bowl, tally them up and provide the stats to the speaker. It's basically like a quick rating of what attendees thought of the talk. This small system does not replace the comment cards that organizers always ask of attendees because this is how attendees elaborate on they rated the talk the way they did. Both systems of rating are important because they deliver two different but equally important types of data to the organizers and the speakers. Anyway, this my two cents.

I really enjoyed GopherCon for a variety of reasons and I would love to attend again. Since the conference I have written a lot more Golang code and the more I write the more I like it. In my mind, Golang should be the goto language for systems programming, DevOps type stuff. While it can easily handle general web development tasks, I'm not sure yet if I would drop Java in favor of Go. I guess I need to keep coding away in both.

Vacation and Hiking in Crested Butte

Last week my family and I took a week's vacation in Crested Butte to do some hiking and relaxing in the beautiful Colorado Rocky Mountains. Although we missed the Crested Butte Wildflower Festival by one week, we still enjoyed the flowers on our hikes.

As we explored the Crested Butte area and enjoyed numerous hikes, we got to spend time together as a family which I really enjoy doing. It's hard to believe that my girls are basically adults now (they are 15 and 20 years old). Bailey even got a second week of altitude camp in for her cross country training by running every day that we were there.

My in-laws even joined us with our two nephews and we all had a very relaxing time together. We even rented some paddle boards and went to Lake Irwin one day. This wasn't something that would have been very easy for me to do, so I enjoyed watching everyone experience the paddle boards for the first time in a cold mountain lake.

Crested Butte is a sleepy little mountain town with a well-known ski resort. We enjoyed hanging out in town, checking out the restaurants and shops, and also finding trails to explore all around the area, including on the ski mountain. Unfortunately, the Crested Butte ski resort was recently acquired by Vail Resorts. This means that it will be completely over-developed soon enough, everything will become too pricy and too crowded with traffic just like all the other resorts it owns. What a drag.

Later in the week, we visited a friend in Lake City and hiked Spring Creek Pass, a high mountain pass with an average grade of 7.5% at 10,901 feet. My phone told me that I climbed 50 flights of stairs that day which was quite an achievement for my withered legs. When we returned to the car, we took a photo by the trailhead sign and Janene had her arm around me and said, 'Your legs are shaking, you need to sit down.' I just chuckled and said, 'Yeah, I know.' Even so, I loved it! I am so thankful to be able to feel that exhaustion from completing a difficult hike.

Forming Scrum Teams

The tongue-in-cheek fable of the chicken and the pig is frequently referenced in Scrum literature to distinguish between people who are committed to the Scrum (the pigs) and people who are only involved (chickens). The reason pigs are committed is they must be sacrificed for the meal whereas chickens are simply involved. When I was software engineer, we used to say, 'It's just us pigs here' when it was a group of software engineers together. Having transitioned to management several years ago, now I say, 'It's just us chickens here.' But I digress.

In this post, I want to discuss my logic for forming Scrum teams, a topic that I have hashed and re-hashed many times throughout my career. So, here I will explain the logic behind how I form software engineering teams.

The Agile Manifesto

The Agile Manifesto lists the following 12 principles:

1. The highest priority is to satisfy the customer via early and continuous delivery of software
2. Welcome changing requirements
3. Deliver working software frequently with a preference to the shorter timescale
4. Business people and developers must work together daily throughout the project
5. Build projects around motivated individuals
6. The most efficient and effective method of conveying information to and within a development team is face-to-face conversation
7. Working software is the primary measure of progress
8. Agile processes promote sustainable development
9. Continuous attention to technical excellence and good design enhances agility
10. Simplicity is essential
11. The best architectures, requirements, and designs emerge from self-organizing teams
12. At regular intervals, the team reflects on how to become more effective, then tunes and adjusts its behavior accordingly

While these 12 principles define much of Agile in the broadest sense, there are variations in different Agile methods such as XP, Scrum, Lean, Kanban, etc. These methods are not meant to be followed dogmatically, instead practices within a particular method are meant to be guidelines. In this post, I will be focused on Scrum, specifically on some of the aspects of team formation. But keep these principles in mind as you read this post.

What Does Scrum Recommend?

The Scrum literature discusses team formation a lot. From this literature, it's easy to learn that there are  some key principles in Scrum focused around forming small, cross-functional teams. But Scrum best practices don't end there. There are many other aspects to building Scrum teams. In this post, I will elaborate on the following:

• Autonomous and self-organizing
• Team composition must be cross-functional
• Ideal team size

Autonomous and Self-Organizing

As stated in many Scrum publications including the Scrum Guide, a Scrum team should consist of a Scrum Master (SM), a Product Owner (PO) and Developers. Scrum teams should be cross-functional and self-organizing, i.e., they must be capable of delivering value each sprint and they are responsible for deciding how they work. This is what I refer to as setting up a team for success. It's the job of the leader to set up the team for success, not only in the structure of the team but also in communicating the expectations for the team. I have always defined expectations in black and white for everyone to see so there is no question about their responsibilities. (In fact, I have even implemented this model across all engineering teams for a product line by working very closely with my management colleagues. There must be consistency of expectations across all the product engineering teams. I cannot stress this enough.)

It is critical that the team members understand and embody their autonomy and empowerment to manage their own work. Typically an engineering manager is involved and accountable, but the team is responsible (from a RACI matrix point of view). In my opinion, the best case scenario is that the engineering manager is more of a coach and not micro-managing everything a team does. Team members must be coached to step up whenever they can and remember to bring everyone along on the journey. This requires well-structured and open communication across the team. Oftentimes, the Scrum Master is the person who guides the team most in terms of self-organization and responsibilities.  It is also important that team members understand the role that everyone plays on the team. Like a set of gears moving together as one, it only works if all cogs are operating correctly and together.

Self-organization is an important aspect for a team to operate well together. Team members cannot operate together without a shared understanding of what this means. In Scrum, this means the PO defines user stores for team and the SM leads the team members in breaking down those user stories into subsequent tasks for estimation and execution. The SM oversees all of this activity and works closely with the PO to determine priorities for the sprint. While people not deeply experienced in Scrum think this is too much for a team to handle or feel that the manager should drive all team activities, this couldn't be further from the truth. As with many things in software, concrete experience teaches a tremendous amount very quickly. As a people leader, it's very easy to see the differences between a mature Scrum team and an immature Scrum team. Like many things in life, reaching a level of maturity as a Scrum team requires proficiency and time, especially if new team members are added over time. As the Agile Manifesto mentions: 

Build projects around motivated individuals. Give them the environment and support they need, and trust them to get the job done.

I work very hard to build trust with my engineering teams as I believe this is the basis for building great teams. I always say that I hire talented individuals who are experts and then I give them room to get the job done. The bottom line is the team is accountable for delivering value every sprint and I have found that building trust is a requirement. I extend trust to them to get the job done and they teach me to trust them by consistently delivering value every sprint.

Team Composition Must Be Cross-Functional

As mentioned above, a Scrum team needs to be cross-functional so as to deliver value every sprint. As much as delivering value is a generic term, when developing microservices it means that the team can deliver, own and operate those microservices itself. The team should not need to rely on other teams for different job functions as doing so amounts to dividing up teams based on function. This type of team organization was an idea from the 1990s and Waterfall methodology that divided up teams based on job function, e.g., front end engineers in one team, back end engineers in another team, QA in it's own team, tech writers in their own team, etc. I've seen this before and it really falls apart quickly because departmental divisions do not work in an Agile environment. It's a mismatch of models. Agile is supposed to be iterative and incremental. When teams are defined by job function they are not set up to be incremental. A true Scrum team must be composed of the constituent parts to deliver the software owned by the team. This requires a cross-section of roles composed of all the necessary disciplines. As mentioned above, a Scrum team should consist of a Scrum Master (SM), a Product Owner (PO) and Developers:

• Scrum Master
• Product Owner
• Front end engineers (developers)
• Back end engineers (developers)
• DevOps engineers (developers)
• QA engineers (developers)
• Tech writer (developers)
• Manager

In the list above, I included a couple of roles that you don't normally see in a Scrum team -- a tech writer and a manager. I also break down the different types of engineers. So, let's discuss each one.

Tech writer: I include a tech writer in my teams whenever I can because I do not consider the product to be Done until everything is actually done including the coding, testing, documentation, performance considerations and operational requirements.

Manager: I include the manager because this person serves a vital role not only in terms of people reporting lines but also in terms of developing a contextual understanding of a team and its work. Speaking from experience, without a manager present, it becomes very easy for software engineering teams to get pulled in many directions. A manager can guard against not only this but also against members of a team reporting to numerous managers which can result in a lot of chaos. 

The other roles are simply a clarification on the types of developers. Furthermore, it illustrates the inclusion of the different types of developer disciplines -- tech writers, QA engineers, DevOps engineers, etc. None of these disciplines or roles should reside on other teams. When this happens, the team will not develop the right level of cohesion and trust necessary to become what Google identified as an effective team (this is a whole separate topic for another blog post).

Ideal Team Size

I have seen Scrum teams with three engineers and Scrum teams with 30 engineers. The goal should be to find an ideal size that works for the products being developed, but I do believe smaller teams are better. According to the experience of many Scrum practitioners, the ideal team size should be 6-8 engineers (give or take one or two) not to exceed 10. From my nearly 20 years of experience practicing Agile, the larger an engineering team grows, the more difficult it becomes to meet, to scope work, to keep track of work, etc. Basically, the larger the team the more complex and time consuming communication becomes. It's not called Agile for no reason -- software engineering teams practicing Scrum are meant to be nimble and iterate quickly. This cannot take place when too many people are involved. 

Consider Brooks' Law about software project management which says adding manpower to a software project that is behind schedule will delay it even longer. This maxim emerged from Fred Brooks eponymous book The Mythical Man-Month. Brooks explains that while cost varies with the number of men and months, progress does not:

“Hence the man-month as a unit for measuring the size of a job is a dangerous and deceptive myth. It implies that men and months are interchangeable. Men and months are interchangeable commodities only when a task can be partitioned among many workers with no communication among them… This is true of reaping wheat or picking cotton; it is not even approximately true of systems programming.”

This is further elaborated by examining the lines of communication for teams of different sizes. As explained in a blog post titled, Lines of Communication and Team Size: Applying Brooks’ Law: 

Most teams in large enterprises have between 10 and 20 members. Even at the low end of that scale, a team has so many lines of communication that progress is bound to be slow.

(source: Lighthouse)

Note the increasing number of lines of communication as a team grows. This is essentially the root cause of why teams beyond the Scrum recommended size become more and more ineffective as they grow. The drag in trying to develop a shared understanding of anything simply becomes more and more complex the larger a team grows. This is typically when people start to complain about there being too much administrative overhead. 

Conclusion

To summarize, if you want to set up your teams for success, then they must be autonomous, self-organizing, cross-functional and sized appropriately for the software being built. This is at least part of the recipe that has helped me to build what I mentioned above that Google calls effective teams. 

Resources

• The Law of Contraints and Brooks' Law
  
• What Does Scrum Mean by Cross Functional Teams?
• Cross Functional Doesn't Mean Everyone Can Do Everything

More Muscle Movement Discovered Recently

Recently I paid a visit to my physical therapist. I only see every 2-3 months to check in and guide me on the physical therapy work that I do on my own. She always checks out my progress to see how my body is doing and this time she found something new -- the tibialis muscles in both my legs are moving now!

The anterior tibialis muscles are what allow you to flex your feet upwards (this is called dorsal flexion). So far, I have only been able to press my feet down (this is called plantar flexion) because this is controlled by the calf muscles. But now that the anterior tibialis muscles are moving ever so slightly, I can begin to rebuild these as well. Just like my calves are taking time, these muscles will also take time to rebuild as well.

I am still working to rebuild my calf muscles and this is slow. My physical therapist reminded me that I'm not just strengthening my calves. I literally had no calf muscles left. So I am rebuilding my calves from nothing which is much more difficult. She also said that at this point, we have no idea if the calves are fully firing or not. In situations where there is nerve damage, you never know if you are getting a full squeeze from the muscle or if the whole muscle is firing yet. This makes the work much, much more difficult. She also told me that this is why most people give up.

My calves are increasing in size and strength, but they are very far from 100% functionality or strength. I still have a long way to go. But I still feel lucky to be where I'm at today with my body still healing.

New Braces and Hiking in Colorado

After about a year-and-a-half of wearing my old carbon fiber AFO braces (Ankle and Foot Orthoses), I recently got a new pair of Phat Braces which are also made of carbon fiber, but have a much better warranty and are widely used by people everywhere.  The big difference between my old braces and the new Phat Braces is that the Phat Braces are taller and stiffer (but they are beginning to soften a bit). They come up my leg to right below my knee which is further than my old braces . This makes them much more stable and it which allows me to balance and walk much easier. They also have some flexible plastic that wraps around the foot (as you can see in the image to the right) which also helps to provide more stability. The biggest benefit about them so far, however, is that it did not take my body six weeks to adjust to them. The previous braces actually took six weeks for my body to adjust and I was in pain the entire time. The company that provided them told me that's just how it goes. Through that adjustment period, I had to have at least a dozen manual adjustments to the carbon fiber (e.g., heat them up, bend out here and there, etc.), probably closer to 18 or so. With the new Phat Braces, I've only had two adjustments and my body has already adjusted to them -- literally less than a week. In fact, I have had the Phat Braces two weeks now and yesterday I did my first true Colorado hike since my injury in April 2014!

Yesterday we decided to go hiking in Evergreen, CO because we were trying to get back to the spot where I proposed to Janene 20 years ago. We thought it would be cool to go back there because later this month Janene and I will be celebrating our 20th wedding anniversary. I was a bit intimidated when we started the hike because of the elevation gain on the trail and the number of large rocks that you hike over on the trail. I did take a single arm crutch with me but it almost made things more difficult because of the angle at which you hold the arm crutch vs. the angle of the rocks on such an uphill elevation. Also, my new braces make going uphill difficult because they are still stiff, but they will soften a bit more in time. But with Janene's help, I completed the hike. Janene did make a good suggestion that instead of using an arm crutch I should consider getting some hiking poles. Because you hold them at a different angle, it could make going uphill and downhill over rocks easier for me. So, I'm going to try some out soon at REI.

Although the distance was not that great (1.7 miles), this was the most uphill/downhill I have done since my spinal cord injury 3.5 years ago -- I actually impressed myself. As proof of the level of workout for my body, my lower back and my hips were really tired after the hike and sore this morning. But I really enjoyed getting out for a hike with Janene and Bailey. So, I'm really looking forward to doing more hiking. I guess I can really start enjoying the fact that we live in Colorado again!