Enabling Others Through Code

At Delphix, we believe that we are most successful when united together as one team. Where your ability to enable other’s success is more important than what you can do with our own two hands. In engineering, this is ultimately represented in the core of our product: the code.

Conversations on code ownership often focus on writing code over understanding code. At Delphix we embrace “weak” code authorship, but that’s only part of the equation. The real question is not who gets to make changes, but what is their responsibility to others when writing code in the first place?

Problems Don’t Respect Boundaries

No matter how well you craft your APIs or modularize your code, real-world problems are going to run across and through your code. In all likelihood it’s not going to be you chasing those problems. So you can choose to put yourself in the critical path, slowing down analysis and impacting your productivity, or you can work to support the problem solvers.

“Programs must be written for people to read, and only incidentally for machines to execute.”
— Structure and Interpretation of Computer Programs, 1985

An old friend worked with an engineer, let’s call him Bob, that insisted that he had a better way of writing C code. He would include a header, <bob.h>, that would provide his own macros like IF_THEN() and insisted that any file he worked on must be written in “Bob C”. Suffice to say it did not work out well for the team. When code is unfamiliar and confusing, the overall cost to the organization is far greater than any imposition style consistency might place on any one developer.

Style rules help people move between different bodies of code, but they must be implemented correctly. One mistake we made at Delphix was failing to build tooling to help people write and verify code with correct style. This led to manual edits and nit-picking during the review process, degrading the quality of the reviews and decreasing engagement in the process. Style rules must come with automation — from IDE configurations, to runtime checks, to build integration.

Yes, There is a Bus

The “bus factor” is the minimum number of team members that have to suddenly disappear from a project before the project stalls due to lack of knowledgeable or competent personnel.

While it can seem impractical to design for the case where someone is going to actually be hit by a moving vehicle, “buses” in the real world are much more subtle. It could be that you’ve been pulled off onto another project, have transitioned roles, left the company, or are on vacation at a critical point.

You need to write code for the future reader assuming neither you nor anyone on your team today is around to help them.

There is a big organizational piece to this, but code is also a critical piece. It starts with good code comments. Comments describe not just what the code is doing, but why it exists and how it was designed. Reference living bodies of information in a consistent fashion, and avoid large design documents separate from the code. At Delphix, we link commits to JIRA issues that are accessible by non-developers and can track customer cases, regressions, etc, that emerge only after the code is pushed.

Code reviews are another useful tool to build knowledge in others. Wielded incorrectly, they can be toxic — full of nitpicking style, attacks on foundational design decisions, and lots of boxes to check. Such environments suggest a gap elsewhere, either in automation to correct nits for you, or in team collaboration to make sure design discussions are hashed out well before the code review stage. But done correctly, reviews keep critical bugs from escaping into the product, validate that code is understandable by others, and build up context outside the primary author.

Your Code Will Misbehave

Nobody writes perfect code. Inevitably, code will be used in ways the author did not anticipate. In these cases, engineers must be able to assess that the code still works under these new conditions — requiring well-designed tests that are accessible to those without deep knowledge of the code.

If a unit test exists, but no one ever runs it, does it actually test anything?

When I joined Delphix there were lots of tests — thousands of them. But each module implemented tests differently, often requiring byzantine manual steps, to the point where no one knew how to run all the tests. Most of us would resort to manual testing and hoping for the best.

Fast, reliable test automation that is accessible to everyone is a necessity, and we’ve come a long way at Delphix. An early misstep, however, was when we allowed projects to integrate separately from their tests. This helped development and QA operate independently, but exposed the organization to incomplete state — without the tools to verify functionality as the product evolved.

I have always loved creating things. It’s why I like woodworking, jazz piano, and cooking. That art of creation is part of what drew me to software engineering. Rules and processes that promote code consistency does not mean we much discard code craftsmanship. Rather, it’s about creating an environment that prioritizes understanding and problem solving through creativity and collaboration, while discouraging passive-aggressive and hostile territorial attacks. Some recommendations:

• Establish consistent style guidelines, but enforce them through automation, not code review.

• Encourage thorough documentation within the code, and minimize reliance on external sources of information.

• Establish peer code review, but make it lightweight and easy to engage the right people.

• Establish rigorous build and test automation, but it must be fast and easy to use.

Culture is a evolving concept, but we’ve laid a strong foundation at Delphix through approaches such as these. A foundation that can grow with the company and hopefully inspire others well beyond the walls of Delphix.