From documentation to empowerment

To effectively make technical decisions we need to share a common understanding on the context in which we work. Thus, we need to know “how did we get here?” and then “where do we want to go?”.

---

Documentation

To understand “how did we get here?”, we need documentation.

Software documentation can have a positive impact on the software development process. It can affect the onboarding experience of new engineers, the distribution of knowledge among the engineering community, and the transparency around architectural evolution as well as the decisions around it.

From the Agile Manifesto we know that working software is considered to be more valuable than comprehensive documentation. Agile documentation are user stories. Still, there are aspects of the software development process that go beyond the scope of user stories and also require to be documented differently. This documentation is usually neglected and there are multiple reasons for this:

1. The value of this documentation is underestimated.
2. Lack of awareness about what and when to document.
3. Significant effort is required to address the previous points.

Yet, good source code is often the best documentation, right?

Yes, but not everything is reflected in the source code. Some of the most valuable pieces of information that tend to get lost during the development process are the reasons to implement something in a certain way and which decisions were made along the way.

Everything doesn’t need to be documented. At Babbel, we concentrate on documenting technical decisions that have significant impact on the way we work, those that shape the trajectory we aim to follow. We document decisions that have major or long-term intentions. This kind of documentation will give us the ability to understand our current situation and will give us context from which to grow. They will support us figuring out “where do we want to go?”.

Using a standard documentation format helps us answering the right questions during the decision-making process. We use a well-known pattern called Architectural Decision Record (ADR) [1][2]. We designed a consice and simple format around this pattern. We agreed to use a common template which implements our standard format to document technical decisions. They are stored as Markdown files and contain the following information:

• Header: The signature of the document. Contains data about the creation, the review, and the status of the document.
• Preface: Contains information regarding the context of this document describing background information, business and technical requirements and constraints, and references to other related documents.
• Decision: All relevant information regarding the evaluation phase of all the different alternatives and the final recommendation, including benefits, potential risks, and consequences.

We agreed to persist the ADR in two forms. Decisions that may affect multiple projects or potentially have a wide impact radius are documented in a dedicated repository. This a global repository accessible to all Babbel engineers, is as any other repository in our organization. Decisions that are local to a particular project, where the impact radius is limited, are documented in the same project’s repository and, thereby, live in close proximity to the relevant source code.

Outdated documentation is often worse than no documentation, and we know it’s hard to keep documentation up-to-date. By documenting technical decisions using the same practices we use for software development, we reduce maintenance overhead and we are able to review this documentation the same way we do for source code.

Reviewing documentation as code creates an opportunity to have open reviews for significant decisions before we implement them. By making reviews open, we win all of the advantages of InnerSourcing. To make the review process more efficient, we defined different groups of engineers responsible for reviewing proposals on different domains. We called these groups Architectural Domain Groups (ADG). They are expected to give feedback and validate the proposal. ADGs are composed of members from different teams, which enables cross-team collaboration throughout the engineering department.

ADGs are also responsible for preventing invalid and obsolete documentation as well as inefficient documentation practices. They should distribute the overhead for the maintenance of the documentation to a minimum group of engineers. The ADG is responsible for:

1. Transparency around documentation topics and changes (e.g. update the documentation agreement, lower entry barriers for engineers, make people aware of documentation changes).
2. Maintenance of general or domain-specific architectural decisions (e.g. create, review and update/deprecate).
3. Evaluation of proposals and assessment of solutions around a new decision record (e.g., collisions to prior decisions, incomplete arguments, unclear assertions, impact radius).
4. ADR life cycle: call for periodic reviews and keep the documentation up-to-date (e.g. review/update/deprecation).

We created a handful of ADGs for the main domains we deal with, such as Backend, Mobile, Frontend, and Infrastructure. Also, there is an ADG that takes care of the process and the other ADGs. We called it Architectural Knowledge Management (AKM).

A generic use case

How teams and ADG should cooperate is described in this workflow chart:

Let’s say, a team is responsible to deliver a solution for a given problem. To implement this solution, the team has made a decision that will change the architectural direction of a component/service. Then the decision-making processes will be something like the following:

1. The team will write an ADR to describe their decision based on the ADR template and create a PR.
2. The team will inform an ADG that this ADR exists by adding them as approver to the PR.
3. The ADG will assess and validate the decision and involve other teams if necessary.
4. The ADG will give feedback about the decision described in the PR, looking to find an agreement in case the decision is questioned.
   1. The team may take the feedback and modify the decision as necessary.
   2. The team may reject the feedback and deliver arguments to backup their decision. This must be documented.
5. The team will modify the ADR to reflect the agreement on the final decision.
6. The team owns the final decision and can implement their solution to the problem.

It’s possible for a team to reject feedback coming from the ADGs. However, this must be documented, as the team will own the final decision and all the ramifications that this may cause in the future. Other engineers that want to implement solutions based on this decision should be supported by this team.

Empowerment

This practice empowers engineers to make decisions and implement them, as they are not made by a central entity. We believe this practice motivates engineers to take more initiative and develop more ownership over their work. Even though any decision must be reviewed and validated by an ADG, the last word belongs to the engineers that are presenting a proposal. They are responsible and will be held accountable for the solutions they implement. With this approach we want to address some of the topics listed in Nehal’s blog post How we work, specifically those points regarding “ownership”:

• Make informed decisions about tools & technology.
• Clean up technical debt.
• Challenge when something is unclear, unfeasible, or disadvantageous.
• Turn product requirements into working software.
• Deliver high quality.
• Contribute to the department and company.

We aim to improve collaboration by creating a channel for cross-team cooperation and by making it a standard step of the development process. Documentation is not a popular topic and the notion of not having a central entity of responsibility was challenging for some. Considerable communication was necessary. We made multiple presentations and went through many open discussions.

Currently, the general acceptance is good and we have a handful of global ADRs and many more in individual repositories. We feel we will reach our goal once this practice is embraced as part of the development process by everyone in the department. We are very optimistic about this, because the whole process is driven by our engineering community.

Photo by visuals on Unsplash