Modularizing our Rails Monolith with Packwerk

At Babbel, the B2B Product & Engineering group is working with a Rails monolith that houses the core business logic for our B2B and B2B2C product offerings; this monolith has grown significantly over the past five years, with contributions from over 20 developers. And one of the active efforts that the team started to help maintain it and manage its growing complexity is the modularization initiative.

Problem and solution

Our group started as one team working mainly on a single Rails app creating Babbel’s initial B2B offering. Within the last 2 years, the team grew continuously and was split into 2 and then 3 teams to evolve and create more product offerings. Now, we have 3 teams working on a Rails monolith at the same time. There are areas of the codebase that are relevant to the 3 teams, and unfortunately, these areas are tightly coupled and complex, which creates a lot of bottlenecks and slows down the pace of development.

As our Rails monolith continues to grow in the course of development, and new business features and requirements are added, it becomes clear that we need to think thoroughly about the structure and organization of our codebase, and we may even have to look beyond the MVC pattern and what Rails offers by default.

The team discussed a couple of alternatives including microservices, Rails engines, and Ruby Gems; we compiled an architectural decision record in which we defined the alternatives and criteria; we systematically compared pros and cons and evaluated what makes sense in our situation. I will not go over each option in this article, but I can say that microservices and Ruby Gems are used widely at Babbel so they were a favorable choice. However, we agreed to go a different direction: we decided to experiment with Packwerk for one quarter.

Packwerk is a library created by Shopify to help modularize Rails apps and enforce boundaries. Essentially, it is a static code analyzer that assumes you have a valid package system and enforces some rules and boundaries between packages in this system. It aims to have high cohesion and loose coupling and enforces communication only through public APIs between these packages.

Implementation

We decided that we want to create at least 3 packages for our experiment and one of these packages should be a dependency of another. We hoped that this would give us a good idea about the usefulness of Packwerk in realizing the benefits of modularization.

It’s important to think carefully about what could be turned into a package as Packwerk does not provide any guidelines regarding that. For us, we had one big entity that’s relevant to the 3 teams so we broke it down into packages and each team now owns the part that’s relevant to them and the different parts communicate through the packages’ public APIs. Another example is external dependencies: we moved all the logic that handles working with an external 3rd party system to a package. The goal was not to turn everything into packages but to do so where it made sense.

For now, we included business logic only in these packages; we didn’t include Rails models, jobs, or specs. We had already created 2 packages as part of our proof of concept that we had built as part of our earlier decision-making process, so we only needed to refine those a bit and create another package.

The implementation consisted of shifting a lot of files around, renaming them as they were under a common namespace now so the names were becoming a bit redundant, and changing method calls everywhere to use the public interfaces of the packages; overall, we had to reflect on the naming of our classes, the arguments that we are passing between these classes and packages, and thus we made things a bit more organized.

We also included Packwerk checks into our CI pipeline to ensure that changes in our pull requests are not violating the boundary checks between our packages.

At the end of our experiment, we arrived at 4 packages (excluding the root package). 

Benefits and challenges

There are a couple of benefits that we get with this approach:

• Loose coupling: by modularizing the code into packages and ensuring that these packages communicate using public APIs only, we are able to change to refactor the private APIs with minimal chance of breaking outside dependencies as long as we can keep the public API the same

$ bin/packwerk check
📦 Packwerk is inspecting 863 files
....................................................................................................................................................................................................................................................................................................E..........................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................
📦 Finished in 0.67 seconds

Dependency violation: ::Memberships::Public::Api belongs to 'app/services/memberships', but 'app/services/scim_memberships' does not specify a dependency on 'app/services/memberships'.
Are we missing an abstraction?
Is the code making the reference, and the referenced constant, in the right packages?

Inference details: this is a reference to ::Memberships::Public::Api which seems to be defined in app/services/memberships/public/api.rb.
To receive help interpreting or resolving this error message, see: https://github.com/Shopify/packwerk/blob/main/TROUBLESHOOT.md#Troubleshooting-violations

• High cohesion: by breaking our business logic into smaller organized chunks (i.e. packages), we are able to easily make sure that these chunks are working towards the same goal

# An example of a package's public API with high cohesion
module Public

  class Api

    class << self

      def create_saml_idp(metadata:, name:)
        ...
      end

      def update_saml_idp(saml_idp:, metadata:, name:)
        ...
      end

      def delete_saml_idp(saml_idp:)
        ...
      end

      def activate_saml_idp(saml_idp:)
        ...
      end

      def deactivate_saml_idp(saml_idp:)
        ...
      end

    end

  end

end

• Better code organization: we had a big app/services directory with lots of files and subdirectories, so it was hard to navigate and find the file you were looking for; by breaking it down into packages, we’re able to find what we’re looking for more easily

# directory structure with packages
  | - services
  |  | - packages
  |  |  | - saml_identity_providers
  |  |  |  | - public
  |  |  | - scim_memberships
  |  |  |  | - public
  |  |  | - memberships
  |  |  |  | - deletion
  |  |  |  | - update
  |  |  |  | - creation
  |  |  |  | - public
  |  |  | - 3rd-party-dependency
  |  |  |  | - public
  |  |  |  |  | - errors

It could be argued that Packwerk is not really needed to get these benefits, however, Packwerk ensures that there’s a contract and that its rules (i.e. dependency, privacy, and visibility boundary checks) are being met. This is arguably better than assuming that developers will consider these benefits when making changes or that the design of the software will ensure they do so.

$ bin/packwerk check
📦 Packwerk is inspecting 863 files
...............................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................
📦 Finished in 0.7 seconds

No offenses detected
No stale violations detected

The main challenge that we have:

• Modularizing business logic only: since we have only included business logic inside our packages, and we have references to Rails models and jobs inside them, this violates Packwerk boundary checks; there are 2 obvious solutions: 
  1. Remove the references and use dependency injection
  2. Move these Rails models and jobs inside the packages

We still haven’t decided which solution works best for us, but I am personally more inclined to the second approach.

Initial impressions and feedback

We had defined criteria to evaluate the success of this experiment with a focus on five areas: Learning curve, Developer experience, Maintainability, Integration, and Community support. We started gathering feedback right after implementation and after a couple of teammates from each of the 3 teams had the chance to learn and work with this approach.

Initial impressions and feedback from the team were very positive. Almost everyone agreed that they see the benefits of this approach and that they wish to continue working with it. The majority of people also reported that it took them only a few hours to a few days to understand and contribute to this approach.

The team also reported that modularization with Packwerk caused a slowdown in terms of development speed, however, most people agreed that the benefits outweigh the slowdown, and edge cases were rarely encountered.

Opinions were mixed when it came to testing; as we don’t include specs in the packages, we often had to tell Packwerk to ignore these files, and fixing this is still one of the challenges that we currently face. Feedback from the team was clear about this point; when asked about the impact of modularization and Packwerk on testing, they responded: 

“The way we implement modularization only for services creates the need to sometimes bypass packwerk”

“rather a negative impact as I haven’t found a nice way to combine it with rails MVC pattern”

Mostly, this approach didn’t have much impact on testing.

People also agreed that Packwerk integrates well with our current tools (e.g. code editors, GitHub Actions) for the most part, and they can find answers and documentation somewhat easily.

Next steps

We’ll continue gathering feedback from the team, and the next step for us will be to modularize more parts of our codebase. We have identified some candidates and will incrementally turn them into packages. We’ll also address the challenges we have by experimenting with one of the approaches mentioned above. 

Our Rails monolith is growing and changing every day, and it’s quite important to keep it steadily and swiftly maintained, and it’s easier to achieve that when changes are contained in packages. This is a progressive step towards maintainability and scalability. 

Header Photo by xavi_cabrera on Unsplash.