Companies should encourage and allocate time for their engineers to be able to contribute back to open-source. It does not only improve the product that they are contributing back to, but enhances the feeling of self-worth which in turn has a big positive impact on the well being of the engineers and the company itself.
“In learning you will teach, and in teaching you will learn.” Phil Collins
You might be wondering what teaching has to do with contributing to a open-source project? In my opinion it is just a small part of one of the many academic processes that every company should embrace, because there is no better way to improve and learn more about a certain topic than to try to bestow and spread the knowledge to eager learners.
Almost every blog post features a fine example. For this post I will be explaining the process of adding a small yet valuable feature in Hashicorp’s terraform-provider-aws plugin.
But first, let me do a small intro.
Having a multilingual and multinational product with many users who are on various platforms and a multitude of browsers makes matters very complex, and sometimes the problems that our customers experience can be solved simply by clearing their web browsers cookies. We all know that doing so can be a tedious job both for our colleagues in Customer Service and our clients, especially given the fact that there are many web browsers out there and not all of our customers are tech-savvy.
For the reasons stated above we have decided to implement a serverless microservice for clearing cookies clear-cookies.babbel.com (Babbel users be warned, pressing the button will clear out your babbel cookies!) running on lambda@edge. With just one press of a button all the cookies that are related to our product on the customer’s machine will be deleted. After reading the cookies from the client, a POST call, which is CSRF protected, will effectively clear all the cookies that have been generated by us. As some of you might already know there is no such thing as deleting a cookie. Instead, the cookies that should be deleted are emptied of all values and their expiry date is set into the past, which effectively “removes” the cookie.
And now let us head on towards describing the process that led to the creation of the pull request against terraform-aws-plugin and how it was done.
The complexity of our systems is rather enormous. In order to help with the workload that emerges with our platform’s intricacy we tend to rely on using the serverless application framework that is provided by AWS all under the veil of terraform as our preferred solution for Infrastructure as Code.
In Babbel we are heavily utilizing many open-source technologies, which by itself is a topic out of scope for this blog. For clarity we will mainly focus on Hashicorp’s terraform-aws-plugin. Lately there have been many new features added in AWS which are not immediately present in the infrastructure configuration tools but may or may not be added at a later date. Under normal circumstances those features are added exceptionally fast in said tools (another fine example of the power that the community has), but some are either unpopular, fall under the radar or are indistinct. Consequently they do not get added in a timely manner if at all. One of those was the include_body option for lambda@edge where it exposes the body of the request/response in the cloudwatch logs. After seeing that it was missing and it was greatly needed we decided that it was time to roll up our sleeves and get down to business.
What we usually do in cases like this at Babbel is that we apply the method of “Eat Your Own Dog Food”, meaning that we maintain our own fork of the terraform-provider-aws, in which we have the features that are either missing or will never be added but are required by us. We build it, test it and use it (in production). So when we are satisfied with the quality, we create a pull request to upstream in GitHub and wait for an approval. Once the feature is approved and merged, we update our fork (again), so that we do not stray too far apart just for simplicity and maintainability sake.
For an appropriate and complete pull request against an official terraform plugin, the maintainer HashiCorp is using a software testing method called acceptance testing, in which the proposed changes are tested with apply, refresh, and destroy cycles. For this particular case we had to manually increase the timeout of the acceptance tests, because the terraform cycles where adding and removing cloudfront resources, which at times can be a little bit slower than usual. Needless to say the tests took two and a half hours to complete.
make testacc TESTARGS='-run=TestAccAWSCloudFrontDistribution'
==> Checking that code complies with gofmt requirements...
TF_ACC=1 go test ./... -v -run=TestAccAWSCloudFrontDistribution -timeout 1000m
? github.com/terraform-providers/terraform-provider-aws [no test files]
=== RUN TestAccAWSCloudFrontDistribution_importBasic
--- PASS: TestAccAWSCloudFrontDistribution_importBasic (828.70s)
=== RUN TestAccAWSCloudFrontDistribution_S3Origin
--- PASS: TestAccAWSCloudFrontDistribution_S3Origin (805.78s)
=== RUN TestAccAWSCloudFrontDistribution_S3OriginWithTags
--- PASS: TestAccAWSCloudFrontDistribution_S3OriginWithTags (1028.92s)
=== RUN TestAccAWSCloudFrontDistribution_customOrigin
--- PASS: TestAccAWSCloudFrontDistribution_customOrigin (830.72s)
=== RUN TestAccAWSCloudFrontDistribution_multiOrigin
--- PASS: TestAccAWSCloudFrontDistribution_multiOrigin (1021.48s)
=== RUN TestAccAWSCloudFrontDistribution_orderedCacheBehavior
--- PASS: TestAccAWSCloudFrontDistribution_orderedCacheBehavior (964.56s)
=== RUN TestAccAWSCloudFrontDistribution_Origin_EmptyDomainName
--- PASS: TestAccAWSCloudFrontDistribution_Origin_EmptyDomainName (2.29s)
=== RUN TestAccAWSCloudFrontDistribution_Origin_EmptyOriginID
--- PASS: TestAccAWSCloudFrontDistribution_Origin_EmptyOriginID (2.09s)
=== RUN TestAccAWSCloudFrontDistribution_noOptionalItemsConfig
--- PASS: TestAccAWSCloudFrontDistribution_noOptionalItemsConfig (832.29s)
=== RUN TestAccAWSCloudFrontDistribution_HTTP11Config
--- PASS: TestAccAWSCloudFrontDistribution_HTTP11Config (748.69s)
=== RUN TestAccAWSCloudFrontDistribution_IsIPV6EnabledConfig
--- PASS: TestAccAWSCloudFrontDistribution_IsIPV6EnabledConfig (804.62s)
=== RUN TestAccAWSCloudFrontDistribution_noCustomErrorResponseConfig
--- PASS: TestAccAWSCloudFrontDistribution_noCustomErrorResponseConfig (919.45s)
PASS
ok github.com/terraform-providers/terraform-provider-aws/aws 8789.627s
...The pull request by itself was trivial and I don’t think it deserves a worthy mention.
So to wrap it up I consider this as a “win-win” situation because:
- We get to use the latest features of AWS (or any other IaaS that we might be using for that matter) without any waiting.
- We can give a feature that has been tested on production back to the community.
- It encourages learning and promotes well being among our engineers.
If you enjoyed this blog post, I’d be very grateful if you’d share it, because sharing is caring! Do not forget to leave your thoughts and ideas below!