Introduction
Feature flags are software engineering practices for applying logical gates to features and functionality with the intention of conditionally making such features available to users in real-time. They control users’ access to features after continuous integration and deployments.
One of Capmo’s core values is learning and growing fast. To be able to do this, we have to take courageous decisions to challenge the status quo, and more importantly, move fast, make mistakes, and learn from them.
In practice, we need a safe way to ship features into production and recover almost instantly if something goes wrong as a result of the new feature shipping without total code rollback.
Motivation
During feature rollout management, e.g., A/B testing of features, we aim to gradually expand our release, i.e., (1%, 5%, 10%, 100%) to users, so that if something goes wrong, only a smaller set of user groups is affected and the features can be rolled back.
According to Martin Fowler, one of the most common arguments for using feature flags in favor of FeatureBranch is that it provides a mechanism for pending features that take longer than a single release cycle.
Imagine you are releasing to production a couple of times a day, but need to build a feature that’s going to take three months to complete. How do you use continuous integration to keep everyone working on the main branch without revealing a half-implemented feature on your releases?
At Capmo, we experience the issues highlighted above a lot. To solve these problems, we have adopted feature flags.
Feature flags have also been useful for us in situations that require simultaneous changes in distributed systems, e.g. delivering a single piece of work on the backend and front end, running migrations, and enabling them all at once when everything is ready.
Approach
We currently use a tool called LaunchDarkly and we are able to achieve feature flag usage and management using various SDKs they provide.
Below is the typical workflow between our application and feature flag evaluation platform
We’ll dive more in-depth into this approach when we get to the implementation section.
But in summary, for every application user, their details are collected, and the flag which serves as the gate to the functionality the user is attempting to access is also sent to the launch darkly platform via the SDK.
Based on the result of the evaluation, the user is able to access the specific functionality or is denied access.
As a result of the approach above, the following comes to mind:
i) Security
Since LaunchDarkly is an external tool that expects us to send some user information to be able to evaluate flags for the user properly, there is a concern about sending the PII (personally identifiable information) of our users to launch-darkly.
We ensure to send a mandatory unique UUID, which our systems use to identify users, which is consistent throughout our application, i.e., userId. Any other information to be shared with LaunchDarkly is optional.
ii) Latency
The majority of platforms that provide infrastructure for feature flag management have unique streaming architectures, which means we can instantly turn features on or off when we need to without introducing any latency to our site. Flag updates take only 200 milliseconds to propagate in our case for LaunchDarkly.
Implementation
The following highlights some key functionalities we leverage and how we implement the usage of feature flags in Capmo.
Variation
The variation is the portion of the LaunchDarkly SDK executed, whenever the user encounters the flag check in-code.
As shown above, the variation determines the value of the feature flag for a specific user. If you are using another platform for your feature flag, this implementation might vary. The LDUser represents the user object type, while the LDFlagValue represents the types of values a feature flag can have.
Individual Targeting
Individual Targeting is the ability to make different flag variations appear to different users. This is useful for us, e.g., in scenarios where the person working on the task wants to be the only one to view the functionality or also enables engineers to work with different versions of the app.
This is also useful when engineers want to test their work with different application versions.
For example, when you switch on your flag, you ensure that you select the user to be targeted as shown in the area marked (1) below. You then switch the flag ON/OFF in the area marked (2) and save all configurations in the area marked (3).
Flag targeting does not need anything to be configured in code for LaunchDarkly, as it is purely a UI action, so we don’t have to worry about any PII (personally identifiable information) being shared.
Percentage Rollouts
Percentage rollout is also important for us to ensure that we can gradually release features to our users, as is done in canary deployments. i.e. releasing to 1%, 5% gradually to 100% of our users.
During the percentage rollout, LaunchDarkly server generates a hash based on both user attributes and the flag’s key. The SDK uses this hash to generate a percentage value for that user. That value, compared to the value set for the percentage rollout value, determines which variation a user receives. The percentage rollout is configured on the dashboard and needs no further changes in code. When combined with targeting, it becomes a super-powerful tool.
Caveat
1. Clean Up
With feature flags, it’s easy to have a lot of clean-up tasks after implementing and rolling out the feature. It becomes messy when you have flags for functionalities that have been successfully released, but there is no need for backward compatibility.
2. Technical Debt
This can lead to a lot of technical debt if not carefully managed, and it makes it possible to accidentally turn off feature flags for functionality already released.
3. Deployments
Feature flags are not great for testing deployments on backend infrastructures or on the network layer of the application. For such scenarios, canary releases are best used. For example, in situations where a new implementation that is controlled by a feature flag that lives in code was deployed via a new version of the application and this version breaks the infrastructure/network layer, this problem cannot be controlled by a feature flag, hence using canary releases helps manage and roll back this new application version.
When not to use Feature Flags
We’ve highlighted scenarios that require the usage of feature flags above in the motivation section and also discussed some further use cases, i.e. individual targeting and percentage rollouts, so it makes sense to discuss when not to use feature flags.
1. When Shipping Security Sensitive Iterations
If your feature deals with authentication or encryption/decryption of sensitive data, it’s safer to wait until it has passed QA before you deploy the code. Deploying it and assuming you can keep it turned off with a feature flag until you’re ready to use it is just too risky.
2. When Staying In Control Is Critical
One of the most common scenarios is if you have an application that can’t connect to the network because
- It’s running on a device without great connectivity.
- You can’t tolerate the latency required to check in with a feature flag configuration over the network.
In the above scenario, the only option for controlling feature flags is to use a configuration file that’s pushed out with the application itself and is hosted locally on the user’s device. That’s very risky because it means there exists a possibility that the user could potentially enable a feature that shouldn’t be enabled.
Conclusion
Lastly, you should ask yourself the question, “Do I really need this feature flag? Could I do without it?” If the answer is yes, then you don’t need it.
From the discussions above it’s easy to conclude that feature flags are a great tool to have in an engineering toolbox. Thanks.
.jpeg)