Lean Startup and Continuous Delivery for SaaS products

At Signal AI we build a SaaS product that enables enterprises to monitor world news in real time.

In the course of last year, we have learned that to keep innovating in the SaaS space we need to find answers to the following two questions:

• What’s the best way to decide what to build next?
• What’s the most effective way to release?

Use Lean Startup principles to discover what to build next

Building the right software is hard. We need to keep a number of factors in mind in order to figure out what the right thing is to build: our long term vision, the market needs as well as the typical constraints of a startup.

At Signal we believe that the Lean Build-Measure-Learn cycle is the right way to create innovative products.

Image Credit: Build Measure Learn

The goal of each iteration is to learn something new that will move the product forward. We formulate an hypothesis, we create enough that will let us validate/disprove the hypothesis and we plan our next step. We use the cycle throughout the feature lifecycle, from idea to working solution. When we are iterating over an existing feature, we measure outcomes with a mixture of qualitative and quantitative feedback to have a balanced perspective.

Continuous Delivery for SaaS

Continuous Delivery is a core engineering practice that supports short and fast iterations and reduces risks. We fully embrace it, trying to push out any new feature as soon as it’s ready. Our engineers can push a new component to production at any given time.

At the same time, we realise that certain business workflows require a set of features bundled together to deliver the full value. For major milestone go-to-market activities also require a degree of coordination with new features hitting production.

How to get the best from both worlds? How to define a release strategy that is mindful of the impact of go-to-market whilst not losing the ability to release as fast as possible while pivot and iterate over an MVP?

We used two guidelines to define our release strategy:

• We want to release as often as possible

  This means the ability to release partial feature set. By early releasing we are able to collect feedback that will help us with finalising the feature set.

• Each release should reach as many people as possible

  At any given time we want to reach as many people as possible, without ignoring the multiplier effect achievable with proper go-to-market. For each incremental improvement going to production we want to extend the number of customers that have access to the feature set.

These guidelines have helped us identify two techniques that are now foundational to our release model: user cohort definition and feature flag.

The first technique is the definition of user cohorts; cohorts are specifically defined in relation to the features we are creating. Sometimes it could be customers who demonstate a strong appetite for innovation, other times customers who have a good idea of how a feature could develop. These early adopters help us learn something new about the nascent feature set, they act as partners in the development cycle.

Signal itself is often the very first cohort; this enables dogfooding for early qualitative feedback and brainstorming. Through opening up internally rough cuts we also crowdsource some QA activities, plus we enable the rest of the organisation to start experiencing what’s coming next.

We use feature flags to enable a given feature set for a cohort. This allows us to have fine grained control of which cohorts will have access to what.

With all these in mind we have identified three stages:

1. Internal

   The feature is open up internally to Signal. This stage is optional. The optionality is based on how much we can learn from this stage.

2. Canary

   We give access to specific cohorts to gain more learning an iterate quickly towards completion. This stage is optional. It’s up to the team to decide if we need this stage and how long the stage should last.

3. General Availability

   Our ultimate goal: the feature is available to all our customer base. We want to get to this stage as quickly as possible.

How we release software at Signal Media

The duration of each stage is driven by the learning opportunity: the more we learn, the more we could iterate over a given stage.

The decision to move to the next stage is not taken by a business sponsor but from the actual team behind the released software.

In order to be effective in steering the company fron one stage to another such a team needs to have some key built-in characteristics:

• Be fully cross-functional.
• Able to act with a high degree of autonomy and trust.

The team has to be set up with all the skill set needed to understand the overall impact of a new feature and assess the right time to move to next stage. A mix of technology, marketing and business is needed to take the best decision.

Also, the cross-functional team has to operate in an environment that promotes high trust and autonomy. Without these two traits the team won’t be able to have the impact that is pursuing, being slowed down by externally forced gateways and checkpoints that will increase the time to market and erode innovation.

Thanks to Patrick Kua and Tim Goodwin for the feedback on the draft of this article.