Agile development and the red line: a point of no return

In the cloud industry, a culture has been established where everyone thinks they must innovate and deliver new features.

It’s not only developers who have this culture. Sales has a tendency to promise new features with the hope that they will help them expand and retain current customers. There are many examples of where a sales team has to fix a customer issue before even thinking about selling.

While new features and customer service are important, what has been successful for our customers and us has been to change this culture and focus on refining and delivering high-quality features.

To do this, ServiceNow created what they call a red line approach.

>See also: Busting the myths of agile development

The red line is built upon the agile development philosophies of customer satisfaction by early and continuous delivery of valuable software and continuous attention to technical excellence and good design.

Finding your red line

A few years ago, the engineering teams implemented a metric that it called the red line. The red line helps find the sweet spot balance for improving the quality of existing features and new feature innovations.

It helps understand how much unnecessary effort businesses cause themselves in support costs and, more importantly, how satisfied or frustrated customers get with various features.

The teams are directly responsible for keeping the ServiceNow cloud functional and embedding greater features, so the red line is an important quality control measure.

The way the red line is determined for each engineering team is simple: it is the total number of issues for features or services developed and operated by a team divided by the total number of customers we have on a monthly basis.

More simply, the red line for each team is the number of issues that they cause our customers on a monthly basis normalised by total customer count. This does mean that every customer issue needs to be categorised and attributed to an engineering team.

For example, if there were ten issues in a given month and you had 100 customers, the red line would be 0.1. If you had 1000 issues, the red line would be 10.0.

The lower the red line number, the better. A key point to note is that an issue is any issue, regardless of severity or priority that the customer raises or that we raise as part of running our operations.

Applying the red line to DevOps

As an engineering organisation, ServiceNow requires monthly accountability to management on the red line number for each team.

Teams, in return, watch the red line on a daily basis to keep the number low – making it real-time and actionable. They are able to make changes and push out updates saving time trying to help customers with a work around.

>See also: How agile development can run in parallel to traditional IT management

The resulting time savings mean the the teams are able to look at every issue that comes in – not just the big ones.

Once the red line was in place, it became clear that many of the issues were the same. Focusing each team on their assigned areas helped them to resolve these repeat issues much more quickly and most issues quickly become non-issues.

Balancing innovation with the red line

If the red line is down or sloping downward, a team can spend a majority of their time innovating new services or features. If the red line slopes up, they need to fix issues.

If the red line is flat, they are balancing quality with innovation. If there is a consistent and significant slope up for a few quarters, changes needed to be made, like diverting some resources to fix issues before they become big problems.

What is key here is understanding the issues that are driving the red line for each engineering team. In some cases, the issues may be driven by poor documentation resulting in a spike in “how do I…?” issues.

Another case may be an engineering team deploying new software that causes performance issues for a series of customers.

This red line helps measure and address these issues so that an organisation can improve product quality and customer satisfaction – thereby improving the success of the solution.

Getting buy-in on the red line

To get buy-in from engineers, the philosophy of the red line is emphasised. They quickly learn how important it is to the company and also learn how to balance between fixing issues and innovation.

Since implementing the red line four years ago, the team has grown from 50 to 700 engineers.

Industry-wide red line adoption

Think of the impact industry-wide adoption of red line philosophy could bring to organisations. The cloud industry must improve the quality of existing features so that customers can realise the impact of the innovation that has taken place. The red line philosophy helps with this.

Now, you have to focus on innovating the features, and that is okay. But as soon as you start getting traction, your engineers need to balance the innovation with fixing customer issues.

>See also: Future shock: The race to embrace agile development

This means putting the right testing in place, ensuring you have the right automation and improving features – alongside working on new features.

By employing the red line philosophy, businesses will be able to change its culture, putting the customer experience as the foundation of the business.

If the industry adopts this same philosophy, it will change the collective culture and balance innovation of features with quality features that help customers to be successful.


Sourced by Allan Leinwand, CTO, ServiceNow

Avatar photo

Nick Ismail

Nick Ismail is a former editor for Information Age (from 2018 to 2022) before moving on to become Global Head of Brand Journalism at HCLTech. He has a particular interest in smart technologies, AI and...

Related Topics