BDD Discovery and OOPSI

Before we get good at delivering continuously, we need to get good at breaking our business goals down into smaller goals and horizons of value.

16904896_lI like to think of these like value waypoints. We need to expertly navigate to these waypoints as quickly as we can.  At each waypoint we’ll check our surroundings and make sure we’re still on track.

We only get value out of iterative delivery practices if we are always working on the most important thing.

BDD addresses this, as it is an outside-in practice. That means we must start with the Outcome (the value)

I’d like to introduce The OOPSI Model, its an extension of Chris Matt’s Feature injection, put together by me and @PeteBuckney.  We’ve added Scenarios to make it OOPSI rather than OOPI. We thought this helped put our BDD practices in context.

OOPSItext

To get to our value waypoints as quickly as we can, we need to be able to defer detailed examination of stuff that isn’t important until later. We want to find the highest value outcomes, then the highest value outputs that achieve those outcomes, then the highest value processes that deliver those outputs, then the highest value scenarios and examples that help clarify the required implementation.   OOPSI is a useful technique to help add more convergent thinking to our software delivery practices. Convergent thinking techniques help us align to the next value waypoint and start small.

If you do it the other way round it ISPOO 😉

You can watch a video of me & Pete talking about BDD Analysis and OOPSI at BDDX 2015

Before we explore OOPSI in detail, let’s take a step back and think about BDD discovery and what we are trying to achieve.

Why do we use examples in our specifications?

sBe

Our goal is to reduce rework, by basically removing defects early, all kinds of defects – those that occur as a result of getting the requirements wrong, those that result from misunderstandings, ambiguities, handoffs.

Our practice of SBE helps us do this by exploring examples.  Not this.

divein

We have to do the analysis and exploration to get these benefits. There is craftsmanship and skill associated with these activities.

We need to do the analysis and synthesis to turn requirements into specifications, there is discussion and exploration required to do this.  It takes skill and practice to come up with the cleanest, leanest and most elegant set of examples you need.  The examples don’t just pop out of the domain expert’s mouth in Given When Then Syntax.

If we jump in too early, we’re likely to end up with a lack of context and masses of duplication in our scenarios

badcukes

We talk a lot at conferences about ‘when cukes go bad’ or ‘how to get the most out of cukes’. This is normally approached as rationalisation and refinement in cucumber. Normally the rationalisation removes duplication by using tables of examples,  tables of data conditions, or tables to define pre-conditions.

If we used more of the Specification by Example practice and collaborated around these tables of examples before we turned them into feature files I think we’d save time.

Tables help us focus on the what and not the how. If we jump into GWT to eagerly we can end up putting the emphaaaaasis on the steps (the how) and lose the value of the what.

It’s hard to spot gaps in a long page of text. It’s much easier to see what’s what by reviewing the tables of data themselves.

OOPSI Mapping

The OOPSI mapping technique can be used to help structure the discovery process, to help ensure that we’re always focusing on the most important thing and to avoid analysis paralysis.

Similar to Matt Wynne’s Example Mapping, this technique can be done collaboratively using post-it notes in small teams. The 3 Amigos (domain expert, test expert and technical expert) follow the OOPSI process using different coloured post-it notes to explore the business requirements and work towards some implementation examples.

OOPSI helps bring together BDD automation with the analysis practices of ATDD and Gojko Adzic’sSpecification by Example

oopzi

Starting with the Outcomes, Outputs, Process etc helps us rationalise in advance. We spot shared processes and relationships between outputs and scenarios, and  we organise our examples around them. The examples rationalise themselves.

A worked example might look like this:

1. Outcomes

A good starting point for the outcome might be the ‘how’ level of an Impact Map.  Or if you work with User Stories you might start with something like this…  a well known BDD starting point…

outcome

2. Outputs

We know that the value normally lies in the outputs of a process, so let’s think about them first.  What outputs might we get from our withdraw cash story?

outputs

Crikey, quite a lot.

Cash, receipts, returned bank cards, an updated balance on a bank statement, updated database values, and even some error messages.

Some of these outputs might be well understood, but some might benefit from a bit more exploration.

It’s really helpful to illustrate outputs with examples.  Draw them.  Understand the data attributes that appear on the outputs.  Use real world examples wherever possible.  I go into this in a bit more detail in my data personas blog.  In this example we might choose to draw the receipt, or mock up the error messages.  This helps drive conversations and get to the important stuff.

We might identify some high level examples here that are just focused on recognising the different outputs that can get generated.  It’s important to recognise that we are often working with examples at different levels of detail.  We always want to look for the most useful format which helps open up conversations.

Now let’s choose our most important output so we don’t get becalmed by analysis paralysis. Our most important output is cash.

Some of the outputs might naturally go together.  In this case when we get cash we normally get a receipt, updated database tables, our card returned and an updated statement.

3. Process

Now we’re onto process.  

What process and interactions are required to generate cash?  We might come up with something like this.

process

We might do a bit of general exploration at the process level and see how our processes and interactions change to support the different outputs.

Collaborative techniques like Jeff Patton’s Story Mapping and Alberto Brandolini’s Event Storming also look at interactions and this might be a really nice place to bring those techniques in.

Making sure we include the process in our discovery activities (as pointed out in Jeff’s User Story Mapping Book) helps bring in the UX and interaction designers, so that everyone can be working to the same vision.

Depending on our problem domain and its complexities, we might also consider other techniques that help open up conversations.  Swim-lane diagrams are particularly useful if you want to explore interfaces and APIs between different systems.

4. Scenarios

Now we have some context to start brainstorming some scenarios that are relevant to this process.  They might be paths through the process to generate the outputs, or just rules. There might not even be a process. (Get 200 points per mile as a business class traveller and 100 points per mile as an economy class traveller doesn’t have a process and is just a rule).

For our ATM scenario we might come up with these key scenarios initially.

scenarios

Let’s take the first one.

scenario 1

If we decided to write this up in Gherkin at this stage it might look like this:

scen1gwtLet’s just think about our use of examples. What do we mean by an example?   To me it’s only an example if it actually clarifies the rule, if it is a representative case.

The reason that we used to find loads of defects in the testing phase of waterfall projects is that it was the first time that our implementation got tested with real world data.

keep it real

The value of the example is in the data used to drive the example

In fact the data is so useful, we might not even need the words to help us understand.  This is where the practice of Specification by Example comes in.

5. Inputs (& Preconditions)

Using our OOPSI model, before we dive into Given, When, Then, we would illustrate our scenario using a table of examples.

SBE1

Ooooh look!, We can illustrate the other two scenarios using the same table.  This supports a Gherkin Scenario Outline (only we haven’t written it yet).  And we realise that the Valid Card and Card Returned Columns aren’t important for the scenarios, so we can take them out.

SBE3

What’s interesting to note here is that the scenario outline name has resolved to customer requests cash.  Notice request cash matches an activity box in our process.

We might find a pattern where high level and happy path scenarios are grouped around important outputs, and lower level examples support the different activity boxes – illustrated by tables of data where necessary.  We could structure our living documentation that way if we wanted.

Collaborating around the data in the table helps us know when we have found new scenarios, and we add them to our OOPSI map.  But since we’re only concerned about the cash output at the moment we’ll park the Invalid Card scenario to worry about later.

When we get to that one, it’ll be a slightly different process & we’ll need a table of examples to help us understand more about valid and invalid cards.  They’ll probably resolve themselves around the activity box ‘check card’.  Interesting.

Liz Keogh wrote a sonnet about deliberate discovery.  I love this excerpt

sonnetAs we explore our examples with real world data, we’re looking for dragons.

The thing that makes one scenario different from another is normally the context. The preconditions, the GIVENs, everything we need to arrange the test.

Liz Keogh also did a blog called Conversational Patterns in BDD where she talks about context questioning. In the ATM example if we were focusing on the cash card returned output we might have asked ‘Is there any scenario where you wouldn’t get your card back?’ And the answer might be ‘Yes, when there is a fraud flag live on the account’ …. Ahhh a fraud flag… what does that look like?

So if we explore the context we find the dragons.  Back to our ATM example.

Ooooh look there’s a dragon

SBE4

We didn’t think about overdraft facility.  We could use the same table and add these examples like this:

SBE5

Using the table helps us notice that if our scenario introduces new columns, we introduce unnecessary permutations of preconditions and we have probably seen a new rule.

And we might spot some more dragons like these:

scen3

Using the table to collaborate around helps us notice that we’ve identified 2 new preconditions that we’re interested in.  The preconditions, the GIVENs are a good thing to collaborate on an agree on.

Get to know your pre-conditions and test input data. Use real world data. Give your data a personality. How many different data personas to you need to define to cover all the examples?

A data persona is a profile of data conditions with a name. They help you drive conversations around examples from discovery through to delivery. And they become ubiquitous examples. You can re-use your data personas all over the place.

So in summary:

  • Don’t dive in to Gherkin too early
  • Do deliberate discovery using OOPSI
  • Value the practice of Specification by Example
  • Slay the dragons using data personas

And use whatever tools and techniques that help find dragons at all different levels of OOPSI.

 

 

Yoda Cake

1 Reply

I had so much fun making this for my son’s 7th Birthday that I had to share….

12541136_10153946542354052_2044292961089838290_n (1)

Make ears out of fondant

Stage 1 – The Body

Stage 2 – The arms

Stage 3 – The head

 

 

Asteroids and Design Thinking

6 Replies

Talk given at Agile on the Bench 15 April 2015

asteroids

I’ve never been a huge fan of user stories

‘As a team member, In order to know what the hell I’m doing, I need a vast and sprawling backlog of 50 million user stories entered into Jira’.  Said no one. Ever.

‘As a team member, In order to contribute to high value business outcomes, I need purpose, direction, context and shared understanding.’  Said me.

I think I’m having a mid-life crisis. I miss the olden days

I miss the structure

  • Section 4.3 had a process flow or diagram
    • Section 4.3.5 had a use case and some key interactions.
      • Section 4.3.5.1 described a scenario
        • Section 4.3.5.1.b described some alternate paths
          • Section 4.3.5.1.b.ii showed some error handling

I miss the context

Remember the functional decomposition tree? In my functional specification, there was a place for everything and everything had a place. Each piece of functionality was neatly broken down into smaller bits of functionality to help me understand. A massive 456 page hierarchy.

I miss the relevance

Screen Shot 2015-04-15 at 19.26.44

Remember the V Model? We had different outputs appropriate for different audiences created by different team roles based on different levels of elaboration. Each level of analysis output had it’s partnering test documentation where the rules were recaptured as tests something like ‘verify that <insert statement from analysis doc here>

I miss the analysis

Years of analysis experience and a multitude of techniques went into massaging Business requirements into functional specifications and design documents for us to build our solutions upon.

Somehow in our new world of iterative software delivery, analysis has become a bit of a dirty word. I think we hold analysis responsible for all the pain we experienced in the olden days.

It’s not analysis’ fault

Analysis was never the problem. The problem was that the structure, context relevance and analysis of the olden days were all based on a pre-conceived solution rather than the goal. Because of this we had to deliver the entire thing in one go to achieve any of the benefits. 2 years later the world had changed.

The problem was that we used to work in silos, delivering ambiguous outputs with lots of hand-offs and too much opportunity for misunderstandings. Inevitably in testing phases we uncovered loads of requirements defects and projects got delayed.

The problem was that there was loads of wastage. Huge documents were produced which were indigestible for reviewers. Each individual would have to re-learn and re-analyse the subject matter to produce the next document in the chain. Testers would find gaps in specifications as they considered how to test the system, causing costly late changes.

Now we do things more collaboratively, we have more conversations. We’ve reduced hand-offs and opportunities for misunderstanding & misinterpretation. We use BDD techniques to collaborate around real-world examples to achieve shared understanding and we reduce wastage by using tests as a specification.

We still need analysis

I think we’ve recently started realising that we need to do the analysis. A conversation from a post-it note isn’t enough. We need to go through the process of turning the business need into a specification (or test) to begin implementation.

The word analysis comes from the Greek ‘to loosen’. It literally means to break down

‘User Story Splitting’ is just a clean way of saying analysis.

Just because we think we’re ‘Agile’ doesn’t mean we’re immune to the horrors of the olden days. If we do too much up-front breaking down of our stories, we can end up in user story hell.  Too many broken down user stories on a backlog represents wastage.

It’s too much up-front breaking down that got us into trouble in the first place.

The whole point of having atomic user stories is to enable us to deliver them incrementally or iteratively in order to get business value as early as possible. We only get value from our iterative software delivery approaches if we are always working on the most important thing.   In order to do this we need to know which scenarios, stories and examples are the most important ones.

Design Thinking

Design Thinking starts with the goal instead of a solution. It investigates alternate paths to achieve that goal and then refines and tests options.

Design thinking is iterative and is described as a sequence of steps (different sources):

  • Empathise, focus, ideate, prototype and test.
  • Define, research, ideate, prototype, choose, implement, and learn.

There key aspects of Design Thinking are:

Analysis and Synthesis. Literally meaning to break down and put together. Analysis breaks down the whole into components to examine it, and synthesis puts the conclusions from that activity back together to form a coherent whole. They always go hand in hand.

Divergent and Convergent Thinking. In the divergent phase we examine options and generate ideas. In the convergent phase we rationalise our ideas and prioritise the best ones. We create choices then make choices.

divergeconverge

In the olden days we may have been doing some pretty good analysis and synthesis as we generated our 456 page functional specification documents. We’d just be doing it on our own, coming up with our own interpretation of ‘synthesis’ not necessarily the same as the next person.

We may have been doing some good convergent thinking, but unfortunately converging on features that delivered little business value or would never get used.

Asteroids

Jeff Patton in his fantastic book ‘User Story Mapping’  compared splitting user stories with the 1980s game of Asteroids.

In Asteroids you start with some really big, asteroids moving around the screen and you have to shoot them up without being squished by them.  As you blast them, they break down into smaller asteroids and start moving faster, and as you break those down they get faster still. The more small, faster moving asteroids you have flying about, the more likely you are to get squished.

You need to pick off the big ones, one at a time or you’re a goner.

When I read this it struck me that this process has a Design Thinking element to it. After we have exploded the big asteroids (divergent thinking) we need choose which one to blow up next (that’s the convergent thinking part) and leave some slower moving ones still floating. This is iterative not just from a story by story perspective, but for each hierarchical level of elaboration.

We only break down what we need to enable us to start the learning process.

It’s this iterative process of creating choices > making choices, that helps us navigate towards value as early as possible. 

We need to start with the goal. We need to break goals down into key business outcomes and impacts, key interactions that help deliver those impacts, then important scenarios and examples. Breaking things down based on the goal helps us stay focused on a shared vision so we can make good choices.

Like a good story, we want things to make sense in the context of a meaningful narrative, so that they are not disconnected from the bigger picture. In order to do this we should embrace analysis techniques like process mapping,  interaction diagrams and User Story Mapping.

IMG_7392

This is an excerpt from Jeff Patton’s User Story Mapping Book.

This recently helped me understand that a story doesn’t have to be a disembodied post-it note in a particular format. All of our outputs at each layer of elaboration are stories, if we use them in the right way. (I like them more now)

We’ve got pretty good at divergent thinking practices in recent years, particularly brainstorming and splitting User Stories, but we need to support this with more convergent thinking practices.

impactmappingGojko Adzic’s Impact Mapping technique uses Design Thinking to explore options and create choices to deliver Business impacts and outcomes. It helps us create a hierarchical product roadmap based on the goal.

This technique is fantastically effective, but the real benefit lies in converging on the next Business value increment or learning milestone that we should focus on.

Other techniques like User Story Mapping include really useful convergent thinking elements. Jeff Patton suggests considering opening game, mid game and end game versions of key interactions. This helps us get feedback at the earliest possible opportunity and defer some of the analysis till later.

We are also doing convergent thinking when we achieve shared understanding.  Collaboration using real world examples helps with this, particularly if we get good at choosing the most important ones to implement first.

Final thoughts…

In the olden days analysis was about functional decomposition of a solution. This decomposition created a hierarchy with context, but only told us where we started – which wasn’t that helpful

In the new world, rather than decomposition our analysis is like navigation. All nodes of our roadmap are linked to the goal. Our map is telling us our destination and options as to how to get there. – Much better.

So analysis is not the bad guy, we just have a new paradigm for analysis activities.

 

 

 

Collaboration, Collaboration, Collaboration

1 Reply

collaboration

Introduction

A Brave New World

Agile is now considered the standard approach to software development.  Many Businesses are experiencing quality issues and are in a state of chaos after rapid introduction of agile methodologies such as Scrum.  Many are in transition towards ‘being Agile’ without having a clear roadmap as to how to get there or even knowing what they are hoping to achieve.  They get the theory, but need a pragmatic approach to adopting agile techniques and achieving the true business benefits without compromising quality.  They need to bring the Business closer to product development.  They need to collaborate.

Background

Agile became popular in response to challenges faced by development teams to respond to business needs quicker and with more agility, to accommodate changing requirements and deliver return on investment at the earliest opportunity.

This is achieved through improving communication between technical and business teams, understanding very clearly which requirements deliver the business value and delivering that business value as early as possible and in line with business expectations (having avoided defects through close collaboration).

Some of the 12 Principles of Agile Software as stated in the 2001 Agile Manifesto;

  • Our highest priority is to satisfy the customer
    through early and continuous delivery
    of valuable software
  • Business people and developers must work
    together     daily throughout the project.
  • Continuous attention to technical excellence
    and good design     enhances agility

Examining the Problem

What we often see however, is a focus on the mechanics of Agile frameworks such as Scrum, and the technical tools and development environments that support them rather than on the underlying principles for success i.e. collaboration.

It’s not just about the technical stuff

Agile has often been advocated by technical teams excited by the shiny new stuff but not necessarily naturally good at working collaboratively.  Business Stakeholders may have half the message, they may have heard Agile jargon and have expectations of some cost–savings or time to market improvement, but have not made the mental switch from cost to investment that underpins the benefits of delivering software in business value increments.

We see project teams thinking they have ‘gone Agile’ because they have thrown their documentation away and are having a 10-minute stand up meeting every morning.

Collaboration is the key to success

After having received technical training for Scrum, team members are generally quick to understand the methodology and principles, but the training can stop short of telling teams how to collaborate.  Team members know they should be having a close dialogue with developers and business users, but don’t have tools or guidance for doing so, and often don’t have predictable access to the Product Owner.

The following problems can occur;

  • Teams fall into ‘mini waterfall’ patterns within each iteration.  This is very expensive as testers sit around doing nothing until enough has been developed for them to test.
  • Defects get raised at the end of an iteration and are progressively pushed to the next, resulting in a final (maybe extended) defect fix iteration to sort everything out.
  • Testers complain that there is not enough information in the user story to test from and feel uncomfortable and under-valued within the agile team.
  • The Product Owner is frequently unavailable and there is no business representation within the iteration.
  • There is a lack of shared understanding of the goal of the iteration, the high level architecture, or how the smaller components fit within the higher level vision
  • There is a total lack of documentation after the project has finished
  • Automation lags behind with high maintenance overhead.
  • Business stakeholders are unable to prioritise requirements and want everything delivered to a fixed deadline.
  • Business stakeholders want detailed estimations and fixed-price costs up front for the whole project.

All of which lead to inevitable delays, rework, quality issues and a very disappointed customer.

Continuous delivery is hard

There are a number of Agile principles focused on regularity of delivery;

  • Our highest priority is to satisfy the customer
    through early and continuous delivery
    of valuable software
  • Deliver working software frequently, from a
    couple of weeks to a couple of months, with a
    preference to the shorter timescale.

There is often more focus on achieving the capability to deliver software continuously, (very challenging) rather than on delivering value early, which is where the true business benefit lies.  Whilst it is a good aspiration in a mature Agile team to have a very short iteration, if you try to get there too quickly without recognising the technical constraints, or without having understood which features deliver the most value, it can be disastrous.

There are several factors that impact the regularity with which a software development initiative can deliver to production, and achieve true continuous delivery.  For each development project we should ask these questions;

  • How many 3rd parties are involved in this development?
  • What is their ability to iterate? Do they have a long acceptance test schedule and infrequent releases?
  • How big is this thing?  Are there many applications, which need to be integrated together, and which require end-to-end integration testing?
  • How easy is it to introduce change to a particular functional area without needing to regression test the whole system?
  • Are there legacy systems involved?
  • Is this green field development?

The Solution

Pragmatism please

We need to recognise that it is not always easy to achieve very regular releases, especially on enterprise projects and in large organisations with lots of legacy systems and points of integration.

If we agree however, that the biggest business benefits lie in delivering value early rather than continuously we are left with a business challenge rather than a technical challenge.

The challenge is to understand which features have the greatest impact on ROI and deliver them first.

We can do this by applying the agile principles around communication, collaboration and prioritisation without necessarily having overcome all of the technical challenges around automation, tools and continuous delivery.

In fact we can use a structure for collaboration and prioritisation for all projects and decide how regularly we can iterate on a project-by-project basis.  We can tackle the continuous delivery challenges in parallel

From Cost to Investment

Agile is a mind-set change for the whole organisation, not just development teams.  Some more Agile Principles;

  • Agile processes promote sustainable development.
    The sponsors, developers, and users should be able
    to maintain a constant pace indefinitely.
  • Simplicity–the art of maximising the amount
    of work not done–is essential.

Users want everything they have asked for.  Business Stakeholders want detailed estimates up front.  Projects often run over on budget and schedule.

Business Stakeholders see development projects as costs (sometimes out of control) rather than as an investment against a Business goal.

This is a particularly painful truth for an organisation trying to adopt agile delivery methods.  With incremental delivery, the whole premise is that you don’t dive into the detail of a requirement until you need to.   That means you might not know how big and complicated a requirement is (and how much it might cost) until you get to it. The cost is fixed and the scope can vary

In most organisations there is some work to do to convince Stakeholders that this is a good idea.

Getting Business Stakeholders comfortable with the idea of flexible scope requires them to fully embrace the idea that the features with the highest business value, contributing most significantly to the ROI will be delivered first and at the earliest possible opportunity.

More so, if there is very clear understanding of the business goals and how features contribute to the ROI, they can make decisions with regard to how much they would like to invest towards the achievement of that goal.  Furthermore they can consider how they would like to distribute that investment across the scope of features, and finally how that translates towards achievement of short term, mid term and long term business goals.

If we collaborated effectively with the Business to get their buy in and agree on all this stuff up front we’d be half way there.

Summary

The Right Product

Too often we get wrapped up in delivering to a certain set of listed requirements rather than achieving the underlying business case.  Through closer collaboration with the Business, a clear vision of the product roadmap and a very clear identification of the highest priority requirements, project teams can get on with delivering the Right Product.

If we are able to give development teams a structure and tools that help them collaborate with Business users more effectively and tackle highest priority requirements first (regardless of how regularly we can deliver to production) then we can be assured we are focusing our efforts in the right direction.

If we focus on collaborative development techniques that reduce misunderstandings around requirements and maximise efficiency, we can reduce defects, reduce costs and get to market quickly.

If we can bridge the gap between technical and business teams, communicate using the same language and plan for collaboration in a predictable and structured way, we get more useful input from the Business.  Development teams with a closer proximity to the Business Domain have higher productivity.

Through collaboration we can allow Business priorities, business language and real world examples to inform the design, resulting in a lean and highly maintainable product.

We can do all this regardless of how well we follow the mechanics of Agile and how well we are doing with our continuous delivery capability.

And when we get good at this stuff, we can start worrying about tackling the harder stuff.