Background
One of the improvements SAFe version 4.5 introduced was incorporating practices from “The Lean Startup” into the framework – specifically the use of benefit hypothesis statements into features and epics. This is a story of how well this worked for us.
We were worried about the standard of feature writing across our teams and also wanted to bring them up to speed with the SAFe 4.5 feature hypothesis thinking. Therefore, we organised a Friday afternoon “Lunch and Learn” session for interested team members.
The main objective was to go over a practical example from one of our feature teams and convert an existing feature into a ‘valuable feature hypothesis statement‘.
[If you are unfamiliar with the Lean Startup concept of hypotheses, please see supporting post: Using Hypothesis Statements for Features in Software Development]
Getting A Feature Benefit Hypothesis Statement
We started with the feature captured below that, no disrespect to team, was not very well written. The definition I like to use for a well written feature is that: ‘someone outside the team can read it once and understand what needs to be done’. At this stage I don’t even think that most of the team understood the feature.
Luckily we had the product owner (PO) in the room. We asked him a few probing questions that went something like this.
Team: Why are we doing this change? What is the benefit?
PO: We need to update the audit report fields for our customers.
Team: Why do they need the additional fields.
PO: So that customers can check for errors before they submit them to us for processing.
Team: Why do they check for errors? How does this work?
PO: All customers have a QA person or someone similar checking these reports to prevent errors being processed. When errors get missed and processed, it wastes time and causes a lot of frustration.
Team: So what exactly is the current problem?
PO: Today, customer auditors can’t see all the relevant information on the audit reports for cross-checking and referencing so many errors are still processed.
Team: What is the intended result of this change?
PO: We’ll reduce the error rate by 95%.
[And BAM! We get our hypothesis]
If we enhance the audit report to include the relevant details for verification by the client, we will reduce error rates by 95% which will enhance the client experience.
Interesting takeout: The PO did not come right out with the “95% error reduction” even though it was in his head from the start – it required a conversation to get his knowledge shared with the rest of the team. This is part of the magic of conversation within collocated teams – and the importance of having a PO who works with the team.
The benefit hypothesis makes the “why” clear to without having to write a long document. Everyone in the room quickly came to the same understanding as to what needed to be done and why. A well worded hypothesis statement helps remove ambiguities and focuses the team on what really needs to be done. [To understand why “why?” is important – see Simon Sinek’s TED Talk below.]
The other (perhaps even more) positive outcome is that it gives “purpose” to the teams’ work. I asked the group, “Would you rather (a) update some fields on an audit report or (b) reduce error rates by 95%?” – unsurprisingly there was unanimous agreement that (b) was the more exciting option.
“Reducing error rates by 95%” gives meaning to the team’s work. I am unlikely to go home and proudly tell my kids I updated some fields on an audit report. But telling them that I helped reduce foreign exchange error rates by 95% makes my job as a developer on a transactional banking system sound sexy and important!
Feature Acceptance Criteria & Slicing
We then moved onto the feature acceptance criteria (AC). The simple way I view feature AC is, “How will we UAT the feature to know that it’s complete and fit for purpose?” (as opposed to user story AC which are the actual unit test cases).
If you are not able to write clear AC you don’t know enough to proceed with the feature so this was a good test of the strength of our feature hypothesis. Once again it was an interesting and valuable discussion with participants throwing various questions at the PO. A summary is below:
Which countries are in scope?
The PO initially said “all countries”. Further conversation sliced it down to two countries (South Africa and Uganda) where there was definite current need – from there it made sense to split the feature into one for each country. South Africa had the most urgent need so we focused on that and the lower priority Uganda feature went onto the backlog (where it will remain until it becomes priority). The requirement that the initial feature for South Africa be scalable for other countries was included as a non-functional requirement.
Attempted slipping in of a production defect
The PO tried to slip a production defect into the AC (the format of the onscreen and printed reports are different). The team deftly managed to slice the defect off the feature (the defect fix is roughly the same size as the eventual feature). The valid defect was added to the team backlog (where the PO can prioritise it against other work to determine when it will get fixed).
Attempted scope creep
The PO also tried to slip a new requirement into the AC – the ability to be able to save audit reports as a .pdf file. Once again the team deftly convinced the PO that this was not part of the minimum viable product (MVP) by referring to the hypothesis statement (i.e. we don’t need to save to .pdf to test the hypothesis). The valid requirement was also added to the team backlog (and the PO gets to decide when it’s priority enough to be built).
Interesting takeout: In my opinion, the PO was doing his job perfectly – trying to get as much as possible into the feature to maximise the customer benefit. Because he’s part of the overall team having the conversation he gets to understand the trade-offs involved with different decisions. When posed with the question, “Do you want to have the feature done in 2 weeks if we just do MVP or do you want to wait for 2 months if we do ‘all this other stuff‘?” a good PO will always go for “small and fast“.
By reducing the feature to a single (highest priority) country and omitting other requirements that (whilst important) had no impact on the hypothesis statement, the feature ended up being at least 10 times smaller than if we’d tried to include everything. Slicing the feature down to get the most value with the least amount of work drastically increases the speed with which it can be built and dramatically reduces risk.
The team left the room knowing exactly what needed to be done. More importantly everyone had agreed on what didn’t need to be done “right now” (i.e. in the next sprint) because they were not MVP (however these requirements have been added to the team backlog and can be done when the PO prioritises them). The PO was also part of all the decisions taken so he should get no nasty surprises when the feature is presented back as “done”.
Conclusion: Super-Quick Delivery
The best part of this story was that two weeks later when I asked the team “How’s the feature going?”, the reply was “It’s already done.”
The team was able to fully design, build and test a valuable feature in less than two weeks (fitting easily into a 2-week sprint). In the past a feature like this would usually take 12 months to deliver (see the supporting post “6 Reasons: From 12 Months to 2 Weeks for Feature Delivery“). By slicing the feature down to its true MVP and the team knowing exactly what was needed and why, the feature flew through the development value stream.
Of course, now the real test is to measure whether our hypothesis proves true and we successfully reduce the error rate by 95% – let’s hope so! We’ll know pretty soon…
Afterward
The actual reduction in client error rates was 80%. As it stands this has been deemed sufficient. There are no plans to build additional features to further reduce the error rates (as there have been higher priority features to work on). Likewise, the production defect has not been fixed (and may never be) since it has never been a priority compared to other more beneficial work.
I delivered a presentation on “The Power of Feature Hypotheses” at Agile Africa 2018 using this and other examples. Here is the video link from the conference: https://youtu.be/CUVX1AiqQak?list=PLp6xQ3fl72zIu8FJjDtBUFUS0w1FPQhPZ
I am also more than happy to submit a speaker proposal for your conference. Feel free to contact me via email: therunningmann@gmail.com or Twitter @runningmann100 / @StuartDMann.
Follow Running Mann:
Good account of what we discussed in the session and what was learnt. Going forward it will definitely provide a mechanism for us to access information on the sessions to further unpack.