One of the critical success factors for our project was our adoption of ATTD and TDD, including frequent test collaboration, engaging the business in writing acceptance test criteria, and robust test automation. Collaborating on writing tests ensured that the entire team understood all stories, which facilitated swarming and cross-skill development. We got questions answered real-time from the business that used to occur in one-off conversations and caused system test failure rework. We saw dramatically reduced failures during exploratory tests which further stabilized velocity. We added 0 manual test cases to our regression bed, and had high quality in production.
Our project team was formed to deliver a business-critical function with a fixed delivery date. Many of the team members were new to line of business that we were asked to support. Early in the project, Acceptance Test-Driven Development, and more specifically Cucumber, was a useful tool for capturing and documenting the expectations of the system. The business team was able to communicate requirements in familiar terms, even if Gherkin was a new syntax. The IT team was able to understand table-driven examples, while learning the business terminology. The two collaborating sides of our project were never completely out of their comfort zones.
Having an automated test suite based on Cucumber gives both our IT team and business team the confidence to release our software to our production environment more frequently than other project teams and without long, manual regression test cycles typical of a company of our size. Our project team knows immediately when a software change unintentionally impacts another part of the system because our automated test suite is integrated into our continuous build environment. This point was clearly on display during a Sprint Review meeting with senior leadership when a series of tests failed during a demo of our application, caused by a pre-demo check-in from a project team member. In turn, we modified the demo to show the failing tests, and how the software would be corrected to prevent defects from leaking into our production baseline.
It's worth calling out one specific example of a Cucumber success story for our project. An entire software release was saved because we were able to debug a troublesome piece of business logic through the creation of a comprehensive, production-like integration test using Cucumber. In this case, we had a series of JUnit tests that were passing, but an integration scenario was failing. The JUnit tests were not comprehensive enough with their use of mock objects and data to uncover a hidden bug in the logic. Through the creation and debug of a real world example using Cucumber, we were able to find the flaw in our logic, correct the software defect, and elevate on schedule.
The product owner was originally skeptical going into training, but she is clearly a fan now, stating that it is "totally worth it" to do ATDD. Her thoughts were mostly about the effects of the workflow changes. Automated testing was more of an after-thought. She could clearly see and articulate the benefits of simply implementing the workflow change associated with ATDD. She said that one day of ATDD specific training, plus a half day of coaching, plus a committed team is all you need to get going. Benefits she sited included:
- Team "happiness factor" increased
- Specifically, the lead developer and tester are much happier
- Less stress on testers, more distributed testing effort across the sprint
- Helped to create/enhance the "we are a team" feeling
- Fewer production defects
- Fewer test environment defects
If you have any of these issues, there's a great chance that adopting ATDD will help in solving it.