As a consultant I have been asked to work across a number of different industries and within a number of different methodologies. One of the constants across all of these variables is the need for User Acceptance Testing (UAT). I have, however, found working with multiple clients that the definition of UAT can vary greatly.
ISTQB defines UAT as:
Acceptance testing carried out by future users in a (simulated) operational environment, focusing on user requirements and needs.ISTQB
My rather simplistic definition is:
Ensuring that the system under test meets the needs of the user as required.James Brodie
I found that, especially in an Agile environment, UAT was being bolted on as part of the sprint and was concentrating on the business components rather than looking at the end-to-end business journeys. I felt that what was being called UAT wasn't actually encompassing what was being tested — it wasn't what the client needed, and it was creating a bottleneck in the Agile process.
A common theme I also observed was that the client would often be working with development third parties, and there would be a lack of confidence in the output being delivered. This would undermine the test process before it even began.
Working initially with one of my retail clients, I recommended that we expand what was being called User Acceptance Test to Acceptance Test, and then call out the different elements we are looking to accept. Acceptance Test would then complement the existing system testing already being performed by the suppliers and the client's own test teams.
The three pillars of acceptance
Witness Acceptance Testing
- The third-party supplier is responsible for the functional testing of their internal solution. The expectation is that the supplier will provide the client with a list of what has, and hasn't, been tested.
- The supplier then provides the End-to-End Test Manager with artefacts demonstrating that the component has been tested.
- These artefacts include an agreed subset of scripts, plus documentation that can be "witnessed" — acting as the acceptance test into Business Component and End-to-End Testing.
Business Component Testing
- This is the acceptance testing of singular business functionality — typically what the client will describe as UAT.
- The output identifies the functionality that will be tested end to end as part of UAT.
User Acceptance Testing
- This is traditional end-to-end UAT.
- Confirming that the solution under test meets the needs of the user as required.
The purpose of this approach is to build business confidence in the system under test.
Witness Acceptance Test has the advantage that it can be used as part of the entry criteria into test, but it also gives the business the tools to ask development — internal or external — to evidence the quality of the testing already undertaken. It stops development from simply throwing code over the fence to hit a deadline, rather than providing a quality deliverable.
Business Component Test mirrors System Test, but from a business perspective. Within an Agile environment, it checks that the end-of-sprint deliverable meets the needs of the user.
By identifying the first two pillars of Acceptance Test, User Acceptance Test is then free to concentrate on the end-to-end solution — and I've found this also helps focus the client's mind on specifically calling out that they are looking at the complete picture.
As ever with an acceptance-based approach, one of the key factors is ensuring you have the right people in the test roles. A tester assigned to acceptance test must have a business background, and I'd recommend that, wherever possible, the business ring-fences its acceptance test team. This means the team gains experience as they work, which also helps with future training.