In part 4 of this series, the authors guided us through implementation and development. Ed.
Once your platform has been implemented and handed over to your team, you will need to make sure that everything works the way you want it to work. After all of the effort and time you have put in, the last thing you want is to fail now. This is where testing comes into play.
What is User Acceptance Testing?
Unfortunately, we know from experience that if KM does not conduct the testing, you can never be certain the product you hand over to your users will work as they expect it to. This is true regardless of how comprehensive the IT department’s quality assurance testing is. The hard truth is that IT professionals see things differently than most users. IT people are not lawyers and do not think like lawyers. As a KM professional, your job is to operate between the technical and legal realms to evaluate whether your users will be able to incorporate into their daily practice the new platform as designed. This crucial step is called user acceptance testing (UAT).
The form and quantity of UAT the KM team performs varies enormously from firm to firm and project to project. We have worked in firms where very little testing is done before launch and in firms where massive testing is done. In truth, the proof is in the pudding – the more effort you put into UAT, the better your outcome will be.
Where do you start?
Our first step in UAT always is to create a test plan. We find this so indispensable that we now create plans even for testing the most minor upgrades to a platform. When creating the test plan, return to your requirements and use cases developed for your business case. You should plan to test every element of functionality a user would touch, understanding the desired outcome based on your requirements. For example, in our search system we determined that ensuring that a filter worked when applied to a set of search results would not be sufficient. Instead, we also needed to check that the list of values for each filter was correct, we could both exclude and apply filter values, the “or” (rather than “and”) logic of the filter values was in place when multiple filter values were applied at once, and we could both scroll through and search for filter values.
Our test plans usually outline a process where each team member mimics a user’s journey through the platform based on our use cases. Depending on what we are testing, we may allocate roles to different testers, with everyone following a slightly different journey through the platform. For example, partners likely use most platforms differently than legal assistants and you need to ensure the functionality works equally well for both. Role-based testing helps you to confirm that your new technology is ready to be released to everyone in the firm.
People doing the testing should note failures in not only pure functionality, but also design. An IT professional testing a grid view may feel that clicking on a column heading and having the results sort is just fine. But, a KM professional will likely recognize that unless a symbol indicates that sort functionality is available, most users will not know the option exists. Your requirements likely included this need early on, but testers should be on the look-out for its having made its way into the actual build.
Because our firm is Canadian and has a major office in French-speaking Quebec, testing bilingual functionality is vital. Recognize and prepare for the fact that your firm may have its own unique jurisdictional requirements or other complexities that must be addressed through a customized UAT process.
Allow sufficient time for UAT
Depending on the complexity of your project and testing, UAT may take some time. In our experience, no test phase ever goes without hitches. Assume you will have multiple issues to report and some elements of the functionality will not work properly on first review. Also prepare for substantial back and forth with IT and the vendor as you work through the issues that testing reveals. During our enterprise search project, we completed several rounds of testing for each component of the project.
Improve inter-departmental communication
Perhaps the most important takeaway from our major projects has been the importance of communication. Communication is critical during testing and it is hard. Communication among team members and between KM and IT, the firm, and external vendor all are critical. Most communication challenges that arise between your team and IT likely can be improved through mutual empathy and understanding.
As observed, most IT professionals speak a different language than legal professionals and you must account for this in the communication style you adopt between departments. For example, during testing, it is tempting for individuals who find major issues to want to report them to IT at once to be addressed quickly. Of course, in some instances this may well be appropriate. But, we have found that waiting to report all issues together generally works better. One of our team members collects feedback from everyone else, consolidates it into a single spreadsheet, and sends it to IT. Every piece of feedback is numbered and, wherever possible, accompanied by screenshots. We have found that the words we used to describe the problems we find often are insufficient to communicate those issues to IT. In fact, we have been tempted to sit beside our IT colleagues and show them what we have experienced on our screens. This rarely is the best use of anyone’s time, nor should it be necessary as visual props should abate confusion and frustration.
Lessons Learned
Engage multiple testers: People have different aptitudes for testing and technical detail. In the same testing round, we have had someone report that everything looks great, while another person reports over 25 issues. So, you will need a team of testers for UAT. You simply cannot do this on your own.
Understand your team’s limitations: During our enterprise search project, the KM team tested the system’s ethical walls compliance. Even though we managed to do it, it was needlessly time-consuming and, in retrospect, this should more properly have been handled by IT. We would not take this on ourselves again. So be realistic – some things are better left to IT.
Know when to stop: After you have signed-off on a component of the project, stop testing that component’s functionality. It sounds simple, but letting go and trusting that the fixes really are in place can be difficult in practice. Now, if in using the platform you stumble across new issues, these of course must be reported. Otherwise, encourage your team to trust your firm’s IT and the vendor and move on to the project’s next stage.
Coming up in part 6: In our next post, we get to the exciting part – how to prepare for launch and take-off.
By Andrea Alliston, Partner, Knowledge Management, and Nicola Shaver, Director of Knowledge Management, Stikeman Elliott LLP
#KnowledgeManagementandSearch