Who Should Define Non-Functional Requirements

Non-functional requirements (NFRs) are the heartbeat of any performance testing exercise. Loosely stated, these are the desired performance attributes and (more importantly) the values that these attributes need to have for performance to become acceptable. Actual application performance is then validated against the NFRs. In a strict sense, performance itself is a non functional attribute - but for the sake of this discussion, we'll assume that performance subsumes other relevant non functional attributes.

When performance testers have initial meetings with other team members to kick start the performance testing exercise, the demands for performance are often stated in terms such as "good", "fast", "efficient" etc. - this, unfortunately, is a very common observation. Performance demands expressed this way are well intentioned but are extremely subjective and open to interpretation. They must be stated in quantifiable values that can be measured.

A likely reason for the subjective description of NFRs is the lack of performance engineering related know-how of the team (particularly the non-technical members). What often happens is an inclination on everyone else's part to entrust the performance tester with the task of defining what is "good", "fast", and "efficient (along with other desirable attributes) and to define values for these attributes. This is understandable since the performance tester has probably had experience with other (similar) projects from which these values can be taken as a general rule of thumb. However all projects are different and conclusions or observations from one cannot be packaged and applied to the others.

It is important that the attributes (and their values) that pertain to user interaction (e.g. web page response times, in a web based application) come from the people who will be using the system and who understand the business well. These need to be defined early on - perhaps at the conceptualization level, prior to the design phase. Other attributes such as efficiency (e.g. hardware resource usage - cpu, memory, disk, network etc) need to be addressed in the design phase - meaning having the necessary hardware in place to meet the earlier requirements, while at the same time considering the costs involved as well as other design considerations. If minimal hardware resources are available, then clear limits on their acceptable usage need to be defined. The best person to define these is the software or enterprise architect. It is he/she who will also define the maximum number of user sessions that can be supported concurrently as well as what happens under conditions of stress (e.g. when one of the application servers goes down and all traffic needs to be handled by the other) or when there are sudden peaks in traffic at certain points of time. An application may have other critical needs such as high availability and scalability, ability to quickly recover from failure, robustness etc. - again it is the software or enterprise architect who must ensure that these are properly addressed.

If the performance tester is brought in from another team (say, a team of specialized performance testers) and the scope of his/her work is restricted to the current release, then it is critical that they do not own the NFR process. It is ultimately the project team that has to live with the software and no matter how hard or painful it may be, they should be the ones to agree on the non functional attributes and their values. The performance tester can certainly advise on what performance attributes should be considered for measurement (and perhaps their possible values also) but the final decision must be of the project. Even here his/her hands are slightly tied - the hardware resource related decisions may have already been made and implemented (in case performance testing wasn't part of the initial discussions - a very likely scenario), so the performance tester can only advise based on ground realities rather than what business would ideally like.