In the present age of globalization and digitization no organization can afford to get too comfortable with the status quo. Constant improvement and perpetual adjusts have become the new standard in business. Companies invest in new solutions and initiate projects in order to attain improvements and sustainable goals. But how can we evaluate these kinds of solutions? And how should their value be defined? Solution evaluations in business analysis offer some insight.
So what is a solution evaluation?
Solution evaluation in business analysis is the assessment of existing, or potential solutions before or after a business goes into operation. Business analysis includes a comprehensive conception of the term “solutions” – it also contains both IT and non-IT components as well as systems, processes, rules, and organizational structures.
The term solution evaluation is derived from the Business Analysis Body of Knowledge Guide (BABOK) version 3, the business analysis authority published by the International Institute of Business Analysis (IIIBA) in Toronto, Canada. Solution evaluation is one of the areas of knowledge in BABOK that is partially represented in the requirements engineering literature, which makes the perspective particularly interesting.
The external and internal perspectives in solution evaluation
The ultimate value of solutions should be defined by cost savings, turnover increases, improved customer and employee satisfaction or better legal compliance. That refers to a business’s externally marketed products or services as well as their internal solutions.
Generally solutions in this context can be considered business requirements. Before solutions can be evaluated it is necessary to have documented the fundamental business requirements and use cases. This must be comprehensive and transparent. Herein lies the first challenge in the evaluation of solutions: unclear or incomplete requirements lead to unsatisfying evaluation criteria or to unsuitable prioritization and eventually to weak solution evaluation. In this context a mask for data capturing in a CRM system could be appropriate for internal use, while it is unusable for use with clients.
Excitement and disappointment
It is important to develop truly relevant evaluation criteria from business requirements in order to achieve meaningful solution evaluation. A solution’s key features are the most relevant factors, they are the focus of evaluations (e.g. a client’s monthly turnover report must be exported as a PDF). Evaluation criteria for functional requirements can often be derived from requirements. But it is often the qualitative execution of solutions that determines whether clients are satisfied or not with a product or service (how “beautiful” is the report? How readable or understandable is it and how quickly can it be made available?) Unfortunately because it is so hard to ascertain the quality of the non-functional requirements, very few people pursue this knowledge during a project. The Kano model of customer satisfaction can be helpful in developing evaluation criteria. It makes it possible for a turnover report to create excitement among customers, and for the report to actually be used.
Requirements and evaluation criteria
Evaluation criteria and their weighting in solution evaluation is no trivial matter. It’s worth the effort though because the questions that come up during these tasks lead to new discussions and help you to understand your stakeholders better. In this regard it is advisable to identify criteria as early as possible. If a development department knows the criteria that are necessary for your products after completing a solution evaluation, it will be able to optimally set its mind on it and satisfy clients.
Evaluation criteria and weighting
Solution evaluation can be used internally both for the evaluation of newly created solutions on the basis of mockups, prototypes or half-finished products as well as for existing solutions such as establishing if a solution has a commercial future.
Externally the solution evaluation is applied in tenders and in individual departments in order find out what the demand is or to find a fitting solution for a delivery.
A weighted evaluation matrix has proved very useful over time. In it the evaluation criteria receive double the amount of attention, summarized in their weighted groups. So in the example below the group “functionality” was given the weight 40%, a third of which stands for the qualitative execution. This method is suitable for attaining a certain level of objectivity in the evaluation of solutions and solution providers, because the endeavor is steered less by each stakeholder’s personal desires and more by objective criteria.
Whether solutions that have been promised by the requirements elicitation actually materialize is a critical success factor in projects. That’s why solutions and value promises need to be continually and iteratively analyzed from the earliest stages. With the help of partial deliveries, mock-ups, prototyping or proof concept installation the quality of the solutions will become visible early — even with regards to aspects that can only be vaguely specified, such as user friendliness.
The use of solutions should constantly be re-evaluated. This also applies to solutions implemented after a company goes into business, even years afterwards. You should always question whether the solution still addresses the business needs or if it has changed in the meantime, or even disappeared. And whether the cost of running it exceed the value. In this regard every end, whether overhauling or re-imagining a system or system component, must be accompanied by a cost/benefit analysis. In this way areas in which investments in new solutions make the most sense are reliably identified. This allows you to establish whether investments will have a sustainably positive effect on the company.
Studies have shown that an insufficiently planned and structured solution evaluation can lead to significant risks. If decisions are based on deficient approaches, then over time they can result in poorly suited solutions in the company, which in turn can cause massive (avoidable) damage. It could also lead to a waste of time, turnover, staff and even customers.