Evaluating Design Ideas
Why Evaluate Design Ideas?
Generating design ideas is only the beginning of the design process. Before committing to one concept, an engineer must evaluate each idea objectively against the requirements set out in the design brief and specification. Without systematic evaluation, decisions are based on personal preference rather than evidence — increasing the risk of selecting a design that fails to meet user needs, cannot be manufactured within budget, or does not satisfy safety requirements. The specification defines what the product must do; evaluation is the process of measuring each design idea against those criteria.
Production of Models
One of the most direct ways to evaluate a design idea is to produce a model. Models can be physical (made from card, foam, block materials or 3D printed) or virtual (3D CAD). A model allows the designer to assess whether the proportions and scale of the design are correct, whether moving parts function as intended, and how the product looks and feels to a user. Unlike a drawing, a model can be picked up, tested and modified. The information obtained from a model depends on its type and fidelity: a rough card model reveals proportions and scale; a breadboard circuit reveals whether the electronic function works; a 3D-printed prototype reveals surface quality, assembly fit and user ergonomics.
The four methods of evaluating design ideas in the specification are: production of models, qualitative comparison with the design brief and specification, ranking matrices, and Quality Function Deployment (QFD).
Qualitative Comparison with the Design Brief and Specification
Qualitative comparison involves working through each criterion in the design brief and specification and making a judgement — usually written as a statement — about whether the design idea meets, partially meets or fails to meet each requirement. For example: "This design meets the aesthetic criterion because the organic shape reflects the brief's requirement for a natural appearance; however, it does not meet the size criterion because the width of 180 mm exceeds the maximum of 150 mm stated in the specification." Qualitative comparison is valuable because it directly links the evaluation to the specification — but it is subjective and does not provide a numerical score that allows direct comparison between multiple design ideas.
Ranking Matrices
A ranking matrix (also called a decision matrix or weighted matrix) provides a more systematic method of comparing multiple design ideas. The criteria from the specification are listed in rows; each design concept is given a column. Each design is scored against each criterion (typically 1–5 or 1–10) and the score is multiplied by a weighting factor that reflects the relative importance of that criterion. The weighted scores for each design are totalled, and the design with the highest total score is identified as the strongest candidate. Ranking matrices can reduce the influence of personal bias by encouraging the evaluator to consider all criteria and use consistent scoring.
To construct a ranking matrix: (1) List all specification criteria in rows — these might include function, aesthetics, cost, safety, size, ease of manufacture and sustainability. (2) Assign a weighting to each criterion on a scale of 1–5, where 5 is most important. (3) List each design concept in columns. (4) Score each concept against each criterion — for example 1–5, where 5 is best. (5) Multiply each score by the criterion weighting to give a weighted score. (6) Sum the weighted scores for each concept. The concept with the highest total is the strongest candidate, but the designer should also check that no critical safety or function criterion has received a very low score, since a high total score in minor areas cannot compensate for a fundamental failure in a critical one.
Quality Function Deployment (QFD)
Quality Function Deployment (QFD) is a more advanced evaluation and planning tool originally developed in Japan. It is sometimes called the House of Quality because of the shape formed by the matrix and its roof section. QFD maps customer requirements (the "whats" — what the customer wants) onto engineering characteristics (the "hows" — how the product will be designed to meet each requirement). The central body of the QFD matrix shows the strength of the relationship between each customer requirement and each engineering characteristic, typically using symbols for strong, medium and weak relationships. The roof section shows any conflicts or trade-offs between engineering characteristics — for example, making a product lighter might conflict with the need for high structural strength. QFD helps engineers prioritise which engineering characteristics have the greatest impact on customer satisfaction.
In a QFD (House of Quality) matrix, the rows list customer requirements (the "whats") and the columns list engineering characteristics (the "hows"). The body shows relationships; the roof shows conflicts or trade-offs between engineering characteristics.
Each evaluation method has strengths and limitations. Production of models is hands-on and reveals real-world issues with proportion, function and assembly that drawings cannot, but models take time and money to produce. Qualitative comparison is quick and directly linked to the specification but is subjective and cannot rank multiple designs numerically. Ranking matrices provide structured numerical comparison of multiple designs but depend on the evaluator assigning sensible, consistent weightings. QFD is highly systematic and reveals relationships and conflicts between engineering requirements, but it is complex to construct and most useful when there are many competing design requirements to manage. In an exam, select the method that best suits the context given in the question.