The accuracy of a measurement system is analyzed by segmenting into two main elements: repeatability (the ability of a particular evaluator to assign the same value or attribute several times under the same conditions) and reproducibility (the ability of several assessors to agree on a set of circumstances). In the case of an attribute measurement system, repeatability or reproducibility problems necessarily pose precision problems. In addition, if global accuracy, repeatability and reproducibility are known, distortions can also be detected in situations where decisions are always wrong. Second, the evaluation of the attribute agreement should be applied and the detailed results of the audit should provide a number of information that will help to understand how evaluation can be the best way to be organized. Assuming that the accuracy rate (or most likely error modes) of the bug tracking system is unknown, it is advisable to check 100 percent of the database for an appropriate framework of the previous history. What`s reasonable? It really depends on that, but to be sure, at least 100 samples should be examined over a last representative period. The definition of the database should take into account how database information should be used: to prioritize projects, investigate the cause or assess performance. 100 audit examples are a good starting point because it gives the analyst an approximate idea of the overall accuracy of the database. However, a bug tracking system is not an ongoing payment.
The assigned values are correct or not; There is no (or should not) grey area. If codes, locations and degrees of gravity are defined effectively, there is only one attribute for each of these categories for a particular error. If z.B. the accuracy rate calculated with 100 samples is 70 per cent, the margin of error is about 9 per cent. At 80 per cent, the margin is 8 per cent, 90 per cent 6 per cent. Of course, more and more samples can be collected to check if more accuracy is needed, but the reality is that if the database is less than 90 percent accurate, the analyst probably wants to understand why. In this example, a repeatability assessment is used to illustrate the idea, and it also applies to reproducibility.