Ordinal scales are commonly used in rating and evaluation processes. These processes usually involve group decision making by means of an experts’ committee. In this paper a mathematical framework based on the qualitative model of the absolute orders of magnitude is considered. The entropy of a qualitatively described system is defined in this framework. On the one hand, this enables us to measure the amount of information provided by each evaluator and, on the other hand, the coherence of the evaluation committee. The new approach is capable of managing situations where the assessment given by experts involves different levels of precision. The use of the proposed measures within an automatic system for group decision making will contribute towards avoiding the potential subjectivity caused by conflicts of interests of the evaluators in the group.