August 2016 Slide of the Month

Screen shot CIRS_August_2016_Slide_of_the_monthIntroduction

An organisation that seeks to improve its productivity should also routinely measure the quality of its decision making (Kahneman, 2011). However, decisions with high uncertainty should be judged by the quality of the process, not just by the consequences. Nevertheless, decisions underpinned by a quality process should lead to better outcomes. Consequently, there is a need to develop, validate and employ frameworks and tools to build quality into the decision-making process in drug development and the regulatory review. Prior studies have lead to the identification of 10 quality decision-making practices (QDMPs) that underpin a quality decision-making process (Donelan et al., 2015). These were generally considered as relevant by both pharmaceutical companies and regulatory agencies (Bujar et al., 2016). To further improve the quality of outcomes in medicines development, the degree of incorporation of these 10 QDMPs into agency and company processes needs to be evaluated.


  • The Quality of Decision-Making Orientation Scheme (QoDoS) consisting of 47 items that measure individual and organisational decision-making approach and influences (Donelan et al. 2015, 2013) was completed by 76 participants from regulatory agencies and pharmaceutical companies.
  • All participants were asked to respond to each of the QoDoS 47 statements using a 5-point Likert scale, where 0=not at all, 1=sometimes, 2=frequently, 3=often and 4=always; Organisational-level agency and company responses were mapped against the 10 Quality Decision Making Practices.


  • A total of 38 individuals from 12 regulatory agencies and 38 individuals from 23 pharmaceutical companies with varying levels of professional experience (range, 2-40 years) and representing medium-sized and large organisations participated in the study
  • There were key differences between company and agency practices; QDMP 3 (‘‘assign values and relative importance to decision criteria’’) is incorporated to a greater extent by companies, whereas QDMP 4 (‘‘evaluate both internal and external influences/biases’’) is embedded more widely by agencies. Agencies incorporate QDMPs to a greater extent compared with companies, but there is room for improvement for both stakeholders.


Companies feel their decision making is influenced to a greater extent by certain biases due to politics, competitors, vested interests of individuals, or incentives compared with agencies, and that QDMP 4 (‘‘evaluate both internal and external influences/biases’’) is generally incorporated to a lesser extent by companies compared with agencies. This finding may be a result of a mixture of complex factors within the company medicine development process, such as the large number of stakeholders involved, as well as the length of the process and the number of key go/no-go decision points. Finally, there was a large difference in responses regarding incorporation of QDMP 3 (‘‘assign values and relative importance to decision criteria’’), with agencies performing this less routinely than companies. This is consistent with the findings from previous research, and one of the recommendations suggested by companies and agencies to improve the quality of their decision making was to make values, preferences, and uncertainty more explicit. As advised by the responders, such an outcome can be achieved through the incorporation of more formal frameworks, such as a benefit-risk framework by agencies as well as more formal approaches to quality decision making within the organisation and by individuals (Bujar et al., 2016)

From: Bujar M, McAuslane N, Salek S, Walker S. 2016. Factors Influencing Quality Decision Making in Medicines Development and Regulatory Review: Biases and Best Practices. Poster presented at: the DIA 2016 52nd Annual Meeting; June 2016; Philadelphia, PA.


Bujar M, McAuslane N, Salek S, Walker S. Quality of regulatory decision-making practices: issues facing companies and agencies. Ther Inn Reg Sci. 2016;DOI: 10.1177/2168479016628573; Centre for Innovation in Regulatory Science. 2004. Building quality into regulatory dossiers and the review process: Knowing and meeting customer expectations. Surrey: Workshop Report; Donelan R, Walker S, Salek S. Factors influencing quality decision-making: regulatory and pharmaceutical industry perspectives. Pharmacoepidemiol Drug Saf. 2015;24: 319-328; Donelan R, Walker S, Salek S. 2016. The development and validation of a generic instrument, QoDoS, for assessing the quality of decision making. Frontiers Pharmacol Pharmaceutical Medicine and Outcome Research). In press; Kahneman D. Thinking fast and slow. New York: Farrar, Straus and Giroux 2011.

Download this slide: