ANALYSIS: Health Technology Assessment Tools
Written by Preetinder S. Gill
Many different definitions exist for health technology assessment (HTA). US Congress’s Office of Technology Assessment defines HTA as a structured analysis of a health technology, a set of related technologies, or a technology-related issue that is performed for the purpose of providing input to a policy decision. The European Observatory on Health Systems and Policies describes HTA a form of policy research that systematically examines the short and long term effects of a health technology or a set of related technologies. HTA can be initiated by any stakeholder of the healthcare system – policy makers, patients’ groups, providers, payers and health care managers in order to evaluate efficacy and effectiveness of 1) a new technology, 2) improvement to an existing technology or 3) change in the operating conditions of a technology. HTA can also encompass economic (cost-benefit) analysis. Additionally, HTA can assess concerns related to safety and ethics. Simply put HTA is decision making tool which systematically analyses effects of technology.
Despite the existence of many definitions of HTA there seems to be a consensus about two keywords: 1) requirements [of stakeholders] and 2) effects [of the technology(ies)]. There are two tools – quality function deployment and failure mode effect analysis – which are employed by product engineers that can be applied to these keywords.
Quality function deployment (QFD) is method widely used in product engineering. QFD is a methodology which is used to transform customer requirements into the design of a system. The QFD ensures that emerging functions of subsystems and components fulfill all the implicit and explicit requirements of the customer.
Failure mode effect analysis (FMEA) is a tool used to identify, rank and mitigate risks. Essentially FMEA involves assessing: 1) severity of a failure mode, 2) likelihood of occurrence of the causes of the failure mode give the preventive measure in place and 3) likelihood that a failure cause or the associated failure mode can be detected given the detection measures in place. This tool was first introduced by the US Military, standard # MIL-P-1629, back in late 1940s. This tool is currently used extensively in many industrial sectors. Joint Commission on Accreditation of Healthcare Organizations’ introduced standard Req. L.D. 5.2 for Health-FMEAs in July 2001.
It must be noted that both QFD and FMEA require teamwork between stakeholders with expertise in different fields. Hence, the importance of the role a team plays in the successful application of these tools cannot be emphasized enough. The QFD and FMEA can be combined in a novel way, described below, in order to supplement the HTA process.
Functional Analysis In HTA
Functional analysis (FA) in HTA must start with formation of a cross-functional team. Typically the core team could have 4 to 7 members who are facilitated by a neutral moderator. Besides the core team members specific subject-matter experts could be invited to meeting sessions, as needed. The four step FA process is described below:
1. The first step of the functional analysis involves collecting the requirements associated with the technology(ies) being assessed. The requirements come from four sources: 1) users/patients, 2) operators/staff, 3) internal guidelines and, 4) regulatory demands. The FA team can research various documents for example product brochures, existing research studies and historical data to identify these requirements. Additionally, the team can brainstorm to identify any requirements which were not found in the literature review. Regulatory demands pertaining to the three types of liabilities: manufacturing defect, design defect and a failure to warn/marketing defects must also be considered as requirements imposed on a technology(ies). The traditional HTAs involve evaluation of effectiveness, efficacy, safety and cost analysis. Hence, they only tend to cover the first two sources of requirements. The FA approach thus, could include additional requirements in the assessment process.
2. The second step involves decomposing the technology being assessed in sub-modules. To illustrate this step let’s consider a hypothetical glucose monitoring device. The overall assembly could be broken in following sub-modules: blood collecting unit, sensing unit, conversion unit, display unit, memory unit and communication unit. The display unit can be further broken down into the following components output port, input port and LED display among others.
3. In the third step the FA team lists the functions for the overall assembly (the technology being reviewed) and the sub modules and components involved. These functions then need to be connected in a means-purpose relationship. In other words relationships need to be defined between each function of sub-module/components and functions of the higher level assembly. The functions of the overall assembly then need to be connected to the requirements identified in step 1. Software such as APIS Informationstechnologien GmbH’s IQ-RM can be used to complete this kind of function analysis.
4. In the fourth step the team checks functional net associated with each requirement to assess whether or not every requirement is satisfied by the functions of the technology being reviewed. Further, the team can identify whether there are functions which don't seem to be associated with any requirement. This could in turn highlight any unintended effects of the technology being reviewed.
Team can quantify FA in terms of the Technology Assessment Score (TAS). TAS is a multiplicative product of three factors: importance, coverage and confidence. In order to calculate the TAS the team can assign each requirement an importance score on a scale of 1 to 10 where 10 is the highest importance. The coverage score of each requirement quantifies how well a particular requirement is fulfilled in terms of the functions of the overall assembly (the technology being reviewed). The team can assign the coverage score to each requirement on scale of 1 to 10 where 10 means perfectly covered requirement. The confidence score quantifies the level of understanding and comfort of the FA team with regards to the technical modalities of the various functions of the overall assembly (the technology being reviewed). In other words a new technology with novel functions will tend to have lower confidence scores whereas well tested and extensively understood functions will tend to have higher confidence scores.
Additionally, in assessing the confidence score the team must consider the underlying sub-functions. The confidence can be also be rated on the scale of 1 to 10 where 10 means absolute confidence. It must be noted while the importance and coverage scores are assigned to each requirement the confidence score is assigned to each function of the overall assembly (the technology being reviewed). The TAS for each requirement is calculated by simply multiplying its importance score, its coverage scores and the lowest confidence score associated with the specific requirement.
In Conclusion
A novel technique adapted from well know product engineering tools - QFD and FMEA – is presented. The healthcare managers and policy makers can use the FA approach for HTA supplement to their decision making by leveraging their team’s expertise.
The FA approach also provides a readily available knowledge base which can be used for quantitatively comparing various technologies. The FA documents can also be used to educate new employees about the workings of components which constitute a specific technology. Lastly, the FA approach can be used as tool for continuous improvement where lessons learnt from one analysis cycle can be applied to next cycle.
