VOL. 10 Nº 2 MAYO – AGOSTO 2019 BUCARAMANGA, COLOMBIA
E-ISSN: 2346-3414
ARTÍCULO DE INVESTIGACIÓN
Scale for measurement of healthcare-associated infection risk in adult patients: development and content validation
Escala para la medición del riesgo de infección asociada a la atención en salud en pacientes adultos: desarrollo y validación de contenido
Escala para a medição do risco de infecção associada à assistência à saúde em pacientes adultos: desenvolvimento e validação de conteúdo
Alba Luz Rodríguez-Acelas1, Wilson Cañon-Montañez2, Miriam de Abreu Almeida3
1Faculty of Nursing, Universidad de Antioquia, Medellín, Colombia. E-mail: aluz.rodriguez@udea.edu.cohttps://orcid.org/0000-0002-7384-3522
2Faculty of Nursing, Universidad de Antioquia, Medellín, Colombia. Corresponding Author. E-mail: wilson.canon@udea.edu.cohttps://orcid.org/0000-0003-0729-5342
3School of Nursing, Universidade Federal do Rio Grande do Sul, Porto Alegre, Brasil. E-mail: miriam.abreu2@gmail.comhttps://orcid.org/0000-0002-4942-9882
Histórico
Receipt date: 09 de febrero de 2019
Approval date: 30 de abril de 2019
How to cite this article: Rodríguez-Acelas AL, Cañon-Montañez W, Almeida MA. Scale for measurement of healthcare-associated infection risk in adult patients: development and content validation. Rev Cuid. 2019; 10(2): e771.http://dx.doi.org/10.15649/cuidarte.v10i2.771
©2019 Universidad de Santander. This is an Open Access article distributed under the terms of the Creative Commons Attribution- NonCommercial (CC BY-NC 4.0). This license lets others distribute, remix, tweak, and build upon your work non- commercially, as long as they credit you for the original creation.
Abstract
Introduction: Hospital settings involve several risk factors related to healthcare-associated infections (HAIs). A method that contributes to prevention and control is identification of risks to enable implementation of preventative measures. It is believed that this identification can be accomplished using scales. The present study attempts to develop and validate the face and content of a new scale for measuring HAIs risk in hospitalized adults. Materials and Methods: A methodological study conducted to develop and validate the face and content of the Adult Inpatients Infection Risk Assessment scale, which underwent evaluation by a committee of 23 experts with experience in HAIs. The scale’s validity was tested using the Content Validity Index (CVI). Results: 15 items were retained in the scale, grouped into two dimensions: intrinsic and extrinsic factors. Certain minor adjustments were needed to improve the clarity of some items. Items’ CVIs ranged from 0.83 to 1.0 and the scale’s mean CVI was 0.90. Discussion: The Adult Inpatients Infection Risk Assessment scale can be used as a technology of low cost for the measurement of the risk of infection, which allows the planning of more accurate and organized interventions of the health team targeting at preventive and safe care during hospitalization. Conclusions: The findings supported the face and content validity of the Adult Inpatients Infection Risk Assessment Scale.
Key words: Risk Assessment; Validation Studies; Infection Control; Patient Safety.
Resumen
Introducción: El ambiente hospitalario envuelve diferentes factores de riesgo relacionados con las infecciones asociadas a la atención sanitaria (IAAS). Un método que contribuye a la prevención y control es la identificación de riesgos que permita la implementación de medidas preventivas. Se cree que dicha identificación se puede llevar a cabo mediante el uso de escalas. El presente estudio intenta desarrollar y validar la validez aparente y el contenido de una nueva escala para medir el riesgo de IAAS en adultos hospitalizados. Materiales y Métodos: Se condujo un estudio metodológico para el desarrollo y la validación aparente y de contenido de la escala de Evaluación de Riesgos de Infección en Pacientes Adultos, la cual se sometió a evaluación por parte de un comité compuesto por 23 expertos con experiencia en IAAS. Se probó la validez de la escala mediante el Índice de Validez de Contenido (IVC). Resultados: Se conservaron 15 ítems de la escala y se agruparon en dos dimensiones: factores intrínsecos y factores extrínsecos. Se necesitaron realizar unos pequeños ajustes para mejorar la claridad de algunos ítems. El IVC de los ítems se ubicó entre 0,83 a 1,0 y el IVC medio de la escala fue de 0,90. Discusión: La escala de Evaluación de Riesgo de Infección en Pacientes Adultos se puede emplear como una tecnología de bajo costo en la medición del riesgo de infección, lo que permite la planeación de intervenciones más precisas y organizadas de parte del equipo de atención médica con el ánimo de prevenir y proveer un cuidado seguro durante la hospitalización. Conclusiones: Los hallazgos soportaron la validación aparente y de contenido de la Escala de Evaluación de Riesgo de Infecciones en Pacientes Adultos.
Palabras clave: Medición de Riesgo; Estudios de Validación; Control de Infecciones; Seguridad del Paciente.
Resumo
Introdução: O ambiente hospitalar abrange diferentes fatores de risco relacionados com as infecções associadas ao atendimento sanitário (IAAS). Um método que contribui com a prevenção e o controle é uma identificação de riscos que permita a implantação de medidas preventivas. Considera-se que essa identificação pode ser realizada mediante o uso de escalas. O presente estudo pretende desenvolver e comprovar a validez aparente, bem como o conteúdo de uma nova escala para medir o risco das IAAS em adultos internados. Materiais e Métodos: Foi realizado um estudo metodológico para o desenvolvimento e a validação aparente e de conteúdo da escala de Avaliação de Riscos de Infecção em Pacientes Adultos, que foi analisada por um comitê constituído por 23 especialistas com experiência em IAAS. A validez da escala foi provada mediante o Índice de Validez de Conteúdo (IVC). Resultados: Foram mantidos 15 itens da escala e agrupados em duas dimensões: fatores intrínsecos e fatores extrínsecos. Foi necessário fazer alguns pequenos ajustes para melhorar a clareza de alguns dos itens. O IVC dos itens foi de 0,83 a 1,0 e o IVC médio da escala foi de 0,90. Discussão: A escala de Avaliação de Risco de Infecção em Pacientes Adultos pode ser utilizada como uma tecnologia de baixo custo para a medição do risco de infecção, permitindo o planejamento de cirurgias mais precisas e organizadas por parte da equipe de atendimento médico, com o intuito de prevenir e oferecer um cuidado seguro durante a internação. Conclusões: Os resultados obtidos demonstram a importância da validação aparente e de conteúdo da Escala de Avaliação de Risco de Infecções em Pacientes Adultos.
Palavras chave: Medição de Risco; Estudos de Validação; Controle de Infecções; Segurança do Paciente.
INTRODUCTION
The prevalence of healthcare-associated infections (HAIs) in developing countries, which is elevated compared with rates in developed countries, is a serious epidemiological and safety problem1. HAIs are the most common group of complications seen in hospitalized patients, they impact on morbidity and mortality, and they significantly increase healthcare system costs2,3.
One study reported that the lack of statistical data on infection rates acquired by surveillance groups is caused by time pressures, resource shortages, and a deficiency of specialized knowledge, since few low and medium income countries have national systems for surveillance of HAIs1.
However, epidemiological profiles can vary between institutions or even across different departments within a single healthcare establishment, according to the type of care provided4, because occurrence of HAIs is dependent on many different conditions or factors5.
Many different risk factors for development of HAIs have been studied in the literature, including variables related to the patients, to treatment, and to the environment6. Additionally, one of the strategies used in prevention of HAIs is identification of risk factors, which enables possible conditions that predispose to increased rates or exacerbation of HAIs to be identified.
It is believed that this identification can be accomplished using risk assessment scales7 and that the success of the endeavor is dependent on a constant search for evidence to aid in healthcare professionals’ decision-making processes.
It is important to develop new measurement tools in healthcare8 because the current tendency is to standardize international norms that contribute to improving teaching, research, and clinical practice. However, researchers have found that such assessment scales are only useful and capable of providing results that are scientifically robust when they have good psychometric properties9.
The quality of health services and their relationship with HAIs is a subject that is attracting the world’s attention as a serious public health problem with a direct impact on the safety of healthcare and is one of the greatest challenges to providing good quality healthcare10.
Concerned with this situation, several international organizations have focused efforts on developing standards, guidelines, and preventative measures based on evidence to improve patient safety, particularly in relation to HAIs. Notable examples include the Centers for Disease Control and Prevention6, the European Centre for Disease Control11, and the Joint Commission on Accreditation of Healthcare Organizations12, among others.
Along these lines, it is recommended that strategies be implemented for constant monitoring of healthcare practices, concentrating on costs and quality for improvement of patient safety13. As part of these efforts, scales can have a major influence on decisions about care, treatment and/or interventions, and formulation of healthcare programs and institutional policies8.
It is clear that use of scales is integral to the process of scientific development in healthcare and they are pervasive from training, through research, to practical application of the knowledge acquired and both the process of construction of scales and their utilization are important, complementing each other and leading to benefits through systematic patient assessment14.
For nurses, in general, it is very important to have access to scales of high psychometric quality that can measure the complex and important constructs involved in guiding nursing care15, since nurses’ involvement in assessments underpins integrated and individualized care for patients and their families16. However, before a scale can be administered and considered trustworthy, it must undergo a process of instrument validation consisting of analysis of its psychometric qualities, designed to improve its application and make it more objective17.
One of the steps in validation of scales is face validation, the purpose of which is to establish the scale’s acceptability in the setting in which it will be employed, which is part of establishing that the scale measures what it appears to measure18, based on experts’ subjective, but criteria-based, judgments15. In turn, content validity is considered an essential step because it represents the first of the mechanisms for associating abstract concepts with observable and measurable indicators19,20.
Both face and content validity can be verified using a committee of experts, which is a process that makes it possible to obtain a collective opinion on a given phenomenon, such as best clinical practice21 and is a widely-used technique and one that has a considerable influence on refining items with more potential for the scale19.
Working from these reflections, and bearing in mind that the many stages of constructing a scale include a variety of types of validation, the objective of this study is to validate the face and content of a new scale for measurement of HAIs risk in hospitalized adults, by consensus between experts.
MATERIALS AND METHODS
Study design
This is a methodological study to evaluate the face and content validation of the RAC (Rodríguez-Almeida-Cañon) Adult Infection Risk Scale, which adopted psychometric procedures recommended in the literature22,23. The research project began with construction of a scale based on a systematic review with meta-analysis24. The first version of the scale was established from that review.
Description of the scale
A literature search followed by analysis of 65 studies24 identified 15 risk factors for HAIs in hospitalized adults, which were transformed into the precursor items for the first version of the scale. It was found that these items fitted a two-dimensional model consisting of intrinsic factors and extrinsic factors, with 8 items in the first dimension and 7 items in the second dimension.
Within this conceptualization, the intrinsic factors are physiological characteristics or conditions of the patient at the time of admission and the extrinsic factors are those that involve the treatment provided25. Additionally, operational definitions for each risk factor were formulated, to aid in comprehension of the concept at the time of assessment by the professional, and reduce confounding bias introduced by the probability of misinterpretation. The scale response options comprise categorical scales, varying by risk factor, for example: nutritional status: normal, underweight, or overweight, etc. Likert scales from 0 to 3 points were then adopted to measure the responses according to the questionnaire items.
The healthcare professional who will eventually apply the scale should choose a single option per question. The total score is then calculated by summing the scores for each response, with total a score that can vary from 4 to 35 and is then categorized as low risk: 4-11, intermediate risk: 12-21, or high risk: ≥22, so that hospitalized adults with higher scores are at higher risk of developing a nosocomial infection. These values and scores were obtained according to the reported prevalence of risk factors independently associated with HAIs, previously identified in a systematic review24.
Procedures
Data collection was conducted from June to December in 2016. The number of experts was chosen using a validated model26, employing statistical tests. There is no consensus in the literature on the definition of experts, but the authors identified clinical experience and theoretical knowledge as important.
The sample size was set according the following parameters of interest: 95% confidence level (the Z -value associated with 95% is = 1.96); sampling error of 10% and expected proportion of experts of 95%. Therefore, the calculation was as follows: n = 1.962*0.95* (1-0.95)/0.102 = 18 experts, and to account for possible losses from the sample, a further 20% was added, estimating losses due to non-return of material or due to incomplete material in the absence of repeat contact with the person in question, resulting in a total of 23 experts from different states in Brazil.
In addition to being health professionals with a Master or Ph.D degree, inclusion criteria defined for the experts were a minimum of two of the following criteria: minimum 2 years’ experience in adult inpatient wards; minimum of 1 year’s experience on a Nosocomial Infection Control Commission, and experience with research related to HAIs and/or publications about HAIs.
Experts were recruited using a snowball-effect sampling strategy, relying on nomination of some researchers, selected participants, via the Lattes platform (a standardized virtual system that combines databases of curricula vitae, research teams, and institutions, in a single information system covering Science and Technology in Brazil), which is administered by the Conselho Nacional de Desenvolvimento Científico (CNPq – National Council for Scientific and Technological Development).
Potential participants were identified by searching for the keywords: construction/validation of scales or instruments and/or infection prevention and control. This type of sample strategy is one option for exchanging knowledge within the scientific community, attempting to capture the most up-to-date knowledge from different settings and experiences, in order to acquire significant results from the many different contexts.
After identification of potential expert participants, an invitation letter was sent by e-mail containing all of the relevant information and indications related to the study. A time limit of 1 month was set for recipients to return this material, and experts were also excluded if they returned incomplete forms and failed to correct omissions even after the material had been returned to them and they had been contacted by the researcher.
The first version of the scale was evaluated in its entirety to determine whether its scope was comprehensive and its items were assessed individually for clarity and relevance. Clarity assessment focused on the wording of items, i.e. on whether the concept was understandable and whether it was evidently congruent with what it was intended to measure. In the relevance assessment, the experts indicated whether the items truly reflect the concepts involved and whether these are relevant and achieve the objectives27.
The validity of content of the scale was tested using the Content Validity Index (CVI). The CVI measures the proportion of experts in agreement on features of the questionnaire and its items, initially each item is analyzed individually and then the instrument is assessed as a whole. The method employs a Likert scale with four levels, as follows: 1-not relevant; 2-somewhat relevant; 3-quite relevant; 4-highly relevant. There is no midpoint to neutral score to avoid ambivalence when interpreting the analyses.
According to experts recommendations19 about development of health measurement scales, items scored as “2” or “3” were revised and items scored as “1” were eliminated, in accordance with the clarity and relevance of each item, in addition to work in the literature doing likewise20,22,23.
Statistical analysis
The CVI for each item (I-CVIs) was calculated using the following formula: CVI= (number of responses scoring 3 or 4) / (total number of responses)20. The entire set of items on the scale was evaluated using the mean of the I-CVIs calculated separately and dividing by the number of items considered in the assessment. The criterion for acceptance between the experts for assessments of the items individually was set at a level of agreement exceeding 0.80, as recommended in the literature22,28.
The CVI for the whole scale was calculated by summing the CVIs calculated separately and dividing by the number of items considered in the assessment, with a cutoff of 0.80 recommended for new instruments8,28, which was adopted for this study.
The data collected were organized using an electronic spreadsheet in Excel and exported to Stata 11.1® for statistical analyses.
Ethical considerations
This study was approved by the Ethics Committee under protocol number 160231 and all experts who were invited to participate accepted signed a free and informed consent form electronically.
RESULTS
Description of the experts
A total of 23 health professionals accepted and completed the data collection questionnaires. Table 1 shows a summary of the profile of these professionals. These results show that there was a predominance of females among the study participants. The majority of young adults is likely to be linked with working age, and the majority of the sample work in the Southeast region, the region with the largest population in Brazil.
Table 1. Demographic characteristics of expert practitioners. Porto Alegre, RS, Brazil, 2017
*Mean ± standard deviation; † n (%).
With relation to academic qualifications, a large proportion of the professionals recruited reported several postgraduate courses, such as: postgraduate certificates, Masters and Ph.D degrees, some completed and others in progress, as shown in Table 1.
Length of experience teaching, in practice, and in research ranged from 3 to 36 years, with a mean of 18.9 years. With regard to job roles, the great majority of the experts worked in several settings, ranging from health services to teaching and research.
Face and content validation
Consensus was achieved with a single round, with analyses for validation of face, which deals with the clarity of each risk factor and their respective operational definitions, and those for response categories and score gradations achieving a mean level of agreement of 0.88. The experts agreed with the dimensions, but recommended small modifications to some items, and those suggestions with support in the literature were accepted and acted on.
Only six experts considered that it was necessary to make some changes in the items, and the item gender remained unchanged because there were no suggestions for modification. Table 2 shows a summary of the experts’ opinions. The dimension of intrinsic factors, which comprises items one to eight (Table 2), had good indices of acceptance by the experts, but certain suggestions were made with relation to some of the questions in order to improve understanding of the content expressed by the scale. Items 3, 6, 7 and 8 were all reformulated to improve understanding, and additional stratification of response categories was performed for items 2, 3, 4, 6 and 8, while the operational definitions for items 4 and 7 were refined to make their concepts clearer. The scores for these items ranged from 0.83 to 1.0, supporting the content validity of the items in the scale.
The dimension of extrinsic factors comprises items nine to fifteen (Table 2) and no changes were suggested to the items themselves, but it was necessary to make some alterations, increasing the number of response categories for items 10, 12, 13, 14 and 15 and refining the operational definitions for items 9, 11, 13, 14 and 15, as suggested by the experts, although the set of items in this dimension nevertheless had agreement scores ranging from 0.83 to 1.0.
Table 2. Experts’ assessments, by Adult Inpatients Infection Risk Assessment scale components. Porto Alegre, RS, Brazil, 2017
NR: No recommendations; R: Recommendations.
Regarding to the relevance of the items, the experts considered that all of them were congruent with the scale construct, with a mean agreement of 0.92. The judges agreed that the items are relevant and applicable to clinical practice (Table 3).
Table 3. Analysis of agreement of the Adult Inpatients Infection Risk Assessment scale. Porto Alegre, RS, Brazil, 2017
CVI: Content Validity Index; S-CVI: Scale-Content Validity Index.
The result of the calculation for the global CVI of this version of the scale was 0.90, indicating that it is representative of the content to be studied for measurement of infection risk in adults, as shown in Table 3.
DISCUSSION
This study developed and evaluated the validity of face and content of the RAC (Rodríguez-Almeida-Cañon) Adult Infection Risk Scale, a new instrument for measuring HAIs risk in hospitalized adults.
No scales were found in the literature that assess the risk factors for occurrence of HAIs in adults and so the findings of this study cannot be correlated with those for other similar scales, since this is an unprecedented study that seeks solutions for problems that involve the profession and human health, filling a gap that exists both in the field of professional practice and in teaching and research and contributing to draw training and professional practice closer together.
According to the World Health Organization, the risk factors for HAIs vary according to the type of healthcare center and the service through which the patient is admitted and are partially different in developing countries. Further research is very much needed to identify models that can be used to make estimates that are more predictive of HAIs1.
The healthcare team, and especially the nursing staff, have moments of essential contact with the patients that enable them to monitor a range of changes. The scale developed in this study can be used as part of screening for risk factors and, consequently of infections.
The results of this study are a supplement to the stages of scale validation, describing tests applied to the scale to test its face and content validity, since analysis by a panel of experts on a given area of knowledge contributes to improving and legitimizing a new instrument under development29.
Therefore, these types of validations are an important step in the effort to provide professionals in clinical practice with a tool that aids in consolidation of a culture of safety and which can have results in safer and higher quality nursing care.
The findings indicate a scale for measurement of infection risk in hospitalized adults comprising two dimensions: intrinsic factors and extrinsic factors. These dimensions are made up of eight and seven items, respectively, with satisfactory face validity indices, as shown by the mean CVI of 0.90, a value recognized as adequate in the literature8.
The overall CVI score was similar for the group of items in each dimension, which occurred because the expert judges awarded agreement values of 3 (agree partially) and 4 (totally agree) to different items that were in the same dimension. It is worth noting that both dimensions have a similar number of items and that the final result showed that they were congruent with each other.
It should be pointed out that the category of extrinsic factors included an infrastructure factor, which refers to the collection of resources that are available to the health professional, including number of staff, equipment, and physical area30. It was decided not to include this item because it was not covered in the literature reviewed and also because the indicators that it contains, such as contaminated air conditioning, contaminated water system, physical design of the department, and others, are central concerns of the managers of healthcare organizations and are not the direct or sole responsibility of the health professionals.
There is no doubt of the importance of this subject, bearing in mind the range of possible complications, but it cannot be concluded that the dimensions and factors identified in this study encompass all that is known on the subject, although they are the elements mentioned with greatest frequency in the literature.
One limitation of this study is that since it was conducted exclusively in Brazil it may not have covered different knowledge about and experience of the subject that exists in other populations. It is suggested that future confirmatory studies should be undertaken in other places in order that cultural differences with relation to the scale’s dimensions and items can be discussed.
Another limitation is related to the fact that the sample was entirely made up of nurses, even though other professionals from the multidisciplinary team were invited to take part. It is therefore recommended that validation by consensus be conducted in other categories of the healthcare professions, since the care provided to people in hospital involves a multidisciplinary team.
CONCLUSIONS
The RAC Adult Infection Risk Scale was considered valid in terms of face and content, bearing in mind the careful and detailed process of assessment of the scale and the experts’ suggestions for improvements to make it better suited to its target public, since this scale is designed to enable identification and measurement of infection risk in hospitalized adults, as an important measure for progress in actions to prevent HAIs. The RAC Adult Infection Risk Scale can be used as a technology of low cost for the measurement of the risk of infection in adult patients, which allows the planning of more accurate and organized interventions of the health team targeting at preventive and safe care during hospitalization. In addition, this tool may favor risk management, patient safety and hospital infection control to achieve best practices and, therefore, contribute to indicators of quality of care31.
Another important step towards this instrument can be considered valid and reliable is to evaluate its reliability and construct validation, which the authors have already accepted as their next study. The intention is that the final result will be a scale that is a practical instrument of use to health professionals for identification of patients at increased risk, enabling them to plan and implement interventions to improve patient safety and reduce the risk of infection among hospitalized adults.
Conflict of interest: The authors declare no conflict of interest.
REFERENCES