Publish in this journal
Journal Information
Vol. 9. Issue 4.
Pages 206-215 (July - August 2013)
Visits
...
Vol. 9. Issue 4.
Pages 206-215 (July - August 2013)
Original Article
DOI: 10.1016/j.reumae.2013.06.001
Full text access
Quality of Care Standards for Nursing Clinics in Rheumatology
Estándares de calidad asistencial para las consultas de enfermería en reumatología
Visits
...
Santiago Muñoz Fernándeza, Pablo Lázaro y De Mercadob,??
Corresponding author
plazaro@taiss.com

Corresponding author.
, Javier Alegre Lópezc, Raquel Almodóvar Gonzálezd, Alberto Alonso Ruize, Francisco Javier Ballina Garcíaf, Ana María Bilbao Cantarerog, Mercedes Cabañas Sáenzb, Rosario García-Vicuñah, Mauricio Mínguez Vegai, Isabel Padró Blanchj, José Andrés Román Ivorrak, Encarnación Roncal Marquetal
a Servicio de Reumatología, Hospital Infanta Sofía, San Sebastián de los Reyes, Madrid, Spain
b Técnicas Avanzadas de Investigación en Servicios de Salud (TAISS), Madrid, Spain
c Sección de Reumatología, Hospital Universitario de Burgos, Burgos, Spain
d Servicio de Reumatología, Hospital Universitario Fundación Alcorcón, Alcorcón, Madrid, Spain
e Servicio de Reumatología, Hospital Universitario de Cruces, Barakaldo, Vizcaya, Spain
f Servicio de Reumatología, Hospital Universitario Central de Asturias, Oviedo, Spain
g Servicio de Reumatología, Hospital Universitario de Gran Canaria Dr. Negrín, Las Palmas de Gran Canaria, Spain
h Servicio de Reumatología, Hospital Universitario de la Princesa, Madrid, Spain
i Servicio de Reumatología, Hospital Clínico Universitario San Juan de Alicante, Alicante, Spain
j Servicio de Reumatología, Hospital del Mar, Barcelona, Spain
k Servicio de Reumatología, Hospital Universitario y Politécnico La Fe, Valencia, Spain
l Servicio de Reumatología, Hospital Universitario La Paz, Madrid, Spain
Article information
Abstract
Full Text
Bibliography
Download PDF
Statistics
Figures (1)
Tables (7)
Table 1. Standards on NCR Initial Considerations (Scale of 1–9).
Table 2. Structure Standards.
Table 3. Processing Standards.
Table 4. Treatment and Follow Up Standards (Scale of 1–9).
Table 5. Health Education Standards (Scale of 1–9).
Table 6. Training Standards/Research of the NCR Nursing Staff.
Table 7. Quality Care Standards (Scale of 1–9).
Show moreShow less
Abstract
Background

Nursing clinics in rheumatology (NCR) are organizational models in the field of nursing care. There are various NCR models, but there is no consensus on its operational definition. Our objective is to develop quality standards to define and characterize a NCR.

Method

Two-round Delphi method. The panel consisted of 67 experts: rheumatologists and nurses of the nursing working group of the Spanish Society of Rheumatology (SSR). The Delphi questionnaire was developed after a literature and experience review from previous SSR projects. The questionnaire consists of 7 sections: general considerations, standards of structure, process, treatment and monitoring, health education, training and research and quality of care. Each item was scored from 1 (least important) to 9 (most important) or by assigning a number (e.g. waiting days). The degree of agreement among the experts was categorized according to the coefficient of variation (CoV) between very high (CoV≤25%) and very low (CoV>100%).

Results

The second round questionnaire (182 items) was answered by 46 panelists (34 rheumatologists and 12 nurses). A very important agreement was reached on the general standards of structure, process, treatment and monitoring, health education and quality of care. Less agreement was observed on standards related to training time, number of recommended nurses’ research projects and publications.

Conclusion

The standards developed in this study would be useful for establishing desirable quality standards of structure and process, and criteria for clinical work, research and teaching that can be used to develop and evaluate the NCRs.

Keywords:
Rheumatology
Nursing
Quality of care
Resumen
Introducción

Las consultas de enfermería en reumatología (CER) son modelos organizativos asistenciales en el ámbito de competencias de enfermería. Hay diversos modelos de CER, pero no existe una definición operacional. El objetivo del proyecto es elaborar estándares de calidad para definir y caracterizar una CER.

Método

Estudio Delphi a dos rondas. El panel estuvo constituido por 67 expertos: reumatólogos y enfermeras del grupo de trabajo de enfermería de la Sociedad Española de Reumatología (SER). El cuestionario se elaboró tras revisión bibliográfica y experiencias de proyectos previos de la SER. El cuestionario consta de 7 apartados: consideraciones generales, estándares de estructura, de proceso, de tratamiento y seguimiento, educación sanitaria, formación e investigación y calidad asistencial. Cada ítem se puntuó de 1 (menos importante) a 9 (más importante) o mediante una cifra. El grado de acuerdo de los expertos se categorizó según el coeficiente de variación (CV) entre muy alto (CV25%) y muy bajo (CV>100%).

Resultados

El cuestionario de la segunda ronda (182 ítems) fue respondido por 46 panelistas (34 reumatólogos y 12 enfermeras). Se obtuvo un grado de acuerdo muy importante en los estándares generales, de estructura, de proceso, de tratamiento y seguimiento, educación sanitaria y calidad asistencial. Se encontró menor acuerdo en los estándares relacionados con el tiempo para formación, número de proyectos de investigación propios de enfermería y de publicaciones recomendables.

Conclusión

Los estándares desarrollados en este estudio permitirían establecer mínimos deseables de calidad de estructura, proceso, labor asistencial, investigadora y docente que se pueden utilizar para desarrollar y evaluar CERs.

Palabras clave:
Reumatología
Enfermería
Calidad asistencial
Full Text
Introduction

Nursing consultations in rheumatology (NCR) are organizational patient-centered models of care in the field of rheumatology nursing skills. NCRs have been developed in recent years, but their implementation in Spain is still piecemeal. There are various models of NCR but there is no operational definition. Although there is little scientific evidence, one might assume that the NCR optimize each professional skill and contribute to improve the efficiency and quality of care by decreasing costs, especially medical, improving outcomes and quality of life effectiveness and perceived patient outcomes. Given the difficulty to find a definition of NCR in the existing literature and its potential relevance to quality of care, we designed this research project with the overall aim of contributing to the operational definition of an NCR and advance the exploration of those aspects of care, teaching and research of the NCR that could help measure quality.

To address this, Delphi rounds were considered the most appropriate method by experts in the field. The Delphi method was initially developed by the Rand Corporation in the United States to make provisions on military matters after World War II.It has subsequently been used in various areas, both as a predictive tool or as a prioritization strategy formulation.1,2 The objective of this technique is to improve judgments and forecasts on issues where there is uncertainty through expert opinion. It is based on the idea that the degree of consensus reached among a group of experts who are familiar with the topics under study gives a very rough idea of the actual size of these items.3 This technique is very useful to know a problem when it cannot be addressed with experimental approaches. The Delphi technique has been used in a variety of health-related studies. For example, it has been used to make predictions,4–7 prioritize design strategies,8–12 and especially to develop standards of quality of care and appropriate use of medical technology.13–20 This method was designed for this project, which aims to develop quality criteria and standards for defining and characterizing the NCR.

Methods

The project was an initiative of the Nursing Task Force of the Spanish Society of Rheumatology (GTESER), carried out by a scientific committee (CC) consisting of 10 rheumatologists and 3 nurses. In addition, two experienced researchers participated in the methodology design. The project development schedule is shown in Fig. 1.

Fig. 1.

Overview of the development of the project.

(0.24MB).

After defining the objectives of the project, we developed a specific questionnaire addressed to experts in the field, to be applied by Delphi methodology. Basically, the Delphi method is carried out through a series of rounds in each of which questionnaires are distributed with specific questions made to a panel of experts who score their opinions on a quantitative scale. In the second round, and the following if any, experts receive “feedback” on the scores of the rest of the group (in the form of the mean score for each question) and have the opportunity to review and change their score if they wish. The process ends when it is considered that it has reached a certain level of consensus or that the information obtained is sufficient. This method ensures the anonymity of responses for each respondent (each responder knows what he has responded, how many have responded to what question, but does not know who has answered what). The anonymity of responses prevents the group from being dominated by powerful or vehement individuals, which sometimes are characteristics not necessarily associated to knowledge. In addition, one of the strengths of the Delphi method is that it identifies the degree of agreement, but does not force consensus, but rather weights the views expressed by the number of respondents and the intensity of their responses.

In this project, we used the Delphi method for 2 rounds, with the distribution of questionnaires via email. To prepare the questionnaire for the first round, we constructed a list of features that could describe an NCR or standards that should be met. To do this, we started from the results and experiences of several SER projects: SERAP,21 ESPERANZA,22 EMAR23 from clinical practice guidelines of the SER24,25 and publication of standards of quality of care in rheumatology.15 The list of features or standards are grouped into the following sections: (1) initial considerations, (2) structural standards, (3) process standards, (4) treatment and monitoring, (5) health education; (6) training/research by the nursing staff of the NCR, and (7) quality of care.

This produced a first draft of the questionnaire sent to members of the CC to verify the relevance of the items, that the statement was not ambiguous and that the way to answer them was appropriate. With the contributions of the CC, the final questionnaire was designed during the first round consisting of 159 items grouped into 7 sections above, and within each section, grouped according to the type of response.

To respond to the questionnaire, a panel of 13 members of the CC and 54 experts (rheumatologists and nurses) was formed.

To rate indicators, the Delphi method was used for 2 rounds.8 Each panelist received by email, in October 2010, an Excel workbook with 3 sheets: (1) introduction and explanation of the project, (2) instructions on how to complete the questionnaire, and (3) the questionnaire of the first round. They set a deadline of 15 days in order to respond to it.

For each of the indicators, the panelists rated a number, depending on the chosen indicator. Sometimes they had to rate on a scale of 1–9 (1=not important and 9=very important), according to the importance they attached to the statement regarding this item. At other times, they were asked to quantify the value of an indicator (e.g. the maximum number of people per nurse at the NCR). To develop the questionnaire for the second round, a descriptive analysis was performed by calculating the mean and standard deviation (SD) of the responses of each indicator. This analysis produced a specific sheet in Excel format, for each of the panelists. This sheet included, for each indicator, the mean, SD and the answer the panelist gave in the first round. Therefore, there were as many different questionnaires as panelists responded to the first round. In addition, with the suggestions given by the panelists in the first round, the final questionnaire was prepared for the second round.

On December 17, 2010, we emailed the questionnaire of the second round to the panelists who responded to the first. This questionnaire consisted of 182 items. Each group included scores (mean and SD) and the actual score of the first round. Each panelist was asked to score points over each item and could maintain the same score of the first round, or modify it.13–15 The panelists were given 15 days to respond.

After receiving the scores, we performed statistical analysis of the second round of ratings. To categorize the variability in responses and its opposite (the agreement), we used the coefficient of variation (CV) categorized with the following breakpoints: CV less than or equal to 25%=very low variability (high agreement), CV greater than 25% and less than or equal to 50%=low variability (high agreement), CV greater than 50% and less than or equal to 75%=average variability (medium agreement), CV greater than 75% and less than or equal to 100%=high variability (low agreement), and greater than 100% CV=very high variability (very low agreement).

Due to the circumstances of this study, which only reflects the views of health professionals and patients, it was not submitted for review by the ethics committee.

Results

Of the 67 panelists initially contacted for the first round of the study, responses were received from 49 (73%): 37 rheumatologists (71%) and 12 nurses (92%).

Of these, 46 panelists (34 rheumatologists and 12 nurses) from 12 Spanish autonomous communities responded to the second round (93% of respondents from the first round).

In Tables 1–7 specifies the items distributed in each domain, with their average score, SD, median, minimum, maximum, CV and degree of agreement.

Table 1.

Standards on NCR Initial Considerations (Scale of 1–9).

Standard  Mean  SD  Median  Lo  Hi  CV, %  Agreement 
An NCR is needed in a department of Rheumatology  8.4  0.9  9.0  6.0  9.0  10.2  Very high 
NCR help improve the quality of care in Rheumatology  8.4  0.7  8.0  7.0  9.0  8.1  Very high 
NCR help improve waiting times in Rheumatology  7.8  1.0  8.0  6.0  9.0  12.5  Very high 
NCR improve the use of resources in the department (cost/effective)  7.9  1.1  8.0  5.0  9.0  13.7  Very high 
NCR reduce costs in the department  7.3  1.4  7.0  4.0  9.0  18.7  Very high 
NCR improve clinical outcomes in patients  7.8  1.0  8.0  5.0  9.0  13.2  Very high 
NCR improve patient satisfaction  8.2  0.8  8.0  6.0  9.0  9.9  Very high 
Table 2.

Structure Standards.

2.1. Equipment/Materials in an NCRC (Scale of 1–9)
Standard  Mean  SD  Median  Lo  Hi  CV, %  Agreement 
Office  8.4  0.8  9.0  7.0  9.0  9.2  Very high 
Computer  8.4  0.7  8.5  6.0  9.0  8.9)  Very high 
Phone  8.7  0.5  9.0  7.0  9.0  6.0  Very high 
Specialized software  7.4  1.1  7.5  5.0  9.0  14.9  Very high 
Databases  7.6  1.1  8.0  5.0  9.0  14.0  Very high 
Educational material for patients  8.7  0.5  9.0  7.0  9.0  5.9  Very high 
Sphingomanometer  8.6  0.6  9.0  7.0  9.0  7.2  Very high 
Scale to weigh and measure patients  8.6  0.7  9.0  6.0  9.0  8.0  Very high 
Crash cart  5.4  1.9  6.0  1.0  9.0  35.2  High 
The location of the NCR must be within the Rheumatology department, next to that of the rheumatologists  8.7  0.5  9.0  7.0  9.0  6.2  Very high 
2.2. Personal/Competency in an NCR (Scale of 1–9)
Standard  Mean  SD  Median  Lo  Hi  CV, %  Agreement 
The NCR nursing staff should be trained to perform:
Visual analog scale  8.8  0.4  9.0  8.0  9.0  4.9  Very high 
Functional capacity Questionnaire: HAQ  8.8  0.4  9.0  8.0  9.0  4.7  Very high 
Painful joint count  7.9  1.3  8.0  4.0  9.0  15.8  Very high 
Swollen joint count  7.9  1.3  8.0  4.0  9.0  16.5  Very high 
RADAI  7.5  1.5  8.0  3.0  9.0  19.9  Very high 
BASFI  8.3  1.0  9.0  4.0  9.0  11.6  Very high 
BASDAI  8.5  0.7  9.0  7.0  9.0  7.7  Very high 
DAS 28  8.0  1.1  8.0  5.0  9.0  13.4  Very high 
SDAI  7.7  1.3  8.0  4.0  9.0  16.5  Very high 
CDAI  7.5  1.3  7.0  4.0  9.0  17.2  Very high 
BASMI  7.6  1.5  8.0  4.0  9.0  19.4  Very high 
Generic health questionnaires (e.g. EuroQoL, SF-12)  8.2  0.9  8.0  6.0  9.0  10.7  Very high 
2.3. Population Needed to Establish an NCR (Continuous Variable)
Standard  Mean  SD  Median  Lo  Hi  CV, %  Agreement 
Maximum number of people per NCR in a health area  212614  76177  200000  50000  400000  35.8  High 
Maximum number of people per NCR nurse  165833  85675  150000  20000  400000  51.7  Intermediate 
Maximum number of rheumatologists who share an NCR nurse  3.9  1.0  4.0  2.0  6.0  24.8  Very high 
Table 3.

Processing Standards.

3.1. NCR Nurse (Scale of 1–9)
Standard  Mean  SD  Median  Lo  Hi  CV, %  Agreement 
Apply protocols for the management of rheumatology patients  8.1  1.0  8.0  5.0  9.0  12.0  Very high 
Established communication circuits between the NCR nursing staff with the rheumatologist, physiotherapist, occupational therapist, psychologist, social worker  8.0  0.9  8.0  5.0  9.0  11.1  Very high 
Participate in regular meetings with other health professionals (rheumatologist, physiotherapist, occupational therapist, psychologist, social worker)  7.9  1.2  8.0  4.0  9.0  14.9  Very high 
Interact with PC physicians  6.8  1.4  7.0  4.0  9.0  21.2  Very high 
Interact with PC nurses  7.5  1.4  8.0  4.0  9.0  18.3  Very high 
Participates with other departments and a multidisciplinary service team  7.4  1.4  8.0  3.0  9.0  19.2  Very high 
Participates in interdepartmental multidisciplinary teams with PC  6.8  1.5  7.0  3.0  9.0  21.7  Very high 
Keeps track of the patients treated  8.4  0.9  9.0  5.0  9.0  10.5  Very high 
Has own agenda  8.5  1.0  9.0  4.0  9.0  11.6  Very high 
3.2. Availability of NCR Appointments (Continuous Variable)
Standard  Mean  SD  Median  Lo  Hi  CV, %  Agreement 
The availability of appointments in the NCR should be immediate (days/week)  3.0  1.0  10.0  44.9  High 
3.3. Essential Assistance Nursing Activities in an NCR (Scale of 1–9)
Standard  Mean  SD  Median  Lo  Hi  CV, %  Agreement 
Overall health assessment by the patient (VAS)  8.5  0.8  9.0  5.0  9.0  9.5  Very high 
VAS pain  8.5  0.8  9.0  5.0  9.0  9.8  Very high 
Rate the quality of life of patients through questionnaires (e.g. Euroquol 5 D, SF-12)  8.2  1.0  8.0  4.0  9.0  12.8  Very high 
Collaboration with doctor in: arthrocentesis/infiltration  8.2  1.2  9.0  3.0  9.0  14.9  Very high 
Resolution of unsolicited consultations  8.0  1.1  8.0  5.0  9.0  13.2  Very high 
Emotional support to patients and families  7.9  1.2  8.0  4.0  9.0  14.7  Very high 
Advice regarding nutrition, exercise  8.3  0.9  8.5  5.0  9.0  10.4  Very high 
Telephone Support  8.6  0.5  9.0  7.0  9.0  6.2  Very high 
Registration of nursing performance  8.5  0.8  9.0  5.0  9.0  9.5  Very high 
Health education to patients and families  8.7  0.6  9.0  7.0  9.0  6.4  Very high 
Patient appointment reminder on the previous day  5.3  1.8  5.0  1.0  9.0  33.1  High 
Mantoux and Booster during the NCR query itself  7.4  1.7  8.0  1.0  9.0  22.4  Very high 
Participation in additional examinations (muscle biopsy of subcutaneous fat)  7.4  1.3  7.5  5.0  9.0  17.2  Very high 
Administering alidated patient self-management and self-efficacy questionnaires and initiating contact with the NCR and periodical follow ups (e.g. Self-efficacy scale for arthritis)  7.7  1.0  8.0  4.0  9.0  13.5  Very high 
For patients with rheumatoid arthritis
Assessment of disease activity: RADAI  7.6  1.3  8.0  3.0  9.0  16.6  Very high 
Collaboration with doctor in the assessment of disease activity: DAS28  7.9  1.1  8.0  4.0  9.0  14.2  Very high 
Assessment of disease activity with DAS28  7.8  1.3  8.0  4.0  9.0  17.3  Very high 
Assessment of disease activity with the SDAI  7.5  1.3  8.0  4.0  9.0  17.4  Very high 
Assessment of disease activity with CDAI  7.3  1.4  7.0  4.0  9.0  18.7  Very high 
functional capacity evaluation: HAQ  8.3  0.9  9.0  6.0  9.0  11.1  Very high 
number of painful joints  7.7  1.4  8.0  3.0  9.0  18.5  Very high 
number of swollen joints  7.5  1.5  8.0  3.0  9.0  20.3  Very high 
For patients with ankylosing spondylitis
Assessment of disease activity: BASDAI  8.4  0.8  9.0  6.0  9.0  9.5  Very high 
functional capacity evaluation: BASFI  8.3  0.7  8.0  6.0  9.0  8.8  Very high 
Collaboration with the performance of BASMI with the rheumatologist  7.8  1.2  8.0  5.0  9.0  15.1  Very high 
For patients with osteoarthritis
Individual information  7.5  1.3  8.0  5.0  9.0  17.4  Very high 
Information in patient groups  7.4  1.4  8.0  1.0  9.0  19.0  Very high 
Information on physical measures (hot-cold)  7.7  1.1  8.0  5.0  9.0  14.8  Very high 
For patients with chronic low back pain
Individual information  7.7  1.3  8.0  5.0  9.0  16.9  Very high 
Information in patient groups  7.3  1.5  8.0  1.0  9.0  20.4  Very high 
Recommendations and individualized advice on exercise and posture  7.7  1.3  8.0  5.0  9.0  17.2  Very high 
Recommendations to groups of patients on exercise and posture  7.4  1.4  8.0  1.0  9.0  18.5  Very high 
For patients with osteoporosis
Individual information  7.7  1.2  8.0  5.0  9.0  15.8  Very high 
Information in patient groups  7.5  1.4  8.0  1.0  9.0  18.6  Very high 
Recommendations for individualized home adequacy to prevent falls  7.7  1.2  8.0  5.0  9.0  15.8  Very high 
Recommendations to patient groups on adequacy of home conditions to prevent falls  7.5  1.3  8.0  1.0  9.0  17.9  Very high 
Completing the FRAX questionnaire  7.1  1.6  7.5  1.0  9.0  23.2  Very high 
For patients with lupus
Individual information  7.6  1.3  8.0  4.0  9.0  16.7  Very high 
Information in patient groups  6.8  1.7  7.0  1.0  9.0  24.6  Very high 
Completing the SLEDAI questionnaire or others  6.9  1.4  7.0  3.0  9.0  20.3  Very high 
3.4. Recording the Activity of the NCR Nurse (Scale of 1–9)
Standard  Mean  DESD  Median  Lo  Hi  CV, %  Agreement 
Assessing the patient (e.g. Annotations of relevant variables, VAS)  8.5  0.8  9.0  4.0  9.0  9.8  Very high 
Nursing diagnosis  7.6  1.5  8.0  3.0  9.0  19.1  Very high 
Activity planning  8.2  1.1  8.0  3.0  9.0  12.9  Very high 
Running the activity  8.2  1.0  8.0  3.0  9.0  12.6  Very high 
Evaluation of the activity  8.1  1.1  8.0  3.0  9.0  13.1  Very high 
Drug sheet  8.3  1.0  9.0  3.0  9.0  12.2  Very high 
Quality of information on patient history regarding:
Objectivity of information (without prejudice of value or personal opinions)  8.2  1.0  8.0  4.0  9.0  12.1  Very high 
Readability  8.3  0.8  8.0  6.0  9.0  9.7  Very high 
The facts should be recorded in a clear and concise manner  8.5  0.7  9.0  6.0  9.0  8.5  Very high 
Date and time should be noted  8.6  0.7  9.0  6.0  9.0  7.6  Very high 
Appropriate use of acronyms  8.3  0.8  8.0  6.0  9.0  9.1  Very high 
Ensure confidentiality of patient data  8.8  0.6  9.0  6.0  9.0  6.5  Very high 
Complete information on the patient history and in:
NCR treatment protocols  8.1  1.4  8.5  2.0  9.0  17.7  Very high 
Reports and statistics  7.7  1.4  8.0  2.0  9.0  18.4  Very high 
Documentats regarding administrative assistance related to clinical processes  7.9  1.4  8.0  2.0  9.0  18.1  Very high 
Nurse signature of all records  8.1  1.5  8.5  2.0  9.0  18.6  Very high 
3.5. Standards of Medical Visitation Time (Continuous Variable)
Standard  Mean  SD  Median  Lo  Hi  CV, %  Agreement 
Minimum time duration of the first visit to the NCR (min)  25  24  15  45  25.5  High 
Minimum time duration of the second visit to the NCR (min)  15  15  10  20  21.2  Very high 
Minimum time duration of the successive visits to the NCR (min)  13  13  20  23.2  Very high 
Maximum waiting time of the patient in the waiting room after the time of appointment before being seen by nurses (min)  24  25  10  60  34.8  High 
Minimum time of a family information session (min)  15  15  30  36.3  High 
Minimum time of a session of family health education (min)  18  18  30  34.1  High 
Maximum hours of nursing consultation by a nurse/day  5.3  1.7  5.0  3.0  15.0  31.3  High 
Maximum waiting time (days) for the first ordinary visit to the rheumatologist  25  10  25  10  60  37.8  High 
Maximum time (days) to wait for the preferential first visit to the rheumatologist  10  30  62.5  Intermediate 
Maximum waiting time (days) between the first visit to the NCR and the second visit  35  15  30  10  100  43.9  High 
Maximum waiting time (days) for successive visits  63  25  60  15  120  39.0  High 
In the NCR appointments should be programmed with a minimum of (months)  6.8  12.4  2.5  0.3  70.0  181.9  Very low 
Percentage of weekly time nurse must be dedicated to assistance in the NCR  61  15  60  20  90  23.9  Very high 
Number of patients per day/nurse  14  15  20  19.4  Very high 
Average desirable ratio of successive nursing consultations per each new visit (over a period of one year)  4.2  1.6  4.0  2.0  10.0  38.9  High 
Desirable frequency of multidisciplinary team and NCR staff (sessions/month)  3.5  1.2  4.0  1.0  8.0  35.0  High 
Table 4.

Treatment and Follow Up Standards (Scale of 1–9).

Standard  Mean  SD  Median  Lo  Hi  CV, %  Agreement 
Continuous monitoring of patients according to drug protocol  8.3  0.9  9.0  5.0  9.0  11.1  Very high 
Medication Management  8.3  0.8  8.0  6.0  9.0  9.1  Very high 
Analytical control of protocolized treatments  8.4  0.9  9.0  4.0  9.0  11.1  Very high 
Pharmacovigilance registration and monitoring of rheumatic disease related drugs  8.1  1.3  8.0  2.0  9.0  16.2  Very high 
Provide and implement the monitoring protocol for biological and nonbiological disease modulating drugs  8.5  0.7  9.0  6.0  9.0  8.5  Very high 
Patients with rheumatic diseases should have access to the NCR, where a nurse will be responsible for coordinating the multidisciplinary care required by the patient  7.5  1.7  8.0  2.0  9.0  22.7  Very high 
Strategies to ensure the safety (adverse effects, interactions, incompatibilities, allergies)  8.4  0.9  9.0  4.0  9.0  10.8  Very high 
Ask the patient about their adcomplianceherence to treatment on each visit  8.5  0.8  9.0  6.0  9.0  9.2  Very high 
Table 5.

Health Education Standards (Scale of 1–9).

Standard  Mean  SD  Median  Lo  Hi  CV, %  Agreement 
Disease information  8.3  1.1  9.0  3.0  9.0  12.7  Very high 
Information on diagnostic tests related to the disease  7.5  1.4  8.0  3.0  9.0  18.5  Very high 
Drug Information  7.9  1.3  8.0  3.0  9.0  16.0  Very high 
Training in self-medication  8.9)  0.3  9.0  8.0  9.0  3.8  Very high 
Joint ergonomic measures  8.5  0.7  9.0  7.0  9.0  8.1  Very high 
Help the patients to actively participate in the management of their disease  8.7  0.6  9.0  7.0  9.0  6.9  Very high 
Health education for self-care  8.7  0.6  9.0  7.0  9.0  6.8  Very high 
Information to contact patient associations  7.6  1.3  8.0  2.0  9.0  17.8  Very high 
Explain precautions that the patient should take into account (e.g. fever, skin or mucosal ulcers)  8.7  0.7  9.0  6.0  9.0  8.2  Very high 
Teaching patients treated to properly administer subcutaneous drugs through injection systems (“pens”, etc.)  9.0  0.2  9.0  8.0  9.0  2.3  Very high 
Provide immunization schedule  8.0  1.0  8.0  5.0  9.0  12.6  Very high 
Verbal Education  8.3  1.2  9.0  2.0  9.0  14.1  Very high 
Written education  8.3  0.8  8.0  6.0  9.0  9.1  Very high 
Audiovisual information  7.0  1.4  7.0  2.0  9.0  19.9  Very high 
Nurse training in relaxation techniques to apply to patients  6.3  1.9  7.0  2.0  9.0  30.0  High 
Education of caregivers of patients with limited mobility  7.4  1.4  8.0  2.0  9.0  19.3  Very high 
Table 6.

Training Standards/Research of the NCR Nursing Staff.

Standard (continuous variable)  Mean  SD  Median  Lo  Hi  CV, %  Agreement 
Number of days for training (a year) for nurses (DUE)  11  10  60  75.3  Low 
Number of days for training (a year) for nursing assistant  13  90  153.1  Very low 
Minimum number of scientific NCR communications at conferences a year  48.7  High 
Minimum number of NCR projects in the last five years  10  60.2  Intermediate 
Minimum number of national NCR publications in the last five years  57.1  Intermediate 
Minimum number of international NCR publications in the last 5 years  44.7  High 
Among the research areas in which the NCR could participate we found (scale of 1–9)
Diagnostic and prognostic factors  6.7  1.4  7.0  2.0  9.0  20.7  Very high 
Efficacy studies (clinical trials) of treatments  7.4  1.2  7.0  3.0  9.0  15.7  Very high 
Efficacy studies (in practice) of treatments  7.6  1.1  8.0  3.0  9.0  14.0  Very high 
Safety Observatories  7.8  0.9  8.0  5.0  9.0  12.0  Very high 
Observatories adherence  8.3  0.7  8.0  6.0  9.0  8.4  Very high 
Genetic testing: biological sampling and clinical information  7.7  1.1  8.0  5.0  9.0  13.7  Very high 
Obtaining blood samples for clinical trials  8.6  0.6  9.0  7.0  9.0  6.8  Very high 
Economic NCR assessment studies  7.3  1.4  8.0  2.0  9.0  19.7  Very high 
The following should be included among the NCR nursing staff training topics (scale of 1–9)
Advances in pharmacological and non-pharmacological treatments  8.3  0.9  8.5  5.0  9.0  10.4  Very high 
Update on diagnosis  7.3  1.4  7.0  2.0  9.0  19.1  Very high 
Update on measurement tools (DAS28, CDAI, HAQ)  8.3  0.8  8.5  6.0  9.0  10.1  Very high 
Healthcare Management  6.4  1.5  6.0  3.0  9.0  24.1  Very high 
Research Methodology  6.9  1.3  7.0  3.0  9.0  19.1  Very high 
Stress Management for monitoring the patient  7.7  1.2  8.0  3.0  9.0  15.9  Very high 
Coaching through motivational talks, seminars, workshops  7.5  1.2  8.0  3.0  9.0  16.4  Very high 
Communication skills to promote kindness, understanding, empathy, sympathy, and creating a climate of trust  8.1  1.3  8.0  2.0  9.0  15.8  Very high 
The NCR should be able to train nursing students and other professionals  7.9  1.4  8.0  1.0  9.0  18.2  Very high 
Table 7.

Quality Care Standards (Scale of 1–9).

Standard (Scale of 1–9)  Mean  SD  Median  Lo  Hi  CV, %  Agreement 
Perform audits on NCR activity and nursing registries  7.1  1.4  7.0  2.0  9.0  19.7  Very high 
Perform NCR management indicators  7.3  1.4  8.0  2.0  9.0  18.4  Very high 
Have strategies for managing complaints and claims  7.5  1.1  8.0  3.0  9.0  14.7  Very high 
Evaluate the agreement of the assessments performed by nurses (e.g. DAS 28, RADAI) with specialists in Rheumatology in order for the results to be reliable  8.0  1.3  8.0  2.0  9.0  17.0  Very high 
If there is little agreement in the assessment between nurses and rheumatologists, carry out joint evaluations to increase the level of agreement  8.0  1.4  8.0  2.0  9.0  17.2  Very high 
Assess the satisfaction of patients  8.5  0.7  9.0  7.0  9.0  7.7  Very high 
Issues to consider in the satisfaction survey of patients seen in the NCR
Care, kindness  8.4  0.7  8.5  7.0  9.0  7.8  Very high 
Information provided by the nursing staff  8.7  0.6  9.0  7.0  9.0  6.7  Very high 
Frequency of visits  7.4  1.3  7.0  2.0  9.0  17.6  Very high 
Visits duration  7.4  1.4  7.0  2.0  9.0  19.0  Very high 
Waiting time to first NCR visit  7.8  1.0  8.0  4.0  9.0  12.3  Very high 
waiting time to be seen once in the NCR office  7.7  1.1  8.0  4.0  9.0  14.9  Very high 
Doubt resolution  8.6  0.7  9.0  6.0  9.0  8.0  Very high 
Accessibility  8.2  1.2  8.0  2.0  9.0  15.1  Very high 
Telephone contact availability  8.5  0.8  9.0  5.0  9.0  8.8  Very high 
physical space in which the query takes place  7.8  1.2  8.0  4.0  9.0  15.3  Very high 
Privacy consultation of the  8.3  0.9  8.5  4.0  9.0  11.3  Very high 
nursing Coordination with physician  8.7  0.7  9.0  5.0  9.0  7.8  Very high 
Standard (continuous variable)
Maximum time to resolve the claims of a patient in the NCR (days)  13  10  30  51.4  Intermediate 
Maximum time to answer any questions a patient has in by the NCR clinic (h)  14  12  12  72  86.0  Low 
Number of months the health quality questionnaires are received by patients of the NCR  10  12  24  36.3  High 

In the items related to general considerations (Table 1) it is worth highlighting the very high degree of agreement in all cases. The panelists believe that an NCR is needed in the Rheumatology department and that it contributes to improving quality of care, waiting times, resource use, costs, clinical outcomes and patient satisfaction.

Regarding structural standards (Table 2), this showed less agreement on the need for a crash cart (Table 2.1, subcategory of equipment) and the maximum number of people per NCR in a health area and for every nurse (Table 2.3, subcategory of population). There was a very high agreement on standards relating to the training the nurses should have to administer different questionnaires, especially the visual analog scale (VAS) and the Health Assessment Questionnaire (HAQ) and metrology training in rheumatoid arthritis regarding the Disease Activity Score 28 (DAS28) and the Bath Ankylosing Spondylitis Ankylosing Spondylitis Disease Activity Index (BASDAI) and the Bath Ankylosing Spondylitis Functional Index (BASFI).

In the items related to process standards we found a very high degree of agreement in all those included in the subcategories of Tables 3.1–3.3, except for the minimum number of days per week that the NCR should be available (Table 3.2) and the need for an appointment reminder to the patient the previous day (Table 3.3). As can be seen in Table 3.5, regarding the time of visit standards, we found a much lesser degree of agreement according to numerous items, highlighting a very low agreement in the minimum estimated time before scheduling appointments.

The items related to standards of care and follow-up (Table 4) and health education (Table 5) had a very high degree of agreement except an agreement only measured as high in “nursing training in relaxation techniques for application to patients” (Table 5).

The items in Table 6, relating to training and research standards, had a greater variability, being medium, low or very low in those regarding days required for continuing education and auxiliary nursing, number of research projects by nursing and number of national publications, respectively. For all other items there was a very high degree of agreement.

The items related to care quality standards (Table 7) had a very high agreement, except the maximum time to resolve claims, the maximum time for clinical queries and months the clinics should distribute questionnaires of perceived quality.

Discussion

To our knowledge, this is the first published study intended to clarify the features an NCR should have.

An important finding in this study was the high degree of agreement (CV<25%) achieved in most of the standards. Those related to initial considerations stand out as most important aspects considered, stating that the NCR is an essential component of Rheumatology departments and helps to improve the quality of care.

The structure standard underscores the need for the NCR to be located within the Rheumatology department. The most important minimum requirements are to have its own phone line, patient education materials, sphingomanometer, and scales to weigh and measure patients. In addition, the NCR nursing staff should be trained primarily for the performance of the VAS, HAQ, DAS28, BASDAI and BASFI.

Within the structure standards, there was also good agreement in establishing the average maximum population (about 200000 inhabitants) to justify establishing an NCR in a health area and the maximum number of rheumatologists who share an NCR nurse (3, 9). In contrast, the agreement was lower in deciding the average maximum population per nurse in an NCR (about 165000 inhabitants).

In the standard process it is worth noting the high degree of agreement that exists in almost all of the items. In the section on the NCR nursing staff, the panelists considered essential that the nursing staff managed their own agenda, kept a patient registry, applied protocols for the management of patients with rheumatologic disease and coordinated communication with the rheumatologist, physical therapist, occupational therapist, psychologist and social worker. Furthermore, the availability of NCR appointments must be at least 3 days/week.

In point of care activities of NCR, health education to patients and families, telephone assistance, global health assessment by the patient (VAS), pain VAS assessment and recording different nursing activities are considered essential. In patients with rheumatoid arthritis, functional capacity assessment by the HAQ is considered necessary. For patients with ankylosing spondylitis, the most important is the assessment of disease activity through the BASDAI and functional capacity assessment using the BASFI. For patients with osteoarthritis, the most important care activities of nursing personnel in the NCR should be to provide information on physical measures (hot-cold) and provide information to patients on an individual basis. For patients with chronic low back pain, individualized information and individualized advice on exercise and posture are the most prominent. For patients with osteoporosis this also applies, as individualized information and recommendations on adequacy of housing to prevent falls are the most necessary. For patients with lupus, individualized information stood out. In all conditions individualized information by the nurse stands over group information.

Panelists considered as important those aspects related to the registry of the NCR nursing staff. The most important item was to note either in the medical record (patient history) or in a database, patient assessment, medication sheet, activity planning, implementation of the activity and its assessment. It is of utmost importance to ensure patient confidentiality of the data. Furthermore, the patient history should always note the date and time in a clear and concise manner. The nurse's signature must appear on all medical records. All this information should also complete NCR treatment protocols.

There was also a high degree of agreement for some of the time standards, but not for the whole. Of note was the ‘very low’ degree of agreement when asked how far in advance the patient should be able to make an NCR appointment. At this point, the panelists issued responses ranging between 0.3 and 70 months. The lack of agreement on this point may be due to different mechanisms established in each department to meet the urgent demands or preference of patients, and the unscheduled tasks and availability for first appointment of different professionals (e.g. residents, physicians, nursing staff, secretary). It is desirable that these responsibilities are well defined to avoid delays or redundancies in unscheduled care.

By studying the results of treatment and monitoring standards, the majority of opinions highlighted the need for a protocol for monitoring disease modulating biological and nonbiological drugs, and applying it. It is also very important to ask the patient for their adherence to treatment at each visit.

Among the NCR health education standards, the panelists believe that it is essential to teach patients treated with subcutaneous drugs to administer them properly and train when in self-medication.

In the block of training and research standards of the NCR staff, there is little agreement among the panelists on how to quantify the value of the indicators. The agreement is very low at the time of deciding the number of days per year of training for nursing assistants, standard on which the panelists ranged from a day to 90 days (mean: 8 days). In the case of nursing training days, and showing better agreement, it scored, on average, 11 days a year. This variability may be explained by the different professionals that made up the group of experts who participated in the study, rheumatologists and nurses belonging to GTESER.

There was more consensus in deciding the research and staff training issues of the NCR. The most valuable lines of research were to obtain blood samples for clinical trials and adherence protocols. The most needed training topics were considered advances in pharmacologic and nonpharmacologic treatments, as well as metrology tools update. It is essential that, with the gradual implementation of NCR units, rheumatologists actively engage nursing staff in research tasks, which are not limited to mere procedure performance. This means minimal training in research methodology as part of the skills to be acquired by any member of the service.

Finally, there was also a low level of agreement to quantify time standards on quality of care. The agreement is low when setting the maximum time to answer any questions from a patient by the NCR. This time ranges from a minimum of one and a maximum of 72h. On the contrary, the agreement is highest in consideration to quality of care. The most important aspect of quality of care is to assess the satisfaction of patients. Among the most important aspects to consider in the satisfaction survey of patients seen in the NCR, information provided by the nursing staff, coordination with the attending physician, resolution of doubts and ease/availability for contacting the NCR stand out.

Although for this project followed the most appropriate methodology for developing such standards, the study is not without limitations. For example, a possible limitation may arise from the representation of the panelists. To avoid this limitation, we sent the questionnaire to rheumatologists and nurses across the country. In the end, panelists from 12 regions and different types of services participated, and it seems unlikely that there is any significant bias in the selection on the panelists and the results derived from their scores.

Another limitation may arise from the lack of completeness of the standards. To avoid missing potentially relevant standards, we searched all those available in previous publications, including15 those of SER itself. With the identified standards, we developed a preliminary list of those that should have an NCR. This list was corrected and extended by SER researchers who had participated in other projects such as the SERAP and ESPERANZA programs. It is therefore unlikely that this list missed any relevant standards, although it is possible that new standards may be defined with increased attention on NCR and their numbers.

Another possible limitations in the development of standards may be due to the influence of the opinions of some experts over others. In this project, influence among panelists was avoided using the Delphi method, since interactions occur in the second round to maintain the anonymity of responses. This method has proved its worth in the development of standards similar to those in this study.14,26–30

In any case, the standards developed in our study are quite consistent with recent recommendations made recently by the European League Against Rheumatism (EULAR) on the role of nurses in the management of chronic inflammatory arthritis.31

In conclusion, the definition of these standards may allow us to establish minimum desirable qualities in different dimensions, such as structure, process and outcome, both in clinical work and in research and teaching. The usefulness of these standards can be multiple, not only for health professionals but also in relationships with patients and patient organizations, institutions and health services and administration.

Ethical ResponsibilitiesProtection of People and Animals

The authors state that no experiments were performed on humans or animals.

Data Confidentiality

The authors state that no patient data appear in this article.

Right to Privacy and Informed Consent

The authors state that no patient data appear in this article.

Funding

Project undertaken by the Spanish Foundation of Rheumatology and sponsored by Abbott.

Conflict of Interest

Dr. D. Santiago Muñoz states the following conflicts of interest: advisorships with Roche, Celgene, and Pfizer. President of the Spanish Society of Rheumatology. Current Participation in clinical trials or studies sponsored by UCB, Roche, Abbott, MSD, Bioiberica, Celgene, Pfizer and the Spanish Foundation of Rheumatology. The other authors declare no conflicts of interest.

Acknowledgments

We thank the panel of experts who responded to the Delphi questionnaires for their time and dedication in preparing their responses, without which this study would not have been possible.

References
[1]
N.C. Dalkey.
The Delphi method: an experimental study of group opinion.
Memorándum RM-5888_PR. The Rand Corporation, Santa Monica, (1967),
[2]
H.A. Linstone, M. Turoff.
The delphi method: techniques and applications,
[3]
R. Pineault, C. Daveluy.
El enfoque por búsqueda de consenso en: “La planificación sanitaria: conceptos, métodos, estrategias”.
Barcelona, (1987),
[4]
J.J. Gérvas, M.M. Pérez Fernández.
La técnica Delphi.
Clínica Rural, 312 (1989), pp. 38-42
[5]
R.P. Hudadk, P.P. Brooke, K. Finstuen.
Forecast 2000: a prediction of skills, knowledge, and abilities required by senior medical treatment facility leaders into the 21st century.
Mil Med, 159 (1994), pp. 494-500
[6]
J. Chin, P.A. Sato, J.M. Mann.
Projections of HIV infections and AIDS cases to the year 2000.
Bull World Health Organ, 68 (1990), pp. 1-11
[7]
E.F. Van Beeck, J.P. Mackenbach.
Future health scenarios as a tool in the surveillance of unintentional injuries.
Health Policy, 40 (1997), pp. 13-28
[8]
J.M. Harrington.
Research priorities in occupational medicine: a survey of United Kingdom medical opinion by the Delphi technique.
Occup Environ Med, 51 (1994), pp. 289-294
[9]
E.J. Hahn, M.K. Rayens.
Consensus for tobacco policy among former state legislators using the policy Delphi method.
Tob Control, 8 (1999), pp. 137-140
[10]
D.W. Bettcher, S. Sapirie, E.H. Goon.
Essential public health functions: results of the international Delphi study.
World Health Stat Q, 51 (1998), pp. 44-54
[11]
M. Gallagher, C. Bradshaw, H. Nattress.
Policy priorities in diabetes care: a Delphi study.
Qual Health Care, 5 (1996), pp. 3-8
[12]
Estudio Delphi sobre estrategias y prioridades frente al SIDA en la Comunidad de Madrid”.
Comunidad de Madrid. Consejería de Salud-Servicio Regional de Salud de Madrid, (1991),
[13]
K. Fitch, S.J. Bernstein, M.D. Aguilar, B. Burnand, J.R. LaCalle, P. Lázaro, et al.
The RAND/UCLA appropriateness method user's manual.
Santa Monica, (2001),
[14]
S.J. Bernstein, P. Lazaro, K. Fitch, M.D. Aguilar, H. Rigter, J.P. Kahan.
Appropriateness of coronary revascularization for patients with chronic stable angina or following an acute myocardial infarction: multinational versus Dutch criteria.
Int J Qual Health Care, 14 (2002), pp. 103-109
[15]
A. Alonso, J. Vidal, J. Tornero, J. Carbonell, P. Lázaro, M.D. Aguilar.
Estándares de calidad asistencial en reumatología.
Reumatol Clin, 3 (2007), pp. 1-20
[16]
J. Carulla, C. Jara, J. Sanz, N. Batista, C. Camps, J. Cassinello, on behalf of the ALGOS Group, et al.
Oncologists’ perceptions of cancer pain management in Spain: the real and the ideal.
Eur J Pain, 11 (2007), pp. 352-359
[17]
K. Fitch, P. Lázaro, M.D. Aguilar, J.P. Kahan, M. van Het Loo, S.J. Bernstein.
European criteria for the appropriateness and necessity of coronary revascularization procedures.
Eur J Cardiothorac Surg, 18 (2000), pp. 380-387
[18]
P. Lázaro, K. Fitch, Y. Martín.
Estándares para el uso apropiado de la angioplastia coronaria transluminal percutánea y cirugía aorto-coronaria.
Rev Esp Cardiol, 51 (1998), pp. 689-715
[19]
N.J. Reavley, A. Ross, E.J. Killackey, A.F. Jorm.
Development of guidelines to assist organisations to support employees returning to work after an episode of anxiety, depression or a related disorder: a Delphi consensus study with Australian professionals and consumers.
BMC Psychiatry, 12 (2012), pp. 135
[20]
P. Lázaro y de Mercado.
Necesidad, adecuación y utilización de servicios sanitarios.
pp. 51-93
[21]
Sociedad Española de Reumatología.
Proyecto SERAP. Evaluación de un modelo de atención de la artritis en España.
(2012),
[22]
S. Muñoz-Fernández, L. Carmona, E. Collantes, J. Mulero, M.J. García-Yébenes, E. de Miguel, Esperanza Group, et al.
A model for the development and implementation of a national plan for the optimal management of early spondyloarthritis: the Esperanza Program.
Ann Rheum Dis, 70 (2011), pp. 827-830
[23]
Sociedad Española de Reumatología.
Proyecto EMAR II. Variabilidad en el manejo de la artritis reumatoide y las espondiloartritis en España.
(2012),
[24]
Sociedad Española de Reumatología Guía de práctica clínica sobre el manejo de los pacientes con Espondiloartritis.
(2012),
[25]
GUIPCAR Group.
Clinical practice guideline for the management of rheumatoid arthritis in Spain.
Spanish Society of Rheumatology, (2007),
[26]
P.G. Shekelle, M. Chassin, R.E. Park.
Assessing the predictive validity of the RAND/UCLA appropriateness method criteria for performing carotid endarterectomy.
Int J Technol Assess Health Care, 14 (1998), pp. 707-727
[27]
P.G. Shekelle, J.P. Kahan, S.J. Bernstein, L.L. Leape, C.J. Kamberg, R.E. Park.
The reproductibility of a method to identify the overuse and underuse of medical procedures.
N Engl J Med, 338 (1998), pp. 1888-1895
[28]
J. McDonnell, A. Meijler, J.P. Kahan, S.J. Bernstein, H. Rigter.
Panellist consistency in the assessment of medical appropriateness.
Health Policy, 37 (1996), pp. 139-152
[29]
R.L. Kravitz, M. Laouri, J.P. Kahan, P. Guzy, T. Sherman, L. Hilborne, et al.
Validity of criteria used for detecting underuse of coronary revascularization.
JAMA, 274 (1995), pp. 632-638
[30]
P.G. Shekelle.
Are appropriateness criteria ready for use in clinical practice?.
N Engl J Med, 344 (2001), pp. 677-678
[31]
Y. Van Eijk-Hustings, A. van Tubergen, C. Boström, E. Braychenko, B. Buss, J. Felix, et al.
EULAR recommendations for the role of the nurse in the management of chronic inflammatory arthritis.
Ann Rheum Dis, 71 (2012), pp. 13-19

Please cite this article as: Muñoz Fernández S, et al. Estándares de calidad asistencial para las consultas de enfermería en reumatología. Reumatol Clin. 2013;9:206–15.

Copyright © 2012. Elsevier España, S.L.. All rights reserved
Idiomas
Reumatología Clínica (English Edition)

Subscribe to our newsletter

Article options
Tools
es en

¿Es usted profesional sanitario apto para prescribir o dispensar medicamentos?

Are you a health professional able to prescribe or dispense drugs?

es en
Política de cookies Cookies policy
Utilizamos cookies propias y de terceros para mejorar nuestros servicios y mostrarle publicidad relacionada con sus preferencias mediante el análisis de sus hábitos de navegación. Si continua navegando, consideramos que acepta su uso. Puede cambiar la configuración u obtener más información aquí. To improve our services and products, we use "cookies" (own or third parties authorized) to show advertising related to client preferences through the analyses of navigation customer behavior. Continuing navigation will be considered as acceptance of this use. You can change the settings or obtain more information by clicking here.