Final program evaluation methods and results of a National Lymphedema Management Program in Togo, West Africa
- DOI
- 10.1016/j.jegh.2013.11.001How to use a DOI?
- Keywords
- Lymphatic filariasis; Public health; Morbidity; Program evaluation; Lymphedema
- Abstract
In order to eliminate Lymphatic Filariasis (LF) as a public health problem, the World Health Assembly recommends an approach which includes interruption of transmission of infection and the alleviation of morbidity. In 2000, the Togolese National Program to Eliminate Lymphatic Filariasis (PNELF) started the annual mass drug administrations and in 2007, the program added a morbidity component for the management of lymphedema. This manuscript describes the methods of an evaluation aimed at assessing the strengths and weaknesses of the Togolese National Lymphedema Morbidity Program. The evaluation was conducted through in-depth interviews with stakeholders at each programmatic level. Interviews focused on message dissemination, health provider training, patient self-care practices, social dynamics, and program impact. The evaluation demonstrated that the program strengths include the standardization and in-depth training of health staff, dissemination of the program’s treatment message, a positive change in the community’s perception of lymphedema, and successful patient recruitment and training in care techniques. The lessons learned from this evaluation helped to improve Togo’s program, but may also provide guidance and strategies for other countries desiring to develop a morbidity program. The methods of program evaluation described in this paper can serve as a model for monitoring components of other decentralized national health programs in low resource settings.
- Copyright
- © 2014 Ministry of Health, Saudi Arabia. Published by Elsevier Ltd.
- Open Access
- This is an open access article under the CC BY-NC-ND license (http://creativecommons.org/licenses/by-nc-nd/4.0/).
1. Introduction
The global disease burden created by infection with Lymphatic Filariasis (LF) is profound. Among the 72 countries (one-fifth) of the world’s population at risk for LF, an estimated 44 million people are symptomatic with lymphedema and/or hydrocele [1–4]. These chronic maladies also render patients susceptible to acute infectious attacks. This morbidity leads to the loss of 5.5 million disability-adjusted life years (DALYs), thus implicating LF and its manifestations as the second leading cause of disability worldwide [5,6]. The impact of disease is also significant as it carries a social stigma and psychological morbidity [7].
Although the international efforts to interrupt transmission have been successful [8], there still remains a lack of morbidity management programs targeting the second pillar of the Global Programme to Eliminate Lymphatic Filariasis. The programs that do exist are often very specialized and geographically focal. This presents both logistical and financial challenges when bringing these approaches to a national or even regional scale [9–12]. Therefore, the morbidity component of the Togolese National Program to Eliminate Lymphatic Filariasis (PNELF) is unique in that it is the first nationally scaled lymphedema management program in Africa, even including non-LF endemic areas. For this reason, a final evaluation of the program was of critical importance. This evaluation would not only strengthen this pilot project, but would also identify areas to improve the concept before applying it in other endemic countries. In summary, this manuscript describes: (1) recommendations for implementing and evaluating a national lymphedema morbidity program; and (2) an evaluation method that could be used to evaluate other decentralized public health programs in low resource settings.
2. Methods
Togo is a West African country with a population of approximately 6 million, divided into six regions and 35 districts; 35% of the population lives below the poverty line. This is a fact of special importance given the overlap between areas of poverty and LF endemicity. According to LF mapping conducted in 2000, 7 of the country’s 35 districts in 3 of its regions were endemic for LF [13]. The morbidity program, as described by Mathieu et al. [14], began in 2005 when Togo’s Ministry of Health collaborated with the CDC and through funding by IMA/USAID developed a novel approach to implement a national lymphedema morbidity program.
2.1. Description of the program
The main components of the program were: (1) train one member of the health staff personnel in lymphedema care at each dispensary in the 35 national health districts, with a “training of trainers” approach, including the national LF coordination team, the district health staff, dispensary nurses, and finally village volunteers; (2) inform people with a swollen leg that care is available at the local dispensary, through health education, information spread by the administrative system (prefect, village chief, and town crier), and media (posters, radio, newspaper, and television); (3) diagnose and train patients on how to care for a swollen leg. This was carried out by the trained dispensary nurse, using standard care techniques recommended by the World Health Organization (WHO) [15]. The soap and towels used in the training were provided free to the patient as a one-time donation “hygiene pack” with the aim of helping patients initiate treatment; (4) provide a support system to motivate the patients. This was executed through a combination of family member support, follow-up visits from a trained village volunteer, and return visits to the dispensary. Clinical data at these visits were documented in a lymphedema-specific patient booklet, the same style that is commonly used in the Togo healthcare system for regular consultations; and (5) integrate lymphedema management into the national curriculum for medical doctors, medical assistants and nurses.
The program was organized slightly differently in LF endemic and non-endemic health districts. The most significant difference was that in non-endemic areas, patient follow-up was the role of a family member and not with village volunteers. Thus, village volunteers in non-endemic districts did not receive the formal training given to volunteers in endemic districts.
2.2. Selection of interview areas
The evaluation encompassed the whole country, but included oversampling in the 3 regions with the 7 previously endemic LF districts. A total of 8 districts were sampled. In order to attempt a comprehensive and balanced program evaluation, and to view healthcare units that functioned differently, the sampling included endemic and non-endemic districts and dispensaries that had both “high” and “low” patient enrollment. Within the region containing three LF endemic districts (‘LF districts’), the LF districts with the lowest and highest number of enrolled lymphedema patients were surveyed, in addition to a randomly selected non-LF district. In the 2 endemic regions which had only 2 endemic districts (both with comparable patient numbers), 1 district was chosen at random. In the 3 regions without LF districts, 1 district was randomly selected (Fig. 1).
Map of Togo: (a) by sampled districts; and (b) by location of dispensaries and villages in the program evaluation, Togo, 2010.
In each district, two dispensaries were selected: the dispensary with the highest number of enrolled patients and one dispensary that had “low” patient numbers reported. The “low” dispensary was chosen randomly from among the five dispensaries with the lowest enrollment. If there were no patient enrollment data available for a district or if all dispensaries reported fewer than five enrolled patients, two dispensaries were chosen at random. A list or map of the dispensary’s catchment area was used to select a random village. From there, a minimum of three to five geographically adjacent villages were visited in each dispensary’s catchment area.
2.3. Selection of participants
In-depth interviews were conducted with each of the five major stakeholders in the program: all persons with a swollen leg and their family members; village volunteers; dispensary nurses; and district staff (either LF focal person or district doctor). In addition, as many village surveys with community members were performed as time would allow.
2.4. Interviews
The evaluation team consisted of an interviewer, transcriber, and translator. The translator was trained in-depth by the CDC team on the goal of each question. The translator spoke fluently in French and Moba (one of the 35 Togolese languages). This meant that in some villages, interviews required translation from English to French to a local language, facilitated by the translator in conjunction with a local village volunteer. During an interview, each member of the team was introduced, and the basic goals of the program evaluation were explained. Next, a series of questions were asked using pre-developed interview guides, each one unique for the group of stakeholders being interviewed. The guides were aimed at addressing five areas of importance: message dissemination, provider training, patient self-care practices, social dynamics, and program impact. The interview structure was conversational, and the interview guide was used to make sure that all the desired questions were addressed. During the interviews, data were collected on a pre-designed data entry form for quick answer recording. The team spent two days in each health district and then feedback was given to each dispensary nurse and/or the district doctor or LF focal point before moving to the next district.
2.5. Data analyses
After each day’s work, the transcribed interview notes were entered into a Microsoft Excel database taking counts of number of times certain topics were mentioned in each interview and organizing key quotes by subject matter. The data were primarily analyzed qualitatively, drawing out centralized themes. Community surveys were graded on percentage of a group responding affirmatively: if everyone in a community group surveyed knew the program message, the survey received a 100% “score”; 75% for more than half; 50% for exactly half; 25% for less than half; and 0% if no one was aware of the message. After recording data in this way, community surveys could be averaged together for a given village or district.
3. Ethical considerations
The study protocol was examined by the CDC and was deemed a program evaluation activity not involving human subjects’ research. The protocol was also submitted to the Ministry of Health in Togo for approval. Verbal consent was obtained before all interviews using the local language and only after providing appropriate details about the information being collected and its future use.
4. Results
The evaluation included visits to 22 dispensaries and over 70 villages in 11 health districts in 6 regions of Togo (Fig. 1). Interviews were conducted with 11 district staff, 20 dispensary nurses, 43 village volunteers, 90 patients, 4 family members of patients, and 87 community surveys. In total, more than 255 participants were involved. Generally, patient interviews took approximately 20 min, village volunteer interviews 20–30 min, and dispensary nurse and district health staff interviews 35–45 min.
4.1. Coverage of the program
With respect to patient recruitment and enrollment, 79% (71/90) of the patients identified during the evaluation to have a swollen leg had been seen at the dispensary for care. Of those 71 patients, 56 (79%) were enrolled in the program. This same recruitment and enrollment data can be analyzed based upon endemic and non-endemic districts and are presented in Table 1. Of the 19 patients not seen at the dispensary, the majority (13/19) stated it was because they were not aware of the treatment message. A closer look at these 13 cases revealed that in 9 cases the person’s community was also unaware of the message; in 3 other cases, the community was aware of the message, but did not inform the patient because they thought the swollen leg was not severe enough to necessitate treatment or qualify for enrollment in the program; and in the last case, the volunteer was aware of the treatment message, but had not informed the patient because he was also a traditional healer and one of his fellow traditional healer colleagues was providing treatment for the patient.
| Total patients | No. of patients seen at dispensary (% total) | No. of patients enrolled (% total seen at dispensary) | |
|---|---|---|---|
| Endemic | 57 | 49 (86%) | 45 (92%) |
| Non-endemic | 33 | 22 (67%) | 11 (50%) |
| Total | 90 | 71 (79%) | 56 (79%) |
Recruitment and enrollment data for patients with a swollen leg, evaluation of the National Lymphedema Morbidity Management Program, Togo, 2010.
Of the 19 patients not seen at the dispensary, the minority (6/19) were in fact aware of the treatment program: 3 reported lack of means (money or transportation); 1 reported having been seen at the dispensary before 2005 and was not aware that this treatment program was different; 1 thought that the treatment would not be worthwhile; and the last 1 did not come as a result of shame related to his disease.
Among the 15 patients that were seen at a dispensary but not subsequently enrolled were several patients in non-endemic districts who were referred to an upper level hospital, but did not have the transportation means to follow-up. The dispensary nurse in one of these areas described that he had no other enrolled patients and did not feel comfortable managing this (stage 5 lymphedema) patient’s care. Another common reason patients were not enrolled in the program was because they had only episodic swelling, which was not recognized as part of the clinical spectrum of lymphedema.
4.2. Training
Each level of the health system consistently reported the training being one of the major strengths of the program. The training sessions were considered to be clear, efficient, and very detailed. This allowed the trainees to “feel comfortable” becoming teachers themselves. The fact that the training was standardized, and all health staff in the country were provided with the same care instructions, was also considered valuable. Besides using the program manual for the training sessions, another incredibly worthwhile learning tool was having a patient with lymphedema attend the training. This aided not only in demonstration and practice, but also made it possible for the trainees to hear about what having the disease meant as a patient. Additionally, having the live practice at the end of the training session was a time during which trainees with more experience in caring for the disease could share thoughts with those that were less experienced. One dispensary nurse mentioned:
“At my training, they did demonstrations with a real patient and then live practice on that patient, which I thought was effective for learning…[an]other strength was that the trainer was very well informed and explained everything well, especially disease [pathology]…[an]other strength was that there was ample time for questions at the end. [Having] more patients to be able to do live practice on would have been good.”
Of note, several trainers also decided to use a self-created PowerPoint slideshow or video. These are elements which were not an official part of the program, but were used to facilitate the learning process.
Dispensary nurses and patients were interviewed regarding the care techniques taught to the patients at the dispensary. When nurses were questioned about which care techniques were taught, 95% of nurses stated that they taught patients how to wash the leg, 95% taught about elevation, and 91% reported that they taught patients exercises. Among patients surveyed, 100% of enrolled patients reported being taught washing techniques, 76% elevation, and 79% exercises. Data were relatively consistent whether the patient came from an endemic or non-endemic district (Table 2). For 33% of the patients in non-LF endemic districts, a family member was trained as was recommended by the program. In endemic districts, where training family member was not officially part of the program, 13% still were. During follow-up in endemic areas, the vast majority of village volunteers reported re-teaching patients the lymphedema management techniques (93% [25/27], 78% [21/27], and 78% [21/27] for washing, elevation, and exercise, respectively). Village volunteers were less convinced that the patients were actually able to follow the care techniques (81% [22/27], 64% [16/25], and 64% [16/25], respectively), and 100% of the patients in the program reported being able to wash initially, but performing the elevation and exercise was considered more difficult (71% and 74% adherence, respectively). At the time of interviews during this evaluation, which for most patients was between one and three years after the initial training, 80% (45/56) of patients stated that they were still doing their washing and 61% (33/54) of the patients stated that they were performing elevation and exercises (Table 3). A factor that influenced adherence was having a basic understanding of the disease mechanism; when patients were aware that the swollen leg was due to fluid buildup caused by defective circulation, they were able to understand the three treatment tenets.
| Total N | n | Percent (%) | ||
|---|---|---|---|---|
| All patients | Washing | 56 | 56 | 100 |
| Elevation | 54 | 41 | 76 | |
| Exercise | 56 | 44 | 79 | |
| Patients from endemic districts | Washing | 47 | 47 | 100 |
| Elevation | 46 | 35 | 76 | |
| Exercise | 47 | 37 | 79 | |
| Patients from non-endemic districts | Washing | 11 | 11 | 100 |
| Elevation | 11 | 8 | 73 | |
| Exercise | 11 | 8 | 73 |
Patient reported initially being trained (by patient subset).
Training of enrolled patients in program’s care techniques, evaluation of the National Lymphedema Morbidity Management Program, Togo, 2010.a
| Patient able to do initially | Patient able to do it after 3 years | ||||||
|---|---|---|---|---|---|---|---|
| Total N | n | Percent (%) | Total N | n | Percent (%) | ||
| All patients | Washing | 56 | 56 | 100 | 56 | 45 | 80 |
| Elevation | 54 | 38 | 70 | 54 | 33 | 61 | |
| Exercise | 56 | 41 | 73 | 56 | 36 | 64 | |
| Patients from endemic districts | Washing | 47 | 47 | 100 | 47 | 41 | 87 |
| Elevation | 47 | 33 | 70 | 47 | 29 | 62 | |
| Exercise | 47 | 34 | 72 | 47 | 33 | 70 | |
| Patients from non-endemic districts | Washing | 11 | 11 | 100 | 11 | 6 | 54 |
| Elevation | 11 | 9 | 82 | 10 | 6 | 60 | |
| Exercise | 11 | 9 | 82 | 11 | 5 | 45 | |
Village volunteer and patient reported adherence to care techniques over time, evaluation of the National Lymphedema Morbidity Management Program, Togo, 2010.
4.3. Health education
Community surveys on message dissemination showed that there was 62% “coverage” rate of message recall among all villages, when averaged as described above. In general, within a given village or set of villages, either everyone knew the message or no one knew the message. Message dissemination was accomplished through the methods that were officially part of the training program and also through avenues created by the dispensary nurse, such as having an announcement made by teachers at school or an informal meeting at the site of a farming collective. District level health staffers that worked in rural areas were more likely to endorse village volunteers as the best method, while those in urban locales more often supported mass media. Sixty-seven percent of patients with a swollen leg (regardless of enrollment status) said that they were aware of the program message. Patients with a swollen leg heard the message mainly from village volunteers and from the dispensary (Table 4). The most effective method to encourage patients to seek care was an active, trained village volunteer.
| Stakeholder/responder | Best message dissemination method | Number of positive responses |
|---|---|---|
| District supervisors (N = 12) | Village volunteers | 5 |
| Mass media | 4 | |
| Town crier | 2 | |
| Dispensary | 1 | |
| Dispensary nurses (N = 21) | Village volunteers | 14 |
| Dispensary | 4 | |
| Town crier | 3 | |
| Lymphedema patients (N = 41) | Village volunteers | 16 |
| Dispensary | 11 | |
| Family member | 5 | |
| Town crier | 4 | |
| Mass media | 4 | |
| Private doctor | 1 |
Optimal methods of message dissemination reported by program stakeholders, evaluation of the National Lymphedema Morbidity Management Program, Togo, 2010.
When radio was used, health staff noted that having the broadcast occur multiple times in several of the different local languages was important. Spreading the treatment message during mass drug administration (MDA) was also effective and resulted in the population having a more complete understanding of the link between LF, lymphedema, hydrocele, and MDA. This helped to reinforce the concept of lymphedema as a sequela of a treatable infectious disease, as opposed to a spiritual disease or curse. One district-level official noted that:
“Originally, LF was thought to be a spiritual disease, so at first they tried to cure it with traditional medicine; but there is no cure. So if we increase sensitization, even a small farmer will know the cause, and it will be a success.”
Forty percent of district supervisors reported a change in community perception regarding LF after dissemination of the message; this opinion was shared among 67% (14/21) of dispensary nurses and 91% (39/43) of village volunteers. Regarding message dissemination, one village volunteer said:
“Before the message dissemination, there was a lot of discrimination, but now it’s just like HIV – it’s just another disease.”
4.4. Follow up
Of patients living in endemic villages, 82% (47/57) were visited at home by a village volunteer and more than half reported help from a family member at home. In LF non-endemic districts, although follow-up from a village volunteer was not an official part of the program, 67% (22/33) of patients were still provided follow-up by a village volunteer. Overall, the patients that had the most consistent follow-up were those that were followed both by a village volunteer and a family member. One dispensary nurse said:
“The best setup would be [to train] both, a synergy between family, ASC [village volunteer] and patient for follow-up.”
Patients thought that the booklets filled out at follow-up visits were helpful for several reasons. First, the drawings on the front and back inside cover helped remind them of the care techniques. Second, patients appreciated being able to read the notes and see improvement in clinical status. Third, patients said that a filled-in booklet was a tangible piece of evidence that someone was visiting them and cared about their health, a fact even appreciated by several illiterate patients. One patient noted:
“I can see the booklet and be reminded of what I have done [by looking at the pictures] and realize that I am not alone. When someone follows up (in the booklet), it is encouraging.”
Although 59% (16/27) of village volunteers reported that patients had difficulty in obtaining supplies, only 49% (13/27) of patients confirmed this fact. The most common problem was finding soap, but two patients mentioned having difficulty with obtaining clean water and dry towels. When speaking about material availability, one village volunteer said:
“…In the early years, the patients were given soap and towels; these materials have run out and now the patients feel abandoned.”
5. Discussion
The Togolese national program to eliminate lymphatic filariasis (PNELF) introduced a novel nationwide approach to alleviate lymphedema morbidity associated with LF. Three years after the program was launched, it was determined that a program evaluation would provide critical information regarding the successes and lessons learned from this pilot project so that similar programs might be introduced elsewhere. A sampling method was elaborated which took important confounding factors into account, such as LF endemicity, high prevalence of patients and/or motivated health staff. The idea behind this methodology was that it could be easily adapted for similar public health programs. The key areas which were addressed through interviews of stakeholders at all levels of the program included: message dissemination, provider training, patient self-care, social dynamics, and program impact.
The dissemination of the health education message has a two-pronged objective: to identify patients and to educate the general population about the existence of LF as an infectious disease. The subsequent method of recruitment which was most likely to bring patients into the program was the use of well-informed village volunteers. A likely reason for this is that many patients with lymphedema have been struggling with the disease for many years and have tried various treatments [7]. When an actual person visited their home and explained the concept of lymphedema treatment as new, prophylactic, and cost-effective, patients were much more likely to visit the dispensary. It seemed that more passive methods, such as mass media, town crier, and posters, were not as effective.
Seventy-nine percent of patients seen at the dispensary were enrolled – a high value for a public health program, especially considering the stigmatizing nature of the disease and the limited knowledge regarding disease etiology. The majority of people suffering from lymphedema that did hear the message went to the dispensary to seek help. Village volunteers seemed incredibly effective at recruiting patients, but when a village volunteer had not been formally trained and lacked detailed information regarding the treatment, as was the case in non-endemic districts, he/she could not make a convincing case for the treatment program.
The patients that had the most consistent follow-up were those that were followed by both a village volunteer and a family member. This setup allowed for a system where initially frequent village volunteer visits transitioned to family member-based care over time. The next best method was to use only a trained village volunteer, which was superior to having only a family member. Although village volunteers were not paid, they considered visiting the patients their “job”, while family members often helped only if they had extra time.
When patients had a good understanding of the treatment goals, expectations could be met and adherence was improved. Patients sometimes had a difficult time understanding that the treatment did not include medications. One dispensary nurse described that “In Africa, people think that treatment is a medicine; when you don’t give someone medicine, they don’t think it is a treatment.”
Throughout patient and health staff interviews, village volunteers were identified as the major feature that allowed the program to function and make morbidity management accessible. In the current program, village volunteers are only officially utilized in the endemic districts, where there are existing LF volunteers. The use of village volunteers who are also involved in other health programs is an innovative piece of the program, and it is recommended to expand their inclusion in all districts, also the non-LF endemic district. Given cost and time constraints, the training need not be as in-depth of a session, but would at least include the basic mechanisms of disease, simple concepts of treatment, and most importantly that the treatment is inexpensive, effective, and easy to perform. With this information, the village volunteers could effectively function in the roles of message disseminators, case finders and patient recruiters. When and if a village volunteer did locate a case, he/she could accompany the patient to the dispensary and receive full training in the care techniques along with the patient. This would allow him/her to perform follow-up.
The above points on the village volunteer’s role in the program are one explanation for why the program generally functioned more effectively in endemic areas. Another possible explanation is the increased community awareness of lymphedema in populations familiar with MDA campaigns for LF. Lastly, it is possible that the dispensary nurses in endemic districts were more proficient at managing the program, possibly due to the fact that endemic areas typically had more practice in managing cases.
The authors observe that a potential limitation of the current study includes the use of non-probability sampling design for the selection of villages in each dispensary’s catchment area. This qualitative evaluation relied on in-depth interviews requiring strategic methods to be put in place to minimize the potential introduction of interviewer and response biases. Interviewer training and data recording methods were standardized among all interviewers to ensure that objective recording procedures were used.
Language translation and back translation measures were completed with care since it was necessary that interviews be conducted in the local language, that responses be back translated into French, and that final results be reported in English. The authors note that collecting data in one language and presenting the findings in another can involve many factors that may influence the quality of translation which in turn may have a direct impact on the validity of the research.
Despite these potential limitations, this study tested a program evaluation method that can help guide programmers to evaluate similar national programs in a way that uses few resources, is easy to implement, is replicable, and is able to procure data with wide geographic coverage from all districts of interest. The authors anticipate that the description of such an evaluation will demonstrate the important role evaluation has in national program implementation and will encourage further use of such evaluations in non-academic settings. Furthermore, the method in which this evaluation was carried out has allowed for important indicators of program quality and adherence to be measured, and encourages the free flow of opinions at all levels of the program structure. The qualitative data have illustrated the strengths and weaknesses of such a program and provide great utility in program monitoring and evaluation.
In conclusion, this complete program evaluation was critical for three reasons. First, the program continues to function in Togo, and lessons learned can help to determine program organization, funding, and delivery in the future. Secondly, there is a deficiency of morbidity programs in nearby LF-endemic countries, several of which are considering the feasibility of initiating national morbidity programs. Being able to understand the strengths and weaknesses of the Togolese national program is invaluable to these health systems. Lastly, the unique method of evaluation described here can help to monitor the effectiveness of these future programs and other public health programs.
Disclaimer
The findings and conclusions in this report are the findings and conclusions of the authors and do not necessarily represent the views of the Centers for Disease Control and Prevention.
References
Cite this article
TY - JOUR AU - Josh Ziperstein AU - Monique Dorkenoo AU - Michel Datagni AU - Naomi Drexler AU - Monica Murphy AU - Yao Sodahlon AU - Els Mathieu PY - 2013 DA - 2013/12/22 TI - Final program evaluation methods and results of a National Lymphedema Management Program in Togo, West Africa JO - Journal of Epidemiology and Global Health SP - 125 EP - 133 VL - 4 IS - 2 SN - 2210-6014 UR - https://doi.org/10.1016/j.jegh.2013.11.001 DO - 10.1016/j.jegh.2013.11.001 ID - Ziperstein2013 ER -