09 – Strengthening Information Systems and Linkages to Care

This section addresses a set of questions relating to impact evaluation of new or existing information systems that have been modified, and especially purpose-specific electronic databases. The general principles of evaluating TB programs apply, i.e. district, province and national official notifications and prevalence surveys should be the principal impact measures, and the care cascade should be enumerated at every feasible step in the process of TB care. Some questions to consider when designing M&E tools and measuring the impact of the information system include the following:

• Can stable baselines and comparison populations be established? Whether the unit considered is a hospital, a district, a province or even perhaps a country, the first question is ‘Impact compared to what?’ Judgment is almost always required, together with time taken to explore how existing systems actually work; and to explain trends, gaps and errors. Programme credibility will depend on how well the surveillance picture can be characterized. Separate aspects of impact will need separate baselines and controls, e.g. DR-TB, children and the elderly; and it may require ingenuity to design a method to disentangle improved reporting from the effect of co-existing TB care interventions.
• Has there been overall increase in laboratory testing? A point that is often overlooked is that without a population-level increase in testing, it is unlikely that missing people with TB will be found. This means that laboratory linkage generally needs to be a parameter of IT system evaluation. Existing systems that notify at the point of treatment initiation may have poor ascertainment of diagnostic testing (especially in private laboratories), and the linking of records to ongoing individual patient care may not be present at all.
• Can repeat measures be triangulated for consistency? It is common to find that large increases in diagnosis or treatment shown in project databases have little effect on existing (usually manual) notifications from a specific geographic area. Common causes include (earlier) testing of people with TB symptoms who would have presented for TB care in any case; and people with TB symptoms travelling within the encompassing area to seek care from the project. Differing inclusion criteria based on head count or cohort membership may affect the comparison of corresponding monthly, quarterly and annual data between systems.
• Are additional notifications seen at district/provincial/national level? Against a completely stable baseline, even a tiny number of additional notifications would be detectable; but in practice, time-series fluctuate so that small increases are not visible against background noise. Fluctuations follow the granularity of programmatic activities, and simple confidence intervals are not generally helpful at district level. Nevertheless, quantifying and understanding time-axis variation may be important, for example, when the ‘signal’ observed at the centre of an intervention attenuates at progressively larger encompassing areas. Observing such a dose–response effect adds credibility to surveillance efforts as a whole.
• Can prevalence be estimated directly? Successive, adequately powered and well-conducted prevalence surveys are the gold standard of TB impact assessment. In the case of IT interventions, such surveys may result in the identification of local factors that will enable notifications to correlate better and therefore act as a more reliable guide to direct control efforts. National surveys are geographically stratified and weighted to be representative of the whole country, but re-analysis of nearby clusters may be a good starting point.