Evaluation

Evaluation helps make the best use of resources and disseminate good practice. Supervised toothbrushing programmes must be evaluated to assess if the programme was implemented as planned and targets for coverage and uptake have been achieved. Furthermore, evaluating supervised toothbrushing programme can capture learning and reflections of key stakeholders, to make recommendations for the future.  

​

​

​

Key evaluation questions and potential sources of information.

 

​Were the supervised toothbrushing programme activities delivered as intended? 

• Quantitative (numerical) data from termly monitoring reports on coverage and uptake.

• Quality audits.

 

Were the short and medium-term outcomes achieved? 

• Using interviews to capture the views of early years/school staff and parents/carers about whether the supervised toothbrushing programme achieved its objectives. 

• Using questionnaires to show changes in oral health knowledge and confidence of staff after undertaking training. 

 

What key learning and insights about supervised toothbrushing programme implementation have been gained?  

• Using data from surveys completed by local oral health improvement staff, any dental practices involved, early years/school staff, parents, and carers. 

• Interviews with key supervised toothbrushing implementation/delivery staff.

 

Long-term outcomes may be evaluated through routinely collected survey or clinical data: 

• National oral health surveys of five-year-olds (e.g., dmft – decayed, missing, filled teeth) which are typically conducted every four years. 

• Number of fillings and extractions carried out by NHS dentists. 

• Number of child dental referrals requiring a general anaesthetic.

 

However, using these outcomes to evaluate a supervised tooth brushing programme requires consideration of the time it takes for changes in tooth decay to be observed (around 24-36 months), the size of the survey sample used and the limitations of existing data sets. 

​

​

​

Key Performance Indicators

 

It is important that monitoring reports, produced by the provider, include information that is specifically related to the supervised toothbrushing programme and does not relate to any other oral health intervention that may co-exist. The provider will need to report on the Key Performance Indicators (KPI) as agreed upon in the service specification. The following are some examples of KPIs that should be included in a monitoring report. 

 

Coverage and Participation 

• Number of eligible settings (according to local authority information).

• Number of eligible children within eligible settings.

• Number of children with consent to participate. 

• How often is supervised brushing taking place (e.g. 5 days per week, 3 days per week).

• Number of participating settings.  

• Number and percentage of participating settings and children in each Index for Multiple Deprivation (IMD) decile.  

• Number of setting staff receiving training each school term. 

• Narrative report of efforts made to engage eligible settings that have declined to participate. 

 

Quality assurance 

• Number of quality audits undertaken in participating settings and method (in person, virtual, self-assessment). 

• Number of suspended settings and reasons stated.

 

Reflections  

• Note any challenges or feedback and efforts made to address these. 

 

​

​

Challenges 

 

Obtaining information directly from local authorities for the following KPIs can sometimes be challenging due to changes in the:  

• total number of eligible settings in each area due to frequent early-year setting closures. 

• total number of registered children within settings. 

 

This can result in the provider having to contact settings directly to ascertain if certain settings are still open and the number of registered children.  

 

To mitigate this challenge there should be a designated contact within the local authority who can provide current information on registered settings and children.