Appendix A — Linkage process

This section provides an overview of the linkage process (see figure under ECHILD database structure) used to create the ECHILD database.

A.1 Data sources

A.1.1 The Department for Education’s National Pupil Database

The Department for Education (DfE) collates and manages data on school children in England in a resource known as the National Pupil Database (NPD) (Jay, Mc Grath-Lone and Gilbert, 2019). For the purposes of the ECHILD linkage, salient points are described below.

Within NPD, each record is associated with natural identifiers (forename, surname, gender, date of birth, postcode) relating to the pupil’s details as known by the submitting organisation at the time the data were submitted. DfE additionally assigns each record in NPD an anonymised Pupil Matching Reference (aPMR). An aPMR is an identifier which is not in itself meaningful: it does not reveal the identity of the pupil. aPMRs are assigned to all records in NPD such that one aPMR should represent one pupil and each pupil should have only one aPMR. This allows records for each pupil to be identified across NPD, both between different datasets and over time, without revealing their identity. As a result, NPD contains a longitudinal record of pupils’ names, addresses, and (potentially) genders over time.

A.1.2 NHS England’s Personal Demographics Service and Master Person Service

NHS England operates the Personal Demographics Service (PDS), a national electronic database of demographic data for patients accessing care in England or services funded by the NHS in England. PDS contains natural identifiers (forename, surname, gender, date of birth, postcode) and NHS number. Each person is assigned a distinct NHS number at birth if born in England, or at the first time of accessing NHS services in England if not otherwise registered. PDS holds a longitudinal record of name and address changes made to NHS services in England over time. This means there may be many records for each person in PDS but all records for a person should be assigned the same NHS number (with some exceptions, see (NHS Primary Care Support England, 2023)).

The Master Person Service (MPS), managed by NHS England (NHS England, 2023l), takes a record of natural identifiers and attempts to link this to a PDS record allowing for some errors and missingness in the recording of the natural identifiers. If this fails and the natural identifiers are sufficiently complete, MPS attempts to link records against a secondary store (MPS bucket) of natural identifiers of persons who previously had contact with the NHS in England and do not have an NHS number, these persons are assigned an “MPS ID”.

The MPS returns a “Person ID” using either:

1. NHS number, if a valid link is found in PDS; or,
2. MPS ID if no link is found in PDS but a valid link is found in the MPS bucket; or,
3. No value if no link is found in either PDS or the MPS bucket.

The Person ID is then encrypted to generate a Token Person ID (TPI), which is not meaningful and does not reveal the person’s identity.

Person IDs, enabling the assignment of Token Person IDs, are also recorded throughout NHS England’s standard data collections, including Hospital Episode Statistics datasets, Emergency Care Datasets, Mental Health Services Datasets, and Community Services Datasets. The “Person ID” is the only routine means of identifying records belonging to the same patient amongst data held by NHS England.

A.2 Linking DfE NPD aPMRs to NHS England TPIs

For the purposes of the ECHILD linkage, the following simplifying assumptions were made:

1. Each aPMR represents at most one real person within NPD;
2. Each TPI represents precisely one real person within NHS England data collections;
3. Each real person represented within NHS England data collections has precisely one TPI.

Essentially, whilst we assume TPIs are perfectly allocated, we only require that aPMRs are not shared. That is, the same real person is permitted to have more than one aPMR.

This linkage task resulted in “N-to-one” links between aPMRs and TPIs: each aPMR linked to at most one TPI but a TPI may have linked to more than one aPMR. However, the vast majority of links made were “1-to-1” (99.4% of linked TPIs linked to only one aPMR).

DfE supplied a “linkage dataset” to NHS England, comprising the natural identifiers (forename, surname, gender, date of birth, postcode) and an aPMR for each record in its NPD.

A.2.1 Linkage Stage 1: Exact link

An initial, simple, linkage stage was used to avoid over-burdening the more resource-intensive MPS trace. Each valid record (e.g., no blank entries) in the linkage dataset was compared to all records in a prepared extract from PDS. A sequential, two part, approach was used. Only records belonging to aPMRs not linked after Part A progressed to Part B. In both parts, an aPMR was considered linked only if all of its associated linkage records were linked to at most one TPI and at least one record was linked to a TPI.

A.2.1.1 Part A: Full forename

A record was deemed linked if compared records had equal (non-empty) values across all of the following fields:

1. Full forename;
2. Full surname;
3. Full date of birth;
4. Full postcode; and,
5. Gender.

A.2.1.2 Part B: Partial forename

A record was deemed linked if compared records had equal (non-empty) values across all of the following fields:

1. First four characters of forename;
2. Full surname;
3. Full date of birth;
4. Full postcode; and,
5. Gender.

A.2.2 Linkage Stage 2: MPS Trace

Only records belonging to aPMRs not linked after Linkage Stage 1 were submitted to MPS. Again, an aPMR was considered linked only if all of its associated linkage records were linked to at most one TPI and at least one record was linked to a TPI.

A.2.3 Linkage results

DfE provided 430M records, comprising sets of identifiers, covering 22.8M aPMRs. NHS England managed to link 22.6M (99.1%) aPMRs to a TPI. 20.1M (88.2%) aPMRs were linked at Stage 1, Part A. A further 1.3M (5.7%) aPMRs were linked at Stage 1, Part B. Stage 2 resulted in the linkage of a further 1.2M (5.3%) aPMRs. Less than 200,000 aPMRs remained unlinked at the end of Stage 2: ~160,000 were not resolvable to any TPI; ~30,000 resolved to more than one TPI and (due to the simplifying assumptions) these were considered invalid and so no link was recorded for any of these aPMRs.

A.2.4 Application of NHS National Data Opt-Outs

The ECHILD Research Database team wished to enable potential participants to opt-out from their data being held within the ECHILD Research Database. Our data suppliers indicated that the only means to (partially) operationalise this was through the non-provision of data held by NHS England relating to participants with a current (at the date of data preparation) NHS National Data Opt-Out (NDOO). This included removing any indication of an identified link between the aPMRs and TPIs for participants with a NDOO. It was, however, not possible to exclude the DfE supplied data relating to these participants. 1.3M (5.8%) of the 22.6M aPMRs for which NHS England had identified a link to a single TPI related to individuals with a registered NDOO as at the date of generation of the data extract.

A.2.5 Linkage Outputs: Pseudonymised bridging file

NHS England produced a pseudonymised bridging file consisting of all aPMRs in the DfE-supplied linkage dataset and their linked TPI (excluding those removed due to the presence of a NHS National Data Opt-Out). All aPMRs that were not linked after Linkage Stage 2, or which related to a participant with a NHS National Data Opt-Out, were omitted from this file.

Administrative Data Research UK (2023) “ECHILD: Linking children’s health and education data for england.” Available at: https://www.adruk.org/our-work/browse-all-projects/echild-linking-childrens-health-and-education-data-for-england-142/.

Benchimol, E.I. et al. (2015) “The REporting of studies Conducted using Observational Routinely-collected health Data (RECORD) statement,” PLoS Med, 12(10), p. e1001885.

Blackburn, R. et al. (2021) “Hospital admissions for stress-related presentations among school-aged adolescents during term time versus holidays in england: Weekly time series and retrospective cross-sectional analysis,” BJPsych Open, 7(6), p. e215.

Blackburn, R. et al. (2022) “COVID-19-related school closures and patterns of hospital admissions with stress-related presentations in secondary school-aged adolescents: Weekly time series,” Br J Psychiatr, 221(5), pp. 655–657.

Boyd, A. et al. (2018) “Understanding hospital episode statistics (HES).” Available at: https://www.closer.ac.uk/wp-content/uploads/CLOSER-resource-understanding-hospital-episode-statistics-2018.pdf.

Clery, A. et al. (2024) “Can administrative data be used to research health visiting in england? A completeness assessment of the community services dataset,” International Journal of Population Data Science, 9(1). doi:10.23889/ijpds.v9i1.2385.

Clinical Practice Research Datalink (2021) “Clinical practice research datalink (CPRD): Hospital episode statistics (HES) admitted patient care (APC) data dictionary.” Available at: https://cprd.com/sites/default/files/2022-02/Data_Dictionary_HES_APC.pdf.

CSCDUG (2023) “The children’s social care data user group (CSCDUG).” Available at: https://cscdug.co.uk/.

Department for Education (2015) “The national pupil database user guide.” Available at: http://doc.ukdataservice.ac.uk/doc/7627/mrdoc/pdf/7627userguide_2015.pdf.

Department for Education (2017) “Exclusions statistics guide.” Available at: https://www.gov.uk/government/publications/exclusions-statistics-guide.

Department for Education (2019) “Unique pupil numbers (UPNs).” Available at: https://www.gov.uk/government/publications/unique-pupil-numbers.

Department for Education (2022) “The school census – what you need to know.” Available at: https://educationhub.blog.gov.uk/2022/10/07/the-school-census-what-you-need-to-know/.

Department for Education (2023a) “Children in need census: Guide to submitting data.” Available at: https://www.gov.uk/guidance/children-in-need-census.

Department for Education (2023b) “Find and explore data in the national pupil database.” Available at: https://find-npd-data.education.gov.uk/.

Department for Education (2023c) “Individualised learner record (ILR) technical documents, guidance and requirements.” Available at: https://guidance.submit-learner-data.service.gov.uk/.

Department for Education (2023d) “NCCIS management information requirement.” Available at: https://www.gov.uk/government/publications/nccis-management-information-requirement.

Department for Education (2023e) “Pupil absence statistics: guide.” Available at: https://www.gov.uk/government/publications/absence-statistics-guide.

Department for Education (2024a) “Alternative provision census.” Available at: https://www.gov.uk/guidance/alternative-provision-ap-census.

Department for Education (2024b) “Apply for department for education (DfE) personal data.” Available at: https://www.gov.uk/guidance/apply-for-department-for-education-dfe-personal-data.

Department for Education (2024c) “Children looked after return: Guide to submitting data.” Available at: https://www.gov.uk/guidance/children-looked-after-return-guide-to-submitting-data.

Department for Education (2024d) “Complete the school census.” Available at: https://www.gov.uk/guidance/complete-the-school-census.

Department for Education (2024e) “Data collection and censuses for schools.” Available at: https://www.gov.uk/education/data-collection-and-censuses-for-schools#guidance_and_regulation.

Department for Education (2024f) “Early years census.” Available at: https://www.gov.uk/guidance/complete-the-early-years-census.

Department for Education (2024g) “Get information about schools.” Available at: https://get-information-schools.service.gov.uk/.

Department for Education (2024h) “Statistics: Children in need and child protection.” Available at: https://www.gov.uk/government/collections/statistics-children-in-need.

Department for Education (2024i) “Statistics: Looked-after children.” Available at: https://www.gov.uk/government/collections/statistics-looked-after-children.

ECHILD (2023) “The ECHILD project.” Available at: https://www.echild.ac.uk.

Emmott, E.H., Jay, M.A. and Woodman, J. (2019) “Cohort profile: Children in need census (CIN) records of children referred for social care support in england,” BMJ Open, 9(2), p. e023771.

Etoori, D. et al. (2022) “Reductions in hospital care among clinically vulnerable children aged 0–4 years during the COVID-19 pandemic,” Arch Dis Child, 107, p. e29.

Feng, Q. et al. (2024a) “Data resource profile: A national linked mother-baby cohort of health, education and social care data in england (ECHILD-MB),” Int J Epidemiol, 53(3). doi:/10.1093/ije/dyae065.

Feng, Q. et al. (2024b) “Data resource profile: ECHILD only-children and siblings (ECHILD-oCSib): A national cohort of linked health, education and social care data on mothers and children in england,” IJPDS, 8(6). doi:/10.23889/ijpds.v8i6.2392.

Gilbert, R. et al. (2017) “GUILD: GUidance for Information about Linking Data sets,” J Pub Health, 40(1), pp. 191–198.

Green, F. et al. (2017) “Who chooses private schooling in britain and why?” Available at: https://www.llakes.ac.uk/wp-content/uploads/2021/03/RP-62.-Green-Anders-Henderson-Henseke.pdf.

Harron, K. et al. (2016) “Linking data for mothers and babies in de-identified electronic health data,” PLoS One, 11(10), p. e0164667.

Harron, K. et al. (2017) “A guide to evaluating linkage quality for the analysis of linked data,” Int J Epidemiol, 46, pp. 1699–1710.

Health Data Research (2023a) “Innovation gateway: Hospital episode statistics accident and emergency (HES a&e).” Available at: https://web.www.healthdatagateway.org/dataset/101a7d68-1675-4b47-bfac-dde7b7b57877.

Health Data Research (2023b) “Innovation gateway: Hospital episode statistics admitted patient care (HES APC).” Available at: https://web.www.healthdatagateway.org/dataset/6599230a-df54-4615-937c-d724d239491f.

Health Data Research (2023c) “Innovation gateway: Hospital episode statistics critical care (HES CC).” Available at: https://web.www.healthdatagateway.org/dataset/d0eeba22-bc3f-4c40-be69-a99f916897a2.

Health Data Research (2023d) “Innovation gateway: Hospital episode statistics outpatients.” Available at: https://web.www.healthdatagateway.org/dataset/d4362d41-0838-4f41-bba3-c3268312c444.

Health Data Research (2023e) “Innovation gateway: Maternity services data set (MSDS).” Available at: https://web.www.healthdatagateway.org/dataset/0681a1e9-3778-4f43-88b3-6540515404b6.

Herbert, A. et al. (2017) “Data resource profile: Hospital episode statistics admitted patient care,” Int J Epidemiol, 46(4), pp. 1093–1093i.

Jay, M.A., Mc Grath-Lone, L. and Gilbert, R. (2019) “Data resource: The national pupil database (NPD),” Int J Pop Data Sci, 4(1), p. 08.

Libuy, N. et al. (2021) “Linking education and hospital data in england: Linkage process and quality,” Int J Pop Data Sci, 6(1), p. 13.

Libuy, N. et al. (2022) “Gestational age at birth, chronic conditions and school outcomes: A population-based data linkage study of children born in england,” Int J Epidemiol, 52(1), pp. 132–143.

Mc Grath-Lone, L., Dearden, L., et al. (2016) “Changes in first entry to out-of-home care from 1992 to 2012 among children in england,” Child Abuse & Neglect, 51, pp. 163–171.

Mc Grath-Lone, L., Harron, K., et al. (2016) “Data resource profile: Children looked after return (CLA),” Int J Epidemiol, 45(3), pp. 716–717f.

Mc Grath-Lone, L., Libuy, N., Harron, K., et al. (2021) “Data resource profile: The education and child health insights from linked data (ECHILD) database,” Int J Epidemiol, 51(1), pp. 17–17f.

Mc Grath-Lone, L., Libuy, N., Etoori, D., et al. (2021) “Ethnic bias in data linkage,” Lancet Digital Health, 3(6), p. e339.

Mc Grath-Lone, L., Etoori, D., et al. (2022) “Changes in adolescents’ planned hospital care during the COVID-19 pandemic: Analysis of linked administrative data,” Arch Dis Chil, 107(10), p. e29.

Mc Grath-Lone, L., Jay, M.A., et al. (2022) “What makes administrative data ‘research-ready’? A systematic review and thematic analysis of published literature,” Int J Pop Data Sci, 7(1), p. 1718.

Morris, H., Lanati, S. and Gilbert, R. (2018) “Challenges of administrative data linkages: Experiences of administrative data research centre for england (ADRC-E) researchers,” Int J Pop Data Sci, 3(2), p. 097.

Nguyen, V.G. et al. (2022) “SEN support from the start of school and its impact on unplanned hospital utilisation in children with cleft lip and palate: A demonstration target trial emulation protocol using ECHILD.” Available at: https://www.medrxiv.org/content/10.1101/2022.04.01.22273280v1.

NHS England (2023a) “About the maternity services data set (MSDS).” Available at: https://digital.nhs.uk/data-and-information/data-collections-and-data-sets/data-sets/maternity-services-data-set/about-the-maternity-services-data-set.

NHS England (2023b) “Birth notification process.” Available at: https://digital.nhs.uk/services/national-care-records-service/birth-notification-process.

NHS England (2023c) “ECDS guidance and documents.” Available at: https://digital.nhs.uk/data-and-information/data-collections-and-data-sets/data-sets/emergency-care-data-set-ecds/ecds-guidance.

NHS England (2023d) “Emergency care data set (ECDS).” Available at: https://digital.nhs.uk/data-and-information/data-collections-and-data-sets/data-sets/emergency-care-data-set-ecds.

NHS England (2023e) “Hospital accident & emergency activity.” Available at: https://digital.nhs.uk/data-and-information/publications/statistical/hospital-accident--emergency-activity.

NHS England (2023f) “Hospital episode statistics data dictionary.” Available at: https://digital.nhs.uk/data-and-information/data-tools-and-services/data-services/hospital-episode-statistics/hospital-episode-statistics-data-dictionary.

NHS England (2023g) “Hospital episode statistics (HES).” Available at: https://digital.nhs.uk/data-and-information/data-tools-and-services/data-services/hospital-episode-statistics.

NHS England (2023h) “Hospital episode statistics (HES) data changes in 2021.” Available at: https://digital.nhs.uk/data-and-information/data-tools-and-services/data-services/hospital-episode-statistics/hospital-episode-statistics-data-changes-in-2021.

NHS England (2023i) “Hospital outpatient activity.” Available at: https://digital.nhs.uk/data-and-information/publications/statistical/hospital-outpatient-activity.

NHS England (2023j) “Introduction to community services data set (CSDS).” Available at: https://digital.nhs.uk/data-and-information/data-collections-and-data-sets/data-sets/community-services-data-set/guidance/introduction.

NHS England (2023k) “Linked HES-ONS mortality data.” Available at: https://digital.nhs.uk/data-and-information/data-tools-and-services/data-services/linked-hes-ons-mortality-data.

NHS England (2023l) “Master person service (MPS).” Available at: https://digital.nhs.uk/services/demographics/master-person-service.

NHS England (2023m) “Mental health services data set (MHSDS).” Available at: https://digital.nhs.uk/data-and-information/data-collections-and-data-sets/data-sets/mental-health-services-data-set/about.

NHS England (2024a) “Hospital admitted patient care activity.” Available at: https://digital.nhs.uk/data-and-information/publications/statistical/hospital-admitted-patient-care-activity.

NHS England (2024b) “Hospital adult critical care activity.” Available at: https://digital.nhs.uk/data-and-information/publications/statistical/hospital-adult-critical-care-activity.

NHS England (2024c) “Mental health data (MHMDS, MHLDDS, MHSDS).” Available at: https://digital.nhs.uk/services/data-access-request-service-dars/dars-products-and-services/data-set-catalogue/mental-health-data.

NHS England (2024d) “Methodological changes.” Available at: https://digital.nhs.uk/data-and-information/find-data-and-publications/statement-of-administrative-sources/methodological-changes.

NHS Primary Care Support England (2023) “Adoptions and gender reassignment.” Available at: https://pcse.england.nhs.uk/help/patient-registrations/adoption-and-gender-re-assignment-processes/.

Office for National Statistics (2023a) “Access the data securely.” Available at: https://www.ons.gov.uk/aboutus/whatwedo/statistics/requestingstatistics/secureresearchservice/accessthedatasecurely#assured-organisational-connectivity-aoc.

Office for National Statistics (2023b) “Secure research service.” Available at: https://www.ons.gov.uk/aboutus/whatwedo/statistics/requestingstatistics/secureresearchservice.

Office for National Statistics (2024) “User guide to birth statistics.” Available at: https://www.ons.gov.uk/peoplepopulationandcommunity/birthsdeathsandmarriages/livebirths/methodologies/userguidetobirthstatistics.

Playford, C.J. et al. (2016) “Administrative social science data: The challenge of reproducible research,” Big Data & Society, 3, p. doi:10.1177/2053951716684143.

Scourfield, J. et al. (2019) “Overview of administrative data on children’s social care in england.” Available at: https://whatworks-csc.org.uk/wp-content/uploads/WWCSC_Overview_of_administrative_data_Oct_2019.pdf.

UK Statistics Authority (2023) “Ethics self-assessment tool.” Available at: https://uksa.statisticsauthority.gov.uk/the-authority-board/committees/national-statisticians-advisory-committees-and-panels/national-statisticians-data-ethics-advisory-committee/ethics-self-assessment-tool/.

University of Bristol (2023) “National pupil database user group.” Available at: http://www.bristol.ac.uk/cmpo/npd-user-group/.