Quality report - further education activity

 

Introduction 

The Department for the Economy (DfE) adheres to the Code of Practice for Statistics and the National Statistician’s guidance on quality.

This document provides a summary of how the following Further Education (FE) publication adheres to the code in terms of quality:

Further education sector activity in Northern Ireland: 2019/20 to 2023/24 | Department for the Economy

The evidence on the quality of this publication is reported in Section 3 and is based on the European Statistical System quality dimensions and principles, most notably the principles on statistical outputs contained in the European Statistics Code of Practice.  A summary of methods used to compile this statistical publication is provided in Section 4 and an assessment of quality using the UK Statistics Authority Administrative Data Quality Assurance matrix is provided in Section 5.

Data Source

Further Education (FE) Colleges are the main providers of further education and training in Northern Ireland (NI).  The sector plays a central role in raising literacy and numeracy levels and in up-skilling and re-skilling the population through a broad range of courses leading to qualifications, particularly at National Qualifications Framework (NQF) and Qualifications and Credit Framework (QCF) levels 2 and 3 and equivalents. Higher Education (at level 4 and above) is also delivered across the FE Colleges.

DfE Statistics and Research Branch (Tertiary Education) uses information from the Consolidated Data Return (CDR) for statistical publication purposes. The CDR is a combined return populated with data from all six of the NI FE Colleges and includes enrolment information for each of their students.  The CDR is facilitated by data consultants employed by the department.

This report outlines the statistical production process from initial collection through to the output of the release.  It identifies potential risks in data quality and how those risks are mitigated. 

Data Quality Report

Dimension – Relevance

The degree to which the statistical product meets user needs in both coverage and content.

Assessment by the Author

Our FE statistical tables and bulletin are designed to provide key points on all enrolments, including regulated enrolments, as well as qualifications and performance.  Performance is provided as the percentage of final year regulated enrolments of students who complete and achieve a qualification.  Information on student numbers is also provided (as one student can have more than one enrolment) and these are broken down further by age, sex and deprivation for example.  Data are also provided on level of education and subject areas and are provided over the most recent five years to allow changes over time to be analysed.

Statistical outputs are used by a variety of customers both internal and external. For example, they are used by DfE policy development officials to monitor existing policies, to allocate funding and for future planning; by other government departments and agencies such as The Executive Office to monitor the reach of FE colleges; by prospective students to inform their choices around further and higher education; and by local businesses to quantify the supply of potentially newly-qualified students in their business area. 

In addition to the bulletin, we provide an extensive range of excel tables for customers to further examine the data. This extends to geographical outputs including by LGD (Local Government District) and by AA (Assembly Area).  We also provide up to date data to the NISRA Data Portal each year, which allows users to create tables and examine their specific areas of interest more closely.  Furthermore, it is clear from the customer feedback, the nature of the questions on FE asked in the Northern Ireland Assembly, and the coverage of FE issues in the local media, for example, that there is great interest in this area, and we can therefore provide ad-hoc bespoke tabular outputs as required.

Further information on how customers’ needs are assessed can be found in the section below titled ‘Assessment of User Needs and Perceptions’. 

Dimension – Accuracy and Reliability

The proximity between an estimate and the unknown true value.

Assessment by the Author

Data Collection

Data are input to the administrative system by the FE Colleges, who are responsible for ensuring the accuracy of the data held on the system.  The FE Colleges use the Tribal system, similar to that used in other FE settings in GB.

In addition to the CDR output, a suite of validations (approx. 200) on the data extract is produced.  Colleges are provided with the outline and assumption for each validation and are required to clear these, which appear as error reports, to ensure data entries are within agreed parameters.

Processing of the CDR, when received by statisticians then includes further validations by way of SPSS syntax.  These validations are in the form of various ‘sense-checks’ of the data via frequencies and cross-tabulations.  If there are any apparent discrepancies or errors in the data, the statistician team will contact the colleges and / or do the following:

• Send a query to the college.
• Use statistical and domain knowledge to inform a quality adjustment to the data.
• Request re-delivery of the data once identified issues have been addressed.

Data declaration

Each October, FE College Principals are required to declare that their end of academic year data is complete and accurate.  The declarations provide assurance that colleges have processed the data in accordance and complied with all policy and guidance circulars and instructions issued by DfE.

The student record covers all students who attend each FE college; therefore, issues of sampling error and estimates are not relevant to this data.  Instead, characteristics of the population in question (students) can be measured directly and comprehensively.  For this data collection, the accuracy of the information returned, and the scale of data are more important.  These issues are kept to a minimum by having in place stringent data quality checks and validations procedures.

Production of statistics

On receipt of data from FE Colleges in the end of year CDR, DfE statisticians produce specific datasets for the FE Sector that are cross verified, across a range of variables.  Prior to publication, DfE’s tables will undergo rigorous checking procedures including peer review of syntax used to analyse data from the CDR database and extensive proof reading of commentary, tables, notes to readers and definitions.

Dimension – Timeliness and Punctuality

Timeliness refers to the time gap between publication and the reference period. Punctuality refers to the gap between planned and actual publication dates.

Assessment by the Author

The CDR collects data from all six NI FE Colleges in the autumn following the academic year to which the data relates.  For each academic year the data collection process opens in August, with a return date of mid-October.  The data quality and validation phase continue through October with final sign off around mid-October.  The CDR is then run to supply the final data to DfE statisticians.

DfE statisticians then spend around eight to ten weeks on validations and data analysis before publishing the final data for the academic year, usually around mid-December, of the same year.  During this analysis period, statisticians develop and quality assure the statistical bulletins.  For the 2023/24 academic year, DfE published the FE Sector Activity Bulletin on 12^th December 2024.   

DfE’s Further Education publication dates are pre-announced through its publication schedule on the DfE website and through the National Statistics release calendar system.

Dimension – Accessibility and Clarity

Accessibility is the ease with which users are able to access the data, also reflecting the format in which the data are available and the availability of supporting information. Clarity refers to the quality and sufficiency of the metadata, illustrations and accompanying advice.

Assessment by the Author

DfE’s Further Education statistics are made available on the DfE website and through a link on the National Statistics publication system. This is supplemented with excel tables on the DfE website and an infographic, which is also shared on the social media accounts for DfE and NISRA.

Internal and external customers who have registered their details with the FE statistician team will have links to the bulletins emailed to them shortly after release.

Tables are available in Excel and open data format. Users can also request further breakdowns and analysis of the data, which along with the tables are provided free of charge.

Dimension – Coherence and Comparability

Coherence is the degree to which data that are derived from different sources or methods, but refer to the same topic, are similar. Comparability

is the degree to which data can be compared over time and domain.

Assessment by the Author

Coherence

The information collected by the CDR regarding students at NI FE Colleges is the only comprehensive source of information on NI FE students and their study choices.

The CDR data for NI FE Colleges covers students who study at any of the six NI FE Colleges and captures all campus locations.  This includes a small number of students who live outside NI.

Comparability

The specification and definitions of data are subject to a major review every few years. From time to time, it is also necessary to change coverage and definitions which can impact the comparability of time series data. When this does happen any discontinuities in the data are made clear and/or data are backdated to take account of these changes.

In each of DfE’s Further Education statistical outputs comparisons are made over a five-year time series.

Dimension – Trade-offs Between Output Quality Components

Trade-offs are the extent to which different aspects of quality are balanced against each other.

Assessment by the Author

The CDR is a consolidated report with data from all six of the NI FE Colleges, with data lifted retrospectively at the end of each academic year.  In order for all the data to be collected on time the data collection process allows some tolerance in respect to missing values for certain variables. This trade-off between timeliness and accuracy allows customers to have information as soon as possible after the period to which it refers.

Dimension – Assessment of User Needs and Perceptions

The processes for finding out about users and uses, and their views on the statistical products.

Assessment by the Author

DfE-based NISRA statisticians ensure that DfE’s Further Education statistical products are relevant to its customers in a number of ways.  Discussions and briefings are held with key policy customers in DfE each year to ensure that the outputs are published in a relevant, accurate and timely manner. Views are also sought on any improvements or changes that customers would like to see implemented and where this is possible it can be incorporated in the next publication. We also carried out a wider readership survey in 2019 to review user’s information requirements and to look at potential future improvements.  We plan to repeat this exercise in due course.

All CDR data collected are subject to a major review cycle every few years to ensure relevance for the current and future needs of customers.  DfE statisticians feed into these reviews, ensuring that the requirements of NI government are addressed.  In addition, any amendments to the system, either new variables or optional changes to responses are agreed with the department and colleges before implementation.  A circular is also sent to each college outlining the changes, for compliance.  For example, in 2024/25 there has been an amendment to the ethnicity responses, in order to bring the answer options in line with those in the Census.

Dimension - Performance, Cost and Respondent Burden

The effectiveness, efficiency and economy of the statistical output.

Assessment by the Author

Much of the costs involved in producing and maintaining the datasets and tables would be incurred regardless of whether or not the publications are produced.

The information used to produce these statistics is collected as part of the application, admission and administrative process within the FE Colleges. Therefore, respondent burden on the students is minimal.

Dimension – Confidentiality, Transparency and Security

The procedures and policy used to ensure sound confidentiality, security and transparent practices.

Assessment by the Author

The following procedures are in place to ensure that data are kept secure and that information released does not identify individuals:

Data are supplied by FE Colleges to DfE Sector Performance Team via a secure web-based transfer system created and maintained by DfE. DfE Sector Performance Team supply data to DfE statisticians via a secure file transfer system. The data are held in DfE on a secure network that can only be accessed by FE statisticians.

To prevent the identification of individuals, figures in publications are rounded to the nearest 5, in line with DfE Statistics and Research Branch (Tertiary Education) rounding strategy, with 0, 1 and 2 rounded to 0. Percentages are rounded to the nearest integer.

Summary of methods used to compile the outputs

Data Collection

Data are supplied by FE Colleges to DfE via a secure web‐based transfer system created and maintained by data consultants employed by the colleges and the department. The data supplied are subject to an extensive quality assurance process.  The colleges use the Tribal system to collect and collate data as part of their everyday operational systems.

The first stage of the quality assurance process in Tribal includes a suite of validation checks, which ensure that the data collected meet specification, dates fall within expected ranges and the information provided within fields of data are consistent. Failures at this stage will cause an entry on a college specific error report, which needs to be fixed before the data can be re‐submitted.  The second stage of quality assurance comprises a verification process whereby frequency counts and cross tabulations are produced via SPSS by DfE statisticians, and any issues are fed back to colleges for verification.  Year-on-year comparisons provide a summary of changes and the level of change in any particular area is examined closely if it falls outside of an expected range.  The final data from each college is signed off by the college principal as being correct.

Contracts in place between the colleges and the department require that the data be of sufficient quality for statutory users' funding and policy purposes and sanctions may be applied should these standards not be met. The quality standards set by statutory users are deemed adequate for the purposes of production of Official Statistics.

Production of statistics

On receipt of data from the FE Colleges, DfE statisticians produce an annual bulletin which outlines all activity at all levels within FE Colleges in NI.  The FE Sector Activity Bulletin is an overarching publication, from which further programme specific data can be extracted, such as for the Essential Skills Bulletin.  Prior to publication, DfE’s statistical tables will undergo rigorous checking procedures including peer review of syntax used to analyse data and extensive proof reading.

Administrative data quality assurance

An assessment of the NI FE Sector Activity Bulletin administrative data was also carried out in accordance with the UK Statistics Authority Administrative Data Quality Assurance matrix and associated toolkit.

The toolkit and matrix are the UK Statistics Authority regulatory standard for the quality assurance of administrative data. The Standard recognises the increasing role that administrative data is playing in the production of official statistics and clarifies what producers of official statistics should do to assure themselves of the quality of these data. The toolkit that supports it sets out the standards for assessing statistics against the Code of Practice for Statistics and the matrix approach to assessment has two components; namely, separate assessments of public interest in our statistics (low, medium, high) and data quality concern about our statistics (low, medium, high). 

	Public interest profile
Level of risk of quality concerns	Lower	Medium	Higher
Low	Statistics of lower quality concern and lower public interest	Statistics of low quality concern and medium public interest	Statistics of a low quality concern and higher public interest
	[A1]	[A1/A2]	[A1/A2]
Medium	Statistics of medium quality concern and lower public interest	Statistics of medium quality concern and medium public interest	Statistics of medium quality concern and higher public interest
	[A1/A2]	[A2]	[A2/A3]
High	Statistics of higher quality concern and lower public interest	Statistics of higher quality concern and medium public interest	Statistics of higher quality concern and higher public interest
	[A1/A2/A3]	[A3]	[A3]

We have assessed the statistical outputs as A2.  The risk of quality concerns has been assessed as medium due to the fact that data are collected from multiple sites and there is an intermediary data supplier.  However, there are safeguards in respect of audit, operational and validation checks. The public interest profile has been assessed as low to medium due to the relatively limited media interest.

Further Quality Assurance measures

Metadata

Full metadata on the data used is available online on the DfE website, in each of the excel tables. This explains the data including many variables and derived variables.

Quality report published

This quality report is updated regularly and published on the DfE website.

Risk Assessment complete

Statistics produced by Analytical Services in DfE undergo an annual risk assessment in order to constantly improve quality assurance measures.

QA Toolkit

The Further Education statistics team within Analytical Services in DfE also use the Administrative Data QA Toolkit to ensure a high level of quality in the four areas associated with data quality.