Brace for the TEF

The WONKHE blog has produced this summary:

The results of the Teaching Excellence Framework (Year 2), which had been due for release in late May, delayed until after the General Election and then again following the election results, will finally be published this Thursday. After all the waiting, we will finally get our hands on the judgements from one of the most significant, and most hotly contested, policy initiatives of recent years.

When the idea was first mooted in the last Conservative manifesto, written by a junior MP by the name of Jo Johnson, few in the higher education sector thought that a TEF would be seen through to implementation. Many expected Labour to lead the government following the 2015 election, or for the idea to be dropped in coalition negotiations that never took place.

At times, this scepticism from some sector leaders about the will to see a TEF through has bordered on denial – so many predictions were confidently made that it would never happen. And not only has TEF survived, it’s also managed to surmount the shifting political sands around it: Brexit, a new Prime Minister, parliamentary opposition, the passage of a Bill, and now another General Election. With Johnson returned, against the predictions of a ‘Jexit’, the future for TEF also now looks more assured.

Despite the political machinations, Jo Johnson’s original vision for the exercise has remained largely intact. Yes, there have been successive delays in tying the outcomes to fee increases, and there will now also be a review of the whole process in 2019 which was mandated by the final tweaks made to the Higher Education and Research Act.

Nonetheless, TEF will be a distinct evaluation of universities’ ‘performance’ for three main reasons:

  1. Unlike almost all higher education rankings and evaluation exercises, research performance will have no bearing on the outcomes.
  2. Unlike other evaluations, students’ entry grades will not be used as a judge of quality.
  3. Unlike other rankings and evaluations, TEF will provide a judgement of relative, rather than absolute, performance through its data benchmarking process.

There are many other features of TEF that make it distinctive from other rankings and evaluations, for example the ‘split’ metrics, which will evaluate the equitability of student outcomes compared to peers from different backgrounds. And crucially (if rather obviously), TEF is a government-backed and branded evaluation giving it extra media credibility. It’s quite possible that this credibility will have some bearing on prospective students’ decision-making.

The current acknowledged hierarchy of universities in the UK, as shaped by newspaper league tables, is primarily based on the linked factors of the age of foundation, research volume and quality, and students’ entry tariff. These are not indicators of teaching quality. For all its faults, TEF will come closer to representing the quality of teaching, learning and student experience at universities than other ranking exercises because it has not included both of these measures. But we should not forget that the definition of ‘teaching’ in TEF is quite a stretch: it is more student experience, or ‘education’ than ‘teaching’, or ‘learning’ for that matter.

Expectation management

Most universities’ will already know the outcome of the exercise based on the benchmarked data supplied to them in advance of making their submissions. Some are likely to be nudged up from Bronze to Silver, and Silver to Gold based on the provider submissions and the TEF panel’s largesse. From the data we know, it is expected that several prestigious universities will find themselves with Bronze ratings. On the flipside, some modern universities not used to topping traditional league tables are expected to perform very strongly and obtain Gold.

There is an outstanding question about the influence that providers’ written submissions will have on the final outcomes. The HEFCE guidance released late last year stated that the metrics would only create an “initial hypothesis” outcome. It also stated (in bold) that “the more clear-cut performance is against the core metrics, the less likely it is that the initial hypothesis will change” as a result of provider submissions.

Nonetheless, TEF chair Chris Husbands and DfE officials have been at pains to stress the importance of the provider submissions and their potential for influencing changing the initial hypotheses. We will only get a sense later this week of the true balance between quantitative and qualitative measures in final outcomes. If qualitative evaluations appear to carry more influence than initially expected in the HEFCE guidance, Husbands and his panel will have to be sure that they can defend their judgements in a relatively objective manner.

There are thus two ‘rings’ of expectation management which could shape the media narrative on outcomes. On the one hand, observers with some knowledge of the exercise and the expected metric outcomes could have expectations confounded if written submissions are particularly influential. On the other, the general public’s expectations of ‘good’ universities could be confounded if the final outcomes closely match the metrics.

Read more:

As a refresher ahead of the results later this week, Wonkhe’s Ant Bagshaw has produced a beginner’s guide to TEF.

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s