In the second of special series recorded live at the PSE-CEPR Policy Forum 2025, we are asking, how good is AI at doing real-world job task? And how can we measure their capability without resorting to technical benchmarks that may not mean much in the workplace?
Since we all became aware of large language models, LLMs scientists have been attempting to evaluate how good they are at performing expert tasks. The results of those tests can show us whether LLMs can be useful complements to our work, or even replacements for us, as many fear. But setting or grading a test to decide whether an LLM can do a problem-solving job task, rather than solve an abstract problem, isn't easy to do. Maria del Rio-Chanona, a computer scientist at UCL, tells Tim Phillips about her innovative work-in-progress, in which she asks an LLM to set a tricky workplace exam, then tells another LLM to take the test – which a third LLM evaluates.
--------
18:03
--------
18:03
S8 Ep33: Autor: Automation and the value of expertise
Recorded live at the PSE-CEPR Policy Forum 2025.
This year the annual Paris School of Economics-PSE Policy Forum is organized around three themes: artificial intelligence and labour reallocation, working conditions and remote work, and inequality in the workplace. In short, what's work going to look like in the future?
Our series of podcasts, recorded live at the event, starts with David Autor’s work on the impact of AI on jobs. Rather than speculate about how soon AI will destroy work, David’s research focuses on which tasks AI will automate, and what that means in terms of the expertise needed to do these jobs in the future. He tells Tim Phillips that some jobs will become more expert and some less – but the employment effects of AI may be the opposite of what many people expect.
--------
29:40
--------
29:40
S8 Ep32: Bonus episode: From soft landings to hard realities
The Bank for International Settlements Annual Economic Report has just dropped, and there’s a markedly less positive tone than last year, when it was celebrating imminent soft landings in the global economy. It warns of a deteriorating outlook for growth, coupled with vulnerabilities in the global financial system.
So, what exactly is the BIS worried about, how can policy and regulation respond, and should central banks start worrying about the next systemic crisis?
Gaston Gelos and Frank Smets are Deputy Heads of the Monetary and Economic Department at the BIS and are also two of the authors who put together the report. They talk to Tim Phillips about why last year’s optimism has disappeared, and how monetary and fiscal policy can adjust to cope with a new era of uncertainty and fragmentation.
--------
32:14
--------
32:14
S8 Ep31: Does better school management boost test scores?
Whether you are looking at the link from education to economic growth, household earnings or individual happiness, there’s no doubt that a better-educated population is good news. But how can policy improve education in a cost-effective way? You might assume that a good route would be to improve the management of schools, but existing research is not conclusive, and often top-down attempts to improve management meets opposition from administrators.
An experiment in Brazil has evaluated a program to improve management using existing resources in Rio de Janeiro. Tiago Cavalcanti, of University of Cambridge, Sao Paulo School of Economics & CEPR, and Felipe Puccioni of the Court of Accounts of Rio de Janeiro came up with the experiment, and they tell Tim Phillips about why universal education doesn’t necessarily mean universal learning – and how they became celebrities on national television when their successful project hit the headlines.
--------
24:09
--------
24:09
S8 Ep30: Do car bans hurt politicians?
Pedestrianised areas, car-free streets, or low traffic neighbourhoods are increasingly visible in major cities. Whether in London, Paris, New York or Barcelona, these changes are always controversial – but does the loud criticism that we often hear in social media or newspapers really represent the views of voters who are affected by these policies?
Elisabet Viladecans-Marsal of the Universitat de Barcelona, Barcelona Institute of Economics and CEPR spoke to Tim Phillips about whether Barcelona’s car-free “Superblocks” were vote-winners or vote-losers for the city’s mayor.
Photo: Cataleirxs
España