Labour market statistics have served governments and researchers well for decades. But they were designed to answer the ‘how many?’ question. They count the unemployed, the employed, those economically inactive. What they cannot answer is the more difficult and more consequential question: which workers are most at risk right now, and what can we do about it before it is too late? For example, the unemployment rate is, by definition, a retrospective measure. It counts people who have already lost their jobs. As a tool for prevention, it is close to useless.
The combination of machine learning and linked data offers a fundamentally different approach – one that shifts the unit of analysis from the aggregate rate to the individual worker, and the temporal orientation from the retrospective to the anticipatory. The data architecture underpinning this approach brings together data sources that are rarely used in combination (i.e. labour data, macro data, Google, search data, remote sensing and geospatial data, among others). For example, national household surveys, such as the Labour Force Survey, provide rich individual-level information: hours worked, contract type, occupation, qualifications, job satisfaction and the subjective experience of work that administrative records can never fully capture. On the other hand, administrative data such as tax records, payroll information and benefit claims provide information that surveys routinely miss. Used separately, each has significant blind spots. Linked together using anonymised identifiers, they create a picture of individual working lives with a resolution that neither dataset alone can approach.
Machine learning works on this linked data by learning from history. A model is trained on records where the outcome is already known: Which workers experienced job loss in the following year? Which moved into in-work poverty? Which left the labour market entirely? The model identifies which combinations of individual, job, employer and local labour market characteristics are predictive of each outcome, drawing on dozens of variables simultaneously in ways that conventional regression analysis cannot. Applied to available data, it produces a risk score for each worker. This is not a categorical label, but a probability that reflects the full complexity of their circumstances.
This is a qualitatively different kind of knowledge. First, it reveals who is at risk in ways that aggregate statistics and even standard cross-tabulations miss. The risk profile of a 45-year-old, part-time, mid-skilled worker in a contracting local industry is distinct from that of a 25-year-old in the same occupation in a growing one. Conventional analysis flattens these distinctions. Machine learning preserves them and can detect interactions between characteristics that no analyst would think to test. Second, it reveals where risk is located. Risk can be mapped at fine geographic levels, uncovering local concentrations of vulnerability that disappear inside regional or national averages. Third, it reveals when risk emerges. Because the model is trained on longitudinal data and applied prospectively, it functions as an early warning system, generating signals of emerging distress before they crystallise into the headline figures that currently trigger intervention.
What this looks like in practice is a shift in the kind of intelligence available to policy makers. Instead of a quarterly statistical release showing that unemployment in a given region is running at a particular rate, decision makers would have access to a map identifying the workers currently carrying the highest predicted risk characterised by employment profile and ranked by vulnerability. Active labour market programmes, skills support and in-work assistance can reach people before they lose their jobs rather than after. This is not a trivial change. The difference between early intervention and crisis response is, for many workers, the difference between a temporary setback and long-term scarring.
None of this comes without caveats. The approach is only as good as the data underlying it. Informal work, unpaid care and gig arrangements remain poorly captured in both surveys and administrative records, which means the workers least protected by existing institutions may also be those least visible to these methods. Also, linking national survey and administrative data to generate individual risk scores raises serious questions about consent, transparency and the potential for harm. Beyond the legal questions sit deeper ones about power. Risk scores generated to support workers could, under different institutional conditions, be repurposed by employers to identify those worth investing in or those worth letting go. Who controls this data, who has access to its outputs and under what conditions are questions that demand active involvement from trade unions, civil society and the workers whose working lives the data represents.
There is also the question of what risk scores cannot tell us. A model can flag that vulnerability is concentrated in a particular postcode or occupational group. It cannot explain why, and explanation is what policy ultimately requires. Techniques that decompose the contribution of individual variables to a model’s predictions now make it possible to explain, in accessible terms, why a given worker carries a particular risk score. However, these are not enough by themselves. Hybrid approaches that combine machine learning risk profiling with targeted qualitative research can bridge the gap between pattern and cause. The model tells you where to look while ethnographic and participatory methods tell you what is actually happening there.
The solutions lie in governance, not in abandoning the approach. Statutory purpose limitation should prevent data assembled for worker support from ever being accessed by employers or parties with adverse interests. Workers identified as high-risk should have a legal right to understand, in plain terms, what has driven their assessment and to challenge it. Independent algorithmic oversight bodies, with authority to audit model deployment and investigate misuse, would provide the institutional check that self-regulation cannot. Mandatory fairness audits can catch the reproduction of historical inequalities before they shape resource allocation. Worker data trusts, governed through trade unions or civil society bodies, offer a mechanism for genuine collective stewardship rather than individual consent that power imbalances easily overwhelm. The workers being modelled must have a structural role in how these tools are built, governed and held to account.
Luis D. Torres is an Associate Professor at Nottingham University Business School in the UK and the Universidad del Desarrollo in Chile. He is also a Data Science Consultant for the United Nations, Finance Director for the European Academy of Occupational Health Psychology and co-founder of Sustaina Value.
Image credit: Claudio Schwarz via Unsplash
