When Public Employment Services Use AI: Legal Risks of Profiling Jobseekers

Public Employment Services (PES) are moving quickly from paper-based intake and manual referrals to digital systems that register jobseekers, predict needs, and match people to vacancies. According to the European Commission’s 2025 capacity review, PES are increasingly adopting digital tools for jobseeker registration, vacancy matching and satisfaction monitoring, and 63% report using AI for profiling or matching. That shift matters because profiling is no longer just an administrative triage tool; when AI shapes who gets job offers, training, outreach, or sanctions-adjacent referrals, it can trigger serious legal exposure. The core risks are familiar but amplified: discrimination, flawed administrative decision-making, data-protection violations, and weak remedies for people who are misclassified by opaque systems.

This guide explains those risks in plain language, while giving students, lawyers, and compliance teams a practical framework for audits, complaints, and litigation strategy. The focus is not whether AI is useful in PES operations; it is whether public authorities can lawfully deploy it, document it, explain it, and correct it when it goes wrong. For readers comparing AI procurement and oversight models, it can help to read alongside our analysis of vendor risk dashboards for AI startups and fairness testing in real-world decision systems. The legal standard is higher when the user is the state.

1. What AI Profiling in Public Employment Services Actually Does

From registration to recommendation engines

In PES settings, AI profiling usually means a system that predicts a jobseeker’s barriers, employability, or likely support needs from data such as age, education, work history, benefits status, location, or prior interactions with the service. Those predictions may determine whether a person is routed to job search help, training, coaching, youth services, or a less intensive pathway. The same system may also rank vacancies, recommend candidates to employers, or flag users for follow-up. The practical effect is that an algorithm can become the first filter in a public service relationship, even if a human caseworker still signs off later.

That is why profiling should not be treated as a neutral technical convenience. It is more like a high-speed case classification system that can harden assumptions into service outcomes. If the system assigns a higher “distance from labor market” score to a young person from a disadvantaged neighborhood, the label may influence what support is offered, how soon, and by whom. Similar risks appear in other automated allocation environments, including AI-driven age verification systems and data-quality-dependent decision systems, where bad inputs can generate confident but wrong outputs.

Why PES adoption is accelerating now

The source report shows that PES are under pressure from changing client demographics, labor-market volatility, staffing constraints, and a push toward skills-based services. A growing share of clients are older, more women are registered, and tertiary education levels have increased, which means the service mix must be more nuanced. At the same time, 81% of PES are identifying green-transition skills and 72% provide green upskilling or reskilling. AI appears attractive because it promises scale, speed, and consistency in a resource-constrained environment. But the same capacity pressures that justify automation also increase the risk of overreliance.

This tension is common in digital public services. Systems get adopted because humans are busy, yet busy services are exactly where governance can slip. The lesson from operational fields like shipping exception playbooks and air traffic controller shortage planning is that high-pressure environments need stronger escalation rules, not weaker ones. PES should approach AI profiling in the same way: as an exception-managed process requiring clear fallback pathways.

AI matching is not just “better search”

Job matching tools are often marketed as smarter search engines, but legally they do more than retrieve vacancies. They infer suitability, suppress options, and shape opportunity. That matters because a mismatch can become self-fulfilling: a person shown only low-skill roles may never be considered for training-linked routes that would improve long-term outcomes. For public bodies, this transforms a convenience tool into a decision-support system with distributive consequences. If the matching model is biased, the service may look efficient while systematically narrowing opportunity.

The operational analogy is a marketplace algorithm that quietly prioritizes certain users and downranks others. We see similar dynamics in AI-powered marketplaces and audience prediction systems. In PES, however, the stakes are higher because the service is public, mandatory for many users, and linked to benefits, training, or employability supports.

2. The Main Legal Exposure: Discrimination and Equality Claims

Direct and indirect discrimination

The most visible risk is that profiling creates unequal treatment. A model may directly use protected characteristics, or it may rely on proxies that reproduce them, such as postal code, education history, language use, benefit history, or frequency of digital contact. Even when a system never sees race or disability directly, it may infer disadvantage in ways that penalize precisely those groups the PES is supposed to support. In practice, discrimination claims often focus less on algorithmic intent and more on unequal outcomes combined with poor justification.

In public employment services, indirect discrimination is especially likely. A model designed to optimize “placement speed” may favor people who already resemble employers’ historical hires, which can penalize women returning to work, disabled jobseekers, older workers, or young people with fragmented histories. The source material notes a rising share of clients aged 55+, a slight increase in women, and stronger youth profiling in the Youth Guarantee context. Those changes make fairness testing essential, because a tool calibrated on yesterday’s client mix may misfire on today’s population. For comparison, see our guide to designing AI-assisted tasks that build skills rather than replacing them, since the same logic applies to public service design: automation should strengthen human capability, not flatten it.

Proxy discrimination and feedback loops

Proxy discrimination occurs when a seemingly neutral variable predicts a protected trait closely enough to reproduce disparities. For example, if a labor-market history variable tracks caregiving interruptions, the model may systematically downgrade women. If a neighborhood code tracks ethnic segregation or poverty, the model may channel applicants into lower-quality services. Over time, these choices can create feedback loops: people receiving lower-intensity support do worse, which “confirms” the model’s initial assumption. That makes the algorithm appear accurate while it is simply entrenching its own bias.

Good audit design should therefore test not only overall accuracy but also subgroup performance, feature sensitivity, and decision cascades. A similar problem appears in retail and marketing systems that optimize for conversion while ignoring downstream harm, which is why our articles on spotting real flash sales and reorder incentives stress visibility into hidden mechanics. For PES, the hidden mechanics are the legal issue.

Evidence collection for equality claims

To support a discrimination complaint, applicants should gather evidence about outputs, comparator cases, and policy documents. Useful proof includes screenshots of profiling results, referral letters, service-pathway records, caseworker notes, and any public description of the system’s inputs or logic. It also helps to obtain data on error rates and service outcomes by age, sex, disability, ethnicity, and other protected groups where lawful and available. If a system is only available to users through an opaque portal, request logs and decision explanations as early as possible.

Lawyers should frame these cases both as individual harm and institutional pattern. One person can show a wrongful referral, but aggregate data can reveal systematic exclusion. If the issue resembles a hiring-screening process rather than a one-off mistake, the strategic model is closer to checking whether employers can avoid hiring mistakes when scaling quickly—see how employers can avoid hiring mistakes when scaling quickly—except that the PES has public-law obligations, not just commercial incentives.

3. Administrative Law Risks: Fairness, Reason-Giving, and Human Review

Reasoned decisions and procedural fairness

Administrative law becomes central whenever AI influences a public decision that affects access to services or support. Even where the algorithm is formally “advisory,” the question is whether humans actually exercise independent judgment or simply rubber-stamp the output. A public authority must generally be able to explain why a person was placed on one pathway rather than another, especially if the decision affects rights, benefits, or significant opportunities. If the authority cannot describe the main factors used, the legal exposure grows quickly.

Procedural fairness also requires intelligibility. A person should not have to reverse-engineer a model to understand why they were treated in a certain way. That is a recurring theme in cases about automated or semi-automated public decisions: the more consequential the output, the more important explanation, review, and correction become. Teams building oversight processes can borrow from the discipline used in surprise patch release response planning, where the first task is to know what changed, when, and who approved it.

Human-in-the-loop is not enough by itself

Many systems claim to include “human oversight,” but that phrase can be misleading. If caseworkers lack time, training, or authority to question the model, oversight is only nominal. A rushed review that simply confirms a score is not a meaningful safeguard. In legal terms, the authority still owns the decision, and so it remains responsible for the lawfulness of the process.

This is a critical audit point: ask whether humans can override the model, whether they are required to justify the override or the acceptance of the result, and whether they see enough information to detect obvious error. In practice, the best controls look like those in a secure-by-default workflow, where safe settings are the default and escalation is deliberate. For PES, that means defaults should favor review, not blind automation.

Recordkeeping and auditability

Administrative law challenges are much easier when the authority has poor records. Unfortunately, that is common in algorithmic systems, especially when procurement, vendor updates, and internal model changes are not documented in a way that legal teams can inspect later. Auditability means more than storing a score; it means keeping the version of the model, the training data summary, feature lists, human override records, performance metrics, and complaint logs. Without this, a service cannot reliably defend itself in review or litigation.

That is why public bodies should treat documentation as part of the decision, not as an afterthought. In the language of operations, you need a clear control surface, much like the difference between control versus ownership in third-party platform lock-in. A PES may buy the tool from a vendor, but it still owns the legal consequences of using it.

4. GDPR and Data-Protection Duties for AI Profiling

Lawful basis, purpose limitation, and minimization

GDPR issues arise immediately because AI profiling depends on the collection and reuse of personal data. A PES must identify a lawful basis, define a specific purpose, and ensure the data collected are relevant and limited to that purpose. “We might need it later” is not a lawful design principle. If the system uses sensitive data, special-category safeguards become even more important, and the service should justify why less intrusive methods would not work.

Purpose limitation matters because public services often inherit data from multiple systems, including benefits, case management, and training platforms. It is tempting to combine all of that into one predictive layer, but broad aggregation can create function creep. A useful analogy comes from the warning signs in vendor risk evaluation: if a vendor cannot explain data flows and dependencies, the buyer cannot responsibly deploy the tool. For PES, this is not just procurement hygiene; it is a compliance obligation.

Automated decision-making and meaningful information

Depending on how the tool is used, GDPR rules on automated decision-making may apply, especially if the system produces decisions with legal or similarly significant effects. Even where Article 22 thresholds are debated in a particular setup, the controller still owes transparency and accountability duties. That means providing meaningful information about the logic involved, the significance of the processing, and the likely consequences. Boilerplate privacy notices rarely satisfy this requirement when a person is actually being profiled.

In practical terms, meaningful information should cover the main inputs, the purpose of the model, the role of human review, how to contest the outcome, and what data are used to train or update the system where relevant. If the explanation is too vague, complainants should ask for records of decision pathways and internal guidance. This is similar to the documentation needed when assessing data quality claims in market feeds: the user needs enough detail to know whether the output can be trusted.

Data subject rights, access requests, and correction

People profiled by PES should be able to exercise access, rectification, restriction, and objection rights where applicable. In many complaints, the crucial move is to request the profiling record itself, not just the final service decision. Ask for the score, the features used, the model version, and any human notes referencing the algorithm. If the authority refuses on the ground that the system is proprietary, that usually does not end the inquiry; public bodies still need to satisfy legal rights even when software is supplied by a vendor.

For practical guidance on documentation packets and evidence preservation, it helps to borrow from process-heavy consumer and logistics playbooks such as inspection-ready document packets. The message is simple: if a dispute is likely, assemble the record before it disappears.

5. What Makes PES AI Harder Than Private-Sector AI

State power and compulsory interaction

PES systems are different from ordinary commercial recommender tools because they sit at the intersection of public power, social protection, and labor-market access. Many users do not freely choose the platform, and they may rely on it to comply with obligations or access support. That makes any erroneous profile more coercive than a mistaken product recommendation. The user may not be able to “opt out” in the same way they would from a commercial app.

This heightened vulnerability means regulators and courts are likely to scrutinize PES profiling more closely than private-sector matching. If the system mislabels a jobseeker as low-potential, the damage may include reduced contact, fewer training opportunities, or delays in placement support. It is the public-sector equivalent of a bad routing decision in a complex logistics chain, which is why supply-chain resilience thinking is useful here; see supply chain resilience stories for a reminder that weak links matter most when the system is under stress.

Unequal digital access and older client bases

The source report notes that PES client bases are aging. That matters because digital literacy, accessibility, and device access vary sharply across age groups and disability status. A tool that works for highly connected users may quietly exclude people who need more assisted pathways. If the AI profile is derived from incomplete interaction data, it may conclude that a person is disengaged when the real problem is accessibility or usability.

These risks are not theoretical. Digital friction can become a source of legal vulnerability when it interacts with social rights. Practical design advice from accessibility-oriented products, such as our piece on supportive design for elderly users, reinforces a key point: systems should be built around actual user needs, not idealized user behavior.

Capacity constraints are not a legal defense

Budget pressure and staffing shortages are real, but they do not eliminate legal duties. If anything, scarce capacity increases the need for conservative deployment, thorough documentation, and targeted human review. Public authorities cannot delegate accountability to a model just because the service is busy. Courts may take context into account, but they will still ask whether the authority had policies to avoid unfairness and privacy violations.

The lesson from operations planning in air traffic control shortage impacts and from anti-deskilling design is the same: stress makes safeguards more necessary, not less. A strained PES should reduce automation risk, not bury it deeper.

6. Practical Audit Framework for Students and Legal Practitioners

Step 1: Map the decision chain

Start by mapping where the AI sits in the workflow. Does it screen new registrants, generate a profile score, rank referrals, recommend training, or influence sanction-like consequences through engagement tracking? Identify which steps are fully automated and which are reviewed by a human. Determine whether the system is used on a one-time basis or continuously updates a profile over time. This map will shape every later legal argument.

A helpful audit question is whether the tool is advisory, determinative, or simply administrative. That distinction matters because the legal standard changes with the practical effect. If the model is effectively binding, the case for due-process style scrutiny strengthens. Think of it as using a workflow audit similar to a real-time inference endpoint: you need to know where tags are added, what they trigger, and whether a human can stop the chain.

Step 2: Test for bias, drift, and proxy features

Request documentation on model inputs, training data, performance metrics, and subgroup error rates. Look for disparities by age, sex, disability, ethnicity, education level, language background, or geography. Test for drift between the population used to train the model and the current PES client base. If the model was trained before the recent shift toward older clients and stronger youth-targeted profiling, its reliability may have deteriorated.

Also examine whether the system uses proxy features that are likely to capture protected traits indirectly. High-risk proxies include location, education gaps, prior benefit history, and digital activity levels. A practical checklist can be built using the same disciplined skepticism recommended in policies for selling AI capabilities and when to restrict use: sometimes the answer is not to deploy at all.

Step 3: Examine notice, consent, and contestability

In PES, consent is often not a realistic basis because the service relationship is unequal and partly compulsory. Focus instead on notice, transparency, and the availability of contestation. Were jobseekers told that profiling occurs, what data are used, and how they can challenge mistakes? Can they obtain a review without unreasonable delay? Is there a clear complaint path, and does the service keep records of corrections?

Contestability should be assessed in practice, not on paper. A complaints form is useless if it goes nowhere or if the same unit that operates the model also adjudicates the complaint without independence. The best process resembles a lifecycle escalation model, the kind discussed in complaint-to-champion lifecycle design, where feedback actually changes the system.

Step 4: Confirm procurement and vendor controls

Procurement files should reveal the service level expectations, explainability commitments, security obligations, audit rights, and model-change notification requirements. If the vendor will not permit inspection of feature logic or update logs, the PES should treat that as a governance red flag. Public agencies need the contractual right to investigate harms and preserve evidence. Without that, the service may find itself unable to respond to a complaint, even when it suspects the system is wrong.

For teams building review files, it can help to borrow the mindset of secure-by-default scripts and inclusive-by-design product responses: insist on built-in safeguards, not optional promises.

7. Remedies and Complaint Strategy

What a complainant can ask for

Affected jobseekers may seek access to their records, correction of inaccurate data, reconsideration of an adverse profile, suspension of automated processing, or a fresh human review. If harm is ongoing, they may also request interim relief from a tribunal or court, depending on the national system. In some cases, evidence preservation should be an immediate request because model logs can be overwritten quickly. Early action matters.

Students and practitioners should prepare a remedies theory before filing. Is the goal to fix one profile, stop a model, force a lawful review, or obtain a declaration that the process is unlawful? Different remedies require different proof. A one-person complaint about a mistaken score can become broader structural relief if the records show systemic lack of auditability. In that sense, complaint strategy is closer to building a campaign than sending a support ticket, as reflected in from complaint to champion.

How to write a strong complaint

A strong complaint should be specific, evidence-based, and legally framed. State the date of the profiling event, the observed harm, the data you believe were used, the requested remedy, and the reasons the process may be unlawful under discrimination, administrative law, and data-protection rules. Ask for the model documentation, policy basis, human review records, and any equality impact assessments. If the authority refuses, that refusal itself may become part of the challenge.

When possible, attach a timeline and a short chronology of events. Include screenshots, letters, and notes from caseworkers. If multiple people experienced similar outcomes, coordinate safely and lawfully to identify a pattern. For evidence organization, the logic is similar to inspection-ready documentation: make it easy for the reviewer to see the problem, the proof, and the requested fix.

When to escalate to data protection authorities or courts

Escalation depends on the nature of the harm and the available forum. If the main issue is unlawful data processing, a data protection authority complaint may be the fastest route. If the issue is a procedurally defective public decision, administrative review or judicial review may be more effective. Equality bodies or labor inspectors may also be relevant depending on the local framework. In some cases, parallel routes are appropriate, particularly where the complaint seeks both record access and substantive correction.

Practitioners should consider whether the case supports a broader challenge to the tool’s deployment. If the AI is used across a region or nationally, a single complaint may have system-wide implications. That is why teams should not wait until litigation to design their evidence strategy. Borrowing from vendor due diligence and fairness testing can make the complaint stronger from day one.

8. A Comparison Table: Legal Risk Areas and What Good Practice Looks Like

The table below summarizes the most important legal risk areas when PES use AI profiling and matching, along with the controls that reduce exposure and the evidence a complainant should seek. It is designed as a quick reference for students, advisors, and compliance teams preparing an audit or legal challenge.

9. Building a Better PES AI Governance Model

Use AI where it supports, not substitutes, judgment

The safest public-sector AI use is usually narrow, assistive, and reversible. Profiling tools should support caseworkers by highlighting issues, not decide the service path on their behalf. Vacancy matching can help surface opportunities, but it should not trap people in a narrow band of “likely” jobs. The aim is augmentation with accountable human oversight, not algorithmic delegation of public discretion.

That principle is consistent with the broader lesson from skills-building AI-assisted task design: tools should expand capability. In PES, that means better service quality, not just faster sorting.

Design for appeal, explanation, and periodic revalidation

Every profiling system should have a clear appeal or review channel, an explanation template for users, and periodic revalidation against current labor-market conditions. Since PES client populations change, models should be recalibrated often enough to prevent drift. A profile built for a pre-change labor market can quickly become inaccurate or unfair. Revalidation should include subgroup analysis, accessibility review, and checks for stale assumptions.

Governance should also distinguish between a technical refresh and a legal re-approval. A model update may improve predictive power while introducing a new fairness issue, so each change should trigger review. This mirrors the discipline in patch management and real-time inference operations, where a small update can have large downstream effects.

Train staff to question, not worship, the score

No governance framework works if staff assume the model is objective just because it is statistical. Caseworkers should be trained to understand false positives, false negatives, proxies, and the limits of prediction. They should know when to override, how to document an exception, and how to escalate suspected bias. Training should also include legal basics so that staff can identify when a user may have a valid complaint.

Pro Tip: In a PES audit, ask not “Does the system have human oversight?” but “Can a trained human realistically detect, explain, and reverse a wrong profile within the service deadline?” That single question often exposes whether the safeguard is real or cosmetic.

10. Bottom Line: What Lawyers Should Remember

The legal theory is converging

When public employment services use AI to profile jobseekers, the legal risks are not isolated. Discrimination law, administrative law, and GDPR usually overlap, and the strongest complaints often invoke all three. If the system cannot be explained, audited, or challenged, the PES is exposed even before a claimant proves a fully discriminatory pattern. Public bodies should assume that documentation gaps will be treated as governance failures.

The source report makes clear that AI use is already widespread in PES profiling and matching. That means the question is no longer whether these systems will be scrutinized, but how well agencies can prove that their models are lawful, fair, and contestable. As public services accelerate digitalization, legal accountability has to keep pace.

What to do next in an audit or complaint

If you are a student, start with the decision chain: who inputs data, who reviews scores, who can override, and what records exist. If you are a practitioner, request the DPIA, procurement file, model documentation, and subgroup performance evidence immediately. If you are preparing a complaint, tie the facts to concrete harms: lost referrals, delayed support, unsuitable job matches, or inaccessible pathways. And if you are advising a PES, treat explainability, auditability, and appeal rights as core infrastructure, not optional extras.

For further perspective on public-service systems under operational stress, see our related explainers on resilience under pressure, restricting risky AI use, and ethical testing frameworks. Those lessons travel well into the legal analysis of AI profiling in PES: if a system affects lives, it needs proof, not promises.

No. AI profiling is not automatically unlawful, but it must comply with discrimination law, administrative law, and data-protection requirements. The legal risk rises when the system materially affects service access, relies on proxy variables, cannot be explained, or lacks meaningful human review. The key issue is not the label “AI,” but whether the process is fair, transparent, and lawful in practice.

2) What is the most common complaint problem in these cases?

The most common problem is opacity. People are told that a system influenced their profile, but they cannot see the inputs, the score, the reasoning, or the review process. Without records, it becomes very difficult to prove discrimination or procedural unfairness. That is why access requests and evidence preservation are often the first steps.

3) Can a jobseeker ask for the algorithm’s logic?

Yes, they can ask for meaningful information about the logic involved, the significance of the processing, and the likely consequences, especially under GDPR transparency principles. They may not always get full source code, but they should receive enough information to understand how the profile was produced and how to challenge it. A refusal to provide useful explanation may itself support a complaint.

4) Does human review fix the legal problem?

Not by itself. Human review only helps if the reviewer has real authority, sufficient time, relevant information, and a genuine ability to override the output. A rubber-stamp review is not meaningful oversight. Courts and regulators will look at how the process works in reality, not just what the policy says.

5) What evidence should lawyers request first?

Ask for the profiling output, the data used, the model version, the human review record, the DPIA, procurement documents, and any equality or impact assessments. Also request retention and audit logs if available. These materials show whether the system was lawful, whether the authority understood the risk, and whether a remedy is feasible.

6) What if the vendor says the model is proprietary?

Proprietary status does not eliminate public-law duties. The PES still has to ensure legality, fairness, and accountability. If the authority cannot explain or defend the system because of vendor secrecy, that is a governance problem for the public body, not a complete shield.

Related Reading

• Vendor Risk Dashboard: How to Evaluate AI Startups Beyond the Hype (Crunchbase Playbook) - Useful for procurement teams reviewing AI vendors before deployment.
• Designing for Fairness: Implementing MIT’s Ethical Testing Framework in Real-World Decision Systems - A practical lens on bias testing and governance.
• When to Say No: Policies for Selling AI Capabilities and When to Restrict Use - Helpful for deciding when a model should not be deployed.
• Preventing Deskilling: Designing AI-Assisted Tasks That Build, Not Replace, Language Skills - Shows how to keep human judgment central in AI-supported workflows.
• From Complaint to Champion: A Lifecycle Playbook to Turn Consumers into Local Advocates - A strong framework for building complaint pathways that actually drive change.