AI in Workplace Safety: An Evaluation Brief

Define the intended use and responsible person

Distinguish a suggested report category, a summary, a risk indication and a decision that changes work. Specify the inputs, intended user and reviewer. NIST’s AI Risk Management Framework provides voluntary context for evaluating AI; it is not a certification of a software product.

NIST: AI Risk Management Framework. Voluntary AI risk guidance; no evidence of a particular product’s safety performance. Checked 20 September 2026.

Use a representative evaluation set

For a text-classification feature, include clear reports, ambiguous wording, different languages and incomplete records resembling the intended work. Keep evaluation examples separate from tuning where possible. Record reference labels, disagreements and conditions covered. No universal record count proves adequate performance for every task.

Measure mistakes and correction work

Count incorrect alerts, missed cases and outputs a reviewer cannot assess. Examine whether errors differ by work setting or language. Time verification and correction as well as generation. A fast suggestion may create more review work, and a plausible summary can omit an important qualifier.

Agree access, retention and escalation

Ask what data leaves the account, which parties process it, whether it is used for training and how deletion works. Identify who may view sensitive records and override an output. Agree a fallback for unavailable or uncertain results. Involve the responsible workplace, privacy and security specialists.

Recheck after changes

Retain the tested version, configuration and evidence. The NIST Playbook offers suggested actions organised around governing, mapping, measuring and managing AI risk. Use that context to agree when changed models, data or work require a new evaluation. Our demo questions are an editorial application, not a prescribed NIST acceptance test.

NIST: AI RMF Playbook. Voluntary suggested actions; no safety-product ranking or universal pass mark. Checked 20 September 2026.

Ask for the limits of the feature in writing

A feature description tells you what a vendor hopes the feature does. Ask for a statement of what it is not designed to do: the languages and input types it was built for, the kinds of report it handles poorly, whether it works on photographs or only on text, and which version of the model is running in your account. If the answer is that the vendor does not know, record that. A written limit gives your reviewers something to test against and gives you a reference when results change after an update.

Watch for reviewers who stop reading

When a suggestion is usually right, a busy reviewer will start accepting it without comparing it to the source text. Decide in advance how you will notice this: sample a number of accepted suggestions each month and check them against the original report, track how often reviewers change an output, and keep the original wording visible beside every generated field. A falling override rate can mean the feature has improved or that people have stopped checking, and only a sample of accepted outputs can tell the two apart.

Keep the human fallback practised

If a feature is later unavailable or withdrawn, the team still has to do the work. Keep the manual route documented and have a few people use it occasionally, so that the skill and the forms are not lost. Include this in the exercise rather than assuming it, by asking the reviewer to complete a record without the feature and noting how long it takes and what is missing.

Check which rules apply to the intended use

The European Commission's page on the AI Act describes four categories of risk and lists AI tools for employment and the management of workers among the high-risk uses, while emotion recognition in workplaces appears among the prohibited practices. When we opened it the page gave 2 December 2027 as the date from which high-risk systems, a category that includes employment uses, face strict obligations. Ask the vendor which category it considers the feature to fall in and why, and ask your legal adviser to confirm. A feature that only suggests a report category is a different matter from one that scores individual workers, and the evaluation should say which of the two is being bought.

European Commission: the AI Act. The Commission's policy page. Dates and scope can change; confirm the current position before relying on this summary. Checked 3 October 2026.

Treat personal data and fairness as part of the test

The UK Information Commissioner's guidance on AI and data protection has chapters on accountability, fairness, bias and discrimination, accuracy, and Article 22 on automated decisions. Free-text incident reports often name people, so the feature's inputs are personal data in many deployments. Ask whether a data protection impact assessment is planned, whether the output ever changes a decision about a named person without review, and who answers when a worker disputes a generated statement about them.

ICO: guidance on AI and data protection. UK data protection guidance, which the ICO marks as under review after the Data (Use and Access) Act. It does not assess any product, and other jurisdictions differ. Checked 3 October 2026.

Read generative features against the NIST profile

NIST published a companion to its AI Risk Management Framework for generative AI, NIST AI 600-1, on 26 July 2024. It is voluntary and cross-sectoral. If the feature writes summaries or answers questions in free text, use it as a prompt for the evaluation set: which risks specific to generated text could occur in your reports, and which test in your exercise would reveal each? Record the ones you decided not to test and the reason.

NIST: Generative AI Profile (NIST AI 600-1). NIST's publication page. It is not a certification and does not score any vendor. Checked 3 October 2026.

Keep a reviewer log during the exercise

During the exercise, log each item with the input, the suggestion, what the reviewer did and, if they changed it, the reason. After a first batch the log shows which kinds of report the feature misreads and which corrections took longest. That is a more useful basis for a decision than a single accuracy percentage, because it tells you where the review effort will go and which cases need a person to read the original text.