A demonstration can produce plausible responses with a few carefully selected inputs and still not be operable in a real workflow. Before enabling an assisted capability, the team must be able to answer specific questions: what decision it supports, what errors it can produce, which cases it must not resolve on its own, and how work continues when the result is not useful.
The goal of evaluating AI functions in PHP applications is not to prove that a model responds well in general. It is to verify that a specific function is sufficiently reliable, traceable, and sustainable for a defined process. This requires designing the evaluation before turning the function into an action available to users.
Define the assisted decision and its boundaries
An assisted function must be described as a verifiable unit of work, not as a generic capability to “use AI.” Define the input it receives, the authorized context it may use, the output it must return, and the action that output may trigger.
For example, “classify incoming requests” needs greater precision: a request with a subject, text, and already processed attachments may return a category, a suggested priority, a confidence level, and a brief explanation. The application may use that output to suggest a work queue, but not to close an incident or automatically reject a customer.
- Input: available fields, expected language, data that must be excluded, and permitted context.
- Output: structured schema, valid values, required fields, and the meaning of each category.
- Action: visible suggestion, reversible automation, or action blocked until review.
- Owner: who corrects results, who decides changes, and who is accountable for the process.
Separating these elements prevents a common error: treating a convincing text output as though it were a valid business decision. If the output feeds an automation, validate the format and permitted values first. A response that does not comply with the schema should not proceed as though it were a correct classification.
Classify harm before measuring quality
Not all errors have the same severity. Confusing two internal labels that an operator can correct in seconds is not equivalent to incorrectly prioritizing a critical incident, assigning work to the wrong team, or exposing information that should not have been disclosed.
Establish a failure taxonomy tied to the operational workflow. You can distinguish between tolerable errors, errors that require review, and blocking errors. This classification determines release thresholds and the type of control required.
- Correctable error: requires a quick edit and does not materially alter service, cost, or a person’s rights.
- Reviewable error: may cause delays, rework, or an inappropriate decision; it must go through a person before producing effects.
- Blocking error: affects security, compliance, money, access, contractual obligations, or decisions that are difficult to reverse. The function must not perform that action on its own.
Also define what “not usable” means. An output may be semantically reasonable but arrive too late, fail to comply with the format, omit decisive data, or be impossible to justify with the available context. Counting these cases separately prevents a single accuracy metric from concealing operational problems.
Build a test set that represents real work
The evaluation set must resemble the inputs the system will receive, not a collection of favorable examples. Start with real processed cases, anonymized and minimized where possible. Remove identifiers and unnecessary data, but retain the elements that explain the difficulty of the decision.
Include diversity in content, length, language, writing style, ambiguity, and data quality. Deliberately add edge cases: requests with contradictory information, incomplete text, internal terms, multiple intents, attachments with no useful text, or instructions inserted by a third party that must not alter the application’s behavior.
Label the verdict, not just an ideal response
For each case, there is not always a single correct output. Record an expected response where appropriate, but also label the permitted level of autonomy:
- Correct: a result that can be suggested or executed within the defined boundary.
- Acceptable: an alternative allowed by the process, even if it is not preferred.
- Requires review: the system can assist, but a person must decide.
- Rejection: the function must state that it cannot produce a valid output or that data is missing.
These labels make it possible to evaluate whether the system knows how to abstain. Forcing it to always classify turns uncertainty into an apparently safe response. Properly handled abstention is an operational capability, not an automatic failure.
Measure results by segment and operational cost
The evaluation must reflect the workflow to be improved. Measure accuracy by case type and harm class, the proportion of outputs that require review, unusable results, response time, and cost per execution or per resolved task. A global average may seem adequate while failing precisely in critical or infrequent cases.
Segment results by relevant categories: request type, language, intake channel, length, presence of incomplete data, and priority. Also review false positives and false negatives separately when the classification triggers a work route. In some workflows, sending too much to review is preferable to leaving an important request unattended.
The acceptance threshold should not be “better than the previous version.” It must state the minimum performance each segment needs, which errors are unacceptable, and what review volume operations can absorb.
Set these criteria before changing instructions, context, data retrieval logic, or provider. This prevents tuning the system until it appears convincing on known examples. Keep part of the test set out of daily iterations to verify whether the change generalizes.
Make evaluation repeatable from the PHP application
The implementation must retain sufficient evidence to repeat a test and explain a discrepancy. It is not necessary to store complete personal data to do so. Store a minimized input or protected reference, the context provided to the function, the structured output, the expected verdict, and the observed verdict.
Version the instruction or prompt, the output schema, validation rules, and any logic that selects context. A change to any of these components can alter the result, even if the PHP code calling the service has not changed.
$evaluationRecord = [
'case_id' => 'support-routing-042',
'instruction_version' => 'instruction-version-id',
'context_version' => 'context-version-id',
'output' => $validatedOutput,
'expected_verdict' => 'review_required',
'observed_verdict' => $observedVerdict,
];The example does not replace access controls, retention, and data minimization. If the input contains sensitive information, define what can be sent, what must be masked, who can access the records, and how long they are needed to audit and improve the workflow.
Run the evaluation automatically before releasing relevant changes. A technical deployment may complete successfully and yet the change may not be ready for a functional release. Enable it gradually: first through internal evaluation, then with a limited group or workflow, and with the ability to stop it without interrupting the main process.
Design human review and continuity during failures
Human review must not become an opaque queue of exceptions. Show the reviewer the relevant input, the proposed output, the reason for review, the suggested action, and the tool’s limits. Prioritize by impact and age, and record corrections with categories that make it possible to detect patterns: insufficient context, ambiguous label, formatting error, out-of-scope case, or business rule not applied.
Use these discrepancies to expand the test set and adjust the process, not only to correct the individual case. If review volume exceeds operational capacity, reduce the scope of automation or improve input quality before expanding exposure.
Also prepare an alternative path. If the service does not respond, exceeds the maximum time, returns an invalid output, or does not reach the required confidence level, the application must preserve the work and route it to the existing manual or deterministic mechanism. Validate types, categories, lengths, and permissions before executing actions; limit reversible operations and require confirmation for sensitive ones.
Checklist before enabling the function
- The assisted decision, its inputs, outputs, and action boundaries are documented.
- Blocking errors have explicit controls and do not depend on textual confidence.
- The test set contains anonymized real cases, edge cases, and incomplete inputs.
- Each case indicates whether it should be resolved, reviewed, or rejected.
- Thresholds are measured by segment and account for review, latency, unusable results, and cost.
- Instructions, context, schema, and results are versioned and auditable.
- Human review has context, priority, and a correction process.
- A manual or deterministic alternative exists for failures, invalid outputs, and overload.
With these controls, the assisted function stops being an isolated demonstration and becomes a capability that product, operations, and technology can evaluate, constrain, and improve responsibly.
