How to measure decision quality with Myaza Trust
Measure decision quality using confirmed downstream outcomes. Review fraud capture and customer friction without silently changing the live policy.
Charles Archibong, Co-founder
· 3 min read
Key takeaways
- An identical outcome replay has no duplicate effect.
- A changed payload with the same key conflicts.
- Incomplete coverage remains visible.
- A shadow experiment does not change live decisions.
- A published improvement has a traceable version.
Measure decision quality using confirmed downstream outcomes. Review fraud capture and customer friction without silently changing the live policy.
Before you begin
Recorded decisions and supported outcome feedback
Authorised reporting access
Use a team member with permission for this control. Check the organisation and environment before changing a setting. Review the current price before a paid check or ongoing enrolment. Screenshots show an example workspace or the public integration reference; they do not show a real customer or a completed paid check.
Set up and use the capability
Step 1: Open Effectiveness and choose a meaningful window
Open Risk Intelligence → Effectiveness with authorised reporting access. Select the evidence window you intend to evaluate. Check whether the report has enough confirmed outcomes and whether the response says its completeness is valid. Not enough evidence is an honest state; it is not zero fraud or perfect performance.
Integration reference: Understand the four headline measures
Step 2: Record what happened after the decision
From your backend, send POST /api/v1/transactions/{transactionId}/outcomes with the final known classification and source. Use fraud, legitimate or inconclusive according to evidence. Include occurredAt and a reason code. Include externalUserId as the ownership guard when available. Keep an idempotency key stable for that exact outcome. Do not label an unresolved chargeback as confirmed fraud.
Integration reference: Record an outcome
Step 3: Keep exact financial evidence with the outcome
When recording loss and recovery, send the applicable currency and exact decimal-string amounts. Keep the confirmed financial event separate from a risk score. Store the transaction and decision reference so the label can be traced to the control that acted. Outcome records are append-only; an unchanged replay is safe, but reusing the key for different data is refused.
Integration reference: Record an outcome
Step 4: Read coverage before judging performance
Review Outcome coverage first, then Fraud captured, Precision and False positives. Low outcome coverage weakens what the other measures can establish. Compare supported segments and per-rule evidence within the same period. Do not call every blocked payment prevented fraud or compare incomplete periods as if they had equal labels.
Integration reference: Read effectiveness evidence
Step 5: Inspect drift and recommendations
Review supported drift, reliability and threshold recommendations. Read the evidence and the next action, not only a headline score. Recommendations cannot change customer decisions automatically. Bounded relationship evidence can help investigation, but it does not replace labelled outcomes. An apparent change may reflect a different customer mix or missing labels rather than a worse control.
Integration reference: Drift, check reliability and relationships
Step 6: Test and publish a deliberate improvement
Backtest the proposed rule or threshold against suitable historical evidence. Use an available shadow experiment when you need comparison without changing live decisions. Review the result, then publish a separately governed rule or system version when authorised. Keep the earlier version and decisions intact. Continue recording outcomes so the improvement can be assessed after activation.
Integration reference: Safe optimisation
Get more from the capability
Automate final dispute and confirmed case outcomes with stable external event IDs, rather than manually labelling only memorable fraud cases.
Track legitimate interrupted customers alongside confirmed fraud to reveal friction.
Review completeness and coverage for every window before using the figures for a policy change.
Test before relying on it
Run the supported Sandbox or simulator scenarios for this product. Where a tool is Production only, check access and scope first, then use only a permitted, deliberately reviewed Production test. A documentation screenshot or a successful page load is not an end-to-end integration test.
An identical outcome replay has no duplicate effect.
A changed payload with the same key conflicts.
Incomplete coverage remains visible.
A shadow experiment does not change live decisions.
A published improvement has a traceable version.
Troubleshooting
All metrics say not enough evidence
Record confirmed outcomes and check the supported window and completeness; do not fill gaps with invented zeroes.
An outcome ownership guard conflicts
Check that externalUserId belongs to the resolved transaction.
A recommendation did not change the policy
That is expected. Review, testing and separate publication are required.
Open the setup and integration references
Read the integration reference
Back to Solutions. Choose the capability you want to activate, then follow its own guide.
Sources
Charles Archibong
Co-founder
Charles Archibong co-founded Myaza Trust. He writes about identity verification, financial technology, and the practical work of building trusted digital services.
