Evaluation before and after release
Test cases built from real business scenarios are run before each change and compared with the previous result. Needs: representative cases and expected results.
IT Technical Support
AI System Assurance & Support
REIT Limited, a Bangladesh-based AI services and technical-delivery company, maintains deployed AI and automation systems so that they keep meeting the expectations written down at launch.
What it is
AI system assurance and support is the ongoing work of keeping deployed AI and automation reliable, secure and fit for purpose. It combines task evaluation, permission reviews, injection testing, incident triage, execution monitoring and usage checks with controlled releases, regression checks and rollback procedures.
Why does a deployed AI system need assurance and support?
Models, prompts, data and vendor interfaces change after launch, and quality drifts without anyone noticing. Assurance measures the system against written tests before each change, monitors it in use, and keeps a rollback ready.
An AI system that worked on launch day changes over time. A vendor updates a model, an API changes its authentication, a prompt is edited without a test, a knowledge source goes stale. Output quality drops quietly, and costs rise without anyone noticing.
Assurance treats a live AI system like any other production system: it is tested against written expectations, watched, changed under control, and rolled back when a change makes it worse.
Where does AI System Assurance & Support fit?
Four situations where this service fits, with what each needs from you.
Monitoring and incident handling
Success rates, failures, stalled runs and unusual cost are watched, and incidents are triaged and recorded. Needs: access to execution logs.
Permission and security review
What each agent and integration can reach is reviewed, and inputs are tested for prompt injection. Needs: the current permission list.
Controlled change
Prompt, model and workflow changes are versioned, regression-tested and released with a rollback path. Needs: a named approver for releases.
What does REIT Limited deliver?
A designed, built, tested and documented engagement with written acceptance checks, an operating guide and a handover. This section lists what is included, what is optional, what is excluded and what stays with you.
What REIT Limited delivers
- Task evaluation against written expectations
- Permission reviews for agents, integrations and accounts
- Prompt-injection testing of inputs and retrieved content
- Incident triage, with an incident record and the fix applied
- Execution monitoring and usage and cost checks
- Controlled releases with regression checks
- Documented rollback procedures
- Maintenance reporting and an owned support channel
Optional additions
- Support for the APIs and integrations the system depends on
- Governance records: prompt and model versions, change history and operating policies
Not included
- Guarantees about the accuracy of any AI model
- Systems REIT Limited has not been given access to inspect
- Third-party outages, beyond detecting and reporting them
Stays with you
- Authority to approve releases and rollbacks
- Ownership of the system, its data and its accounts
- Decisions on acceptable quality thresholds
| Capability | What it covers | Typical tools and connections |
|---|---|---|
| Task evaluation | Test cases from real scenarios with expected results; accuracy and completion measured before and after release | Evaluation set, pass criteria |
| Permission review | What each AI system can read, write and trigger, checked against what it needs | Access register |
| Injection testing | Attempts to make the system ignore its instructions or reveal data, run as a standing test set | Adversarial test set |
| Execution monitoring | Success rates, failures, stalled runs, agent behaviour, token use and cost anomalies | Monitor, alerts, usage report |
| Incident triage | Failed or unusual runs logged, classified, fixed and recorded | Incident log, change record |
| Controlled releases | Regression checks before every change, with a one-step rollback | Release checklist, version history |
| Maintenance reporting | A monthly report of quality measures, incidents, cost and an improvement plan | Monthly assurance report |
Supported systems and connections
- Workflows built by REIT Limited
- Agents built by REIT Limited
- Inherited AI systems that can be inspected
- Model provider usage data
- Helpdesk and alerting tools
A connection is confirmed in the assessment, after checking the interface, permissions and vendor limits of each system.
Existing clients raise an incident or a change request by email to support@reit.com.bd or through the support request form. This is the owned support channel named in each assurance agreement.
How does the process work?
The stages are colour-coded the same way across this website: input, processing, human approval, outcome.
- InputSignal or changeAn alert, a failed check or a proposed change arrives.
- ProcessingTriage and testThe issue is assessed and regression checks are run.
- ProcessingPrepare the fixA versioned change and its rollback are prepared.
- Human approvalRelease approvalA named person approves the release.
- OutcomeRelease and reportThe change goes live and the record is updated.
What result can it produce?
One engagement, with the measures agreed before the pilot and reported after launch.
Case studySample case study
An inherited quoting assistant brought under evaluation and change control
Ostrander Insurance Brokers · Insurance broking, United States
- 95%Answer accuracy on the evaluation sethigher, was 81%
- USD 2,090Monthly model costlower, was USD 2,900
- 3 of 3Releases with a regression runhigher, was 0 of 4
The problem
A quoting assistant built by a former contractor gave inconsistent answers after a model update. Nobody could say what had changed or how accurate it was.
What was built
- An evaluation set of 350 real questions with expected answers, run before every release
- Permission review, prompt-injection tests and a log of every model, prompt and retrieval change
- Monitoring of failure rate, token use and cost, with alerts and a monthly improvement plan
Where a person approves
The operations manager signs off each release after the regression run; rollback to the previous version takes one step.
The result
Measured answer accuracy rose from 81% to 95% over three releases, and monthly model cost fell by 28% after unused context was removed.
Systems and tools
- Evaluation harness
- Prompt and configuration repository
- Model provider usage API
- Alerting
- Monthly assurance report
Limit found
Accuracy on rare policy types remains below target because few examples exist; those questions are routed to a broker.
- Before launch
- After launch
- 81%Inherited
- 88%Release 1
- 92%Release 2
- 95%Release 3
View data table
| Period | Value |
|---|---|
| Inherited | 81% |
| Release 1 | 88% |
| Release 2 | 92% |
| Release 3 | 95% |
Sample data
- Sector
- Insurance broking
- Country
- United States
- Size
- 110 staff
- Timeline
- Review 3 weeks, then monthly assurance
What does the published evidence say?
Figures from named surveys and official statistics, each with its source.
27%
of respondents whose organisations use generative AI say employees review all content it creates before the content is used.
McKinsey & Company, 12 March 2025Survey of 1,491 respondents, July 2024
86%
of knowledge workers who use AI say they treat its output as a starting point, not a final answer.
Microsoft, 5 May 2026Survey of 20,000 AI-using knowledge workers in 10 markets
21%
of respondents say their organisations have a mature governance model in place for agentic AI.
Deloitte Insights, 24 April 2026Survey of 3,235 leaders in 24 countries
How is quality controlled?
Through acceptance checks agreed before the build, named approvers for sensitive actions, access limited to what you grant, monitoring with alerts, written recovery steps and a named owner after handover.
- Evaluation
- Failures, output quality and task completion are measured against agreed cases.
- Monitoring
- Failed, stalled and unusual runs raise alerts to the owned support channel.
- Rollback
- Every release has a tested way back to the previous version.
- Records
- Incidents, fixes and changes are logged and summarised in a maintenance report.
- Access
- Permissions are reviewed on a schedule and reduced where they exceed need.
- Limits
- Assurance reduces risk and shows evidence; it does not make an AI system error-free.
What does it cost and how do we start?
Work starts with a paid AI assessment of one process (USD 500, usually 5 working days). A scoped pilot follows (from USD 2,500), then implementation and support, each priced in a written proposal.
AI assessment
USD 500
5 working days
One process reviewed for fit, dependencies and risk.
Scoped pilot
from USD 2,500
3 to 4 weeks
One bounded use case built and tested on real cases.
Production implementation
from USD 6,000
6 to 10 weeks
The accepted pilot hardened, connected and launched.
Sample pricesPrices exclude third-party licences, model usage and taxes. The proposal sets the final price for your scope. Invoices are issued in USD; GBP, EUR and AUD are available on request.
The first step is an AI assessment of one process or one bounded task. It establishes whether the work is a sound candidate, what it depends on, and what a pilot would cover. An enquiry is a request for that conversation; it is not an appointment or an order.
Build work is quoted in the proposal that follows the assessment. Third-party licences, model usage and hosting are recurring costs and are shown separately from implementation. Ongoing support is a separate arrangement with its own scope. See How We Work for how the stages differ.
Questions buyers ask
How is an AI system tested, monitored and rolled back?
Testing uses cases built from real scenarios with expected results, run before and after each change. Monitoring watches success rates, failures, stalled runs and cost. Rollback means every release keeps a tested route back to the previous version, so a change that makes results worse can be reversed quickly and the cause investigated.
What happens when the AI gives a wrong answer?
The case is recorded as an incident, the cause is traced through the logs, and a fix is prepared and regression-tested before release. Where the error could affect a customer, the approval step in the workflow is the first safeguard. No assurance process removes errors entirely; it makes them visible and correctable.
Can you support an AI system that another team built?
Possibly. It depends on whether the system can be inspected: its prompts, workflows, logs and permissions. The assessment reviews what exists and states what can be supported, what must be documented first, and what cannot be taken on. Support starts only once that scope is agreed.
Start with one process
Tell us the process or task you want to improve, the systems it touches and the result you need.
- Assessment before any build
- A person approves customer-facing actions
- Registered in Bangladesh, RJSC No. C-169247/2021