Service intake
Most facilities AI is sold on a demo, and a demo cannot tell you whether the product runs in a real portfolio, who keeps it accurate, or what it costs over three years. Seven questions, each with the evidence to ask for, will tell you far more than a feature list.
In this guide, we explain why facilities AI needs its own checklist, walk through the seven questions and the red flags behind each one, and show a 30-day evaluation you can run on a single workflow.
Key takeaways
- A live agent has a named customer, a defined workflow and a record of every action it took.
- Ask each of the seven questions in writing, and ask for the evidence behind each answer.
- Check who runs the agent after launch, because accuracy depends on tuning as well as on the model.
- Test one workflow for 30 days on your own records, and measure a baseline before you start.
- Compare vendors on a three-year cost that includes platform, delivery and running cost.
Why does facilities AI need its own evaluation checklist?
Gartner predicts that more than 40% of agentic AI projects will be canceled by the end of 2027, because of rising costs and unclear business value. It also estimates that only about 130 of the thousands of vendors selling agentic AI are genuine, and that many others have relabelled assistants and chatbots as agents.
For a facilities leader, that means the risk sits in the buying decision. A general checklist covers security and price, but it does not ask whether an agent can dispatch a technician, read a vendor contract or work inside the permissions of your CMMS. The seven questions below are written for that job.
What are the seven questions to ask an AI vendor?
Each question has a reason behind it, a piece of evidence you can request, and a red flag that should slow you down.
| Question | Why it matters | Evidence to ask for | Red flag |
|---|---|---|---|
| 1. Is it live today? | A roadmap item cannot run your work orders next quarter. | The agents in production, the workflow each one handles and a customer you can speak to. | The answer is a demo script and no customer. |
| 2. Who runs it after launch? | Accuracy depends on someone tuning rules and reviewing exceptions. | The named team that builds, tunes and supports the agent, and what your team is expected to do. | Support ends at go-live, or it is handed to a third-party integrator. |
| 3. What happens when it is wrong? | Every agent makes mistakes, so the correction path decides the risk. | How errors are flagged, who approves, how corrections are made and how accuracy is reported. | There is no approval step and no record of what the agent decided. |
| 4. What does it touch, and who owns the data? | An agent acts on your records, so access needs to match your policies. | The systems it reads and writes, its permissions, where data is held and how customers are isolated. | The agent has wider access than the person who asked for the task. |
| 5. How long until the first outcome? | A long setup delays the value you are paying for. | A plan for one workflow with a start, a review point and a success measure. | The timeline depends on a full platform rollout first. |
| 6. Is there proof at a comparable customer? | Results at a similar portfolio are the best predictor of yours. | A customer story with the workflow, the period and how the measure was defined. | The numbers have no period, no definition or no named customer. |
| 7. What is the three-year cost? | Licence price alone hides delivery, tuning and running cost. | One figure covering platform, delivery and running cost for the workflows in scope. | Tuning and support are billed separately and not quoted. |
How do you read the answers?
The table tells you what to ask. This section explains what a strong answer sounds like, grouped by the decision it supports.
Capability: Is it live, and who keeps it accurate?
A strong answer to the first two questions names a customer, a workflow and a team. For example, an agent that is live for invoice checks should come with the purchase orders it compares against and the person who reviews its exceptions. A platform built for facilities, such as the one we describe in why facilities needs a vertical agentic platform, brings that knowledge on the first day, while a general platform learns it at your expense.
Control: What happens when it is wrong, and what does it touch?
Look for an agent that acts as the user and inside that user's permissions on the records in your CMMS, with approval steps for actions that carry risk and a trace of each action that a reviewer can replay. If the vendor cannot show you a single run from start to finish, the audit trail is a promise and not a feature.
Delivery and value: How fast is the first outcome, and who has seen it?
Ask for one workflow, one review date and one measure, then ask to speak to a customer who has run something similar. For the cost question, request a single three-year figure and ask what it excludes, so that you can compare vendors on the same basis.
How do you run a 30-day evaluation?
Answers on paper are the start. The strongest evidence comes from a small trial on your own records.
- Pick one workflow: Choose a recurring task such as invoice checks or work order dispatch, with enough volume to matter and rules you can state clearly.
- Measure today: Record the time per task, the error rate and the number of exceptions, so that every later result has something to be compared with.
- Run it on real cases: Use your own records and a sample you did not prepare for the vendor, and ask to see each exception with its reason.
- Compare and decide: Score the vendor's answers to the seven questions against what you saw, and use the measures in our guide to measuring AI agent performance to judge accuracy, exceptions and hours returned.
Take the seven questions to a live demo.
Our FM Ops Engineers answer each one on a live portfolio and show the evidence behind it.
How does Facilio answer the seven questions?
We run our AI agents for facility operations with FM Ops Engineers who build, tune and support them with your team. This is how we answer each question, so you can hold other vendors to the same standard.
| Question | Our answer |
|---|---|
| 1. Is it live today? | Our invoice processing and contract validation agents are live at Charter Hall, and our dispatch and vendor coordination agents run in production. |
| 2. Who runs it after launch? | Our FM Ops Engineers build, tune and support the agents with your team through rollout and afterwards. |
| 3. What happens when it is wrong? | Soft flags go to a person with the reason, hard failures return to the vendor with an explanation, and every decision is recorded. |
| 4. What does it touch? | The agents work with the records in your Facilio platform, under the same security controls as the CMMS, including regional data residency and per-customer isolation. |
| 5. How long until the first outcome? | We scope one workflow with a review point and a measure, and we confirm the timeline during scoping and not in advance. |
| 6. Is there proof at a peer? | Charter Hall, a diversified property funds manager, runs invoice processing and contract validation on one operating layer across its portfolio. |
| 7. What is the three-year cost? | We scope cost by workflow, and we include delivery and running cost in the figure we share with you. |
How is Facilio different from the tools you use today?
The bottom line: What should you do first?
Send the seven questions to every vendor on your shortlist and ask for written answers with evidence. Then choose one workflow and run the 30-day evaluation on your own records.
The vendors that can answer clearly, name a customer and show a trace of each action are the ones worth taking further.
See the AI agents live
Put our answers to the seven questions to the test
Talk with our FM Ops Engineers about your workflow, and we will show what runs on a live portfolio and what stays with your team.
Frequently asked questions about evaluating facilities AI
How do I tell whether a facilities AI agent is live or only a demo?
Ask which customers run the agent in production today, for which workflow, and whether you can speak to one of them. A live agent has a named customer, a defined workflow and a record of what it did. A roadmap item has a demo script and no customer.
What should an AI vendor be able to show me in a pilot?
The vendor should run the agent on your own records for one workflow, show the accuracy against your team's verdict, list the exceptions it handed to a person, and give you the trace of every action it took.
What should an audit trail for an AI agent include?
It should show what the agent read, which rule or record it compared against, what it decided, who approved the decision when approval was needed, and when each action happened, so a reviewer can replay any run.
How long should a pilot of facilities AI take?
One workflow over about 30 days is a practical target for an evaluation. The right length depends on how ready your data is and how many exceptions the workflow produces, so agree the length and the success measures before you start.
Why ask for a three-year cost?
A three-year figure that combines platform, delivery and running costs lets you compare vendors on the same basis and shows whether the cost of tuning and support is included or billed separately.
About the Author
More from Facilio
