How to evaluate AI-native service management

Every service management vendor now says it has AI. Most of them have a chatbot in front of the same ticket queue, with the same database underneath it. An AI-native platform is one where the agent resolves the request and people handle what it hands off, and that only works if the agent can see what is actually true about your organization. Here is how to tell the difference in a demo.

1. What does the agent know before the request arrives?

Ask the vendor to show you the record for one employee: their manager, cost center, laptop, installed apps, group memberships, and the last time each of those facts was confirmed. Then ask where each field came from. If the answer is "an admin entered it" or "a discovery scan found the hardware," the agent is reasoning from a database that will be wrong in six months. If the answer names your identity provider, HR system and device manager, with timestamps, the platform is maintaining the picture itself.

2. Does it resolve, or does it route?

Take a request that touches three systems, such as a new hire who needs a laptop, an Okta group and a Workday update, with a manager's approval in the middle. Watch what happens. Routing looks like a well-formatted ticket landing in the right queue. Resolving looks like the approval request going to the manager, the three changes executing after it, and a person seeing the result rather than the work.

3. Who maintains the automation a year in?

Ask how many workflows a typical customer has after twelve months and who owns them. Platforms that automate by writing a workflow per case accumulate hundreds, and each one breaks when an upstream tool changes its API or an org chart moves. Ask what happens to a workflow when the identity provider changes an endpoint. Ask whether workflows are versioned and can be rolled back, and whether a non-engineer can read and change one. Then ask to see a workflow that has run unchanged for six months.

4. Can it tell you what a change will affect before it runs?

Pick a change in the demo, such as removing a group or retiring a server, and ask the platform what depends on it. This is the test of whether relationships are first-class data or inferred at runtime. A platform that stores the graph answers in seconds; one that does not will tell you to check the CMDB.

5. Where do approvals and policy live?

Automation without governance is a liability. Ask to see the rule that decides which requests run automatically, which need a person, and who that person is. The rule should be data the IT team controls, not logic buried in each workflow. Ask whether the agent can be prevented from taking an action, not just asked nicely.

6. How is AI priced?

This one tells you what the vendor believes AI is. Metered pricing, by action, credit or "assist," treats AI as a scarce premium feature to be rationed, and it means your bill rises every time the agent does more work. Per-seat pricing for the service desk plus an AI add-on per agent means the AI cost scales with the team you were hoping to shrink. Ask for the price at 20% autonomous resolution and at 80%. If the second number is much larger, the incentives are misaligned.

7. Can your own AI assistant use it?

Your employees already use Claude, ChatGPT or Copilot. Ask whether those assistants can act in the platform through an open protocol such as MCP, as the signed-in user, with the same permissions and the same audit log. Vendors that only offer their own assistant are building a walled garden around work that increasingly starts somewhere else.

8. What happens when the agent is wrong?

It will be, sometimes. Ask to see the log of an AI decision: what it saw, what it did, and why. Ask how a workflow is rolled back. Ask whether model outputs are checked against policy before anything runs. A vendor that cannot show the trail is asking you to trust a black box with production access.

9. How long until it is doing real work?

Ask for the date the agent resolved its first real ticket at a reference customer, counted from contract signature. Months means a traditional implementation with an AI label. Days means the platform built its own understanding of the organization from the tools already connected.

10. What does the vendor commit to?

Ask what happens if the agent is not resolving a meaningful share of requests after the first two months. The answer tells you how confident the vendor is in its own product.

A note on STLabs

We wrote this guide because these are the questions we want to be asked. STLabs is built on Axiom™, a live model of your organization that answers question one; it resolves requests end to end with policy-gated approvals; it charges per employee with no AI meter; and it has an open MCP server. Book a demo and ask us all ten.