Most companies do not fail at generative AI because the technology cannot do it. They fail because nobody can say which process deserves an assistant, and nobody agreed beforehand what success would look like. So the thesis produced two things: a method for picking the process, and a method for proving whether it worked.
What I did
- Change-request classification and incident pattern recognition, both live in production
- 14 use cases evaluated. 2 shipped. Impact measured.
How I found the candidates
The first attempt was interviews and a questionnaire. Two and a half weeks, 75 percent drop-out, and one pattern that explains it: the departments with the highest automation need have the least time to answer questions about it. Experts also tend to propose the process that annoys them rather than the one with the most leverage.
So I put the selection framework into a Rovo agent as its system prompt and pointed it at the process wiki. It came back with candidates nobody had named in the interviews, because they looked too trivial or sat too deep in the documentation. That is the point where the matrix stopped being a paper exercise. It is machine-readable, and that is what makes it scale.
How I picked the process
Every process gets scored on three axes and the weighted result decides, not enthusiasm. Data readiness and process fit count 40 percent each, organisational feasibility 20. The reasoning is that a motivated department cannot compensate for missing data, but bad data will sink a project no matter how motivated everyone is.
- Data. Where does the information actually live, on a scale from people's heads and paper up to a database or an API. Plus whether it is current and consistent, whether there is history to learn from, and whether a technical interface exists at all.
- Process. How often it runs, how stable the rules are, and what decisions are based on: gut feeling, experience, or recognisable patterns. Plus how much of the work is actually text.
- Organisation. Are the people who own the process available in the project window, and how much value do they themselves expect from it.
One criterion runs the other way on purpose. If a process follows purely rule-based logic, its score goes down, not up. That case does not need a language model, it needs an if statement, and scoring it as a win is how organisations end up putting AI on problems that a rule already solves.
What the agents check
The change assistant verifies seven mandatory fields on the ticket, the link to the correct configuration item in the CMDB, and whether the approver selection follows the rules. The problem assistant searches for similar incidents and scores them: three to five is a possible cluster, more than five or three within four weeks is a confirmed one, and at that point it recommends opening a problem ticket. The pile it works on is 37,795 historical incidents with around 451 new ones a month.
How I measured it
- A before and after evaluation on time, cost and quality, extended with established acceptance instruments so that user judgement is a number and not an anecdote.
- The incident assistant cut analysis from roughly 30 to 60 minutes down to about two.
- The change assistant did the opposite. It cost time and paid it back in field completeness, which turned out to be the metric that mattered in that process.
- A sample of 15 fully traced incidents: 60 percent reached the right team straight away, 13 percent went through three or more reassignments, and around 80 percent were never linked to the similar cases that already existed.
What I took away
Two assistants, two opposite results, and a single success metric would have declared one of them a failure. Whichever number you pick up front decides what you are allowed to see afterwards. The other finding was less comfortable: what held adoption back was almost never technical feasibility, it was process readiness and the organisation around it.