We Need a Service Desk Automation Benchmark, People

Everyone who makes service desk automation software says their AI is the best. A benchmark would help MSPs decide who’s right.

If you’re an MSP eager to capitalize on the productivity-boosting power of AI automation (and if you’re not, you might want to get eager to avoid falling way behind your peers), there have never been more options for realizing that goal. Which is a blessing, of course, but also a curse, because the people responsible for those options at companies ranging from Kaseya and ConnectWise to Pia and Thread and way, way beyond all say their AI automation is the best.

And they can’t all be right, so how do you make a choice? Build a lab and test them all out in an endless series of in-house bake-offs? Flip a coin and hope for the best? This is something that’s been on my mind for months, and the launch of Kaseya Intelligence is as good a reason as any to bring it up here.

I think we need a benchmark.

Now, I say this knowing that AI benchmarks are flawed. They can be gamed for one thing, they create perverse incentives to “write to the test” rather than build what people need, and they’re not always easy for laypeople to understand fully.

On the other hand, assuming it’s designed and maintained by a neutral third party, a benchmark is objective, and it gives MSPs otherwise stuck with hunches and guesswork something to go by when evaluating solutions. Having one specifically for service desk automation would, I think, be helpful.

It’s also something I truly, deeply don’t know how to create. So I’m just putting this out there in the hopes that someone who agrees that a service desk AI benchmark would do more good than harm will build one.

I promise to write all about it here in Channelholic if you do. Writing I mostly know how to do.