How to Test AI-Based Systems

Wednesday 6th March 2024, 6:30 pm.

Speaker: Bryan Jones

Venue: Computershare Limited, Edinburgh House, 4 North St. Andrew Street, Edinburgh EH2 1HJ

Refreshments and networking from 6:00 pm.

This event is free of charge and open to all, though registration is required.

Synopsis

The stories we are fed about AI seem to fall into two categories:

1. It’s our saviour! It’s going to be wonderful!

2. It’s going to take all our jobs and destroy humanity!

The media feed us so many simplified stories designed to stimulate our imaginations with science fiction fantasies, both utopian and dystopian. The law and governance around AI is also so far behind the curve that it can confuse the issues, often propagating misconceptions, and making life harder for those wanting to use AI sensibly. The truth is far more nuanced and requires us to understand much more than just the hype.

But there is an elephant in the AI room. Conversations tend to focus on Generative-AI. Very few people talk about how to actually test AI-based systems.

Whether we like it or not, AI is here to stay, and as testers it’s our job to understand those nuances, to understand what the actual risks are, and how to mitigate them.

So … how do you test an AI system?

This talk addresses some of the risks, opportunities, and ethics involved. It also lays out a framework for testing AI-based systems, exploring the technology itself, along with the techniques, skill sets, and thinking patterns that we as testers need to adopt to remain relevant, as the fourth Industrial Revolution kicks in for real.

AI is evolving. We need to as well!

About the speaker

Having graduated from Staffordshire University (1988) with one of the first IT degrees in the country, I decided that the best way to be able to stick my nose into everything that was going on, was to be an Integration Tester. So, I started on Safety Critical Real-Time avionics systems. In the 35 years since then I have spent 18 years as a Freelance Test Consultant and the last 13 years working for a Top 5 European Digital Services Company, most recently as Director of Quality Engineering & Testing.

In that time I have worked in most industry sectors, on a wide variety of projects for many different clients including: Jaguar Land Rover, Rolls Royce Jet Engines, Boeing, PowerGen, Barclays Bank, BarclayCard, Britannia Building Society, Yorkshire Water, Tesco, Vodafone, various government departments, a challenger bank and many others, including a brief stint at easyJet.

In Summary, I am a career testing professional with a lifetime obsession with testing and quality, not to mention an insatiable curiosity and learning addiction.

I am a certified Psychotherapist and Hypnotherapist. I have also done impromptu street hypnosis. I used to play bass in several bands, including an Irish Folk band and a Trash Metal band. I’m currently learning to play electric guitar. I take part in off-road motorcycle rallies on my 1200cc BMW that weighs 250kg – not a practical rally bike! I also used to train and ride horses (a lot!), but my horse is now retired.