Thursday, June 15, 2023

Vetting the AI

 Ever wonder why anyone cares what schools were attended, what GPA was earned, what papers were published in peer reviewed journals?

Or how senior someone is and whether or not they are respected in their vocation?

We screen prospective candidates for important positions - we just don't let anyone become a Supreme Court Justice who "feels like it" (and most of them can explain what is a woman). We wouldn't pay attention to Jordan Peterson if he were just a grumpy Canadian convenience store owner.

This process of checking out "experts" before we place them in important roles is called "vetting". We do it all the time in everyday life.

Do you know why we pay more attention to a 65yr old's advice than the noise from a 20yr old's pie hole? Because the senior citizen likely knows much more than the college kid with their preferred pronouns written on their shirt.  Older people generally know more because they've lived more.

Who is vetting the AIs in use?

Lets say someone custom designs and trains an AI application to decide who to parole. The idea is to reduce the manpower & time (cost) to make thousands of decisions. Guaranteed that the people using the AI application will NOT understand it, nor will they be able to evaluate it's performance. If the AI has a bias, how will it be detected? How will the errors be rectified? (This exact scenario happened).

Again, who is vetting the AI?

The media, as usual, has got it wrong. The first problem we face with with AI isn't SkyNet (Terminator), the first problem is vetting the AI applications before they are in the hands of  unsophisticated users.

We pay attention to Dr. Jordan Peterson because of his education, his teaching, his publishing, his clinical work and his public speaking. About 35 years of "work experience" we can reference and review. Peterson, like him or not, is vetted.

Finally, who is vetting the AI?