Six Sigma, American Excellence in Manufacturing, and AI

Think of a product that you've purchased in the US your entire life. I'm thinking about chips, the snack food. The large American company, Frito-Lay, uses quality assurance techniques taught in Six Sigma (symbol omitted) classes to ensure a consistent experience across their products. So, think about every bag of Frito-Lay's chips that you have opened and enjoyed your entire life. For the normal person, that's probably hundreds, if not thousands, of bags of chips. Now, have you ever once found in your bag, of name brand Frito-Lay chips, a burnt chip, or a foreign object, or even inconsistent seasoning? I'm going to go out on a limb and guess that since 1990, most American's have never encountered that. You can use your own personal experience, but you can also search through social media history. You ever see a social media post that says, "OMG, look at what I found in my bag of Frito's?" I'm going to guess probably not, and if so, it's extremely rare. How rare? Well, Six Sigma uses a measurement called defects per million opportunities (DPMO), so the number of defects isn't really per-bag, but something else. But, still, the chances of encountering a defect become so rare, that it's like a once in an lifetime occurrence.

And, everyone has a story, maybe. I hesitate to use this example, because I once opened a bag of Doritos, and found a clump of seasoning at the bottom of the bag. I don't remember what decade this was, but it did happen. Still, it's an amazingly consistent product. So much so, that when I did find that one defect, I wasn't even mad; I just found it curious.

What is this about? This is about American's expectation of quality. American's expect to rarely, if never, encounter defects. Is that a reasonable expectation? At scale, I would say yes it is.

As this AI technology is being deployed, we are seeing lots of defects. The deployment is in places where a Six Sigma level of excellence is not only expected, but also required. This is law enforcement and military. But, as social media post suggest, and as personal experience suggest, we are nowhere near that level of excellence. Last week, someone got a speeding ticket, when the AI clearly hallucinated a license plate number; when trying to deal with the ticket, the police officer, use to this level of excellence, gave the person the runaround. The plate was unreadable, the car was completely different, and still, the police officer, use to American excellence, assumed the product must be correct; this is "Super Intelligence" from an American company after all. Earlier this year, it was a guy who was arrested in a casino for trespassing, when he wasn't trespassing, but the AI gave a 100% match with someone who was previously kicked out of the casino; this casino lost a customer, and a lawsuit. Last week, I asked AI to put a GTag into my website, and it hallucinated the tie number. If I hadn't checked, my site would be sending tracking information to someone else.

So, what's the point of this? The major AI companies say, you can't use this shit for war; it's not safe. Part of that seems to be their motivation to hoard the tech for themselves through regulation, but part of it seems to be that the technology is unreliable.

Even if we do reach Six Sigma levels of reliability with this technology, someone has to be responsible for it. Because, there is that one time that I got that bad bag of chips. I suspect that any ethics in CS class would agree; I believe any ethics in anything would agree, if it's actual ethics. But, people also need to understand technology that they are using. As I said, this technology is useful, and today engineers are suggesting to trust AI 100% of the time, but, and I cringe when I say this, but that's danger.

I wonder and worry if we could even expect people to understand the technology they are using. In the example of the guy arrested in the casino, the arresting officer had a print out that suggested 100% confidence in a match, and he had the guys "Real ID." Guess which one the arresting police officer trusted. It wasn't the government issued Real ID. He trusted the printout from some AI company, and that was a mistake. But, I don't blame him. He lives in this fantasyland where he has never encountered a burn chip before. Why would he expect the AI to be wrong?

I suspect the issue of quality and defects plagues all of the tech sector. For instance, what is the failure rate of emails sent from my personal Gmail account, which I've had since their first week or two of beta. I've had the gmail account for over twenty years, and I wouldn't be surprised if 50% of the emails I send go straight to the void. It's absurd that we accept this state of technology, but seems that failure in tech is almost expected. But, not with the AI. People seem to expect it to work as if it's magic, and it's simply not. There's a gap between expectations and reality.

Side Note: I had to remove all occurrences of the sigma sign to get this to post, meaning all versions of "Six Sigma" are spelled out. You can see the attached screen grab of the original essay from my journal. Seems that the sigma sign causes an issue with language detection on the site.