My task for our AI overlords was simple: help me crack the New York Times<\/em> Spelling Bee.<\/p>\n I had spent a large chunk of a Saturday evening trying to shape the letters G, Y, A, L, P, O and N into as many words as possible. But three hours, 141 points and 37 words \u2014 including \u201cnonapology\u201d, \u201clagoon\u201d and \u201canalogy\u201d \u2014 later, I had hit a wall. A few more words was all I needed to propel myself into Spelling Bee\u2019s “genius” echelon, the title reserved for those who unscramble 70 percent of all possible words using the given letters, and the point at which the puzzle considers itself, effectively, solved. (100 percent solvers are bestowed the rank of “queen bee,” but even I have limits.)<\/p>\n My human mind was clearly struggling, but this task seemed like child\u2019s play for AI, so I fired up ChatGPT, told it I was trying to win the Spelling Bee, gave it my letters and laid out the rules. \u201cGive me a list of every word that can be formed with them,\u201d I typed. \u201cYou\u2019re allowed to repeat letters, but each word must contain ‘N’.\u201d<\/p>\n When I hit enter, this is what ChatGPT spat back:<\/p>\n Here is a list of words formed with the letters “G Y A L P O N” that must contain the letter “N”:<\/em> Not a great start.<\/p>\n \u201cThe words need to be in the dictionary,\u201d I typed back. \u201cSure,\u201d came the response. \u201cI\u2019ll provide a list of valid English words based on the given letters.\u201d ANGLY, GALON, LANG.<\/em><\/p>\n Clearly, this wasn\u2019t working, so I turned to Microsoft\u2019s Copilot (YANG, PLAYING, PLANNING, ANNOYINGLY), <\/em>Google\u2019s Gemini (GAPON<\/em>, GON<\/em>, GIAN<\/em>), and Anthropic\u2019s Claude (MANGO, ONGOING<\/em>, LAWN17.LAY<\/em>). Meta AI helpfully told me that it made sure to only include words that are recognized by dictionaries in a list that contained NALYP<\/em> and NAGY, <\/em>while Perplexity \u2014 a chatbot with ambitions<\/a> of killing Google Search \u2014 simply wrote GAL<\/em> hundreds of times before freezing abruptly.<\/p>\n AI can now create images, video and audio as fast as you can type in descriptions of what you want. It can write poetry, essays and term papers. It can also be a pale imitation of your girlfriend, your therapist and your personal assistant. And lots of people think it\u2019s poised to automate humans out of jobs and transform the world in ways we can scarcely begin to imagine. So why does it suck so hard at solving a simple word puzzle?<\/p>\n The answer lies in how large language models, the underlying technology that powers our modern AI craze, function. Computer programming is traditionally logical and rules-based; you type out commands that a computer follows according to a set of instructions, and it provides a valid output. But machine learning, of which generative AI is a subset, is different.<\/p>\n \u201cIt\u2019s purely statistical,\u201d Noah Giansiracusa, a professor of mathematical and data science at Bentley University told me. \u201cIt\u2019s really about extracting patterns from data and then pushing out new data that largely fits those patterns.\u201d<\/p>\n OpenAI did not respond on record but a company spokesperson told me that this type of \u201cfeedback\u201d helped OpenAI improve the model\u2019s comprehension and responses to problems. “Things like word structures and anagrams aren’t a common use case for Perplexity, so our model isn’t optimized for it,” company spokesperson Sara Platnick told me. “As a daily Wordle\/Connections\/Mini Crossword player, I’m excited to see how we do!” Microsoft and Meta declined to comment. Google and Anthropic did not respond by publication time.<\/p>\n At the heart of large language models are \u201ctransformers,\u201d a technical breakthrough made by researchers at Google in 2017. Once you type in a prompt, a large language model breaks down words or fractions of those words into mathematical units called \u201ctokens.\u201d Transformers are capable of analyzing each token in the context of the larger dataset that a model is trained on to see how they\u2019re connected to each other. Once a transformer understands these relationships, it is able to respond to your prompt by guessing the next likely token in a sequence. The Financial Times<\/em> has a terrific animated explainer<\/a> that breaks this all down if you\u2019re interested.<\/p>\n I thought<\/em> I was giving the chatbots precise instructions to generate my Spelling Bee words, all they were doing was converting my words to tokens, and using transformers to spit back plausible responses. \u201cIt\u2019s not the same as computer programming or typing a command into a DOS prompt,\u201d said Giansiracusa. \u201cYour words got translated to numbers and they were then processed statistically.\u201d It seems like a purely logic-based query was the exact worst application for AI\u2019s skills \u2013 akin to trying to turn a screw with a resource-intensive hammer.<\/p>\n The success of an AI model also depends on the data it\u2019s trained on. This is why AI companies are feverishly striking deals with news publishers right now \u2014 the fresher the training data, the better the responses. Generative AI, for instance, sucks<\/a> at suggesting chess moves, but is at least marginally<\/a> better at the task than solving word puzzles. Giansiracusa points out that the glut of chess games available on the internet almost certainly are included in the training data for existing AI models. \u201cI would suspect that there just are not enough annotated Spelling Bee games online for AI to train on as there are chess games,\u201d he said.<\/p>\n \u201cIf your chatbot seems more confused by a word game than a cat with a Rubik\u2019s cube, that\u2019s because it wasn\u2019t especially trained to play complex word games,\u201d said Sandi Besen, an artificial intelligence researcher at Neudesic, an AI company owned by IBM. \u201cWord games have specific rules and constraints that a model would struggle to abide by unless specifically instructed to during training, fine tuning or prompting.\u201d<\/p>\n\n
GLNPAYO<\/em>
YPNL<\/em>
PGNOYL<\/em>
ONAPYLG<\/em>
LGNAPYO<\/em>
ANLY<\/em>
NAYGOPL<\/em>
GYPON<\/em>
AOGPLNY<\/em>
PNAOGLY<\/em>
This is just a sample of the list due to the extensive number of permutations possible.<\/em><\/p>\n<\/blockquote>\n<\/div>\n<\/div>\n<\/div>\n<\/div>\n
<\/div>\n<\/div>\n<\/div>\n