I had ChatGPT take an IQ test, so you don't have to
I've been curious to test out how ChatGPT would score on an IQ test. So thats what I did, Pretty surprised by the results. Let's dissect.
First, Welcome all new friends (631 👋)
So excited to see you all here, it’s been a hectic week, with loads of new AI sightings, more GEN-1 videos, voice cloning from Elevenlabs and OpenAI buying ai.com
Putting ChatGPT through an IQ test
In this post, I’m trying to scratch my own itch. Been curious to see how ChatGPT would perform on an IQ test. To be clear, I’m the bottleneck for moving results back and forth via copy/paste. Ideally, ChatGPT would take the test without human intervention, maybe someone is up for making that possible?
This is probably not the first time someone attempts this, but I wanted to check for myself and thought sharing it with you would be interesting.
I’m pretty impressed by the score. If this is the new baseline, it will be extremely interesting to see where we are 2 years down the line.
Let’s dive in.
General IQ score = 89
Let’s just get the general score out of the way, TLDR; IQ of 89.
“ChatGPT’s General IQ Score of 89 shows how able their mind is in general. Anyone with a General IQ Score this high is considered to be of average intelligence.”
“This score is better than 23.17% of all persons taking this test. 70% of all occupations can be comprehended with a General IQ this high. ChatGPT should be able to handle most academic challenges.”
ChatGPT scored higher than its General IQ Score in 5 individual ability categories.
2 of these better scores could be called statistically significant and may indicate special abilities, or that they were distracted on those parts of the IQ Test that counted more heavily in the other ability categories.
Time to break down the entire IQ score
This is a complete breakdown of the IQ test, everything is in here to digest. I’ll include the full written breakdown of each category below.
Spatial Skill
“Understanding what changes will occur when conditions vary is a deep and powerful ability of the mind. All invention and creativity of every sort is based upon this ability. Although test problems usually involve the manipulation of objects in space, persons with a stronger ability to spatially manipulate can also be expected to use this ability to be able to better predict how social and psychological situations would change due to variation.”
ChatGPT’s Spatial Skill IQ score of 93 is not significantly different from their General IQ score.
This score is better than 32.04% of all persons taking this test.
Logical
This is the ability to determine if a set of rules has been correctly followed. This ability is most useful in combination with other mental skills listed above. Those with strong logical ability are quicker to see where a given set of conditions is going to lead, have a strong sense of justice, and better understand–from an intellectual analysis–the benefits of harmony.
ChatGPT's Logical IQ score of 87 is not significantly different from their General IQ score.
This score is better than 19.31% of all persons taking this test.
Spelling
The ability to spell can indicate general intelligence. Remembering a set sequence of letters indicates the mind’s ability to retrieve remembered facts. Learning how to spell and use the words of a language is almost a complete IQ test in itself. Although poor spellers with high IQ scores can be found, it is rare, and in general–everything else being equal–the better spellers have higher IQ scores.
ChatGPT's Spelling IQ score of 100 is exceptionally higher than their General IQ score.
This score is better than 50% of all persons taking this test.
Short Term Memory
The ability to remember things for a short period of time allows the mind to check back and retrieve facts needed to complete a problem-solving operation. This ability becomes more critical when problems have many aspects that need consideration and/or need to be solved mentally. This ability strongly determines how efficiently one handles the many aspects of normal life. If your short-term memory ability is strong you are much less likely to seem inattentive or “slow to get it” to others.
ChatGPT's Short Term Memory IQ score of 87 is not significantly different from their General IQ score.
This score is better than 19.31% of all persons taking this test.
Rote Utilization
This is the ability to take a set of memorized facts and mentally extract and/or operate with or upon the facts within the set that are pertinent to the problem at hand. Persons with more of this ability can be expected to spell well, remember telephone and other numbers easily, be more adroit in procedural operations, and have a stronger foundation for tasks that require the use of memorized material.
ChatGPT's Rote Utilization IQ score of 92 is not significantly different from their General IQ score.
This score is better than 29.69% of all persons taking this test.
Algebraic
This is the ability of the mind to abstractly handle quantities and qualities. Persons who are strong in this ability can more quickly and more deeply understand analogies, stories, derivations, equalities, and hierarchical structures.
ChatGPT's Algebraic IQ score of 79 is significantly lower than their General IQ score.
This score is better than 8.08% of all persons taking this test.
General Knowledge
Knowledge that is casually picked up and remembered can indicate intelligence because persons with higher intelligence will exhibit greater retention of those pieces of information that are encountered less often. Because higher intelligence allows a person to have a deeper appreciation of the connectivity of facts that may seem disparate to others of lesser intelligence, memory of such facts becomes easier.
ChatGPT's General Knowledge IQ score of 81 is significantly lower than their General IQ score.
This score is better than 10.26% of all persons taking this test.
Visual Apprehension
This is the ability of the mind to mentally picture visual information and to be able to extract portions of that information for separate use. A person whose visual apprehension is strong enjoys a richer, more creative appreciation of visual aspects of experiences.
ChatGPT's Visual Apprehension IQ score of 82 is not significantly different from their General IQ score.
This score is better than 11.51% of all persons taking this test.
Geometric
How well one can comprehend geometric relationships of lines, sides, planes, angles, and topological properties strongly determines one’s ability to make sense of visual information. The strength of one’s geometric ability can strongly determine how quickly knowledge can be absorbed if it is presented visually.
ChatGPT's Geometric IQ score of 77 is significantly lower than their General IQ score.
This score is better than 6.26% of all persons taking this test.
Vocabulary
Knowing the meaning of words is an ability that directly increases along with the increase of general intelligence. The meaning of a word is more easily remembered with higher intelligence because it takes more intelligence to understand and correctly use words based on the subtle differences between words with similar meanings and to comprehend difficult concepts which are sometimes symbolized by a single word.
ChatGPT's Vocabulary IQ score of 87 is not significantly different from their General IQ score.
This score is better than 19.31% of all persons taking this test.
Intuition
Intuition is defined as the ability of the mind to develop answers to questions without consciously dealing with the problem at hand. Often a question will provoke your mind to answer without using conscious processing time, and the answer is said to come “out of the blue” or “suddenly, it just struck me”. Of all the many abilities of the mind, this is one of the most often used. Just knowing what to do is often an automatic process that occurs without much conscious figuring. Those with stronger intuition make fewer mistakes and can seem luckier, wiser, or more mature.
ChatGPT's Intuition IQ score of 90 is not significantly different from their General IQ score.
This score is better than 25.25% of all persons taking this test.
Computational Speed
If you can correctly solve a variety of problems faster than another person, you may be demonstrating a generally more orderly internal arrangement of your mind’s problem-solving methods. While speed cannot be the sole factor in determining overall superiority in one’s mental operations, faster computational speed will often indicate that comprehension of a problem was more complete. With everything else being equal, a person with faster computational speed will be better at tasks that require the synthesis of many bits of information.
ChatGPT's Computational Speed IQ score of 115 is exceptionally higher than their General IQ score.
This score is better than 84.13% of all persons taking this test.
*Disclaimer, I was adding lag with copy/pasting over input/results, so I’m pretty sure it would score a lot higher.
Something that stuck with me
70% of all occupations can be comprehended with a General IQ this high. ChatGPT should be able to handle most academic challenges.
Some people are responding to AI with plain fear, others in some sort of utopian futuristic high. I think we need to settle somewhere in the middle. Like with most things in society.
The genie is out of the bottle, and there is no putting it back. So we will have to learn to live with AI and its implications. Good and bad. In this post, we are just looking at one single LLM, but the reality is that there is a heap of them and that list will only keep growing.
We have not looked at other AI models, I think the real inflection point will be when either they are combined (easier) or when there is a single general-purpose model that can do real multi-modality (harder). Then we will see some really intense speed of development. But if we just cycle back to this:
70% of all occupations can be comprehended with a General IQ this high.
This is true right now, 70% of occupations can be comprehended with the IQ of ChatGPT. That does not translate to its ability to perform these occupations, keep that in mind. But its ability to comprehend. I would not be surprised if we are close to 100% really soon, and then the metric to watch out for will be: Their ability to perform an occupation with or without supervision.1 Regardless, we are living through some pretty amazing times, it’s an AI renaissance at least a decade in the making.
Your thoughts
What’s your take on this? How fast do you think the large language models will advance in tests like these? Was this output something you had expected?
Let me know by replying to this email or by responding on substack.
This test was made to scratch my own itch
This is for fun, it’s not a scientific result nor was it done in a controlled environment. It did not include any visual tasks. If we want to explore scientific results we should rely on the constant benchmarks for intelligence and for there to be a certified IQ test in a controlled environment. So please do not consider this state-of-the-art or a solid scientific source.
If you liked this please consider liking it and leaving a comment. It helps me with reaching new readers.
You can also share this post with anyone you think would be interested.
Use the link below
a forward-looking statement, might not age well or I might be completely wrong.
Big thanks for this. Looking forward for the exact same but with ChatGPT-4.