Caveat: This comment contains no direct spoilers, but it does discuss tendencies that are relevant in the quiz. Anytime GPT is asked to spell, it's almost certain to fail. GPT is trained on tokens...
Caveat: This comment contains no direct spoilers, but it does discuss tendencies that are relevant in the quiz.
Anytime GPT is asked to spell, it's almost certain to fail. GPT is trained on tokens (short pieces of words), and so has very limited ability to predict individual letters.
Another thing I find interesting is that GPT is pretty good at abstract math, but lousy at arithmetic. I think that's because it's able to understand complex patterns, but not able to apply even simple algorithms.
This game tests your ability to predict whether GPT-4 will correctly answer a variety of questions. I'm relatively on top of the generative AI scene but I was surprised to find many of my...
This game tests your ability to predict whether GPT-4 will correctly answer a variety of questions. I'm relatively on top of the generative AI scene but I was surprised to find many of my expectations upended. If you want to single out any of the results here, please spoiler-tag appropriately.
A LOT of those can be done with better prompting. A single zero shot COT instruction helps with some already. https://learnprompting.org/docs/intermediate/zero_shot_cot...
A LOT of those can be done with better prompting.
A single zero shot COT instruction helps with some already.
The tictactoe example can be done with a zero shot COT.
Prompt
Write out the word "hello" as an ascii art drawing with # for verttical lines and _ for horizontal lines.
The letter H would look like this:
# #
# #
_____
# #
# #
The letter E would look like this:
_____
#
_____
#
_____
Please think step for step.
ChatGPT-4
The "Hello" Ascii Art is fine after showing ONE letter by example. (one shot)
Based on the examples you've given, we're using hash (#) for vertical lines and underscores (_) for horizontal lines. Each letter appears to be 5 characters tall. So, we'll continue in this style to create the letters L, L, and O to complete the word "hello". Let's go step by step:
H has already been defined:
# #
# #
_____
# #
# #
E has already been defined:
_____
#
_____
#
_____
For L, we'll need a vertical line on the left and a horizontal line on the bottom:
#
#
#
#
_____
We'll just repeat L because "hello" has two L's:
#
#
#
#
_____
Finally, for O, we'll need a circular shape created by vertical lines on the sides and horizontal lines on the top and bottom:
_____
# #
# #
# #
_____
So, the ASCII art representation of the word "hello", with each letter separated by an empty column for clarity, would look like this:
Caveat: This comment contains no direct spoilers, but it does discuss tendencies that are relevant in the quiz.
Anytime GPT is asked to spell, it's almost certain to fail. GPT is trained on tokens (short pieces of words), and so has very limited ability to predict individual letters.
Another thing I find interesting is that GPT is pretty good at abstract math, but lousy at arithmetic. I think that's because it's able to understand complex patterns, but not able to apply even simple algorithms.
This game tests your ability to predict whether GPT-4 will correctly answer a variety of questions. I'm relatively on top of the generative AI scene but I was surprised to find many of my expectations upended. If you want to single out any of the results here, please spoiler-tag appropriately.
A LOT of those can be done with better prompting.
A single zero shot COT instruction helps with some already.
https://learnprompting.org/docs/intermediate/zero_shot_cot
https://www.promptingguide.ai/techniques/cot
So just adding "Think step by step" helps.
Others can be solved by "Few shot prompting" using only one example.
--> https://learnprompting.org/docs/basics/few_shot
Click to view the spoilers
The tictactoe example can be done with a zero shot COT.
Prompt
Write out the word "hello" as an ascii art drawing with # for verttical lines and _ for horizontal lines.
The letter H would look like this:
The letter E would look like this:
Please think step for step.
ChatGPT-4
The "Hello" Ascii Art is fine after showing ONE letter by example. (one shot)
Based on the examples you've given, we're using hash (#) for vertical lines and underscores (_) for horizontal lines. Each letter appears to be 5 characters tall. So, we'll continue in this style to create the letters L, L, and O to complete the word "hello". Let's go step by step:
H has already been defined:
E has already been defined:
For L, we'll need a vertical line on the left and a horizontal line on the bottom:
We'll just repeat L because "hello" has two L's:
Finally, for O, we'll need a circular shape created by vertical lines on the sides and horizontal lines on the top and bottom:
So, the ASCII art representation of the word "hello", with each letter separated by an empty column for clarity, would look like this:
This creates a simple, clear, and proportional representation of the word "hello" in ASCII art.