What changes when you make AI 'think deeply'? Comparing reasoning settings in ChatGPT and Gemini

When asking questions to an AI, there are times when you want an immediate answer and times when you want it to take its time and think.

Recent AI services provide features such as "Thinking," "Reasoning," or "Thought Level" to adjust for this difference.

However, even when settings like "Standard," "Extended," or "High Reasoning" appear on the screen, it is difficult to understand how the answers actually change.

Here, using ChatGPT and Gemini as examples, we will examine what happens when you change the AI's reasoning settings.

Note that the available models and setting items vary depending on your subscription plan, account, and service availability.

"Thinking deeply" is not a feature for making answers longer

First, it is important to understand that "strengthening reasoning" and "making the answer text longer" are different things.

Reasoning-related settings are intended to allow the AI to use more inference during the processing stage before generating an answer.

Therefore, just because you strengthen the reasoning does not necessarily mean the answer will become longer.

Rather, where differences are more likely to appear is in

Problems with many conditions

Problems comparing multiple candidates

Problems requiring the construction of procedures

Mathematics and logic problems

Problems involving organizing contradictory conditions

These are questions that are difficult to answer with simple knowledge retrieval alone.

On the other hand, for simple questions like 'Tell me the meaning of this word' or 'Summarize this in 100 characters,' there are cases where the perceived difference is small even if the reasoning is strengthened.

ChatGPT allows you to adjust the strength of reasoning

In ChatGPT, depending on the plan and model you are using, you can adjust the reasoning strength from the model selection screen.

In the current configuration, in addition to options for fast responses like Instant, there are cases where options that use stronger reasoning, such as Medium, High, and Extra High, are provided.

For free and Go users, there are configurations where "Think" can be used for difficult questions.

The important thing is that the same options are not displayed to all users.

Operating and verifying

First, open ChatGPT.

Create a new chat and check the model selection area at the top of the screen or near the input field.

If available, you can select models such as "Instant" or "Thinking," or settings related to the reasoning level there.

If detailed settings are displayed in paid plans or similar, change the reasoning level before sending the same question.

What is important here is not to change the question itself.

For example, use a problem like the following.

There are three people: A, B, and C.

A is faster than B, and B is slower than C.

Can you determine the order of speed for the three people?

If it cannot be determined, please explain what can be understood.

I will send this question with both light and heavy reasoning settings.

Comparing the answers, it is easier to see that rather than simply increasing the amount of text, the AI attempts to organize the conditions before reaching a conclusion.

However, if the problem itself is too simple, there is almost no difference even if you change the settings.

Gemini has 'thinking levels'

In Gemini as well, you can adjust the thinking level in the available models.

In the current guidance, 'Standard Thinking' and 'Extended Thinking' are provided, and under certain usage conditions, 'Deep Think' is also available.

Standard Thinking is intended for normal questions, while Extended Thinking is positioned for use when solving complex problems.

Deep Think is a feature that performs more advanced reasoning, and its use is subject to conditions such as the applicable plan.

Operate and verify

Open Gemini and check the available models.

If you can change the thinking level, check the settings from the model or mode selection screen.

First, ask a question in the standard state.

Next, execute the same question with a higher thinking level, such as extended thinking.

Do not change the question text here either.

For comparison, problems like the following are suitable.

The price of a product was increased by 20% and then decreased by 20%.

Compared to the initial price, what is the percentage change in the final price?

Please also explain the calculation process.

Even if it looks like a simple calculation, it is easy to be swayed by the intuition that 'it goes up by 20% and then down by 20%, so it returns to the original value'.

In problems like this, it is easy to compare how the AI processes the conditions in order.

When comparing reasoning settings, use the 'same question'

The worst thing you can do when comparing the reasoning capabilities of AI is to change the question for each setting.

For example,

Simple questions for normal mode

Difficult questions for high-reasoning mode

In a comparison like this, you cannot tell if the difference is due to the AI or the question.

When comparing,

It is important to change only the reasoning settings while keeping the model, question, and conditions the same.

This becomes important.

Furthermore, it is easier to see the differences by looking at the following parts, not just the answers.

Organization of conditions

When a question has multiple conditions, check whether each condition is correctly captured.

Stability until the conclusion

Check whether the AI changes its judgment midway or misses any conditions.

Handling of exceptions

Check whether it can handle exceptions, such as 'However, if this condition applies, the conclusion changes.'

Time to answer

Increasing reasoning strength may result in longer response times.

Usage

Depending on the service, more advanced models or higher levels of reasoning may consume more usage limits.

Gemini indicates that more advanced models or higher levels of reasoning consume more usage.

In other words, increasing reasoning strength has not only the benefit of 'potentially thinking more deeply,' but also the aspect of 'using more time and usage limits.'

Increasing reasoning strength does not guarantee a correct answer every time

This is an important point.

The reasoning function is not a switch that automatically turns AI responses into correct answers.

It does not mean the AI possesses information it didn't have before, and if you provide a false premise, it may simply think more thoroughly based on that premise.

Also, just because time is spent on reasoning does not necessarily mean it is suitable for questions that require checking the latest information.

For example,

"Tell me today's news"

and

"Think of the best combination based on multiple conditions"

are questions that require different AI capabilities.

For the former, the freshness of information and search functionality are important.

For the latter, reasoning ability to organize conditions and derive a conclusion is important.

In other words, when using AI, you need to think according to the nature of the question rather than "always choosing the strongest setting."

The concept of switching between "normal" and "reasoning"

For everyday questions, the normal setting is often sufficient.

For tasks like paraphrasing text, simple summarization, general questions, or short brainstorming, prioritizing response speed is rarely an issue.

On the other hand, it is worth testing reasoning settings for questions with many conditions.

For example,

'If A, use this condition; if B, use this condition. However, if C, there is an exception. Which one should be chosen in the end?'

These are the types of questions.

In such problems, the AI needs to organize the conditions one by one.

Also, instead of setting reasoning to high from the start, you can get an answer with the standard setting and, if you are not satisfied, retry the same question with the reasoning setting enabled.

This way, you can avoid using more time or usage quota than necessary.

If you are testing three AIs under the same conditions,

when comparing ChatGPT and Gemini, it is also important not to simply equate the service names with superiority or inferiority.

Even with similar names like "Thinking," "Extended Thinking," or "High," the internal mechanisms and the meaning of the settings are not necessarily identical across different services.

Therefore,

"Which is better, ChatGPT's High or Gemini's Extended?"

Rather than a simple comparison like

"How do the answers differ when changing the reasoning settings for the same question?"

it is more practical to look at

When comparing, it is a good idea to prepare one question like the following.

There are Condition A, Condition B, and Condition C.

Prioritizing A creates constraints on B, and prioritizing B creates constraints on C.

Please organize each condition and separate the options that are valid from those that are not.

For parts that cannot be determined, please also explain the reason why they cannot be determined.

Input this question into each AI in the same way.

Then,

whether the conditions were correctly identified

whether the relationships between the conditions were organized

whether it avoided forcing a conclusion on parts that cannot be determined

how the time taken to answer compares

how the quality of the answer changed due to the setting changes

will be checked.

Even just doing this allows for a fairly concrete understanding of 'what reasoning capability actually is'.

The 'strength' of AI is not just one thing

When looking at AI services, it is easy to focus only on the term 'high-performance model'.

However, in reality,

having knowledge

being able to look up the latest information

being able to understand images and files

being able to reason through complex conditions

being able to continue long processes

and so on, AI capabilities have multiple facets.

What reasoning settings adjust is primarily the part concerning 'how deeply to process a problem'.

Therefore, rather than thinking that 'maximizing reasoning will make the AI smarter in every way,' it is easier to think about whether 'this question is worth having the AI think deeply about.'

First, try sending the same question twice

If you have never used reasoning features before, there is no need to conduct a difficult experiment.

Choose one question with multiple conditions that you use in your daily ChatGPT or Gemini interactions.

Send it with the standard settings first.

Next, if available, change to settings like 'Thinking' or 'Extended Thinking' and send the exact same question.

Then, instead of just looking at the length of the response, check:

'Did it overlook any conditions?'

'Is the basis for its judgment organized?'

'How much does the time taken to respond differ?'

Check it.

Once you perform this comparison, terms like "Thinking" or "Reasoning" that appear on AI model selection screens will no longer be just difficult technical jargon.

What is important when using AI is not always choosing the most powerful model.

It is about choosing how much you want the AI to think based on your question.