I taught ChatGPT rules, but it broke them. The difference between 'memory' and 'application' I discovered through investigation

It all started with a single word: 'Ore'.

When you use ChatGPT for a long time, you sometimes get it to remember your own rules.

For example, in this case, the user had given ChatGPT a rule,

“Do not refer to yourself using a specific first-person pronoun”

as a rule to follow.

However, while having it investigate some information in great detail and explain the results,

ChatGPT suddenly started referring to itself as 'Ore'.

“Wait, didn't you remember that rule?”

When pointed out, ChatGPT apologized and corrected itself.

Up to this point, it's the kind of thing that happens occasionally when using AI.

But right after that, something bothered me a little.

Even though it forgot one rule, it was still following another.

The user had given ChatGPT another rule.

That was,

“When apologizing, properly say 'I'm sorry'”

as a rule.

When I pointed out that ChatGPT had messed up the first-person pronoun rule, the response it gave was properly,

“I'm sorry”

as its response.

This raises a question.

It failed to follow the first-person rule.

However, it is following the rules for when it apologizes.

This means,

it might not have forgotten all the rules themselves, right?

I thought.

“Remembering” and “being able to reflect it in every answer” are not the same thing

From here, I asked ChatGPT itself for the reason while also checking OpenAI's official information.

In the official OpenAI memory FAQ, it is explained that ChatGPT's memory is a mechanism for remembering useful information from chats and files to personalize future conversations.

On the other hand, the memory overview also clearly states that it does not include all information obtained from past conversations.

Furthermore, custom instructions are also explained as a feature for conveying content that you want ChatGPT to consider when generating responses.

In other words,

“ChatGPT possessing that information”

and

“being able to reflect that information 100% in every sentence it generates”

should be considered separately.

Why was it able to follow the “I'm sorry” rule?

From this point on, since OpenAI has not disclosed the internal processing for this conversation, this is a hypothesis based on the phenomena that actually occurred.

What was quite easy to understand this time was the difference in the types of rules.

The apology rule is,

“ChatGPT is pointed out a mistake”

“It becomes a situation to apologize”

“Use the rule for when apologizing”

This means the timing for application is very easy to understand.

On the other hand,

the rule of "do not use this first-person pronoun"

is different.

This is not just for specific situations,

but must be strictly adhered to from the beginning to the end of the response.

Moreover, the time it got the first-person pronoun wrong this time was when it was investigating a significant amount of information to create a long response.

Investigating information.

Comparing multiple sources.

Verifying numbers.

Separating facts from speculation.

Structuring the text.

On top of that, it also maintains the speaking style that is set by default.

I cannot conclude that "it becomes easier to forget rules when investigating" based solely on this experience.

However,

the fact remains that when creating complex responses, stylistic rules that should be applied at all times did partially break down.

That fact remains.

ChatGPT itself does not explain that it will "always follow" them.

This is also interesting.

OpenAI also has a mechanism called "traits" that adjusts ChatGPT's speaking style and tone.

In the official explanation, it is described as something that works in combination with selected personalities, custom instructions, memory, etc., and that ChatGPT 'attempts to follow' the style you have set.

I think this point is quite important.

AI personalization is not

a mechanism that is 'absolutely fixed like an if-statement in a program once set'.

At least in this experience,

it is not.

even rules that it should remember can sometimes fail to be reflected in the response.

Conversely, it has not forgotten everything.

Therefore, rather than 'amnesia',

it is better to think that it sometimes misses information during the stage of applying remembered information to the response

which fits this phenomenon better.

This might be important the longer you use ChatGPT.

At first, it was simply

a matter of

'It got the first-person pronoun wrong again'.

But when I think about it, it was quite important.

Customizing ChatGPT for yourself,

how it speaks.

how it addresses you.

how it constructs sentences.

Work-related rules.

Prohibited items.

As you increase these kinds of things,

you need to look at not just 'whether it remembers' but 'whether it is being properly applied in the current response'.

It becomes necessary to check.

Especially when having it create long texts or research large amounts of information,

rather than completely leaving it to the AI thinking,

'It's a rule I decided before, so it's fine,' it is better to check the completed response once.

In this case too, if the worker hadn't noticed the difference in the first-person pronoun, it would have remained that way.

AI was not 'absolute once it remembers'.

To summarize what I learned this time,

ChatGPT has mechanisms like memory and custom instructions to personalize conversations.

However,

remembering something and reflecting it perfectly in text every time are not the same thing.

And what was interesting was,

even right after it broke the first-person rule, it properly reflected another rule for when apologizing.

It didn't forget everything.

But it also can't use everything perfectly.

If you use ChatGPT for a long time to make it closer to your own dedicated environment, I think knowing this difference will make it easier to work with.

'Why don't you follow the rules even though you remember them?'

When you think that.

Perhaps it didn't "forget," but rather,

it just wasn't applied correctly in that response.