(No this is not a post about ‘Dear Esther’ or ‘Everybody Has Gone To Rapture’, but please go check out those awesome games after first reading this.)

Imagine a hallway containing a room.

Inside there is a man named ‘John’ he is not allowed out and no one is allowed in.

He has been completely isolated for some time provided food and water through a contactless delivery system.

A woman who works at the lab has now recently been instructed to begin periodically communicating with John by slipping him notes under his door.

She has not been told anything else. She does not know who is in the room or why. …One morning after, she begins following this instruction.

John wakes up to a note in his room.

He is both immediately curious and hopeful of what it might say; however, to great disappointment when he picks it up to read it he finds that it is written in Chinese, a language he does not know at all.

John quickly notices that now also placed in his room are boxes full of papers pencils and erasers as well as books containing comprehensive instructions for how to appropriately respond in Chinese to statements and questions the woman might write.

In the books, John can locate and match the symbols the woman writes and then write the relevant corresponding symbols that the book provides: facilitating a conversation.

At first john is confused but, desperate for social interaction, he soon engages with the woman writing back notes according to the book’s instructions.

As time passes the woman begins to like and then fall for John (believing she is communicating with a charismatic man who was fluent in Chinese); of course however, John has no idea what he is saying…

With the advent and rapid development of digital computing and artificial intelligence the question concerning when, or if, a machine could be considered intelligent, able to think and possess an understanding, is only becoming increasingly relevant.

If John can convince the woman he is intelligent or fluent in Chinese based on what he outputs from the room, but can do so without ever actually understanding Chinese and what he is saying, then the output of John’s behavior is not sufficient in qualifying him as intelligent, or in this case, possessing an actual understanding of what he is saying; and so likewise, if a computer program can follow syntactic rules well enough to provide an output that can convince a person that it is thinking and fully understanding: that does not necessarily mean the same.

It is this experiment challenges the philosophical positions known as functionalism and computationalism.

Functionalism arguing that mental states are to be identified by what they do rather than by what they are made of; and computationalism arguing that the mind is an information processing system and consciousness is merely a form of computation.

According to the Chinese rooms proposition, however, for intelligence of mind to actually exist there needs to be understanding, which is inevitably always missing from a digital computer.

This poses the question though, “What constitutes understanding?”

Obviously John understanding Chinese or not is essentially binary, but can John still understand what he is doing in the situation, without understanding Chinese?

In other words does John’s understanding of the fact that he is in taking and sending out messages in Chinese still constitute some form of understanding? On what level is a computational system required to understand what it is doing for it to be considered sufficient enough for intelligence?

Is it a total comprehensive understanding that is required? A partial understanding in a specific way? How does one determine this?

How does one verify it after all?

Do we as humans really understand what we mean? Even when we know what we are saying. Ultimately, if the door to John’s room was suddenly opened and the woman saw John showing emotion and effort as he wrote his responses down from the instruction books, clearly understanding not the language, but the significance of it, his role in the exchange: Would this change things?

Would the woman still find him worthy of love and possessing of a sufficient enough understanding to be intelligent; likewise if a machine functioned as if it understood behaved as if it did and made the explicit claim that it felt as though it did all based on a program’s code, do we believe it?

How do we treat it? What are the ethics now needed ?