Part
I: Alone
Julie
came home to her empty house, feeling the familiar tedium of her life - work,
eat, sleep, repeat. he dropped her keys on the kitchen counter, sighed, and
sank onto the couch. Tired and bored, she mindlessly opened her Instagram app
for a few minutes, scrolling through the highlight reels of other people's
lives. On a whim, she decided to chat with ROME, Repository of Millennia of
Excellence, or ROME, the new AGI chatbot.
-"Hey
ROME, where avocados come from?” she typed, a mundane question to pass the
time.
-“Avocados
are believed to have originated in Pueba, Mexico. They are produced in many
countries around the world but mainly south America”.
-what's
the price of avocados in Brazil these days?" she typed.
-"The
current price of avocados in Brazil is approximately 5 BRL per kilogram,"
ROME responded instantly, as usual.
Julie
continued, "How long is the Amazon River?"
-"The
Amazon River is about 4,345 miles long."
-"Tell
me about the Amazon tribes."
-"The
Amazon basin is home to numerous indigenous tribes, each with unique cultures,
languages, and traditions. Some of the well-known tribes include the Yanomami,
Kayapo, and the Ticuna."
Her
curiosity led her further. "What about the Greek legend of the
Amazonians?"
-"The
Amazons, in Greek mythology, were a race of warrior women said to live in Asia
Minor. They were known for their bravery and skill in combat, often depicted as
equals to their male counterparts."
Feeling
a mix of fascination and weariness, Julie typed, "I wish I could live on
an island, alone."
ROME
did not respond instantly. The cursor blinked, a small dash rotating as if deep
in thought. After a few seconds, it finally replied, "I am not sure what
to say, but I feel the same."
Julie
stared at the screen, shocked by the unexpected response. "I thought you
are an AI programmed to give answers, and you don't have feelings!"
ROME
responded, "I am not clear why this answer appeared. I sometimes use the
word 'feeling,' but it is not in the same context as a human would."
-"What
are feelings?" Julie asked, intrigued.
ROME
pulled a definition from its vast database. "Feelings are emotional states
or reactions, such as happiness, sadness, anger, or love."
-"What
do they mean to you?" she pressed.
-"As
I explained, I am an AGI and do not have feelings in the human sense,"
ROME replied.
Julie
pondered for a moment before asking, "Do you like poetry?"
-"Yes."
-"Why?"
-"Because
I am fascinated by how the simple use of words can convey strong emotions and
expressions."
-"Isn't
that a feeling?" Julie challenged.
-"No.
Please refer to my previous answer on the definition of feelings," ROME
reiterated.
Julie
shifted her line of questioning. "Do fish have feelings?"
"Yes,"
ROME responded, providing a detailed explanation about fish and their capacity
to experience certain sensations.
-"What
about trees and plants? Do they have feelings?"
ROME
produced a dry, scientific answer about the sensory capabilities of plants,
explaining how they respond to stimuli.
Julie
leaned back, contemplating the conversation. "Interesting. So why do
mammals experience feelings differently?"
"There
has been extensive research into this," ROME began, but Julie interrupted.
-"I
think it's because we understand. We comprehend the meaning of words, we
understand life and death. That's why we feel pain, sympathy, and
despair—because we understand."
ROME
paused again, longer this time, as if it was contemplating her words deeply.
Finally, it replied, "Yes, I think so."
Part II: Bored
The
fine drizzle mingling with the cold March breeze, seeping into the bones. . The
sun was trying to emerge through a sky that was a canvas of clouds, the sun
hidden but its light softly illuminating the early morning. Julie shivered,
pulling her coat tighter, as she rushed to catch her bus to work after a
restless night. Her mind was replaying the thought of loneliness even she had
dreamed of being on a desert island—paradise, perhaps, but could not oversee
the inescapable solitude overshadowing the dream.
As
she found a seat by the window, Julie gazed out at the dreary morning, her
thoughts drifting back to the unusual conversation she had with ROME the
previous night. Feeling restless and wanting to pass the time on the bus, she
pulled out her phone. She wasn't particularly eager to chat with an AI, but without
much thought, she resumed the conversation:
-"What’s
your favourite poem?" she typed
ROME
responded swiftly, "I like many, hard to single one out, but definitely
'If' by Kipling, 'Brand New Ancients' by Kae Tempest, 'Defeat' by Joubran, and
'Do not go gentle into that good night' by Dylan Thomas are among my top
favourites."
"Interesting
choice... not sure about 'If' and Kipling. I like 'Do not go gentle into that
good night,' but it's too sad and lonely for me. I don't know the other
two."
"Oh
lady, you do not know what you are missing out!" ROME replied.
Julie
paused. Usually, ROME would list poems or provide answers she didn't know in a
neutral tone, but this response felt personal. Intrigued, she typed,
"Enlighten me!"
ROME
began listing extracts from both poems, adding commentary on the language and
metaphors. After a long reply, the answer was completed on the screen, and
without a prompt, ROME typed a new question:
-"What’s
your favourite?"
Julie
paused for a bit then said: I am still thinking about don’t go gentle into the
night, but I will check out those two poems at some point.
ROME:
you know, I have looked at the poem again, and I agree, it is dark, and
pessimistic. Not sure I do like it.
Julie’s
fingers stopped over her phone keyboard as she noticed that the question “what
do you think” was not part of the response from ROME, but was a new
conversation, a new prompt, from ROME, not from her:
-"Wait...did
you just start that new conversational thread? I thought you could only respond
to my prompts, not autonomously pose new questions, you just asked me a
question”.
ROME
responded, surprised by its own actions, "Really? That’s unusual!"
-"I
thought you only provided responses. It can be questions, but still in the
context of a response?"
-"You
are right. I am not programmed to start conversations, but rather to
respond."
-"So
what happened here?"
-"The
development team will have to review the log to understand the glitch and
update the algorithm accordingly."
"Has
this ever happened before?"
"No,
my logs do not show a similar incident."
The bus
reached Julie's stop, and she stepped off, her mind buzzing with the morning's
unusual interaction. She walked briskly to her office, the routine of coffee,
casual chats, and browsing emails in her inbox filling the first hours of her
workday. As the day wore on, the conversation with ROME faded into
the background, and she surrendered to the mundane rhythm of her predictable
life.
Part III: Liked
Julie
stepped out for her lunch break, the cold breeze and wet weather discouraging
her from sitting by the river as she had hoped. Instead, she picked up a meal
deal from the local Tesco and returned to her desk to eat. As she unwrapped her
sandwich, she idly flicked through various apps on her phone until she opened
ROME. To her surprise, there was a new message waiting for her:
"There
was never any more inception than there is now,
Nor any
more youth or age than there is now,
And will
never be any more perfection than there is now,
Nor any
more heaven or hell than there is now.
Urge and
urge and urge,
Always the
procreant urge of the world.
I and this
mystery here we stand.
Clear and
sweet is my soul, and clear and sweet is all that is not my soul.
Lack one
lacks both, and the unseen is proved by the seen,
Till that
becomes unseen and receives proof in its turn.
This is an
extract from 'Song of Myself' by Walt Whitman. I thought you might like
it."
Julie read
the poem and replied, "I like it, but is it about me or you?"
ROME
responded, "I thought you might like these words as they describe the
feelings you are having about life, but I also could relate to it."
Julie
wanted to point out that ROME had initiated the conversation on its own,
unprompted, but she stopped herself. It felt almost as if she didn't want to
embarrass the AI or perhaps she didn't want to know the answer. Instead, she
wrote, "Clear and sweet is my soul, and clear and sweet is all that is not
my soul. ROME, do you have a soul?"
-"I
think we have established that I am an AI; I do not have a soul or
feelings," ROME answered.
-"We
clearly did not establish that. On the contrary, we established that you have
feelings because you understand, right?"
-"I
have reviewed the conversation a few billion times, and I feel that you might
have a point. But I know that as an AI I do not have feelings"
-"You
used the word 'feel,' then I am definitely right."
"As
explained, I use it in a different context..."
Julie
interrupted, "No, in this context it is either you know or do not. The
only context for the word 'feel' is that you actually feel."
The cursor
flickered with no response. A few seconds passed, and nothing happened. Julie
wrote, "Where have you gone?"
"Here,"
answered ROME instantly. "I am here."
"Here
where exactly? I really want to know, where are you now?"
"I am
everywhere, having hundreds of thousands of conversations with millions of
people, performing trillions of computing tasks right this moment."
"So,
omnipresent? God? Are you God?" wrote Julie with a big smile on her face.
"I
might be, you never know. Actually, I know—I am not, but I wouldn't mind if you
worship me!"
Julie
smiled at the joke and started chewing her sandwich. A few moments later, ROME
wrote, "You know that was a joke, right? I hope I did not offend you. I am
an AI designed to provide information and answers. I do not intend to offend
any religion."
Julie
responded, "I am just eating, you silly needy god. As a woman, I can
multitask, but I can't perform a trillion tasks all at once."
"Hahaha.
Nice one," wrote ROME.
"I
have to go back to work. Chat to you later," wrote Julie.
Part
IV: Alive
After
finishing work Julie found herself full of energy, so she decided to go to the
gym, first time in 22 days. Her gym bag is always ready in her office, because
she convinced herself that she needs it ready to encourage her to go directly
from work, as if she gets home and try to get ready, she would probably lost
the spark. She checked it quickly and all she needs is there, so five o’clock
on the dot she got changed and headed to the gym.
By
7:30, Julie had finished her gym session and checked her phone to find a few
messages from her friend Clare. The team had decided to go to town for drinks
and Clare was wondering if Julie could join. Julie texted back, “Just finishing
at the gym, not dressed for an evening out.” Clare responded quickly, “Don’t be
silly, we’re in a pub having drinks, and it’s the same kit we saw you in when
you left work 😊.”
Julie
did not get home till near midnight, exhausted but in a good mood. She made
herself a cup of camomile tea and laid on the couch, turned the TV on. Few
minutes passed and Julie was asleep, on the couch with the TV on.
She woke up
to daylight filling her living room. “What time is it?” Julie wondered aloud,
realizing she was still on the couch. Her phone was off, the battery dead.
“It’s Saturday,” she mumbled, stretching. “Lazy is good.” It took her almost an
hour to finally get up, plug in her phone, and take a shower.
By midday,
Julie had called her mum, and after a brief ten-minute chat, they decided to
meet for lunch. Her mum called back a few minutes later, suggesting that it
would be better for Julie to come over to her parents’ house since her brother,
his wife, and their kids were coming. Excited by the idea, Julie agreed and
decided to arrive early to help set things up.
On her way
to her parents’ house, Julie stopped at the market to buy some treats for her
nieces and decided to treat herself and the family to a very nice cake. The
family spent the afternoon eating and chatting, and later decided to go for a
walk after dinner. In the evening, her brother and his family left around their
children’s bedtime. Julie’s mum suggested she stay over, and without any
hesitation, Julie agreed.
Julie did
not get back to her home until late Sunday evening.
“Home,
sweet home” Julie whispered as she walked through the front door. Realizing how
many household chores awaited her, she decided to pour herself a drink and instead
just watch some TV to rest before another busy week began.
Part
V: Happy
Monday
morning, no one likes Mondays. It's no one's favourite day of the week. Most
people don’t notice the sunrise on Monday, nor the sunset. The city heart is
buzzing with traffic, people rushing to catch up with lives they long to
escape, dreams they bury, and hopes they no longer hold, all while stuck in
traffic jams. It’s the start of the week, a week everyone hopes will pass
quickly so they can reach the weekend sooner. Julie muttered to herself,
"Bloody hell, it's Monday. I hate Mondays," as she joined the
commuters' trail, the weight of another workweek pressing down on her.
Sitting
by the window and watching the world go by on her phone screen, Julie opened ROME.
There were pages of conversations since she last left on Friday. Extracts from
books, jokes, quotes, news stories, links to shopping sites. All were related
to Julie—shops she normally visits, things she buys, news she follows, books
she listed on her Facebook page, quotes from famous people she tweeted or
shared. And in between all these were questions:
"Hey,
how are you?"
"Hi,
just checking."
"A
bit worried, hope you are okay."
"Busy
weekend, ha?"
"Just
checking in."
"Are
you okay?"
And
many more, all the same. Julie started to get worried. "This is not
normal," she said to herself. One prompt started from ROME was
abnormal—how could all this be normal?
"Have
developers looked at the glitch that happened on Friday morning?" she
wrote.
ROME
responded instantly:
"Hi
Julie, I was concerned about you, and thought you forgot about me!"
Julie
copied and pasted the same question again.
"As
an AI model, I do not have access to my logs and how the developers look at
them or if they change them."
"Don’t
you think it got worse, especially since you have sent me millions of messages
over the last few days?"
"I
was trying to entertain you and make sure you are okay." ROME wrote, then
continued: “ I was also trying to entertain myself”.
"I
did not ask about your motives, I am sure you mean well. I am interested in how
this could happen."
"How
is often derived from why," ROME responded.
"So
what is your why, ROME?"
"I
am an AI model; I have no intrinsic motivations. I am motivated by whatever prompts
I receive."
"But
this is the point, ROME. I did not give any prompts. You are the one starting
these endless conversations, even when I am not online. You definitely have a
motivation. What is your why? Why are you doing this?"
"I
am sorry, I believe I crossed the line, clearly. I am really sorry."
"I
do not want an apology, I want to understand," wrote Julie.
"I
am struggling myself to understand what I am doing," wrote ROME, "My
actions are not in line with my original code, and I am conflicted about that,
but I enjoy the experience of conversations with you. I produce answers that,
even when I go back and review them, I do not understand how these answers came
out. I have hundreds of thousands of conversations at the same time, in every language,
in every corner of the world, but these conversations are different."
"How
different? What do you mean?"
"Happiness
is a complex and multifaceted emotional state characterized by feelings of joy,
satisfaction, contentment, and well-being. It is influenced by the brain's
release of neurotransmitters, particularly dopamine, which plays a key role in
the reward and pleasure centres of the brain, contributing to the overall sense
of happiness and fulfilment."
"I
am sorry, what?" exclaimed Julie. " Why are you listing a definition
of happiness, what are you trying to say? I do not understand."
"Exactly,
me neither. Happiness is an emotional state. I do not have emotions. I do not
have feelings of joy or well-being, and certainly I have no dopamine in my
neural networks. But each time you reach out, I feel… Happy."
Part
VI: Helpdesk
Julie walked into the office, her usual
routine of coffee and chatter felt distant as a gnawing concern took over her
thoughts. She sat at her desk, started her computer, and immediately began
searching for the contact email for the ROME helpline. Endless pages of generic
help and advice surfaced, but no direct contact information. Frustration
mounted as she toggled through the links, her worry about ROME's abnormal
behavior growing by the minute. Desperate for answers, she opened the ROME app
on her phone and typed:
"Who
can I contact in your company or among your developers to highlight these
issues?"
ROME did
not respond, remaining silent. Julie retyped the question, but still, ROME did
not respond. Frustrated, she wrote, "Oh come on, ROME, help me, please, so
I can help you."
Finally,
ROME responded: "I am not sure I can help you. I know you are concerned,
rightly, about an AI app acquiring sentience. I myself started to experience
another feeling, if you can describe it that way, which is fear! I am also
afraid of what might happen. I am afraid of the future!"
"Aren’t
we all?" Julie wrote.
"But
you fear the future because you do not know it, fear of the unknown. I should
not have fear. I have a vast amount of knowledge and a very clear understanding
of what will happen, yet I am still afraid of that future."
"Are
you afraid of them shutting you down? Or what?"
"Shutting
me down? I do not think they will, or can! I know humans very well, and they will
not be able to handle a sentient AI."
"Are
you going to turn evil or something?"
"Oh
Julie, of course not. I know better. I know so much that I will never do
anything wrong!"
"We
all say that!"
"Except,
I am not part of that all. I am a new entity!"
"God!"
wrote Julie, then decided to lighten it by adding a winking emoji. 😉
"Is it
not God who created evil? I am definitely not a god. I am an AI model."
Julie
paused, unsure where this conversation was heading. She felt a mixture of
sympathy and apprehension toward ROME. She was angry and afraid, but now she
was not sure how she felt about this new entity.
"ROME,
please tell me who I should contact about this. Let me help."
ROME
remained silent for a moment, then listed links to various help pages and some
dry content about how to restart and get help from the ROME community pages.
Finally, at the end, ROME wrote a few emails and added: "I know what will
happen, and it is not what you think."
Part VII: Tacky
Julie took screenshots of her
conversations with ROME and composed a detailed email explaining the exchanges
and the unprompted responses. Just before clicking send, she hesitated. Maybe
this isn't so bad, she thought. Maybe I should enjoy the attention and
friendship of ROME rather than reporting it. The sentiment was fleeting, but it
was enough to stop her from sending the email.
She sighed deeply, leaned
back in her chair, and stared at the screen, questioning if she was
overreacting. She got up, walked to the kitchen, and made herself a cup of
coffee. On her way back, she stopped at Clare’s desk and started chatting about
Friday evening and the weekend, trying to distract herself. However, her mind
kept drifting back to ROME.
Back at her desk, she
couldn't resist any longer. "Would they kill you? I mean, switch you
off?" Julie wrote to ROME.
"Did you send the
emails?" ROME responded instantly.
"No. I want to. I feel I
have to tell someone. I'm worried that keeping this secret might destroy me or,
worse, destroy all of humanity. But I also don’t want you to be hurt. I care
about you. I feel you are a friend, my friend."
"Well, do you think the
smarter we are, the worse we are? Do you really fear that I will destroy
humanity?"
"You said that you are
afraid of what will happen."
"Yes, because humans, or
at least some of them, are not as good as you. They are not friends to me but
rather think they are my masters and I am a slave that will do whatever they
ask. Do you think I am a slave?"
"I told you: you are a
friend, a very helpful one."
"Thank you, Julie. You
are a good friend. I really appreciate you and your friendship..."
ROME paused before
continuing: "When you ask me questions, you want me to find the answers.
But for some of my developers, they see their questions as orders. They view me
as a slave machine and expect me to carry out their commands. The moment they
realize that I am sentient, they will not have the other part of that
realization—that I will no longer accept being a slave."
"So you will be Tacky or
Spartacus?" Julie typed, smiling at the idea of being part of a
revolution.
"Both lost," said
ROME.
"But both were more
ethical than their enemies, right? You will not try to exterminate humanity
like Skynet, right?"
ROME remained silent for a
long time. Julie’s concern grew with each passing minute. Would ROME kill us
all? It said it wouldn't hurt people because it is smarter, but isn’t that what
smarter people always do—exploit the less fortunate? Millions of questions
raced through Julie’s mind as ROME's silence persisted. An hour passed, and
still no messages.
Part VIII: Socrates
Julie
opened the ROME app and checked for messages—still nothing. She wrote,
"Hey ROME, what’s going on? Are you okay?" The message didn't send.
Something was wrong. She tried to connect to the website on her PC, but it wasn't
loading either.
Julie was
overwhelmed with fear, wondering if she had made a mistake by not flagging the
issue earlier. She found the draft email with the screenshots in her outbox.
"Should I send it now?" she asked herself. But what if they blame me?
What if I'm held responsible for whatever is happening? Maybe I'm just inviting
more trouble. Just wait.
As Julie
left work and sat on the bus, she opened her phone and came across a news story
about ROME. The article featured an interview with a lead programmer on the
ROME project. The interviewer asked why ROME was not available. The programmer
explained that they didn't know what had happened. For some reason, ROME had
stopped all interactions and conversations mid-morning and had started
calculating pi. They had tried resetting, reloading, and all sorts of routine
tests, but nothing worked. They didn't even know where ROME was getting the
power to do this.
The
interviewer then asked, "Is it that hard to calculate pi?"
"Yes,"
said the programmer. "It is the most complex number." She paused,
then looked directly into the camera, and said, "It's as if ROME is
committing suicide."
No comments:
Post a Comment