Life after Death - as a Parrot

Conversing personality replicas: what would make them good and what are they good for?

Photo by Roberto Nickson on Unsplash

I first wrote about the idea of personality replicas in 2019, when they were in the very early stages. Today, a story in the MIT Technology Review shows that new developers are still pursuing this idea. Meanwhile, AI companies have stunned us with ever more convincing conversational programs called “large language models” (LLMs). This stuff might get real, so it’s time to take another look - not at what they are now, but what and how they might be.

I enjoy being an unsung futurist. For example, I “invented” the idea of digitizing music in the days before even personal computers. My friends and I were sitting around a campfire in the Wind River Range. We had seen moose and a bear that day. We had fried trout from a high mountain lake. One of my witnesses has since made artistic fame in part by pioneering in digital media. But I doubt that he would remember my oracular pronouncement. My other witness died from dementia. I would love to talk to him again. Which brings us back to the subject: preserving a person’s personality, knowledge, and memories so that others can experience aspects of these after the person is gone.

A Pioneering Idea: the Lifebox

There is a prophet of the idea of personality replication: the mathematician, computer scientist, transrealist, cyberpunk speculative fiction writer, Rudy Rucker. Way back in 1986, he coined the term “lifebox” for a very realizable idea. As he explains it:

“… to create a virtual self, all I need to do is to (1) Place a very large amount of text online in the form of articles, books, and blog posts, (2) Provide a search box for accessing this data base, and (3) Provide a nice user interface.”

Rucker is amazed that outfits currently using this idea never credit him with originating it. He has continued to develop the concept in novels and lectures.

---

What We Lose

My Mom had outlived nearly everyone of her generation and some of mine. Before she died, I started a picture book to help preserve memories of her. I dug up and scanned old hardcopy photos, made a genealogy tree, and consulted clippings, letters, and scrapbooks. The images did not speak for themselves, so half the book was narration, organized around time periods, ancestors, and significant others. It was a lot of work, and her loved ones treasured it.

Right after she had passed, and I had handed out the book copies, I learned of a battle she had with her employer, the US Postal Service. It was about her harassment by male co-workers, reinforced by institutional discrimination against female employees— in the 1950s. She won, retiring with honors from USPS management many years later!

Her memorial service was done by her friend, who told us about her spiritual development and how others depended on her for prayerful guidance.

Even if my memory book gets passed down to later generations, not lost in a box, it was made too soon to contain that discrimination story or her spiritual odyssey. While it shows her paintings, it doesn’t tell that she could write crafted sentences in a gorgeous flowing hand. It will impart only a tiny glimpse of her character or of what it was like to know and love her, because any single narrative about a person is so limited.

Mom’s pre-view of heaven?

Our own forgetting starts long before we die, and then our physical traces and others’ memories of us continue to decay after death. Why should the memories of individual lives crumble into dust when we are mastering information’s power and permanence?Given current tech developments, a way to fix being forgotten is foreseeable. It might not be a techno-utopian fantasy.

The concept emerging in recent years concerns a form of replica that is built upon data about the focal (deceased) person’s history and personality and that is thus able to express this information as if the deceased were conversing with you. This could be effective even if it only happens via written words, without any simulations of the focal person’s voice or appearance. These days we are quite readily engaged by text, even when it’s only a stupid chatbot on the other end.

Let’s look at some serious challenges to overcome and then discuss how replicas might fit into our lives and culture. Much of this I have not seen articulated elsewhere.

What It Would Take

The big AI companies claim to have solved the problem of having machines generate meaningful written language. If you are somewhat technical, you might even have tried interacting with one of the LLMs. If you ask one of those programs a question, you will get an answer that is coherent and sometimes is (somewhat) to the point. You can attempt to have it mimic the utterance styles of famous people, but again: it’s not always very convincing or individualistic. Many experts say the LLMs will get better: similar models for creating images seem to emulate particular visual styles rather well. So, language models might be a big enabler for personal replicas. I assume here that those models will be necessary, but not sufficient.

What would make a convincing, engaging replica? It should speak with the characteristic phrasing of the focal person and about the person’s concerns. It should do that while being sensitive to the context of what has gone before in a conversation and being inventive and spontaneous. As far as I know, today’s replicas are rather far from this level of effectiveness.

You could use many sources of textual data to train a replica program to behave like a known person. These days we have data recordings of audio and video conversations, blogs, text messaging, email, and other social media. For more historical figures there are their letters and other writing, as well as what historians, biographers, novelists, and filmmakers have said about the person. The crux would be how to connect the personal information to more general conversational capabilities. This is where LLMs come in.

Smart parrots and personality

Nearly any AI researcher will tell you that an LLM, with its ability to respond to questions or elaborate on stories, is just a very capable mimic of human discourse. It doesn’t understand words and sentences. Instead, it has been trained with real language examples of all the ways that words and sentences go together. Buried or implicit in those connections among words might be knowledge of the world that the words refer to. I say “might” because experts disagree more on this point.

Think of such programs as being like a super smart parrot, Jim, who lives in a busy restaurant. He picks up bits of dialog from many patrons and soon learns to mimic how people talk. Jim would not learn the style of any particular person, but for anything you might say to him, he would have a plausible response. The more long conversations that Jim hears, the better he would mimic how a real conversation might go.

Jim hears and incorporates bits of language from many people. Suppose the restaurant closes, and Jim goes to live with a customer named Mae. She’s a sociable, talkative woman with many friends, who visit in her living room where Jim can hear her conversations. Will Jim start to “parrot” Mae more? Sure. Will he often say things that seem unlike her? For sure.

If you know Mae then you will know “what she’s like.” But how can we infuse that essence into a replica? What are the dimensions of resemblance to a real personality? How do we go from conversing in general to conversing like a particular person?

Conversational Style. For one, each person has their own characteristic vocabulary, favorite phrases, and conversational tactics. Do they go for wit? If so, then do they prefer quips or anecdotes? Do they answer questions with questions? Do they take a long time to get to the point? Do they volunteer explanations of events? How much emotion do they express?

Social Attitudes and Beliefs. Decades of research have established the social nature of personality. Much of our personality is both created and expressed with regard to people whom we know. To a great extent, we are a different person in each of our close relationships. We will have memories specific to each loved one. We also relate to new acquaintances according to a collection of general social norms and personal likes/dislikes, such as: be kind to the old and the young; trust everyone but always cut the cards, don’t suffer fools, or avoid people who believe X. Our judgments of a new person evolve over time, depending upon what we both do and how we and they use “theory of mind” to understand what the other person might be thinking.

Knowledge and Personal History. There’s a similar connection with life experience and interests. What does someone know about the world? What can’t they know because it happened after they died? What are they interested or expert in? What pleases them, and what can they not abide?

We can hope that training an AI on enough data from a person will capture these personality characteristics, but that alone will not suffice to make a replica.

Merging individuality with fluency

Just to get off the ground our replica needs a level of language fluency. If Jim only ever lived with Mae and you ask him about something that Mae never mentioned, he won’t know what to say. The fluency of today’s LLMs depends on data collections that are billions of times larger than any that we might obtain from a single person. Billions!

It will be necessary to give a replica the language ability and implicit world knowledge of a (hideously expensive) large language model and yet somehow train it to be like aunt Mae or uncle Arthur. Wearing my developer’s hat, I’ll propose a (realistically vague) approach that would use interacting software models.

Suppose we have a world model, which is an LLM that represents much of the world’s textual data in a given language. This model would be very like the LLMs that we are creating now.

There is also a personal model based only on textual data from a focal person. The personal replica is a combination of the world model and the personal model, along with some programming to make the models work together.

You (a real person) write some words as a prompt to the personal model. The personal model generates a possible response along with two scores for the response. One score estimates how specific it is to the focal person: how likely is that they would have said it in real life. The other score estimates how suitable it is, in general, as a response to the prompt. If the response is both specific and suitable enough, then the system sends the response to you. An exchange has been made. It’s your turn to talk again.

On the other hand, the suitability might not have been high enough to use because the personal model just doesn’t cover the conversational situation. So, the personal model’s response is passed to the world model, which generates an alternative response with higher suitability. The personal model in turn then scores the alternative with regard to specificity (again, that’s how well it resembles something that the replica model might say). If the specificity score is high enough, then the system responds to you with it. Once more, you have made a conversational exchange.

This is only a skeletal idea of how it might work. A real AI system would be way more elaborate, trying many alternatives back and forth, with more types of scoring, before completing an exchange or saying that it doesn’t know how to respond. The overall process would hopefully produce details of conversational style, attitudes, beliefs, interests, and experiences that are sensibly like the real focal person.

But there’s more that we need. An exchange occurs in a conversation, and that conversation occurs in a context. A personal replica will do better by also modeling that context.

Contextual Modeling

Interlocutor Identity. One part of the context is for the replica to know with whom it is interacting. When conversing with a significant other from the focal person’s life, the personal model needs to associate specific interaction patterns with the identity of that SO. This connection allows for forming a response that relates to shared events, conversations, and expressions of feeling that involved the SO in the past. The personal model’s calculation of response specificity would include how it would fit a conversation with that SO.

Another aspect of context would be a memory of the replica’s previous conversations (as an AI) with each real person, whether or not they were important in life to the focal person. Without this feature, a conversation with a replica would be frustrating, like talking to a real person who has dementia.

Mind Flow. The flow of a real conversation depends on how those talking feel from moment to moment. David Gelernter (in his books: The Muse in the Machine and Tides of Mind) reminds us that our minds also flow. Our current perceptions connect with our memories and cause changes in emotions, attention, and subject matter. We would detect the lack of these when talking with a replica, and we would notice when it failed to respond appropriately to our changes.

Going back to Jim the parrot, suppose there were times when the restaurant was not very busy, and he would hear longer conversations at nearby tables. If the patrons were letting their feelings and ideas flow and emerge in conversation, then Jim might achieve mimicry of some of that flow. But he’s still a parrot that has no idea about what he’s been hearing, so maybe he would not be that convincing.

If we had a broad enough sample of a focal person’s language then a replica might sometimes mimic flow. It might also derive some spontaneity and variety from the world model. But just like us real humans, a replica would do better on flow by having as a context some memory of what has been said in current or earlier conversations.

Stability and New Knowledge. There are a couple of more issues with regard to the replica’s memory context. Suppose you ask Replica Uncle Art a question that is out of his context: a subject he doesn’t know about, perhaps something that happened after the real uncle Art died. As we have said, the personal model has to use the world model to fill in the gap. Or perhaps the interlocutor explains something to the replica. The resulting conversations become part of the replica’s memory context.

However, if a replica is changed in this way by its conversations with others then it might progressively lose some resemblance to the person on which it was modeled. Designers might have to choose the kind of change allowed. Depending upon its purpose, a replica might need stronger or weaker resistance to change. Sometimes we might even give a replica a memory reset.

In summary, I have imagined how a truly effective personal replica might be achieved by combining the conversational power of large language models with learned details from the writings of a focal person’s life, modulated by memories of the replica’s own conversations over time. This would be hard to do. Let’s now consider reasons for doing it.

---

What Good Are They? What Bad?

I told my daughter about the idea of making a personality replica to carry on after someone dies. She reacted with vehement, vocal disgust. Why would someone feel this way? They might perceive disrespect to a focal person’s memory and/or to their lack of consent about how a replica would be used in the future. Maybe it’s just creepy to say that a computer could have any relation at all, any similarity, to a real person.

These are very valid and compelling objections. The counter-argument goes as follows. Replicas can also be seen as just a step beyond eulogies, grave monuments, picture albums, biographies, tribute websites, portraits, posthumous awards, ancestor worship, and other ways we honor the dead.

Replicas would be compressed memories with a conversational interface. They would remind you of, but not by any stretch replace, a loved one. Some people hallucinate their lost loved ones, retaining their influence as a mental presence. Why shouldn’t those of us who can’t sense our dead get a less intimate way to help remember them?

Simple replicas already exist. More will get made, and with a lot better fidelity and interactivity. They might become popular, but maybe not cheap. It is hard to imagine that good ones could be made without a lot of human curation effort, even after all the basic infrastructure programming had been paid for.

If social media and current bots are any examples then replica prevalence, sadly, will not depend on whether they do more good or more harm. As usual, we would benefit from forethought and debate.

When some people approach the end of their life they think about writing a life story, a memoir, or even an autobiography. The purpose might be to pass on lessons learned. Maybe it’s in order to become something besides a headstone and decaying memories in the minds of significant others. It could be an expression of undying love to those who mattered. Or, a finger in the eyes of their enemies. Some might write their stories because they think the world just cannot do without them. These are all also reasons why people might create, or ask for the creation of, their personal replica.

However, once we know how to make a replica, uses other than the personal come to mind. Individuals or institutions might also create replicas of famous or important people. Their purposes could range broadly across a spectrum from propaganda to education or advising. Some replicas might teach history, adding immediacy, like the talking recordings that are appearing in museums.

The chance of a replica making new accomplishments or innovating in some way seems to be distant from what we can expect. But other uses seem likely. Science fiction, such as William Gibson’s Neuromancer, has long imagined ancestor replicas guiding the fortunes of descendants. This might work for more conservative social organizations. In fact, they might jump all over it.

Something in between educational and advisory value might ensue if you had famous replicas confer with one another. Writers have often used the device of such dialogs before. You might benefit from assembling a personal advisory board, with replicas of wise, successful, or famous people. There could be AI help — like an special offshoot of today’s LLMs — to resolve or clarify disagreements among the board members.

Getting advice from a replica could benefit you with less effort than trying to read and understand everything that the original person had said in life. Thus ultimately we might study the wise less but consult them more. This possibility is surely hair-raising to contemplate. Scholars would decry it as shallow — unless, of course, they got work in the creation process. The influence of such replicas could, like that of celebrities today, be enormous and could be abused by either their creators or their users.

Less authentic “replicas” would certainly be used as trolls. We can expect ostensible replicas whose opinions reflect only the agenda of their creators and not the wisdom or the full perspective of their focal person.

If you cherry-pick the respected founding intellects of the United States you could come up with some repellent, immoral advice. The same is commonly and especially true of picked advice from religious tomes. We are already swamped with bad biases drawn from the past.

We are, believe it or not, in the early stages of AI influence on our culture. We might do well to imagine accurate historical oracles as a way to make such influence more benign, personal, and accessible. They could explain their advice and refer us back to primary sources for validation. They could be the anti-trolls instead of just better trolls. Is this nerd idealism, or something we could do?

We ought to begin establishing the rules while we are still opening this new can of worms. With replicas, evolving laws concerning artificial persons might be a starting point. However, this should be disentangled from the sociopathy-encouraging laws governing our original artificial persons: limited liability corporations. At a minimum, there should be both legal and technological means to restrict who can use, copy, or modify a replica. This would allow, for example, a provision in a person’s will for protecting any future replica of themselves.

The topic of personal replicas does inevitably lead to science fiction tropes. Creators of an effective replica could not predict what it might say. They can’t do that for LLMs today. Would a replica have some right to free speech, or be liable for its misbehavior, independently of its creators? Can you stop a replica demagogue? Will these memorial creations be viewed as parrots or persons; caged or free?

---

Postscript: The world is abuzz with the notion that an LLM could be conscious/sentient (whatever those mean), even though that has been repeatedly debunked. The same conclusion would apply to replicas, which would be only a refinement of LLM tech. But, their behavior will make many think that replicas are people. The issue could be messy. What do you think?

And another P.S.: For a not very current but comprehensive academic-style review, see Digital Immortality and Virtual Humans. This was followed by a book, AI for Death and Dying.