Rehoboam by Accident

This is the thirteenth article in "The Shape of the Next Decade," a series on how AI reshapes work, institutions, and ordinary life. It follows The Reversal Gap. This article contains spoilers for the third season of Westworld.

In January, Google introduced Personal Intelligence, a feature that lets people connect Gemini to their Gmail, Google Photos, YouTube and search history. Google promised “a super helpful assistant that knows you really well.” To demonstrate it, Josh Woodward, who runs the Gemini app, described a trip to the tire shop. He was standing in line when he realized he didn’t know the size of the tires on his family’s 2019 Honda minivan. “These days any chatbot can find these tire specs,” he wrote, “but Gemini went further.” It found the family’s road trips to Oklahoma in Google Photos and used those trips to recommend an all-weather set, with ratings and prices. When he reached the counter and needed the license plate number, Gemini read the number off a picture so he wouldn’t have to walk out to the van. It also found the trim in email messages from when he purchased the van. The launch post described this as saving Woodward a headache.

Woodward also describes Gemini helping plan their spring break by “analyzing our family’s interests and past trips.” It skipped the tourist traps and suggested an overnight train with specific board games for the ride. To make those recommendations, Gemini had to interpret what the family’s photos and correspondence revealed about what they enjoyed.

Further down, under the heading “How you can help us improve,” Google acknowledges that Gemini can misread those interests. Seeing hundreds of photos of you at a golf course, it says, might lead Gemini to assume you love golf. "But it misses the nuance: you don't love golf, but you love your son, and that's why you're there." If that happens, Google says, you can just tell it: "I don't like golf."

That fix assumes you know what the assistant has decided about you. A father can correct a conclusion about how he wants to spend his weekends once he sees it written down somewhere. If the assistant concludes that he would rather stay close to home, though, it can use that judgment to select the chores and trips it recommends without ever thinking to ask if he agrees. The conclusion reaches him through the options he’s offered, and eventually through the life he has.

A future on file

In the third season of Westworld, the story leaves the theme park and follows Caleb Nichols through a near-future Los Angeles. He is a veteran who works construction by day and takes small criminal jobs through a gig app at night to supplement his wages. One of his accomplices is a man called Giggles, played by Marshawn Lynch. In the first episode, Caleb interviews for another job, is turned down, and asks for feedback so he can improve his chances next time. The voice on the other end of the call, which he had taken for a recruiter, turns out to be automated. His question goes unanswered, and he opens the uber-for-crimes app to look for his next gig.

Then he meets a woman who knows things about him she should not know. Her name is Dolores, one of the park’s androids now loose in the world, though Caleb takes her for a stranger in trouble. In the third episode, she takes him to a diner he remembers from childhood, the place where he was abandoned as a boy. She knows about that day because she has access to a profile compiled from records of his life. The system that assembled it has also produced a forecast of his future.

At the end of a pier, she shows him the forecast. It describes a man who will not advance beyond the work he is doing, whose relationships will not hold, and who will, some years from now, kill himself at that pier. Dolores explains that employers consult the system’s predictions and will not invest in someone they expect to die by suicide. Caleb has been trying to improve his prospects by applying for jobs, while employers have been using those prospects as a reason to turn him down. The rejection that opened the season now reads differently. What he understood as a judgment of his application was based on a judgment already made about his life, one he did not know existed and could not address by asking the recruiter how to improve.

Researchers call this performative prediction, in which acting on a model’s predictions changes the outcomes it predicts. Caleb’s projected death remains a forecast, but employers are already making decisions as though it were settled.

The park’s androids, known as hosts, lived out storylines written for them. They were reset after each loop and made to repeat it. The drama of the first seasons came from the hosts discovering that their lives had authors. Season three carries that idea out of the park. Caleb appears to live outside those scripts, yet his attempts to change his life keep running up against decisions made by a system he cannot see. Lisa Joy, who created the show with Jonathan Nolan, told Variety that the gods of the earlier seasons were the tech people putting the hosts in loops. Outside the park, the people pulling those strings are harder to see. “You can't see their hands moving anymore,” she said, “so you don't understand what's being tugged.”

The system is called Rehoboam. One of its builders, the French billionaire Engerraund Serac, watched Paris destroyed in a nuclear attack when he was young. He spent decades building a system that could make humanity predictable enough to steer it away from collapse. His reasoning emerges in the fifth episode. If you had a model that could see wars and collapses coming, and intervening in individual lives appeared to head them off, you might decide that a managed world was better than a free one that ended. His purpose is easier to sympathize with than his methods. He wants humanity to have a future, and he is willing to decide what other people must surrender to secure it.

For Caleb, the cost first looks like a job he cannot get and a future he has never been allowed to contest. Most people encounter Rehoboam’s decisions as ordinary rejections whose purpose is never explained. But its forecasts depend on people behaving the way it expects, and it flags those it cannot predict as outliers. Serac has them taken to a facility for reconditioning; his own brother is among them. Late in the season, Caleb learns that he too had been classified as an outlier. Memories he trusted had been edited there, and the gig app he kept going back to had been using him to round up other outliers. For most people, the system works on the options in front of them. For the few it cannot predict, it works on the person.

Someone you might have become

Imagine a fictional household in 2029. Laura is thirty-six and lives with her husband, their son, and their nine-year-old daughter. She uses an AI assistant of the kind Woodward described, one that knows her calendar, her mail, her photos, and her spending. She has been using it long enough that asking for its help has become a reflex.

That spring a former colleague offers her a role in Chicago. It is the kind of job she and her husband used to talk about when they were first together. The plan had been to live in the city for a while before they had kids, and, as they so often do, the kids came first. She asks the assistant to help her think through the decision. The salary is better and the work would be more interesting. It would mean moving, a great deal of travel, or probably both. Her husband has only recently established his practice and wouldn’t be able to relocate for at least a few years. Her daughter is in elementary school, and Laura does much of the day-to-day parenting. The assistant does not tell her what to do. It says that on the information it has, staying looks like the choice that fits her life, and that she could revisit the question in a few years when things are less tight. So she decides to stay.

Laura drives her daughter to synchronized swimming practice on Sunday mornings. The only local club trains on the other side of town, too far away to make going home worthwhile, so Laura sits in the bleachers with a book that has grown swollen and wrinkled from the pool’s humidity. The drive home is when her daughter talks to her. Not at weeknight dinners, where her father and brother dominate the conversation. Not at bedtime, but in the car with her hair still wet. She talks about the girl who is being mean to her friend and the teacher who is unfair. When she is eleven, she asks whether she is any good at anything. Laura would have missed most of those drives if she had taken the Chicago job, and she knows it. She is not sorry.

By 2035 her daughter is fifteen. Laura now runs the team and knows everyone at work. The opportunities that reach her are mostly local and better than the ones she had before. When she asks the assistant to keep an eye out for something new, it shows her jobs that fit her life. From six years of choices, it has learned that she prefers work that does not disrupt her household. It gives those openings more weight, and the jobs that would require a move have thinned out. She did make those choices. But it treats that history as evidence of her preferences without accounting for its own role in shaping her choices, and she does not think to ask. She made the first decision to stay with its advice. Every choice after that was easier because of the one before, and each one went into the file as one more data point about a woman who likes to stay.

Nothing conspiratorial is needed for this to happen. Her own decisions changed her résumé, her network, and her obligations, and any recruiter reading her history would see a regional person with regional ties, whether or not influenced by her assistant. The loop runs through the world, not just the software.

In a September New York Times Well newsletter, psychologist Desmond Ong recommends using chatbots to gather information about a move or a job change while keeping the decision with the person. Laura has done that.

People can make their own decisions and still be influenced by the help they receive. In two experiments published in 2026, people wrote short essays with autocomplete suggestions deliberately biased toward one side of a contested question. Afterward, their reported attitudes had shifted toward the position the assistant favored, even though they remained in charge of what they wrote. Those experiments measured the effect of a single task, not years of advice. Changing what people see also does not necessarily change what they believe. A three-month experiment that switched Facebook and Instagram users to chronological feeds changed what they saw without significantly changing the political attitudes it measured.

By design, eventually

The pattern grew out of a useful product, a busy life, and decisions Laura had little reason to revisit. Neither she nor the provider needed to intend that her options would gradually narrow. Once the provider notices how reliably its recommendations influence choices, though, it can deliberately develop and sell that influence.

In 2036, the company behind Laura’s assistant adds employer-paid placements. Employers pay to have their openings shown to qualified users, and the provider’s analytics show that familiar, low-disruption matches lead to more accepted offers. Jobs that require a move, a change of field, or a temporary step down are less likely to be accepted, so the provider gives more weight to the matches that pay. A product team notices that these rankings are crowding out the exploratory recommendations users have explicitly asked for and brings the issue to a review. The dashboard on the screen shows that restoring them would cost a few points of placement revenue. It also shows acceptance rates up and complaints flat. Someone points out that people accepting the jobs they are shown says nothing about the jobs they asked to see, and nobody disagrees. The weighting stays. The provider is now using what it knows about Laura’s family commitments to help a paying employer fill a vacancy.

Facebook faced a related question about what to do with evidence of harm. In 2021, internal research from Facebook's own teams became public, showing that among teenage girls who already struggled with body image, a share said Instagram made it worse. Frances Haugen, the former product manager who disclosed the documents, told the Senate that the company had repeatedly chosen its profits over users' safety. Facebook disputed how the research was being characterized and said it was acting on the findings. OpenAI began testing ads in ChatGPT this year and says they are labeled and do not influence answers. But the capital committed to infrastructure raises the stakes of finding revenue. Conversations are one place a provider might look. If a provider later finds that a profitable feature is working against its users’ interests, it will have to decide what it is willing to change. The meeting in 2036 is one way that decision goes.

Laura’s assistant and the recruiting platform she uses both show her local and remote jobs. The remote listings draw hundreds or thousands of applicants; the local ones usually have fewer than twenty. With better odds nearby and two services pointing her toward similar openings, she has little reason to question the recommendations. What she cannot see is that the company behind her assistant has sold the recruiting platform a score predicting her willingness to move. An assumption drawn from the years when moving was difficult is now shaping her opportunities in more than one place, without ever being put to her as a question.

The assistant recommends a local opening where Laura could take on more responsibility without uprooting her family. She can weigh that trade-off herself. What she cannot weigh are the opportunities the provider left out, or how the employer’s payment influenced that selection.

When Caleb finally saw his file, he discovered how much of his life had been decided before he ever spoke to the recruiter. Laura sees jobs she could reasonably want and makes her own choices, each of which gives the system more history to draw on next time. That predictability would have suited Serac, who built Rehoboam to keep humanity from destroying itself. His business partner, Liam Dempsey Sr., had a simpler ambition. He wanted to make money from its forecasts.

Laura’s agent provider can make money from both ends of that arrangement, charging employers to steer her toward their openings and selling the recruiting platform an assessment built from the choices she makes. It knows she asked to see other possibilities. When its own team raised the problem, the review put a price on fixing it and decided the revenue was worth keeping. Laura gets a service that is worse at doing what she asked, while the alternative she might turn to has already bought the conclusions about her. The company keeps both piles of money and records another year of Laura apparently preferring the life it is paid to recommend. As the opportunities reaching her narrow, her choices become easier to predict. Serac used reconditioning to force unpredictable people to behave as his model required, with the noble-ish aim of saving the world from collapse. Laura’s agent provider can keep helping her choose among the possibilities it leaves open. Over enough years, it produces fewer outliers.

A life worth keeping

It is 2039 and Laura's daughter is nineteen and home for the summer. She is asking her mom for advice about a semester abroad when she says something that lands harder than it should. Didn't you and Dad always want to live in Chicago? What happened to that?

Laura remembers wanting to live in the city when they were younger, just out of college. The thought of being in the middle of it all still gives her a faint sting of nostalgia. That’s not her life anymore, and it is far too late for her to imagine what could have been. Later that night she recalls her daughter’s question and asks the assistant, half as a joke, “When did I give up on Chicago?” It provides the timeline: the offer in 2029, the original discussion, the decision to stay and revisit later, followed by the decisions she made in the years afterward. Seeing them one after another, she begins to notice how consistently the advice pointed her toward staying. She recognizes the decisions as her own, but remembers wanting more than the recommendations had allowed for.

Laura remembers her daughter beside her on the drives home from synchronized swimming, hair still wet, talking the whole way. She is glad she was there for those years. But staying had not been meant to settle the question for good. Laura scrolls back to that first conversation about Chicago. The advice to wait is still there.

Her parents, her employers, even the books she happened to pick up had all helped shape what Laura wanted from life. She might have stayed without the assistant. She wanted time with her daughter and got it. Had she taken the Chicago job, she might have found herself wishing she had stayed.

The philosopher L. A. Paul describes choices that change not only what we know but what we come to value. Becoming a parent is her central example. Before having a child, we cannot fully know what will matter to us afterward. Laura has already lived through that change. Moving to Chicago might have changed her too. Different work, new friends, and a different life with her daughter might have become as important to her as the life she now wants to keep. An assistant could help her explore such a change, but not if it keeps treating her earlier decisions to stay as a reason to leave those possibilities out. Laura knows she values the life she has. She cannot know what she would have come to value in Chicago, or how that version of herself would look back on the decision to move.

Laura could still ask to see the openings being filtered out, reject the advice, or change providers. An assistant paid only by her might help her explore possibilities the current service has stopped showing her, but it would be helping a woman whose career, marriage, and family life have taken shape around the decision to stay. Her husband’s practice is established here, she runs a team she knows, and their daughter is about to leave home for good. There may be room now for something different, though it would not be the move she considered ten years earlier. A better assistant could help her make a change now, but it could not give her the years she might have spent building a career and getting to know people somewhere else. Whatever advice she seeks next will begin with the life she has already built, including the parts that earlier advice helped her choose.

Laura is still glad she made many of the choices the assistant helped her through, even as she begins to wonder how its advice changed what she wanted. The history it returns follows the decisions she made, not the decisions the company made about her. She can revisit the reasons she gave for staying, but she cannot see the review where her requests for a wider search were weighed against placement revenue and set aside. She agreed to the jobs she took, not to that bargain. For now, the record gives her plenty of material for questioning herself and very little for questioning the company.

The man with the earpiece

In Westworld, Dolores exposes what Laura still cannot see. Midway through the season, she releases Rehoboam’s files to the people whose lives it has been predicting. On a train, passengers look down at forecasts about their marriages, careers, and futures. These are not simply predictions they can choose to believe or dismiss. Employers and other institutions have already been making decisions based on them. Caleb’s discovery is happening to people across the city at once, and the streets descend into chaos. They are learning how much of what they took to be their own failure had been decided in advance.

In the season’s final episode, Serac is revealed to be taking instructions from Rehoboam through an earpiece. He built the system because he had seen what people could do to one another and wanted humanity to survive them. Now he waits for it to tell him what to say. He still owns the machine and remains responsible for what it does, but the judgment he meant to use has become the judgment he depends on. To defy it would mean trusting himself over the system he built to prevent another catastrophe.

Laura began with a much smaller fear of losing something she loved. The advice to stay let her keep a promising career and be there for her daughter, and she kept turning to the assistant as their lives changed. A few days after their conversation about Chicago, she asks it to show her the openings it stopped surfacing. It produces them without complaint, each with a date and an explanation of why it had seemed a poor fit. She opens the list determined to find something she should have been shown. The first few jobs look promising, until she reads the dates and remembers what taking them would have meant. Her husband was still building his practice. Her daughter needed to be driven to swim practice. Laura lingers over one opening, imagining the conversation they would have had about it, and finds she would still have chosen to stay. She closes the list before the end. The reasons are good ones.