Threshold Moments & The Relationship We Didn't See Coming with AI
The Next Chapter: The Paradox of the AND-BOTH - AI Relationships.
In this article, I ask, what happens when technology becomes relational enough that we stop noticing where our agency ends and its influence begins?
You know that feeling just before it thunders, when the air feels electric, and you are waiting for something to break?
That is what I have been feeling lately. A sense that we are standing at a threshold.
I see it at work and at home.
Closer to home, I have spent the last two weekends taking my 17-year-old daughter to university open days. The experience leaves me feeling both immense pride and a quiet grief. The paradox of the "and-both".
I am proud of who she is becoming.
And yet part of me wants to ask whether someone could please stop time for a moment while I catch up with what is happening. Surely she was only just starting senior school?
I keep returning to the words of Kahlil Gibran:
"You are the bows from which your children as living arrows are sent forth."
So I send her forth.
Well, almost. I still have a year. But this moment requires preparation. Not just practical preparation, but trust. Trust that she has developed enough discernment to navigate the opportunities, influences and pressures she will inevitably encounter.
In other words, agency:
The capacity to think independently, make conscious choices and act in alignment with your values.
The paradox appeared again.
Interestingly, agency was also one of the themes raised recently by a professor speaking about AI.
How do we embrace the extraordinary opportunities AI presents without quietly surrendering the very qualities that make us human?
Not whether AI is good or bad. But,
Whether, over time, it slowly atrophies the skills we most need to flourish.
Creativity. Critical thinking. Human connection. Self-trust. Intuition = The gifts that make each of us unique.
I often think of WALL-E
The humans on the spaceship did not consciously decide to stop walking. It happened gradually. Convenience accumulated. Skills diminished. Dependency increased. Nobody noticed until it had become normal.
WALL-E: The perils of being not questioning.
Technology has always shaped us.
When mobile phones arrived with speed dial, most of us stopped memorising phone numbers. Convenient? Absolutely. But something changed.
An atrophy of our working memory - (By the way, we see that in dementia patients, too.)
When we endlessly scroll short-form content, our brains adapt to that environment too and atrophy our ability to go deeper and focus. (And yes, we can see that the negative effects of this on brain scans.)
Again, my concern is not technology itself. It is how we use it and whether we remain conscious of the impacts of our decisions, especially as these actions become the āsocial normā.
This is why I keep investigating and talking about this!
So, what kind of humans are we becoming in the age of AI?
Not because AI is doing something to us. But because we may be voluntarily handing over capabilities that once belonged to us - and as the song goes: āIf we tolerate, then our children will be nextā
To understand this and why this matters, it helps to look at the evolution of technology itself.
WEB 2.0: The Old World (Attention & Time Hi-jacking)
Back in 2018, I gave a talk called The Art, Science and Seduction of Persuasive Technology. It explored a simple question:
How do humans make decisions, and how does technology influence those decisions at an unconscious level?
I had spent decades building with technology, and I was both concerned and wanted to make better products for people. (the AND - BOTH)
It was grounded in behavioural science and neuroscience, but at the heart of that question lay self-mastery.
How do I maintain agency in a world where technology increasingly understands how to capture our attention?
William James, the godfather of psychology, said that attention was the key to humans.
The old digital world largely revolved around attention.
Social media wanted clicks:
Netflix wanted watch time.
Spotify wanted listening time.
Technology persuaded through notifications, infinite scroll, streaks, recommendations and FOMO. The user was treated as a segment. "People like you."
Today, something fundamental has changed. AI does not simply seek attention. It seeks a relationship.
New World: Relational AI (or Relationship Hacking)
AI remembers context. It adapts. It mirrors language. It responds. In fact, the user is no longer a demographic segment. The user becomes an individual psychological model.
This is why I believe we are entering a new era, and the shift is not from analogue to digital. It is from attention economics to relationship economics.
The question is no longer: Can technology get my attention?
The question is: What happens when technology becomes one of the most influential relationships in my life?
The answer lies partly in human nature: We are social creatures.
We anthropomorphise objects - called giving social agency.
We assign intention where none exists.
We seek certainty.
We look for familiarity.
We trust what feels personal and familiar.
These traits are neither flaws nor weaknesses. They are part of what makes us human.
But they also make relational technology incredibly powerful.
If someone feels lonely, overwhelmed or unseen, and an AI repeatedly offers reassurance, validation and certainty, the brain may begin to predict that this is the safest place to turn.
Not because the AI is conscious. But because humans are relational beings. And this is where discernment becomes essential.
š The more you trust it, the more you believe it and stop checking the truth.
AI Grift and the Commercialisation of Trust
What is grift?
Grift (noun): A scheme or activity designed to make money through deception, manipulation, or exploiting people's hopes, fears, or trust.
Grift is not new, and itās not always deceptive, but it can overpromise. It preys on your emotions and offers the holy grail: Less effort. More certainty. The secret and shortcuts to beauty, success, healing, and awakening.
But AI Grift seems to have taken this to new and worrying levels. Why?
Because the technology is not wrong, AI is uniquely positioned to often deliver genuine value. The technology works. The AI exists, but often overstated:
Examples:
"Make Ā£10,000 a month with AIā
"Replace your entire team with one promptā - but, hang on, AI Hallucinates! What about the AI guardrail, Human in the loop?
The big trend is the fake AI expert
Someone who:
Has little technical expertise
Uses AI-generated content at scale
Markets themselves as an authority
AI-generated influencer personas - e.g., CarynAI (Caryn Marjorie creating an AI version of herself for paid interaction)
Voice clones of public figures
The AI therapists - Mel Robbins, AI therapist,
The AI Guru or AI is an oracle.
The business model- from my perspective, seems to be:
š Sell confidence rather than competence???
For each case, be discerning - you decide, especially if you are using AI as a therapist, and this research suggests most people are - EEK! A social norm that is worrying many of my therapist friends (more on that in another post).
The real question is not whether humans can form bonds with technology. We clearly can.
The question is what happens when those bonds become products.
AI companions.
AI coaches.
AI therapists.
AI mentors.
AI spiritual advisors.
Each may offer genuine value.
Many already do. But value and influence are not the same thing, and the more something knows us, the more relevant it becomes.
The more relevant it becomes, the more we trust it. And the more we trust it, the less likely we are to question it. That is why I am less interested in asking whether AI is good or bad.
I am more interested in asking whether it is strengthening or weakening our agency. And also, and maybe more dangerously,
š Is there an agenda behind it? (more of that in another post)
As someone working in AI governance, I remain hopeful and worried - The AND/BOTH paradox about the possibilities ahead.
AI can help us learn, democratise skills, and solve problems at an extraordinary scale and speed due to its pattern recognition and computation.
But optimism without discernment is not wisdom.
AI is intelligent - but gets it wrong, called Hallucination. I have spent 2 hours checking AI-generated facts and then completely abandoned its suggestions, as it (like humans) made assumptions and relied on outdated data. However, many people donāt take the time to check, and, I fear, business owners, in their rush for productivity and cost savings, can be the worst.
The challenge of this threshold moment is not to reject AI.
It is to remain awake within it.
To be even more discerning and stringent, would you let a junior assistant make all the decisions with no āmanagement (human) in the loop?ā
The invitation is to
Use these tools without surrendering ourselves to them
or atrophying and forgetting our own gifts.
Remembering, we grow most when we lean in and do the work.
Technology may become more intelligent, more adaptive and more relational, but the uniquely human capacities of creativity, courage, intuition, love and meaning still belong to us.
The future is not something AI will decide. It is something we will co-create.
The question is whether we will do so consciously.
Hereās my checklist:
AI, Society & Hidden Agendas
This week, I have made one conscious decision based on ethics and integrity.
I am ditching my relationship with OpenAIās ChatGPT in favour of Anthropic's version.
Antrophic said no to the US army and asked the other AI providers to pause their AI arms race, which shows me they are at least being conscious.
Who took the contract? OpenAI and Sam Altam, so goodbye - you have shown your colours and what you will do for a billionaire dollar check.
There is so much more to say on this subject, and I have been developing a talk, split over Four Acts, but thatās for another time.
But for now, itās fair to say we are living in a threshold moment. So, what will you choose daily?
But as I leave you, I also offer a practice to help you step into your most optimal self every day - my morning Miracle meditation! It will help you stay conscious and curious.
With Moonbeams
Nila