Double Feature! Cult Design & My Stance on AI
This month we have a very special double feature! I had planned and promised to write about cults- what makes a cult, how do they work, etc. But, I’ve also been asked a few times not about my stance on AI, so this month you’re just getting both!
Scroll down to the next cat picture for my AI stance; we’re doing cult stuff first.
So, if you’ve read Secondhand Origin Stories or Names in Their Blood you’re not going to be surprised that book 3 is diving further into the genetic engineering cult that created Solomon and that whole side of the family.
Which means, fundamentally, that I need to design a cult from the ground up.
That means theology, rituals, hierarchy, methods of enforcement, even aesthetics.
I mentioned recently on Tumblr that I was struggling with developing weird rituals and beliefs without them being distracting or becoming the “reason” the cult is creepy and bad. (I’m not interested in reinforcing the framing that unfamiliar religious practices are bad and scary) and someone online asked me, very reasonably, why the cult’s beliefs have to be weird.
And then I remembered that most people don’t actually know much about what makes a cult.
And that seemed like an interesting and worthwhile subject for this month’s newsletter.
I, personally, favor Dr Steven Hassan’s BITE model of understanding and evaluating cults. The acronym stands for four domains of control exerted by cults.
Behavior Control
Information Control
Thought Control
Emotion Control
Let’s break it down.
Behavior control is, at it’s base, a combo of reward/punishment, and material control. Whan I say material control I mean things like having your money, food, appearance, housing, family, and major life choices (like who’s married and who gets to raise kids). Once you have total control of someone’s family and material needs, you can meet out rewards and punishments with drastically lowered risk of them pushing back. It’s a multi-directional hostage situation. Many of the overt horrors of cults and most obvious tactics are found under this heading, since this allows for extremes of punishment.
But to obtain this level of control, you need the rest of the acronym.
Information control is about cutting members (and, if possible, potential members) off from external influences. This means no unauthorized books, TV, Internet. Critically, it also means mediating and controlling every conversation with outsiders that goes beyond smalltalk. Which is part of why aggressive evangelism is often a must. This evangelism gives members a strict set of talking points that keeps them from getting involved in conversations that could give them access to broader worldview or people who might want to help them get out of the cult. Only the most trusted insiders; those highest in the pecking order, have access to the outside world. They have the most rewards, and so the most to loose if they step out of line.
This, like so many things in a cult, is about differentiating the community of insiders- the believers, from “corrupting” external influences. Things like phone line/Internet monitoring and buddy systems help keep this enforced. Evangelism is the stated goal, but isolation is the real function. Did you ever wonder why door to door or streetcorner evangelism is a thing? You can see that it clearly isn’t effective- this behavior repels most people. But the negative reactions of the “outsiders” reinforced that the outside world is dangerous and cruel. Safety and warmth only exist inside, and only for members in good standing.
Thought control is about completely warping a person’s perception of reality. A lot of people are at least passingly aware from pop culture that cults may use meditation, hypnotic states, fatigue, hunger, substances and other altered states in order to make people more malleable. But a lot of the techniques cults use to influence member’s thoughts are far more mundane.
Every thought must be categorized as good or bad. Bad thoughts are thoughts that aren’t compliant with cult priorities, doctrine and hierarchy, and they are pursued and punished at least as much as “bad” actions. People are taught to aggressively police their own thoughts as well as each other’s (but not those at the top of the hierarchy, whose grasp of the situation is accepted as purer, more complex, and generally beyond basic members). Critical thinking and analysis are rejected in favor of faith in the system. They have to be, to ensure leadership is completely safe from external standards or criticism. “Thought stopping” is a technique that replaces actual thought with simplistic slogans or sayings. For example, dismissing anything that promotes unauthorized empathy as “woke”. What does that mean? It doesn’t matter. It’s woke and therefor bad.
Emotion control is mainly about fear and shame. It is imperative that people are terrified of what will happen if they leave the group, so that they will tolerate abuse and control. Consequences are invented, threatened, and, where possible for earthly beings, enforced. Making independent decisions must be made terrifying. Sin or the system’s equivalent must be interfering if a member would reject the cult’s control. Social shunning must be made even more terrifying. The whole outside world must be made of danger. On the flip side, those who are doing things correctly must be in a state of perpetual peace and joy, and showered with praise and affection. Which means if you’re not in a state of perpetual peace and joy, it’s because you’re doing something wrong. Other feelings - like anger or dissatisfaction, that isn’t directed at outsiders, is a sign that you are doing/thinking Wrong Things and are going to be In Trouble.
This means that your feelings become suspect, invalid and unreal. Only the approved doctrine and emotions are safe and can protect you.
And there you have it. If your feelings are suspect and unreliable, your thoughts are suspect and reliable, you rely on the hierarchy to provide for your basic needs and decisions, and you’re cut off from influences that might break these cycles, you’re well and truly trapped, and not even allowed to know that.
You, penpal, might notice that there’s actually not much in there about God or anything. The actual material doesn’t need to be explicitly religious. The framework can be spiritual, financial, wellness-oriented, or political. It doesn’t actually matter. What matters is the mechanisms of absolute control.
So that brings us back to the question I got online- why does the cult need to have weird beliefs?
The answer is that to create an in group and an out group, a leader needs to convince their followers to buy into things that alienate them from the society around them. They have to see a stark contrast between themselves and outsiders. They have to convince potential members that they are offering something special and unique. Which means they can’t be doing what everybody else is used to.
The point of me making the cult practices isn’t to shock and repel the readers with unfamiliar tradition and doctrine. The point is that my fictional people have to do and say things that other fictional people think is weird. So that those outsiders can reinforce to the cult members that the outside world is weird and scary and will not accept or help them.
Critically, this does not mean that if you know someone in a cult, and you want to help them, that you should try to engage them in critical thinking via debate. Because by the time they are evangelizing they are well ensconced in thought-stopping techniques. In fact, trying to debate them may put you at risk of recruitment.
Some folks may be reading this and finding it familiar. The simple fact is- the US has a cult problem. And we have for over 100 years. In fact, the cult I wrote about in an early newsletter was part of a huge cult boom in the US. I’m not going to get into why that is right now- how long are these newsletters even supposed to be?
But if you or someone you know have experienced what I’m describing here, please check out this link here. The way out is more complicated than debate, but it can and does happen, and once you’re out, healing can begin.
My Official Stance on AI
Ok, I understand that since I have a positive depiction of a sentient AI in my books people are wondering about my thoughts and feelings on this controversial real world subject. So, here is my take, with links.
Part of the problem is that “AI” is being used to describe WAY too many things.
I love AI that exists to detect and understand patterns beyond what human brains are good for. AI that creates testable models for understanding climate change or detecting cancer cells before a human brain can- and that are still supervised by humans- are genuinely wonderful. I welcome them. I’m excited about them.
But hiding under that same name is a whole lot of other crap.
So here are some things I don’t like.
Firstly, in this age of disinformation, I am openly suspicious of things like ChatGPT that provide answers that have had all their attribution skimmed off them. It’ll answer- but where did it get that answer? Do I trust that source? Encouraging reliance on these functions discourages critical thinking and evaluation as well as keeping people doing the actual labor of finding answers from getting the credit. And I like falling down a rabbit hole of locating an expert and devouring all their good hard work.
Secondly, generative AI is an absolutely ludicrous drain on our power grids/water supplies, and is producing disgusting amounts of carbon emissions. I am angry but not shocked to see that new reports are suggesting that the emissions are up to 600% worse even than the already bleak estimates provided by the people trying to sell everyone on their product. Now is not the time to play around with carbon emissions, especially for something so unnecessary. For Gods sake, look at those hurricanes and droughts.
Thirdly, things like generative AI are theft machines, explicitly being sold as replacement for human beings. They steal from people who took the time and effort to learn a skill, in order to try to replace them- usually to avoid paying. And for what? So people who can’t be bothered to learn skills can have crappy to mediocre products for cheap. Even Zuckerberg has admitted that skimming this labor, without permission much less compensation, is iffy in terms of copyright. But he views what he steals as being close to worthless, until it’s time for him to profit off of it. He claimed that creators who don’t want their work scraped for corporate profit can have that option, but anyone who posts art or work online and doesn’t want it used for training data knows that platform after platform has made posting online without “agreeing” to this functionally impossible. Additionally, Microsoft AI CEO Mustafa Suleyman argued earlier this year that anything put on the internet is “freeware” and fair game for scraping and profiting off of, which is absolutely buckwild.
These are not people who are trying to help creators. They say this work has no value- until we need it. Not enough value we will pay the people who made it, of course, but enough value that we must not be prevented from taking it for our own profit.
I am of the exact right age to remember when a Native American woman in her 20s, from my own Minnesota, was subjected to 7 years of trials as the music industry tried to destroy her for uploading 24 songs. At one point she was ordered to pay 1.2 Million in damages. This was eventually brought down to about $200k, which was the cost of a nice house in the area at the time.
My peers and I were constantly being told that even downloading pirated songs was illegal and every tape had a warning that the FBI would ruin your life if you copied a VHS tape.
Somehow, downloading media from huge companies for private consumption is bad, but stealing from millions of unattributed creators so you can profit off it it is fine?
Truly, I dream of a future where corporations are held as accountable as poor people.
Fourthly, a bunch of AI’s reportedly miraculous functionality isn’t even actual computation. “AI” is a cover companies are using to obscure using traumatized people in less financially competitive countries who are doing a bunch of labor for $2 an hour. Even when employees are in the US, companies have laid off skilled expert workers and then hired them back on to mindlessly clean up the low quality crap the AI is putting out, without attribution, to make it look like the machines are putting out better work than they are. Journalism and translation have been hit particularly hard.
Fifthly, opaque algorithms and “AI” are being used to escape responsibility. UnitedHealth Insurance recently got in hot water for using AI to accept or reject claims. They were already notorious for inappropriately denying claims (a habit which, to be clear, does result in deaths) but now they have an AI to blame, so that no actual humans can ever be held accountable. Humans are known to put too much faith in the “objective” conclusions of machines, which makes pushing back against AI made “decisions” harder. This makes our corporations even more inflexible and inhumane. They did not need more help in that department.
Finally- even if we enacted Universal Basic Income and had a completely green energy grid that could easily handle this massive strain without devouring the all the drinking water in the area, I wouldn’t want art, music or books by AI. Very simply- if nobody could be bothered to make it, then I can’t be bothered to care about it. Art is how we connect to each other. I want art that connects me to it’s markers.
Only people who don’t make things think that the real work is coming up with “the idea”. Ideas are easy. It’s the execution that brings the idea to life. And that is the bit people are trying to shortcut through. And why? Creating things is wonderful! Why would we outsource that? If you want to make something go make something! Yes, it might suck, but improving at skills slowly is good for your brain and your heart.
So- do I like AI? I dislike cancer and climate change. When AI is working against those forces, I like it. When they are contributing to those, as well as corporate greed and lack of accountability, I am against it. And I resent the relentless messaging from those who seek to profit off AI’s seedy underbelly that their uses for AI will be ubiquitous and inevitable.
And that is my official statement on AI.
As always, thank you so much for joining me, and I will see you in the comment section (available in browser) or next month in your inbox!
Best wishes,
Lee Brontide