Do Sufficiently Large Organizations Start Acting Like Malevolent AIs? (pt 1)

(and Society is an Extremely Large Organization)

What do I mean by malevolent AI?

AI typically refers to any kind of intelligence or ability to learn possessed by machines. Malevolent AI occurs when a machine pursues its programmed objectives in a way that humans find horrifying or immoral. For example, a machine programmed to make paperclips might decide that the easiest way to maximize paperclip production is to enslave humans to make paperclips for it. Superintelligent AI is AI that has figured out how to make itself smarter and thus keeps getting smarter and smarter. (Should we develop malevolent superintelligent AI, then we’ll really have something to worry about.)

Note: people who actually study AI probably have better definitions than I do.

While we like to think of ourselves (humans) as unique, thinking individuals, it’s clear that many of our ideas come from other people. Chances are good you didn’t think up washing your hands or brushing your teeth by yourself, but learned about them from your parents. Society puts quite a bit of effort, collectively speaking, into teaching children all of the things people have learned over the centuries–from heliocentrism to the fact that bleeding patients generally makes diseases worse, not better.

Just as we cannot understand the behavior of ants or bees simply by examining the anatomy of a single ant or single bee, but must look at the collective life of the entire colony/hive, so we cannot understand human behavior by merely examining a single human, but must look at the collective nature of human societies. “Man is a political animal,” whereby Aristotle did not mean that we are inherently inclined to fight over transgender bathrooms, but instinctively social:

Hence it is evident that the state is a creation of nature, and that man is by nature a political animal. And he who by nature and not by mere accident is without a state, is either above humanity, or below it; he is the ‘Tribeless, lawless, hearthless one,’ whom Homer denounces—the outcast who is a lover of war; he may be compared to a bird which flies alone.

Now the reason why man is more of a political animal than bees or any other gregarious animals is evident. Nature, as we often say, makes nothing in vain, and man is the only animal whom she has endowed with the gift of speech. And whereas mere sound is but an indication of pleasure or pain, and is therefore found in other animals (for their nature attains to the perception of pleasure and pain and the intimation of them to one another, and no further), the power of speech is intended to set forth the expedient and inexpedient, and likewise the just and the unjust. And it is a characteristic of man that he alone has any sense of good and evil, of just and unjust, and the association of living beings who have this sense makes a family and a state. –Aristotle, Politics

With very rare exceptions, humans–all humans, in all parts of the world–live in groups. Tribes. Families. Cities. Nations. Our nearest primate relatives, chimps and bonobos, also live in groups. Primates are social, and their behavior can only be understood in the context of their groups.

Groups of humans are able to operate in ways that individual humans cannot, drawing on the collective memories, skills, and knowledge of their members to create effects much greater than what could be achieved by each person acting alone. For example, one lone hunter might be able to kill a deer–or if he is extremely skilled, hardworking, and lucky, a dozen deer–but ten hunters working together can drive an entire herd of deer over a cliff, killing hundreds or even thousands. (You may balk at the idea, but many traditional hunting societies were dependent on only a few major hunts of migrating animals to provide the majority of their food for the entire year–meaning that those few hunts had to involve very high numbers of kills or else the entire tribe would starve while waiting for the animals to return.)

Chimps have never, to my knowledge, driven megafauna to extinction–but humans have a habit of doing so wherever they go. Humans are great at what we do, even if we aren’t always great at extrapolating long-term trends.

But the beneficial effects of human cooperation don’t necessarily continue to increase as groups grow larger–China’s 1.3 billion people don’t appear to have better lives than Iceland’s 332,000 people. Indeed, there probably is some optimal size–depending on activity and available communications technology–beyond which the group struggles to coordinate effectively and begins to degenerate.

CBS advises us to make groups of 7:

As it turns out, seven is a great number for not only forming an effective fictional fighting force, but also for task groups that use spreadsheets instead of swords to do their work.

That’s according to the new book Decide & Deliver: 5 Steps to Breakthrough Performance in Your Organization (Harvard Business Press).

Once you’ve got 7 people in a group, each additional member reduces decision effectiveness by 10%, say the authors, Marcia W. Blenko, Michael C. Mankins, and Paul Rogers.

Unsurprisingly, a group of 17 or more rarely makes a decision other than when to take a lunch break.

Princeton blog reports:

The trope that the likelihood of an accurate group decision increases with the abundance of brains involved might not hold up when a collective faces a variety of factors — as often happens in life and nature. Instead, Princeton University researchers report that smaller groups actually tend to make more accurate decisions, while larger assemblies may become excessively focused on only certain pieces of information. …

collective decision-making has rarely been tested under complex, “realistic” circumstances where information comes from multiple sources, the Princeton researchers report in the journal Proceedings of the Royal Society B. In these scenarios, crowd wisdom peaks early then becomes less accurate as more individuals become involved, explained senior author Iain Couzin, a professor of ecology and evolutionary biology. …

The researchers found that the communal ability to pool both pieces of information into a correct, or accurate, decision was highest in a band of five to 20. After that, the accurate decision increasingly eluded the expanding group.

Couzin found that in small groups, people with specialized knowledge could effectively communicate that to the rest of the group, whereas in larger groups, they simply couldn’t convey their knowledge to enough people and group decision-making became dominated by the things everyone knew.

If you could travel back in time and propose the idea of democracy to the inhabitants of 13th century England, they’d respond with incredulity: how could peasants in far-flung corners of the kingdom find out who was running for office? Who would count the votes? How many months would it take to tally up the results, determine who won, and get the news back to the outlying provinces? If you have a printing press, news–and speeches–can quickly and accurately spread across large distances and to large numbers of people, but prior to the press, large-scale democracy simply wasn’t practical.

Likewise, the communism of 1917 probably couldn’t have been enacted in 1776, simply because society at that time didn’t have the technology yet to gather all of the necessary data on crop production, factory output, etc. (As it was, neither did Russia of 1917, but they were closer.)

Today, the amount of information we can gather and share on a daily basis is astounding. I have at my fingertips the world’s greatest collection of human knowledge, an overwhelming torrent of data.

All of our these information networks have linked society together into an increasingly efficient meta-brain–unfortunately, it’s not a very smart meta-brain. Like the participants in Couzin’s experiments, we are limited to what “everyone knows,” stymied in our efforts to impart more specialized knowledge. (I don’t know about you, but I find being shouted down by a legion of angry people who know less about a subject than I do one of the particularly annoying features of the internet.)

For example, there’s been a lot of debate lately about immigration, but how much do any of us really know about immigrants or immigrant communities? How much of this debate is informed by actual knowledge of the people involved, and how much is just people trying to extend vague moral principles to cover novel situations? I recently had a conversation with a progressive acquaintance who justified mass-immigration on the grounds that she has friendly conversations with the cabbies in her city. Heavens protect us–I hope to get along with people as friends and neighbors, not just when I am paying them!

One gets the impression in conversation with Progressives that they regard Christian Conservatives as a real threat, because that group that can throw its weight around in elections or generally enforce cultural norms that liberals don’t like, but are completely oblivious to the immigrants’ beliefs. Most of our immigrants hail from countries that are rather more conservative than the US and definitely more conservative than our liberals.

Any sufficiently intelligent democracy ought to be able to think critically about the political opinions of the new voters it is awarding citizenship to, but we struggle with this. My Progressive acquaintance seems think that we can import an immense, conservative, third-world underclass and it will stay servile indefinitely, not vote its own interests or have any effects on social norms. (Or its interests will be, coincidentally, hers.)

This is largely an information problem–most Americans are familiar with our particular brand of Christian conservatives, but are unfamiliar with Mexican or Islamic ones.

How many Americans have intimate, detailed knowledge of any Islamic society? Very few of us who are not Muslim ourselves speak Arabic, and few Muslim countries are major tourist destinations. Aside from the immigrants themselves, soldiers, oil company employees, and a handful of others have spent time in Islamic countries, but that’s about it–and no one is making any particular effort to listen to their opinions. (It’s a bit sobering to realize that I know more about Islamic culture than 90% of Americans and I still don’t really know anything.)

So instead of making immigration policy based on actual knowledge of the groups involved, people try to extend the moral rules–heuristics–they already have. So people who believe that “religious tolerance is good,” because this rule has generally been useful in preventing conflict between American religious groups, think this rule should include Muslim immigrants. People who believe, “I like being around Christians,” also want to apply their rule. (And some people believe, “Groups are more oppressive when they’re the majority, so I want to re-structure society so we don’t have a majority,” and use that rule to welcome new immigrants.)

And we are really bad at testing whether or not our rules are continuing to be useful in these new situations.

 

Ironically, as our networks have become more effective, our ability to incorporate new information may have actually gone down.

The difficulties large groups experience trying to coordinate and share information force them to become dominated by procedures–set rules of behavior and operation are necessary for large groups to operate. A group of three people can use ad-hoc consensus and rock-paper-scissors to make decisions; a nation of 320 million requires a complex body of laws and regulations.

But it’s getting late, so let’s continue this discussion in the next post.