Part 2 - The Peril
Also published on the Atlantic Council's FutureSource Blog
In my previous blog, I outlined what living in an Answer Engine-powered Internet might look like. There is much to be excited for as we move toward the future where the Internet is woven into every facet of life. Think how far we’ve come already and how quickly it happened. Think about your relationship with your mobile device. This merging of Internet and life is inevitable; in fact it is already almost a fait accompli. There is also an array of dangers to be concerned about. The most worrisome negatives must be understood and addressed if we are to have any hope of mitigating them.
In the future Internet, privacy as we have known it is gone. The global mesh of cameras and sensors combined with a powerful Answer Engine AI will ensure that the most information that can be known will be known, although a secrecy arms race will likely still be boiling under the surface. Every human and every artifact he or she owns can be tracked and accounted for in real time.
Yet I do not think we are headed for an uncontrollable transparent society. Instead I foresee social contracts of the future that are immensely more nuanced and codified than they are today, especially at the individual level.
Lovers might grant each other full privacy permissions, while new acquaintances will be insulated from each other with myriad permissions doors. Every piece of your networked experience will be sharable, but you choose who is allowed to share it, for how long, and even at what price. Access to cameras, health data, location, even inferred mood will be available to share with others. Everything you do and say, but also increasingly, everything you think or even feel will be accessible to anyone with the key to these doors, or a good crowbar. Perhaps governments won’t even need a key; they may weasel their way in at a design level, requiring guaranteed access as the cost of selling computing products within their borders. Every user’s most intimate data will potentially be accessible to every other user - but the individual will control access to much of this information.
As well, the Answer Engine paradigm will mostly forgo providing huge swaths of broad information, instead providing small, calculated snippets of densely relevant information. As I forecast in my previous post, such a capability will be a boon for inquiry. However, it also presents a dark problem. Even modern Google responds to queries by placing advertisement links at the top and links prioritized by more dispassionate calculated relevance below. While the ad link might be a fine answer, it is also a terrible answer; it got the top spot not from pure relevance but by buying its way there.
The Answer Engine world could see people ferried about their daily lives not by the best answers, but by the answers that paid the most to be delivered. As we continue to offload internal memory to our networked world so too will we offload some amount of inference and even decision-making. Existing in and relying on an Answer Engine Internet opens your mind up to a powerful new form of direct manipulation: the answers to your questions delivered by the top bidder, the solutions to your life’s daily problems offered by the entity with the biggest wallet. Who is in control then?
With huge swaths of society being literally controlled by advertisers, the potential for dystopia is alarming. More worrying still, there are efforts currently underway by Facebook and others to gather data and synthesize inference about people so nuanced that it begins to form a picture of who they actually are, not just what they like. Advertisements taking the form of help, of solutions. Ads tricking people into thinking they are the best answer by being a great answer. Not actually the best answer mind you, merely a great answer, one that makes you spend money and feel happy.
Consider this hypothetical: The Answer Engine knows you’re feeling sad based off body sensor data, the wistful social posts you’ve been making, and the fact that you just got a digi-paper from your boss telling you that you’re fired. It also knows you like sweets. All on its own and at a perfectly calculated moment, it proposes that you stop worrying about your future and buy a treat for yourself from Cand-Corp instead. Wow, thanks Answer Engine, you always know how to cheer me up! Given this level of context and insight, ads might move from annoyance to friend. But of course, this friend only wants one thing from you, and it will use every trick imaginable to get it, all while giving you vapid nothings dressed up as wisdom in return.
Hopefully, the future will be characterized by seeking balance, by being aware of what one might gain by accepting a technology and what one might lose. Understanding the forces at play will be instrumental to retaining power and agency. To accept blindly is to be powerless. The majority of Earth’s population could end up as mere cyber-serfs who have no knowledge of how the technologies that define their existence function and thus are basically at the mercy of whatever entities they have trusted with their data - and their lives. They could become "ignorant masses" trapped inside the information pool, being ferried about by a handful of mega-corps and savvy individuals, blindly following while given a shallow illusion of control. Yet to swim completely upstream and escape the pool will become increasingly difficult. Some trends are unavoidable, at least not without forgoing integration with the rest of society. There will simply be fewer and fewer places left to hide. Joining this potentially dystopian future could become less of a choice and more of a silent oppression; either join or be left behind.
Billions more humans are still waiting to be connected to the Internet. These next decades will see the completion of the transition from the Internet as a tool, to the Internet as part of existence itself. This transition will be rocky, it will change who we are, and it will open up cans of worms previously unimaginable. Many already use the Internet for the most nefarious of ends, and they have abilities at their disposal that were once the sole purview of only the world’s most powerful governments.
Still, regardless of risks and perils, this is what we humans do. We’ve been absorbing our technologies into our lives since before recorded history. Now we cook food, we wear clothes, we live in structures, we use electricity, and we are connected to the Internet. This network of networks is a basic utility now, something that all are entitled to. It is driving the biggest change that humanity has ever undergone, a growth in knowledge and power of truly exponential proportions. Yet knowledge and power are just tools to forward ambition. Good and evil both will be empowered. The opportunities for failure are high, but then so too are the opportunities for success.
Personally, I am optimistic. Being aware of potential future pitfalls helps us avoid them, or at least know where to look and to try to mitigate their impact. Now is not the time for pessimism in the face of massive change. Rather, it is the time to find the balance, to assert control, to be proactive in shaping our future. I believe optimism and pessimism alike create self-fulfilling prophecies, and I prefer to pick the better prophecy.