The tension between ‘what is’ and ‘what can be’ is omnipresent in technological design. The ‘what can be’ side of the tension further striates into what ‘ought be’ and ‘what can afford to be’ in an industrial economic setting. To me, nowhere is the tension between ‘what is’ and ‘what can be’ more apparent than with digital computers. These devices are substrates for logical operations, and as increasingly diverse communities of people have integrated them into their practices we see a flowering of implementations in software. Yet the initial boundary conditions of the history of computing powerfully shape what it is – where computing has been is the ground for us as we stretch to search for what it can be.
“The devices and systems of technology are not natural phenomena but the products of human design, that is, they are the result of matching available means to desired ends at acceptable cost. The available means ultimately do rest on natural laws, which define the possibilities and limits of the technology. But desired ends and acceptable costs are matters of society.” (Mahoney, 122)
So far ‘desired ends’ of the computational society have been seeded with industrial concerns and perspectives.
“the computer industry was, more than anything else, a continuation of the pre-1945 office equipment industry and in particular of the punched card machine industry.” (Mahoney, 126; quoting Haigh) “But making it universal, or general purpose, also made it indeterminate. Capable of calculating any logical function, it could become anything but was in itself nothing (well, as designed, it could always do arithmetic).” (Mahoney, 123)
Thus, in the 1970s, humanist artists began wading into computation, and we have witnessed an explosion of ‘high level’ creativity as to what the metamedium of ‘computation’ can actually do for us as meaning-makers. Ideas flourished that saw the computer as not just a machine for counting, but a substrate for human imagination. Yet the histories of computing set the devices we compute with on a path that has shaped its form: a device with baked-in logics that we recombine. The histories of computing feature engineering, science and data analysis as the kernel of the computer’s unfolding into the wider sociotechnical ecosystem. Art was tacked on later as an affordance of having enough 1/0s to spare. Computer programs are precise manipulations of the state of an electro-atomic system we call a computer. Yet human language too manipulates other electro-atomic systems (aka, other humans) in a much more blurry and imprecise way - yet this blurriness leaves room for emergence, and this I think is the key to the future direction of computing itself.
I am struck more and more each day by the 20th century origins of computing, and harden my resolve to lean more and more into what the 21st century of computing looks like. The future will see the “front” and “back” of computation merge into a holistic loop where generative logics allow computers to learn as they are used. The loops in our minds will be further augmented by loops through machines that begin to not just manipulate saved libraries, but increasingly generate new forms. We are, I think, at a profound crossroads in the path: will computing be continually defined by linear “processing”, or can we move it toward continuous relational inference? I think we must move to the latter, for the affordances of the future will enable and demand new human-scale ways to program computers. We are in the midst of a latent programming revolution.
This thinking has been culminating for me with the input of this class and my continued experience with the Microsoft Surface. The Surface device that I am typing this on is perhaps the perfect symbol for the crossroads that personal computing is currently in. The Surface has two distinct interface modes: the touchscreen/pen digitizer, and the keyboard. The mouse is unified with the digitizer pen decently well, but the keyboard remains a realm unto itself.
I am finding it increasingly jarring to coexist in free-flowing writing inside of digital inking applications and interfacing with programming.
To this day when writing to a computer at the level of its logical comprehension we are forced to bring our hands together and cramp over an un-changing keyboard. We input 1/0 commands into the machine through keys that correspond to symbols, which in sequence will (when interpreted) illicit the electrical state of the computer to evolve step by step as fast as the system clock allows.
The more I use a pen on a grid, the more I believe that there is potentially another way to program.
The work of von Neumann and others who pioneered the study of cellular automata has shown me that computing does not have to be about direct control using predefined symbol sets, but rather can be about boundary conditions and evolution.
I wonder if we cannot use digitizer grids and pens to allow human operators to sketch with computers. Already much of the power of the computer comes to us via adding abstraction. To edit a photo with machine code directly would be impossibly tedious, but thanks to many layers of abstraction I can use a tool like photoshop to move around thousands of pixels and billions of transistors in large strokes.
Programming languages have been path dependent upon 20th century paradigms. To me, programming a digital computer feels like playing with a near-infinite movable type: there are libraries of modules that I arrange in patterns to produce sequences which instruct the machine and can even mean something to a person.
Yet I wonder, is that the only way to program computers? Must we only use rigid pre-delineated symbols?
I think we can begin to write higher level programming environments that allow us to write to our computers, not type, but actually write.
I discovered a groundbreaking paper recently which shows that a unification between the way humans reason and the way computers process might be increasingly possible and fruitful.
Researchers Lake, Salakutdinov and Tenenbaum instantiated a “machine learning” concept by creating a “Bayesian program learning (BPL) framework, capable of learning a large class of visual concepts from just a single example and generalizing in ways that are mostly indistinguishable from people.” Using digital inking they developed a technique to parse drawn symbols via vector and temporal relational information and allow the computer to generate further symbols from these inputs.