SIGGRAPH 2017 & The Birthing Of Interactive External Imagination

uf1.jpg

I attended SIGGRAPH 2017 in LA and was struck by how mature the technologies are. It felt like a capstone moment, when so much of what I had dreamt of over the recent years of my life started to look tantalizingly possible. This post is a collection links from myself and others, please click around and explore the future As We May Think - There has long existed a dream of what an augmenting medium could be about, and a profound new chapter feels close at hand if we choose to build it...

 

At SIGGRAPH I was delighted to see Ken Perlin in the interesting demo #MeetMike. I recorded his talk on my phone, so it's not the best quality, but ironically the main thing I want to share is not the visuals, although they are incredibly impressive, but rather Perlin's message: we need to work with humanity as we build social computer experiences. Perlin would rather see a cartoon with great expressivity than a photo-realistic avatar with rigid muscles. Perlin is a pioneering researcher and thinker currently working on HoloJam and Chalktalk.

That said, the #MeetMike demo was extremely impressive in showing a glimmer of what a photo-realistic avatar could be like. The possibility has been lurking, but Mike Seymour and his team brought it to life and gave it an application, and they did an amazing job!

In my touring around the booths I encountered Imverse in the back of the room with a cool first take on making the creation of Mixed Reality really easy. Talking with Javier the CEO was really fun, his tool already has some of the elements of Perlin's HoloJam, but could use a lot more interactivity in creating real-time advanced animations, as Javier agreed and is working on. Unlike HoloJom though, Imverse is a commercially-ready application that works on more standard VR gear, without the need of a fully tracked room, making it ripe for experimentation by us mere mortals without lab-space!

Another highlight: I was really blown away to wear ODG's Mixed Reality glasses because they were so light! They run Android and will cost a bit over $1,000. Their form factor made me think, "yup, this one will actually get used in the field like it should." They're a self-contained computer, which limits graphics but mobility is my favorite element of what makes Mixed and Augmented Reality so special - computing out in the real world, with our hands free! 

With Apple's AR kit and small glasses like this we are getting ready for Mixed (or Augmented) reality sooner than I thought! 

But before any of this we also need to think hard about computation and what we're trying to do - what is the Center Of "Why"?

And I'm intent to explore where we've already been to make sure we're always breaking new ground and/or refining the garden we're already in. A recent ACM panel gives a great crash history through the present with the ethics.

Most of the improvements we need are in conceptual understanding of computation - SIGGRAPH showed me that the raw technology is ready for anything! So it's up to us to imagine a more robust version of just what it means to "use a computer". We need smarter interfaces and a more expressive form of programming, and we can begin experimenting today even before all the gadgets are ready.

If I talked with you at SIGGRAPH I probably told you I'm tired of clicking little buttons and using inert spaces which I must instruct EVERY element of the action I wish to perform. I mean it. I want to paint with math and math with paint. I believe that computing is a shared substrate for all of human endeavor, and I know it needs to let us be fully human within it. 

In graduate school I started exploring the research that is currently present for making a much more expressive interface for computing based on the the mighty pen, hypercharged by the affordances of a digital substrate: As We May Sketch

I am fascinated by the dream that we might eventually Converse with Computers in new forms of conversation, even if it is just to make graphics in a more user-friendly way.

There are already methods for approaching "artificial" intelligence in computers that will let us switch from thinking of machine learning as a data problem to having machine learning become an expressive experience allowing us to bridge rigid logics with blurry imaginations.

I am struck to my core that computing can and ought to move beyond us as explicit instructors and toward us as cooperative participants with an increasingly dynamic and intelligent substrate ready to hold our creations and let our minds soar. We in our physical bodies and full humanity are at the center of the vision of the future.

Still beyond our wonderful maturing pen-based tablets with 2D motion screens, we are entering into a time when graphics can move outside of rectangles - a time to explore more Humane Representations of Thought. We do need to keep humanity in mind, both our soaring potentials and our consistent ambitions and nagging flaws, books like Rainbow's End help situate the affordances against stories.

Back in 2015 I wrote a collection of three scenes from a School Astride the Metaverse, an attempt to envision a school that bridges material and virtual reality into a kind of whole:

One about Expression Capture Day and some musing from a teacher.

One about a typical Final Assignment

And one in which a kid educates the system...

This vision is not complete, it is playground to think about possible styles of school in the future. The first two are grounded, and the last one is more fanciful. There's an entire arena to go into with mixed reality and even more simulation space games - I'm working on it, but for now want to re-surface these since they are starting to look almost conservative in light of this years' tech I witnessed!

This is not meant as a description of the future, but rather a constellation of ideas about how the future of education might look in a world where technology keeps getting better and cheaper, yet overall amounts of money spent on education remain relatively constant. This is not a utopia, this is meant to explore the feeling of what certain compromises and perspectives might manifest as. You certainly do not have to agree with or love or even like this vision, but I do hope that you want to discuss it and the ideas within since this is a world not too far away. Mere decades really for some parts of it; other parts are less clear...

It's important to think and converse about holistic visions for the future of education, not just individual technology artifacts in the classroom and moderate systems deployments. This is my attempt to help add a little flavor to the conversation.

Let's make a future where learning is an adventure and allow new generations to soar past our wildest dreams.