Beyond VR: Computer assisted dreaming

I first played with VR in 1983/1984 while working in the missile industry. Back then we didn’t call it VR, we just called it simulation but it was actually more intensive than VR, just as proper flight simulators are. Our office was a pair of 10m wide domes onto which video could be projected, built decades earlier, in the 1950s I think. One dome had a normal floor, the other had a hydraulic platform that could simulate being on a ship. The subject would stand on whichever surface was appropriate and would see pretty much exactly what they would see in a real battlefield. The missile launcher used for simulation was identical to a real one and showed exactly the same image as a real one would. The real missile was not present of course but its weight was simulated and when the fire button was pressed, a 140dB bang was injected into the headset and weights and pulleys compensated for the 14kg of weight, suddenly vanishing from the shoulder. The experience was therefore pretty convincing and with the loud bang and suddenly changing weight, it was almost as hard to stand steady and keep the system on target as it would be in real life – only the presumed fear and knowledge of the reality of the situation was different.

Back then in 1983, as digital supercomputers had only just taken over from analog ones for simulation, it was already becoming obvious that this kind of computer simulation would one day allow ‘computer assisted dreaming’. (That’s one of the reasons I am irritated when Jaron Lanier is credited for inventing VR – highly realistic simulators and the VR ideas that sprung obviously from them had already been around for decades. At best, all he ‘invented’ was a catchy name for a lower cost, lower quality, less intense simulator. The real inventors were those who made the first generation simulators long before I was born and the basic idea of VR had already been very well established.)

‘Computer assisted dreaming’ may well be the next phase of VR. Today in conventional VR, people are immersed in a computer generated world produced by a computer program (usually) written by others. Via trial and feedback, programmers make their virtual worlds better. As AI and sensor technology continue rapid progress, this is very likely to change to make worlds instantly responsive to the user. By detecting user emotions, reactions, gestures and even thoughts and imagination, it won’t be long before AI can produce a world in real time that depends on those thoughts, imagination and emotions rather than putting them in a pre-designed virtual world. That world would depend largely on your own imagination, upskilled by external AI. You might start off imagining you’re on a beach, then AI might add to it by injecting all sorts of things it knows you might enjoy from previous experiences. As you respond to those, it picks up on the things you like or don’t like and the scene continues to adapt and evolve, to make it more or less pleasant or more or less exciting or more or less challenging etc., depending on your emotional state, external requirements and what it thinks you want from this experience. It would be very like being in a dream – computer assisted lucid dreaming, exactly what I wanted to make back in 1983 after playing in that simulator.

Most people enjoy occasional lucid dreams, where they realise they are dreaming and can then decide what happens next. Making VR do exactly that would be better than being trapped in someone else’s world. You could still start off with whatever virtual world you bought, a computer game or training suite perhaps, but it could adapt to you, your needs and desires to make it more compelling and generally better.

Even in shared experiences like social games, experiences could be personalised. Often all players need to see the same enemies in the same locations in the same ways to make it fair, but that doesn’t mean that the situation can’t adapt to the personalities of those playing. It might actually improve the social value if each time you play it looks different because your companions are different. You might tease a friend if every time you play with them, zombies or aliens always have to appear somehow, but that’s all part of being friends. Exploring virtual worlds with friends, where you both see things dependent on your friend’s personality would help bonding. It would be a bit like exploring their inner world. Today, you only explore the designer’s inner world.

This sort of thing would be a superb development and creativity tool. It could allow you to explore a concept you have in your head, automatically feeding in AI upskilling to amplify your own thoughts and ideas, showing you new paths to explore and helping you do so. The results would still be extremely personal to you, but you on a good day. You could accomplish more, have better visions, imagine more creative things, do more with whatever artistic talent you have. AI could even co-create synthetic personas, make virtual friends you can bond with, share innermost thoughts with, in total confidence (assuming the company you bought the tool from is trustworthy and isn’t spying on you or selling your details, so maybe best not to buy it from Facebook then).

And it would have tremendous therapeutic potential too. You could explore and indulge both enjoyable and troublesome aspects of your inner personality, to build on the good and alleviate or dispel the bad. You might become less troubled, less neurotic, more mentally healthy. You could build your emotional and creative skills. You could become happier and more fulfilled. Mental health improvement potential on its own makes this sort of thing worth developing.

Marketers would obviously try to seize control as they always do, and advertising is already adapting to VR and will continue into its next phases of development. Your own wants and desires might help guide the ‘dreaming’, but marketers will inevitably have some control over what else is injected, and will influence algorithms and AI in how it chooses how to respond to your input. You might be able to choose much of the experience, but others will still want and try to influence and manipulate you, to change your mindset and attitudes in their favour. That will not change until the advertising business model changes. You might be able to buy devices or applications that are entirely driven by you and you alone, but it is pretty certain that the bulk of products and services available will be at least partly financed by those who want to have some control of what you experience.

Nevertheless, computer-assisted dreaming could be a much more immersive and personal experience than VR, being more like an echo of your own mind and personality than external vision, more your own creation, less someone else’s. In fact, echo sounds a better term too. Echo reality, ER, or maybe personal reality, pereal, or mental echo, ME. Nah, maybe we need Lanier to invent a catchy name again, he is good at that. That 1983 idea could soon become reality.

 

The future of rubbish quality art

Exhibit A: Tracey Emin – anything at all from her portfolio will do.

Exhibit B: What I just knocked up in 5 minutes:

Exploration of the real-time gravitational interaction of some copper atoms

A recent work, I can Cu Now

As my obvious  artistic genius quickly became apparent to me, I had a huge flash of inspiration and produced this:

Investigating the fundamental essence of futurology and whether the process of looking into the future can be fully contained within a finite cultural bottle.

Trying to bottle the future

I have to confess that I didn’t make the beautiful bottle, but even Emin only has a little personal  input into some of the works she produces and it is surely obvious that my talent in arranging this so beautifully is vastly greater than that of the mere sculptor who produced the vase, or bottle, or whatever. Then, I produced my magnum opus, well so far, towards the end of my five minutes of exploration of the art world. I think you’ll agree I ought immediately to be assigned Professor of Unified Arts in the Royal Academy. Here it is, if I can see well enough to upload it through my tears of joy at having produced such insight.

Can we measure the artistic potential of a rose?

This work needs no further explanation. I rest my case.

The future of questions

The late Douglas Adams had many great ideas. One of the best was the computer Deep Thought, built to answer The question of ‘life, the universe and everything’ that took 6 million years to come up with the answer 42. It then had to design a far bigger machine to determine what the question actually was.

Finding the right question is often much harder than answering it. Much of observational comedy is based on asking the simplest questions that we just happen never to have thought of asking before.

A good industrial illustration is in network design. A long time ago I used to design computer communication protocols, actually a pretty easy job for junior engineers. While doing one design, I discovered a flaw in a switch manufacturer’s design that would allow data networks to be pushed into a gross overload situation and crashed repeatedly by a single phone call. I simply asked a question that hadn’t been asked before. My question was “can computer networks be made to resonate dangerously?” That’s the sort of question bridge designers have asked every time they’ve built a bridge since roman times, with the notable exception of the designers of London’s Millennium Bridge, who had to redesign their’s. All I did was apply a common question from one engineering discipline to another. I did that because I was trained as a systems engineer, not as a specialist. It only took a few seconds to answer in my head and a few hours to prove it via simulation, so it was a pretty simple question to answer (yes they can), but it had taken many years before anyone bothered to ask it.

More importantly, that question couldn’t have been asked much before the 20th century, because the basic knowledge or concept of a computer network wasn’t there yet. It isn’t easy to think of a question that doesn’t derive from existent culture (which includes the full extent of fiction of course). As new ideas are generated by asking and answering questions, so the culture gradually extends, and new questions become possible. But we don’t ask them all, only a few. Even with the culture and knowledge we already have at any point, it is possible to ask far more questions, and some of them will lead to very interesting answers and a few of those could change the world.

Last night I had a dream where I was after-dinner speaking to some wealthy entrepreneurs (that sort of thing is my day job). One of them challenged me that ideas were hard to come by and as proof of his point asked me why the wheel had never been reinvented (actually it is often reinvented, just like the bicycle – all decent engineers have reinvented the bicycle to some degree at some point, and if you haven’t yet, you probably will. You aren’t allowed to die until you have). Anyway, I invented the plasma caterpillar track there and then as my answer to show that ideas are ten a penny and that being an entrepreneur is about having the energy and determination to back them, not the idea itself. That’s why I stick with ideas, much less work. Dreams often violate causality, at least mine do, and one department of my brain obviously contrived that situation to air an idea from the R&D department, but in the dream it was still the question that caused the invention. Plasma caterpillar tracks are a dream-class invention. Once daylight appears, you can see that they need work, but in this case, I also can see real potential, so I might do that work, or you can beat me to it. If you do and you get rich, buy me a beer. Sorry, I’m rambling.

How do you ask the right question? How do you even know what area to ask the right question in? How do you discover what questions are possible to ask? Question space may be infinite, but we only have a map of a small area with only a few paths and features on it. Some tools are already known to work well and thousands of training staff use them every day in creativity courses.

One solution is to try to peel back and ask what it is you are really trying to solve. Maybe the question isn’t ‘what logo should we use?’ but ‘what image do we want to present?’, or is it ‘how can we appeal to those customers?’ or ‘how do we improve our sales?’ or ‘how do we get more profit?’ or ‘how can we best serve shareholders?’. Each layer generates different kinds of answers.

Another mechanism I use personally is to matrix solutions and industries, applying questions or solutions from one industry to another, or notionally combining random industries. A typical example: Take TV displays and ask why can’t makeup also change 50 times a second? If the answer isn’t obvious, look at how nature does displays, can some of those techniques be dragged into makeup? Yes, they can, and you could make smart makeup using similar micro-structures to those that butterflies and beetles use and use the self-organisation developing in materials science to arrange the particles automatically.

Dragging solutions and questions from one area to another often generates lots of ideas. Just list every industry sector you can think of (and nature), and all the techs or techniques or procedures they use and cross reference every box against every other. By the time you’ve filled in every box, it will be long overdue to start again because they’ll all have moved on.

But however effective they are, these mechanistic techniques only fill in some of the question space and some can be addressed at least partly by AI. There is still a vast area unexplored, even with existing knowledge. Following paths is fine, but you need to explore off-road too. Group-think and cultural immersion stand in the way of true creativity. You can’t avoid your mind being directed in particular directions that have been ingrained since birth, and some genetic.

That leads some people to the conclusion that you need young fresh minds rather than older ones, but it isn’t just age that determines creativity, it is susceptibility to authority too, essentially thinking how you’re told to think. Authority isn’t just parents and teachers, or government, but colleagues and friends, mainly your peer group. People often don’t see peers as authority but needing their approval is as much a force as any. I am frequently amused spotting young people on the tube that clearly think they are true individuals with no respect for authority. They stick out a mile because they wear the uniform that all the young people who are individuals and don’t respect authority wear. It’s almost compulsory. They are so locked in by the authority and cultural language of those they want to impress by being different that they all end up being the same. Many ‘creatives’ suffer the same problem, you can often spot them from a distance too, and it’s a fairly safe bet that their actual creativity is very bounded. The fact is that some people are mentally old before they leave school and some die of old age yet still young in mind and heart.

How do you solve that? Well, apart from being young, one aspect of being guided down channels via susceptibility to authority is understanding the rules. If you are too new in a field to know how it works, who everyone is, how the tools work or even most of the basic fundamental knowledge of the field, then you are in an excellent position to ask the right questions. Some of my best ideas have been when I have just started in a new area. I do work in every sector now so my mind is spread very thinly, and it’s always easy to generate new ideas when you aren’t prejudiced by in-depth knowledge. If I don’t know that something can’t work, that you tried it ages ago and it didn’t, so you put it away and forgot about it, then I might think of it, and the technology might well have moved on since then and it might work now, or in 10 years time when I know the tech will catch up. I forget stuff very quickly too and although that can be a real nuisance it also minimizes prejudices so can aid this ‘creativity via naivety’.

So you could make sure that staff get involved in other people’s projects regularly, often with those in different parts of the company. Make sure they go on occasional workshops with others to ensure cross-fertilization. Make sure you have coffee areas and coffee times that make people mix and chat. The coffee break isn’t time wasted. It won’t generate new products or ideas every day but it will sometimes.

Cultivating a questioning culture is good too. Just asking obvious questions as often as you can is good. Why is that there? How does it work? What if we changed it? What if the factory burned down tomorrow, how would we rebuild it? Why the hell am I filling in this form?

Yet another one is to give people ‘permission’ to think outside the box. Many people have to follow procedures in their jobs for very good reasons, so they don’t always naturally challenge the status quo, and many even pursue careers that tend to be structured and ordered. There is nothing wrong with that, each to their own, but sometimes people in any area might need to generate some new ideas. A technique I use is to present some really far future and especially seemingly wacky ones to them before making them do their workshop activity. Having listened to some moron talking probable crap and getting away with it gives them permission to generate some wacky ideas too, and some invariably turn out to be good ones.

These techniques can improve everyday creativity but they still can’t generate enough truly out of the box questions to fill in the map.

I think what we need is the random question generator. There are a few random question generators out there now. Some ask mathematical questions to give kids practice before exams. Some just ask random pre-written questions from a database. They aren’t the sort we need though. We won’t be catapulted into a new era of enlightenment by being asked the answer to 73+68 or ones that were already on a list. Maybe I should have explored more pages on google, but most seemed to bark up the wrong tree. The better approach might be to copy random management jargon generators. Tech jargon ones exist too. Some are excellent fun. They are the sort we need. They combine various words from long categorized lists in grammatically plausible sequences to come out with plausible sounding terms. I am pretty sure that’s how they write MBA courses.

We can extend that approach to use a full vocabulary. If a question generator asks random questions using standard grammatical rules and a basic dictionary attack, (a first stage filtration process) most of the questions filtering through would still not make sense (e.g, why are all moons square?). But now we have AI engines that can parse sentences and filter out nonsensical ones or ones that simply contradict known facts and the web is getting a lot better at being machine-comprehensible. Careful though, some of those facts might not be any more.

After this AI filtration stage, we’d have a lot of questions that do make sense. A next stage filtration could discover which ones have already been asked and which of those have also been answered, and which of those answers have been accepted as valid. These stages will reveal some questions still awaiting proper responses, or where responses are dubious or debatable. Some will be about trivia, but some will be in areas that might seem to be commercially or socially valuable.

Some of the potentially valuable ones would be suited to machines to answer too. So they could start using spare cycles on machines to increase knowledge that way. Companies already do this internally with their big data programs for their own purposes, but it could work just as well as a global background task for humanity as a whole, with the whole of the net as one of its data sources. Machines could process data and identify potential new markets or products or identify social needs, and even suggest how they could be addressed and which companies might be able to do them. This could increase employment and GDP and solve some social issues that people weren’t even aware of.

Many would not be suited to AI and humans could search them for inspiration. Maybe we could employ people in developing countries as part of aid programs. That provides income and utilizes the lack of prejudice that comes with unfamiliarity with our own culture. Another approach is to make the growing question database online and people would make apps that deliver randomly selected questions to you to inspire you when you’re bored. There would be enough questions to make sure you are usually the first to have ever seen it. When you do, you could rate it as meaningless, don’t care, interesting, or wow that’s a really good question, maybe some other boxes. Obviously you could also produce answers and link to them too. Lower markings would decrease their reappearance probability, whereas really interesting ones would be seen by lots of people and some would be motivated to great answers.

Would it work? How could this be improved? What techniques might lead us to the right questions? Well, I just asked those ones and this blog is my first attempt at an answer. Feel free to add yours.

 

 

The future of creativity

Another future of… blog.

I can play simple tunes on a guitar or keyboard. I compose music, mostly just bashing out some random sequences till a decent one happens. Although I can’t offer any Mozart-level creations just yet, doing that makes me happy. Electronic keyboards raise an interesting point for creativity. All I am actually doing is pressing keys, I don’t make sounds in the same way as when I pick at guitar strings. A few chips monitor the keys, noting which ones I hit and how fast, then producing and sending appropriate signals to the speakers.

The point is that I still think of it as my music, even though all I am doing is telling a microprocessor what to do on my behalf. One day, I will be able to hum a few notes or tap a rhythm with my fingers to give the computer some idea of a theme, and it will produce beautiful works based on my idea. It will still be my music, even when 99.9% of the ‘creativity’ is done by an AI. We will still think of the machines and software just as tools, and we will still think of the music as ours.

The other arts will be similarly affected. Computers will help us build on the merest hint of human creativity, enhancing our work and enabling us to do much greater things than we could achieve by our raw ability alone. I can’t paint or draw for toffee, but I do have imagination. One day I will be able to produce good paintings, design and make my own furniture, design and make my own clothes. I could start with a few downloads in the right ballpark. The computer will help me to build on those and produce new ones along divergent lines. I will be able to guide it with verbal instructions. ‘A few more trees on the hill, and a cedar in the foreground just here, a bit bigger, and move it to the left a bit’. Why buy a mass produced design when you can have a completely personal design?

These advances are unlikely to make a big dent in conventional art sales. Professional artists will always retain an edge, maybe even by producing the best seeds for computer creativity. Instead, computer assisted and computer enhanced art will make our lives more artistically enriched, and ourselves more fulfilled as a result. We will be able to express our own personalities more effectively in our everyday environment, instead of just decorating it with a few expressions of someone else’s.

However, one factor that seems to be overrated is originality. Anyone can immediately come up with many original ideas in seconds. Stick a safety pin in an orange and tie a red string through the loop. There, can I have my Turner prize now? There is an infinitely large field to pick from and only a small number have ever been realized, so coming up with something from the infinite set that still haven’t been thought of is easy and therefore of little intrinsic value. Ideas are ten a penny. It is only when it is combined with judgement or skill in making it real that it becomes valuable. Here again, computers will be able to assist. Analyzing a great many existing pictures or works or art should give some clues as to what most people like and dislike. IBM’s new neural chip is the sort of development that will accelerate this trend enormously. Machines will learn how to decide whether a picture is likely to be attractive to people or not. It should be possible for a computer to automatically create new pictures in a particular style or taste by either recombining appropriate ideas, or just randomly mixing any ideas together and then filtering the new pictures according to ‘taste’.

Augmented reality and other branches of cyberspace offer greater flexibility. Virtual objects and environments do not have to conform to laws of physics, so more elaborate and artistic structures are possible. Adding in 3D printing extends virtual graphics into the physical domain, but physics will only apply to the physical bits, and with future display technology, you might not easily be able to see where the physical stops and the virtual begins.

So, with machine assistance, human creativity will no longer be as limited by personal skill and talent. Anyone with a spark of creativity will be able to achieve great works, thanks to machine assistance. So long as you aren’t competitive about it, (someone else will always be able to do it better than you) your world will feel nicer, more friendly and personal, you’ll feel more in control, empowered, and your quality of life will improve. Instead of just making do with what you can buy, you’ll be able to decide what your world looks, sounds, feels, tastes and smells like, and design personality into anything you want too.