Before beginning work on SoundSelf, I spent most of my time making sound effects for independent games like Antichamber, The Stanley Parable and Capsule. In this video, I go through some of the unique sound design challenges SoundSelf presents to me, and how I’ve approached those challenges.
Special thanks to the awesome Tina Rodriguez for her musical generosity.
Here are some more ideas I’ve been experimenting with. Aside from “fields”, I think all of these are pretty beautifully successful, and I look forward to exploring them more fully in game.
This is based on a wind-powered sculpture a friend of mine posted on facebook.
This is the most direct exploration of mandala forms I’ve experimented with so far. It began as an attempt to create gravitational orbits, but I abandoned that in favor of greater symmetry, which I think works well for SoundSelf in general.
This one’s really meh. I was experimenting with interference and aliasing, but it just… meh. That was a rabbit hole that lasted most of the day… but once I abandoned it…
This came out much more awesomely. It was inspired by this weird thing.
BTW, I totally love it when people send me cool trippy gifs and artstuff. It really really inspires me. So if there’s a thing you’d like to share, please do so in the Greenlight comments! A soundselfy version of it may end up in the game.
SoundSelf currently has one audio-visual form (or “story”, as we call it) – a tunnel of light with an orb in the center. Exploring this story has given us a lot of leverage to fine-tune the interactivity. But we want SoundSelf to contain many different stories and seamlessly move between them as you play.
I’ve spent the last week jamming on new ideas. Most of them are pretty uninteresting, but some have a lot of potential! Below are some of the first sketches that could be expanded into future stories.
I’ll be live-streaming the rest of the jam this week! You can tune into my twitch.tv channel and click “follow” to be alerted when I start the broadcast. Don’t be shy! I want to hear your voice!
This form comes from creating an array of simple shapes, and have them constantly rush towards the player.
This is a radically different sort of visual form, and it’s super elegant. By determining the particle’s shape based on its location in x/y/z space, the center being zero (no variation), a blinking circle appears in the center. Totally accidental, but awesome emergent beauty like this is what makes these systems so powerful (more on that from Evan in a future blog post). This form could be expanded to feel like star systems, oceanic particles, or geometric forms like grids and spirals.
Normally, when I use the harmonograph the resolution (how many lines make up the curve) is like a ceiling limiting the complexity of the shape. Here, I’ve tried sharply reducing that ceiling, so changes in resolution itself have a dramatic impact on the shape. I don’t necessarily see myself building a story around this, but knowing that adjusting resolution is itself an aesthetic tool is valuable.
Let’s work together.
Returning from the weeks of festival hopping, we’re becoming increasingly aware of the therapeutic potential of SoundSelf. I’ve had the great fortune of meeting people with stories of either themselves or close relations struggling with any number of neurological challenges, from autism to schizophrenia. They’ve told me they believe SoundSelf could be a tool for them. I would like to be able to help, and I would also love to use your knowledge to make the experience more powerful.
While I have some reach in this quirky corner of the game world, I don’t know how to navigate the web of conferences and blogs that medical and mindfulness practitioners use. Additionally, researching the means for applying SoundSelf therapeutically, and then executing on that knowledge, have an uncertain time cost that we must balance against development.
If you believe you can use SoundSelf to help yourself or a friend, or if you think it would be valuable to your field of expertise, please let us know. I am eager to share and learn.
This week’s blog post is brought to you by Food House and GameCity.
I’ve just returned from the wonderful GameCity festival in Nottingham. Part of my contribution to the festival was this short talk on hacking perception, hypnosis, and illusions. If you’ve been following SoundSelf for a while, we go into some detail in what makes the experience get under your skin. For those of you who haven’t been following SoundSelf, this video is a lovely and brief introduction to the project.
(25 minutes, plus conversation at end)
PART 1 – HISTORY
PART 2 – ILLUSIONS
PART 3 – THE PERCEPTION OF SELF
PART 4 – HOW WE HACK IT
Since Burning Man this year, most of my SoundSelf time has been spent showing SoundSelf, describing SoundSelf, and developing new ideas. We’ve been preparing our Kickstarter rewards, showing the game at Indiecade, and in the coming two weeks we’ll be at GameCity (Nottingham),Gamercamp (Toronto) and MineCon (Orlando).
But while software development has slowed down, research has not. Here are a few of things that have excited and inspired me in the last few weeks:
I discovered this album through a Radiolab short devoted to it. I absolutely recommend checking it out.
Trance states are accessed through repetition. This is why many meditation techniques include a mantra, it’s why trance music (and music by definition) is based on repetition repetition, it’s why SoundSelf uses chanting (repeated voice) as it’s controller, and it’s why you see delusional people repeating the same words again and again.
Dawn of Midi’s album is as much a tribute to repetition as Ravel’s Bolero. But that repetitive structure is reduced to an elemental focus – the beat.
As a student of trance, of particular interest to me is the way the different beats work together. In stark contrast with shamanic drumming, there’s no global tempo in Dysnomia. Often three or four percussive patterns will be in play at the same time, each instrument following its own distinct beat. I found my mind shifting between the different instruments: as I rested my mind on a new rhythm, a formerly dominant rhythm would phase in and out of synchronization and syncopation.
Listening to this album has given me a new frame of reference for thinking about rhythm (and repetition at all, for that matter) as a tool for trance induction in SoundSelf.
The LACMA has a retrospective right now for light artist James Turrell. If you’re in LA… DO NOT MISS IT!
Most of Turrell’s pieces occupy an entire room, as they are designed to envelop participants in unique and sometimes illusory experiences of light.
The pieces are difficult to describe because they are not symbolic in nature. They are unique experiences of light and perception, as indescribably experiential as floating in a sense deprivation chamber is.
Turrell’s current project is (and has been for the past twenty-something years) converting a volcanic crater into a sort of modern-day Stone Henge: temples paying tribute to the sun, the sky, and human perception at dance with the cosmos.
The Singularity Project is artist David Tamargo’s entry in the Indiecade / Oculus VR Jam. It takes abstract fine art and explodes it into kinetic 3D. If you’re into SoundSelf, you’ll be into this.
I really love it when art makes me feel small and puts my life into geological scale. This is a to-scale puppet of a prehistoric ichthyosaur that once lived in the now dried lake bed that plays host to the Burning Man festival. And it’s pregnant.
Daniel Kahneman puts his long career of brain research into this deep but accessible model that describes the brain as simultaneously working in two radically different ways: fast and slow. It’s a wonderful read for anybody interested in biases and learning.
A short book with a deep and narrow focus: Trance. How it happens, the flavors it comes in, how to control it, and how it is often used irresponsibly.
Evan Balster here. I’m SoundSelf’s back-end engineer and co-designer. Helping Robin to understand new mathematical concepts that play into the design and development of the program is a big part of my job, and today I’ll be doing some of that explanation for you. Consider this the first installment of…
What you see below is the beginning of a new visual generative system for SoundSelf. Click it to visit our interactive ShaderToy page.
This is the default formulation of the shader, which can be manipulated with the mouse.
It’s loosely based on a concept called Cymatics, and computed as a sum of sines much like our existing harmonograph system. Instead of geometry, however, the sum computes an intensity value per pixel which is represented as color.
Move the ring of control nodes near the center for more regular, intense shapes.
The sine waves being summed in this demonstration are functions of distance from a point. Like ripples in a pond, they spread out and weaken. As more are added in regular or irregular arrangements we can get complex shapes as seen above. The default arrangement is a ring of control points arranged using the golden ratio.
We’re also experimenting with other wave functions which may offer more possibilities, as seen here:
This uses a combination of three circular ripples, a horizontal “bar” ripple and a vertical “bar” ripple. Symmetric patterns can often resemble faces.
The shader above was produced in a short time yesterday after a long discussion about what emergent system of visuals we should explore after the harmonograph. Emergence is a concept that is fundamental to the creation of SoundSelf; it is the phenomenon of complex behaviors or manifestations arising from simple rules that interact. The simpler the rules, and the more complex the manifestations, the deeper the emergence runs.
Because the harmonograph is deeply emergent, we very frequently see things in SoundSelf that we’ve never seen before. Interesting, sometimes beautiful forms that were not intentfully designed into the system are a common occurrence. Every oscillation has the power to transform the entire harmonograph, and with many tens of these at play the range of possibilities is immense.
Cymatics is one of the systems we discussed. In essence, if sand or dust is placed on a plate and that plate is subjected to harmonic vibrations, the plate will form standing waves as a function of the vibration and ts overlapping reflections. The sand will pile into intriguing patterns of points and lines which represent areas of stability, which differ based on the plate’s shape and size and the vibration’s source and frequency.
For a very simple home demonstration, fill a wine glass with water, near to the brim, dip your finger in the water and slide it across the lip to make the glass “sing”. A nearly full glass produces a deep sound and standing waves can be seen in the water, rotating in time with the vibration source (your finger).
This model, unlike the others, simulates the reflection patterns seen in square-plate cymatics. However, it’s expensive.
My initial evaluation of this system was discouraging — the actual forces and behaviors at work are prohibitively complex. However, the underlying concepts of waves and reflections, cancellation and stability are somewhat simpler. Distilling this process to a more representational method without concern for physical realism gives rise to a number of possibilities for simple, emergent systems. It would be accurate to say these are “inspired by cymatics” more than anything.
The experiments above are just scratching the surface of what can be done here. We’ll see what else we can find in this peculiar world of ripples. And of course, we’ll make sure there’s far, far more to be found in SoundSelf through the power of emergence.
Last month we brought a unique installation version of SoundSelf to [url=burningman.org]Burning Man[/url]. Making this unusual architectural iteration of our video game took a lot of work, a lot of sacrifice, and was only possible because our generous Kickstarter backers pushed us $7,000 past our goal.
Check out the rest of the loving messages visitors left in our Guest Book on our Facebook Page!
Photos by Dimitri Kouri and Michael Nusbaum.
For the first time since our Kickstarter, you can now pre-order or buy alpha access to SoundSelf!
Gone is the Windows-only prototype of yore! We’ve still got a long way to go, but this is the beginning of something special that’s already exceeding my expectations. Here’s what’s new from the KickStarter prototype:
- COMPLETELY overhauled music, COMPLETELY overhauled imagery.
- Much more intimate reaction to your voice. It feels more alive now than it ever has before!
- Oculus Rift support!
- More lightweight – won’t necessarily melt your machine – not literally anyway!
- Mac and Linux support!
We’re also migrating our forums to Steam! So come say hi on our Greenlight community page.