Semester 1 is over, meaning no more sidetrack assignments. I have a year from now to produce a thesis paper.
In review, semester 1 was a very helpful exercise and a lifestyle shift in a way I produce work and talk about things. Academia has a different mode of operating and going through that space changed a lot of things for me.
The best tangible thing that came out of this is of course the interactive piece of work that I’ve produced for Pigsty. I was able to take it a bit further for another assignment, introducing motion control. Using kinect, I was interrogating the play space of the application from the point of view of a participant, observer, the software itself and misalignment between their literal and metaphorical vantage points. I’m hoping to put a few more features into the app, like screenshot saving, some user options and make kinect control available for public download, but perhaps after some of the exhibitions I’m pitching this work for.
WASD, LMB to clear visuals. Gamepad compatible.
Best played with headphones in the dark.
Unity is drastically changing it’s audio tools in the 5th release.
New compression format, lower memory foot print;
resource handling improvements;
audio metadata access;
performance profiling Audio mixer “first offering” (more big features to go?)(interactive audio tools next)
mixing & mastering.
DSP effects anywhere in the chain
ADSR etc in editor
runtime editing (mix while the game is playing)
modeled on existing DAWs (eg. Ableton, Logic etc)
ability to create & chain together multiple mixers
routing of sound groups as hierarchies (groups mixed down together)
3D positioning, doppler etc (spacial sound concepts) applied to source before the mixer.
effects are stacked sequentially (like in a daw)
native plugin effects – custom DSP, custom UI
Snapshots – save states of your edits (awesome!)
transition between states at runtime (even awesomer!)
expose individual effect parameters – can be script drivensends / returns
ducking (side chain compression) – any audio group
A very exciting update for audio in Unity. A very slack presentation, however as an overview, makes me wonder whether I should just hold out till 5.0 or if we will end up using both Unity native and Fmod sound at the same time.
Picked up a kinect on trademe for nzd$30 because why not. Would be cool to set up a motion controlled sound experiment.
Was looking through MS SDK toolkit and their demos to see how it works. You can’t be too close or too far from the camera. Luckily my room has just enough space to capture whole skeleton.
Haven’t been able to get it to work with Unity yet, but there are plenty of resources online. For tonight I’m happy sending the green stickman back to infinity.
If you were able to discover this effect in the previous builds, if you just touch a base zone and go out of it without completely coming through, it triggers off, leaving you in the “zero zone” where the last played bass goes through a low pass filter. I really love the floating feeling of the LP filtering based on velocity, so I made a scene that isolates that effect, using the most appropriate track out of the old 4.
There is something about this that just feels good. It’s meditative and hypnotizing. I wonder how such qualities of carrying out certain actions for sheer pleasure be used as game mechanics or in game actions in respect to game states or mechanics in Swordy.
Made the wall collision impact sound more pleasant, but the main thing, was a spontaneous piece of code I added and the subsequent discovery. I was running the app in dual game/editor screen while trying to convert my friend Jenna to Unity, and duplicated the cube. Both were controllable and both triggered the collision sounds and background track graphics. So why stop at 2?
Press A(xbox) / X(ps) (or left click on a mouse) to spawn more cubes!
There’s no limit, and the clone function runs every frame if you keep it pressed (not advisable). GO! 😀
On the ground floor, this was an exercise in coding in Unity and fmod one shot / event parameter manipulation integration test. Otherwise, it’s also an exercise in visual language for a sonic environment that I’ve set up. And further exploration in spacial relationships and sound.
I spent longer figuring out how to do the RGB split post effect on bound collision than anything. I feel like every sound event must be accompanied by visual feedback, that way both sight and hearing senses get stimulated at the same time, and there’s reinforcement of what triggers the sound event or the visual effect, otherwise known as Juice.
recommended: run in windowed mode, max settings with a controller (or keyboard wasd)
I realize I’ve been lying to myself for a long time. I’ve been telling myself that “in the time that it would take me to get any good at writing code, I could get proportionately better at something else”.
Now I find myself in a place where I neither did code, nor learned that “other” thing I would supposedly be better at in the time I could’ve learned to code.
As hard as it is, I need to make steps towards code proficiency beyond simple scripts. Implementing my own audio and setting up relationships to drive my sound design is a perfect opportunity to stop giving myself excuses and get it done.
I want to purposefully avoid using tools like playmaker and node based programming aids. Even though I could make things faster, and it’s fine for prototyping, it is not good for production and my learning.
Thankfully, my team @frogshark is here to help 😀
time to get over it and do some code.
Drawing on the mistakes of the first attempt at audio blending, I’ve simplified the setup. Now, all samples get scrubbed simultaneously. Based on the parameter knob, only one plays at any given time. There is no timing issues the first test presented because the playhead doesn’t skip, but keeps looping smoothly. The experience is much more seamless, there are no fades and the sample switch over instantly at the right time.
I also realized the samples didn’t actually loop. It felt like they did because it was long enough, but I needed to add a loop region for it to actually jump to start.
That’s also something to consider for optimization sake. No point loading a 14 second loop with 14 repeats into memory when I can just do one, except, that the instrument that I used to design the loop has subtle randomization happening on the drumkit letting it sound more natural, something I can’t do to easily in fmod unless I assemble the drum kit there from individual samples.
Next: figure out how to avoid hard coding event names into fmod handler script and recognize the available parameters automatically. That way I can make connections in the inspector and prefab the pieces instead of creating a script for every event or knob I wan to modify.
Also, I should make something with more sounds and more interactivity.
I couldn’t figure out how to make the loops be stacked with synchronous playback while having them switch blended on the value. Instead, I set up a system where loop regions for every intensity level had sends based on the main parameter knob. Every loop region has a range of that parameter within which the loop repeats. If I turn the knob, the playhead jumps to another loop for that range of values, and plays it from the beginning.
It was too abrupt at first, because the playhead isn’t jumping to the same time in the bar, but to the start of the loop region. Because the are no bars, these are freeform events within events with audio files inside of them. To alleviate the harsh pop, I added ADSR to every loop. That fades off and blends them smoother, although there’s some doubling up going on, or they go quiet with quick changes from value to value trying to catch up. Everything feels desynchronized still.
I need to test a system with loops that just blend and stack with each other well, to keep the playhead in place while swapping out samples.