Revision as of 15:59, 29 December 2012

UDKOSC is an implementation of the Unreal3 engine built using the Unreal Development Kit(UDK) with an integrated OscPack Windows dll linked to custom UnrealScript classes.

By linking custom UnrealScript classes to Ross Bencina's excellent OscPack Open Sound Control c++ implementation, actions and motions taken by virtual actors in the Unreal3 game engine are encapsulated as OSC messages over UDP and used to control dynamic music and sound synthesis systems in a manner similar to q3osc.

UDKOSC Seeking projectiles tracking Target projectiles

UDKOSC was first used in performance at the 2010 September 7,8 performances of Tele-Harmonium and Perkussionista in Milano, Italy at the 2010 MiTo (Milano-Torino) Festival.

Helpful Console Commands

Overview

UDKOSC is the fourth project in a lineage of integrating virtual worlds and environments with interactive audio and music servers with a common goal of creating shared and distributed musical performance spaces. Like our custom Sirikata implementation for the 2009 MiTo festival, q3osc and Maps & Legends before, UDKOSC combines a game-based virtual world implementation (in this case the Unreal Development Kit or UDK) with customized classes to stream game-data to interactive sound-servers using Open Sound Control messages over UDP. In this manner, gestures, motions and actions from actors in virtual space can be transformed in real-time into control messages for complex audio and musical software systems.

While the UDK offers only free access to the engine's UnrealScript programming layer (leaving the core C++ engine accessible only through significant financial investment) UnrealScript has a provision for binding Windows Win32 .dll's to UnrealScript classes, enabling code such as a C++ implementation of Open Sound Control (in this case, Ross Bencina's OSCPack) to provide bi-directional communication between game engine and sound-server. By then creating mirrored data structures both in UnrealScript classes and in the Windows .dll, data can be passed back and forth between the UDK and any OSC-enabled control system or software, including ChucK, Supercollider, Pure Data (PD) and Max/MSP.

For UDKOSC, a custom Windows oscpack_1_0_2.dll was compiled with specific extern methods and mirrored data structures to communicate with the game engine, both to stream specific game data out as OSC over UDP (including Pawn XYZ coordinate positioning, XYZ coordinate tracking for in-game projectiles, state and XYZ coordinate info for Static Mesh and Interp actor classes) and to send control data from iPad controllers and from the audio-engine itself back into the game to control Global GameInfo parameters such as world Gravity and GameRate, as well as specific positioning information for projecties and static actors.

Custom Functionalities

Player Pawn outputs location, landed, crouch events over OSC

Homing projectiles (ShockBalls, SeekingShockBalls)

Can be assigned to track other projectiles (ShockBalls), UTVolumes or targeted locations (via left-mouse fire trigger)

Can be resized; size param is sent over OSC

Bounce and Destroyed events tracked over OSC

Instance numbering and class name/type tracked over OSC

ShockBall projectiles can target XYZ coordinates input via OSC

InterpActors track position deltas over OSC

takeDamage, bump and touch events output OSC

Triggers fire triggeron, triggeroff events over OSC

Gravity and GameSpeed can be set via OSC

OSC can be used to control player movement (X, Y, Z vector directions, Speed and Jump)

OSC can be used to control camera movement (X, Y, Z, Pitch, Yaw, Roll, Speed)

Exec Methods

Exec methods can be run from the UDK command-line in-game. The command-line can be accessed by pressing "~" for the full drop-down terminal window or by pressing "TAB" for the single-line window-bottom command-line. These are the custom UT3OSC exec methods that have been implemented to this point:

Getting Started

Follow the Subversion installation instructions below to install files to the correct locations.

There are only 4 locations where files need to be instlled:

UDK-2010-07\Development\Src\UT3OSC: these are the custom OSC UnrealScript classes

UDK-2010-07/Binaries/Win32/UserCode: the oscpack windows dll

UDK-2010-07/UDKGame/Config: the custom default config files

UDK-2010-07/UDKGame/Content/Maps: custom maps go here

Open UnrealFrontend to compile UT3OSC

?game=UT3OSC.UT3OSC should be pasted into the "Extra Options" field

check the "show log" check box to see debug messages in the console

in 2012-05 sometimes baking is required to run project (not sure why)

Launch UDKOSC

DETAILS COMING SOON

Launch Map

DETAILS COMING SOON

Supercollider

Default input port 57120

SETUP OSC CLIENT FOR MONITORING

DETAILS COMING SOON

Timeline

12/20/2012

Created installer package and uploaded to ccrma servers. Users can just install UDKOSC from the windows installer now, though the udkosc .dll must be manually added to the project's Binaries/Win32/UserCode/ directory.

12/20/2012

updated master branch with V1 working versions of OSC input control for Pawns and functional OSC output for Pawns

added branch 'dev' to github repo for dev changes, master will remain (relatively) stable and dev will be for ongoing work

8/22/2010

Set ctrl-F to call WalkFly exec to toggle pawn state between Fly and Walk

8/19/2010

Added OSCProj_SeekingShockBall, and OSCWeap_SeekingShockRifle for glowing balls with seeking properties, taken from the OSCProj_SeekingRocket code

Added media/ScreenShots dir to svn for archiving ingame snaps

8/18/2010

Cleanup on UDK-side of multi-touch control; now shockballs don't explode on collision with other shockballs and when initOSCFingerTouches is TRUE, will seek out the next available finger touch location currently being received.

Created Max/MSP test patch for control testing.

8/17/2010

Added initial OSC Input to control seeking of ShockBalls to be used as controlled by 5 iPad finger touches.

/fingers X Y Z on/off (all floats, for fingers 1-5)

8/16/2010

Smoothed out homing functionality for SeekingRockets; now make nice smooth arcs.

8/01/2010

Checked in modifications to July UDK to svn at ccrma. Note only modified files/custom files were checked in, including the source for the oscpack windows dll, and these should be checked out into a 2010-07 install of the UDK.

7/31/2010

OSCInterpActor class

- outputs position data on move, bump and damage as well
- Kismet script tested successfully to spit out position data while object moves

DLL Binding

"A single UnrealScript class can bind to only a single DLL. The DLL to bind to is specified with the DLLBind directive, and the DLL name to bind to is specified in parentheses. Do not include a path or .DLL extension. DLLs can only be loaded from the Binaries\Win32\UserCode folder."
- http://udn.epicgames.com/Three/DLLBind.html