Screenings in Miami

Street: Festival of Music, Art and Performance in Miami presents Sinus Aestum on December 6, 2012. 

Clonal Colonies: Fresh Runners is part of MADATAC 04 in Madrid, starting December 10, in Programme #4.

Upcoming Screenings: Louisville, Bulldog Bytes, CSUS Festival of New American Music

The following venues will be screening Clonal Colonies in the coming months:

November 5, 2012
CSUS Festival of New American Music

Sacramento City College
Sacramento, California, USA

March 26, 2013
Bulldog Bytes Concert Series
South Carolina State University
Orangeburg, South Carolina, USA

Sinus Aestum will be screened Nov 7, 2012 at 

University of Louisville School of Music New Music Festival

Louisville, Kentucky, USA 

Forms (process) by Memo Aktin, Quayola +

Forms (process) from Memo Akten on Vimeo.

This provides some exceptional food for thought regarding the khyāl gesture mapping challenge. Akten and team demonstrate powerfully that there is a sweet spot to be found between a direct mapping of body motion and a more abstracted visualisation.

Why is this so effective?

The multiple layers of material originate from the same physical motion, but have independence from each-other and from that motion.

The original motion is, in a sense, providing the impulse that shapes the behavior of different systems. The resulting system behaviors don't just "track" the motion directly, but also clearly reflect the velocity and trajectory of that motion. These velocity patterns cause changes in the behaviors that echo past the time sequences that shape them. Like the "memory trail" one might have of the original motion, the systems emphasize that the dynamics of the motion have perceptual implications beyond the specific narrow windows where they occur.

Because these layered systems both have their own behaviors and are impulsed by the same original motion, there is an engaging "counterpoint" arising between the different layers, all pointing clearly back to the character of the originating motion — even while that original motion has been eliminated from the picture. (I have a long-standing fascination with the idea that audio-visual complexes could achieve coherence due to their relationship to an underlying system that is neither audio nor visual and is not directly perceived).

It is interesting that the left particle system helps emphasise the body mass and its distribution, while the other systems seem to emphasise more the trajectories of points on that mass. I find having both there quite compelling; they tell different aspects of the story.

In the right-hand system, the spline curves seem to shift even after they have been drawn, perhaps even more so with wide arcs of high velocity. It seems likely simply that those points take on an initial velocity and trajectory and that velocity decays over time. Lovely idea.

I'm still scratching my head a bit on the center system — though some kind of spring model seems at play.

If one applies some similar techniques to the khyāl gesturing, it seems that there is a risk that the time independence of the visual systems could blur, rather than highlight, the motion-to-music relationship. But this is certainly worth experimenting with.

Further, it is worth considering what other higher-order aspects (beyond velocity) in the originating motion could fruitfully be applied to shaping system behavior.

Colonal Colonies in Cartes Flux 2012

Clonal Colonies Movement I will be screening in Cartes Flux, Espoo, Finland, 15-22 October 2012

Duplicating Objects in Blender 2.6 Python Scripting

Duplicating objects with Blender 2.6 Python scripting is not quite as straightforward as one might hope, and for me a web search failed to return a simple, clear solution. If found my best solution by searching through the Blender scripts addons folder for files containing the 'copy()' function.

One has to create a new mesh, then copy the data of the source object, then link to the scene. Here's the short function I'm using to do this.

# The following function is adapted from 
# Nick Keeline "Cloud Generator" addNewObject 
# from object_cloud_gen.py (an addon that comes with the Blender 2.6 package)
#
def duplicateObject(scene, name, copyobj):

    # Create new mesh
    mesh = bpy.data.meshes.new(name)

    # Create new object associated with the mesh
    ob_new = bpy.data.objects.new(name, mesh)

    # Copy data block from the old object into the new object
    ob_new.data = copyobj.data.copy()
    ob_new.scale = copyobj.scale
    ob_new.location = copyobj.location

    # Link new object to the given scene and select it
    scene.objects.link(ob_new)
    ob_new.select = True

    return ob_new

Towards Essential Body Relationships

Though I am intending to map the motions of Tofail Ahmed to an abstract visualisation, I want to ensure that important perceptual aspects of the body motion translate effectively. So it is helpful to analyze the performance motions. My intent is to identify high-priority relationships or parameters that I will seek to honour in the abstract visualisation. What "honouring" can and will mean in practice remains to be seen.

Gesture Types

Borrowing terms from Martin Clayton's study of gesture in Khyāl performance (2007), the performances I recorded of Mr Ahmed contained physical gestures that serve as "markers", "illustrators" and "emblems".

Marker gestures indicate a specific time point in musical structure, such as beating a pulse, identifying a downbeat, etc.

Illustrators appear analogous to the melodic flow/motion. The vast majority of the motion falls into this category. It is one vast territory that covers a lot that is very interesting — and very difficult to talk about analytically. This, perhaps, is precisely why it is so valuable!

Emblems, or symbolic gestures, are more based on cultural convention and can be translated relatively readily into a verbal equivalent. An example would be indicating approval with a hand wave.

Emblems would be the most problematic type of gesture given my intent. An emblem is at high risk of disappearing in an abstraction, given the very precise body-arrangement and audience reading it entails. Fortunately, there are very few emblems in the performance I am working with. The closest is an invocation-type emblem. This general position — palms close together, often in front of the face — is important. It creates the impression of focus and preparation, and occurs at the beginning of the first and last phrases of the performance. It also appears at the start of 11 other  phrases of the 60 total phrases. This invocation emblem is also often acting as a marker of the start of phrases. It is doing dual duty.

Invocation: Hand position at start of phrase 1

I am wagering that the element of proximity/closedness is as crucial here as the fact that this can be read as a sign of invocation/gathering.

If the invocation emblem often marks the start of a phrase, the most common marker of a phrase start involves the fingers oriented towards each other horizonally, at mid-body level. The example below is appears at the start of phrase 3:

Rest Position: Hand position at start of phrase 3

I am calling this the rest position. 27 of the 60 phrases start with the hands arranged in this general position. Five others are in closely related positions.

Though there are true full-rest positions of hands on knees (at start) or hands in lap (at end), these seem like outliers that won't be useful as a base position, since none of the Illustrator gestures operate in those spaces.

Measures

Given the above clues, I created some measures to help explore what body relationships might be highly correlated to the structure of the music. Basically, I am assuming that if my abstract visualisations at least clearly carry some of large scale articulators of phrasing, the details within will "take care of themselves".

Proximity to Rest Position (PRP): Clayton (2007) used distance of hand from a rest position in his analysis. I am taking a similar approach, measuring the distance from each finger tip to the neutral position and summing those distances. [Aug 10: However, I might consider using a different term, like neutral position, since Clayton appeared to use rest position to apply to true resting of hands on the legs.]

I did this in Blender by creating an Empty at the rest position and an Empty to represent the distance. I animated the Z axis of the latter with a Driver:

Finger Tip Proximity (FP) and Thumb+Finger Tip Proximity (FTP): The finger tips are in close proximity in both the invocation and the neutral position. So it could be of value to simply measure the distance between the finger tips.

Plus, I took a more refined measurement of hand tip proximity is to take the distance between the finger tips and between the thumb tips, and average them.

As it turns out, close finger tip proximity is often closely related to the beginning of phrases. However, finger tip proximity also closes in fairly often during other mid-phrase events.

Shoulder to Hand Proximity (SHP): The close-to-body versus far-from-body contrast also seems important. As the arms straighten, hands move away from the body. So a simply measure of the degrees of far-from-body is the distance between the shoulder and the hand. This measure sums the left and right shoulder-to-wrist distances.

Hand Closedness (HC): This sums the left and right finger-tip to wrist distances, to provide a measure of the open versus closed state of the hand.

Video

The above video shows the first 18 phrases, with graphic representation of the above measures. This is enough to convince me that Proximity to Rest Position is an excellent candidate to focus on. (But how?)

I could spend a lot of time now analyzing the relationships of these measurements to the music. But that probably will not actually help me attain my immediate goals, so I might have to set that aside for another time.

References

Clayton, M. (2007) "Time, Gesture and Attention in Khyāl Performance". Asian Music v.38 n.2.

Tofail Motion Capture Mapping Test "simple-03"

This test makes me confident that the hierarchical motion mapping idea is worth pursing:


This is a very simple mapping of the upper body motion of Tofail Ahmed as he sings an khyal alaap in raag Bhairavi.

The spheres follow the middle finger and thumb endpoints, with the camera in the viewer position facing Tofail.

The planks are linked in a parenting hierarchy and receive local rotations of bones in the skeleton. (The capacity to do this is easily missed in Blender: when identifying the target object of a Copy Rotations constraint, one indicate the armature, then a second drop down will appear that lets one indicate the target bone.)

Also, to smooth out issues I was having with sudden shifts in the thorax and pelvis rotations, I substituted in the head at the thorax point and made that the root. So the line is head : clavicle: humerus : radius : <hand : finger > <thumb>.

An empty provides parenting to the root, and the empty is rotated 180° on each axis across the whole performance. This provides a foundation of slow, continuous motion to match, if you will, the fundamental drone. The head block, then, applies the rotations from the head as an offset to this base angle.

I'm intrigued by the juxtaposition of the direct position mapping (the balls) with the hierarchical mapping (the planks).