Maya: Animation Event Definition Tool

I've been working on a new set of tools to define animation event data in Maya and export it into Unity for consumption by the AssetPostProcessor. This has been kind of a cool learning experience, because before now I really didn't have much to do with the Animation pipe, which has been an entirely manual process for the animators.

Previously clips were cut up and events were defined by hand in Unity once the .fbx files had been exported from Maya. This wasn't such a bad thing, except that the tools in Unity for defining events were very un-intuitive. The aim of doing all this was to allow the animators the ability to work in a tool they were familiar with, and get the AssetPostProcessor to do most of the boring legwork on the Unity side.

The Maya interface. 
The Maya tool is pretty simple right now, and responsive enough that the animators don't hate me. Navigating to different clips or events will move to that time range on the slider. The event list is directly correlated to events called by the engineers in engine, so they are only available as pre-set items in a drop down.

The data is stored in the scene as attributes on empty nodes. On export time, the data is compiled into dictionaries that are then written into a .JSON file named after the .fbx.

The end result is something like this:

Data for an 'Attack' animation. The first clip is the entire duration, the next three are the intro, loop and outro. In the anim events the unit fires it's weapons three times, once on frame 5, then on 9 and 13. 'Index' is used for the muzzle position on a unit or turret, as those are the only entities in game making use of the system. I would prefer to be using named bones for more flexibility. 
Once this animation is out there, the engineering team pull it into the AssetPostProcessor and generate the required clips and event metadata from the contents. 

All in all, the Maya side is pretty basic at the moment, and implemented on a project that currently has pretty basic animation requirements. Even so, the idea of feeding data into the AssetPostProcessor is something that really appeals to me, and the .JSON format has been pretty easy to write to from Python. 

Eventually I would like to be able to define custom VFX events on named bones, and mark up non-event related data like whether you want an asset to automatically generate lightmap UV's on an imported model or not. There is a lot of potential there. 

ZBrush Speedsculpt: "Hydrosol" Cargo ship.

Theme this week was 'starship' so I made something up. I didn't want to have some big weapon covered star ship, so I went with a cargo freighter design.

"Hydrosol", sculpt time 50 minutes, although I took 10 minutes to sketch out a rough idea before I started.

Solids on top, liquids down the bottom. 

Zbrush Speedsculpt: Alien Pilot Guy... thing...

Hooray for Zbrush lumpy aliens! No particular theme this week, so just kinda sculpted away... Sculpted in under an hour.

Could also pass for an alien optometrist... 

Perforce: Delete empty change lists using Python

Some of the tools I've written allow a user to check out chains of files- like Maya files as well as any exported FBX files and their unity metadata etc, all in one nice pass. This is great, but it has one annoying side effect. If the user presses the button twice the files are added to another change list, but the one they were just in stays there, empty. Press it a number of times and suddenly you have an army of zombie phantom changelists whose only purpose is to clutter up your pending list and annoy you. 

I have been using the P4Python API and found it to be... well... its kinda crap. It would be great if it had a 'delete all empty pending changelists' function, but it doesn't seem to. So I wrote one of my own. 

Beware! If you don't like hacking output out of strings the following code will make you cringe... 

There's my target...
import subprocess

"""Get a list of changelists from the command line. Try to delete them (will delete if empty)"""
# sInfo= subprocess.STARTUPINFO() # Use this instead of the code below to hide the output window. Kinda handy if you don't like annoying artists.  
# sInfo.dwFlags |= subprocess.STARTF_USESHOWWINDOW # Hide the cmd window
# p = subprocess.Popen(command, stdout=subprocess.PIPE, startupinfo=sInfo) # Same as below, but no cmd window. 

workspace = 'YourWorkspace'
# Get the pending changes on the local client using the P4 console commands and reading the output.
command = "p4 changes -s pending -c %s" % workspace 
p = subprocess.Popen(command, stdout=subprocess.PIPE) 
temp = p.stdout.read()

# Split the block of text at the line endings. This should give you one changelist per line. 
raw_data = temp.split('\n') 

print raw_data

# Now go through each line and split the text at the spaces. 
# This should end up with something that reads like ['Change', '99999', 'on' etc etc...
for line in raw_data: 
    target = line.rstrip().split(" ")
    if len(target) > 1: # Skip any list with single elements. That aint got what we want. 
        changeNum = target[1] # The changeNum should be the second element
        command = 'p4 change -d %s' % (changeNum) # Attempt to delete the changelist. P4 refuses to delete changelists with items.  
        p = subprocess.Popen(command, stdout=subprocess.PIPE) # Lets get some info back again. 
        temp = p.stdout.read() # Tell us about what you just did. 
        print temp # Good boy. 

The Result... poor 76610 got what was coming to him...

Output will be something like this:
["Change 76610 on 2014/02/06 by ****@**** *pending* '[EmptyChangeListToNuke] '\r", 
"Change 76607 on 2014/02/06 by ****@**** *pending* '[ToolDev] Stuff '\r", '']

Change 76610 deleted.
Change 76607 has 35 open file(s) associated with it and can't be deleted.




Use Python to use JavaScript to get Photoshop to do stuff... and tell you about it!

Its been pretty well established that you can send a .jsx file to Photoshop using a subprocess call in Python. The tricky part is then getting Photoshop to send some information back.

This is my awesome hacky method of passing information from Photoshop to Python. It relies on being able to write to a temporary text file from Photoshop and then reading that information back into Python. This method relies on actually being able to write data to disk... if that's a problem I suspect you could do the same to the console standard output and get the same results... somehow? Maybe?

Because writing to the disk was not a problem in this situation, I went with a temp file solution. The only kinda tricky part was making my program wait for the return value to be passed. Subproces.call() returns a value of 1 or 0 from the shell, but this only indicates that the program successfully (or not) opened.

Its highly likely that whatever script you passed to Photoshop as part of the Subprocess call will still be executing by the time your Python comes to the section where you want to read the return data. In this case, your Python code will likely be reading old data from the temp file, or, no data at all.

In this case, I was fine with having my program wait until the data it needed was available. I did this by doing a check to see if the temporary output text file had been modified. Once this condition was met, the file was opened in Python and the contents were pulled back into the main program.

I've seen some people recommending using a JSON file to do this, which is something I might look into if I need more complex feedback than a single line.

Here is an example of a Python script which builds a .jsx file, sends it to Photoshop, waits for a return value and then prints the return value out to the console.

"""
Example which builds a .jsx file, sends it to photoshop and then waits for data to be returned. 
"""
import os
import subprocess
import time
import _winreg

# A Mini Python wrapper for the JS commands...
class PhotoshopJSWrapper(object):
    
    def __init__(self):
        # Get the Photoshop exe path from the registry. 
        self.PS_key = _winreg.OpenKey(_winreg.HKEY_LOCAL_MACHINE, 
                                      "SOFTWARE\\Adobe\\Photoshop\\12.0")
        self.PS_APP = _winreg.QueryValueEx(self.PS_key, 'ApplicationPath')[0] + 'Photoshop.exe'          

        # Get the path to the return file. Create it if it doesn't exist.
        self.return_file = 'c:\\temp\\ps_temp_ret.txt'
        if not os.path.exists('c:\\temp\\'):
            os.mkdir('c:\\temp\\')
        
        # Ensure the return file exists...
        with open(self.return_file, 'w') as f:
                f.close()  
            
        # Establish the last time the temp file was modified. We use this to listen for changes. 
        self._last_mod_time = os.path.getmtime(self.return_file)         
        
        # Temp file to store the .jsx commands. 
        self.temp_jsx_file = "c:\\temp\\ps_temp_com.jsx"
        
        # This list is used to hold all the strings which eventually become our .jsx file. 
        self._commands = []    
    
    # This group of helper functions are used to build and execute a jsx file.
    def js_new_command_group(self):
        """clean the _commands list. Called before making a new list of commands"""
        self._commands = []

    def js_execute_command(self):
        """Pass the commands to the subprocess module."""
        self._compile_commands()
        self.target = '"' + self.PS_APP +'"' + " " +  '"' + self.temp_jsx_file + '"'
        print self.target
        ret = subprocess.Popen(self.target) 
    
    def _add_command(self, command):
        """add a command to the commands list"""
        self._command_list.append(command)

    def _compile_commands(self):
        with open(self.temp_jsx_file, "wb") as f:
            for command in self._commands:
                f.write(command)
           
    # These are the strings used to build the .jsx file.  
    def js_create_document(self, varName, w, h, docName):
        """
        Javascript command to create a new document. Returns varname as 
        a reference to the jsx variable. 
        """
        self._mode = " NewDocumentMode.RGB" # Hard set, but easy to add as a python var
        self._init_fill = "DocumentFill.WHITE" # Hard set, but easy to add as a python var
        self._PaR = 1.0 # Hard set, but easy to add as a python var
        self._BpC = "BitsPerChannelType.EIGHT" # Hard set, but easy to add as a python var 
        
        self._com = (
            """
            %s = app.documents.add(%s, %s, 72, "%s", %s, %s, %s, %s);
            """ % (varName, w, h, docName, self._mode, self._init_fill, self._PaR, self._BpC)
            )
        self._commands.append(self._com)
        return varName # Return the name we used for the jsx var, we can use this later in the Python code

    
    def js_write_data_out(self, returnRequest):
        """ An example of getting a return value"""
        self._com = (
            """
            var retVal = %s; // Ask for some kind of info about something. 
            
            // Write to temp file. 
            var datFile = new File("/c/temp/ps_temp_ret.txt"); 
            datFile.open("w"); 
            datFile.writeln(String(retVal)); // return the data cast as a string.  
            datFile.close();
            """ % (returnRequest)
        )
        self._commands.append(self._com)
        
        
    def read_return(self):
        """Helper function to wait for PS to write some output for us."""
        # Give time for PS to close the file...
        time.sleep(0.1)        
        
        self._updated = False
        while not self._updated:
            self._this_mod_time = os.path.getmtime(self.return_file)
            if str(self._this_mod_time) != str(self._last_mod_time):
                self._last_mod_time = self._this_mod_time
                self._updated = True
        print "Return Detected"
        
        f = open(self.return_file, "r+")
        self._content = f.readlines()
        f.close()      
        self._ret = []
        for item in self._content:
            self._ret.append(str(item.rstrip()))
        return self._ret
    
    
# An interface to actually call those commands. 
class PhotoshopJSInterface(object):
    
    def __init__(self):
        
        self.psCom = PhotoshopJSWrapper()
    
    def create_new_document(self, x, y, docName):
        """Compile a command to create a new document"""
        self.psCom.js_new_command_group() # Clears the command list. 
        self.docRef = self.psCom.js_create_document('docRef', x, y, docName) # Adds the new document command to the list. 
        self.psCom.js_write_data_out(self.docRef + ".activeLayer.name") # Get the document's active layer name. 
        self.psCom.js_execute_command()
        
        # Now I find the return value. 
        self.layerName = self.psCom.read_return()[0]
        print "Current active layer:", self.layerName
        
        
PS = PhotoshopJSInterface()
PS.create_new_document(512, 512, 'My Amazing Document')

Now, in my mind this is pretty handy, and could be extended to a point where it could become a viable Python API for Photoshop. One thing I do want to look into is using Socket control to talk to the Photoshop application directly, replacing the use of the Subprocess module. Maybe it would be possible to then get information back without writing to a temp file. Has anyone tried this?

Texture Monkey: Starting to look like a real tool.

Stuff is happening...
After a little while in the field it came to be pretty apparent that the win32com module can't be relied upon to consistently work on everyone's machines. I spent a good amount of time trying to work out why, but came up with no solution that worked consistently on everyone's workstations. Errors like ('Member not found, None, None) would constantly pop up in their debug feedback, but not on mine. Talk about frustrating. So I got pretty annoyed and just ripped the win32com components out. 

This was a pretty big undertaking, because apart from the GUI, win32com stuff made up about 80% of the remaining code. The P4 functionality wasn't touched, because its operating through it's own native API. 

The solution I went with was to write a wrapper around the JavaScript equivalent of what I was using the win32com module for. Each of these commands would be put into a list, which is then compiled into a temporary .jsx file. Finally this file is sent to Photoshop using the subprocess module. 

To deal with getting information from Photoshop a similar process was used. A JavaScript command is called to write the requested data to a file on disk, while Python waits until it can see that the file has been modified. Once Python can see that Photoshop has completed writing to the file, the contents are read and fed back into the main program. 

Benefits/Cons
Benefits? Right off the bat, the JavaScript executes much more quickly than the win32com commands. This is pretty cool, especially when coupled with the fact that it now works on all the artists machines without mysterious bugs. Yet. Also, I still get to keep writing the tool in Python.

The biggest con is that I can't help but feel that its a mother of a hack. I mean, writing JavaScript on the fly from Python to send to a program which can only feed data back through a temporary text file? Man. Awful. But it works. But I feel dirty. But it works. But what about those ugly chunks of strings pretending to be JavaScript? But it works. Yes it works. The structure of the code behind the scenes feels like its taken a train to the chest, but it works. My next couple of days will be pulling it together and making the code a little easier on the eyes.

First Speedsculpt 2014- Bear

Zbrush, one hour. Felt like doing something a little more natural looking. Kinda feel I could have pushed the character a little further... maybe I will later.

Its a happy/bored bear...