Back to MEL

I've used 3dsMax for years, and one of my favorite plugins for it was the good old pitsNpeaks. Using it you could bake in really nice dirt passes into a mesh's vertex color quickly and painlessly.

I'm now on the hunt for an equivalent script for Maya, just because it's so very handy.

Styling it up...

Now that I've gotten to grips with making Python scripts that work, I'm paying more attention to readability and style. I've had the PEP 8 Style guide for Python Code recommended to me as a good standard to follow.

TextureMonkey v1.0

Well, here is TextureMonkey V1.0 with the Alienbrain integration. If anybody ends up using it I'd like to hear about how it goes. Also, I'm interested in hearing about the way you integrate it into Photoshop. I integrated it using a javascipt and a call from a batch file, but I'm sure there is a more streamlined way of doing it.

In order to get it set up, make sure you fill in the variables up top with your project specifics. These are all called throughout the script to keep it tidy.

And here it is:
##############################################################################
#
# Photoshop texture exporter v1.0
# (C) Pete Hanshaw, 2012
# http://peterhanshawart.blogspot.com.au/
# Inspired by recursive Photoshop layer export script by
# Adam Pletcher
# http://techarttiki.blogspot.com.au/
#
##############################################################################
#
# Checks to see if a PSD file is open.
# For the currently active file, exports various 24-bit TGA textures based on
# layer groups found in the PSD.
#
# If Alienbrain is present attempts to check out the export files if required
# Does not check out source file- this should be left to the artists to ensure
# they are working with the latest revision PSD before modifying it.
#
# Requires the Win32 Extensions:
# http://python.net/crew/mhammond/win32/
#
##############################################################################
 
#Import required modules
from win32com.client.dynamic import Dispatch
from stat import *
import os
from sys import exit
import pythoncom
 
pythoncom.OleInitialize()

#Alienbrain variables
ab_project = "yourProjectHere"
ab_server = "alienbrainServerHere"

#Login name
ab_user_name = "yourUserNameHere" 

#Alienbrain workspace
abWorkspace = "\\\\Workspace\\yourProjectHere\\"

#the relative paths we want to export to
sourceTarget = "art\\source\\textures"
assetTarget = "art\\assets\\textures"
exportTarget = "art\\exports\\textures"

#project HDD location
projectRoot = "Z:\\yourProjectHere\\"

#map layer group names to export names. Currently includes:
#Diffuse, Normal, Specular, Illuminance, Gloss, alpha
exportTypes = {'d': '_d', 'n': '_n', 's': '_s', 'i' : '_i', 'g': '_g', 'a':'_a'}


#name check function
def name_check(name_to_check, exportTypes):
    """Checks the PSD name for bad naming convention.

    Specifically, makes sure that if an artist has already included _d or _a etc 
    on the end of the source file name, it does not get included in the final 
    export output name.
    """

    #create a variable containing the raw name
    doc_name = name_to_check
    
    #Create a test case to test against the PSD name that has the last two 
    #characters removed
    test_name = doc_name[:-2]

    #for each of the different conventions that an artist may have used...
    for key, name in exportTypes.items():

        #Create a test name to match against the psd name
        test_case = test_name + name

        #match them against each other. If we have a match, remove whatever the
        #additional two characters were and return a clean doc name
        if test_case == doc_name:
            doc_name = doc_name[:-2]
            return doc_name
            break
            
    #If we did not get a match in the for loop we can pass through an 
    #unmodified doc_name 
    return doc_name    


#Function to check permissions of files
def check_permissions(abWorkspace, project_path, relative_path, doc_name, layer_name):
    """Checks to see if an asset file is writable.
    
    If it is not writable, attempts to check it out from alienbrain.
    """
    check_path = project_path + relative_path + "\\" + doc_name + layer_name + ".tga"
        
    if os.access(check_path, os.W_OK):
        print "Files writable"
        return
    else:
        print "\n", check_path, "is not writable."
        print "Monkey wants to check that out for you!"
        #set the base path for the exports
        as_base_path = abWorkspace + relative_path + "\\" + doc_name + exportTypes[lsName] + ".tga"
        print "Checking out ", as_base_path

        com_param.Reset()
        com_param.Command = "CheckOut"
        com_param.SetParamIn("ShowDialog", "0")
       
        com_nxn.RunCommand ( as_base_path, com_param.Command, com_param.xml )

        #check the writability of the file again
        if os.access(check_path, os.W_OK):
            print "Check out successful"
            return
            

        else:
            os.system("color 0c")
            print "Checking out ", as_base_path, " failed"
            print "Please check out the export files manually"
            doc.ActiveHistoryState = saved_state            
            os.system("PAUSE")
            exit(0)

#Begin the script            
if (__name__ == '__main__'):
    #COM dispatch for Photoshop
    try:
        psApp = Dispatch('Photoshop.Application')
 
    except:
        os.system("color 0c")
        print "OOPS! Something went wrong..."
        print "The dispatch to Photoshop did not work"
        print "Texture monkey hides in shame..."
        os.system("PAUSE")
        exit(0)

    #attempt to establish a connection to the AB server
    com_nxn = Dispatch("NxNNamespace.NxNNamespaceHelper")
    com_param = Dispatch("NxNXMLHelper.NxNXMLParameter" )
    com_param.Command = "ProjectLoadEx"
    com_param.SetParamIn(ab_user_name, ab_project)
    com_param.SetParamIn(ab_server, "alienbrainServer")

    # Define 24bit Targa save options
    options = Dispatch('Photoshop.TargaSaveOptions')
    PsTargaBitsPerPixels = 24
    options.Resolution = PsTargaBitsPerPixels
    options.AlphaChannels = False
    options.RLECompression = False

    # Define 32bit Targa save options
    optionsAlpha = Dispatch('Photoshop.TargaSaveOptions')
    PsTargaBitsPerPixels = 32
    optionsAlpha.Resolution = PsTargaBitsPerPixels
    optionsAlpha.AlphaChannels = True
    optionsAlpha.RLECompression = False
 
    # Define PSD save options
    psdOptions = Dispatch('Photoshop.PhotoshopSaveOptions')
    psdOptions.annotations = False
    psdOptions.alphaChannels = True
    psdOptions.layers = True
    psdOptions.spotColors = True
    psdOptions.embedColorProfile = True
 
    #Get the currently active document
    try:
        doc = psApp.activeDocument
 
    #Gracefully end if a document is not open
    except:
        os.system("color 0c")
        print "OOPS! Something went wrong..."
        print "You need to have an active Photoshop Doc to use this script."
        os.system("PAUSE")
        exit(0)
 
    #Get the document name and strip it's extensionprojectRoot
    sourceFile = projectRoot + sourceTarget + '\\' + doc.name
    doc_name = os.path.splitext(doc.name)[0]

    #Change the names to lower case
    sourceFile = sourceFile.lower()
    doc_name = doc_name.lower()
    
    #check to see if the source file can be saved
    try:
        doc.SaveAs(sourceFile, psdOptions)
        print "Saving original doc", sourceFile
    except Exception:
        os.system("color 0c")
        print "\nThe PSD is not writable!\n"
        print "Monkey doesn't like messing with source files!"
        print "\nManually check out ", sourceFile," and try again!\n"
        os.system("PAUSE")
        exit(0)

    #Assuming the PSD is writable, lets go ahead and initialise alienbrain. 
    print "Monkey wants to load AlienBrain"
    tmp = com_nxn.RunCommand ( abWorkspace, com_param.Command, com_param.xml )
    if ( com_param.WasSuccessful ):
        print "SUCCESS: Alienbrain loaded."
    else:
        print "ERROR: Project failed to load"
        print "TextureMonkey won't be able to check out files for you!"
    
    #Check the PSD name for redundant extensions
    doc_name = name_check(doc_name, exportTypes)
    
    print "Exporting from ", sourceFile
    #print "Using base export name ", doc_name
    
    #Get our layer sets from the currently open doc
    layerSets = doc.LayerSets
 
    #Check if there are any layerSets in the current doc
    if (len(layerSets) > 0):
        # first hide all root-level layers
        for layer in doc.Layers:
            layer.Visible = False
 
        # ... and layerSets
        for layerSet in layerSets:
            layerSet.Visible = False
 
        # Loop through each LayerSet (aka Group)
        for layerSet in layerSets:
            lsName = layerSet.Name.lower()
 
            #save the current setup
            saved_state = doc.activeHistoryState
 
            if (lsName in exportTypes):
                layerSet.Visible = True # Make the group visible

                #make the asset and export file names
                exportFile = projectRoot + exportTarget + '\\' + doc_name + exportTypes[lsName] + '.tga'
                assetFile = projectRoot + assetTarget + '\\' + doc_name + exportTypes[lsName] + '.tga'
                
                #See if the export files are writable. If not, they are probably
                #not checked out!
                if (os.path.exists(exportFile)):
                    check_permissions(abWorkspace, projectRoot, exportTarget, doc_name, exportTypes[lsName])

                #Do the same thing for the asset file        
                if (os.path.exists(assetFile)):
                    check_permissions(abWorkspace, projectRoot, assetTarget, doc_name, exportTypes[lsName])
                
                #Flatten the image temporarily to allow TGA export
                doc.flatten

                #Check to see if the export requires alpha options
                if lsName == "a":
                    #Do our exports
                    doc.SaveAs(exportFile, optionsAlpha)
                    doc.SaveAs(assetFile, optionsAlpha)
                    print 'exporting: ', exportFile
                    print 'exporting: ', assetFile

                else:
                    #Do our exports
                    doc.SaveAs(exportFile, options)
                    doc.SaveAs(assetFile, options)
                    print 'exporting: ', exportFile
                    print 'exporting: ', assetFile
 
                #Go back to the saved history state
                doc.ActiveHistoryState = saved_state 
 
                #Hide the layer to make way for the others...
                layerSet.Visible = False

        #now that its all done, make the layer sets visible again
        for layerSet in layerSets:
            layerSet.Visible = True
        
        #Save the PSD to retain any changes and return control to the humans.
        doc.SaveAs(sourceFile, psdOptions)
        print "Reverting control to humans!"
        print "Monkey work done!\n"
 
    # If there are no layer sets present, 'gracefully' end the script.
    else:
        os.system("color 0c")
        print "OOPS! Something went wrong..."
        print "The file has no groups to export!"
        exit(0)

Python command line text color

Want an error message with a difference? Color Module is an excellent reference if you want to get your command line popping out in any number of different colors. Windows only.

TextureMonkey and Alienbrain

Well that wasn't so hard... using the same win32com interface as the one I am using in to control PhotoShop, I have been able to check out files from Alienbrain within the TextureMonkey script with minimum fuss.

Having access to Ben Deda's Alienbrain python interface was a massive help in this! I suggest anyone trying to drive AlienBrain with python to have a good look through it.

The script is now pretty self sufficient.
  • It allows a user to export all their maps from a Photoshop document to specified folders
  • It names the exported textures appropriately, based on the document and layer names 
  • It is smart enough to know when to export a texture with an Alpha  channel
  • It can check file permissions and attempt to check out files if required
  • If the files are checked out by another user, AlienBrain shows a dialogue of who has it checked out
  • If it can't obtain file write permissions it closes down and informs the user.   
Two issues with the I would like to resolve to make it as user friendly as possible:
  • Currently every time the script runs, it attempts a new connection with Alienbrain. ping Alienbrain somehow so that it doesn't need to keep connecting every time the script is run, and will only attempt to connect if a connection is not present. 
  • hide the cmd window when the script is executing for as little distraction as possible.

TextureMonkey- next steps

So I have got the script into a working form. It's still got some ugly bits, but it does the job its intended to do well enough to be popular with the other artists and is already showing a lot of promise.

So what next? 
  • Currently the script has no idea how to handle exceptions properly, so that's totally on the todo list.
  • The script doesn't like it when the files are not writable, and isn't smart enough at the moment to know what to do about it. 
  • I'm still convinced that I can copy the output TGA's to at least one of the output directories. Although I'm not sure how much time this will save, I'd still like to give it a try. The script execution is pretty rapid, so the savings would only really become apparent on Photoshop files that are spitting out a lot of different maps.
So my goals over the next couple of weeks are:
  • Implement effective exception handling
  • Somehow integrate the script with Alienbrain asset management to at least check-out the files if they are not writable. I don't intend to do anything about the check-in process, as its totally out of the scope of the script, but I may do it with a different one in the future. 
  • Investigate the copy TGA option and see if it actually saves time.
Back to eeeeit......

TextureMonkey in the pipeline...

While testing out the textureMonkey script I came across a couple of little hurdles to get it working within the current pipeline.

The most annoying bug came from the naming of the PSD files themselves. Many had been named with the _d, _n and _s already in the PSD name, in order to work with a Photoshop action that was already doing a basic version of what TextureMonkey was intended for- saving the flattened PSD as a TGA in a couple of different directories.

When TextureMonkey was used on these files, it came out with an ugly export name, eg:

'psd_name_d_d'
'psd_name_d_n'
'psd_name_d_s'

Lameness, but a reasonably easy fix. I added a basic function that checks for the extensions in the PSD name, and if they were there, to remove them from the export file name:

def name_check(name_check):
    """Checks the PSD name for the old naming convention.
    """
    doc_name = name_check
    old_name = ('_d', '_n', '_s', '_a', '_t', '_D', '_N', '_S', '_A', '_T')

    #Create a test case to test against the PSD name
    name_check = doc_name[:-2]
    for name in old_name:  

        test_case = name_check + name

        if test_case == doc_name:
            doc_name = doc_name[:-2]
            print "It looks like the file is using the old naming conventions"
            print "Monkey wants to fix that for you!"
            print "TADA! Removed ", name, "from", doc_name, "source"
            print "Proceeding"
            return doc_name
            break
            
    #If we did not pick up anything in the for loop 
    #we can pass through an unmodified doc_name 
    return doc_name
   

Works pretty well, and does the job. Ideally, all the source files wouldn't have the extensions already in the name, but its not a show stopper.

-Pete

Running the automation from Photoshop

So, there is probably a better way of doing it, but I have been able to run the script from within photoshop using a javascript that calls a .bat file that then calls my python script. It's a convoluted way of doing it, but it does allow me to directly call textureMonkey using an action from within photoshop.

The javascript looks like:
var textureMonkey = new File("/z/Tools/textureMonkey/textureMonkey.bat");
textureMonkey.execute();
and the .bat file called looks like...
@echo off
call python Z:\Tools\textureMonkey\textureMonkey.pyw
The whole thing has been very easy to set up for the art team and is already saving time that would have otherwise been spent navigating folders and changing file names. -Pete

More scripting reference

In addition to the adobe com reference and VB scripting reference, I've been taking a look through another site called tranberry to learn more about photoshop scripting. I'll post if anything handy comes up.

Photoshop Automated Texture Exporting

Python Texture Monkey
Requires the Win32 Extensions: http://python.net/crew/mhammond/win32/

So the script has progressed far enough to be usable, but still has several crude elements, such as the way it exports the TGAs into each separate folder.  

Currently the script works like so:
  •  Using python grabs whatever doc is currently active in Photoshop and looks through it for specified group names
  • Exports each of these groups as a TGA in specified directories with a specific suffix depending on the original group name- eg the diffuse 'd' group becomes 'sourceTextureName_d'
  • Saves original doc and reverts control to user
  • Can export:
    • 'd' - Diffuse as 'fileName_d.tga'
    • 'n' - Normal as 'fileName_n.tga'
    • 's' - Spec as 'fileName_s.tga'
    • 'i' - Illuminance as 'fileName_s.tga'
    • 'a' - Texture with alpha as 'fileName_a.tga'
    • 'g' - Gloss as 'fileName_g.tga'
This script is based on a recursive layer export script by Adam Pletcher. The key differences between my script and Adam's are:
  • My new script exports TGA files rather than PNGs 
  • The script runs on the currently active Photoshop doc rather than recursively scanning through a folder. This is handy as it can be integrated directly into a user's workflow for rapidly updating work. 
  • Due to my current test project setup, TGA files are exported to two different folders- a Maya 'asset' location and an engine 'export' location. This can be easily modified by the savvy end user.
Intended future refinements:
  • Adding a button in Photoshop. Most artists don't like working from the command prompt! :-)
  • Optimizing the TGA export. As the export and asset TGA files are essentially the same thing, it might work out quicker to export it once to the asset folder and then copy the output over to the export folder.
  • Easily changed options for different export formats 

##############################################################################
#
# Photoshop texture exporter
# Author: Pete Hanshaw, 2012
# http://peterhanshawart.blogspot.com.au/
# Inspired by recursive Photoshop layer export script by
# Adam Pletcher
# http://techarttiki.blogspot.com.au/
#
##############################################################################
#
# Checks to see if a PSD file is open.
# For the currently active file, exports various 24-bit TGA textures based on
# layer groups found in the PSD.
#
# Requires the Win32 Extensions:
# http://python.net/crew/mhammond/win32/
#
##############################################################################

#Import required modules
from win32com.client.dynamic import Dispatch
import os
from sys import exit
import pythoncom

pythoncom.OleInitialize()

#Define our target directories for each of our exports
#Change this for whatever project you are working on
sourceTarget = r'c:\textureWork\source'
assetTarget = r'c:\textureWork\asset'
exportTarget = r'c:\textureWork\export'

#map layer group names to export names. Currently includes:
#Diffuse, Normal, Specular, Illuminance, Gloss, alpha
exportTypes = {'d': '_d', 'n': '_n', 's': '_s', 'i' : '_i', 'g': '_g', 'a':'_a'}

if (__name__ == '__main__'):

    #COM dispatch for Photoshop
    try:
        psApp = Dispatch('Photoshop.Application')

    except:
        print "OOPS! Something went wrong..."
        print "The dispatch to Photoshop did not work"

    # Define 24bit Targa save options
    options = Dispatch('Photoshop.TargaSaveOptions')
    PsTargaBitsPerPixels = 24
    options.Resolution = PsTargaBitsPerPixels
    options.AlphaChannels = False
    options.RLECompression = False

    # Define 32bit Targa save options
    optionsAlpha = Dispatch('Photoshop.TargaSaveOptions')
    PsTargaBitsPerPixels = 32
    optionsAlpha.Resolution = PsTargaBitsPerPixels
    optionsAlpha.AlphaChannels = True
    optionsAlpha.RLECompression = False

    # Define PSD save options
    psdOptions = Dispatch('Photoshop.PhotoshopSaveOptions')
    psdOptions.annotations = False
    psdOptions.alphaChannels = True
    psdOptions.layers = True
    psdOptions.spotColors = True
    psdOptions.embedColorProfile = True

    #Get the currently active document
    try:
        doc = psApp.activeDocument

    #Gracefully end if a document is not open
    except:
        print "OOPS! Something went wrong..."
        print "You need to have an active Photoshop Doc to use this script."
        print "Goodbye!"
        exit(0)

    #Get the document name and strip it's extension
    sourceFile = sourceTarget + '\\' + doc.name
    doc_name = os.path.splitext(doc.name)[0]
    print "Operating on ", sourceFile

    #Get our layer sets from the currently open doc
    layerSets = doc.LayerSets

    #Check if there are any layerSets in the current doc
    if (len(layerSets) > 0):
        # first hide all root-level layers
        for layer in doc.Layers:
            layer.Visible = False

        # ... and layerSets
        for layerSet in layerSets:
            layerSet.Visible = False

        # Loop through each LayerSet (aka Group)
        for layerSet in layerSets:
            lsName = layerSet.Name.lower()

            #save the current setup
            saved_state = doc.activeHistoryState

            if (lsName in exportTypes):
                layerSet.Visible = True # Make the group visible

                #make the asset and export file names
                exportFile = exportTarget + '\\' + doc_name + exportTypes[lsName] + '.tga'
                assetFile = assetTarget + '\\' + doc_name + exportTypes[lsName] + '.tga'
 
                #If the file exists, delete it
                #This is somewhat clumsy, but does the job
                if (os.path.exists(exportFile)):
                    os.remove(exportFile)

                if (os.path.exists(assetFile)):
                    os.remove(assetFile)

                #Since the export file is just a copy of the asset file, it
                #would be faster to just copy it using the o.s. rather than save
                #it all over again!

                #Flatten the image temporarily to allow TGA export
                doc.flatten

                #Check to see if the export requires alpha options
                if lsName == "a":
                    #Do our exports
                    doc.SaveAs(exportFile, optionsAlpha)
                    doc.SaveAs(assetFile, optionsAlpha)
                    print 'exporting: ', exportFile
                    print 'exporting: ', assetFile

                else:
                    #Do our exports
                    doc.SaveAs(exportFile, options)
                    doc.SaveAs(assetFile, options)
                    print 'exporting: ', exportFile
                    print 'exporting: ', assetFile

                #Go back to the previous history state
                doc.ActiveHistoryState = saved_state 

                #Hide the layer to make way for the others...
                layerSet.Visible = False

        #now that its all done, make the layer sets visible again
        for layerSet in layerSets:
            layerSet.Visible = True

        #Save the PSD to retain any changes and return control to the humans.
        doc.SaveAs(sourceFile, psdOptions)
        print "Saving original document as ", sourceFile
        print "Reverting control to humans!"
        print "All done!" 

    # If there are no layer sets present, gracefully end the script.
    else:
        print "OOPS! Something went wrong..."
        print "The file has no groups to export!"
        print "Goodbye!"
        exit(0)

The Code... or is it?

So I got it working, but now I need to find a decent way of displaying code on my blog! Stay tuned!

Automated Texture Exports

I've got far enough in Python to start taking apart other people's scripts and reassembling them to do the things I want them to do. Towards this end I have taken the Photoshop layer exporter script by Adam Pletcher and modified it in several key ways to adapt it to the pipeline I'm currently working in.

I found the adobe com reference of moderate help, as well as the VB scripting reference, especially when I needed to search for specific things like how to step back to a previous history state.

 The key differences between my script and Adam's are:

  • My new script exports TGA files rather than PNGs 
  • The script runs on the currently active Photoshop doc rather than recursively scanning through a folder. This is handy as it can be integrated directly into a user's workflow for rapidly updating work. 
  • Due to my current test project setup, TGA files are exported to two different folders- a Maya 'asset' location and an engine 'export' location. 
  • The source PSD is saved and control is returned to the user, rather than closing the doc. There is a bug with this step I need to stomp on before I post up a code sample. 

The key areas I am experiencing problems are:

  • Saving as TGA format has lead to a couple of hurdles concerning CS2's insistence on saving the TGA as a 'copy' if there are layers present. I get around this by saving the history state, flattening the doc, exporting the current layer, then reverting to my saved history state. The downside is that post flattening, the document thinks it's meant to stay as a TGA format, complete with the export type layer name tagged on the end, even once the layers have been restored. 
  • I intend to get around this by adding a final step that saves the document with the original .psd extension in a source folder, which I am currently working on now. 
  • Optimizing the TGA export. As the export and asset TGA files are essentially the same thing, it might work out quicker to export it once to the asset folder and then copy the output over to the export folder. 

 Additional things I intend to add:

  • Intelligent support for '_a' one bit alpha textures 
  • Support for '_g' gloss map textures
  • Easily changed options for different formats 
That's all for now!


-Pete