|Page (1) of 1 - 08/14/02||email article||print page|
How To Set Up an After Effects Render Farm (Single-Page Version)The cheap and easy way to get the most out of the After Effects Production Bundle
Render farms allow you to take advantage of multiple computers churning out frames of the same project all at the same time. Most of you, no doubt, have some old PCs or Macs sitting around doing nothing. So why not put them to use to save you a significant amount of time, even with low-end computers helping out?
For this example, I'm going to use a dual 1 GHz G4 as my primary system and two old 400 MHz G4 systems in the render farm, all running Mac OS X. This is not the only way to do it by any means, but I happen to have these machines handy, which is really the point: A render farm takes advantage of equipment you already have lying around, and it makes tremendous use out of these systems.
Note:This tutorial covers the network render feature of the After Effects 5.5 Production Bundle. You can network render with the standard version as well, but the process is more laborious. See your online help documentation for setting up a network render using the After Effects standard edition.
First some preliminaries. The After Effects network rendering process is not flawless. First, it has a few quirks during the initial use. Second, it renders image sequences rather than movie files, though this shouldn't prove to be a problem for anyone. (It takes about one second per 100 frames to convert an image sequence into a movie in an NLE or QuickTime Pro.) And third, the Windows and Mac OS X network Render Engines are not compatible with one another. So, if you have a mixed environment, you'll have to boot your Macs in OS 9 for the time being, until Adobe updates its software to run correctly between these two platforms. (If you have the Mac version of After Effects 5.5, it runs on both Mac OS 9 and Mac OS X without having to do any additional installations.)
Nevertheless, the Render Engine is efficient in the extreme, which more than makes up for the little oddities you might encounter at first. In fact, it's so efficient as to be almost unbelievable. But we'll get to the results at the end....
Whether you're using Macs or PCs (or combinations of the two), the setup is the same. You begin, of course, by connecting all of your computers to be used in the render farm onto a physical network. (For a look at how to network Macintosh and Windows systems together, visit http://www.macwindows.com/tutorial.html. This link will take you to the MacWindows site, which is not affiliated with DMN.)
If you already know how to network computers and mount remote volumes, you can skip ahead to the next section, "Installing the Render Engine."
Next, create a folder somewhere that is accessible by all of the systems on your network. This will be your Watch Folder. For this example, I'll be placing the Watch Folder on the Desktop of my primary system, and I'll be calling it simply "AE Watch Folder."
To make it accessible to all of the machines on my all-Mac network, I'll need to turn on File Sharing, and then I'll need to mount my primary machine's hard drive on the remote machines. No problem.
1. Turn on AppleTalk in your Network System Preferences (Mac OS X) or Chooser (Mac OS 9) on all of your systems.
2. Turn on File Sharing in your Sharing System Preferences (Mac OS X) or File Sharing Control Panel (Mac OS 9) on the system that contains your Watch Folder--in this case my primary system.
You're now sharing your disk across the network. Now you need to mount this volume on each of the machines you plan to use in the render farm. So go to each machine and do one of the following:
- In Mac OS X, go to the Finder and select the Go menu. From there, choose "Connect to Server." The volume containing the Watch Folder will appear in the list, and you just select it, and then type in your user name and password.
- In Mac OS 9, select the Chooser from the Apple Menu. Click on the AppleShare icon, and then click on the volume that contains your Watch Folder. Type in your user name and password in the dialog that appears.
Simple Mac networking. Nothing difficult about it.
Installing the Render Engine
Now you just need to install the After Effects Render Engine on all of the systems on your network. This engine is included in your After Effects installation CD. To install it, run the After Effects installer. But instead of "Easy Install," choose "Custom." When the dialog appears, select "Render Engine" and "Plug-Ins" from the list. Leave the others unchecked. (You won't need help files or Adobe Online support for the network renderers.) If you're on an old version of the Mac OS, you might need to restart the computer following the installation, which means you'll also need to remount the volume containing the Watch folder, as described above.
In addition, you will have to install any third-party plugins you're using onto the network render machines. Make certain that your license allows for additional installations or additional render-only installations. You can't use network rendering to render third-party effects unless the plugins are installed on all machines used in the network render.
Finally, you'll need to make sure that all of the computers used in the render have different names. It will also help you to assign different names to each hard drive on the network rather than using the default "Macintosh HD." To change the name of your hard drive, click on its name on the Desktop, hit return, and type a new name. To change the name of your computer, open the Sharing System Preference in Mac OS X or the File Sharing Control Panel in Mac OS 9.
Important tip:The Render Engine is a stripped-down version of After Effects whose sole purpose is to render files across a network. The main After Effects application can also be used in conjunction with the network render engines to include your primary machine in the render. But, as an excellent option, you can also install the Render Engine on your primary system in addition to the regular After Effects application. This Render Engine will take over rendering on your main system and will free up your main application so that you can keep working in it while the separately running Render Engine is doing chugging away. On a dual-processing Mac, this is particularly valuable because After Effects doesn't really take advantage of both processors, so you have plenty of CPU power to spare, despite the fact that you'll have two versions of the same program chugging away on processor-intensive tasks.
Running the Render Engine for the first time
To run the network render, you have to take a few simple steps, which will be different the first time you use it from the subsequent times you use it, owing the the quirks I've mentioned earlier. I'm going to assume this is your first time using an After Effects render farm. Even if it's not, you'll need to follow this first-time setup for each new machine on which you install the Render Engine.
First, open up your composition and work on it as you always would. Save the project wherever you want. When you're done, rather than choosing Composition > Make Movie, you'll choose Composition > Add to Render Queue. At this point, your Render Queue window will pop open, and you'll see your composition listed there. (You can work on any number of compositions in a network render queue, and they will be rendered in the order in which they have been added.)
Now there are two or three things you have to change in your comp settings in the Render Queue window. First, if one is not already specified, you have to specify a name and location in the "Output to" field. If one is already specified for you, skip this step. Otherwise, click on the "Not specified" link, and choose a name and output location. The actual location doesn't matter. Your network render settings will override the location you choose.
Next, you must specify an Output Module. To do so, click on the pull-down menu located next to the "Output Module" setting in your current comp, and choose "Multi-Machine Sequence." This is a basic multi-machine template that's included with After Effects. You can click on the link after you've selected this setting to change individual parameters. By default, the module outputs RGB plus alpha, millions of colors+ and premultiplied color. Change these settings as you see fit.
Next, you need to specify the Render Settings. Again, select the pull-down menu next to the Render Settings setting, and choose "Multi-Machine Settings." To check or change these settings, just click on the setting after you've selected it.
You're now done with the settings, so all that's left is getting your files into the Watch Folder we created earlier. You can do this but one way, and here it is. From the File menu, choose "Collect Files." You can only choose this option from a Production Bundle version of After Effects. Once you select it, you will be asked to save your project, and then you'll be presented with a dialog box.
Adjust your settings as in the picture below, with the "Enable Watch Folder" and "Change Render output to" options checked specify the maximum number of machines you want to have working on this project, and then click "Collect."
Another dialog will appear asking you to locate the folder where you wat the files collected. Choose the Watch Folder you created earlier, and click "Save."
After Effects will now collect all of your project files and move them into the Watch Folder. It will also generate the all-important files needed for the other Render Engines on the network to be able to work together on this project.
Once you've done this, go to each of the machines on your network, launch the After Effects Render Engine, and select File > Watch Folder. In the dialog that appears, choose the Watch Folder that you have already created. Repeat this on all of your machines, including your primary machine, if you want it to be included in the network render process. As mentioned above, you can also free up your main application by running the Render Engine on your primary system in addition to the main application--in other words, you can run After Effects 5.5 and the After Effects 5.5 Render Engine on the same machine at the same time.
Once you've chosen the Watch Folder in each Render Engine, including your main system--whether you do it from the main application or the Render Engine version--they will start chugging out frames to your specified Output Folder, which will be located in your Project Folder inside the Watch Folder.
Subsequent uses of the Render Engine
Of course, it would be a pain to have to go to each machine and select a Watch Folder every single time you wanted to perform a network render. Fortunately, this is a one-time process. After you select the Watch Folder manually the first time, you can just leave those machines running, and they'll automatically start rendering anytime you collect files into the Watch Folder in the future. So, essentially, after the first render, you can stuff your networked computers into a closet and forget about them (which is a particular benefit if you're using Windows PCs in your render farm).
So, for all subsequent renders, you need only add the comp to the Render Queue and use the Collect Files function (as described above) on your primary system. Everything else is automatic. Your Render Engine clients automatically return to "Search" mode when they've completed all of the available renders.
What's the benefit?
I mentioned above that using the After Effects Render Engine can boost render speeds tremendously. In fact, the Render Engine is so efficient that you will see a performance boost disproportionate to the combined processing power of the machines on your network.
Here's what I mean.
For this tutorial, I rendered two complex but fairly short projects in three different ways. First, I rendered them using only my primary dual 1 GHz G4. Then I rendered them using the primary machine plus one networked 400 MHz G4. Then I rendered them with one additional 400 MHz G4, for a total of three machines: a dual 1 GHz G4 and two 400 MHz G4s.
The dual 1 GHz machine had 1.5 GB RAM. The first 400 MHz G4 had 960 MB RAM. And the third had 448 MB RAM. They were all networked through a cheap ($75) 10/100Base-T switch, available at your local computer retailer.
Dual 1 GHz G4
Dual 1 GHz G4 + 400 MHz G4
Dual 1 GHz G4 + 400 MHz G4 + 400 MHz G4
50 minutes, 12 seconds
34 minutes, 24 seconds
26 minutes, 1 second
Project 1 consisted of two solid layers, both functioning in 3D space. Both rotated on the Y axis, but did so at different rates. The top layer included Fractal Noise, Depth of Field and Fog 3D. The bottom layer use Fractal Noise, Vector paint, Glow, Scatter, Strobe Light and Depth of Field. On top of it, I used a custom camera.
Dual 1 GHz G4
Dual 1 GHz G4 + 400 MHz G4
Dual 1 GHz G4 + 400 MHz G4 + 400 MHz G4
31 minutes, 43 seconds
22 minutes, 46 seconds
17 minutes, 34 second
Project 2 used the same footage as the previous test, as well as an adjustment layer and a still. The adjustment layer used Advanced Lightning with a high degree of complexity. The footage, sandwiched in the middle, included a Color Difference Key, a heavy Mesh Warp, Iris Wipe and Curves.
You can see from these results that this Render Engine doesn't mess around. At no extra expense to me--in other words, using only equipment I had sitting around--I cut my render time nearly in half. And these are not high-end render stations I'm using here. These are virtually stock, two-year-old G4s giving me this performance boost. At best, all things being equal, these machines combined should have given me nothing more than a 40 percent performance increase, but I guess there's something to be said for the low overhead of the standalone Render Engine.
At any rate, for these kinds of gains, you should be willing to dig through your garage for some old, disused computers to add to the network or even head over to the Goodwill and look for some used systems. Couldn't hurt.
Potential Stumbling blocks
Now, don't expect that you can keep scaling up this render farm infinitely. There are diminishing returns based on the data flow over your network. The optimal number of machines will depend on the speed of your network and, to some extent, your network equipment. For example, I'm using a network switch rather than a simple hub, which helps direct the flow of network traffic better.
Also, you might encounter some stumbling blocks in the setup. I addressed one (the initial manual setup), which is not documented anywhere else, as far as I could see. Adobe keeps another list of potential problems at http://www.adobe.com:80/support/techdocs/28372.htm.
If you do experience problems, After Effects keeps an HTML-based log of everything that happens during the network render, which is accessible in the Project Folder that's created in the Watch Folder. You can browse projects by the name of the render and the names of the computers involved in the render.
That's it. Enjoy your newfound productivity.
Contact the author: Dave Nagel is the producer of Creative Mac and Digital Media Designer; host of several World Wide User Groups, including Synthetik Studio Artist, Adobe Photoshop, Adobe InDesign, Adobe LiveMotion, Creative Mac and Digital Media Designer; and executive producer of the Digital Media Net family of publications. You can reach him at firstname.lastname@example.org.
Related Keywords:How To Set Up an After Effects Render Farm