Photo-realism is one of the ultimate goals for many CG artists, and it's also one of the most difficult to achieve. Even if you're relatively new to 3D computer graphics however, today's tools and workflow techniques make photo-realism very obtainable. Here are eight techniques to help you get there:
1. Bevel, Bevel, Bevel: Forgetting to bevel or chamfer edges is one of the most common errors committed by beginning 3D artists. There are almost no razor sharp edges in nature, and even most man-made objects have a slight roundness where two opposing surfaces meet. Beveling helps bring out detail, and really sells the realism of your model by allowing edges to properly catch highlights from your lighting solution. Using the bevel (or chamfer tool in 3ds Max) is one of the first things you should learn as a modeler. If you're new enough to 3D that you're unsure how to create a beveled edge, chances are you could truly benefit from a good introductory tutorial, or even a training subscription.
2. Learn to Use Linear Workflow: Even though linear workflow has been around for years, it's still a confusing and complicated idea for beginners. I'm won't try to completely explain the theory here (there's just too much to say), but I do want to make sure you're at least aware that these techniques exist. The need for linear workflow essentially comes down to the fact that your monitor displays images in a different color space (sRGB) than what is output by your render engine (linear). In order to combat this, artists must take the necessary steps to apply gamma correction to a render.
But linear workflow actually goes pretty far beyond simple gamma corrections—it's all about eschewing old techniques and workarounds (most of which are based on outdated math), and moving toward true physically based lighting solutions.
There's a lot more to say on linear workflow, and thankfully it's been discussed exhaustively over the past few years. Here's a link that I found useful when learning the theory behind the process—he links out to quite a few sources, so there's plenty of reading to be done. The second link is a Digital Tutors course that deals specifically with linear workflow in Maya 2012.
Linear Workflow and Gamma
Linear Workflow in Maya 2012
3. Use IES Light Profiles for Photometric Lighting: Alongside the rise of linear workflow, 3D artists (especially those working in architectural visualization) have begun using files called IES light profiles to more realistically mimic real world lighting. IES profiles were originally created by manufacturers like General Electric as a way to digitally quantify photometric lighting data. Because IES light profiles contain accurate photometric information regarding light shape, luminance, and falloff, 3D developers have seized the opportunity to add IES support in most major 3D packages. Why spend hours trying to mimic real-world lighting when you can use an IES profile and have the real thing? CG Arena has a nice article with some great pictures to give you an idea what an IES light profile looks like:
Understanding IES Lights
4. Use Depth of Field: Depth of field (blurred background) effects are one of the easiest ways to increase the realism of your renders, because it's something we associate closely with real life photography. Using a shallow depth of field helps isolate your subject, and can improve your composition by leaps and bounds when it's uses in appropriate situations. Depth effects can be calculated at render time from within your 3D package, or applied in post-production using a z-depth pass and lens blur in Photoshop. Applying the effect in post is by far the quicker route, however setting up depth of field within your primary app gives you more control over the effect.
5. Add Chromatic Abberation: I know the name sounds complicated, but adding chromatic abberation to your renders is probably the easiest technique on this list. Chromatic abberation occurs in real-world photography when a lens fails to render all color channels at the same convergence point. The phenomenon is manifest as "color fringing," where high contrast edges show a subtle red or blue outline. Because chromatic abberation does not naturally occur in CG lighting, 3D artists have developed ways to fake the phenomenon by offsetting the red and blue channel of a render by a pixel or two in Photoshop Chromatic abberation can add realism to a render, but it can also detract from one when the effect is overdone. Don't be afraid to try it out, but remember that subtlety is your best friend. As I said, chromatic abberation is pretty darn easy to apply and Digital Tutors has a free two minute tutorial to show you how:
Visual Guide to Chromatic Abberation
6. Use Specular Maps: Most artists learn to use specular maps pretty early on, but it definitely warrants a mention for anyone who's not already on board. Specular maps tell you render engine which parts of your model should have high specularity (glossiness) and which should be more diffuse. Using specular maps increases realism, because let's face it—most objects in nature don't display uniform glossiness, but when you leave the specular map off, that's exactly how your model will render. Even for objects that do have relatively uniform glossiness (glazed ceramics, polished metal) you should still use a spec map to help bring out surface irregularities from scratches, dings, and dents.
7. Grunge it Up: You don't see the "error of perfection" as much as you did in the early days of CG, but for those of you who need a reminder: don't be afraid to add some dirt and grit to your models and textures. Most real world objects aren't clean and pristine, so leaving your models that way can come off as lazy and will almost certainly undermine your quest for photo-realism. It doesn't just have to be textural details either—try adding large scale cracks and destruction to some of your models, especially if you're working on FPS style game environments. Keep the idea of non-perfection in mind when you're populating your scenes too. Unless you're going for a very polished architectural showroom type render, scatter some props naturally throughout your scene to make the space look lived in.
8. Add Asymetry: The ability to turn on symmetry when modeling or sculpting a character is a great luxury—it means that as modelers we only have to do half the work and never have to worry ourselves over one eye being bigger than the other, or making sure the left cheekbone lines up with the right one (you know, those pesky problems that trouble traditional painters and sculptors). But when it comes time to do a final detail pass and pose your model, it's a great idea to turn off symmetry and add some sort of asymmetric variance to your character. Whether it's in the pose, costume, or textural detail, asymmetry will make your models more lifelike, and chances are you'll end up with a more dynamic and successful final image.