Pre-production
Following are the steps which I studied and undertook to reach the desired result.
- Setup
- Capture
- Assigning digital data to Actor/skeleton to Character
- Conversion of Captured data in digital format
- Retweeting and renders
- Conversion of Captured data in skeleton
- Calibration
Setup
Markers
Markers are made from retro reflective tape, when light hits an object it bounces off to different angle but when u deal with retro reflective tapes light bounce straight back to the camera. Retro reflective tape is used in road signs, safety gear or safety jackets, it reflects back the light and tells the actual position or direction by reflecting light. Marker placement on an actor’s body have specific reason, it defines the joint placement and the moment. Some rules to follow on how many markers and the placement of the markers on the actor. Markers gets tracked with tracking software and are send to 3d software where character gets assigned, animation gets weird moment is the markers are not placed on right position. Certain template is used for all the actors to follow the marker names. The software which detects the markers fill out one template which helps to build a human stricture. The information from the cameras is fed into special unit which is design to take the analogue signal to digital signal and then send it to computer, computer takes the data from cameras and reads in digital format which the converter does collected in one box which gets analogue signals and which are converted to digital signals which are recorded into computers.
Cameras
Different cameras are used for different techniques of Motion Capture. Performance capture studio uses cameras with different technology. Inside the camera there is a filter which only allows a certain wavelength of light to actually pass thru. This light is reflected back from the reflective markers.
Cameras with depth is also used for motion capture. It creates an infrared depth around the character which helps to track the character. Need more than 2 cameras to get 3 dimensional data.
Stage
Set up cameras on walls and point it to the center of the room. More number of cameras more accurate the data you get. Set up all the markers on the ground to get the size of the area all the camera will cover. Set up a stand of markers which denotes the height. Calculate the safe volume from all the cameras. Check if you see all the points from the camera view. Calculate the length of the safe area and mark it on the ground considering one center point. It takes days or couple of weeks to setup the stage according to the area.
If the characters motion is very fast where the markers doesn’t get detected or if the character is hiding behind a prop object the markers are not visible. In such case the markers should be rearranged to define the skeleton. Only biggest limitation for doing motion capture is area. It is very important to know the anatomy of what u are capturing. If capturing a creature like dragon or bird or very important for an actor to understand its anatomy. More and more rehearsals should be done by the actors before shooting as much time and money is involved in this process. Every morning calibrate of camera should be done to get perfect motion capture. Most of the films, animation and the games uses prop for motion capture, in such case the props should be rigged with markers to get detected in 3d software. It becomes very easy for animators that props are also motion captured.
Calibration
Calibration is representing stage, camera and characters in 3D world. For different methods have different process of calibration. System Calibration and Subject Calibration are the two most important things to do. Generally a calibration stand and a calibration wand is used to define the axis and the height of the 3d space. In a performance capture studio cameras are pointing toward the actor, these cameras should cover the entire working area. Calibration is done correctly when all the cameras are pointing at one axis (XYZ).
Capture
Motion Capture system takes patience and experience. The process will not be quick until you’ve done it several times and are used to how it works. Motion capture never works automatically. However,
With enough planning, even the first time you work all the way through a motion capture project, You will have a clear idea about how you get from one end to the other.
Pre-production is one of the most important process in Films, Animation or Games. This part of process allows you to break everything in parts and prepares you before going on any further step. Preproduction always start with jotting down ideas on paper. It is a process by which client can get a bigger picture of what the final output, before actually creating everything in CG. Preproduction gives you is a roadmap to the production.
Script is a Narration of a story which is written down on paper to better understanding. Scripts is used in all the aspects of Films and Animation, it is also used in games.
Storyboard is to narrate each and every shot by drawing in in a thumbnail box with details. This decides all the aspects, such as Camera work, shot length, shot size, moment, dialog, narration till sound. Each and every sequence is broken down into scenes and every scene is further segregated into shots..
Animatic is pre visualization of entire animation which is animated version of storyboard with sound. Length of the shot is also decided in animatic. Camera moments are added to animatic which decides the cinematography
To understand capture session, pre-production is important. Capture session is recoding all the animation. All the capture session should start with Tpose, this is how 3d software will understand and can connect actor to character. Capture session doesn’t take most of the production time.
All the actors should rehears well before capturing session. It is important for actors to understand the process of motion capture system, markers and the area of acting.
Â
Conversion of Captured data in digital format
Data can be converted into different formats depending upon the software. Following are the data types which software understands.
TRC (track row column) file format has information about all the markers and has detailed information about the session and subject. The first three lines of the .trc file is header which denotes file path (from where the data is stored), frame rate (data rate in this case 60 frames per second), frame number (indicates total number of frames with starting frame and end frame) and the number of markers (total number of markers used to shoot the character). Then its followed by two rows and column which contains all the joints and coordinates (name of all the markers and their X’s y’s and z’s to avoid confusion it is noted as x1y1z1,x2y2z2 etc.) Then one blank row. Followed by a row data containing frame number and a column data containing all the coordinates’ data of all the markers. In short .trc file is the data collected from all the cameras and stored in one file.
C3D (coordinate 3 dimensional) file format holds data of the markers and associated parameters within a single file. In a C3D file the first record is used as a header, and the data section contains 3D point coordinates and analog channel data in a specified format. Each point coordinate is stored as a 16-bit integer or 32-bit float. The data are stored sequentially by frames, and within a frame they are stored sequentially by point number, followed by the analog channel data.
The ability to store 3D positional and analog data in both processed and unprocessed form.
Store subject information e.g. ID, age at trial, with physical parameters such as weight, leg length etc.
BVH file format was originally developed by Bio vision, the name BVH stands for Bio vision hierarchical data. As a way to provide skeleton hierarchy information in addition to the motion data. BHV file is nothing but a skeleton which has all the motion capture data.
CSM format is an ASCII file that is used to import positional marker data from various motion capture systems into Character Studio to animate bipedal characters. This has all the motion capture data similar to c3d and trc file format. Biped provides direct input of CSM files from disk, including comprehensive key frame reduction, footstep extraction, and talent figure and pose calibration to provide a fast and accurate means of import for large volumes of positional data stored in the CSM format. CSM marker values are time-varying xyz locations which are typically generated by reflective markers and optical motion capture systems. These xyz values may also be derived from other motion capture sensing devices that generate positional data in a world-coordinate system. CMS file supports specific name format. The markers which gets imported in max with the specific name format
Similarly there are other formats like ASF, BIP, and AOA etc.
Conversion of Captured data in skeleton
Characterization of a character in motion builder (create Rig)
Characterization is a process which defines the character which can be readable in all the software’s. Characterization will define the joints of the character. For character should always be in T-pose. Basic character requirement for retargeting Motion capture data.
Rigging requirement for motion capture: The rig should be very much simple FK joints which is skinned to the character model, and have certain number of joints which are required. As all the motion data is directly assigned to the joints. Character should be in T-pose to start with. Motion builder has ability to capture motion capture data. Always export Maya character in .fbx format because that is the format which motion builder in very familiarized. In .fbx file both the joints and the mesh gets exported, if we have camera or lights in the scene that also adds
Up in .fbx file. In motion builder .fbx file format has a wonderful option of selecting the things which are needed to select while exporting a fbx file. A motion capture file can have multiple takes in one .fbx files, which can be selected while importing it in motion builder. Process once the data is applied on 3d character
Retweeting and Renders
It is very easy to load the animation in 3d software but it is not exactly the way you want it. There may be animation flaws due to an error due to recording process or due to retargeting animation. When motion capture data is transferred on rigged character, all the joints gets key, it is usually one per frame. When you have millions of key frames it becomes difficult to refine animation. In such case Animation layers is used, Animation layers is similar to layering system in another software. A new layer gets created on top of existing layer preserving all the keys, Animation can be tweaked in graph editor and dope sheet. In graph editor each and every key can be manipulated and tweaked and in dope sheet. It gives us an overview of the animation of several objects at once. Spline curve in graph editor should be smooth and shouldn’t have any rumbles. Rumbles in curves means jerks. Self-penetration of the character and sliding of the foot are the most common corrections which is done in cleaning the MoCap data. Once all the motion is applied to 3D characters, it is time to look at the motion by either loading everything into the game engine or rendering all the frames using the camera that has been set up in a 3D application based on cinematographic decisions made in the preproduction stage. For this production pipeline should be finalized with all the camera angles
