Candece Seling

The ability to depict medical content clearly using animation in a user-friendly software environment is becoming a necessary tool in the biomedical industry. By using commercial, off-the-shelf (COTS) 3D software, designers can affordably achieve high levels of accuracy resulting in multiple-use products that meet clients' demands and deadlines. This paper discusses the use of affordable COTS 3D digital animation software to create animations in support of the development of an advanced prosthetic arm at The Johns Hopkins University Applied Physics Laboratory (JHU/APL).

Figure 1. Animation showing motor control and sensory feedback 

Figure 2 . Conceptual illustration depicting a possible neural chip implant that allows the wearer to move the prosthesis and experience sensory feedback. Photo Source: Used with permission, University of Utah

Figure 3. Illustration showing a screen shot of the Poser program depicting numerical inputs used to create exact movement.

Figure 4. Animation depicting a user navigating through the Flash-based Graphical User Interface (GUI). The navigation allows the user to view the 16 animations at the same time the DARPA specifications are shown.

Figure 5. Animation showing a sampling of all the linear simulations put together.

Illustrations and Animations
created by Candece Seling and Kevin McCafferty © 2006

Background: The Trend Toward Animation

Animation is sometimes needed to depict certain concepts that cannot be conveyed clearly through static illustrations. Partly because of popular culture, medical animation has become a biocommunicator's must-have. Television shows such as Crime Scene Investigation (CSI) and Discovery Channel Health have popularized animation in the medical community. Using commercial off-the-shelf (COTS) 3D modeling and animation software can be a cost-effective way to provide medical animations. Software applications like these have a variety of canned and editable models ready for animating and rendering. The user fills in the detail work such as texturing, lights, camera angles, and model animation. The rendering engines do the rest. It is a great way to quickly create highly effective imagery. Advanced software programs designed for more seasoned developers can take years to learn completely. By using canned models that are already rigged for animation, a novice may be able to produce high-quality digital animations in an afternoon. This is an effective solution for projects with tight budgets and even tighter schedules. Consequently, a multimedia team at The Johns Hopkins University Applied Physics Laboratory (JHU/APL) used COTS software to create 16 animations in support of a prosthetics development project.

The Defense Advanced Research Projects Agency (DARPA) recently announced a call for proposals for the Revolutionizing Prosthetics contract that aims for the creation of a next-generation prosthetic arm. The $30.4 million, four-year contract was awarded to JHU/APL in early 2006. Subject matter experts from the Army, DARPA, the Food and Drug Administration (FDA), and the National Institutes of Health (NIH) evaluated JHU/APL's proposal. The project’s goal is the ambitious development of a neurally integrated arm allowing for near-biological movement. JHU/APL will lead an interdisciplinary team of universities, government agencies, and private organizations to accomplish this vision.

At JHU/APL, thousands of engineers and scientists work to solve critical problems of national and global importance. JHU/APL is developing technologies to support Department of Defense initiatives, including saving and improving the lives of soldiers. The Biomedicine Branch of JHU/APL's National Security Technology Department (NSTD) supports initiatives in the prevention, mitigation, and recovery of wounded U.S. troops.

JHU/APL's Visual Information, Editorial, and Web Services (VIEWS) works as a team to depict highly technical concepts to government sponsors, the research community, and the public. In preparing the proposal, the VIEWS team needed to produce clear, comprehensible animations quickly; the animations needed to represent JHU/APL's design concept for a prosthetic arm with the advanced capabilities specified by DARPA.

The Need for Prosthetics

Due to advances in body armor technology, soldiers are surviving wounds that would have been fatal in the past. Injuries from Improvised Explosive Devices (IEDs) have changed the nature of soldiers' wounds, causing more extremity injuries that result in amputations. A New England Journal of Medicine study (Gawande 2004) indicates that the percentage of victims suffering limb amputation is twice that of previous wars: "Blast injuries are also producing an unprecedented burden of what orthopedists term mangled extremities — limbs with severe soft-tissue, bone, and often vascular injuries.... Whether or not to amputate is one of the most difficult decisions in orthopedic surgery."

Advancing the State of Prosthetic Technology

The most advanced prostheses in existence have a maximum of 6 degrees of movement. However, they can only accomplish one movement at a time. The DARPA contract aims to achieve 22 degrees of movement, nearly that of a natural arm. The controls must fit within the size constraints of a natural human arm, and the weight of the arm is limited to 8 lbs. The weight limit merits further consideration because the arm will not be attached to, and supported by the skeletal system. The prosthesis will also incorporate sensory feedback and spatial location technologies. Neural integration will enable wearers to to move the arm and differentiate sensation if they touch something hot or cold (Figures 1 and 2). Imagine what it must mean to the wearer of the prosthesis to be able to feel the warmth of a loved one while holding hands. This is the type of holistic and all-encompassing recovery that DARPA and JHU/APL are striving to achieve for amputees. As Army Col. Geoffrey S.F. Ling, manager of the Revolutionizing Prosthetics program for DARPA, has said, "What's available commercially is woefully inadequate. We also set the bar really high. We want to give them back their lives."

The Need for Animation

To respond to the DARPA Request for Proposal, JHU/APL had to submit a proposal that included a technical document accompanied by descriptive and explanatory visuals. JHU/APL needed the clarity and comprehensiveness of animations to help win the contract.

The researchers requested the advice of multimedia professionals to communicate their design concepts effectively. They realized that the concepts they were trying to convey could remain ambiguous if only explained verbally. Therefore, the simulation of movement would be necessary to make those concepts lucid. If a single picture is worth a thousand words, then an animation at 30 frames per second is worth 30,000 words per second. Thus, the biomedical team wanted a series of animations to help DARPA visualize the JHU/APL scientists' proposed conceptual design for the 22 degrees of precision arm movements the prosthesis wearer could achieve.

Phase 1: Evaluation of Requirements

The visualization process began with a meeting with the biomedical team to discuss the specific visual challenge. To make their proposal stand out professionally, the team wanted powerful, modern medical animations that have the precision and punch of those aired on health-related television shows. In addition, the animations had to accurately display the precise arm movements DARPA specified. The biomedical team and the VIEWS graphics team evaluated each of the requirements to determine how the 3D human model could depict the specified movements realistically.

After assessing the challenge, the VIEWS team realized that we could not use video for this proposal because the upper-extremity prosthesis has not yet been created. Showing a highly detailed design would be misleading as to the final appearance of the prosthesis. The most appropriate solution—and the best return on investment—was to create animations using 3D digital models. Once we created the digital models, it would be easy to re-render the scene using different camera angles, allowing the models to be viewed from any desired perspective. Reusing the models in varying scenarios would save time and money both during the proposal preparation and in reporting future development milestones. The Revolutionizing Prosthetics contract is a four-year project, which will require milestone briefings to the sponsor. As the technology advances and discoveries are made, the project will continually need updated visuals. Taking the time to create reusable 3D models at the outset will afford the highest efficiency and keep production time low.

In order to produce the 16 animations, we needed to define the requirements and scope of the project. The first task was to create a 3D digital model of a soldier's arm. Second, we needed to animate the arm using specifications DARPA provided: We also had to input specific numeric values to recreate the percentage of arm movement. This was going to be the most difficult task. Third, we had to deliver the animations through a custom Graphical User Interface (GUI) that can be viewed on multiple platforms. The GUI needed to be user-friendly and show animations and written specifications on the same screen at the same time. Finally, the project had a tight budget and needed to be completed quickly.

Phase II: Creating the 3D Digital Human Model

After defining the requirements, we reviewed our software options. The tools available to us included: Adobe Creative Suite™, After Effects™, and Flash™; Autodesk Maya™; Curious Labs™ and eFrontier Poser™; along with various other image editing applications.

We considered creating the 3D digital model from scratch using Maya. Considering the schedule, we decided that this would be too time consuming. Animating it would also be difficult because the human digital model would need to be rigged (set-up) for motion which can be time consuming at the novice level. We researched the possibility of purchasing a 3D digital model but it proved to be too costly.

We decided to review the parameters and let the project determine the tool rather than the other way around. We investigated off-the-shelf software in our possession and determined that using Poser's digital human model was a practical solution. Poser is an affordable, packaged software application that includes ready-made 3D digital character models. The models are already rigged with inverse kinematics for movement, which would save valuable animating time. After considering the options, we determined that by using Poser we could manipulate and animate its pre-built human model, thereby meeting the client's tight deadline and keeping costs low. Once we decided to use Poser, we started production. We chose the most suitable male model the software offered and started the process of editing his clothes to resemble a soldier's camouflage uniform.

Phase III: Animating the Model

When the model was put together in camouflage fatigues, it was time to keyframe the arm and hand movements for the animations. Keyframing is a process of choosing a particular body part, adding a keyframe (or reference point) at the beginning of a particular movement's timeline, and inserting another keyframe at the desired end position. On the latter keyframe, we chose a specific joint (the elbow) and entered a numerical percentage to instruct the software to move the arm to the exact position needed (Figure 3). Once the final position was input, the software performed a process called "in-betweening" or "tweening," which created a smooth movement from one position to the next. After analysing the degree per second specification we knew where to put the final keyframe. Each movement was numerically entered to ensure precision. The biomedical team members were comfortable with the algorithms in the software articulating the percentage of movement.

Phase IV: Production of Interactive Deliverable

All 16 animations had to be easily navigable, and the GUI had to display the scientific specifications while the animation was running (Figure 4). We decided that Macromedia Flash was the best option available to us because we could author interactive content rich with video and animation. Flash is recognized as an industry standard for creating digital experiences.

The multimedia team recommended exporting the animation to a stand-alone Flash player to allow the client to view the animations from a CD on Macintosh or PC platforms. This approach afforded the versatility we were aiming for because the CDs had to be presented with the written proposal and delivered to many people outside the JHU/APL organization. Knowing this is a four-year contract, we chose Flash for the GUI due to its ability to repurpose. We can export the animations for the Web; as Shockwave Flash (swf) files that can be placed into PowerPoint™ with the use of third-party software; and as a single, continuously running animation.

Once the proposal was accepted and JHU/APL announced the launch of the project, the media began to request the animation for use in television news broadcasts. CBS News wanted to show all of the movements in a smooth sequence. We were able to combine all 16 animations into a single, linear animation CBS could include in a video feature about the Revolutionizing Prosthetics Project (Figure 5).

Conclusion

Our decision to use inexpensive COTS 3D software not only made it possible to create our animations quickly but also allowed us to create another animation to be included in the PBS Science Investigators broadcast in 2007. This animation utilizes the same software and procedure we used to create the previous 16 animations. Once again the deadline was incredibly short. There was no time to learn new techniques. Repurposing the material economically promises to provide an excellent return on investment for many years to come as the Revolutionizing Prosthetics Project progresses toward its completion in 2009.

Reference

Gawande, A. 2004. "Casualties of War -- Military Care for the Wounded From Iraq and Afghanistan." N Engl J Med 351:2471–2475, No. 24.

Bibliography

CRS Report for Congress, United States Military Casualty Statistics: Operation Iraqi Freedom and Operation Enduring Freedom. Hannah Fischer, June 8, 2006. Order Code RS22452. http://www.fas.org/sgp/crs/natsec/RS22452.pdf
    
U.S. Department of Defense Casualty Status for Operation Iraqi Freedom (OIF) and Operation Enduring Freedom (OEF) [http://www.defenselink.mil/news/casualty.pdf]

Author
Candece Seling graduated with a BFA in Scientific Illustration from the University of Georgia and is a biomedical illustrator and graphic artist for The Johns Hopkins University Applied Physics Laboratory. She is on the Revolutionizing Prosthetics team and specializes in multimedia communications. Candece has done illustrations for NASA and the University of North Carolina Veterinary Hospital. She is also pursuing a MFA in Integrated Design at the University of Baltimore.
Email: candece.seling@jhuapl.edu