In serra’s 3D animation course, I learned many valuable skills and knowledge, which laid a solid foundation for my future animation creation. Here are the key things I learned in the course and the results of my self-exploration:

1. Use TouchDesigner to transform videos

By learning TouchDesigner, I have mastered how to transform and process videos in a real-time environment. This powerful tool allows me to create complex visuals and apply them in live performances and interactive installations. This has not only improved my technical abilities, but also expanded my creative horizons, allowing me to express my creativity more freely.

2. Use VJ software Resolume Arena

During the course, we learned how to use the Resolume Arena in depth. This VJ software allows me to control video and audio content during live performances for real-time mixing and effects processing. I learned how to set up layers, apply effects, and work with MIDI controllers to make my performances more dynamic and visually impactful.

3. Import to UE using motion capture technology

In terms of motion capture technology, I learned how to capture character movements using the Vicon system and import that data into the Unreal Engine. This process involves cleaning and optimizing the data to make the captured actions appear more natural and fluid in the virtual environment. This skill has greatly improved my efficiency and effectiveness in character animation.

4. Sync iphone with UE by livelink

By learning how to sync my phone with Unreal Engine, I was able to use the phone’s sensors and camera for real-time control and monitoring. This allows me to achieve more immersive and interactive experiences in virtual reality and augmented reality projects, and also opens up more possibilities for my future creations.

5. Self-learning: New feature for animation in UE5.4

Outside of the course, I took the initiative to learn new features of the animation section in Unreal Engine 5.4. This includes advanced features such as the use of animation blueprints and character binding. These new features not only make my character animations more flexible and delicate, but also give me a deeper understanding of the control of the virtual character.

Summary in the end:
In general, this 3D animation course enables me to master a variety of cutting-edge animation techniques, and makes me comprehensively improve in real-time video processing, live performance, motion capture and virtual engine application. Through courses and self-learning, I not only improved my technical ability, but also broadened my creative ideas, and accumulated rich experience and inspiration for future animation creation. Thanks to the support and help of teachers and students, this learning experience will become a valuable asset in my career.

This artifact’s process in this week:

Mixamo to UE5 by mixamo_converter

By communicating with serra, you can increase the animations you look around

Add eyebrows’ bone to the girl and do the girl’s ik config:

Add animation sequence to the girl and the prince:

use animation blueprint to mix two animation sequences together in one person:

Modify the animation sequence because the existing animation sequence does not match the scene, and also add the eyebrows’ animation: [Below are some screenshots of the changes made to almost every animated sequence to fit the scene]

set camera and put the character inside the level sequence:

make the texture to the phone on the table:

Use the same principle to change the texture of the girl’s painting on the wall of the original castle. And after the prince travels back, the phone map is changed to the castle.

Rendering: [setting and export]

Compositing and Cutting video:

Introduction:

In last conversation with serra, erra suggested adding emoticons to the characters. Since the character has eyebrows, serra suggested animating them.
To achieve the expression animation, I am going to make bones for the eyebrows of the model for better control, as well as animation.
I studied a lot of tutorials and found that UE5.4, as the latest version, opens an editing plugin and you can create bones and skins in UE. Therefore, I updated to the latest version and went through the steps to achieve the animation of the character’s face (eyebrows).

Add eyebrows bone for prince in order to do some face animation with eyebrows:

edit weight for eyebrows:

key the animation for eyebrows:

I combed through the content of each shot and the required animation, and the following is my manuscript

I need the prince to make a turn when he walks, so I learned to create a new path in the blueprint through learning, and modify the path to make the prince walking animation conform to the scene I envisioned. Here is part of my process:

Add blueprint–‘spline’, use it to do the walk follow the way I want

Inside the prince’s room, I need three different shots. The first shot, you need to follow the hero from the balcony to the room, follow the upper body, but at this time try not to appear on the wall of the painting, leaving the audience a certain suspense. In the second shot, I want to focus on the animation of his facial expressions, so I will give a close-up of the prince’s face to show his movements and expressions when he looks at the painting. The third is a close-up of the photo, from the prince’s back, the camera moves up, the image appears in the frame, and zooms in quickly. So you can achieve the prince through the effect.、

set cameras:

choose a woman character‘s picture and change the texture for the frame

For the modern scene, I have added the PostProcessVolume box here, because from the Prince’s perspective, the girl’s world is linear, not three-dimensional like his own world. And the fact is, the world Prince is in is the game world, which will give the audience a surprising sense of reversal at the end.

PostProcessVolume Edit [In this use of two existing materials, after the numerical modification, a superposition is used]

set camera and edit animation sequence for the coffee scene:

key two cars animations:

make girl’s setting animation and put it in scene:

this week’s outcome:

Course Introduction:
This course is designed to teach students how to perform visually using Resolume Arena 6. Resolume Arena 6 is a powerful real-time video mixing software, widely used in live shows, stage shows, VJ (video jockey) shows and other fields. Through this course, students will learn how to create, edit, and manipulate visual content and master the skills to synchronize this content with music and other performance elements.

Course objectives:

• Master the basic functions and interface operation of Resolume Arena 6.
• Learn to create and manage video clips and layers.
• Explore the application of video effects and transformations.
• Understand how to synchronize visual effects with music and other live performance elements.

Here are the steps in the course

Download the VJ app:

https://www.resolume.com/download/files?file=Resolume_Arena_6_1_3_rev_63441_No_Footage_Installer.exe

Import the files to the VJ app:

Add sources in this app:

shortcuts–edit keyboard:

piano key:

Effects — strobe — Edit keyboard — space — piano&Invert

Edit MIDI

Content:

It’s been an amazing adventure. The hero is a prince who has always been in love with the girl of his dreams. Until one day he looked at the girl in the painting and accidentally crossed into a completely strange world. When he woke up and looked around, he miraculously found the girl in the dream, he ran towards her regardless of everything, and wanted to talk to the girl, but found that the girl did not know him, and even thought he was a strange person. When the girl gets up to get her coffee, the prince notices a strange box (mobile phone) making a noise, and after he touches it curiously, he finds himself back in his original world. The girl came back after taking the coffee to find that the boy had already disappeared, instead, the game on the mobile phone showed ’game over‘.

storyboard:

Scene determination:

1. If you want a cartoonish narrative, the model you choose may be cartoonish and concise.
2. The selected character model and scene model need to be consistent
3. Make some modifications to the downloaded model scene
4. The prince will live in a castle setting, while the girls will choose a modern setting

Download scenes – castle and coffee shop:

Here, I changed the original scene, adjusted the position of the throne, modified the standing position of the guards and the decorations behind the throne, and modified the hall of the first floor to a higher floor, so that there will be more space, so that the interior of the castle is very magnificent. At the same time, I took into account the narrative of the story and, therefore, added the blue element representing the color of the country’s flag in this scene. I changed the color of the wall around the throne to blue, which contrasted with the prince in red clothes, and more highlighted the difference between the characters and the characters’ background.

Revamp the existing scene in order to match my story:

In the modern scene, the original model building map shows a huge parking lot outside the shopping mall, which is not consistent with my preset scene, because I want the prince to wake up in a meadow, and I don’t want to quickly reflect the prince through a modern world. Therefore, I modified the map here, changed the open space in front of the supermarket into a green lawn, and generated certain plants randomly through the brush. In this way, when the prince wakes up and looks around, the audience realizes that the prince has gone to another world, so there is a sense of contrast and mystery.

In the modern scene of the cafe, I modified the original layout. Since both the prince and the girl are dressed in red, the use of red floor tiles here would be particularly crowded, so I changed the RGB of the ground map in the blueprint to make the floor show a cool color like blue. At the same time, I changed the seating from a low stool to a high chair, so that the main man and woman could sit side by side and appear in the same shot.

The above is part of the scene modification show, in addition to the prince’s bedroom and other modifications. However, the main scenes and main props have been placed above, which can be well adapted to the production of subsequent animations. After I finished the scene, I began to experiment with camera placement to see what kind of shots and angles would be more appropriate for my animation. Below is a screenshot of my shot attempt.

Setting the shots:

Course Introduction:
This course is designed to teach students how to import TouchDesigner projects into Unreal Engine 5 (UE5), combining the power of both to achieve more complex and efficient visuals. Through this course, students will learn the basic operation of TouchDesigner and UE5, as well as data transfer and synchronization techniques between the two.

In this class, I mainly learned two main knowledge points, the first is the method of importing touchdesigner in ue; The second direction is how to smoothly import motion capture animations into ue5.

Open the plug-in and use the advanced version through the link given by the teacher. This version can smoothly import the touchdesigner file into ue.

Export the touchdesigner file:

Create a random cube and import the touchdesigner video into the cube as a material, so that you can view the touchdesigner file in the file

Part two：

Part two：How to import vicon motion capture files into ue smoothly

Import FBX file: In the Content browser, right-click and select “Import to…” Options.
Navigate to the folder where you saved the FBX file and select the FBX file you want to import.
Import Settings: In the Import dialog, make sure the following options are set correctly:
Skeleton: Select the skeleton that matches the FBX file. If this is the first import, you can select “None” and a new skeleton will be created.
Animation: Make sure the “Import Animations” option is selected.
Mesh: If the imported file contains mesh data, you can also import it.

use ViconFemale_v2 to import Emily:

Import Animation sequence:

do ik rig for Vicon:

Check the skeleton:

Once the import is complete, locate the imported Skeleton in the content browser. Double click to open and check that the bone structure is correct.
Ensure that bone names and hierarchies are consistent with the model.

IK the Mannequins and Vicon:

Edit the Vicon pose and Mannequins, let both of them in the same pose:

Adjust the animation:

Find the imported Animation in the content browser. Double-click to open and check that the animation plays correctly.
The animation editor allows you to adjust the animation speed and loop Settings.

export animation sequence:

This week’s class, I came to the motion capture classroom. The teacher took us step by step to start from scratch, wearing specially made black clothes with anchor points. In front of the green screen, make sure that there are no reflections on anyone in the room, as these reflections will affect the setting of the scene.

Emily trying the Vicon to record some pose:

The teacher taught us how to set up the camera so that every anchor point of the performer could be successfully identified. Because misidentification can distort the final captured action, a lot of time was spent setting up the camera and testing the integrity of the action and the motion capture so that the subsequent imported action could be more smooth.

Set the camera: