mmd loader and runtime for Babylon.js
2024-04-17.16-32-48_1.mp4
Music: ピチカートドロップス
Model: YYB Piano dress Miku
Stage: ガラス片ドーム
Motion: https://www.nicovideo.jp/watch/sm31508557
Camera: https://www.nicovideo.jp/watch/sm36273873
Web Assembly Runtime With Integrated Physics Demo (experimental) - most advanced method
Music: メランコリ・ナイト
Model: YYB Hatsune Miku_10th
Motion / Camera: https://www.nicovideo.jp/watch/sm41164308
First, install the package.
npm add @babylonjs/core @babylonjs/havok babylon-mmd
Here is the code to build a scene with a simple MMD model and play a VMD animation.
// side effects that register the loader
import "babylon-mmd/esm/Loader/pmxLoader";
// side effects that register the animation runtime
import "babylon-mmd/esm/Runtime/Animation/mmdRuntimeCameraAnimation";
import "babylon-mmd/esm/Runtime/Animation/mmdRuntimeModelAnimation";
async function build(canvas: HTMLCanvasElement, engine: Engine): Scene {
// If you don't want full SDEF support on shadow / depth rendering, you can comment out this line as well. While using SDEF can provide similar results to MMD, it comes with a higher cost.
SdefInjector.OverrideEngineCreateEffect(engine);
const scene = new Scene(engine);
scene.ambientColor = new Color3(0.5, 0.5, 0.5);
const camera = new MmdCamera("mmdCamera", new Vector3(0, 10, 0), scene);
const hemisphericLight = new HemisphericLight("light1", new Vector3(0, 1, 0), scene);
hemisphericLight.intensity = 0.3;
hemisphericLight.specular = new Color3(0, 0, 0);
hemisphericLight.groundColor = new Color3(1, 1, 1);
const directionalLight = new DirectionalLight("DirectionalLight", new Vector3(0.5, -1, 1), scene);
directionalLight.intensity = 0.7;
CreateGround("ground1", { width: 60, height: 60, subdivisions: 2, updatable: false }, scene);
// Use havok physics engine for rigid body/joint simulation
const havokInstance = await HavokPhysics();
const havokPlugin = new HavokPlugin(true, havokInstance);
scene.enablePhysics(new Vector3(0, -9.8 * 10, 0), havokPlugin);
// MMD runtime for solving morph, append transform, IK, animation, physics
const mmdRuntime = new MmdRuntime(scene, new MmdPhysics(scene));
mmdRuntime.register(scene);
// For synced audio playback
const audioPlayer = new StreamAudioPlayer();
audioPlayer.source = "your_audio_path.mp3";
mmdRuntime.setAudioPlayer(audioPlayer);
// You can also run the animation before it loads. This will allow the audio to run first.
mmdRuntime.playAnimation();
// create a youtube-like player control
new MmdPlayerControl(scene, mmdRuntime, audioPlayer);
const vmdLoader = new VmdLoader(scene);
mmdRuntime.setCamera(camera);
const cameraMotion = await vmdLoader.loadAsync("camera_motion_1", "your_camera_motion_path.vmd");
camera.addAnimation(cameraMotion);
camera.setAnimation("camera_motion_1");
const model = await loadAssetContainerAsync("path/to/your_file.pmx", scene)
.then(result => {
result.addAllToScene();
return result.meshes[0] as MmdMesh;
});
const mmdModel = mmdRuntime.createMmdModel(model);
const modelMotion = await vmdLoader.loadAsync("model_motion_1", "your_model_motion_path.vmd");
mmdModel.addAnimation(modelMotion);
mmdModel.setAnimation("model_motion_1");
return scene;
}
If your model uses textures such as TGA, don't forget to also import the side effects for TGA textures.
// side effects for TGA textures
import "@babylonjs/core/Materials/Textures/Loaders/tgaTextureLoader";
The optimized custom formats, BPMX (BabylonPMX) and BVMD (BabylonVMD), handle most of the validation and parsing process required for loading at the conversion stage.
As a result, you can provide users with an incredibly fast loading experience.
Furthermore, it is also useful when you want to protect the original model/motion files since they cannot be loaded in MMD.
To perform pmx conversion, please visit below link and load the desired models directory. Then select the model you want to convert and click on "convert".
then you can load the converted files like below.
// side effects that register the loader
import "babylon-mmd/esm/Loader/Optimized/bpmxLoader";
const model = await loadAssetContainerAsync("path/to/your_file.bpmx", scene)
.then(result => {
result.addAllToScene();
return result.meshes[0] as MmdMesh;
});
To perform vmd conversion, please visit below link and load the desired motions. Then, simply click on "convert". By default, it is possible to merge one camera motion and one model motion together.
then you can load the converted files like below.
const bvmdLoader = new BvmdLoader(scene);
const motion = await bvmdLoader.loadAsync("motion_1", "your_motion_path.bvmd");
Please refer to the documentation for more detailed instructions on how to use it.
Parser
- PMX Parser
- PMD Parser
- VMD Parser
- VPD Parser
- Optimized Custom PMX like format
- Optimized Custom VMD like format
PMX / PMD Loader
Mesh
- Geometry load
- Build Bone / MorphTarget
Material / Shading
- Basic material parameters load
- Custom MMD toon fragment shader
- Spherical Deformation support
- WebGL 1.0 / 2.0 support
- WebGPU support
VMD / VPD Loader
- Basic animation load / custom retargeting
- Load animation using the Babylon.js
Animation
container - Babylon.js
Animation
Humanoid to MMD bone retargeting
Animation Runtime
- Basic animation load
- MMD morph system support
- Solve Append transform
- Solve IK
- Play audio / sync with animation
- Basic animation player UI
- Animation blending in custom animation runtime
- Force Humanoid Model Support
Physics Runtime
- Solve Rigid body / Joint
- WASM based runtime for threaded bullet physics and MMD IK
- PMX 2.1 support (because 2.1 spec not implemented in MMD)
- Self shadow, Ground shadow spec support
Support this project
Advanced technical support is available for those supporting the project
Contact me via email or discord
email: [email protected]
discord: noname0310
PMX (Polygon Model eXtended) 2.0, 2.1 File Format Specifications