简体中文

2024-05-24

Current

  • new :rainbow: Licenses is no longer required to use MediaPipe4U (all plugin) in the UE editor (you can still use M4U even though the license file has expireds).
  • new :rainbow: Unreal Engine version 5.4 support, I skipped UE 5.3, I’m sorry for that.
  • new :rainbow: Control Rig supported (preview). Add a new animation blueprint node ControlRigFromPose that used to convert FK poses to Control Rig Blueprints pose.
  • new :fire: Upgrade to latest goole mediapipe holistic task api, got better performace for holistic tracking.
  • new :rainbow: Add StopAsync to MediaPipeHolisticComponent to support asynchronous stop Mediapipe.
  • new :rainbow: MediaPipeHolisticComponent Display “Raw mode” support for Android platform.
  • [fix] :bug: #130: When packaging a game, the mediapipe model file is missing.
  • [fix] :bug: #124: BVH Recorder crash if skeleton bone index was not used for compact space pose.
  • [fix] :bug: The AnchorWidgetInCanvas function exceeds the size limit when scaling the frame image.
  • improve :rose: Optimize android performace, now Android platfrom can reach 15-20 fps (CPU: Qualcomm Snapdragon 870 ).
  • improve :rose: The Mediapipe4U solver is now adaptive to LOD changes and motion capture can also be done when the LOD changes.
  • [remove] :nauseated_face: LLM plugin has been removed. Because of the high latency of the LLM and the frequent updates of the llama.cpp project for a long time, there is no stable version. So I remove it, and maybe it will come back in the future.
  • [remove] :nauseated_face: Since this version, the 5.0.x version of UnrealEngine is no longer supported. Because 5.0.X requires VS 2019 and VS 2019 is no longer installed on my development PC. Sorry for that, but the older versions of the 5.0.x plugin are still available.
  • [break change] :pill: ASRCaptureComponent change name to SpeechCaptureComponent;

2023-10-10

  • new :rainbow: MediaPipe4U Mobile Android packaging support, the development process is fully compatible with Windows.
  • new :fire: Both StartCamera and StartImageSource now support truly blueprint-friendly asynchronous functions.
  • improve Optimize frame pipline, huge performance improve, now Windows CPU can reach 25 fps (CPU: AMD 3600, Frame Size: 1280 * 720).

2023-08-29

  • [fix] Fails if the file path contains characters from a Non-English language.
  • [fix] Error may occur after packaging in LLM , ASR.
  • [fix] Animation blueprint node error when open blueprint editor: #104, #101

2023-08-17

  • [fix] Fixed the issue that the speech model import interface was block (#96).
  • [fix] The llm inference process may have caused a memory leak.

2023-08-11

  • new Support warm-up when large language models are loaded, solving the problem of slow response to the first round of dialogue.
  • new LipSync supports Remap assets, and curve names don’t have to follow Facebook’s Viseme name, and can be mapped at runtime.
  • new SpeechActor adds new blueprint functions (IsTTSReady/IsTTSReady) to check if the model is loaded.
  • [imporve] SpeechActor::IsTTSSpeaking You can now accurately determine whether tts is still speaking.
  • [fix] Speech may create an invalid empty folder on disk due to encoding issues.
  • [fix] Issues fixed: #95, #96

2023-08-07

  • new :rainbow: Add LLM Speech feature (MediaPipe4ULLMSpeech plugin), integrated MediaPipe4U Speech into LLM, can automatically read the result that generated by LLM.
  • new :rainbow: Released a new ONNX voice model pack (better sound quality), adding four new voice model packs: Chinese, Chinese male voice, English, English female voice（Speech Model Release Page）.
  • new TTS inference supports Onnx engine and can get a higher quality vocoder.
  • new Add StartCameraAsync function, prevent UI block while camera starting.
  • new TTS blueprint functions added to MediaPipeSpeechActor (some functions in LocalTTSComponent no longer support access in blueprint).
  • improve TTS supports text reading queue, which can read multiple texts one by one (the SpeakText function on MediaPipeSpeechActor add the StopPrevious parameter).
  • improve Removes the pdb file to further reduce the package size.
  • improve When the NvAR SDK is not installed, packaging will no longer fail, but will give a warning message.
  • [fix] Fixed that if UE Editor’s LiveLink UI is opened when start mocap, the program will crash.
  • [fix] Fix program crash when access UI in ASR speech recognited events (#90).
  • [fix] Fixed the Pose Asset Baker tool crash on UE 5.2 (#88).
  • [fix] A small probability that TTS can not be stopped.
  • [fix] An encoding issue exists in TTS/ASR parsing folders that cause the wrong folder being created on disk.

2023-07-18

  • new Supports updating licenses in the plugin settings view （Documentation）.
  • new Supports updating the voice model in plugin settings view （Documentation）.
  • new Support for updating licenses from blueprints（Documentation）.
  • improve When using Pose Tracker annotation drawing, ROI red wireframes are no longer drawn (details: #81).
  • improve GStreamer no longer include PC (C++ package config) files when packaging project, reducing the packaging size.
  • [fix] Temporarily remove FixMediaPipe as it causes leg jumps (details: #81).