English | 简体中文

ComfyUI-MieNodes is a plugin for the ComfyUI ecosystem, offering a series of utility nodes designed to simplify workflows and enhance efficiency.

Currently, the following services are supported:

• ZhiPu 智谱 — Open Platform (zhipu key) + Coding / Token Plan (zhipu_code key)
• SiliconFlow 硅基流动 (siliconflow key)
• Kimi 月之暗面 (kimi key)
• DeepSeek (deepseek key)
• Gemini — multimodal-aware (image input is forwarded as inline data) (gemini key)
• Bailian 阿里云百炼 (bailian key)
• Bailian 阿里云百炼 Token Plan (sk-sp- key, bailian_token_plan) — Credits-based subscription (个人版, cn-beijing endpoint) covering the Qwen3.8/3.7 flagships plus DeepSeek, GLM and multimodal families; the key rotates when the plan is re-purchased
• Bailian 阿里云百炼 Coding Plan (sk-sp- key scoped to the coding host, bailian_coding) — programming subscription covering both the qwen3-coder family and general chat models (qwen3.7-plus, glm-5, MiniMax-M2.5, kimi-k2.5)
• MiniMax — standard Open Platform (sk-... key, minimax_open_platform) + Token Plan / Coding Plan (sk-cp-... key, minimax)
• Xiaomi MiMo — standard Open Platform (sk-... key, mimo) + Token Plan / Coding Plan (tp-... key, mimo_token_plan)
• Ollama (local) — SetOllamaLLMServiceConnector, no API key required (the Authorization header is sent but ignored), ollama config_key (key is usually left empty)
• Any OpenAI-compatible endpoint via SetGeneralLLMServiceConnector (custom base URL + model, openai_compatible key)
• Volcano Ark Doubao (doubao key; doubao-seed-* model ids or ep- endpoint ids)
• Baidu Qianfan ERNIE (qianfan key)
• iFLYTEK Spark (spark key = APIPassword, X2 /x2/ endpoint)
• OpenAI (openai key)
• xAI Grok (grok key)
• OpenRouter (openrouter key; one key routes 400+ vendor models)
• Anthropic Claude (anthropic key, official OpenAI-compat layer)
• SiliconFlow Jev / System One decision models (reuse the siliconflow key via SetSiliconFlowJevLLMServiceConnector + /v1/systemone; pair with the CallJevDecision node for structured routing — not a chat model)
• LM Studio and other local OpenAI-compatible servers (use the "General OpenAI-compatible" node, e.g. base URL http://127.0.0.1:1234/v1/chat/completions)

If you wish to use other large language model (LLM) services that cannot be connected through SetGeneralLLMServiceConnector, please submit an issue or a pull request for feedback.

Several providers (notably ZhiPu and SiliconFlow) offer free-tier models. The free lineup changes often, so check the provider's website for what is currently free and enter the model id into the connector's custom_model field.

This workflow demonstrates how to connect to an LLM service and generate a response based on a given prompt, you can use any LLM service supported.

This workflow demonstrates how to load an image, generate a detailed description using the Florence2 model, and compose context prompts with the help of large language models. The results are displayed both in English and Chinese using the Show Anything node.

• Image Loader: Loads the input image.
• Florence2 Model Loader & Describe Image: Generates a detailed caption for the image.
• Set SiliconFlow LLM Service Connector & Kontext Prompt Generator: Uses LLM to create context-aware prompts.
• Show Anything: Displays the generated result.

This workflow checks the connection to the LLM service and displays the result.

This workflow demonstrates how to add and remove a custom preset.

This workflow focuses on generating an artistic prompt for an ethereal digital portrait using the LLM connector and advanced prompt generator nodes.

• Prompt Generator: Produces detailed prompts for creative tasks.
• Show Anything: Displays the generated result.

This workflow routes a user prompt through the Bernini task-aware prompt enhancer (bytedance/Bernini, Apache 2.0) and writes an enriched, task-shaped prompt back into the graph. The 13 task types supported in the dropdown are:

• t2i / t2v — text-to-image / text-to-video
• i2i / i2v — image-to-image editing / image-to-video
• r2i / r2v — subject-driven image / video generation (reference image of a subject)
• ri2i — reference-image-guided image editing (source image + reference image; MieNodes extension)
• v2v / mv2v — video-to-video editing / multi-source v2v
• vi2v — video editing with a reference image
• rv2v / vrc2v — reference-guided video editing
• ads2v — ads insertion in video

Reference images and source video frames are forwarded to the LLM as image_url content parts, so the model can see what it's rewriting about. The dropdown labels are bilingual (code + 中文) and stay readable in either language.

The plugin provides a suite of nodes for prompt enhancement, offering:

1. A collection of preset workflows that leverage large language models to automatically generate high-quality prompts from image and text inputs.
2. Advanced prompt optimization, including automatic translation and enrichment of details, enabling richer, more expressive outputs for various creative tasks.
3. MiniMax H3 Storyboard / Loop Plan nodes — MiniMaxH3StoryboardGenerator turns a concept + target shot count into a structured storyboard (beats, shot types, camera moves, transitions, durations); MiniMaxH3LoopPromptGenerator turns that storyboard into a plan_json that plugs straight into the ComfyUI-MiniMaxH3-Context-Loop Production Plan node (plan_json_input), writing per-clip H3 prompts with explicit chain-continuation rules and a shared prompt_prefix. The Loop Plan node also accepts an IMAGE batch socket — wire the same LoadImage outputs once into both nodes, the node captions each image and routes them to the right upstream picture sink (i2va uses only the first; fl2va cycles 1..N; ref2va activates all every scene). The per-scene transition (e.g. "从图一到图二" / "A→B→A→B") is written in user_input.

The H3 storyboard / loop pipeline has four units, named from the upstream ComfyUI-MiniMaxH3-Context-Loop project (docs/README.md + H3_CHAIN_FORMAT_GUIDE.md).

• Storyboard shot — planning unit. One JSON entry from MiniMaxH3StoryboardGenerator (shot_type, camera_movement, transition_in, duration_seconds, narrative_beat, characters, props, notes).
• Plan entry (plan_json["shots"][i]) — execution unit. One entry = one Context-Loop generation. This repo enumerates them as clip_index / clip_count (legacy variable names, kept for diff stability). Default ids look like scene_01.
• Upstream Scene — H3_CHAIN_FORMAT_GUIDE.md: "one planned H3 generation." Each plan entry corresponds to one Scene.
• Upstream Segment — same source: "the delivered media saved for one scene." On disk: segments/clip_NNNN.<id>.mp4.

Vocabulary notes (not units)

• prompt_prefix is not an upstream unit. None of the six official example_workflows/ plan_json strings emit a top-level prompt_prefix field; the H3_CHAIN_FORMAT_GUIDE.md music-video template shows one, but the official examples chose to put the same identity / style content inside each shot's prompt so each scene is self-contained. In this repo, prompt_prefix is an extension field introduced by MiniMaxH3LoopPromptGenerator to share whole-video style across scenes — it gets prepended to every scene's prompt by the generator, but it is not a vocabulary word in upstream plans.
• @-aliases (@courier_arrival, @greenhouse_delivery) are not generated by this node. They live upstream in Project Asset Carousel / Ref2V Tagged Conditioning. Our pipeline emits bare <Picture N> / <Subject N> only.
• Intra-Scene [Shot 1] — loop-pipeline section opener, not a cut. The multi-cut [Shot N] At 00:03.500 form is H3-model-native and lives only in prompts/h3/system_t2v.txt (standalone T2V); do not mix.
• MiniMax-H3 model caps — Output duration | 4–15 seconds per MiniMax-AI/MiniMax-H3/README.md; H3-Max 5–15 s per the platform API. Our MiniMaxH3LoopPromptGenerator caps every scene at 14 s.

Files in nodes/llm/prompts/h3_loop/ still carry the legacy shot_*.txt prefix; that prefix is historical and refers to the Scene-level unit. Do not rename without updating _SHOT_USER_TEMPLATE in minimax_h3_loop_prompts.py.

The plugin provides the following utility nodes, with a focus on dataset file management tasks in LoRA training workflows:

1. Batch edit caption files (Insert/Append/Replace operations).
2. Batch rename files, add prefixes, and format file numbering for specific file types.
3. Synchronize image and caption files, with support for automatically creating or removing .txt files to match image files.
4. Batch read caption files, with support for extracting all file contents for analysis and summarization by large language models.
5. Batch convert image files, enabling conversion of all image files to a specified format (.jpg or .png).
6. Batch delete files with the specified extension and optional prefix.
7. Remove duplicated (same content) image files in the specified directory.
8. Save any data as a file in TOML, JSON, or TXT format.
9. Compare two files (in TOML or JSON format) and return the differences.

The plugin also provides utility nodes for general-purpose tasks:

1. Display any input as a string.
2. Download files from huggingface, hf-mirror, github or anywhere to models folder.
3. Connect to large language models (LLMs) for advanced prompt generation, enhancement, translation, and style adaptation.
4. Translate any text to another language (English, Chinese, etc.).

Function: Batch rename files and add a prefix and numbering.
Parameters:

• directory (str): Path to the directory.
• file_extension (str): File extension to operate on (e.g., .jpg, .txt).
• numbering_format (str): Numbering format (### means three digits).
• update_caption_as_well (bool): Whether to also rename .txt files with the same name.
• prefix (str, optional): Prefix to add to the file name.

Function: Batch delete files with the specified extension and optional prefix.
Parameters:

• directory (str): Path to the directory.
• file_extension (str): File extension to delete (e.g., .jpg, .txt).
• prefix (str, optional): Prefix to check before deleting files.

Before:

After:

Function: Perform operations on text files (Insert, Append, Replace, or Remove).
Parameters:

• directory (str): Path to the directory.
• operation (str): Type of operation (insert, append, replace, remove).
• file_extension (str, optional): File extension to operate on (e.g., .txt).
• target_text (str, optional): Text to replace or remove (only used for Replace or Remove operations).
• new_text (str, optional): New content to insert, append, or replace.

Before:

After:

Function: Add caption files (.txt files with the same name) for image files in a directory.
Parameters:

• directory (str): Path to the directory.
• caption_content (str): Content to populate in the caption file (e.g., "nazha,").

Before:

After:

Function: Summarize the content of all text files in a directory.
Parameters:

• directory (str): Path to the directory.
• add_separator (bool): Whether to add a separator between file contents.
• save_to_file (bool): Whether to save the summarized content to a file.
• file_extension (str, optional): File extension to operate on (e.g., .txt).
• summary_file_name (str, optional): Name of the file to save the summary.

Function: Convert all images in a specified directory to the target format.
Parameters:

• directory (str): Path to the directory.
• target_format (str): Target image format (jpg or png).
• save_original (bool): Whether to retain the original files after conversion.

Before:

After:

Function: Remove duplicate image files in a specific directory.
Parameters:

• directory (str): Path to the directory.
• max_distance_threshold (int): Maximum Hamming distance threshold for identifying duplicates.

Before:

After:

Function: Print the input content as a string.
Parameters:

• anything (*): The input content to display.

Function: Save data to a file in either TOML, JSON, or TXT format.
Parameters:

• data (*): The data to save.
• directory (str): The directory to save the file in.
• file_name (str): The name of the output file.
• save_format (str): The format to save the data in ("json", "toml", or "txt").

Function: Compare two files and return the differences. Parameter:

• file1_path (str): The path to the first file.
• file2_path (str): The path to the second file.
• file_format (str): The format of the files ("json" or "toml").

Function: Format a Python str.format-style template with autogrowing positional value inputs (Bernini Conditioning style UX).
Parameters:

• template (str): The format string. Supports {0}, {1}, ... positional placeholders and standard format specs (e.g. {0:>5}, {0:.2f}). Use {{ / }} to emit a literal brace.
• value_0 ... value_15 (str, optional): The positional arguments, one per slot. The first 2 slots (value_0, value_1) are visible when the node is dropped; the next slot appears automatically each time the last visible slot gets a connection, up to 16. Unconnected slots are treated as the empty string.
  Output:
• result (str): The formatted string. On a malformed template, the raw template is returned and a diagnostic is logged so the user can still see it on the wire.
  Example: template "{0} + {1} = {2}" with value_0="1", value_1="2", value_2="3" produces "1 + 2 = 3".

Function: Hash a STRING into a short hex digest for use in cache-key filenames. Companion to ImageHash|Mie for inputs that are already STRING (e.g. a SAM3 prompt word); keeps the prompt-derived component of a cache key short, deterministic, and free of filename-hostile characters (spaces, slashes).
Parameters:

• text (str, required): The STRING to hash. None / empty yields a constant sentinel ("0" * length) so an unconnected input still produces a valid filename component rather than crashing.
• length (int, optional): Number of hex chars to keep. Default 12, range [4, 64]. Values outside the range are clamped.
  Output:
• hash (str): Lowercase hex of sha256(text.encode("utf-8"))[:length].
  Example: text="human" (default length 12) returns "79a5478768d2".

Function: Download files from Hugging Face, hf-mirror, GitHub, or any other source to the models folder. Parameters:

• url (str): The URL of the file to download.
• save_path (str): The path to save the downloaded file.
• override (bool): Whether to override the existing file if it already exists.
• use_hf_mirror (bool): Whether to use the Hugging Face mirror URL.
• rename_to (str, optional): The new name for the downloaded file (optional).
• hf_token (str, optional): The Hugging Face token for authentication (optional).
• trigger_signal (*, optional): A signal to trigger the download (optional).

Every Set*LLMServiceConnector node can pull its API key from a local JSON file instead of being typed into the node's api_token input. Copy mie_llm_keys.json.example to mie_llm_keys.json next to this README and fill in only the services you use; unused keys can stay empty. The file is read by core.utils.load_plugin_config and is intentionally not committed, so each install / clone keeps its own secrets.

The default config_key for each connector matches the JSON key in the example file. Leave it at the default to read the matching entry, or set a custom value to read a different entry from the same file.

Key prefixes collide across vendors (sk- is shared by Bailian PAYG, MiniMax Open, MiMo Open, DeepSeek, Kimi and SiliconFlow; sk-sp- by Bailian Token Plan, Bailian Coding Plan and SiliconFlow Coding Plan), and several platforms have subscription tiers with their own endpoint and key. docs/LLM_SERVICE_SELECTION_GUIDE.md resolves all of it: a key-prefix → connector table, a provider/use-case table, the endpoint reference, current model lists per connector, and a pitfalls section (401 tier mix-ups, retired model names, providers that pin sampling parameters).

It is written as an LLM-readable instruction document, so there are two ways to use it:

• Ask an assistant to fill it in for you — point your AI assistant at the guide (paste it, or add it to the system prompt / agent instructions) and describe your key or your provider, e.g. "I have an sk-sp- key from Alibaba Bailian and I bought the Coding Plan" or "I want the cheapest model that can read images". The guide teaches it to walk the decision tree, pick the right Set*LLMServiceConnector, drop the key into the matching mie_llm_keys.json slot, and warn you about the tier traps.
• Read it yourself — §2 is a lookup table from key prefix to connector, §5 lists the model ids currently in each dropdown.

Keep the two in sync: services/llm.py is the source of truth, and the guide MUST be updated whenever a connector, endpoint or dropdown changes.

Resolution rules (matches core.utils.resolve_token):

1. The connector loads the JSON at the path given by config_file (default: mie_llm_keys.json in the plugin root — the same folder as this README).
2. It looks up the entry named by config_key (default per the table above; override on the node to read a different entry).
3. With prefer_local_config=True (default), the JSON file wins. The api_token node input is used only when the file is missing or the entry is empty. Flip to False to make the node input win — useful when you keep a placeholder in the file but want to override the key per workflow.

To use a different config file (for example, a shared team config kept outside the plugin folder), set config_file on the node to its absolute path.

Function: Connect to a locally running Ollama instance via its OpenAI-compatible /v1/chat/completions endpoint (Ollama 0.3+). Ollama does not require an API key — the Authorization: Bearer ... header is sent but ignored by the server, so the api_token field can be left empty (or set to anything). Parameters:

• host (str): bare Ollama server URL. Default http://127.0.0.1:11434. Common variants:
  • http://127.0.0.1:11434 (default, local install)
  • http://192.168.x.x:11434 (LAN host running Ollama, with OLLAMA_HOST=0.0.0.0 set on the server)
  • http://host.docker.internal:11434 (Ollama running in Docker, reached from another container on the same host)
• model (str): the name of a locally installed Ollama model. Free-form text — type whatever you have (qwen2.5, llama3.2, deepseek-r1, llava, qwen2.5vl:7b, etc.). Pull new models with ollama pull <name> on the host. Empty falls back to qwen2.5.
• api_token (str, optional): placeholder only. Ollama ignores the value, but you can fill it in if you run Ollama behind a reverse proxy that needs auth.
• config_file / config_key / prefer_local_config: standard config plumbing (default config_key=ollama matches the example JSON).
• timeout (int, optional, default 60s): per-request HTTP timeout. Set higher (e.g. 120-180s) if you pull a large model that takes a long time to load on first call. The base class retries on timeout, but bumping the single-attempt timeout avoids the wasted first attempt.

Notes:

• Vision models (llava, llama3.2-vision, qwen2.5vl, gemma3, etc.) work through the OpenAI-compat layer: just connect an IMAGE input to the downstream CallLLMService node.
• Reasoning models (deepseek-r1, qwen3) emit <think>...</think> chains before the final answer; the base class strips them automatically.
• Ollama-level knobs (num_ctx, keep_alive, custom Modelfile) are NOT exposed on the OpenAI-compat endpoint. Configure those via Modelfile on the Ollama host.
• Streaming, JSON mode, tools, and seed are all supported by Ollama's OpenAI-compat layer; the connector stays non-streaming for parity with the other Set*LLMServiceConnector nodes.

Function: Connect to the MiniMax Open Platform API with a standard eyJ... (JWT) key. Parameters:

• api_token (str): API key, or leave empty to use the minimax_open_platform entry in mie_llm_keys.json.
• model_select: one of MiniMax-M2.7, MiniMax-M2.7-highspeed, MiniMax-M2.5, MiniMax-M2.5-highspeed, Custom. Default: MiniMax-M2.7.
• config_file / config_key / prefer_local_config: standard config plumbing.

Function: Connect to the MiniMax Token Plan / Coding Plan endpoint with an sk-cp-... prefixed key. Parameters:

• api_token (str): Token Plan API key, or leave empty to use the minimax entry in mie_llm_keys.json.
• model_select: MiniMax-M3 (default), MiniMax-M2.7, MiniMax-M2.7-highspeed, MiniMax-M2.5, MiniMax-M2.5-highspeed, Custom.
• config_file / config_key / prefer_local_config: standard config plumbing.

The MiniMax connectors share a per-connector image-detail sanitization step that drops detail: "auto" from image_url parts before sending — MiniMax rejects that value with HTTP 400. The OpenAI-compat services (SiliconFlow, ZhiPu, Kimi, etc.) are unaffected by this.

Function: Connect to the Xiaomi MiMo Open Platform API with a standard sk-... API key. Parameters:

• api_token (str): API key, or leave empty to use the mimo entry in mie_llm_keys.json.
• model_select: mimo-v2.5-pro (default, Pro / deep thinking, 1M context, 128K output), mimo-v2.5 (Omni / full modality, 1M context), mimo-v2-omni (Omni, 256K context), mimo-v2-flash (Flash / low cost, 256K context), mimo-v2-pro (Pro, older), Custom.
• config_file / config_key / prefer_local_config: standard config plumbing.

Function: Connect to the Xiaomi MiMo Token Plan / Coding Plan endpoint with a tp-... prefixed API key. The Token Plan is a fixed-fee subscription with its own base URL (token-plan-cn.xiaomimimo.com) and billing; the model lineup is shared with the standard tier. Parameters:

• api_token (str): Token Plan API key, or leave empty to use the mimo_token_plan entry in mie_llm_keys.json.
• model_select: same list as the standard connector, default mimo-v2.5-pro.
• config_file / config_key / prefer_local_config: standard config plumbing.

The MiMo connectors post to the OpenAI-compatible /v1/chat/completions endpoint but use max_completion_tokens (not the legacy max_tokens) and drop top_k / n / response_format from the payload — the MiMo docs never show these and they are likely to 400. Defaults are temperature=1.0, top_p=0.95. The image-detail sanitization is the same as MiniMax: detail: "auto" is stripped from image_url parts; explicit low / high from the caller is preserved.

Function: Rewrite a user prompt using a Bernini task-aware system prompt. Optionally forward media (1 source image or a batch of source video frames, plus 0+ image references and 0+ video-reference frames) so the LLM can see what it's rewriting about. Parameters:

• llm_service_connector (LLMServiceConnector): any connector from the list above.
• task_type: one of the 13 task types (dropdown shows code - 中文 labels). Default: t2i - 文生图. The ri2i (扩展) entry is a MieNodes extension task.
• user_prompt (str): the raw instruction to rewrite.
• source (IMAGE, optional): the thing being operated on. For image-source tasks (i2i / ri2i / i2v) pass a single image; for video-source tasks (v2v / mv2v / vi2v / rv2v / vrc2v / ads2v) pass a batch of video frames. Unused by r2i / r2v.
• reference_images (IMAGE, optional): 0+ image references. The subject for r2i / r2v, the guidance material for ri2i / vi2v / rv2v / vrc2v, fallback anchor for i2v.
• reference_video (IMAGE, optional): 0+ video-reference frames. Used by ads2v (ad video to insert into the source video) and vrc2v (reference video content alongside subject images). Forwarded in full - the LLM picks the relevant frames.
• video_frames (int, 1-16): how many of source to forward when the task treats it as video frames (ignored for image-source tasks). Default: 3.
• image_detail (auto / low / high): OpenAI-style image detail hint.
• temperature / top_p / max_tokens: standard sampling parameters.

Routing detail:

• t2i and t2v use specialized Chinese system prompts (T2I_A14B_EN_SYS_PROMPT, T2V_A14B_EN_SYS_PROMPT).
• The other 11 tasks use the per-task system prompt in bernini_prompts.SYSTEM_PROMPTS and route reference material through the appropriate *_TEMPLATE.
• r2i, r2v, rv2v, vrc2v, ri2i return JSON-mode outputs ({"rewritten_text": "..."}); the node parses the JSON and returns the inner string.

ComfyUI-MieNodes is under active development and will expand its features in future updates. Planned additions include:

• Automatic reverse caption generation for images.
• Support for complex node chaining to improve data flow integration.

As the author is a content creator, the plugin will also include many practical tools developed to address specific video production needs. Stay tuned for more updates!

• Bilibili: @黎黎原上咩
• YouTube: @SweetValberry

Some features in this project are inspired by, or build directly on the work of, the following open-source projects. We are grateful to the original authors and contributors.

• Bernini by the Bernini Team at ByteDance (Chenchen Liu, Junyi Chen, Lei Li, Lu Chi, Mingzhen Sun, Zhuoying Li, Yi Fu, Ruoyu Guo, Yiheng Wu, Ge Bai, Zehuan Yuan). The 12 task-aware system prompts and user templates used by the BerniniPromptGenerator node (under nodes/llm/bernini_prompts.py) are a verbatim copy of the upstream bytedance/Bernini prompt library (Apache 2.0). Routing logic in bernini_prompt_generator.py is a ComfyUI-friendly port of bernini.prompt_enhancer.PromptEnhancer from the same upstream repo.

• rgthree-comfy by @rgthree. The SimpleText and RichText canvas annotation nodes (in js/textNodes.js and odes/common/text_nodes.py) are a direct re-implementation of rgthree's Label node. The transparent LiteGraph shell, the LGraphCanvas.prototype.drawNode wrapper, the Canvas oundRect/draw(ctx) rendering, and the no-op INPUT_TYPES / oop Python shell all follow rgthree's approach. RichText reuses the same idea and adds a DOM widget for sanitized Markdown rendering. Many thanks to rgthree for the clean reference implementation and for the years of work that have made ComfyUI's node ecosystem so much better.