o
¶ Ïiã± ã @ sæ d Z ddlZddlmZ ddlZddlZddlZddlZddl Zddl
Zddl Zddl Zddl
ZddlZddlZddlZddlZddlZddlZddlZddlZddlZddlmZ erbddlmZ dd„ ZG dd „ d eƒZ G d
d „ d ƒZ G d d
„ d
e ƒZ G dd„ de ƒZ G dd„ dƒZ G dd„ de ƒZ!i fdd„Z"dd„ Z#i fdd„Z$G dd„ de ƒZ%i fd d „Z&i fd d „Z'd i fd!d"„Z(i fd#d$„Z)i fd%d&„Z*i fd'd(„Z+d)d*„ Z,di fd+d,„Z-di fd-d.„Z.G d/d0„ d0e ƒZ/i fd1d2„Z0dS )3aÎ
This file is part of ComfyUI.
Copyright (C) 2024 Comfy
This program is free software: you can redistribute it and/or modify
it under the terms of the GNU General Public License as published by
the Free Software Foundation, either version 3 of the License, or
(at your option) any later version.
This program is distributed in the hope that it will be useful,
but WITHOUT ANY WARRANTY; without even the implied warranty of
MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
GNU General Public License for more details.
You should have received a copy of the GNU General Public License
along with this program. If not, see .
é N)ÚEnum)Ú
TYPE_CHECKING)Ú HookGroupc C s˜ | j d }|dkr | S || }| d |… } || j d kr7tj| g|| j d | d || j d … g dd} | j d }||krB| S tj| g| ddS )Nr é ©Údim)ÚshapeÚtorchÚcat)ÚtensorÚtarget_batch_sizeÚbatched_numberÚcurrent_batch_sizeÚ per_batch© r ú./mnt/c/Users/fbmor/ComfyUI/comfy/controlnet.pyÚbroadcast_image_to. s
6
r c @ s e Zd ZdZdZdS )Ú StrengthTyper é N)Ú__name__Ú
__module__Ú __qualname__ÚCONSTANTÚ LINEAR_UPr r r r r ? s r c @ sp e Zd Zdd„ Zdddg fdd„Zdd „ Zd
d „ Zd d
„ Zdd„ Zdd„ Z dd„ Z
dd„ Z dd„ Z ddd„Z
dS )Ú ControlBasec C s| d | _ d | _d| _d| _d | _d | _d| _d | _d| _d| _ i | _
d | _ g | _ t
j| _d| _g | _d | _d | _dd„ | _d S )Nç ð?©ç r Fé ú
nearest-exactc S ó | S ©Nr ©Úar r r ÚW ó z&ControlBase.__init__..)Úcond_hint_originalÚ cond_hintÚstrengthÚtimestep_percent_rangeÚ
latent_formatÚvaeÚglobal_average_poolingÚtimestep_rangeÚcompression_ratioÚupscale_algorithmÚ
extra_argsÚprevious_controlnetÚ extra_condsr r Ú
strength_typeÚ concat_maskÚextra_concat_origÚ extra_concatÚ extra_hooksÚpreprocess_image©Úselfr r r Ú__init__D s& zControlBase.__init__r r Nc C sp || _ || _|| _| jd ur|d u rt d¡ || _| ¡ | _| j r6t
| jƒdkr6| j t
dgggg¡¡ | S )NzXWARNING: no VAE provided to the controlnet apply node when this controlnet requires one.r r )r& r( r) r* ÚloggingÚwarningr+ Úcopyr5 r4 ÚlenÚappendr r )r: r' r( r) r+ r6 r r r Ú
set_cond_hintY s
zControlBase.set_cond_hintc C s> || j d ƒ|| j d ƒf| _| jd ur | j ||¡ d S d S )Nr r )r) r- r1 Úpre_run©r: ÚmodelÚ percent_to_timestep_functionr r r rB f s
ÿzControlBase.pre_runc C s
|| _ | S r! )r1 )r: Ú
controlnetr r r Úset_previous_controlnetk s z#ControlBase.set_previous_controlnetc C s* | j d ur
| j ¡ d | _d | _d | _d S r! )r1 Úcleanupr' r6 r- r9 r r r rH o s
zControlBase.cleanupc C s g }| j d ur|| j ¡ 7 }|S r! )r1 Ú
get_models©r: Úoutr r r rI w s
zControlBase.get_modelsc C s6 g }| j d ur
| | j ¡ | jd ur|| j ¡ 7 }|S r! )r7 r@ r1 Úget_extra_hooksrJ r r r rL } s
zControlBase.get_extra_hooksc C s– | j |_ | j|_| j|_| j|_| j|_| j|_| j|_| j ¡ |_| j |_ | j
¡ |_
| j |_ | j |_ | j
¡ |_
| jrB| j ¡ nd |_| j|_d S r! )r& r( r) r, r. r/ r* r0 r> r+ r2 r3 r4 r5 r7 Úcloner8 ©r: Úcr r r Úcopy_to… s zControlBase.copy_toc C s | j d ur | j |¡S dS )Nr )r1 Ú inference_memory_requirements©r: Údtyper r r rQ – ó
z)ControlBase.inference_memory_requirementsc C sž g g g dœ}|D ]n}|| }t ƒ }tt|ƒƒD ]^}|| } | d urn| jr8tj| ddd dd| jd | jd ¡} | |vr`| | ¡ | j t
j krM| | j 9 } n| j t
j
kr`| | j tt|ƒ| ƒ 9 } |d urn| j|krn| |¡} || | ¡ qq|d urÍdD ]O} || }
tt|| ƒƒD ]@}|| | } |t|
ƒkrŸ|
| ¡ q‹| d urË|
| d u r®| |
|< q‹|
| jd | jd k rÃ| |
| |
|< q‹| |
| |
|< q‹q}|S ) N)ÚinputÚmiddleÚoutput)r é T)r Úkeepdimr r rX r )ÚsetÚranger? r, r ÚmeanÚrepeatr Úaddr3 r r r( r ÚfloatrS Útor@ ) r: ÚcontrolÚ control_prevÚ output_dtyperK ÚkeyÚcontrol_outputÚ
applied_toÚiÚxÚoÚprev_valr r r Ú
control_merge› sF (
ð
€õ zControlBase.control_mergec C s || j |< d S r! )r0 )r: ÚargumentÚvaluer r r Ú
set_extra_argÄ s zControlBase.set_extra_argr! )r r r r; rA rB rG rH rI rL rP rQ rk rn r r r r r C s
)r c sn e Zd Zdddddddgejddd„ f
‡ fdd„ Zd d
„ Zd d „ Z‡ fd
d„Z‡ fdd„Z ‡ fdd„Z
‡ Z S )Ú
ControlNetNFr Úyc C r r! r r" r r r r$ É r% zControlNet.c s| t ƒ ¡ || _|| _|d ur tjj| j|tj ¡ d| _ || _
|| _ d | _ || _
|| _| j|7 _|| _| | _|
| _d S )N)Ú load_deviceÚoffload_device)Úsuperr; Ú
control_modelrq ÚcomfyÚ
model_patcherÚCoreModelPatcherÚmodel_managementÚunet_offload_deviceÚcontrol_model_wrappedr. r, Úmodel_sampling_currentÚmanual_cast_dtyper* r2 r3 r4 r8 ) r: rt r, r. r* rq r| r2 r3 r4 r8 ©Ú __class__r r r; É s
zControlNet.__init__c
C sF d }| j d ur| j |||||¡}| jd ur0|d | jd ks(|d | jd k r0|d ur.|S d S | jj}| jd ur<| j}| jd u s^|jd | j | jjd ks^|jd | j | jjd kr2| jd ure| `d | _| j}| j d urx|| j
¡ 9 }n | j d urt dƒ‚t
j | j|jd | |jd | | jd¡| _| | j¡| _| j d ur½t
jjd d
} | j | j dd¡¡| _t
j | ¡ | j d urÊ| j | j¡| _t| jƒdkr(g }
| jD ]D} | | jj ¡} t
j | | jjd | jjd | jd¡} | j | jj k r
| d¡} t
jj | | jjd dd } |
t
j | | jjd ¡¡ q×t!j"| jg|
dd | _| jj|j |d | _|jd | jjd krIt#| j|jd |ƒ| _| $d
|d ¡} | j% &¡ }
| j'D ]} | $| d ¡}|d urqt
j( )|||j ¡|
| < qY| j* +|¡}| j* ,||¡}| jd| |¡| j| |¡t
j -| |j |¡dœ|
¤Ž}| j.||d dS )Nr r r rX zuThis Controlnet needs a VAE but none was provided, please use a ControlNetApply node with a VAE input and connect it.éÿÿÿÿéþÿÿÿÚcenterT)Úonly_currently_usedr )ÚdevicerS Úcrossattn_controlnetÚ c_crossattn)rh ÚhintÚ timestepsÚcontext)rc r )/r1 Ú get_controlr- rt rS r| r' r r. r+ Úspacial_compression_encoder* Ú
ValueErrorru ÚutilsÚcommon_upscaler& r/ r8 rx Ú
loaded_modelsÚencodeÚmovedimÚload_models_gpuÚ
process_inr? r5 r` rƒ ÚndimÚ unsqueezeÚrepeat_to_batch_sizer@ r r
r Úgetr0 r> r2 Ú
model_baseÚconvert_tensorr{ ÚtimestepÚcalculate_inputÚcast_to_devicerk )r: Úx_noisyÚtÚcondr
Útransformer_optionsrb rS r. rŽ Ú to_concatrO rˆ ÚextraÚtempr™ ra r r r r‰ Ú sh
$
D
.
&
€ 6zControlNet.get_controlc C ó4 t d | j| j| jd}| j|_| j|_| |¡ |S ©N©r, rq r| )ro r, rq r| rt rz rP rN r r r r> ó
zControlNet.copyc s t ƒ ¡ }| | j¡ |S r! )rs rI r@ rz rJ r} r r rI rT zControlNet.get_modelsc s t ƒ ||¡ |j| _d S r! )rs rB Úmodel_samplingr{ rC r} r r rB $ s zControlNet.pre_runc s d | _ tƒ ¡ d S r! )r{ rs rH r9 r} r r rH ( s zControlNet.cleanup) r r r r r r; r‰ r> rI rB rH Ú
__classcell__r r r} r ro È s *> ro c s0 e Zd Z‡ fdd„Z‡ fdd„Zdd„ Z‡ ZS )ÚQwenFunControlNetc s@ | j }t t| j dƒ¡| _ ztƒ |||||¡W || _ S || _ w )Nr )r( ÚmathÚsqrtÚmaxrs r‰ )r: rœ r rž r
rŸ Úoriginal_strengthr} r r r‰ . s
z QwenFunControlNet.get_controlc s t ƒ ||¡ | d|j¡ d S )NÚ
base_model)rs rB rn Údiffusion_modelrC r} r r rB 9 s zQwenFunControlNet.pre_runc C r£ r¤ )r© r, rq r| rt rz rP rN r r r r> = r¦ zQwenFunControlNet.copy)r r r r‰ rB r> r¨ r r r} r r© - s r© c @ s@ e Zd ZG dd„ dejjejjƒZ G dd„ dejjejjƒZ
dS )ÚControlLoraOpsc s< e Zd Z d dedededdf‡ fdd„
Zd d
„ Z‡ ZS ) zControlLoraOps.LinearTNÚ in_featuresÚ out_featuresÚbiasÚreturnc s2 t ƒ ¡ || _|| _d | _d | _d | _d | _d S r! )rs r; r± r² ÚweightÚupÚdownr³ )r: r± r² r³ rƒ rS r} r r r; F s
z ControlLoraOps.Linear.__init__c
C s t jj| |dd\}}}| jd ur4tjj ||t | jj dd| j
j dd¡ | j j
¡ |j¡ |¡}n tjj |||¡}t j | |||¡ |S ©NT)Ú offloadabler )Ú start_dim)ru ÚopsÚcast_bias_weightr¶ r ÚnnÚ
functionalÚlinearÚmmÚflattenr· Úreshaperµ r ÚtyperS Úuncast_bias_weight©r: rU rµ r³ Úoffload_streamrh r r r ÚforwardP s
Fz ControlLoraOps.Linear.forward)TNN)r r r ÚintÚboolr; rÇ r¨ r r r} r ÚLinearE s ÿÿ
rÊ c s6 e Zd Z d
‡ fdd„ Zdd „ Z‡ ZS ) zControlLoraOps.Conv2dr r TÚzerosNc sb t ƒ ¡ || _|| _|| _|| _|| _|| _d| _d| _ || _
| | _ d | _ d | _
d | _d | _d S )NFr )rs r; Ú in_channelsÚ out_channelsÚ kernel_sizeÚstrideÚpaddingÚdilationÚ
transposedÚoutput_paddingÚgroupsÚ padding_moderµ r³ r¶ r· ) r: rÌ rÍ rÎ rÏ rÐ rÑ rÔ r³ rÕ rƒ rS r} r r r; Z s
z ControlLoraOps.Conv2d.__init__c
C s° t jj| |dd\}}}| jd ur.control_lora_opsN©r r r r r r r Úcontrol_lora_ops’ ó rá c @ rÞ rß rà r r r r rá • râ Ú
operationsrS > Úlora_controlnetz.upz.downú.weightú.biasÚ__r )rs rB Ú model_configÚ unet_configr> ÚpoprÛ r r| Ú get_dtyper° ru r» Údisable_weight_initÚ manual_castÚcldmro rt r` rx Úget_torch_devicer¯ Ú
state_dictrŒ Úset_attr_paramÚendswith)
r: rD rE Úcontrolnet_configrS rá r¯ ÚsdÚkrµ r} r r rB Š s8
0*€ýzControlLora.pre_runc C s t | j| jd}| |¡ |S )N)r, )rÚ rÛ r, rP rN r r r r> ¬ s
zControlLora.copyc s | ` d | _ tƒ ¡ d S r! )rt rs rH r9 r} r r rH ± s zControlLora.cleanupc C s t | ¡}|S r! )r rI rJ r r r rI ¶ s
zControlLora.get_modelsc C s& t j | j¡t j |¡ t | |¡ S r! )ru rŒ Úcalculate_parametersrÛ rx Ú
dtype_sizer rQ rR r r r rQ º s &z)ControlLora.inference_memory_requirements)
r r r r; rB r> rH rI rQ r¨ r r r} r rÚ ƒ s " rÚ c
C s¤ t j | dd¡}| dd ¡}|d u r&t j | ¡}t|jƒ}t jj d||d}t j
¡ }t j ||¡}| dd ¡}|d u rEt j j
||dd}t j ¡ } |||||| fS )NÚ TrS r ©Ú model_paramsÚsupported_dtypesÚ weight_dtypeÚcustom_operations©Údisable_fast_fp8)ru Úmodel_detectionÚmodel_config_from_unetr– rŒ rü ÚlistÚsupported_inference_dtypesrx Ú
unet_dtyperï Úunet_manual_castr» Úpick_operationsry )
rô rÜ rè r rü r rq r| rã rr r r r ró ½ s
ró c C sN | j |dd\}}t|ƒdkrt d |¡¡ t|ƒdkr%t d |¡¡ | S )NF©Ústrictr úmissing controlnet keys: {}ú unexpected controlnet keys: {})Úload_state_dictr? r< r= ÚformatÚdebug)rt rô ÚmissingÚ
unexpectedr r r Úcontrolnet_load_state_dictÑ s r c C s¾ t j | d¡}t||d\}}}}}}t j |d¡} | D ]}
| |
||
< q d} | d¡jd } | dkr5d} t jjj d | | |||d œ|j
¤Ž}
t |
|ƒ}
t j
¡ }d
|_t |
d|| ||d }|S )
Nrø ©rÜ zjoint_blocks.{}.Fúpos_embed_input.proj.weightr é T)Ú
num_blocksÚcontrol_latent_channelsrã rƒ rS r )r. r* r4 rq r| r )ru r Úconvert_diffusers_mmditró Ú count_blocksr– r rî Úmmditro ré r Úlatent_formatsÚSD3Ú shift_factor)rô rÜ Únew_sdrè rã rq r r| rr r rõ r4 r rt r* ra r r r Úload_controlnet_mmditÜ s "
r c s$ e Zd Z‡ fdd„Zdd„ Z‡ ZS )ÚControlNetSD35c sX | j jr| j jj|jj ¡ dd\}}n| j jj|jj ¡ dd\}}tƒ ||¡ d S )NFr )
rt Ú double_y_embÚorig_y_embedderr r¯ Ú
y_embedderrð Ú
x_embedderrs rB )r: rD rE r r r} r r rB ò s zControlNetSD35.pre_runc C r£ r¤ )r r, rq r| rt rz rP rN r r r r> ù r¦ zControlNetSD35.copy)r r r rB r> r¨ r r r} r r ñ s r c C sŠ d}d| v rt | d¡ ¡ ƒ}|dk}|dk}i }tjjD ]}|d | v r0| |d ¡||d < q | D ]}| | ||< q3|} | d j}|d d }d| } |}
| |
} tj |d¡} tj
¡ }
tj ¡ }tj j dd }tj
||
¡}| d
d ¡}|d u r…tjj||d d }tjjjd dd
| ||d |d k| |
d|||d }t|| ƒ}tj ¡ }dd„ }|r³dd„ }n|r¹dd„ }t|d||
||d}|S )Nr Ú control_typer r r zy_embedder.mlp.0.weighté@ ztransformer_blocks.{}.)rú rý Trþ é i ) Úimg_sizeÚ
patch_sizeÚin_chansÚ
num_layersÚmain_model_doubler Úattention_head_dimÚnum_attention_headsÚadm_in_channelsrƒ rS rã c S r r! r r" r r r r$ 2 r% z&load_controlnet_sd35..c S s | d d d S )Néÿ g à?r r" r r r r$ 4 s c S s d| S )Nr r r" r r r r$ 6 s )r. r* rq r| r8 )Úroundrê Úitemru rŒ ÚMMDIT_MAP_BASICr r r rx rï ry r r r– r» r rî Ú dit_embedderÚControlNetEmbedderr r r r )rô rÜ r# Ú
canny_cnetÚ
depth_cnetr rõ Ú y_emb_shapeÚdepthÚ hidden_sizeÚ num_headsÚhead_dimr rq rr r r| rã rt r* r8 ra r r r Úload_controlnet_sd35 s` €
õ
r; c C sf t | |d\}}}}}}tjjjj|||d}t|| ƒ}tj ¡ } g d¢}
t |d| |||
t
j d} | S )Nr )rã rƒ rS )Útext_embedding_maskÚencoder_hidden_states_t5Útext_embedding_mask_t5Úimage_meta_sizeÚstyleÚ cos_cis_imgÚ sin_cis_imgr )r. r* rq r| r2 r3 ) ró ru ÚldmÚhyditrF ÚHunYuanControlNetr r ÚSDXLro r r ) Úcontrolnet_datarÜ rè rã rq r r| rr rt r* r2 ra r r r Úload_controlnet_hunyuandit= s
rH Fc C sj t | |d\}}}}}}tjjjjd||||dœ|j¤Ž} | | ¡} t| | ƒ} ddg}
t | |||
d} | S )Nr )Ú mistolinerã rƒ rS rp Úguidance)rq r| r2 r )
ró ru rC ÚfluxrF ÚControlNetFluxré Úprocess_unet_state_dictr ro ) rô rI rÜ rè rã rq r r| rr rt r2 ra r r r Ú$load_controlnet_flux_xlabs_mistolineH s "
rN c C sÚ t j | d¡}t||d\}}}}}}| D ]} | | || < qd}
d} | |v r-|| jd }
| d¡jd d } d}
| d kr?d
}
t jjjj dd
|
| |||d œ|j
¤Ž}t ||ƒ}t j
¡ }d d
g}t|d||
|||d}|S )Nrø r r z controlnet_mode_embedder.weightzpos_embed_input.weightr é Fr T)Ú latent_inputÚnum_union_modesr rã rƒ rS rp rJ ©r. r* r4 rq r| r2 r )ru r r ró r r– rC rK rF rL ré r r ÚFluxro )rô rÜ r rè rã rq r r| rr rõ rQ Ú
union_cnetr r4 rt r* r2 ra r r r Ú load_controlnet_flux_instantxQ s$ &
rU c C s˜ t | |d\}}}}}}| d¡jd }d} d}
|dkr"|d } d}
tjjjjd | |||d œ|j¤Ž} t | | ƒ} tj
¡ } g }
t | d| |
|||
d
}|S ) Nr ú controlnet_x_embedder.weightr r FéD r$ T)Úextra_condition_channelsrã rƒ rS rR r )
ró r– r ru rC Ú
qwen_imagerF ÚQwenImageControlNetModelré r r ÚWan21ro )rô rÜ rè rã rq r r| rr r rX r4 rt r* r2 ra r r r Ú load_controlnet_qwen_instantxi s "
r\ c C sú t j ¡ }t j | ¡}| d|¡}t j ||¡}| dd ¡}|d u r+t jj||dd}| d j d }| d j d }| d } | d j d }
t
d| j d t
d|
ƒ ƒ} t j j j
j||| |
d
d d |t j ¡ |d
} t| | ƒ} t j ¡ }
t| d|
d||g d}|S )NrS rý Trþ úcontrol_img_in.weightr r z!control_blocks.0.attn.to_q.weightz#control_blocks.0.attn.norm_q.weighté é< )r é é é$ é0 )
Úcontrol_in_featuresÚ inner_dimr, r+ Únum_control_blocksr* Úinjection_layersrã rƒ rS FrR )ru rx rï rŒ rü r– r r» r r r¬ rC rY rF ÚQwenImageFunControlNetModelry r r r[ r© )rô rÜ rq rü r r| rã r± re Ú block_weightr+ r, rD r* ra r r r Úload_controlnet_qwen_funz sH
ö
ö rj c C s t j | ddi¡S )Nzsingle_controlnet_blocks.zcontrolnet_single_blocks.)ru rŒ Ústate_dict_prefix_replace)rô r r r Úconvert_mistoline¨ s rl c ( C s$ | }d| ¡ v rt||dS d|v rt||dS d }d }d|v rëtj |¡}tj |¡}d|d< d|d< d }d
}|rad d g} | D ]}
d
||
¡} d ||
¡} | |vrVd} n| || < q@|d7 }|s:d }d
}|rŸd d g} | D ]+}
|d kryd |
¡} nd |d |
¡} d |d |
¡} | |vr”d |
¡} d}| || < qm|d7 }|sgi }
|D ]}||v r²| |¡|
|| < q£d|v rÖ|d j
d |d< t | ¡ ƒD ]}| dd¡}| |¡|
|< qÆ| ¡ }t
|ƒd krèt d |¡¡ |
}nZd |v r'd |v rút||dS d |v rd |v r
t||dS t||dS d |v rt||dS d!|v r&t||dS n d"|v r7d#|v r7t||dS d$|v rEtt|ƒd
|d%S d&}d}d'}||v rWd
}|}d(}n||v r_d)}nt||d}|d u rot d*¡ |S |d u r†tj ||d
¡}t |jƒ}|j}| d+d ¡}|d u r«tj |¡}|d u r¢tj ¡ g}tj j d,||d-}tj ¡ }tj !||¡}| d.d ¡}|d u rÉtj" #||¡}||d/< ||d+< tj $¡ |d0< | d1¡ |d2 |¡ j
d |d3< tj%j%j&d>i |¤Ž} |r[d4|v rA|d ur.WeightsLoaderNrà r r r r Ú
WeightsLoader5 râ rp r r r
r, r¥ r )4ÚkeysrH rÚ ru r Ú unet_config_from_diffusers_unetrŒ Úunet_to_diffusersr rê r r Úreplacer? r< r= rN r; r r\ rU rj rl Úload_t2i_adapterÚerrorr r ré r– rü rx r rï r r» r ry rî ro r‘ Úmodel_state_dictÚ
startswithrà rS r` rƒ r r½ rØ rt r r
)(rð rD rÜ rG ró r Údiffusers_keysÚcountÚloopÚsuffixÚsÚk_inÚk_outr rõ Únew_kÚ
leftover_keysÚpth_keyÚpthrd ÚprefixÚnetrè r rü rq r| rã rt Úmodel_sdrh Úc_mÚsd_keyÚcdrp Úwr r r, ra r r r Úload_controlnet_state_dict¬ s
÷
ô€
ÿ
€ú
r‹ c C sr | ¡ }d|vr tj | ¡d }| d¡s| d¡r d|d< ttjj| dd||d}|d u r7t
d | ¡¡ |S ) Nr, r Ú_shuffleÚ
_shuffle_fp16T)Ú safe_load)rD rÜ zCerror checkpoint does not contain controlnet or t2i adapter data {}) r> ÚosÚpathÚsplitextrò r‹ ru rŒ Úload_torch_filer< rv r )Ú ckpt_pathrD rÜ ÚfilenameÚcnetr r r Úload_controlnetG s r– c s6 e Zd Zd
‡ fdd„ Zdd„ Zdd„ Zdd „ Z‡ ZS ) Ú
T2IAdapterNc sD t ƒ ¡ || _|| _d | _|| _|| _|d u r tj ¡ }|| _
d S r! ) rs r; Ú t2i_modelÚ channels_inÚ
control_inputr. r/ ru rx rï rƒ )r: r˜ r™ r. r/ rƒ r} r r r; T s
zT2IAdapter.__init__c C s4 | j j}t || ¡| }t || ¡| }||fS r! )r˜ Úunshuffle_amountrª Úceil)r: ÚwidthÚheightr› r r r Úscale_image_to_ s zT2IAdapter.scale_image_toc C sæ d }| j d ur| j |||||¡}| jd ur0|d | jd ks(|d | jd k r0|d ur.|S d S | jd u sQ|jd | j | jjd ksQ|jd | j | jjd krœ| jd urX| `d | _d | _| |jd | j |jd | j ¡\}}tj
| j ||| j d¡
¡ | j¡| _| jdkrœ| jjd dkrœtj| jddd| _|jd | jjd kr²t| j|jd |ƒ| _| jd u rÕ| j |j¡ | j | j¡ | | j |j¡¡| _| j ¡ i } | jD ]}
ttdd „ | j|
ƒƒ| |
< qÚ| | ||j¡S )
Nr r r rX r T)rY c S s | d u rd S | ¡ S r! )rM r" r r r r$ „ s z(T2IAdapter.get_control..)r1 r‰ r- r' r r. rš rŸ ru rŒ r r& r/ r_ r` rƒ r™ r r\ r r˜ rS Úcpur Úmaprk ) r: rœ r rž r
rŸ rb r rž rš rõ r r r r‰ e s8
$B
(&
zT2IAdapter.get_controlc C s$ t | j| j| j| jƒ}| |¡ |S r! )r— r˜ r™ r. r/ rP rN r r r r> ˆ s
zT2IAdapter.copyr! )r r r r; rŸ r‰ r> r¨ r r r} r r— S s
#r— c C s6 d}d}d| v r | d } d| v rFi }t dƒD ]$}t dƒD ]}d |d | ¡|d ||¡< q d |d ¡|d |¡< qd
|d < tj | |¡} | ¡ }d |v rb| d jd
}tjjj |g d¢dd} nŒd|v r·| d jd
}| d jd }
| d jd } d} t
t dd„ |ƒƒ}
t |
ƒdkrŽd} d}|dks˜|dkršd}tjjj
||
|
d |
d |
d gd d… d| d| |d} n7d|v rÑtjjjj| d jd
g d¢d } d }d }n d |v rìtjjjj| d jd
d!g d¢d"} d
}d}nd S | | ¡\}}t |ƒdkrt d# |¡¡ t |ƒdkrt d$ |¡¡ t| | j||ƒS )%Nr r Úadapterz&adapter.body.0.resnets.0.block1.weightrO r zbody.{}.zadapter.body.{}.resnets.{}.zadapter.body.{}.rø zadapter.zbody.0.in_conv.weightr )i@ i€ é r£ )ÚcinÚchannelsÚnums_rbzconv_in.weightr zbody.0.block2.weightFc S s
| d¡S )Nzdown_opt.op.weight)rò r" r r r r$ ¥ s
z"load_t2i_adapter..Té i )r¤ r¥ r¦ ÚksizeÚskÚuse_convÚxlzbackbone.0.0.weight)r rO r r` é3 é7 é; é? )Úc_inÚ proj_blocksé Úbilinearzbackbone.10.blocks.0.weightzbackbone.0.weightÚlarge)r° Úbottleneck_moder± zt2i missing {}zt2i unexpected {})r[ r ru rŒ rk rq r Ú t2i_adapterr¢ Ú
Adapter_lightr Úfilterr? ÚAdapterrC ÚcascaderF ro r r< r= r
r— Úinput_channels)Út2i_datarÜ r. r/ Úprefix_replacerg Újrq r¤ Úmodel_adÚchannelr¨ rª Ú down_optsr« r r r r r ru sV :"$ru )1Ú__doc__r Úenumr rª r r< Ú comfy.utilsru Úcomfy.model_managementÚcomfy.model_detectionÚcomfy.model_patcherÚ comfy.opsÚcomfy.latent_formatsÚcomfy.model_baseÚcomfy.cldm.cldmÚcomfy.t2i_adapter.adapterÚ comfy.ldm.cascade.controlnetÚcomfy.cldm.mmditÚcomfy.ldm.hydit.controlnetÚcomfy.ldm.flux.controlnetÚ comfy.ldm.qwen_image.controlnetÚcomfy.cldm.dit_embedderÚtypingr Ú comfy.hooksr r r r ro r© r° rÚ ró r r r r; rH rN rU r\ rj rl r‹ r– r— ru r r r r Ú s` e? : = . :