2024-08-27T12:42:42,668 Created temporary directory: /tmp/pip-build-tracker-6b5q52yn 2024-08-27T12:42:42,669 Initialized build tracking at /tmp/pip-build-tracker-6b5q52yn 2024-08-27T12:42:42,670 Created build tracker: /tmp/pip-build-tracker-6b5q52yn 2024-08-27T12:42:42,670 Entered build tracker: /tmp/pip-build-tracker-6b5q52yn 2024-08-27T12:42:42,671 Created temporary directory: /tmp/pip-wheel-r79_e81j 2024-08-27T12:42:42,675 Created temporary directory: /tmp/pip-ephem-wheel-cache-s48t32mn 2024-08-27T12:42:42,717 Looking in indexes: https://pypi.org/simple, https://www.piwheels.org/simple 2024-08-27T12:42:42,720 2 location(s) to search for versions of lightllm: 2024-08-27T12:42:42,720 * https://pypi.org/simple/lightllm/ 2024-08-27T12:42:42,720 * https://www.piwheels.org/simple/lightllm/ 2024-08-27T12:42:42,721 Fetching project page and analyzing links: https://pypi.org/simple/lightllm/ 2024-08-27T12:42:42,721 Getting page https://pypi.org/simple/lightllm/ 2024-08-27T12:42:42,723 Found index url https://pypi.org/simple/ 2024-08-27T12:42:42,938 Fetched page https://pypi.org/simple/lightllm/ as application/vnd.pypi.simple.v1+json 2024-08-27T12:42:42,939 Skipping link: No binaries permitted for lightllm: https://files.pythonhosted.org/packages/52/6f/b70cba2598eacf4afff2c5f8679dac9c1dd830f7462623a761397741c9ad/lightllm-0.0.1-py3-none-any.whl (from https://pypi.org/simple/lightllm/) (requires-python:>=3.9) 2024-08-27T12:42:42,940 Found link https://files.pythonhosted.org/packages/24/f7/a4eb391f04d43375339fdcbf54009d1fe3637c4c87def5fcc6391806b3be/lightllm-0.0.1.tar.gz (from https://pypi.org/simple/lightllm/) (requires-python:>=3.9), version: 0.0.1 2024-08-27T12:42:42,941 Fetching project page and analyzing links: https://www.piwheels.org/simple/lightllm/ 2024-08-27T12:42:42,942 Getting page https://www.piwheels.org/simple/lightllm/ 2024-08-27T12:42:42,943 Found index url https://www.piwheels.org/simple/ 2024-08-27T12:42:43,097 Fetched page https://www.piwheels.org/simple/lightllm/ as text/html 2024-08-27T12:42:43,098 Skipping link: not a file: https://www.piwheels.org/simple/lightllm/ 2024-08-27T12:42:43,099 Skipping link: not a file: https://pypi.org/simple/lightllm/ 2024-08-27T12:42:43,117 Given no hashes to check 1 links for project 'lightllm': discarding no candidates 2024-08-27T12:42:43,119 Collecting lightllm==0.0.1 2024-08-27T12:42:43,121 Created temporary directory: /tmp/pip-unpack-zxv911_h 2024-08-27T12:42:43,329 Downloading lightllm-0.0.1.tar.gz (198 kB) 2024-08-27T12:42:44,070 Added lightllm==0.0.1 from https://files.pythonhosted.org/packages/24/f7/a4eb391f04d43375339fdcbf54009d1fe3637c4c87def5fcc6391806b3be/lightllm-0.0.1.tar.gz to build tracker '/tmp/pip-build-tracker-6b5q52yn' 2024-08-27T12:42:44,072 Running setup.py (path:/tmp/pip-wheel-r79_e81j/lightllm_ed0f3a808c8f4c2e9e2d8fd196b7c760/setup.py) egg_info for package lightllm 2024-08-27T12:42:44,073 Created temporary directory: /tmp/pip-pip-egg-info-x_xbisfa 2024-08-27T12:42:44,074 Preparing metadata (setup.py): started 2024-08-27T12:42:44,075 Running command python setup.py egg_info 2024-08-27T12:42:45,277 running egg_info 2024-08-27T12:42:45,278 creating /tmp/pip-pip-egg-info-x_xbisfa/lightllm.egg-info 2024-08-27T12:42:45,305 writing /tmp/pip-pip-egg-info-x_xbisfa/lightllm.egg-info/PKG-INFO 2024-08-27T12:42:45,308 writing dependency_links to /tmp/pip-pip-egg-info-x_xbisfa/lightllm.egg-info/dependency_links.txt 2024-08-27T12:42:45,310 writing requirements to /tmp/pip-pip-egg-info-x_xbisfa/lightllm.egg-info/requires.txt 2024-08-27T12:42:45,312 writing top-level names to /tmp/pip-pip-egg-info-x_xbisfa/lightllm.egg-info/top_level.txt 2024-08-27T12:42:45,313 writing manifest file '/tmp/pip-pip-egg-info-x_xbisfa/lightllm.egg-info/SOURCES.txt' 2024-08-27T12:42:45,741 reading manifest file '/tmp/pip-pip-egg-info-x_xbisfa/lightllm.egg-info/SOURCES.txt' 2024-08-27T12:42:45,743 adding license file 'LICENSE' 2024-08-27T12:42:45,755 writing manifest file '/tmp/pip-pip-egg-info-x_xbisfa/lightllm.egg-info/SOURCES.txt' 2024-08-27T12:42:45,868 Preparing metadata (setup.py): finished with status 'done' 2024-08-27T12:42:45,872 Source in /tmp/pip-wheel-r79_e81j/lightllm_ed0f3a808c8f4c2e9e2d8fd196b7c760 has version 0.0.1, which satisfies requirement lightllm==0.0.1 from https://files.pythonhosted.org/packages/24/f7/a4eb391f04d43375339fdcbf54009d1fe3637c4c87def5fcc6391806b3be/lightllm-0.0.1.tar.gz 2024-08-27T12:42:45,873 Removed lightllm==0.0.1 from https://files.pythonhosted.org/packages/24/f7/a4eb391f04d43375339fdcbf54009d1fe3637c4c87def5fcc6391806b3be/lightllm-0.0.1.tar.gz from build tracker '/tmp/pip-build-tracker-6b5q52yn' 2024-08-27T12:42:45,880 Created temporary directory: /tmp/pip-unpack-fafx1gza 2024-08-27T12:42:45,881 Created temporary directory: /tmp/pip-unpack-67brqyfi 2024-08-27T12:42:45,881 Building wheels for collected packages: lightllm 2024-08-27T12:42:45,886 Created temporary directory: /tmp/pip-wheel-18cpawgl 2024-08-27T12:42:45,886 Building wheel for lightllm (setup.py): started 2024-08-27T12:42:45,887 Destination directory: /tmp/pip-wheel-18cpawgl 2024-08-27T12:42:45,888 Running command python setup.py bdist_wheel 2024-08-27T12:42:46,953 running bdist_wheel 2024-08-27T12:42:47,089 running build 2024-08-27T12:42:47,089 running build_py 2024-08-27T12:42:47,121 creating build 2024-08-27T12:42:47,121 creating build/lib 2024-08-27T12:42:47,122 creating build/lib/lightllm 2024-08-27T12:42:47,123 copying lightllm/__init__.py -> build/lib/lightllm 2024-08-27T12:42:47,125 creating build/lib/lightllm/common 2024-08-27T12:42:47,126 copying lightllm/common/mem_manager.py -> build/lib/lightllm/common 2024-08-27T12:42:47,128 copying lightllm/common/infer_utils.py -> build/lib/lightllm/common 2024-08-27T12:42:47,130 copying lightllm/common/req_manager.py -> build/lib/lightllm/common 2024-08-27T12:42:47,132 copying lightllm/common/__init__.py -> build/lib/lightllm/common 2024-08-27T12:42:47,133 copying lightllm/common/build_utils.py -> build/lib/lightllm/common 2024-08-27T12:42:47,135 copying lightllm/common/int8kv_mem_manager.py -> build/lib/lightllm/common 2024-08-27T12:42:47,136 copying lightllm/common/ppl_int4kv_mem_manager.py -> build/lib/lightllm/common 2024-08-27T12:42:47,138 copying lightllm/common/mem_utils.py -> build/lib/lightllm/common 2024-08-27T12:42:47,139 copying lightllm/common/ppl_int8kv_mem_manager.py -> build/lib/lightllm/common 2024-08-27T12:42:47,142 creating build/lib/lightllm/server 2024-08-27T12:42:47,142 copying lightllm/server/tokenizer.py -> build/lib/lightllm/server 2024-08-27T12:42:47,144 copying lightllm/server/api_models.py -> build/lib/lightllm/server 2024-08-27T12:42:47,146 copying lightllm/server/api_lightllm.py -> build/lib/lightllm/server 2024-08-27T12:42:47,149 copying lightllm/server/metrics.py -> build/lib/lightllm/server 2024-08-27T12:42:47,150 copying lightllm/server/req_id_generator.py -> build/lib/lightllm/server 2024-08-27T12:42:47,152 copying lightllm/server/build_prompt.py -> build/lib/lightllm/server 2024-08-27T12:42:47,154 copying lightllm/server/__init__.py -> build/lib/lightllm/server 2024-08-27T12:42:47,156 copying lightllm/server/io_struct.py -> build/lib/lightllm/server 2024-08-27T12:42:47,158 copying lightllm/server/api_server.py -> build/lib/lightllm/server 2024-08-27T12:42:47,160 copying lightllm/server/api_tgi.py -> build/lib/lightllm/server 2024-08-27T12:42:47,162 copying lightllm/server/multimodal_params.py -> build/lib/lightllm/server 2024-08-27T12:42:47,164 copying lightllm/server/sampling_params.py -> build/lib/lightllm/server 2024-08-27T12:42:47,167 creating build/lib/lightllm/utils 2024-08-27T12:42:47,168 copying lightllm/utils/infer_utils.py -> build/lib/lightllm/utils 2024-08-27T12:42:47,170 copying lightllm/utils/net_utils.py -> build/lib/lightllm/utils 2024-08-27T12:42:47,171 copying lightllm/utils/__init__.py -> build/lib/lightllm/utils 2024-08-27T12:42:47,173 copying lightllm/utils/petrel_helper.py -> build/lib/lightllm/utils 2024-08-27T12:42:47,175 copying lightllm/utils/health_check.py -> build/lib/lightllm/utils 2024-08-27T12:42:47,176 copying lightllm/utils/graceful_utils.py -> build/lib/lightllm/utils 2024-08-27T12:42:47,178 copying lightllm/utils/start_utils.py -> build/lib/lightllm/utils 2024-08-27T12:42:47,180 copying lightllm/utils/log_utils.py -> build/lib/lightllm/utils 2024-08-27T12:42:47,182 creating build/lib/lightllm/models 2024-08-27T12:42:47,183 copying lightllm/models/__init__.py -> build/lib/lightllm/models 2024-08-27T12:42:47,185 creating build/lib/lightllm/common/basemodel 2024-08-27T12:42:47,186 copying lightllm/common/basemodel/splitfuse_infer_struct.py -> build/lib/lightllm/common/basemodel 2024-08-27T12:42:47,188 copying lightllm/common/basemodel/__init__.py -> build/lib/lightllm/common/basemodel 2024-08-27T12:42:47,189 copying lightllm/common/basemodel/infer_struct.py -> build/lib/lightllm/common/basemodel 2024-08-27T12:42:47,191 copying lightllm/common/basemodel/basemodel.py -> build/lib/lightllm/common/basemodel 2024-08-27T12:42:47,194 creating build/lib/lightllm/common/basemodel/layer_infer 2024-08-27T12:42:47,195 copying lightllm/common/basemodel/layer_infer/__init__.py -> build/lib/lightllm/common/basemodel/layer_infer 2024-08-27T12:42:47,196 copying lightllm/common/basemodel/layer_infer/base_layer_infer.py -> build/lib/lightllm/common/basemodel/layer_infer 2024-08-27T12:42:47,198 copying lightllm/common/basemodel/layer_infer/transformer_layer_infer.py -> build/lib/lightllm/common/basemodel/layer_infer 2024-08-27T12:42:47,200 copying lightllm/common/basemodel/layer_infer/post_layer_infer.py -> build/lib/lightllm/common/basemodel/layer_infer 2024-08-27T12:42:47,201 copying lightllm/common/basemodel/layer_infer/pre_layer_infer.py -> build/lib/lightllm/common/basemodel/layer_infer 2024-08-27T12:42:47,203 creating build/lib/lightllm/common/basemodel/triton_kernel 2024-08-27T12:42:47,204 copying lightllm/common/basemodel/triton_kernel/__init__.py -> build/lib/lightllm/common/basemodel/triton_kernel 2024-08-27T12:42:47,206 copying lightllm/common/basemodel/triton_kernel/apply_penalty.py -> build/lib/lightllm/common/basemodel/triton_kernel 2024-08-27T12:42:47,208 copying lightllm/common/basemodel/triton_kernel/multimodal_emb.py -> build/lib/lightllm/common/basemodel/triton_kernel 2024-08-27T12:42:47,210 copying lightllm/common/basemodel/triton_kernel/dequantize_gemm_int8.py -> build/lib/lightllm/common/basemodel/triton_kernel 2024-08-27T12:42:47,212 copying lightllm/common/basemodel/triton_kernel/copy_kv_index_to_req.py -> build/lib/lightllm/common/basemodel/triton_kernel 2024-08-27T12:42:47,214 copying lightllm/common/basemodel/triton_kernel/quantize_gemm_int8.py -> build/lib/lightllm/common/basemodel/triton_kernel 2024-08-27T12:42:47,216 copying lightllm/common/basemodel/triton_kernel/dequantize_gemm_int4.py -> build/lib/lightllm/common/basemodel/triton_kernel 2024-08-27T12:42:47,219 copying lightllm/common/basemodel/triton_kernel/splitfuse_copy_kv_index_to_req.py -> build/lib/lightllm/common/basemodel/triton_kernel 2024-08-27T12:42:47,221 copying lightllm/common/basemodel/triton_kernel/destindex_copy_kv.py -> build/lib/lightllm/common/basemodel/triton_kernel 2024-08-27T12:42:47,223 creating build/lib/lightllm/common/basemodel/cuda_kernel 2024-08-27T12:42:47,224 copying lightllm/common/basemodel/cuda_kernel/__init__.py -> build/lib/lightllm/common/basemodel/cuda_kernel 2024-08-27T12:42:47,226 copying lightllm/common/basemodel/cuda_kernel/ppl_wquant.py -> build/lib/lightllm/common/basemodel/cuda_kernel 2024-08-27T12:42:47,227 copying lightllm/common/basemodel/cuda_kernel/fast_llm_wquant.py -> build/lib/lightllm/common/basemodel/cuda_kernel 2024-08-27T12:42:47,229 copying lightllm/common/basemodel/cuda_kernel/ppl_awquant.py -> build/lib/lightllm/common/basemodel/cuda_kernel 2024-08-27T12:42:47,231 copying lightllm/common/basemodel/cuda_kernel/lmdeploy_wquant.py -> build/lib/lightllm/common/basemodel/cuda_kernel 2024-08-27T12:42:47,233 creating build/lib/lightllm/common/basemodel/layer_weights 2024-08-27T12:42:47,234 copying lightllm/common/basemodel/layer_weights/pre_and_post_layer_weight.py -> build/lib/lightllm/common/basemodel/layer_weights 2024-08-27T12:42:47,236 copying lightllm/common/basemodel/layer_weights/base_layer_weight.py -> build/lib/lightllm/common/basemodel/layer_weights 2024-08-27T12:42:47,237 copying lightllm/common/basemodel/layer_weights/__init__.py -> build/lib/lightllm/common/basemodel/layer_weights 2024-08-27T12:42:47,239 copying lightllm/common/basemodel/layer_weights/hf_load_utils.py -> build/lib/lightllm/common/basemodel/layer_weights 2024-08-27T12:42:47,240 copying lightllm/common/basemodel/layer_weights/transformer_layer_weight.py -> build/lib/lightllm/common/basemodel/layer_weights 2024-08-27T12:42:47,242 creating build/lib/lightllm/common/basemodel/layer_infer/template 2024-08-27T12:42:47,243 copying lightllm/common/basemodel/layer_infer/template/pre_layer_infer_template.py -> build/lib/lightllm/common/basemodel/layer_infer/template 2024-08-27T12:42:47,245 copying lightllm/common/basemodel/layer_infer/template/__init__.py -> build/lib/lightllm/common/basemodel/layer_infer/template 2024-08-27T12:42:47,246 copying lightllm/common/basemodel/layer_infer/template/transformer_layer_infer_template.py -> build/lib/lightllm/common/basemodel/layer_infer/template 2024-08-27T12:42:47,248 copying lightllm/common/basemodel/layer_infer/template/transformer_layer_infer_template_wquant.py -> build/lib/lightllm/common/basemodel/layer_infer/template 2024-08-27T12:42:47,250 copying lightllm/common/basemodel/layer_infer/template/post_layer_infer_template.py -> build/lib/lightllm/common/basemodel/layer_infer/template 2024-08-27T12:42:47,251 copying lightllm/common/basemodel/layer_infer/template/transformer_layer_infer_template_awquant.py -> build/lib/lightllm/common/basemodel/layer_infer/template 2024-08-27T12:42:47,253 creating build/lib/lightllm/server/visualserver 2024-08-27T12:42:47,254 copying lightllm/server/visualserver/__init__.py -> build/lib/lightllm/server/visualserver 2024-08-27T12:42:47,256 copying lightllm/server/visualserver/manager.py -> build/lib/lightllm/server/visualserver 2024-08-27T12:42:47,259 creating build/lib/lightllm/server/health_monitor 2024-08-27T12:42:47,259 copying lightllm/server/health_monitor/__init__.py -> build/lib/lightllm/server/health_monitor 2024-08-27T12:42:47,261 copying lightllm/server/health_monitor/manager.py -> build/lib/lightllm/server/health_monitor 2024-08-27T12:42:47,263 creating build/lib/lightllm/server/detokenization 2024-08-27T12:42:47,264 copying lightllm/server/detokenization/decode.py -> build/lib/lightllm/server/detokenization 2024-08-27T12:42:47,266 copying lightllm/server/detokenization/__init__.py -> build/lib/lightllm/server/detokenization 2024-08-27T12:42:47,267 copying lightllm/server/detokenization/manager.py -> build/lib/lightllm/server/detokenization 2024-08-27T12:42:47,270 creating build/lib/lightllm/server/embed_cache 2024-08-27T12:42:47,271 copying lightllm/server/embed_cache/__init__.py -> build/lib/lightllm/server/embed_cache 2024-08-27T12:42:47,272 copying lightllm/server/embed_cache/manager.py -> build/lib/lightllm/server/embed_cache 2024-08-27T12:42:47,274 copying lightllm/server/embed_cache/interface.py -> build/lib/lightllm/server/embed_cache 2024-08-27T12:42:47,276 copying lightllm/server/embed_cache/utils.py -> build/lib/lightllm/server/embed_cache 2024-08-27T12:42:47,278 creating build/lib/lightllm/server/httpserver 2024-08-27T12:42:47,279 copying lightllm/server/httpserver/__init__.py -> build/lib/lightllm/server/httpserver 2024-08-27T12:42:47,280 copying lightllm/server/httpserver/manager.py -> build/lib/lightllm/server/httpserver 2024-08-27T12:42:47,283 creating build/lib/lightllm/server/router 2024-08-27T12:42:47,284 copying lightllm/server/router/stats.py -> build/lib/lightllm/server/router 2024-08-27T12:42:47,286 copying lightllm/server/router/pause_strategy.py -> build/lib/lightllm/server/router 2024-08-27T12:42:47,287 copying lightllm/server/router/token_load.py -> build/lib/lightllm/server/router 2024-08-27T12:42:47,289 copying lightllm/server/router/__init__.py -> build/lib/lightllm/server/router 2024-08-27T12:42:47,290 copying lightllm/server/router/manager.py -> build/lib/lightllm/server/router 2024-08-27T12:42:47,293 creating build/lib/lightllm/server/visualserver/model_infer 2024-08-27T12:42:47,294 copying lightllm/server/visualserver/model_infer/model_rpc.py -> build/lib/lightllm/server/visualserver/model_infer 2024-08-27T12:42:47,296 copying lightllm/server/visualserver/model_infer/__init__.py -> build/lib/lightllm/server/visualserver/model_infer 2024-08-27T12:42:47,297 creating build/lib/lightllm/server/embed_cache/impl 2024-08-27T12:42:47,299 copying lightllm/server/embed_cache/impl/naive_memory_cache.py -> build/lib/lightllm/server/embed_cache/impl 2024-08-27T12:42:47,300 copying lightllm/server/embed_cache/impl/__init__.py -> build/lib/lightllm/server/embed_cache/impl 2024-08-27T12:42:47,302 creating build/lib/lightllm/server/router/model_infer 2024-08-27T12:42:47,303 copying lightllm/server/router/model_infer/infer_batch.py -> build/lib/lightllm/server/router/model_infer 2024-08-27T12:42:47,306 copying lightllm/server/router/model_infer/model_rpc.py -> build/lib/lightllm/server/router/model_infer 2024-08-27T12:42:47,308 copying lightllm/server/router/model_infer/__init__.py -> build/lib/lightllm/server/router/model_infer 2024-08-27T12:42:47,309 creating build/lib/lightllm/server/router/dynamic_prompt 2024-08-27T12:42:47,310 copying lightllm/server/router/dynamic_prompt/__init__.py -> build/lib/lightllm/server/router/dynamic_prompt 2024-08-27T12:42:47,312 copying lightllm/server/router/dynamic_prompt/radix_cache.py -> build/lib/lightllm/server/router/dynamic_prompt 2024-08-27T12:42:47,314 copying lightllm/server/router/dynamic_prompt/shared_arr.py -> build/lib/lightllm/server/router/dynamic_prompt 2024-08-27T12:42:47,317 creating build/lib/lightllm/server/router/req_queue 2024-08-27T12:42:47,318 copying lightllm/server/router/req_queue/base_queue.py -> build/lib/lightllm/server/router/req_queue 2024-08-27T12:42:47,320 copying lightllm/server/router/req_queue/__init__.py -> build/lib/lightllm/server/router/req_queue 2024-08-27T12:42:47,322 creating build/lib/lightllm/server/router/model_infer/mode_backend 2024-08-27T12:42:47,323 copying lightllm/server/router/model_infer/mode_backend/__init__.py -> build/lib/lightllm/server/router/model_infer/mode_backend 2024-08-27T12:42:47,325 copying lightllm/server/router/model_infer/mode_backend/base_backend.py -> build/lib/lightllm/server/router/model_infer/mode_backend 2024-08-27T12:42:47,327 creating build/lib/lightllm/server/router/model_infer/mode_backend/diverse_backend 2024-08-27T12:42:47,328 copying lightllm/server/router/model_infer/mode_backend/diverse_backend/post_process.py -> build/lib/lightllm/server/router/model_infer/mode_backend/diverse_backend 2024-08-27T12:42:47,330 copying lightllm/server/router/model_infer/mode_backend/diverse_backend/__init__.py -> build/lib/lightllm/server/router/model_infer/mode_backend/diverse_backend 2024-08-27T12:42:47,332 copying lightllm/server/router/model_infer/mode_backend/diverse_backend/impl.py -> build/lib/lightllm/server/router/model_infer/mode_backend/diverse_backend 2024-08-27T12:42:47,334 creating build/lib/lightllm/server/router/model_infer/mode_backend/beamsearch 2024-08-27T12:42:47,335 copying lightllm/server/router/model_infer/mode_backend/beamsearch/pre_process.py -> build/lib/lightllm/server/router/model_infer/mode_backend/beamsearch 2024-08-27T12:42:47,337 copying lightllm/server/router/model_infer/mode_backend/beamsearch/post_process.py -> build/lib/lightllm/server/router/model_infer/mode_backend/beamsearch 2024-08-27T12:42:47,340 copying lightllm/server/router/model_infer/mode_backend/beamsearch/__init__.py -> build/lib/lightllm/server/router/model_infer/mode_backend/beamsearch 2024-08-27T12:42:47,341 copying lightllm/server/router/model_infer/mode_backend/beamsearch/impl.py -> build/lib/lightllm/server/router/model_infer/mode_backend/beamsearch 2024-08-27T12:42:47,344 creating build/lib/lightllm/server/router/model_infer/mode_backend/continues_batch 2024-08-27T12:42:47,345 copying lightllm/server/router/model_infer/mode_backend/continues_batch/pre_process.py -> build/lib/lightllm/server/router/model_infer/mode_backend/continues_batch 2024-08-27T12:42:47,347 copying lightllm/server/router/model_infer/mode_backend/continues_batch/post_process.py -> build/lib/lightllm/server/router/model_infer/mode_backend/continues_batch 2024-08-27T12:42:47,349 copying lightllm/server/router/model_infer/mode_backend/continues_batch/__init__.py -> build/lib/lightllm/server/router/model_infer/mode_backend/continues_batch 2024-08-27T12:42:47,351 copying lightllm/server/router/model_infer/mode_backend/continues_batch/impl.py -> build/lib/lightllm/server/router/model_infer/mode_backend/continues_batch 2024-08-27T12:42:47,353 copying lightllm/server/router/model_infer/mode_backend/continues_batch/impl_for_return_all_prompt_logprobs.py -> build/lib/lightllm/server/router/model_infer/mode_backend/continues_batch 2024-08-27T12:42:47,355 creating build/lib/lightllm/server/router/model_infer/mode_backend/splitfuse 2024-08-27T12:42:47,356 copying lightllm/server/router/model_infer/mode_backend/splitfuse/pre_process.py -> build/lib/lightllm/server/router/model_infer/mode_backend/splitfuse 2024-08-27T12:42:47,358 copying lightllm/server/router/model_infer/mode_backend/splitfuse/__init__.py -> build/lib/lightllm/server/router/model_infer/mode_backend/splitfuse 2024-08-27T12:42:47,359 copying lightllm/server/router/model_infer/mode_backend/splitfuse/impl.py -> build/lib/lightllm/server/router/model_infer/mode_backend/splitfuse 2024-08-27T12:42:47,361 creating build/lib/lightllm/server/router/req_queue/continues_batch 2024-08-27T12:42:47,362 copying lightllm/server/router/req_queue/continues_batch/__init__.py -> build/lib/lightllm/server/router/req_queue/continues_batch 2024-08-27T12:42:47,364 copying lightllm/server/router/req_queue/continues_batch/impl.py -> build/lib/lightllm/server/router/req_queue/continues_batch 2024-08-27T12:42:47,366 copying lightllm/server/router/req_queue/continues_batch/beam_impl.py -> build/lib/lightllm/server/router/req_queue/continues_batch 2024-08-27T12:42:47,368 creating build/lib/lightllm/server/router/req_queue/splitfuse 2024-08-27T12:42:47,369 copying lightllm/server/router/req_queue/splitfuse/__init__.py -> build/lib/lightllm/server/router/req_queue/splitfuse 2024-08-27T12:42:47,371 copying lightllm/server/router/req_queue/splitfuse/impl.py -> build/lib/lightllm/server/router/req_queue/splitfuse 2024-08-27T12:42:47,373 creating build/lib/lightllm/models/internlm_xcomposer 2024-08-27T12:42:47,375 copying lightllm/models/internlm_xcomposer/__init__.py -> build/lib/lightllm/models/internlm_xcomposer 2024-08-27T12:42:47,376 copying lightllm/models/internlm_xcomposer/internlm_visual.py -> build/lib/lightllm/models/internlm_xcomposer 2024-08-27T12:42:47,378 copying lightllm/models/internlm_xcomposer/infer_struct.py -> build/lib/lightllm/models/internlm_xcomposer 2024-08-27T12:42:47,380 copying lightllm/models/internlm_xcomposer/model.py -> build/lib/lightllm/models/internlm_xcomposer 2024-08-27T12:42:47,382 creating build/lib/lightllm/models/qwen2 2024-08-27T12:42:47,383 copying lightllm/models/qwen2/__init__.py -> build/lib/lightllm/models/qwen2 2024-08-27T12:42:47,385 copying lightllm/models/qwen2/infer_struct.py -> build/lib/lightllm/models/qwen2 2024-08-27T12:42:47,387 copying lightllm/models/qwen2/model.py -> build/lib/lightllm/models/qwen2 2024-08-27T12:42:47,389 creating build/lib/lightllm/models/gemma_2b 2024-08-27T12:42:47,390 copying lightllm/models/gemma_2b/__init__.py -> build/lib/lightllm/models/gemma_2b 2024-08-27T12:42:47,392 copying lightllm/models/gemma_2b/model.py -> build/lib/lightllm/models/gemma_2b 2024-08-27T12:42:47,394 creating build/lib/lightllm/models/llama_quik 2024-08-27T12:42:47,395 copying lightllm/models/llama_quik/__init__.py -> build/lib/lightllm/models/llama_quik 2024-08-27T12:42:47,397 copying lightllm/models/llama_quik/model.py -> build/lib/lightllm/models/llama_quik 2024-08-27T12:42:47,399 creating build/lib/lightllm/models/baichuan7b 2024-08-27T12:42:47,400 copying lightllm/models/baichuan7b/__init__.py -> build/lib/lightllm/models/baichuan7b 2024-08-27T12:42:47,402 copying lightllm/models/baichuan7b/model.py -> build/lib/lightllm/models/baichuan7b 2024-08-27T12:42:47,404 creating build/lib/lightllm/models/internlm2_wquant 2024-08-27T12:42:47,405 copying lightllm/models/internlm2_wquant/__init__.py -> build/lib/lightllm/models/internlm2_wquant 2024-08-27T12:42:47,406 copying lightllm/models/internlm2_wquant/model.py -> build/lib/lightllm/models/internlm2_wquant 2024-08-27T12:42:47,408 creating build/lib/lightllm/models/yi 2024-08-27T12:42:47,409 copying lightllm/models/yi/__init__.py -> build/lib/lightllm/models/yi 2024-08-27T12:42:47,410 copying lightllm/models/yi/model.py -> build/lib/lightllm/models/yi 2024-08-27T12:42:47,412 creating build/lib/lightllm/models/baichuan2_7b 2024-08-27T12:42:47,413 copying lightllm/models/baichuan2_7b/__init__.py -> build/lib/lightllm/models/baichuan2_7b 2024-08-27T12:42:47,415 copying lightllm/models/baichuan2_7b/model.py -> build/lib/lightllm/models/baichuan2_7b 2024-08-27T12:42:47,417 creating build/lib/lightllm/models/llama_awquant 2024-08-27T12:42:47,417 copying lightllm/models/llama_awquant/__init__.py -> build/lib/lightllm/models/llama_awquant 2024-08-27T12:42:47,419 copying lightllm/models/llama_awquant/model.py -> build/lib/lightllm/models/llama_awquant 2024-08-27T12:42:47,421 creating build/lib/lightllm/models/baichuan13b 2024-08-27T12:42:47,422 copying lightllm/models/baichuan13b/__init__.py -> build/lib/lightllm/models/baichuan13b 2024-08-27T12:42:47,423 copying lightllm/models/baichuan13b/model.py -> build/lib/lightllm/models/baichuan13b 2024-08-27T12:42:47,425 creating build/lib/lightllm/models/qwen_wquant 2024-08-27T12:42:47,426 copying lightllm/models/qwen_wquant/__init__.py -> build/lib/lightllm/models/qwen_wquant 2024-08-27T12:42:47,427 copying lightllm/models/qwen_wquant/model.py -> build/lib/lightllm/models/qwen_wquant 2024-08-27T12:42:47,430 creating build/lib/lightllm/models/llama_wquant 2024-08-27T12:42:47,431 copying lightllm/models/llama_wquant/__init__.py -> build/lib/lightllm/models/llama_wquant 2024-08-27T12:42:47,432 copying lightllm/models/llama_wquant/model.py -> build/lib/lightllm/models/llama_wquant 2024-08-27T12:42:47,435 creating build/lib/lightllm/models/internlm 2024-08-27T12:42:47,436 copying lightllm/models/internlm/__init__.py -> build/lib/lightllm/models/internlm 2024-08-27T12:42:47,437 copying lightllm/models/internlm/model.py -> build/lib/lightllm/models/internlm 2024-08-27T12:42:47,439 creating build/lib/lightllm/models/internlm_wquant 2024-08-27T12:42:47,440 copying lightllm/models/internlm_wquant/__init__.py -> build/lib/lightllm/models/internlm_wquant 2024-08-27T12:42:47,442 copying lightllm/models/internlm_wquant/model.py -> build/lib/lightllm/models/internlm_wquant 2024-08-27T12:42:47,444 creating build/lib/lightllm/models/mistral 2024-08-27T12:42:47,445 copying lightllm/models/mistral/__init__.py -> build/lib/lightllm/models/mistral 2024-08-27T12:42:47,446 copying lightllm/models/mistral/infer_struct.py -> build/lib/lightllm/models/mistral 2024-08-27T12:42:47,448 copying lightllm/models/mistral/model.py -> build/lib/lightllm/models/mistral 2024-08-27T12:42:47,450 creating build/lib/lightllm/models/starcoder_wquant 2024-08-27T12:42:47,451 copying lightllm/models/starcoder_wquant/__init__.py -> build/lib/lightllm/models/starcoder_wquant 2024-08-27T12:42:47,453 copying lightllm/models/starcoder_wquant/model.py -> build/lib/lightllm/models/starcoder_wquant 2024-08-27T12:42:47,455 creating build/lib/lightllm/models/minicpm 2024-08-27T12:42:47,456 copying lightllm/models/minicpm/__init__.py -> build/lib/lightllm/models/minicpm 2024-08-27T12:42:47,457 copying lightllm/models/minicpm/model.py -> build/lib/lightllm/models/minicpm 2024-08-27T12:42:47,460 creating build/lib/lightllm/models/chatglm2 2024-08-27T12:42:47,461 copying lightllm/models/chatglm2/__init__.py -> build/lib/lightllm/models/chatglm2 2024-08-27T12:42:47,462 copying lightllm/models/chatglm2/model.py -> build/lib/lightllm/models/chatglm2 2024-08-27T12:42:47,464 creating build/lib/lightllm/models/internlm2 2024-08-27T12:42:47,465 copying lightllm/models/internlm2/__init__.py -> build/lib/lightllm/models/internlm2 2024-08-27T12:42:47,467 copying lightllm/models/internlm2/model.py -> build/lib/lightllm/models/internlm2 2024-08-27T12:42:47,469 creating build/lib/lightllm/models/starcoder2 2024-08-27T12:42:47,469 copying lightllm/models/starcoder2/__init__.py -> build/lib/lightllm/models/starcoder2 2024-08-27T12:42:47,471 copying lightllm/models/starcoder2/model.py -> build/lib/lightllm/models/starcoder2 2024-08-27T12:42:47,473 creating build/lib/lightllm/models/stablelm 2024-08-27T12:42:47,474 copying lightllm/models/stablelm/__init__.py -> build/lib/lightllm/models/stablelm 2024-08-27T12:42:47,475 copying lightllm/models/stablelm/model.py -> build/lib/lightllm/models/stablelm 2024-08-27T12:42:47,477 creating build/lib/lightllm/models/qwen 2024-08-27T12:42:47,478 copying lightllm/models/qwen/__init__.py -> build/lib/lightllm/models/qwen 2024-08-27T12:42:47,480 copying lightllm/models/qwen/infer_struct.py -> build/lib/lightllm/models/qwen 2024-08-27T12:42:47,481 copying lightllm/models/qwen/model.py -> build/lib/lightllm/models/qwen 2024-08-27T12:42:47,484 creating build/lib/lightllm/models/llama 2024-08-27T12:42:47,484 copying lightllm/models/llama/splitfuse_infer_struct.py -> build/lib/lightllm/models/llama 2024-08-27T12:42:47,486 copying lightllm/models/llama/__init__.py -> build/lib/lightllm/models/llama 2024-08-27T12:42:47,488 copying lightllm/models/llama/yarn_rotary_utils.py -> build/lib/lightllm/models/llama 2024-08-27T12:42:47,489 copying lightllm/models/llama/infer_struct.py -> build/lib/lightllm/models/llama 2024-08-27T12:42:47,491 copying lightllm/models/llama/model.py -> build/lib/lightllm/models/llama 2024-08-27T12:42:47,493 creating build/lib/lightllm/models/bloom 2024-08-27T12:42:47,494 copying lightllm/models/bloom/__init__.py -> build/lib/lightllm/models/bloom 2024-08-27T12:42:47,496 copying lightllm/models/bloom/model.py -> build/lib/lightllm/models/bloom 2024-08-27T12:42:47,498 creating build/lib/lightllm/models/baichuan2_13b 2024-08-27T12:42:47,499 copying lightllm/models/baichuan2_13b/__init__.py -> build/lib/lightllm/models/baichuan2_13b 2024-08-27T12:42:47,500 copying lightllm/models/baichuan2_13b/model.py -> build/lib/lightllm/models/baichuan2_13b 2024-08-27T12:42:47,503 creating build/lib/lightllm/models/starcoder 2024-08-27T12:42:47,504 copying lightllm/models/starcoder/__init__.py -> build/lib/lightllm/models/starcoder 2024-08-27T12:42:47,505 copying lightllm/models/starcoder/infer_struct.py -> build/lib/lightllm/models/starcoder 2024-08-27T12:42:47,507 copying lightllm/models/starcoder/model.py -> build/lib/lightllm/models/starcoder 2024-08-27T12:42:47,509 creating build/lib/lightllm/models/qwen_vl 2024-08-27T12:42:47,511 copying lightllm/models/qwen_vl/__init__.py -> build/lib/lightllm/models/qwen_vl 2024-08-27T12:42:47,512 copying lightllm/models/qwen_vl/qwen_visual.py -> build/lib/lightllm/models/qwen_vl 2024-08-27T12:42:47,514 copying lightllm/models/qwen_vl/model.py -> build/lib/lightllm/models/qwen_vl 2024-08-27T12:42:47,517 creating build/lib/lightllm/models/llava 2024-08-27T12:42:47,518 copying lightllm/models/llava/llava_visual.py -> build/lib/lightllm/models/llava 2024-08-27T12:42:47,520 copying lightllm/models/llava/__init__.py -> build/lib/lightllm/models/llava 2024-08-27T12:42:47,521 copying lightllm/models/llava/model.py -> build/lib/lightllm/models/llava 2024-08-27T12:42:47,523 creating build/lib/lightllm/models/mixtral 2024-08-27T12:42:47,524 copying lightllm/models/mixtral/__init__.py -> build/lib/lightllm/models/mixtral 2024-08-27T12:42:47,526 copying lightllm/models/mixtral/infer_struct.py -> build/lib/lightllm/models/mixtral 2024-08-27T12:42:47,528 copying lightllm/models/mixtral/model.py -> build/lib/lightllm/models/mixtral 2024-08-27T12:42:47,530 creating build/lib/lightllm/models/internlm_xcomposer/layer_infer 2024-08-27T12:42:47,531 copying lightllm/models/internlm_xcomposer/layer_infer/__init__.py -> build/lib/lightllm/models/internlm_xcomposer/layer_infer 2024-08-27T12:42:47,532 copying lightllm/models/internlm_xcomposer/layer_infer/transformer_layer_infer.py -> build/lib/lightllm/models/internlm_xcomposer/layer_infer 2024-08-27T12:42:47,535 creating build/lib/lightllm/models/internlm_xcomposer/layer_weights 2024-08-27T12:42:47,535 copying lightllm/models/internlm_xcomposer/layer_weights/__init__.py -> build/lib/lightllm/models/internlm_xcomposer/layer_weights 2024-08-27T12:42:47,537 copying lightllm/models/internlm_xcomposer/layer_weights/transformer_layer_weight.py -> build/lib/lightllm/models/internlm_xcomposer/layer_weights 2024-08-27T12:42:47,539 creating build/lib/lightllm/models/qwen2/layer_infer 2024-08-27T12:42:47,540 copying lightllm/models/qwen2/layer_infer/__init__.py -> build/lib/lightllm/models/qwen2/layer_infer 2024-08-27T12:42:47,541 copying lightllm/models/qwen2/layer_infer/transformer_layer_infer.py -> build/lib/lightllm/models/qwen2/layer_infer 2024-08-27T12:42:47,544 creating build/lib/lightllm/models/qwen2/layer_weights 2024-08-27T12:42:47,545 copying lightllm/models/qwen2/layer_weights/pre_and_post_layer_weight.py -> build/lib/lightllm/models/qwen2/layer_weights 2024-08-27T12:42:47,546 copying lightllm/models/qwen2/layer_weights/__init__.py -> build/lib/lightllm/models/qwen2/layer_weights 2024-08-27T12:42:47,548 copying lightllm/models/qwen2/layer_weights/transformer_layer_weight.py -> build/lib/lightllm/models/qwen2/layer_weights 2024-08-27T12:42:47,550 creating build/lib/lightllm/models/gemma_2b/layer_infer 2024-08-27T12:42:47,551 copying lightllm/models/gemma_2b/layer_infer/__init__.py -> build/lib/lightllm/models/gemma_2b/layer_infer 2024-08-27T12:42:47,552 copying lightllm/models/gemma_2b/layer_infer/transformer_layer_infer.py -> build/lib/lightllm/models/gemma_2b/layer_infer 2024-08-27T12:42:47,554 copying lightllm/models/gemma_2b/layer_infer/pre_layer_infer.py -> build/lib/lightllm/models/gemma_2b/layer_infer 2024-08-27T12:42:47,556 creating build/lib/lightllm/models/gemma_2b/triton_kernel 2024-08-27T12:42:47,557 copying lightllm/models/gemma_2b/triton_kernel/__init__.py -> build/lib/lightllm/models/gemma_2b/triton_kernel 2024-08-27T12:42:47,558 copying lightllm/models/gemma_2b/triton_kernel/gelu_and_mul.py -> build/lib/lightllm/models/gemma_2b/triton_kernel 2024-08-27T12:42:47,561 creating build/lib/lightllm/models/gemma_2b/layer_weights 2024-08-27T12:42:47,562 copying lightllm/models/gemma_2b/layer_weights/pre_and_post_layer_weight.py -> build/lib/lightllm/models/gemma_2b/layer_weights 2024-08-27T12:42:47,564 copying lightllm/models/gemma_2b/layer_weights/__init__.py -> build/lib/lightllm/models/gemma_2b/layer_weights 2024-08-27T12:42:47,565 copying lightllm/models/gemma_2b/layer_weights/transformer_layer_weight.py -> build/lib/lightllm/models/gemma_2b/layer_weights 2024-08-27T12:42:47,567 creating build/lib/lightllm/models/llama_quik/layer_infer 2024-08-27T12:42:47,569 copying lightllm/models/llama_quik/layer_infer/__init__.py -> build/lib/lightllm/models/llama_quik/layer_infer 2024-08-27T12:42:47,570 copying lightllm/models/llama_quik/layer_infer/transformer_layer_infer.py -> build/lib/lightllm/models/llama_quik/layer_infer 2024-08-27T12:42:47,573 creating build/lib/lightllm/models/llama_quik/cuda_kernel 2024-08-27T12:42:47,573 copying lightllm/models/llama_quik/cuda_kernel/quik_awquant.py -> build/lib/lightllm/models/llama_quik/cuda_kernel 2024-08-27T12:42:47,575 copying lightllm/models/llama_quik/cuda_kernel/__init__.py -> build/lib/lightllm/models/llama_quik/cuda_kernel 2024-08-27T12:42:47,577 creating build/lib/lightllm/models/llama_quik/layer_weights 2024-08-27T12:42:47,579 copying lightllm/models/llama_quik/layer_weights/__init__.py -> build/lib/lightllm/models/llama_quik/layer_weights 2024-08-27T12:42:47,580 copying lightllm/models/llama_quik/layer_weights/qlinear.py -> build/lib/lightllm/models/llama_quik/layer_weights 2024-08-27T12:42:47,582 copying lightllm/models/llama_quik/layer_weights/transformer_layer_weight.py -> build/lib/lightllm/models/llama_quik/layer_weights 2024-08-27T12:42:47,585 creating build/lib/lightllm/models/baichuan7b/layer_weights 2024-08-27T12:42:47,586 copying lightllm/models/baichuan7b/layer_weights/__init__.py -> build/lib/lightllm/models/baichuan7b/layer_weights 2024-08-27T12:42:47,587 copying lightllm/models/baichuan7b/layer_weights/transformer_layer_weight.py -> build/lib/lightllm/models/baichuan7b/layer_weights 2024-08-27T12:42:47,589 creating build/lib/lightllm/models/internlm2_wquant/layer_weights 2024-08-27T12:42:47,590 copying lightllm/models/internlm2_wquant/layer_weights/__init__.py -> build/lib/lightllm/models/internlm2_wquant/layer_weights 2024-08-27T12:42:47,591 copying lightllm/models/internlm2_wquant/layer_weights/transformer_layer_weight.py -> build/lib/lightllm/models/internlm2_wquant/layer_weights 2024-08-27T12:42:47,594 creating build/lib/lightllm/models/yi/layer_weights 2024-08-27T12:42:47,595 copying lightllm/models/yi/layer_weights/__init__.py -> build/lib/lightllm/models/yi/layer_weights 2024-08-27T12:42:47,596 copying lightllm/models/yi/layer_weights/transformer_layer_weight.py -> build/lib/lightllm/models/yi/layer_weights 2024-08-27T12:42:47,598 creating build/lib/lightllm/models/baichuan2_7b/layer_infer 2024-08-27T12:42:47,599 copying lightllm/models/baichuan2_7b/layer_infer/__init__.py -> build/lib/lightllm/models/baichuan2_7b/layer_infer 2024-08-27T12:42:47,600 copying lightllm/models/baichuan2_7b/layer_infer/transformer_layer_infer.py -> build/lib/lightllm/models/baichuan2_7b/layer_infer 2024-08-27T12:42:47,602 creating build/lib/lightllm/models/baichuan2_7b/layer_weights 2024-08-27T12:42:47,603 copying lightllm/models/baichuan2_7b/layer_weights/pre_and_post_layer_weight.py -> build/lib/lightllm/models/baichuan2_7b/layer_weights 2024-08-27T12:42:47,605 copying lightllm/models/baichuan2_7b/layer_weights/__init__.py -> build/lib/lightllm/models/baichuan2_7b/layer_weights 2024-08-27T12:42:47,607 creating build/lib/lightllm/models/llama_awquant/layer_infer 2024-08-27T12:42:47,608 copying lightllm/models/llama_awquant/layer_infer/__init__.py -> build/lib/lightllm/models/llama_awquant/layer_infer 2024-08-27T12:42:47,609 copying lightllm/models/llama_awquant/layer_infer/transformer_layer_infer.py -> build/lib/lightllm/models/llama_awquant/layer_infer 2024-08-27T12:42:47,612 creating build/lib/lightllm/models/llama_awquant/layer_weights 2024-08-27T12:42:47,612 copying lightllm/models/llama_awquant/layer_weights/__init__.py -> build/lib/lightllm/models/llama_awquant/layer_weights 2024-08-27T12:42:47,614 copying lightllm/models/llama_awquant/layer_weights/transformer_layer_weight.py -> build/lib/lightllm/models/llama_awquant/layer_weights 2024-08-27T12:42:47,617 creating build/lib/lightllm/models/baichuan13b/layer_infer 2024-08-27T12:42:47,617 copying lightllm/models/baichuan13b/layer_infer/__init__.py -> build/lib/lightllm/models/baichuan13b/layer_infer 2024-08-27T12:42:47,619 copying lightllm/models/baichuan13b/layer_infer/transformer_layer_infer.py -> build/lib/lightllm/models/baichuan13b/layer_infer 2024-08-27T12:42:47,621 creating build/lib/lightllm/models/baichuan13b/layer_weights 2024-08-27T12:42:47,622 copying lightllm/models/baichuan13b/layer_weights/__init__.py -> build/lib/lightllm/models/baichuan13b/layer_weights 2024-08-27T12:42:47,624 copying lightllm/models/baichuan13b/layer_weights/transformer_layer_weight.py -> build/lib/lightllm/models/baichuan13b/layer_weights 2024-08-27T12:42:47,626 creating build/lib/lightllm/models/qwen_wquant/layer_infer 2024-08-27T12:42:47,627 copying lightllm/models/qwen_wquant/layer_infer/__init__.py -> build/lib/lightllm/models/qwen_wquant/layer_infer 2024-08-27T12:42:47,628 copying lightllm/models/qwen_wquant/layer_infer/transformer_layer_infer.py -> build/lib/lightllm/models/qwen_wquant/layer_infer 2024-08-27T12:42:47,631 creating build/lib/lightllm/models/qwen_wquant/layer_weights 2024-08-27T12:42:47,631 copying lightllm/models/qwen_wquant/layer_weights/__init__.py -> build/lib/lightllm/models/qwen_wquant/layer_weights 2024-08-27T12:42:47,633 copying lightllm/models/qwen_wquant/layer_weights/transformer_layer_weight.py -> build/lib/lightllm/models/qwen_wquant/layer_weights 2024-08-27T12:42:47,636 creating build/lib/lightllm/models/llama_wquant/layer_infer 2024-08-27T12:42:47,636 copying lightllm/models/llama_wquant/layer_infer/__init__.py -> build/lib/lightllm/models/llama_wquant/layer_infer 2024-08-27T12:42:47,638 copying lightllm/models/llama_wquant/layer_infer/transformer_layer_infer.py -> build/lib/lightllm/models/llama_wquant/layer_infer 2024-08-27T12:42:47,640 creating build/lib/lightllm/models/llama_wquant/layer_weights 2024-08-27T12:42:47,641 copying lightllm/models/llama_wquant/layer_weights/__init__.py -> build/lib/lightllm/models/llama_wquant/layer_weights 2024-08-27T12:42:47,643 copying lightllm/models/llama_wquant/layer_weights/transformer_layer_weight.py -> build/lib/lightllm/models/llama_wquant/layer_weights 2024-08-27T12:42:47,645 creating build/lib/lightllm/models/internlm/layer_infer 2024-08-27T12:42:47,646 copying lightllm/models/internlm/layer_infer/__init__.py -> build/lib/lightllm/models/internlm/layer_infer 2024-08-27T12:42:47,648 copying lightllm/models/internlm/layer_infer/transformer_layer_infer.py -> build/lib/lightllm/models/internlm/layer_infer 2024-08-27T12:42:47,651 creating build/lib/lightllm/models/internlm/layer_weights 2024-08-27T12:42:47,652 copying lightllm/models/internlm/layer_weights/__init__.py -> build/lib/lightllm/models/internlm/layer_weights 2024-08-27T12:42:47,653 copying lightllm/models/internlm/layer_weights/transformer_layer_weight.py -> build/lib/lightllm/models/internlm/layer_weights 2024-08-27T12:42:47,656 creating build/lib/lightllm/models/internlm_wquant/layer_infer 2024-08-27T12:42:47,657 copying lightllm/models/internlm_wquant/layer_infer/__init__.py -> build/lib/lightllm/models/internlm_wquant/layer_infer 2024-08-27T12:42:47,659 copying lightllm/models/internlm_wquant/layer_infer/transformer_layer_infer.py -> build/lib/lightllm/models/internlm_wquant/layer_infer 2024-08-27T12:42:47,661 creating build/lib/lightllm/models/internlm_wquant/layer_weights 2024-08-27T12:42:47,662 copying lightllm/models/internlm_wquant/layer_weights/__init__.py -> build/lib/lightllm/models/internlm_wquant/layer_weights 2024-08-27T12:42:47,663 copying lightllm/models/internlm_wquant/layer_weights/transformer_layer_weight.py -> build/lib/lightllm/models/internlm_wquant/layer_weights 2024-08-27T12:42:47,666 creating build/lib/lightllm/models/mistral/layer_infer 2024-08-27T12:42:47,667 copying lightllm/models/mistral/layer_infer/__init__.py -> build/lib/lightllm/models/mistral/layer_infer 2024-08-27T12:42:47,669 copying lightllm/models/mistral/layer_infer/transformer_layer_infer.py -> build/lib/lightllm/models/mistral/layer_infer 2024-08-27T12:42:47,672 creating build/lib/lightllm/models/mistral/triton_kernel 2024-08-27T12:42:47,673 copying lightllm/models/mistral/triton_kernel/token_attention_nopad_reduceV.py -> build/lib/lightllm/models/mistral/triton_kernel 2024-08-27T12:42:47,674 copying lightllm/models/mistral/triton_kernel/__init__.py -> build/lib/lightllm/models/mistral/triton_kernel 2024-08-27T12:42:47,676 copying lightllm/models/mistral/triton_kernel/context_flashattention_nopad.py -> build/lib/lightllm/models/mistral/triton_kernel 2024-08-27T12:42:47,678 copying lightllm/models/mistral/triton_kernel/token_attention_nopad_att1.py -> build/lib/lightllm/models/mistral/triton_kernel 2024-08-27T12:42:47,680 copying lightllm/models/mistral/triton_kernel/token_attention_softmax_and_reducev.py -> build/lib/lightllm/models/mistral/triton_kernel 2024-08-27T12:42:47,682 creating build/lib/lightllm/models/starcoder_wquant/layer_infer 2024-08-27T12:42:47,683 copying lightllm/models/starcoder_wquant/layer_infer/__init__.py -> build/lib/lightllm/models/starcoder_wquant/layer_infer 2024-08-27T12:42:47,685 copying lightllm/models/starcoder_wquant/layer_infer/transformer_layer_infer.py -> build/lib/lightllm/models/starcoder_wquant/layer_infer 2024-08-27T12:42:47,687 creating build/lib/lightllm/models/starcoder_wquant/layer_weights 2024-08-27T12:42:47,688 copying lightllm/models/starcoder_wquant/layer_weights/__init__.py -> build/lib/lightllm/models/starcoder_wquant/layer_weights 2024-08-27T12:42:47,690 copying lightllm/models/starcoder_wquant/layer_weights/transformer_layer_weight.py -> build/lib/lightllm/models/starcoder_wquant/layer_weights 2024-08-27T12:42:47,693 creating build/lib/lightllm/models/minicpm/layer_infer 2024-08-27T12:42:47,693 copying lightllm/models/minicpm/layer_infer/__init__.py -> build/lib/lightllm/models/minicpm/layer_infer 2024-08-27T12:42:47,695 copying lightllm/models/minicpm/layer_infer/transformer_layer_infer.py -> build/lib/lightllm/models/minicpm/layer_infer 2024-08-27T12:42:47,697 creating build/lib/lightllm/models/minicpm/layer_weights 2024-08-27T12:42:47,698 copying lightllm/models/minicpm/layer_weights/pre_and_post_layer_weight.py -> build/lib/lightllm/models/minicpm/layer_weights 2024-08-27T12:42:47,700 copying lightllm/models/minicpm/layer_weights/__init__.py -> build/lib/lightllm/models/minicpm/layer_weights 2024-08-27T12:42:47,702 copying lightllm/models/minicpm/layer_weights/transformer_layer_weight.py -> build/lib/lightllm/models/minicpm/layer_weights 2024-08-27T12:42:47,704 creating build/lib/lightllm/models/chatglm2/layer_infer 2024-08-27T12:42:47,705 copying lightllm/models/chatglm2/layer_infer/__init__.py -> build/lib/lightllm/models/chatglm2/layer_infer 2024-08-27T12:42:47,707 copying lightllm/models/chatglm2/layer_infer/transformer_layer_infer.py -> build/lib/lightllm/models/chatglm2/layer_infer 2024-08-27T12:42:47,709 creating build/lib/lightllm/models/chatglm2/triton_kernel 2024-08-27T12:42:47,711 copying lightllm/models/chatglm2/triton_kernel/__init__.py -> build/lib/lightllm/models/chatglm2/triton_kernel 2024-08-27T12:42:47,712 copying lightllm/models/chatglm2/triton_kernel/rotary_emb.py -> build/lib/lightllm/models/chatglm2/triton_kernel 2024-08-27T12:42:47,715 creating build/lib/lightllm/models/chatglm2/layer_weights 2024-08-27T12:42:47,716 copying lightllm/models/chatglm2/layer_weights/pre_and_post_layer_weight.py -> build/lib/lightllm/models/chatglm2/layer_weights 2024-08-27T12:42:47,718 copying lightllm/models/chatglm2/layer_weights/__init__.py -> build/lib/lightllm/models/chatglm2/layer_weights 2024-08-27T12:42:47,719 copying lightllm/models/chatglm2/layer_weights/transformer_layer_weight.py -> build/lib/lightllm/models/chatglm2/layer_weights 2024-08-27T12:42:47,722 creating build/lib/lightllm/models/internlm2/layer_weights 2024-08-27T12:42:47,723 copying lightllm/models/internlm2/layer_weights/pre_and_post_layer_weight.py -> build/lib/lightllm/models/internlm2/layer_weights 2024-08-27T12:42:47,725 copying lightllm/models/internlm2/layer_weights/__init__.py -> build/lib/lightllm/models/internlm2/layer_weights 2024-08-27T12:42:47,727 copying lightllm/models/internlm2/layer_weights/transformer_layer_weight.py -> build/lib/lightllm/models/internlm2/layer_weights 2024-08-27T12:42:47,730 creating build/lib/lightllm/models/starcoder2/layer_infer 2024-08-27T12:42:47,731 copying lightllm/models/starcoder2/layer_infer/__init__.py -> build/lib/lightllm/models/starcoder2/layer_infer 2024-08-27T12:42:47,732 copying lightllm/models/starcoder2/layer_infer/transformer_layer_infer.py -> build/lib/lightllm/models/starcoder2/layer_infer 2024-08-27T12:42:47,735 creating build/lib/lightllm/models/starcoder2/layer_weights 2024-08-27T12:42:47,736 copying lightllm/models/starcoder2/layer_weights/pre_and_post_layer_weight.py -> build/lib/lightllm/models/starcoder2/layer_weights 2024-08-27T12:42:47,738 copying lightllm/models/starcoder2/layer_weights/__init__.py -> build/lib/lightllm/models/starcoder2/layer_weights 2024-08-27T12:42:47,745 copying lightllm/models/starcoder2/layer_weights/transformer_layer_weight.py -> build/lib/lightllm/models/starcoder2/layer_weights 2024-08-27T12:42:47,747 creating build/lib/lightllm/models/stablelm/layer_infer 2024-08-27T12:42:47,748 copying lightllm/models/stablelm/layer_infer/__init__.py -> build/lib/lightllm/models/stablelm/layer_infer 2024-08-27T12:42:47,750 copying lightllm/models/stablelm/layer_infer/transformer_layer_infer.py -> build/lib/lightllm/models/stablelm/layer_infer 2024-08-27T12:42:47,752 creating build/lib/lightllm/models/stablelm/layer_weights 2024-08-27T12:42:47,753 copying lightllm/models/stablelm/layer_weights/pre_and_post_layer_weight.py -> build/lib/lightllm/models/stablelm/layer_weights 2024-08-27T12:42:47,755 copying lightllm/models/stablelm/layer_weights/__init__.py -> build/lib/lightllm/models/stablelm/layer_weights 2024-08-27T12:42:47,757 copying lightllm/models/stablelm/layer_weights/transformer_layer_weight.py -> build/lib/lightllm/models/stablelm/layer_weights 2024-08-27T12:42:47,759 creating build/lib/lightllm/models/qwen/layer_infer 2024-08-27T12:42:47,760 copying lightllm/models/qwen/layer_infer/__init__.py -> build/lib/lightllm/models/qwen/layer_infer 2024-08-27T12:42:47,762 copying lightllm/models/qwen/layer_infer/transformer_layer_infer.py -> build/lib/lightllm/models/qwen/layer_infer 2024-08-27T12:42:47,764 creating build/lib/lightllm/models/qwen/layer_weights 2024-08-27T12:42:47,765 copying lightllm/models/qwen/layer_weights/pre_and_post_layer_weight.py -> build/lib/lightllm/models/qwen/layer_weights 2024-08-27T12:42:47,767 copying lightllm/models/qwen/layer_weights/__init__.py -> build/lib/lightllm/models/qwen/layer_weights 2024-08-27T12:42:47,768 copying lightllm/models/qwen/layer_weights/transformer_layer_weight.py -> build/lib/lightllm/models/qwen/layer_weights 2024-08-27T12:42:47,771 creating build/lib/lightllm/models/llama/layer_infer 2024-08-27T12:42:47,778 copying lightllm/models/llama/layer_infer/__init__.py -> build/lib/lightllm/models/llama/layer_infer 2024-08-27T12:42:47,780 copying lightllm/models/llama/layer_infer/transformer_layer_infer.py -> build/lib/lightllm/models/llama/layer_infer 2024-08-27T12:42:47,782 copying lightllm/models/llama/layer_infer/post_layer_infer.py -> build/lib/lightllm/models/llama/layer_infer 2024-08-27T12:42:47,785 copying lightllm/models/llama/layer_infer/pre_layer_infer.py -> build/lib/lightllm/models/llama/layer_infer 2024-08-27T12:42:47,787 creating build/lib/lightllm/models/llama/triton_kernel 2024-08-27T12:42:47,789 copying lightllm/models/llama/triton_kernel/token_attention_nopad_reduceV.py -> build/lib/lightllm/models/llama/triton_kernel 2024-08-27T12:42:47,791 copying lightllm/models/llama/triton_kernel/token_attention_nopad_softmax.py -> build/lib/lightllm/models/llama/triton_kernel 2024-08-27T12:42:47,793 copying lightllm/models/llama/triton_kernel/flash_decoding.py -> build/lib/lightllm/models/llama/triton_kernel 2024-08-27T12:42:47,794 copying lightllm/models/llama/triton_kernel/ppl_int4kv_copy_kv.py -> build/lib/lightllm/models/llama/triton_kernel 2024-08-27T12:42:47,796 copying lightllm/models/llama/triton_kernel/__init__.py -> build/lib/lightllm/models/llama/triton_kernel 2024-08-27T12:42:47,798 copying lightllm/models/llama/triton_kernel/splitfuse_context_flashattention_nopad.py -> build/lib/lightllm/models/llama/triton_kernel 2024-08-27T12:42:47,800 copying lightllm/models/llama/triton_kernel/context_flashattention_nopad.py -> build/lib/lightllm/models/llama/triton_kernel 2024-08-27T12:42:47,803 copying lightllm/models/llama/triton_kernel/gqa_flash_decoding.py -> build/lib/lightllm/models/llama/triton_kernel 2024-08-27T12:42:47,804 copying lightllm/models/llama/triton_kernel/silu_and_mul.py -> build/lib/lightllm/models/llama/triton_kernel 2024-08-27T12:42:47,806 copying lightllm/models/llama/triton_kernel/ppl_quant_copy_kv.py -> build/lib/lightllm/models/llama/triton_kernel 2024-08-27T12:42:47,808 copying lightllm/models/llama/triton_kernel/flash_decoding_stage1.py -> build/lib/lightllm/models/llama/triton_kernel 2024-08-27T12:42:47,811 copying lightllm/models/llama/triton_kernel/token_attention_nopad_att1.py -> build/lib/lightllm/models/llama/triton_kernel 2024-08-27T12:42:47,813 copying lightllm/models/llama/triton_kernel/ppl_fp16_flash_decoding.py -> build/lib/lightllm/models/llama/triton_kernel 2024-08-27T12:42:47,815 copying lightllm/models/llama/triton_kernel/ppl_int8kv_flash_decoding.py -> build/lib/lightllm/models/llama/triton_kernel 2024-08-27T12:42:47,816 copying lightllm/models/llama/triton_kernel/rmsnorm.py -> build/lib/lightllm/models/llama/triton_kernel 2024-08-27T12:42:47,818 copying lightllm/models/llama/triton_kernel/gqa_flash_decoding_stage2.py -> build/lib/lightllm/models/llama/triton_kernel 2024-08-27T12:42:47,820 copying lightllm/models/llama/triton_kernel/rotary_emb.py -> build/lib/lightllm/models/llama/triton_kernel 2024-08-27T12:42:47,822 copying lightllm/models/llama/triton_kernel/gqa_flash_decoding_stage1.py -> build/lib/lightllm/models/llama/triton_kernel 2024-08-27T12:42:47,824 copying lightllm/models/llama/triton_kernel/token_attention_softmax_and_reducev.py -> build/lib/lightllm/models/llama/triton_kernel 2024-08-27T12:42:47,826 copying lightllm/models/llama/triton_kernel/flash_decoding_stage2.py -> build/lib/lightllm/models/llama/triton_kernel 2024-08-27T12:42:47,828 copying lightllm/models/llama/triton_kernel/ppl_int4kv_flash_decoding.py -> build/lib/lightllm/models/llama/triton_kernel 2024-08-27T12:42:47,829 copying lightllm/models/llama/triton_kernel/gqa_decode_flashattention_nopad.py -> build/lib/lightllm/models/llama/triton_kernel 2024-08-27T12:42:47,832 creating build/lib/lightllm/models/llama/layer_weights 2024-08-27T12:42:47,833 copying lightllm/models/llama/layer_weights/pre_and_post_layer_weight.py -> build/lib/lightllm/models/llama/layer_weights 2024-08-27T12:42:47,835 copying lightllm/models/llama/layer_weights/__init__.py -> build/lib/lightllm/models/llama/layer_weights 2024-08-27T12:42:47,836 copying lightllm/models/llama/layer_weights/transformer_layer_weight.py -> build/lib/lightllm/models/llama/layer_weights 2024-08-27T12:42:47,839 copying lightllm/models/llama/layer_weights/ds_load_utils.py -> build/lib/lightllm/models/llama/layer_weights 2024-08-27T12:42:47,841 creating build/lib/lightllm/models/bloom/layer_infer 2024-08-27T12:42:47,842 copying lightllm/models/bloom/layer_infer/__init__.py -> build/lib/lightllm/models/bloom/layer_infer 2024-08-27T12:42:47,843 copying lightllm/models/bloom/layer_infer/transformer_layer_infer.py -> build/lib/lightllm/models/bloom/layer_infer 2024-08-27T12:42:47,846 copying lightllm/models/bloom/layer_infer/post_layer_infer.py -> build/lib/lightllm/models/bloom/layer_infer 2024-08-27T12:42:47,847 copying lightllm/models/bloom/layer_infer/pre_layer_infer.py -> build/lib/lightllm/models/bloom/layer_infer 2024-08-27T12:42:47,850 creating build/lib/lightllm/models/bloom/triton_kernel 2024-08-27T12:42:47,851 copying lightllm/models/bloom/triton_kernel/token_attention_nopad_reduceV.py -> build/lib/lightllm/models/bloom/triton_kernel 2024-08-27T12:42:47,853 copying lightllm/models/bloom/triton_kernel/token_attention_nopad_softmax.py -> build/lib/lightllm/models/bloom/triton_kernel 2024-08-27T12:42:47,855 copying lightllm/models/bloom/triton_kernel/token_flashattention_nopad.py -> build/lib/lightllm/models/bloom/triton_kernel 2024-08-27T12:42:47,857 copying lightllm/models/bloom/triton_kernel/__init__.py -> build/lib/lightllm/models/bloom/triton_kernel 2024-08-27T12:42:47,858 copying lightllm/models/bloom/triton_kernel/context_flashattention_nopad.py -> build/lib/lightllm/models/bloom/triton_kernel 2024-08-27T12:42:47,860 copying lightllm/models/bloom/triton_kernel/token_attention_nopad_att1.py -> build/lib/lightllm/models/bloom/triton_kernel 2024-08-27T12:42:47,862 copying lightllm/models/bloom/triton_kernel/layernorm.py -> build/lib/lightllm/models/bloom/triton_kernel 2024-08-27T12:42:47,865 creating build/lib/lightllm/models/bloom/layer_weights 2024-08-27T12:42:47,866 copying lightllm/models/bloom/layer_weights/pre_and_post_layer_weight.py -> build/lib/lightllm/models/bloom/layer_weights 2024-08-27T12:42:47,868 copying lightllm/models/bloom/layer_weights/__init__.py -> build/lib/lightllm/models/bloom/layer_weights 2024-08-27T12:42:47,869 copying lightllm/models/bloom/layer_weights/hf_load_utils.py -> build/lib/lightllm/models/bloom/layer_weights 2024-08-27T12:42:47,871 copying lightllm/models/bloom/layer_weights/transformer_layer_weight.py -> build/lib/lightllm/models/bloom/layer_weights 2024-08-27T12:42:47,873 creating build/lib/lightllm/models/starcoder/layer_infer 2024-08-27T12:42:47,874 copying lightllm/models/starcoder/layer_infer/__init__.py -> build/lib/lightllm/models/starcoder/layer_infer 2024-08-27T12:42:47,876 copying lightllm/models/starcoder/layer_infer/transformer_layer_infer.py -> build/lib/lightllm/models/starcoder/layer_infer 2024-08-27T12:42:47,877 copying lightllm/models/starcoder/layer_infer/pre_layer_infer.py -> build/lib/lightllm/models/starcoder/layer_infer 2024-08-27T12:42:47,879 creating build/lib/lightllm/models/starcoder/layer_weights 2024-08-27T12:42:47,880 copying lightllm/models/starcoder/layer_weights/pre_and_post_layer_weight.py -> build/lib/lightllm/models/starcoder/layer_weights 2024-08-27T12:42:47,882 copying lightllm/models/starcoder/layer_weights/__init__.py -> build/lib/lightllm/models/starcoder/layer_weights 2024-08-27T12:42:47,884 copying lightllm/models/starcoder/layer_weights/transformer_layer_weight.py -> build/lib/lightllm/models/starcoder/layer_weights 2024-08-27T12:42:47,886 creating build/lib/lightllm/models/qwen_vl/layer_infer 2024-08-27T12:42:47,887 copying lightllm/models/qwen_vl/layer_infer/__init__.py -> build/lib/lightllm/models/qwen_vl/layer_infer 2024-08-27T12:42:47,889 copying lightllm/models/qwen_vl/layer_infer/pre_layer_infer.py -> build/lib/lightllm/models/qwen_vl/layer_infer 2024-08-27T12:42:47,891 creating build/lib/lightllm/models/mixtral/layer_infer 2024-08-27T12:42:47,892 copying lightllm/models/mixtral/layer_infer/__init__.py -> build/lib/lightllm/models/mixtral/layer_infer 2024-08-27T12:42:47,893 copying lightllm/models/mixtral/layer_infer/transformer_layer_infer.py -> build/lib/lightllm/models/mixtral/layer_infer 2024-08-27T12:42:47,896 creating build/lib/lightllm/models/mixtral/layer_weights 2024-08-27T12:42:47,897 copying lightllm/models/mixtral/layer_weights/__init__.py -> build/lib/lightllm/models/mixtral/layer_weights 2024-08-27T12:42:47,899 copying lightllm/models/mixtral/layer_weights/transformer_layer_weight.py -> build/lib/lightllm/models/mixtral/layer_weights 2024-08-27T12:42:48,032 /usr/local/lib/python3.11/dist-packages/setuptools/_distutils/cmd.py:66: SetuptoolsDeprecationWarning: setup.py install is deprecated. 2024-08-27T12:42:48,033 !! 2024-08-27T12:42:48,034 ******************************************************************************** 2024-08-27T12:42:48,035 Please avoid running ``setup.py`` directly. 2024-08-27T12:42:48,036 Instead, use pypa/build, pypa/installer or other 2024-08-27T12:42:48,036 standards-based tools. 2024-08-27T12:42:48,038 See https://blog.ganssle.io/articles/2021/10/setup-py-deprecated.html for details. 2024-08-27T12:42:48,038 ******************************************************************************** 2024-08-27T12:42:48,040 !! 2024-08-27T12:42:48,041 self.initialize_options() 2024-08-27T12:42:48,069 installing to build/bdist.linux-armv7l/wheel 2024-08-27T12:42:48,070 running install 2024-08-27T12:42:48,094 running install_lib 2024-08-27T12:42:48,120 creating build/bdist.linux-armv7l 2024-08-27T12:42:48,120 creating build/bdist.linux-armv7l/wheel 2024-08-27T12:42:48,122 creating build/bdist.linux-armv7l/wheel/lightllm 2024-08-27T12:42:48,123 creating build/bdist.linux-armv7l/wheel/lightllm/common 2024-08-27T12:42:48,124 copying build/lib/lightllm/common/mem_manager.py -> build/bdist.linux-armv7l/wheel/./lightllm/common 2024-08-27T12:42:48,127 copying build/lib/lightllm/common/infer_utils.py -> build/bdist.linux-armv7l/wheel/./lightllm/common 2024-08-27T12:42:48,128 copying build/lib/lightllm/common/req_manager.py -> build/bdist.linux-armv7l/wheel/./lightllm/common 2024-08-27T12:42:48,130 copying build/lib/lightllm/common/__init__.py -> build/bdist.linux-armv7l/wheel/./lightllm/common 2024-08-27T12:42:48,132 copying build/lib/lightllm/common/build_utils.py -> build/bdist.linux-armv7l/wheel/./lightllm/common 2024-08-27T12:42:48,134 creating build/bdist.linux-armv7l/wheel/lightllm/common/basemodel 2024-08-27T12:42:48,135 creating build/bdist.linux-armv7l/wheel/lightllm/common/basemodel/layer_infer 2024-08-27T12:42:48,136 copying build/lib/lightllm/common/basemodel/layer_infer/__init__.py -> build/bdist.linux-armv7l/wheel/./lightllm/common/basemodel/layer_infer 2024-08-27T12:42:48,138 copying build/lib/lightllm/common/basemodel/layer_infer/base_layer_infer.py -> build/bdist.linux-armv7l/wheel/./lightllm/common/basemodel/layer_infer 2024-08-27T12:42:48,140 copying build/lib/lightllm/common/basemodel/layer_infer/transformer_layer_infer.py -> build/bdist.linux-armv7l/wheel/./lightllm/common/basemodel/layer_infer 2024-08-27T12:42:48,142 creating build/bdist.linux-armv7l/wheel/lightllm/common/basemodel/layer_infer/template 2024-08-27T12:42:48,143 copying build/lib/lightllm/common/basemodel/layer_infer/template/pre_layer_infer_template.py -> build/bdist.linux-armv7l/wheel/./lightllm/common/basemodel/layer_infer/template 2024-08-27T12:42:48,145 copying build/lib/lightllm/common/basemodel/layer_infer/template/__init__.py -> build/bdist.linux-armv7l/wheel/./lightllm/common/basemodel/layer_infer/template 2024-08-27T12:42:48,146 copying build/lib/lightllm/common/basemodel/layer_infer/template/transformer_layer_infer_template.py -> build/bdist.linux-armv7l/wheel/./lightllm/common/basemodel/layer_infer/template 2024-08-27T12:42:48,148 copying build/lib/lightllm/common/basemodel/layer_infer/template/transformer_layer_infer_template_wquant.py -> build/bdist.linux-armv7l/wheel/./lightllm/common/basemodel/layer_infer/template 2024-08-27T12:42:48,150 copying build/lib/lightllm/common/basemodel/layer_infer/template/post_layer_infer_template.py -> build/bdist.linux-armv7l/wheel/./lightllm/common/basemodel/layer_infer/template 2024-08-27T12:42:48,152 copying build/lib/lightllm/common/basemodel/layer_infer/template/transformer_layer_infer_template_awquant.py -> build/bdist.linux-armv7l/wheel/./lightllm/common/basemodel/layer_infer/template 2024-08-27T12:42:48,154 copying build/lib/lightllm/common/basemodel/layer_infer/post_layer_infer.py -> build/bdist.linux-armv7l/wheel/./lightllm/common/basemodel/layer_infer 2024-08-27T12:42:48,156 copying build/lib/lightllm/common/basemodel/layer_infer/pre_layer_infer.py -> build/bdist.linux-armv7l/wheel/./lightllm/common/basemodel/layer_infer 2024-08-27T12:42:48,158 copying build/lib/lightllm/common/basemodel/splitfuse_infer_struct.py -> build/bdist.linux-armv7l/wheel/./lightllm/common/basemodel 2024-08-27T12:42:48,160 copying build/lib/lightllm/common/basemodel/__init__.py -> build/bdist.linux-armv7l/wheel/./lightllm/common/basemodel 2024-08-27T12:42:48,161 copying build/lib/lightllm/common/basemodel/infer_struct.py -> build/bdist.linux-armv7l/wheel/./lightllm/common/basemodel 2024-08-27T12:42:48,164 creating build/bdist.linux-armv7l/wheel/lightllm/common/basemodel/triton_kernel 2024-08-27T12:42:48,165 copying build/lib/lightllm/common/basemodel/triton_kernel/__init__.py -> build/bdist.linux-armv7l/wheel/./lightllm/common/basemodel/triton_kernel 2024-08-27T12:42:48,167 copying build/lib/lightllm/common/basemodel/triton_kernel/apply_penalty.py -> build/bdist.linux-armv7l/wheel/./lightllm/common/basemodel/triton_kernel 2024-08-27T12:42:48,168 copying build/lib/lightllm/common/basemodel/triton_kernel/multimodal_emb.py -> build/bdist.linux-armv7l/wheel/./lightllm/common/basemodel/triton_kernel 2024-08-27T12:42:48,171 copying build/lib/lightllm/common/basemodel/triton_kernel/dequantize_gemm_int8.py -> build/bdist.linux-armv7l/wheel/./lightllm/common/basemodel/triton_kernel 2024-08-27T12:42:48,173 copying build/lib/lightllm/common/basemodel/triton_kernel/copy_kv_index_to_req.py -> build/bdist.linux-armv7l/wheel/./lightllm/common/basemodel/triton_kernel 2024-08-27T12:42:48,175 copying build/lib/lightllm/common/basemodel/triton_kernel/quantize_gemm_int8.py -> build/bdist.linux-armv7l/wheel/./lightllm/common/basemodel/triton_kernel 2024-08-27T12:42:48,177 copying build/lib/lightllm/common/basemodel/triton_kernel/dequantize_gemm_int4.py -> build/bdist.linux-armv7l/wheel/./lightllm/common/basemodel/triton_kernel 2024-08-27T12:42:48,180 copying build/lib/lightllm/common/basemodel/triton_kernel/splitfuse_copy_kv_index_to_req.py -> build/bdist.linux-armv7l/wheel/./lightllm/common/basemodel/triton_kernel 2024-08-27T12:42:48,182 copying build/lib/lightllm/common/basemodel/triton_kernel/destindex_copy_kv.py -> build/bdist.linux-armv7l/wheel/./lightllm/common/basemodel/triton_kernel 2024-08-27T12:42:48,184 creating build/bdist.linux-armv7l/wheel/lightllm/common/basemodel/cuda_kernel 2024-08-27T12:42:48,185 copying build/lib/lightllm/common/basemodel/cuda_kernel/__init__.py -> build/bdist.linux-armv7l/wheel/./lightllm/common/basemodel/cuda_kernel 2024-08-27T12:42:48,187 copying build/lib/lightllm/common/basemodel/cuda_kernel/ppl_wquant.py -> build/bdist.linux-armv7l/wheel/./lightllm/common/basemodel/cuda_kernel 2024-08-27T12:42:48,188 copying build/lib/lightllm/common/basemodel/cuda_kernel/fast_llm_wquant.py -> build/bdist.linux-armv7l/wheel/./lightllm/common/basemodel/cuda_kernel 2024-08-27T12:42:48,190 copying build/lib/lightllm/common/basemodel/cuda_kernel/ppl_awquant.py -> build/bdist.linux-armv7l/wheel/./lightllm/common/basemodel/cuda_kernel 2024-08-27T12:42:48,192 copying build/lib/lightllm/common/basemodel/cuda_kernel/lmdeploy_wquant.py -> build/bdist.linux-armv7l/wheel/./lightllm/common/basemodel/cuda_kernel 2024-08-27T12:42:48,194 creating build/bdist.linux-armv7l/wheel/lightllm/common/basemodel/layer_weights 2024-08-27T12:42:48,195 copying build/lib/lightllm/common/basemodel/layer_weights/pre_and_post_layer_weight.py -> build/bdist.linux-armv7l/wheel/./lightllm/common/basemodel/layer_weights 2024-08-27T12:42:48,197 copying build/lib/lightllm/common/basemodel/layer_weights/base_layer_weight.py -> build/bdist.linux-armv7l/wheel/./lightllm/common/basemodel/layer_weights 2024-08-27T12:42:48,199 copying build/lib/lightllm/common/basemodel/layer_weights/__init__.py -> build/bdist.linux-armv7l/wheel/./lightllm/common/basemodel/layer_weights 2024-08-27T12:42:48,200 copying build/lib/lightllm/common/basemodel/layer_weights/hf_load_utils.py -> build/bdist.linux-armv7l/wheel/./lightllm/common/basemodel/layer_weights 2024-08-27T12:42:48,202 copying build/lib/lightllm/common/basemodel/layer_weights/transformer_layer_weight.py -> build/bdist.linux-armv7l/wheel/./lightllm/common/basemodel/layer_weights 2024-08-27T12:42:48,204 copying build/lib/lightllm/common/basemodel/basemodel.py -> build/bdist.linux-armv7l/wheel/./lightllm/common/basemodel 2024-08-27T12:42:48,207 copying build/lib/lightllm/common/int8kv_mem_manager.py -> build/bdist.linux-armv7l/wheel/./lightllm/common 2024-08-27T12:42:48,209 copying build/lib/lightllm/common/ppl_int4kv_mem_manager.py -> build/bdist.linux-armv7l/wheel/./lightllm/common 2024-08-27T12:42:48,210 copying build/lib/lightllm/common/mem_utils.py -> build/bdist.linux-armv7l/wheel/./lightllm/common 2024-08-27T12:42:48,212 copying build/lib/lightllm/common/ppl_int8kv_mem_manager.py -> build/bdist.linux-armv7l/wheel/./lightllm/common 2024-08-27T12:42:48,215 creating build/bdist.linux-armv7l/wheel/lightllm/server 2024-08-27T12:42:48,216 copying build/lib/lightllm/server/tokenizer.py -> build/bdist.linux-armv7l/wheel/./lightllm/server 2024-08-27T12:42:48,219 creating build/bdist.linux-armv7l/wheel/lightllm/server/visualserver 2024-08-27T12:42:48,220 creating build/bdist.linux-armv7l/wheel/lightllm/server/visualserver/model_infer 2024-08-27T12:42:48,221 copying build/lib/lightllm/server/visualserver/model_infer/model_rpc.py -> build/bdist.linux-armv7l/wheel/./lightllm/server/visualserver/model_infer 2024-08-27T12:42:48,223 copying build/lib/lightllm/server/visualserver/model_infer/__init__.py -> build/bdist.linux-armv7l/wheel/./lightllm/server/visualserver/model_infer 2024-08-27T12:42:48,224 copying build/lib/lightllm/server/visualserver/__init__.py -> build/bdist.linux-armv7l/wheel/./lightllm/server/visualserver 2024-08-27T12:42:48,225 copying build/lib/lightllm/server/visualserver/manager.py -> build/bdist.linux-armv7l/wheel/./lightllm/server/visualserver 2024-08-27T12:42:48,228 copying build/lib/lightllm/server/api_models.py -> build/bdist.linux-armv7l/wheel/./lightllm/server 2024-08-27T12:42:48,229 copying build/lib/lightllm/server/api_lightllm.py -> build/bdist.linux-armv7l/wheel/./lightllm/server 2024-08-27T12:42:48,232 copying build/lib/lightllm/server/metrics.py -> build/bdist.linux-armv7l/wheel/./lightllm/server 2024-08-27T12:42:48,233 copying build/lib/lightllm/server/req_id_generator.py -> build/bdist.linux-armv7l/wheel/./lightllm/server 2024-08-27T12:42:48,235 copying build/lib/lightllm/server/build_prompt.py -> build/bdist.linux-armv7l/wheel/./lightllm/server 2024-08-27T12:42:48,237 copying build/lib/lightllm/server/__init__.py -> build/bdist.linux-armv7l/wheel/./lightllm/server 2024-08-27T12:42:48,239 creating build/bdist.linux-armv7l/wheel/lightllm/server/health_monitor 2024-08-27T12:42:48,240 copying build/lib/lightllm/server/health_monitor/__init__.py -> build/bdist.linux-armv7l/wheel/./lightllm/server/health_monitor 2024-08-27T12:42:48,241 copying build/lib/lightllm/server/health_monitor/manager.py -> build/bdist.linux-armv7l/wheel/./lightllm/server/health_monitor 2024-08-27T12:42:48,244 creating build/bdist.linux-armv7l/wheel/lightllm/server/detokenization 2024-08-27T12:42:48,245 copying build/lib/lightllm/server/detokenization/decode.py -> build/bdist.linux-armv7l/wheel/./lightllm/server/detokenization 2024-08-27T12:42:48,247 copying build/lib/lightllm/server/detokenization/__init__.py -> build/bdist.linux-armv7l/wheel/./lightllm/server/detokenization 2024-08-27T12:42:48,249 copying build/lib/lightllm/server/detokenization/manager.py -> build/bdist.linux-armv7l/wheel/./lightllm/server/detokenization 2024-08-27T12:42:48,251 copying build/lib/lightllm/server/io_struct.py -> build/bdist.linux-armv7l/wheel/./lightllm/server 2024-08-27T12:42:48,253 copying build/lib/lightllm/server/api_server.py -> build/bdist.linux-armv7l/wheel/./lightllm/server 2024-08-27T12:42:48,256 creating build/bdist.linux-armv7l/wheel/lightllm/server/embed_cache 2024-08-27T12:42:48,257 copying build/lib/lightllm/server/embed_cache/__init__.py -> build/bdist.linux-armv7l/wheel/./lightllm/server/embed_cache 2024-08-27T12:42:48,259 creating build/bdist.linux-armv7l/wheel/lightllm/server/embed_cache/impl 2024-08-27T12:42:48,260 copying build/lib/lightllm/server/embed_cache/impl/naive_memory_cache.py -> build/bdist.linux-armv7l/wheel/./lightllm/server/embed_cache/impl 2024-08-27T12:42:48,262 copying build/lib/lightllm/server/embed_cache/impl/__init__.py -> build/bdist.linux-armv7l/wheel/./lightllm/server/embed_cache/impl 2024-08-27T12:42:48,264 copying build/lib/lightllm/server/embed_cache/manager.py -> build/bdist.linux-armv7l/wheel/./lightllm/server/embed_cache 2024-08-27T12:42:48,266 copying build/lib/lightllm/server/embed_cache/interface.py -> build/bdist.linux-armv7l/wheel/./lightllm/server/embed_cache 2024-08-27T12:42:48,268 copying build/lib/lightllm/server/embed_cache/utils.py -> build/bdist.linux-armv7l/wheel/./lightllm/server/embed_cache 2024-08-27T12:42:48,270 copying build/lib/lightllm/server/api_tgi.py -> build/bdist.linux-armv7l/wheel/./lightllm/server 2024-08-27T12:42:48,272 copying build/lib/lightllm/server/multimodal_params.py -> build/bdist.linux-armv7l/wheel/./lightllm/server 2024-08-27T12:42:48,274 creating build/bdist.linux-armv7l/wheel/lightllm/server/httpserver 2024-08-27T12:42:48,275 copying build/lib/lightllm/server/httpserver/__init__.py -> build/bdist.linux-armv7l/wheel/./lightllm/server/httpserver 2024-08-27T12:42:48,276 copying build/lib/lightllm/server/httpserver/manager.py -> build/bdist.linux-armv7l/wheel/./lightllm/server/httpserver 2024-08-27T12:42:48,279 copying build/lib/lightllm/server/sampling_params.py -> build/bdist.linux-armv7l/wheel/./lightllm/server 2024-08-27T12:42:48,281 creating build/bdist.linux-armv7l/wheel/lightllm/server/router 2024-08-27T12:42:48,283 creating build/bdist.linux-armv7l/wheel/lightllm/server/router/model_infer 2024-08-27T12:42:48,283 copying build/lib/lightllm/server/router/model_infer/infer_batch.py -> build/bdist.linux-armv7l/wheel/./lightllm/server/router/model_infer 2024-08-27T12:42:48,286 copying build/lib/lightllm/server/router/model_infer/model_rpc.py -> build/bdist.linux-armv7l/wheel/./lightllm/server/router/model_infer 2024-08-27T12:42:48,289 copying build/lib/lightllm/server/router/model_infer/__init__.py -> build/bdist.linux-armv7l/wheel/./lightllm/server/router/model_infer 2024-08-27T12:42:48,290 creating build/bdist.linux-armv7l/wheel/lightllm/server/router/model_infer/mode_backend 2024-08-27T12:42:48,292 creating build/bdist.linux-armv7l/wheel/lightllm/server/router/model_infer/mode_backend/diverse_backend 2024-08-27T12:42:48,293 copying build/lib/lightllm/server/router/model_infer/mode_backend/diverse_backend/post_process.py -> build/bdist.linux-armv7l/wheel/./lightllm/server/router/model_infer/mode_backend/diverse_backend 2024-08-27T12:42:48,295 copying build/lib/lightllm/server/router/model_infer/mode_backend/diverse_backend/__init__.py -> build/bdist.linux-armv7l/wheel/./lightllm/server/router/model_infer/mode_backend/diverse_backend 2024-08-27T12:42:48,297 copying build/lib/lightllm/server/router/model_infer/mode_backend/diverse_backend/impl.py -> build/bdist.linux-armv7l/wheel/./lightllm/server/router/model_infer/mode_backend/diverse_backend 2024-08-27T12:42:48,299 copying build/lib/lightllm/server/router/model_infer/mode_backend/__init__.py -> build/bdist.linux-armv7l/wheel/./lightllm/server/router/model_infer/mode_backend 2024-08-27T12:42:48,301 creating build/bdist.linux-armv7l/wheel/lightllm/server/router/model_infer/mode_backend/beamsearch 2024-08-27T12:42:48,302 copying build/lib/lightllm/server/router/model_infer/mode_backend/beamsearch/pre_process.py -> build/bdist.linux-armv7l/wheel/./lightllm/server/router/model_infer/mode_backend/beamsearch 2024-08-27T12:42:48,304 copying build/lib/lightllm/server/router/model_infer/mode_backend/beamsearch/post_process.py -> build/bdist.linux-armv7l/wheel/./lightllm/server/router/model_infer/mode_backend/beamsearch 2024-08-27T12:42:48,306 copying build/lib/lightllm/server/router/model_infer/mode_backend/beamsearch/__init__.py -> build/bdist.linux-armv7l/wheel/./lightllm/server/router/model_infer/mode_backend/beamsearch 2024-08-27T12:42:48,308 copying build/lib/lightllm/server/router/model_infer/mode_backend/beamsearch/impl.py -> build/bdist.linux-armv7l/wheel/./lightllm/server/router/model_infer/mode_backend/beamsearch 2024-08-27T12:42:48,311 creating build/bdist.linux-armv7l/wheel/lightllm/server/router/model_infer/mode_backend/continues_batch 2024-08-27T12:42:48,312 copying build/lib/lightllm/server/router/model_infer/mode_backend/continues_batch/pre_process.py -> build/bdist.linux-armv7l/wheel/./lightllm/server/router/model_infer/mode_backend/continues_batch 2024-08-27T12:42:48,314 copying build/lib/lightllm/server/router/model_infer/mode_backend/continues_batch/post_process.py -> build/bdist.linux-armv7l/wheel/./lightllm/server/router/model_infer/mode_backend/continues_batch 2024-08-27T12:42:48,316 copying build/lib/lightllm/server/router/model_infer/mode_backend/continues_batch/__init__.py -> build/bdist.linux-armv7l/wheel/./lightllm/server/router/model_infer/mode_backend/continues_batch 2024-08-27T12:42:48,318 copying build/lib/lightllm/server/router/model_infer/mode_backend/continues_batch/impl.py -> build/bdist.linux-armv7l/wheel/./lightllm/server/router/model_infer/mode_backend/continues_batch 2024-08-27T12:42:48,320 copying build/lib/lightllm/server/router/model_infer/mode_backend/continues_batch/impl_for_return_all_prompt_logprobs.py -> build/bdist.linux-armv7l/wheel/./lightllm/server/router/model_infer/mode_backend/continues_batch 2024-08-27T12:42:48,321 copying build/lib/lightllm/server/router/model_infer/mode_backend/base_backend.py -> build/bdist.linux-armv7l/wheel/./lightllm/server/router/model_infer/mode_backend 2024-08-27T12:42:48,324 creating build/bdist.linux-armv7l/wheel/lightllm/server/router/model_infer/mode_backend/splitfuse 2024-08-27T12:42:48,325 copying build/lib/lightllm/server/router/model_infer/mode_backend/splitfuse/pre_process.py -> build/bdist.linux-armv7l/wheel/./lightllm/server/router/model_infer/mode_backend/splitfuse 2024-08-27T12:42:48,327 copying build/lib/lightllm/server/router/model_infer/mode_backend/splitfuse/__init__.py -> build/bdist.linux-armv7l/wheel/./lightllm/server/router/model_infer/mode_backend/splitfuse 2024-08-27T12:42:48,328 copying build/lib/lightllm/server/router/model_infer/mode_backend/splitfuse/impl.py -> build/bdist.linux-armv7l/wheel/./lightllm/server/router/model_infer/mode_backend/splitfuse 2024-08-27T12:42:48,331 copying build/lib/lightllm/server/router/stats.py -> build/bdist.linux-armv7l/wheel/./lightllm/server/router 2024-08-27T12:42:48,332 copying build/lib/lightllm/server/router/pause_strategy.py -> build/bdist.linux-armv7l/wheel/./lightllm/server/router 2024-08-27T12:42:48,334 copying build/lib/lightllm/server/router/token_load.py -> build/bdist.linux-armv7l/wheel/./lightllm/server/router 2024-08-27T12:42:48,336 copying build/lib/lightllm/server/router/__init__.py -> build/bdist.linux-armv7l/wheel/./lightllm/server/router 2024-08-27T12:42:48,337 copying build/lib/lightllm/server/router/manager.py -> build/bdist.linux-armv7l/wheel/./lightllm/server/router 2024-08-27T12:42:48,340 creating build/bdist.linux-armv7l/wheel/lightllm/server/router/dynamic_prompt 2024-08-27T12:42:48,341 copying build/lib/lightllm/server/router/dynamic_prompt/__init__.py -> build/bdist.linux-armv7l/wheel/./lightllm/server/router/dynamic_prompt 2024-08-27T12:42:48,342 copying build/lib/lightllm/server/router/dynamic_prompt/radix_cache.py -> build/bdist.linux-armv7l/wheel/./lightllm/server/router/dynamic_prompt 2024-08-27T12:42:48,345 copying build/lib/lightllm/server/router/dynamic_prompt/shared_arr.py -> build/bdist.linux-armv7l/wheel/./lightllm/server/router/dynamic_prompt 2024-08-27T12:42:48,347 creating build/bdist.linux-armv7l/wheel/lightllm/server/router/req_queue 2024-08-27T12:42:48,348 copying build/lib/lightllm/server/router/req_queue/base_queue.py -> build/bdist.linux-armv7l/wheel/./lightllm/server/router/req_queue 2024-08-27T12:42:48,351 copying build/lib/lightllm/server/router/req_queue/__init__.py -> build/bdist.linux-armv7l/wheel/./lightllm/server/router/req_queue 2024-08-27T12:42:48,353 creating build/bdist.linux-armv7l/wheel/lightllm/server/router/req_queue/continues_batch 2024-08-27T12:42:48,354 copying build/lib/lightllm/server/router/req_queue/continues_batch/__init__.py -> build/bdist.linux-armv7l/wheel/./lightllm/server/router/req_queue/continues_batch 2024-08-27T12:42:48,355 copying build/lib/lightllm/server/router/req_queue/continues_batch/impl.py -> build/bdist.linux-armv7l/wheel/./lightllm/server/router/req_queue/continues_batch 2024-08-27T12:42:48,357 copying build/lib/lightllm/server/router/req_queue/continues_batch/beam_impl.py -> build/bdist.linux-armv7l/wheel/./lightllm/server/router/req_queue/continues_batch 2024-08-27T12:42:48,360 creating build/bdist.linux-armv7l/wheel/lightllm/server/router/req_queue/splitfuse 2024-08-27T12:42:48,361 copying build/lib/lightllm/server/router/req_queue/splitfuse/__init__.py -> build/bdist.linux-armv7l/wheel/./lightllm/server/router/req_queue/splitfuse 2024-08-27T12:42:48,363 copying build/lib/lightllm/server/router/req_queue/splitfuse/impl.py -> build/bdist.linux-armv7l/wheel/./lightllm/server/router/req_queue/splitfuse 2024-08-27T12:42:48,365 copying build/lib/lightllm/__init__.py -> build/bdist.linux-armv7l/wheel/./lightllm 2024-08-27T12:42:48,367 creating build/bdist.linux-armv7l/wheel/lightllm/utils 2024-08-27T12:42:48,368 copying build/lib/lightllm/utils/infer_utils.py -> build/bdist.linux-armv7l/wheel/./lightllm/utils 2024-08-27T12:42:48,369 copying build/lib/lightllm/utils/net_utils.py -> build/bdist.linux-armv7l/wheel/./lightllm/utils 2024-08-27T12:42:48,371 copying build/lib/lightllm/utils/__init__.py -> build/bdist.linux-armv7l/wheel/./lightllm/utils 2024-08-27T12:42:48,372 copying build/lib/lightllm/utils/petrel_helper.py -> build/bdist.linux-armv7l/wheel/./lightllm/utils 2024-08-27T12:42:48,375 copying build/lib/lightllm/utils/health_check.py -> build/bdist.linux-armv7l/wheel/./lightllm/utils 2024-08-27T12:42:48,376 copying build/lib/lightllm/utils/graceful_utils.py -> build/bdist.linux-armv7l/wheel/./lightllm/utils 2024-08-27T12:42:48,378 copying build/lib/lightllm/utils/start_utils.py -> build/bdist.linux-armv7l/wheel/./lightllm/utils 2024-08-27T12:42:48,380 copying build/lib/lightllm/utils/log_utils.py -> build/bdist.linux-armv7l/wheel/./lightllm/utils 2024-08-27T12:42:48,382 creating build/bdist.linux-armv7l/wheel/lightllm/models 2024-08-27T12:42:48,384 creating build/bdist.linux-armv7l/wheel/lightllm/models/internlm_xcomposer 2024-08-27T12:42:48,385 creating build/bdist.linux-armv7l/wheel/lightllm/models/internlm_xcomposer/layer_infer 2024-08-27T12:42:48,386 copying build/lib/lightllm/models/internlm_xcomposer/layer_infer/__init__.py -> build/bdist.linux-armv7l/wheel/./lightllm/models/internlm_xcomposer/layer_infer 2024-08-27T12:42:48,388 copying build/lib/lightllm/models/internlm_xcomposer/layer_infer/transformer_layer_infer.py -> build/bdist.linux-armv7l/wheel/./lightllm/models/internlm_xcomposer/layer_infer 2024-08-27T12:42:48,390 copying build/lib/lightllm/models/internlm_xcomposer/__init__.py -> build/bdist.linux-armv7l/wheel/./lightllm/models/internlm_xcomposer 2024-08-27T12:42:48,391 copying build/lib/lightllm/models/internlm_xcomposer/internlm_visual.py -> build/bdist.linux-armv7l/wheel/./lightllm/models/internlm_xcomposer 2024-08-27T12:42:48,394 copying build/lib/lightllm/models/internlm_xcomposer/infer_struct.py -> build/bdist.linux-armv7l/wheel/./lightllm/models/internlm_xcomposer 2024-08-27T12:42:48,396 copying build/lib/lightllm/models/internlm_xcomposer/model.py -> build/bdist.linux-armv7l/wheel/./lightllm/models/internlm_xcomposer 2024-08-27T12:42:48,398 creating build/bdist.linux-armv7l/wheel/lightllm/models/internlm_xcomposer/layer_weights 2024-08-27T12:42:48,399 copying build/lib/lightllm/models/internlm_xcomposer/layer_weights/__init__.py -> build/bdist.linux-armv7l/wheel/./lightllm/models/internlm_xcomposer/layer_weights 2024-08-27T12:42:48,400 copying build/lib/lightllm/models/internlm_xcomposer/layer_weights/transformer_layer_weight.py -> build/bdist.linux-armv7l/wheel/./lightllm/models/internlm_xcomposer/layer_weights 2024-08-27T12:42:48,403 creating build/bdist.linux-armv7l/wheel/lightllm/models/qwen2 2024-08-27T12:42:48,404 creating build/bdist.linux-armv7l/wheel/lightllm/models/qwen2/layer_infer 2024-08-27T12:42:48,405 copying build/lib/lightllm/models/qwen2/layer_infer/__init__.py -> build/bdist.linux-armv7l/wheel/./lightllm/models/qwen2/layer_infer 2024-08-27T12:42:48,407 copying build/lib/lightllm/models/qwen2/layer_infer/transformer_layer_infer.py -> build/bdist.linux-armv7l/wheel/./lightllm/models/qwen2/layer_infer 2024-08-27T12:42:48,409 copying build/lib/lightllm/models/qwen2/__init__.py -> build/bdist.linux-armv7l/wheel/./lightllm/models/qwen2 2024-08-27T12:42:48,411 copying build/lib/lightllm/models/qwen2/infer_struct.py -> build/bdist.linux-armv7l/wheel/./lightllm/models/qwen2 2024-08-27T12:42:48,412 copying build/lib/lightllm/models/qwen2/model.py -> build/bdist.linux-armv7l/wheel/./lightllm/models/qwen2 2024-08-27T12:42:48,415 creating build/bdist.linux-armv7l/wheel/lightllm/models/qwen2/layer_weights 2024-08-27T12:42:48,416 copying build/lib/lightllm/models/qwen2/layer_weights/pre_and_post_layer_weight.py -> build/bdist.linux-armv7l/wheel/./lightllm/models/qwen2/layer_weights 2024-08-27T12:42:48,417 copying build/lib/lightllm/models/qwen2/layer_weights/__init__.py -> build/bdist.linux-armv7l/wheel/./lightllm/models/qwen2/layer_weights 2024-08-27T12:42:48,419 copying build/lib/lightllm/models/qwen2/layer_weights/transformer_layer_weight.py -> build/bdist.linux-armv7l/wheel/./lightllm/models/qwen2/layer_weights 2024-08-27T12:42:48,421 creating build/bdist.linux-armv7l/wheel/lightllm/models/gemma_2b 2024-08-27T12:42:48,423 creating build/bdist.linux-armv7l/wheel/lightllm/models/gemma_2b/layer_infer 2024-08-27T12:42:48,424 copying build/lib/lightllm/models/gemma_2b/layer_infer/__init__.py -> build/bdist.linux-armv7l/wheel/./lightllm/models/gemma_2b/layer_infer 2024-08-27T12:42:48,425 copying build/lib/lightllm/models/gemma_2b/layer_infer/transformer_layer_infer.py -> build/bdist.linux-armv7l/wheel/./lightllm/models/gemma_2b/layer_infer 2024-08-27T12:42:48,427 copying build/lib/lightllm/models/gemma_2b/layer_infer/pre_layer_infer.py -> build/bdist.linux-armv7l/wheel/./lightllm/models/gemma_2b/layer_infer 2024-08-27T12:42:48,429 copying build/lib/lightllm/models/gemma_2b/__init__.py -> build/bdist.linux-armv7l/wheel/./lightllm/models/gemma_2b 2024-08-27T12:42:48,431 creating build/bdist.linux-armv7l/wheel/lightllm/models/gemma_2b/triton_kernel 2024-08-27T12:42:48,432 copying build/lib/lightllm/models/gemma_2b/triton_kernel/__init__.py -> build/bdist.linux-armv7l/wheel/./lightllm/models/gemma_2b/triton_kernel 2024-08-27T12:42:48,433 copying build/lib/lightllm/models/gemma_2b/triton_kernel/gelu_and_mul.py -> build/bdist.linux-armv7l/wheel/./lightllm/models/gemma_2b/triton_kernel 2024-08-27T12:42:48,435 copying build/lib/lightllm/models/gemma_2b/model.py -> build/bdist.linux-armv7l/wheel/./lightllm/models/gemma_2b 2024-08-27T12:42:48,438 creating build/bdist.linux-armv7l/wheel/lightllm/models/gemma_2b/layer_weights 2024-08-27T12:42:48,439 copying build/lib/lightllm/models/gemma_2b/layer_weights/pre_and_post_layer_weight.py -> build/bdist.linux-armv7l/wheel/./lightllm/models/gemma_2b/layer_weights 2024-08-27T12:42:48,441 copying build/lib/lightllm/models/gemma_2b/layer_weights/__init__.py -> build/bdist.linux-armv7l/wheel/./lightllm/models/gemma_2b/layer_weights 2024-08-27T12:42:48,442 copying build/lib/lightllm/models/gemma_2b/layer_weights/transformer_layer_weight.py -> build/bdist.linux-armv7l/wheel/./lightllm/models/gemma_2b/layer_weights 2024-08-27T12:42:48,444 creating build/bdist.linux-armv7l/wheel/lightllm/models/llama_quik 2024-08-27T12:42:48,446 creating build/bdist.linux-armv7l/wheel/lightllm/models/llama_quik/layer_infer 2024-08-27T12:42:48,447 copying build/lib/lightllm/models/llama_quik/layer_infer/__init__.py -> build/bdist.linux-armv7l/wheel/./lightllm/models/llama_quik/layer_infer 2024-08-27T12:42:48,448 copying build/lib/lightllm/models/llama_quik/layer_infer/transformer_layer_infer.py -> build/bdist.linux-armv7l/wheel/./lightllm/models/llama_quik/layer_infer 2024-08-27T12:42:48,451 copying build/lib/lightllm/models/llama_quik/__init__.py -> build/bdist.linux-armv7l/wheel/./lightllm/models/llama_quik 2024-08-27T12:42:48,452 copying build/lib/lightllm/models/llama_quik/model.py -> build/bdist.linux-armv7l/wheel/./lightllm/models/llama_quik 2024-08-27T12:42:48,454 creating build/bdist.linux-armv7l/wheel/lightllm/models/llama_quik/cuda_kernel 2024-08-27T12:42:48,455 copying build/lib/lightllm/models/llama_quik/cuda_kernel/quik_awquant.py -> build/bdist.linux-armv7l/wheel/./lightllm/models/llama_quik/cuda_kernel 2024-08-27T12:42:48,457 copying build/lib/lightllm/models/llama_quik/cuda_kernel/__init__.py -> build/bdist.linux-armv7l/wheel/./lightllm/models/llama_quik/cuda_kernel 2024-08-27T12:42:48,460 creating build/bdist.linux-armv7l/wheel/lightllm/models/llama_quik/layer_weights 2024-08-27T12:42:48,461 copying build/lib/lightllm/models/llama_quik/layer_weights/__init__.py -> build/bdist.linux-armv7l/wheel/./lightllm/models/llama_quik/layer_weights 2024-08-27T12:42:48,462 copying build/lib/lightllm/models/llama_quik/layer_weights/qlinear.py -> build/bdist.linux-armv7l/wheel/./lightllm/models/llama_quik/layer_weights 2024-08-27T12:42:48,465 copying build/lib/lightllm/models/llama_quik/layer_weights/transformer_layer_weight.py -> build/bdist.linux-armv7l/wheel/./lightllm/models/llama_quik/layer_weights 2024-08-27T12:42:48,467 creating build/bdist.linux-armv7l/wheel/lightllm/models/baichuan7b 2024-08-27T12:42:48,468 copying build/lib/lightllm/models/baichuan7b/__init__.py -> build/bdist.linux-armv7l/wheel/./lightllm/models/baichuan7b 2024-08-27T12:42:48,470 copying build/lib/lightllm/models/baichuan7b/model.py -> build/bdist.linux-armv7l/wheel/./lightllm/models/baichuan7b 2024-08-27T12:42:48,472 creating build/bdist.linux-armv7l/wheel/lightllm/models/baichuan7b/layer_weights 2024-08-27T12:42:48,473 copying build/lib/lightllm/models/baichuan7b/layer_weights/__init__.py -> build/bdist.linux-armv7l/wheel/./lightllm/models/baichuan7b/layer_weights 2024-08-27T12:42:48,474 copying build/lib/lightllm/models/baichuan7b/layer_weights/transformer_layer_weight.py -> build/bdist.linux-armv7l/wheel/./lightllm/models/baichuan7b/layer_weights 2024-08-27T12:42:48,476 copying build/lib/lightllm/models/__init__.py -> build/bdist.linux-armv7l/wheel/./lightllm/models 2024-08-27T12:42:48,477 creating build/bdist.linux-armv7l/wheel/lightllm/models/internlm2_wquant 2024-08-27T12:42:48,479 copying build/lib/lightllm/models/internlm2_wquant/__init__.py -> build/bdist.linux-armv7l/wheel/./lightllm/models/internlm2_wquant 2024-08-27T12:42:48,480 copying build/lib/lightllm/models/internlm2_wquant/model.py -> build/bdist.linux-armv7l/wheel/./lightllm/models/internlm2_wquant 2024-08-27T12:42:48,482 creating build/bdist.linux-armv7l/wheel/lightllm/models/internlm2_wquant/layer_weights 2024-08-27T12:42:48,483 copying build/lib/lightllm/models/internlm2_wquant/layer_weights/__init__.py -> build/bdist.linux-armv7l/wheel/./lightllm/models/internlm2_wquant/layer_weights 2024-08-27T12:42:48,484 copying build/lib/lightllm/models/internlm2_wquant/layer_weights/transformer_layer_weight.py -> build/bdist.linux-armv7l/wheel/./lightllm/models/internlm2_wquant/layer_weights 2024-08-27T12:42:48,487 creating build/bdist.linux-armv7l/wheel/lightllm/models/yi 2024-08-27T12:42:48,488 copying build/lib/lightllm/models/yi/__init__.py -> build/bdist.linux-armv7l/wheel/./lightllm/models/yi 2024-08-27T12:42:48,490 copying build/lib/lightllm/models/yi/model.py -> build/bdist.linux-armv7l/wheel/./lightllm/models/yi 2024-08-27T12:42:48,492 creating build/bdist.linux-armv7l/wheel/lightllm/models/yi/layer_weights 2024-08-27T12:42:48,493 copying build/lib/lightllm/models/yi/layer_weights/__init__.py -> build/bdist.linux-armv7l/wheel/./lightllm/models/yi/layer_weights 2024-08-27T12:42:48,495 copying build/lib/lightllm/models/yi/layer_weights/transformer_layer_weight.py -> build/bdist.linux-armv7l/wheel/./lightllm/models/yi/layer_weights 2024-08-27T12:42:48,497 creating build/bdist.linux-armv7l/wheel/lightllm/models/baichuan2_7b 2024-08-27T12:42:48,499 creating build/bdist.linux-armv7l/wheel/lightllm/models/baichuan2_7b/layer_infer 2024-08-27T12:42:48,500 copying build/lib/lightllm/models/baichuan2_7b/layer_infer/__init__.py -> build/bdist.linux-armv7l/wheel/./lightllm/models/baichuan2_7b/layer_infer 2024-08-27T12:42:48,501 copying build/lib/lightllm/models/baichuan2_7b/layer_infer/transformer_layer_infer.py -> build/bdist.linux-armv7l/wheel/./lightllm/models/baichuan2_7b/layer_infer 2024-08-27T12:42:48,503 copying build/lib/lightllm/models/baichuan2_7b/__init__.py -> build/bdist.linux-armv7l/wheel/./lightllm/models/baichuan2_7b 2024-08-27T12:42:48,504 copying build/lib/lightllm/models/baichuan2_7b/model.py -> build/bdist.linux-armv7l/wheel/./lightllm/models/baichuan2_7b 2024-08-27T12:42:48,507 creating build/bdist.linux-armv7l/wheel/lightllm/models/baichuan2_7b/layer_weights 2024-08-27T12:42:48,508 copying build/lib/lightllm/models/baichuan2_7b/layer_weights/pre_and_post_layer_weight.py -> build/bdist.linux-armv7l/wheel/./lightllm/models/baichuan2_7b/layer_weights 2024-08-27T12:42:48,510 copying build/lib/lightllm/models/baichuan2_7b/layer_weights/__init__.py -> build/bdist.linux-armv7l/wheel/./lightllm/models/baichuan2_7b/layer_weights 2024-08-27T12:42:48,512 creating build/bdist.linux-armv7l/wheel/lightllm/models/llama_awquant 2024-08-27T12:42:48,513 creating build/bdist.linux-armv7l/wheel/lightllm/models/llama_awquant/layer_infer 2024-08-27T12:42:48,514 copying build/lib/lightllm/models/llama_awquant/layer_infer/__init__.py -> build/bdist.linux-armv7l/wheel/./lightllm/models/llama_awquant/layer_infer 2024-08-27T12:42:48,515 copying build/lib/lightllm/models/llama_awquant/layer_infer/transformer_layer_infer.py -> build/bdist.linux-armv7l/wheel/./lightllm/models/llama_awquant/layer_infer 2024-08-27T12:42:48,518 copying build/lib/lightllm/models/llama_awquant/__init__.py -> build/bdist.linux-armv7l/wheel/./lightllm/models/llama_awquant 2024-08-27T12:42:48,519 copying build/lib/lightllm/models/llama_awquant/model.py -> build/bdist.linux-armv7l/wheel/./lightllm/models/llama_awquant 2024-08-27T12:42:48,521 creating build/bdist.linux-armv7l/wheel/lightllm/models/llama_awquant/layer_weights 2024-08-27T12:42:48,522 copying build/lib/lightllm/models/llama_awquant/layer_weights/__init__.py -> build/bdist.linux-armv7l/wheel/./lightllm/models/llama_awquant/layer_weights 2024-08-27T12:42:48,523 copying build/lib/lightllm/models/llama_awquant/layer_weights/transformer_layer_weight.py -> build/bdist.linux-armv7l/wheel/./lightllm/models/llama_awquant/layer_weights 2024-08-27T12:42:48,526 creating build/bdist.linux-armv7l/wheel/lightllm/models/baichuan13b 2024-08-27T12:42:48,527 creating build/bdist.linux-armv7l/wheel/lightllm/models/baichuan13b/layer_infer 2024-08-27T12:42:48,528 copying build/lib/lightllm/models/baichuan13b/layer_infer/__init__.py -> build/bdist.linux-armv7l/wheel/./lightllm/models/baichuan13b/layer_infer 2024-08-27T12:42:48,530 copying build/lib/lightllm/models/baichuan13b/layer_infer/transformer_layer_infer.py -> build/bdist.linux-armv7l/wheel/./lightllm/models/baichuan13b/layer_infer 2024-08-27T12:42:48,532 copying build/lib/lightllm/models/baichuan13b/__init__.py -> build/bdist.linux-armv7l/wheel/./lightllm/models/baichuan13b 2024-08-27T12:42:48,533 copying build/lib/lightllm/models/baichuan13b/model.py -> build/bdist.linux-armv7l/wheel/./lightllm/models/baichuan13b 2024-08-27T12:42:48,536 creating build/bdist.linux-armv7l/wheel/lightllm/models/baichuan13b/layer_weights 2024-08-27T12:42:48,537 copying build/lib/lightllm/models/baichuan13b/layer_weights/__init__.py -> build/bdist.linux-armv7l/wheel/./lightllm/models/baichuan13b/layer_weights 2024-08-27T12:42:48,538 copying build/lib/lightllm/models/baichuan13b/layer_weights/transformer_layer_weight.py -> build/bdist.linux-armv7l/wheel/./lightllm/models/baichuan13b/layer_weights 2024-08-27T12:42:48,541 creating build/bdist.linux-armv7l/wheel/lightllm/models/qwen_wquant 2024-08-27T12:42:48,542 creating build/bdist.linux-armv7l/wheel/lightllm/models/qwen_wquant/layer_infer 2024-08-27T12:42:48,543 copying build/lib/lightllm/models/qwen_wquant/layer_infer/__init__.py -> build/bdist.linux-armv7l/wheel/./lightllm/models/qwen_wquant/layer_infer 2024-08-27T12:42:48,544 copying build/lib/lightllm/models/qwen_wquant/layer_infer/transformer_layer_infer.py -> build/bdist.linux-armv7l/wheel/./lightllm/models/qwen_wquant/layer_infer 2024-08-27T12:42:48,546 copying build/lib/lightllm/models/qwen_wquant/__init__.py -> build/bdist.linux-armv7l/wheel/./lightllm/models/qwen_wquant 2024-08-27T12:42:48,548 copying build/lib/lightllm/models/qwen_wquant/model.py -> build/bdist.linux-armv7l/wheel/./lightllm/models/qwen_wquant 2024-08-27T12:42:48,550 creating build/bdist.linux-armv7l/wheel/lightllm/models/qwen_wquant/layer_weights 2024-08-27T12:42:48,551 copying build/lib/lightllm/models/qwen_wquant/layer_weights/__init__.py -> build/bdist.linux-armv7l/wheel/./lightllm/models/qwen_wquant/layer_weights 2024-08-27T12:42:48,553 copying build/lib/lightllm/models/qwen_wquant/layer_weights/transformer_layer_weight.py -> build/bdist.linux-armv7l/wheel/./lightllm/models/qwen_wquant/layer_weights 2024-08-27T12:42:48,555 creating build/bdist.linux-armv7l/wheel/lightllm/models/llama_wquant 2024-08-27T12:42:48,556 creating build/bdist.linux-armv7l/wheel/lightllm/models/llama_wquant/layer_infer 2024-08-27T12:42:48,557 copying build/lib/lightllm/models/llama_wquant/layer_infer/__init__.py -> build/bdist.linux-armv7l/wheel/./lightllm/models/llama_wquant/layer_infer 2024-08-27T12:42:48,559 copying build/lib/lightllm/models/llama_wquant/layer_infer/transformer_layer_infer.py -> build/bdist.linux-armv7l/wheel/./lightllm/models/llama_wquant/layer_infer 2024-08-27T12:42:48,561 copying build/lib/lightllm/models/llama_wquant/__init__.py -> build/bdist.linux-armv7l/wheel/./lightllm/models/llama_wquant 2024-08-27T12:42:48,563 copying build/lib/lightllm/models/llama_wquant/model.py -> build/bdist.linux-armv7l/wheel/./lightllm/models/llama_wquant 2024-08-27T12:42:48,565 creating build/bdist.linux-armv7l/wheel/lightllm/models/llama_wquant/layer_weights 2024-08-27T12:42:48,565 copying build/lib/lightllm/models/llama_wquant/layer_weights/__init__.py -> build/bdist.linux-armv7l/wheel/./lightllm/models/llama_wquant/layer_weights 2024-08-27T12:42:48,567 copying build/lib/lightllm/models/llama_wquant/layer_weights/transformer_layer_weight.py -> build/bdist.linux-armv7l/wheel/./lightllm/models/llama_wquant/layer_weights 2024-08-27T12:42:48,570 creating build/bdist.linux-armv7l/wheel/lightllm/models/internlm 2024-08-27T12:42:48,571 creating build/bdist.linux-armv7l/wheel/lightllm/models/internlm/layer_infer 2024-08-27T12:42:48,572 copying build/lib/lightllm/models/internlm/layer_infer/__init__.py -> build/bdist.linux-armv7l/wheel/./lightllm/models/internlm/layer_infer 2024-08-27T12:42:48,573 copying build/lib/lightllm/models/internlm/layer_infer/transformer_layer_infer.py -> build/bdist.linux-armv7l/wheel/./lightllm/models/internlm/layer_infer 2024-08-27T12:42:48,575 copying build/lib/lightllm/models/internlm/__init__.py -> build/bdist.linux-armv7l/wheel/./lightllm/models/internlm 2024-08-27T12:42:48,577 copying build/lib/lightllm/models/internlm/model.py -> build/bdist.linux-armv7l/wheel/./lightllm/models/internlm 2024-08-27T12:42:48,580 creating build/bdist.linux-armv7l/wheel/lightllm/models/internlm/layer_weights 2024-08-27T12:42:48,580 copying build/lib/lightllm/models/internlm/layer_weights/__init__.py -> build/bdist.linux-armv7l/wheel/./lightllm/models/internlm/layer_weights 2024-08-27T12:42:48,582 copying build/lib/lightllm/models/internlm/layer_weights/transformer_layer_weight.py -> build/bdist.linux-armv7l/wheel/./lightllm/models/internlm/layer_weights 2024-08-27T12:42:48,585 creating build/bdist.linux-armv7l/wheel/lightllm/models/internlm_wquant 2024-08-27T12:42:48,586 creating build/bdist.linux-armv7l/wheel/lightllm/models/internlm_wquant/layer_infer 2024-08-27T12:42:48,587 copying build/lib/lightllm/models/internlm_wquant/layer_infer/__init__.py -> build/bdist.linux-armv7l/wheel/./lightllm/models/internlm_wquant/layer_infer 2024-08-27T12:42:48,589 copying build/lib/lightllm/models/internlm_wquant/layer_infer/transformer_layer_infer.py -> build/bdist.linux-armv7l/wheel/./lightllm/models/internlm_wquant/layer_infer 2024-08-27T12:42:48,591 copying build/lib/lightllm/models/internlm_wquant/__init__.py -> build/bdist.linux-armv7l/wheel/./lightllm/models/internlm_wquant 2024-08-27T12:42:48,592 copying build/lib/lightllm/models/internlm_wquant/model.py -> build/bdist.linux-armv7l/wheel/./lightllm/models/internlm_wquant 2024-08-27T12:42:48,595 creating build/bdist.linux-armv7l/wheel/lightllm/models/internlm_wquant/layer_weights 2024-08-27T12:42:48,596 copying build/lib/lightllm/models/internlm_wquant/layer_weights/__init__.py -> build/bdist.linux-armv7l/wheel/./lightllm/models/internlm_wquant/layer_weights 2024-08-27T12:42:48,597 copying build/lib/lightllm/models/internlm_wquant/layer_weights/transformer_layer_weight.py -> build/bdist.linux-armv7l/wheel/./lightllm/models/internlm_wquant/layer_weights 2024-08-27T12:42:48,600 creating build/bdist.linux-armv7l/wheel/lightllm/models/mistral 2024-08-27T12:42:48,601 creating build/bdist.linux-armv7l/wheel/lightllm/models/mistral/layer_infer 2024-08-27T12:42:48,602 copying build/lib/lightllm/models/mistral/layer_infer/__init__.py -> build/bdist.linux-armv7l/wheel/./lightllm/models/mistral/layer_infer 2024-08-27T12:42:48,603 copying build/lib/lightllm/models/mistral/layer_infer/transformer_layer_infer.py -> build/bdist.linux-armv7l/wheel/./lightllm/models/mistral/layer_infer 2024-08-27T12:42:48,605 copying build/lib/lightllm/models/mistral/__init__.py -> build/bdist.linux-armv7l/wheel/./lightllm/models/mistral 2024-08-27T12:42:48,607 copying build/lib/lightllm/models/mistral/infer_struct.py -> build/bdist.linux-armv7l/wheel/./lightllm/models/mistral 2024-08-27T12:42:48,609 creating build/bdist.linux-armv7l/wheel/lightllm/models/mistral/triton_kernel 2024-08-27T12:42:48,609 copying build/lib/lightllm/models/mistral/triton_kernel/token_attention_nopad_reduceV.py -> build/bdist.linux-armv7l/wheel/./lightllm/models/mistral/triton_kernel 2024-08-27T12:42:48,612 copying build/lib/lightllm/models/mistral/triton_kernel/__init__.py -> build/bdist.linux-armv7l/wheel/./lightllm/models/mistral/triton_kernel 2024-08-27T12:42:48,613 copying build/lib/lightllm/models/mistral/triton_kernel/context_flashattention_nopad.py -> build/bdist.linux-armv7l/wheel/./lightllm/models/mistral/triton_kernel 2024-08-27T12:42:48,615 copying build/lib/lightllm/models/mistral/triton_kernel/token_attention_nopad_att1.py -> build/bdist.linux-armv7l/wheel/./lightllm/models/mistral/triton_kernel 2024-08-27T12:42:48,617 copying build/lib/lightllm/models/mistral/triton_kernel/token_attention_softmax_and_reducev.py -> build/bdist.linux-armv7l/wheel/./lightllm/models/mistral/triton_kernel 2024-08-27T12:42:48,619 copying build/lib/lightllm/models/mistral/model.py -> build/bdist.linux-armv7l/wheel/./lightllm/models/mistral 2024-08-27T12:42:48,621 creating build/bdist.linux-armv7l/wheel/lightllm/models/starcoder_wquant 2024-08-27T12:42:48,623 creating build/bdist.linux-armv7l/wheel/lightllm/models/starcoder_wquant/layer_infer 2024-08-27T12:42:48,624 copying build/lib/lightllm/models/starcoder_wquant/layer_infer/__init__.py -> build/bdist.linux-armv7l/wheel/./lightllm/models/starcoder_wquant/layer_infer 2024-08-27T12:42:48,625 copying build/lib/lightllm/models/starcoder_wquant/layer_infer/transformer_layer_infer.py -> build/bdist.linux-armv7l/wheel/./lightllm/models/starcoder_wquant/layer_infer 2024-08-27T12:42:48,627 copying build/lib/lightllm/models/starcoder_wquant/__init__.py -> build/bdist.linux-armv7l/wheel/./lightllm/models/starcoder_wquant 2024-08-27T12:42:48,629 copying build/lib/lightllm/models/starcoder_wquant/model.py -> build/bdist.linux-armv7l/wheel/./lightllm/models/starcoder_wquant 2024-08-27T12:42:48,631 creating build/bdist.linux-armv7l/wheel/lightllm/models/starcoder_wquant/layer_weights 2024-08-27T12:42:48,632 copying build/lib/lightllm/models/starcoder_wquant/layer_weights/__init__.py -> build/bdist.linux-armv7l/wheel/./lightllm/models/starcoder_wquant/layer_weights 2024-08-27T12:42:48,634 copying build/lib/lightllm/models/starcoder_wquant/layer_weights/transformer_layer_weight.py -> build/bdist.linux-armv7l/wheel/./lightllm/models/starcoder_wquant/layer_weights 2024-08-27T12:42:48,636 creating build/bdist.linux-armv7l/wheel/lightllm/models/minicpm 2024-08-27T12:42:48,638 creating build/bdist.linux-armv7l/wheel/lightllm/models/minicpm/layer_infer 2024-08-27T12:42:48,639 copying build/lib/lightllm/models/minicpm/layer_infer/__init__.py -> build/bdist.linux-armv7l/wheel/./lightllm/models/minicpm/layer_infer 2024-08-27T12:42:48,640 copying build/lib/lightllm/models/minicpm/layer_infer/transformer_layer_infer.py -> build/bdist.linux-armv7l/wheel/./lightllm/models/minicpm/layer_infer 2024-08-27T12:42:48,642 copying build/lib/lightllm/models/minicpm/__init__.py -> build/bdist.linux-armv7l/wheel/./lightllm/models/minicpm 2024-08-27T12:42:48,644 copying build/lib/lightllm/models/minicpm/model.py -> build/bdist.linux-armv7l/wheel/./lightllm/models/minicpm 2024-08-27T12:42:48,646 creating build/bdist.linux-armv7l/wheel/lightllm/models/minicpm/layer_weights 2024-08-27T12:42:48,647 copying build/lib/lightllm/models/minicpm/layer_weights/pre_and_post_layer_weight.py -> build/bdist.linux-armv7l/wheel/./lightllm/models/minicpm/layer_weights 2024-08-27T12:42:48,648 copying build/lib/lightllm/models/minicpm/layer_weights/__init__.py -> build/bdist.linux-armv7l/wheel/./lightllm/models/minicpm/layer_weights 2024-08-27T12:42:48,650 copying build/lib/lightllm/models/minicpm/layer_weights/transformer_layer_weight.py -> build/bdist.linux-armv7l/wheel/./lightllm/models/minicpm/layer_weights 2024-08-27T12:42:48,652 creating build/bdist.linux-armv7l/wheel/lightllm/models/chatglm2 2024-08-27T12:42:48,653 creating build/bdist.linux-armv7l/wheel/lightllm/models/chatglm2/layer_infer 2024-08-27T12:42:48,654 copying build/lib/lightllm/models/chatglm2/layer_infer/__init__.py -> build/bdist.linux-armv7l/wheel/./lightllm/models/chatglm2/layer_infer 2024-08-27T12:42:48,656 copying build/lib/lightllm/models/chatglm2/layer_infer/transformer_layer_infer.py -> build/bdist.linux-armv7l/wheel/./lightllm/models/chatglm2/layer_infer 2024-08-27T12:42:48,658 copying build/lib/lightllm/models/chatglm2/__init__.py -> build/bdist.linux-armv7l/wheel/./lightllm/models/chatglm2 2024-08-27T12:42:48,660 creating build/bdist.linux-armv7l/wheel/lightllm/models/chatglm2/triton_kernel 2024-08-27T12:42:48,660 copying build/lib/lightllm/models/chatglm2/triton_kernel/__init__.py -> build/bdist.linux-armv7l/wheel/./lightllm/models/chatglm2/triton_kernel 2024-08-27T12:42:48,662 copying build/lib/lightllm/models/chatglm2/triton_kernel/rotary_emb.py -> build/bdist.linux-armv7l/wheel/./lightllm/models/chatglm2/triton_kernel 2024-08-27T12:42:48,664 copying build/lib/lightllm/models/chatglm2/model.py -> build/bdist.linux-armv7l/wheel/./lightllm/models/chatglm2 2024-08-27T12:42:48,666 creating build/bdist.linux-armv7l/wheel/lightllm/models/chatglm2/layer_weights 2024-08-27T12:42:48,667 copying build/lib/lightllm/models/chatglm2/layer_weights/pre_and_post_layer_weight.py -> build/bdist.linux-armv7l/wheel/./lightllm/models/chatglm2/layer_weights 2024-08-27T12:42:48,669 copying build/lib/lightllm/models/chatglm2/layer_weights/__init__.py -> build/bdist.linux-armv7l/wheel/./lightllm/models/chatglm2/layer_weights 2024-08-27T12:42:48,671 copying build/lib/lightllm/models/chatglm2/layer_weights/transformer_layer_weight.py -> build/bdist.linux-armv7l/wheel/./lightllm/models/chatglm2/layer_weights 2024-08-27T12:42:48,673 creating build/bdist.linux-armv7l/wheel/lightllm/models/internlm2 2024-08-27T12:42:48,674 copying build/lib/lightllm/models/internlm2/__init__.py -> build/bdist.linux-armv7l/wheel/./lightllm/models/internlm2 2024-08-27T12:42:48,676 copying build/lib/lightllm/models/internlm2/model.py -> build/bdist.linux-armv7l/wheel/./lightllm/models/internlm2 2024-08-27T12:42:48,678 creating build/bdist.linux-armv7l/wheel/lightllm/models/internlm2/layer_weights 2024-08-27T12:42:48,679 copying build/lib/lightllm/models/internlm2/layer_weights/pre_and_post_layer_weight.py -> build/bdist.linux-armv7l/wheel/./lightllm/models/internlm2/layer_weights 2024-08-27T12:42:48,680 copying build/lib/lightllm/models/internlm2/layer_weights/__init__.py -> build/bdist.linux-armv7l/wheel/./lightllm/models/internlm2/layer_weights 2024-08-27T12:42:48,682 copying build/lib/lightllm/models/internlm2/layer_weights/transformer_layer_weight.py -> build/bdist.linux-armv7l/wheel/./lightllm/models/internlm2/layer_weights 2024-08-27T12:42:48,685 creating build/bdist.linux-armv7l/wheel/lightllm/models/starcoder2 2024-08-27T12:42:48,686 creating build/bdist.linux-armv7l/wheel/lightllm/models/starcoder2/layer_infer 2024-08-27T12:42:48,687 copying build/lib/lightllm/models/starcoder2/layer_infer/__init__.py -> build/bdist.linux-armv7l/wheel/./lightllm/models/starcoder2/layer_infer 2024-08-27T12:42:48,689 copying build/lib/lightllm/models/starcoder2/layer_infer/transformer_layer_infer.py -> build/bdist.linux-armv7l/wheel/./lightllm/models/starcoder2/layer_infer 2024-08-27T12:42:48,691 copying build/lib/lightllm/models/starcoder2/__init__.py -> build/bdist.linux-armv7l/wheel/./lightllm/models/starcoder2 2024-08-27T12:42:48,692 copying build/lib/lightllm/models/starcoder2/model.py -> build/bdist.linux-armv7l/wheel/./lightllm/models/starcoder2 2024-08-27T12:42:48,694 creating build/bdist.linux-armv7l/wheel/lightllm/models/starcoder2/layer_weights 2024-08-27T12:42:48,695 copying build/lib/lightllm/models/starcoder2/layer_weights/pre_and_post_layer_weight.py -> build/bdist.linux-armv7l/wheel/./lightllm/models/starcoder2/layer_weights 2024-08-27T12:42:48,697 copying build/lib/lightllm/models/starcoder2/layer_weights/__init__.py -> build/bdist.linux-armv7l/wheel/./lightllm/models/starcoder2/layer_weights 2024-08-27T12:42:48,698 copying build/lib/lightllm/models/starcoder2/layer_weights/transformer_layer_weight.py -> build/bdist.linux-armv7l/wheel/./lightllm/models/starcoder2/layer_weights 2024-08-27T12:42:48,700 creating build/bdist.linux-armv7l/wheel/lightllm/models/stablelm 2024-08-27T12:42:48,702 creating build/bdist.linux-armv7l/wheel/lightllm/models/stablelm/layer_infer 2024-08-27T12:42:48,703 copying build/lib/lightllm/models/stablelm/layer_infer/__init__.py -> build/bdist.linux-armv7l/wheel/./lightllm/models/stablelm/layer_infer 2024-08-27T12:42:48,704 copying build/lib/lightllm/models/stablelm/layer_infer/transformer_layer_infer.py -> build/bdist.linux-armv7l/wheel/./lightllm/models/stablelm/layer_infer 2024-08-27T12:42:48,706 copying build/lib/lightllm/models/stablelm/__init__.py -> build/bdist.linux-armv7l/wheel/./lightllm/models/stablelm 2024-08-27T12:42:48,707 copying build/lib/lightllm/models/stablelm/model.py -> build/bdist.linux-armv7l/wheel/./lightllm/models/stablelm 2024-08-27T12:42:48,710 creating build/bdist.linux-armv7l/wheel/lightllm/models/stablelm/layer_weights 2024-08-27T12:42:48,711 copying build/lib/lightllm/models/stablelm/layer_weights/pre_and_post_layer_weight.py -> build/bdist.linux-armv7l/wheel/./lightllm/models/stablelm/layer_weights 2024-08-27T12:42:48,713 copying build/lib/lightllm/models/stablelm/layer_weights/__init__.py -> build/bdist.linux-armv7l/wheel/./lightllm/models/stablelm/layer_weights 2024-08-27T12:42:48,714 copying build/lib/lightllm/models/stablelm/layer_weights/transformer_layer_weight.py -> build/bdist.linux-armv7l/wheel/./lightllm/models/stablelm/layer_weights 2024-08-27T12:42:48,717 creating build/bdist.linux-armv7l/wheel/lightllm/models/qwen 2024-08-27T12:42:48,718 creating build/bdist.linux-armv7l/wheel/lightllm/models/qwen/layer_infer 2024-08-27T12:42:48,719 copying build/lib/lightllm/models/qwen/layer_infer/__init__.py -> build/bdist.linux-armv7l/wheel/./lightllm/models/qwen/layer_infer 2024-08-27T12:42:48,721 copying build/lib/lightllm/models/qwen/layer_infer/transformer_layer_infer.py -> build/bdist.linux-armv7l/wheel/./lightllm/models/qwen/layer_infer 2024-08-27T12:42:48,723 copying build/lib/lightllm/models/qwen/__init__.py -> build/bdist.linux-armv7l/wheel/./lightllm/models/qwen 2024-08-27T12:42:48,724 copying build/lib/lightllm/models/qwen/infer_struct.py -> build/bdist.linux-armv7l/wheel/./lightllm/models/qwen 2024-08-27T12:42:48,726 copying build/lib/lightllm/models/qwen/model.py -> build/bdist.linux-armv7l/wheel/./lightllm/models/qwen 2024-08-27T12:42:48,728 creating build/bdist.linux-armv7l/wheel/lightllm/models/qwen/layer_weights 2024-08-27T12:42:48,729 copying build/lib/lightllm/models/qwen/layer_weights/pre_and_post_layer_weight.py -> build/bdist.linux-armv7l/wheel/./lightllm/models/qwen/layer_weights 2024-08-27T12:42:48,731 copying build/lib/lightllm/models/qwen/layer_weights/__init__.py -> build/bdist.linux-armv7l/wheel/./lightllm/models/qwen/layer_weights 2024-08-27T12:42:48,732 copying build/lib/lightllm/models/qwen/layer_weights/transformer_layer_weight.py -> build/bdist.linux-armv7l/wheel/./lightllm/models/qwen/layer_weights 2024-08-27T12:42:48,735 creating build/bdist.linux-armv7l/wheel/lightllm/models/llama 2024-08-27T12:42:48,736 creating build/bdist.linux-armv7l/wheel/lightllm/models/llama/layer_infer 2024-08-27T12:42:48,737 copying build/lib/lightllm/models/llama/layer_infer/__init__.py -> build/bdist.linux-armv7l/wheel/./lightllm/models/llama/layer_infer 2024-08-27T12:42:48,739 copying build/lib/lightllm/models/llama/layer_infer/transformer_layer_infer.py -> build/bdist.linux-armv7l/wheel/./lightllm/models/llama/layer_infer 2024-08-27T12:42:48,741 copying build/lib/lightllm/models/llama/layer_infer/post_layer_infer.py -> build/bdist.linux-armv7l/wheel/./lightllm/models/llama/layer_infer 2024-08-27T12:42:48,743 copying build/lib/lightllm/models/llama/layer_infer/pre_layer_infer.py -> build/bdist.linux-armv7l/wheel/./lightllm/models/llama/layer_infer 2024-08-27T12:42:48,745 copying build/lib/lightllm/models/llama/splitfuse_infer_struct.py -> build/bdist.linux-armv7l/wheel/./lightllm/models/llama 2024-08-27T12:42:48,747 copying build/lib/lightllm/models/llama/__init__.py -> build/bdist.linux-armv7l/wheel/./lightllm/models/llama 2024-08-27T12:42:48,748 copying build/lib/lightllm/models/llama/yarn_rotary_utils.py -> build/bdist.linux-armv7l/wheel/./lightllm/models/llama 2024-08-27T12:42:48,750 copying build/lib/lightllm/models/llama/infer_struct.py -> build/bdist.linux-armv7l/wheel/./lightllm/models/llama 2024-08-27T12:42:48,752 creating build/bdist.linux-armv7l/wheel/lightllm/models/llama/triton_kernel 2024-08-27T12:42:48,753 copying build/lib/lightllm/models/llama/triton_kernel/token_attention_nopad_reduceV.py -> build/bdist.linux-armv7l/wheel/./lightllm/models/llama/triton_kernel 2024-08-27T12:42:48,755 copying build/lib/lightllm/models/llama/triton_kernel/token_attention_nopad_softmax.py -> build/bdist.linux-armv7l/wheel/./lightllm/models/llama/triton_kernel 2024-08-27T12:42:48,757 copying build/lib/lightllm/models/llama/triton_kernel/flash_decoding.py -> build/bdist.linux-armv7l/wheel/./lightllm/models/llama/triton_kernel 2024-08-27T12:42:48,759 copying build/lib/lightllm/models/llama/triton_kernel/ppl_int4kv_copy_kv.py -> build/bdist.linux-armv7l/wheel/./lightllm/models/llama/triton_kernel 2024-08-27T12:42:48,761 copying build/lib/lightllm/models/llama/triton_kernel/__init__.py -> build/bdist.linux-armv7l/wheel/./lightllm/models/llama/triton_kernel 2024-08-27T12:42:48,763 copying build/lib/lightllm/models/llama/triton_kernel/splitfuse_context_flashattention_nopad.py -> build/bdist.linux-armv7l/wheel/./lightllm/models/llama/triton_kernel 2024-08-27T12:42:48,765 copying build/lib/lightllm/models/llama/triton_kernel/context_flashattention_nopad.py -> build/bdist.linux-armv7l/wheel/./lightllm/models/llama/triton_kernel 2024-08-27T12:42:48,767 copying build/lib/lightllm/models/llama/triton_kernel/gqa_flash_decoding.py -> build/bdist.linux-armv7l/wheel/./lightllm/models/llama/triton_kernel 2024-08-27T12:42:48,769 copying build/lib/lightllm/models/llama/triton_kernel/silu_and_mul.py -> build/bdist.linux-armv7l/wheel/./lightllm/models/llama/triton_kernel 2024-08-27T12:42:48,771 copying build/lib/lightllm/models/llama/triton_kernel/ppl_quant_copy_kv.py -> build/bdist.linux-armv7l/wheel/./lightllm/models/llama/triton_kernel 2024-08-27T12:42:48,773 copying build/lib/lightllm/models/llama/triton_kernel/flash_decoding_stage1.py -> build/bdist.linux-armv7l/wheel/./lightllm/models/llama/triton_kernel 2024-08-27T12:42:48,775 copying build/lib/lightllm/models/llama/triton_kernel/token_attention_nopad_att1.py -> build/bdist.linux-armv7l/wheel/./lightllm/models/llama/triton_kernel 2024-08-27T12:42:48,777 copying build/lib/lightllm/models/llama/triton_kernel/ppl_fp16_flash_decoding.py -> build/bdist.linux-armv7l/wheel/./lightllm/models/llama/triton_kernel 2024-08-27T12:42:48,779 copying build/lib/lightllm/models/llama/triton_kernel/ppl_int8kv_flash_decoding.py -> build/bdist.linux-armv7l/wheel/./lightllm/models/llama/triton_kernel 2024-08-27T12:42:48,781 copying build/lib/lightllm/models/llama/triton_kernel/rmsnorm.py -> build/bdist.linux-armv7l/wheel/./lightllm/models/llama/triton_kernel 2024-08-27T12:42:48,783 copying build/lib/lightllm/models/llama/triton_kernel/gqa_flash_decoding_stage2.py -> build/bdist.linux-armv7l/wheel/./lightllm/models/llama/triton_kernel 2024-08-27T12:42:48,785 copying build/lib/lightllm/models/llama/triton_kernel/rotary_emb.py -> build/bdist.linux-armv7l/wheel/./lightllm/models/llama/triton_kernel 2024-08-27T12:42:48,787 copying build/lib/lightllm/models/llama/triton_kernel/gqa_flash_decoding_stage1.py -> build/bdist.linux-armv7l/wheel/./lightllm/models/llama/triton_kernel 2024-08-27T12:42:48,789 copying build/lib/lightllm/models/llama/triton_kernel/token_attention_softmax_and_reducev.py -> build/bdist.linux-armv7l/wheel/./lightllm/models/llama/triton_kernel 2024-08-27T12:42:48,791 copying build/lib/lightllm/models/llama/triton_kernel/flash_decoding_stage2.py -> build/bdist.linux-armv7l/wheel/./lightllm/models/llama/triton_kernel 2024-08-27T12:42:48,792 copying build/lib/lightllm/models/llama/triton_kernel/ppl_int4kv_flash_decoding.py -> build/bdist.linux-armv7l/wheel/./lightllm/models/llama/triton_kernel 2024-08-27T12:42:48,794 copying build/lib/lightllm/models/llama/triton_kernel/gqa_decode_flashattention_nopad.py -> build/bdist.linux-armv7l/wheel/./lightllm/models/llama/triton_kernel 2024-08-27T12:42:48,801 copying build/lib/lightllm/models/llama/model.py -> build/bdist.linux-armv7l/wheel/./lightllm/models/llama 2024-08-27T12:42:48,804 creating build/bdist.linux-armv7l/wheel/lightllm/models/llama/layer_weights 2024-08-27T12:42:48,804 copying build/lib/lightllm/models/llama/layer_weights/pre_and_post_layer_weight.py -> build/bdist.linux-armv7l/wheel/./lightllm/models/llama/layer_weights 2024-08-27T12:42:48,807 copying build/lib/lightllm/models/llama/layer_weights/__init__.py -> build/bdist.linux-armv7l/wheel/./lightllm/models/llama/layer_weights 2024-08-27T12:42:48,808 copying build/lib/lightllm/models/llama/layer_weights/transformer_layer_weight.py -> build/bdist.linux-armv7l/wheel/./lightllm/models/llama/layer_weights 2024-08-27T12:42:48,810 copying build/lib/lightllm/models/llama/layer_weights/ds_load_utils.py -> build/bdist.linux-armv7l/wheel/./lightllm/models/llama/layer_weights 2024-08-27T12:42:48,812 creating build/bdist.linux-armv7l/wheel/lightllm/models/bloom 2024-08-27T12:42:48,813 creating build/bdist.linux-armv7l/wheel/lightllm/models/bloom/layer_infer 2024-08-27T12:42:48,814 copying build/lib/lightllm/models/bloom/layer_infer/__init__.py -> build/bdist.linux-armv7l/wheel/./lightllm/models/bloom/layer_infer 2024-08-27T12:42:48,816 copying build/lib/lightllm/models/bloom/layer_infer/transformer_layer_infer.py -> build/bdist.linux-armv7l/wheel/./lightllm/models/bloom/layer_infer 2024-08-27T12:42:48,818 copying build/lib/lightllm/models/bloom/layer_infer/post_layer_infer.py -> build/bdist.linux-armv7l/wheel/./lightllm/models/bloom/layer_infer 2024-08-27T12:42:48,820 copying build/lib/lightllm/models/bloom/layer_infer/pre_layer_infer.py -> build/bdist.linux-armv7l/wheel/./lightllm/models/bloom/layer_infer 2024-08-27T12:42:48,822 copying build/lib/lightllm/models/bloom/__init__.py -> build/bdist.linux-armv7l/wheel/./lightllm/models/bloom 2024-08-27T12:42:48,824 creating build/bdist.linux-armv7l/wheel/lightllm/models/bloom/triton_kernel 2024-08-27T12:42:48,825 copying build/lib/lightllm/models/bloom/triton_kernel/token_attention_nopad_reduceV.py -> build/bdist.linux-armv7l/wheel/./lightllm/models/bloom/triton_kernel 2024-08-27T12:42:48,827 copying build/lib/lightllm/models/bloom/triton_kernel/token_attention_nopad_softmax.py -> build/bdist.linux-armv7l/wheel/./lightllm/models/bloom/triton_kernel 2024-08-27T12:42:48,829 copying build/lib/lightllm/models/bloom/triton_kernel/token_flashattention_nopad.py -> build/bdist.linux-armv7l/wheel/./lightllm/models/bloom/triton_kernel 2024-08-27T12:42:48,831 copying build/lib/lightllm/models/bloom/triton_kernel/__init__.py -> build/bdist.linux-armv7l/wheel/./lightllm/models/bloom/triton_kernel 2024-08-27T12:42:48,832 copying build/lib/lightllm/models/bloom/triton_kernel/context_flashattention_nopad.py -> build/bdist.linux-armv7l/wheel/./lightllm/models/bloom/triton_kernel 2024-08-27T12:42:48,834 copying build/lib/lightllm/models/bloom/triton_kernel/token_attention_nopad_att1.py -> build/bdist.linux-armv7l/wheel/./lightllm/models/bloom/triton_kernel 2024-08-27T12:42:48,836 copying build/lib/lightllm/models/bloom/triton_kernel/layernorm.py -> build/bdist.linux-armv7l/wheel/./lightllm/models/bloom/triton_kernel 2024-08-27T12:42:48,838 copying build/lib/lightllm/models/bloom/model.py -> build/bdist.linux-armv7l/wheel/./lightllm/models/bloom 2024-08-27T12:42:48,840 creating build/bdist.linux-armv7l/wheel/lightllm/models/bloom/layer_weights 2024-08-27T12:42:48,841 copying build/lib/lightllm/models/bloom/layer_weights/pre_and_post_layer_weight.py -> build/bdist.linux-armv7l/wheel/./lightllm/models/bloom/layer_weights 2024-08-27T12:42:48,843 copying build/lib/lightllm/models/bloom/layer_weights/__init__.py -> build/bdist.linux-armv7l/wheel/./lightllm/models/bloom/layer_weights 2024-08-27T12:42:48,845 copying build/lib/lightllm/models/bloom/layer_weights/hf_load_utils.py -> build/bdist.linux-armv7l/wheel/./lightllm/models/bloom/layer_weights 2024-08-27T12:42:48,846 copying build/lib/lightllm/models/bloom/layer_weights/transformer_layer_weight.py -> build/bdist.linux-armv7l/wheel/./lightllm/models/bloom/layer_weights 2024-08-27T12:42:48,849 creating build/bdist.linux-armv7l/wheel/lightllm/models/baichuan2_13b 2024-08-27T12:42:48,850 copying build/lib/lightllm/models/baichuan2_13b/__init__.py -> build/bdist.linux-armv7l/wheel/./lightllm/models/baichuan2_13b 2024-08-27T12:42:48,851 copying build/lib/lightllm/models/baichuan2_13b/model.py -> build/bdist.linux-armv7l/wheel/./lightllm/models/baichuan2_13b 2024-08-27T12:42:48,854 creating build/bdist.linux-armv7l/wheel/lightllm/models/starcoder 2024-08-27T12:42:48,855 creating build/bdist.linux-armv7l/wheel/lightllm/models/starcoder/layer_infer 2024-08-27T12:42:48,856 copying build/lib/lightllm/models/starcoder/layer_infer/__init__.py -> build/bdist.linux-armv7l/wheel/./lightllm/models/starcoder/layer_infer 2024-08-27T12:42:48,858 copying build/lib/lightllm/models/starcoder/layer_infer/transformer_layer_infer.py -> build/bdist.linux-armv7l/wheel/./lightllm/models/starcoder/layer_infer 2024-08-27T12:42:48,859 copying build/lib/lightllm/models/starcoder/layer_infer/pre_layer_infer.py -> build/bdist.linux-armv7l/wheel/./lightllm/models/starcoder/layer_infer 2024-08-27T12:42:48,861 copying build/lib/lightllm/models/starcoder/__init__.py -> build/bdist.linux-armv7l/wheel/./lightllm/models/starcoder 2024-08-27T12:42:48,863 copying build/lib/lightllm/models/starcoder/infer_struct.py -> build/bdist.linux-armv7l/wheel/./lightllm/models/starcoder 2024-08-27T12:42:48,864 copying build/lib/lightllm/models/starcoder/model.py -> build/bdist.linux-armv7l/wheel/./lightllm/models/starcoder 2024-08-27T12:42:48,867 creating build/bdist.linux-armv7l/wheel/lightllm/models/starcoder/layer_weights 2024-08-27T12:42:48,868 copying build/lib/lightllm/models/starcoder/layer_weights/pre_and_post_layer_weight.py -> build/bdist.linux-armv7l/wheel/./lightllm/models/starcoder/layer_weights 2024-08-27T12:42:48,870 copying build/lib/lightllm/models/starcoder/layer_weights/__init__.py -> build/bdist.linux-armv7l/wheel/./lightllm/models/starcoder/layer_weights 2024-08-27T12:42:48,872 copying build/lib/lightllm/models/starcoder/layer_weights/transformer_layer_weight.py -> build/bdist.linux-armv7l/wheel/./lightllm/models/starcoder/layer_weights 2024-08-27T12:42:48,874 creating build/bdist.linux-armv7l/wheel/lightllm/models/qwen_vl 2024-08-27T12:42:48,875 creating build/bdist.linux-armv7l/wheel/lightllm/models/qwen_vl/layer_infer 2024-08-27T12:42:48,876 copying build/lib/lightllm/models/qwen_vl/layer_infer/__init__.py -> build/bdist.linux-armv7l/wheel/./lightllm/models/qwen_vl/layer_infer 2024-08-27T12:42:48,878 copying build/lib/lightllm/models/qwen_vl/layer_infer/pre_layer_infer.py -> build/bdist.linux-armv7l/wheel/./lightllm/models/qwen_vl/layer_infer 2024-08-27T12:42:48,880 copying build/lib/lightllm/models/qwen_vl/__init__.py -> build/bdist.linux-armv7l/wheel/./lightllm/models/qwen_vl 2024-08-27T12:42:48,882 copying build/lib/lightllm/models/qwen_vl/qwen_visual.py -> build/bdist.linux-armv7l/wheel/./lightllm/models/qwen_vl 2024-08-27T12:42:48,884 copying build/lib/lightllm/models/qwen_vl/model.py -> build/bdist.linux-armv7l/wheel/./lightllm/models/qwen_vl 2024-08-27T12:42:48,886 creating build/bdist.linux-armv7l/wheel/lightllm/models/llava 2024-08-27T12:42:48,887 copying build/lib/lightllm/models/llava/llava_visual.py -> build/bdist.linux-armv7l/wheel/./lightllm/models/llava 2024-08-27T12:42:48,889 copying build/lib/lightllm/models/llava/__init__.py -> build/bdist.linux-armv7l/wheel/./lightllm/models/llava 2024-08-27T12:42:48,890 copying build/lib/lightllm/models/llava/model.py -> build/bdist.linux-armv7l/wheel/./lightllm/models/llava 2024-08-27T12:42:48,892 creating build/bdist.linux-armv7l/wheel/lightllm/models/mixtral 2024-08-27T12:42:48,893 creating build/bdist.linux-armv7l/wheel/lightllm/models/mixtral/layer_infer 2024-08-27T12:42:48,894 copying build/lib/lightllm/models/mixtral/layer_infer/__init__.py -> build/bdist.linux-armv7l/wheel/./lightllm/models/mixtral/layer_infer 2024-08-27T12:42:48,896 copying build/lib/lightllm/models/mixtral/layer_infer/transformer_layer_infer.py -> build/bdist.linux-armv7l/wheel/./lightllm/models/mixtral/layer_infer 2024-08-27T12:42:48,898 copying build/lib/lightllm/models/mixtral/__init__.py -> build/bdist.linux-armv7l/wheel/./lightllm/models/mixtral 2024-08-27T12:42:48,899 copying build/lib/lightllm/models/mixtral/infer_struct.py -> build/bdist.linux-armv7l/wheel/./lightllm/models/mixtral 2024-08-27T12:42:48,901 copying build/lib/lightllm/models/mixtral/model.py -> build/bdist.linux-armv7l/wheel/./lightllm/models/mixtral 2024-08-27T12:42:48,903 creating build/bdist.linux-armv7l/wheel/lightllm/models/mixtral/layer_weights 2024-08-27T12:42:48,904 copying build/lib/lightllm/models/mixtral/layer_weights/__init__.py -> build/bdist.linux-armv7l/wheel/./lightllm/models/mixtral/layer_weights 2024-08-27T12:42:48,906 copying build/lib/lightllm/models/mixtral/layer_weights/transformer_layer_weight.py -> build/bdist.linux-armv7l/wheel/./lightllm/models/mixtral/layer_weights 2024-08-27T12:42:48,908 running install_egg_info 2024-08-27T12:42:48,939 running egg_info 2024-08-27T12:42:48,965 writing lightllm.egg-info/PKG-INFO 2024-08-27T12:42:48,967 writing dependency_links to lightllm.egg-info/dependency_links.txt 2024-08-27T12:42:48,969 writing requirements to lightllm.egg-info/requires.txt 2024-08-27T12:42:48,971 writing top-level names to lightllm.egg-info/top_level.txt 2024-08-27T12:42:49,087 reading manifest file 'lightllm.egg-info/SOURCES.txt' 2024-08-27T12:42:49,106 adding license file 'LICENSE' 2024-08-27T12:42:49,123 writing manifest file 'lightllm.egg-info/SOURCES.txt' 2024-08-27T12:42:49,125 Copying lightllm.egg-info to build/bdist.linux-armv7l/wheel/./lightllm-0.0.1-py3.11.egg-info 2024-08-27T12:42:49,135 running install_scripts 2024-08-27T12:42:49,150 creating build/bdist.linux-armv7l/wheel/lightllm-0.0.1.dist-info/WHEEL 2024-08-27T12:42:49,152 creating '/tmp/pip-wheel-18cpawgl/lightllm-0.0.1-py3-none-any.whl' and adding 'build/bdist.linux-armv7l/wheel' to it 2024-08-27T12:42:49,154 adding 'lightllm/__init__.py' 2024-08-27T12:42:49,156 adding 'lightllm/common/__init__.py' 2024-08-27T12:42:49,157 adding 'lightllm/common/build_utils.py' 2024-08-27T12:42:49,159 adding 'lightllm/common/infer_utils.py' 2024-08-27T12:42:49,160 adding 'lightllm/common/int8kv_mem_manager.py' 2024-08-27T12:42:49,161 adding 'lightllm/common/mem_manager.py' 2024-08-27T12:42:49,162 adding 'lightllm/common/mem_utils.py' 2024-08-27T12:42:49,164 adding 'lightllm/common/ppl_int4kv_mem_manager.py' 2024-08-27T12:42:49,165 adding 'lightllm/common/ppl_int8kv_mem_manager.py' 2024-08-27T12:42:49,166 adding 'lightllm/common/req_manager.py' 2024-08-27T12:42:49,168 adding 'lightllm/common/basemodel/__init__.py' 2024-08-27T12:42:49,170 adding 'lightllm/common/basemodel/basemodel.py' 2024-08-27T12:42:49,171 adding 'lightllm/common/basemodel/infer_struct.py' 2024-08-27T12:42:49,172 adding 'lightllm/common/basemodel/splitfuse_infer_struct.py' 2024-08-27T12:42:49,174 adding 'lightllm/common/basemodel/cuda_kernel/__init__.py' 2024-08-27T12:42:49,175 adding 'lightllm/common/basemodel/cuda_kernel/fast_llm_wquant.py' 2024-08-27T12:42:49,176 adding 'lightllm/common/basemodel/cuda_kernel/lmdeploy_wquant.py' 2024-08-27T12:42:49,177 adding 'lightllm/common/basemodel/cuda_kernel/ppl_awquant.py' 2024-08-27T12:42:49,179 adding 'lightllm/common/basemodel/cuda_kernel/ppl_wquant.py' 2024-08-27T12:42:49,180 adding 'lightllm/common/basemodel/layer_infer/__init__.py' 2024-08-27T12:42:49,182 adding 'lightllm/common/basemodel/layer_infer/base_layer_infer.py' 2024-08-27T12:42:49,183 adding 'lightllm/common/basemodel/layer_infer/post_layer_infer.py' 2024-08-27T12:42:49,184 adding 'lightllm/common/basemodel/layer_infer/pre_layer_infer.py' 2024-08-27T12:42:49,185 adding 'lightllm/common/basemodel/layer_infer/transformer_layer_infer.py' 2024-08-27T12:42:49,187 adding 'lightllm/common/basemodel/layer_infer/template/__init__.py' 2024-08-27T12:42:49,188 adding 'lightllm/common/basemodel/layer_infer/template/post_layer_infer_template.py' 2024-08-27T12:42:49,189 adding 'lightllm/common/basemodel/layer_infer/template/pre_layer_infer_template.py' 2024-08-27T12:42:49,191 adding 'lightllm/common/basemodel/layer_infer/template/transformer_layer_infer_template.py' 2024-08-27T12:42:49,192 adding 'lightllm/common/basemodel/layer_infer/template/transformer_layer_infer_template_awquant.py' 2024-08-27T12:42:49,194 adding 'lightllm/common/basemodel/layer_infer/template/transformer_layer_infer_template_wquant.py' 2024-08-27T12:42:49,195 adding 'lightllm/common/basemodel/layer_weights/__init__.py' 2024-08-27T12:42:49,197 adding 'lightllm/common/basemodel/layer_weights/base_layer_weight.py' 2024-08-27T12:42:49,198 adding 'lightllm/common/basemodel/layer_weights/hf_load_utils.py' 2024-08-27T12:42:49,199 adding 'lightllm/common/basemodel/layer_weights/pre_and_post_layer_weight.py' 2024-08-27T12:42:49,200 adding 'lightllm/common/basemodel/layer_weights/transformer_layer_weight.py' 2024-08-27T12:42:49,202 adding 'lightllm/common/basemodel/triton_kernel/__init__.py' 2024-08-27T12:42:49,203 adding 'lightllm/common/basemodel/triton_kernel/apply_penalty.py' 2024-08-27T12:42:49,204 adding 'lightllm/common/basemodel/triton_kernel/copy_kv_index_to_req.py' 2024-08-27T12:42:49,208 adding 'lightllm/common/basemodel/triton_kernel/dequantize_gemm_int4.py' 2024-08-27T12:42:49,210 adding 'lightllm/common/basemodel/triton_kernel/dequantize_gemm_int8.py' 2024-08-27T12:42:49,211 adding 'lightllm/common/basemodel/triton_kernel/destindex_copy_kv.py' 2024-08-27T12:42:49,213 adding 'lightllm/common/basemodel/triton_kernel/multimodal_emb.py' 2024-08-27T12:42:49,215 adding 'lightllm/common/basemodel/triton_kernel/quantize_gemm_int8.py' 2024-08-27T12:42:49,217 adding 'lightllm/common/basemodel/triton_kernel/splitfuse_copy_kv_index_to_req.py' 2024-08-27T12:42:49,219 adding 'lightllm/models/__init__.py' 2024-08-27T12:42:49,220 adding 'lightllm/models/baichuan13b/__init__.py' 2024-08-27T12:42:49,222 adding 'lightllm/models/baichuan13b/model.py' 2024-08-27T12:42:49,223 adding 'lightllm/models/baichuan13b/layer_infer/__init__.py' 2024-08-27T12:42:49,224 adding 'lightllm/models/baichuan13b/layer_infer/transformer_layer_infer.py' 2024-08-27T12:42:49,226 adding 'lightllm/models/baichuan13b/layer_weights/__init__.py' 2024-08-27T12:42:49,227 adding 'lightllm/models/baichuan13b/layer_weights/transformer_layer_weight.py' 2024-08-27T12:42:49,229 adding 'lightllm/models/baichuan2_13b/__init__.py' 2024-08-27T12:42:49,230 adding 'lightllm/models/baichuan2_13b/model.py' 2024-08-27T12:42:49,232 adding 'lightllm/models/baichuan2_7b/__init__.py' 2024-08-27T12:42:49,233 adding 'lightllm/models/baichuan2_7b/model.py' 2024-08-27T12:42:49,234 adding 'lightllm/models/baichuan2_7b/layer_infer/__init__.py' 2024-08-27T12:42:49,235 adding 'lightllm/models/baichuan2_7b/layer_infer/transformer_layer_infer.py' 2024-08-27T12:42:49,237 adding 'lightllm/models/baichuan2_7b/layer_weights/__init__.py' 2024-08-27T12:42:49,238 adding 'lightllm/models/baichuan2_7b/layer_weights/pre_and_post_layer_weight.py' 2024-08-27T12:42:49,240 adding 'lightllm/models/baichuan7b/__init__.py' 2024-08-27T12:42:49,241 adding 'lightllm/models/baichuan7b/model.py' 2024-08-27T12:42:49,242 adding 'lightllm/models/baichuan7b/layer_weights/__init__.py' 2024-08-27T12:42:49,243 adding 'lightllm/models/baichuan7b/layer_weights/transformer_layer_weight.py' 2024-08-27T12:42:49,245 adding 'lightllm/models/bloom/__init__.py' 2024-08-27T12:42:49,246 adding 'lightllm/models/bloom/model.py' 2024-08-27T12:42:49,248 adding 'lightllm/models/bloom/layer_infer/__init__.py' 2024-08-27T12:42:49,249 adding 'lightllm/models/bloom/layer_infer/post_layer_infer.py' 2024-08-27T12:42:49,250 adding 'lightllm/models/bloom/layer_infer/pre_layer_infer.py' 2024-08-27T12:42:49,252 adding 'lightllm/models/bloom/layer_infer/transformer_layer_infer.py' 2024-08-27T12:42:49,253 adding 'lightllm/models/bloom/layer_weights/__init__.py' 2024-08-27T12:42:49,254 adding 'lightllm/models/bloom/layer_weights/hf_load_utils.py' 2024-08-27T12:42:49,256 adding 'lightllm/models/bloom/layer_weights/pre_and_post_layer_weight.py' 2024-08-27T12:42:49,257 adding 'lightllm/models/bloom/layer_weights/transformer_layer_weight.py' 2024-08-27T12:42:49,259 adding 'lightllm/models/bloom/triton_kernel/__init__.py' 2024-08-27T12:42:49,261 adding 'lightllm/models/bloom/triton_kernel/context_flashattention_nopad.py' 2024-08-27T12:42:49,262 adding 'lightllm/models/bloom/triton_kernel/layernorm.py' 2024-08-27T12:42:49,263 adding 'lightllm/models/bloom/triton_kernel/token_attention_nopad_att1.py' 2024-08-27T12:42:49,265 adding 'lightllm/models/bloom/triton_kernel/token_attention_nopad_reduceV.py' 2024-08-27T12:42:49,266 adding 'lightllm/models/bloom/triton_kernel/token_attention_nopad_softmax.py' 2024-08-27T12:42:49,268 adding 'lightllm/models/bloom/triton_kernel/token_flashattention_nopad.py' 2024-08-27T12:42:49,269 adding 'lightllm/models/chatglm2/__init__.py' 2024-08-27T12:42:49,271 adding 'lightllm/models/chatglm2/model.py' 2024-08-27T12:42:49,272 adding 'lightllm/models/chatglm2/layer_infer/__init__.py' 2024-08-27T12:42:49,274 adding 'lightllm/models/chatglm2/layer_infer/transformer_layer_infer.py' 2024-08-27T12:42:49,275 adding 'lightllm/models/chatglm2/layer_weights/__init__.py' 2024-08-27T12:42:49,277 adding 'lightllm/models/chatglm2/layer_weights/pre_and_post_layer_weight.py' 2024-08-27T12:42:49,278 adding 'lightllm/models/chatglm2/layer_weights/transformer_layer_weight.py' 2024-08-27T12:42:49,280 adding 'lightllm/models/chatglm2/triton_kernel/__init__.py' 2024-08-27T12:42:49,281 adding 'lightllm/models/chatglm2/triton_kernel/rotary_emb.py' 2024-08-27T12:42:49,283 adding 'lightllm/models/gemma_2b/__init__.py' 2024-08-27T12:42:49,284 adding 'lightllm/models/gemma_2b/model.py' 2024-08-27T12:42:49,286 adding 'lightllm/models/gemma_2b/layer_infer/__init__.py' 2024-08-27T12:42:49,287 adding 'lightllm/models/gemma_2b/layer_infer/pre_layer_infer.py' 2024-08-27T12:42:49,289 adding 'lightllm/models/gemma_2b/layer_infer/transformer_layer_infer.py' 2024-08-27T12:42:49,290 adding 'lightllm/models/gemma_2b/layer_weights/__init__.py' 2024-08-27T12:42:49,291 adding 'lightllm/models/gemma_2b/layer_weights/pre_and_post_layer_weight.py' 2024-08-27T12:42:49,293 adding 'lightllm/models/gemma_2b/layer_weights/transformer_layer_weight.py' 2024-08-27T12:42:49,294 adding 'lightllm/models/gemma_2b/triton_kernel/__init__.py' 2024-08-27T12:42:49,296 adding 'lightllm/models/gemma_2b/triton_kernel/gelu_and_mul.py' 2024-08-27T12:42:49,297 adding 'lightllm/models/internlm/__init__.py' 2024-08-27T12:42:49,298 adding 'lightllm/models/internlm/model.py' 2024-08-27T12:42:49,300 adding 'lightllm/models/internlm/layer_infer/__init__.py' 2024-08-27T12:42:49,301 adding 'lightllm/models/internlm/layer_infer/transformer_layer_infer.py' 2024-08-27T12:42:49,303 adding 'lightllm/models/internlm/layer_weights/__init__.py' 2024-08-27T12:42:49,304 adding 'lightllm/models/internlm/layer_weights/transformer_layer_weight.py' 2024-08-27T12:42:49,306 adding 'lightllm/models/internlm2/__init__.py' 2024-08-27T12:42:49,307 adding 'lightllm/models/internlm2/model.py' 2024-08-27T12:42:49,308 adding 'lightllm/models/internlm2/layer_weights/__init__.py' 2024-08-27T12:42:49,310 adding 'lightllm/models/internlm2/layer_weights/pre_and_post_layer_weight.py' 2024-08-27T12:42:49,311 adding 'lightllm/models/internlm2/layer_weights/transformer_layer_weight.py' 2024-08-27T12:42:49,313 adding 'lightllm/models/internlm2_wquant/__init__.py' 2024-08-27T12:42:49,314 adding 'lightllm/models/internlm2_wquant/model.py' 2024-08-27T12:42:49,315 adding 'lightllm/models/internlm2_wquant/layer_weights/__init__.py' 2024-08-27T12:42:49,317 adding 'lightllm/models/internlm2_wquant/layer_weights/transformer_layer_weight.py' 2024-08-27T12:42:49,318 adding 'lightllm/models/internlm_wquant/__init__.py' 2024-08-27T12:42:49,320 adding 'lightllm/models/internlm_wquant/model.py' 2024-08-27T12:42:49,321 adding 'lightllm/models/internlm_wquant/layer_infer/__init__.py' 2024-08-27T12:42:49,322 adding 'lightllm/models/internlm_wquant/layer_infer/transformer_layer_infer.py' 2024-08-27T12:42:49,324 adding 'lightllm/models/internlm_wquant/layer_weights/__init__.py' 2024-08-27T12:42:49,325 adding 'lightllm/models/internlm_wquant/layer_weights/transformer_layer_weight.py' 2024-08-27T12:42:49,327 adding 'lightllm/models/internlm_xcomposer/__init__.py' 2024-08-27T12:42:49,328 adding 'lightllm/models/internlm_xcomposer/infer_struct.py' 2024-08-27T12:42:49,330 adding 'lightllm/models/internlm_xcomposer/internlm_visual.py' 2024-08-27T12:42:49,331 adding 'lightllm/models/internlm_xcomposer/model.py' 2024-08-27T12:42:49,333 adding 'lightllm/models/internlm_xcomposer/layer_infer/__init__.py' 2024-08-27T12:42:49,334 adding 'lightllm/models/internlm_xcomposer/layer_infer/transformer_layer_infer.py' 2024-08-27T12:42:49,336 adding 'lightllm/models/internlm_xcomposer/layer_weights/__init__.py' 2024-08-27T12:42:49,337 adding 'lightllm/models/internlm_xcomposer/layer_weights/transformer_layer_weight.py' 2024-08-27T12:42:49,339 adding 'lightllm/models/llama/__init__.py' 2024-08-27T12:42:49,341 adding 'lightllm/models/llama/infer_struct.py' 2024-08-27T12:42:49,342 adding 'lightllm/models/llama/model.py' 2024-08-27T12:42:49,344 adding 'lightllm/models/llama/splitfuse_infer_struct.py' 2024-08-27T12:42:49,345 adding 'lightllm/models/llama/yarn_rotary_utils.py' 2024-08-27T12:42:49,346 adding 'lightllm/models/llama/layer_infer/__init__.py' 2024-08-27T12:42:49,348 adding 'lightllm/models/llama/layer_infer/post_layer_infer.py' 2024-08-27T12:42:49,349 adding 'lightllm/models/llama/layer_infer/pre_layer_infer.py' 2024-08-27T12:42:49,351 adding 'lightllm/models/llama/layer_infer/transformer_layer_infer.py' 2024-08-27T12:42:49,353 adding 'lightllm/models/llama/layer_weights/__init__.py' 2024-08-27T12:42:49,354 adding 'lightllm/models/llama/layer_weights/ds_load_utils.py' 2024-08-27T12:42:49,356 adding 'lightllm/models/llama/layer_weights/pre_and_post_layer_weight.py' 2024-08-27T12:42:49,357 adding 'lightllm/models/llama/layer_weights/transformer_layer_weight.py' 2024-08-27T12:42:49,359 adding 'lightllm/models/llama/triton_kernel/__init__.py' 2024-08-27T12:42:49,361 adding 'lightllm/models/llama/triton_kernel/context_flashattention_nopad.py' 2024-08-27T12:42:49,362 adding 'lightllm/models/llama/triton_kernel/flash_decoding.py' 2024-08-27T12:42:49,364 adding 'lightllm/models/llama/triton_kernel/flash_decoding_stage1.py' 2024-08-27T12:42:49,365 adding 'lightllm/models/llama/triton_kernel/flash_decoding_stage2.py' 2024-08-27T12:42:49,366 adding 'lightllm/models/llama/triton_kernel/gqa_decode_flashattention_nopad.py' 2024-08-27T12:42:49,368 adding 'lightllm/models/llama/triton_kernel/gqa_flash_decoding.py' 2024-08-27T12:42:49,369 adding 'lightllm/models/llama/triton_kernel/gqa_flash_decoding_stage1.py' 2024-08-27T12:42:49,370 adding 'lightllm/models/llama/triton_kernel/gqa_flash_decoding_stage2.py' 2024-08-27T12:42:49,371 adding 'lightllm/models/llama/triton_kernel/ppl_fp16_flash_decoding.py' 2024-08-27T12:42:49,373 adding 'lightllm/models/llama/triton_kernel/ppl_int4kv_copy_kv.py' 2024-08-27T12:42:49,374 adding 'lightllm/models/llama/triton_kernel/ppl_int4kv_flash_decoding.py' 2024-08-27T12:42:49,375 adding 'lightllm/models/llama/triton_kernel/ppl_int8kv_flash_decoding.py' 2024-08-27T12:42:49,376 adding 'lightllm/models/llama/triton_kernel/ppl_quant_copy_kv.py' 2024-08-27T12:42:49,378 adding 'lightllm/models/llama/triton_kernel/rmsnorm.py' 2024-08-27T12:42:49,379 adding 'lightllm/models/llama/triton_kernel/rotary_emb.py' 2024-08-27T12:42:49,381 adding 'lightllm/models/llama/triton_kernel/silu_and_mul.py' 2024-08-27T12:42:49,382 adding 'lightllm/models/llama/triton_kernel/splitfuse_context_flashattention_nopad.py' 2024-08-27T12:42:49,384 adding 'lightllm/models/llama/triton_kernel/token_attention_nopad_att1.py' 2024-08-27T12:42:49,385 adding 'lightllm/models/llama/triton_kernel/token_attention_nopad_reduceV.py' 2024-08-27T12:42:49,386 adding 'lightllm/models/llama/triton_kernel/token_attention_nopad_softmax.py' 2024-08-27T12:42:49,388 adding 'lightllm/models/llama/triton_kernel/token_attention_softmax_and_reducev.py' 2024-08-27T12:42:49,389 adding 'lightllm/models/llama_awquant/__init__.py' 2024-08-27T12:42:49,391 adding 'lightllm/models/llama_awquant/model.py' 2024-08-27T12:42:49,392 adding 'lightllm/models/llama_awquant/layer_infer/__init__.py' 2024-08-27T12:42:49,394 adding 'lightllm/models/llama_awquant/layer_infer/transformer_layer_infer.py' 2024-08-27T12:42:49,395 adding 'lightllm/models/llama_awquant/layer_weights/__init__.py' 2024-08-27T12:42:49,397 adding 'lightllm/models/llama_awquant/layer_weights/transformer_layer_weight.py' 2024-08-27T12:42:49,398 adding 'lightllm/models/llama_quik/__init__.py' 2024-08-27T12:42:49,400 adding 'lightllm/models/llama_quik/model.py' 2024-08-27T12:42:49,401 adding 'lightllm/models/llama_quik/cuda_kernel/__init__.py' 2024-08-27T12:42:49,403 adding 'lightllm/models/llama_quik/cuda_kernel/quik_awquant.py' 2024-08-27T12:42:49,615 adding 'lightllm/models/llama_quik/layer_infer/__init__.py' 2024-08-27T12:42:49,618 adding 'lightllm/models/llama_quik/layer_infer/transformer_layer_infer.py' 2024-08-27T12:42:49,622 adding 'lightllm/models/llama_quik/layer_weights/__init__.py' 2024-08-27T12:42:49,625 adding 'lightllm/models/llama_quik/layer_weights/qlinear.py' 2024-08-27T12:42:49,628 adding 'lightllm/models/llama_quik/layer_weights/transformer_layer_weight.py' 2024-08-27T12:42:49,849 adding 'lightllm/models/llama_wquant/__init__.py' 2024-08-27T12:42:49,850 adding 'lightllm/models/llama_wquant/model.py' 2024-08-27T12:42:49,852 adding 'lightllm/models/llama_wquant/layer_infer/__init__.py' 2024-08-27T12:42:49,853 adding 'lightllm/models/llama_wquant/layer_infer/transformer_layer_infer.py' 2024-08-27T12:42:49,855 adding 'lightllm/models/llama_wquant/layer_weights/__init__.py' 2024-08-27T12:42:49,856 adding 'lightllm/models/llama_wquant/layer_weights/transformer_layer_weight.py' 2024-08-27T12:42:49,858 adding 'lightllm/models/llava/__init__.py' 2024-08-27T12:42:49,859 adding 'lightllm/models/llava/llava_visual.py' 2024-08-27T12:42:49,860 adding 'lightllm/models/llava/model.py' 2024-08-27T12:42:49,862 adding 'lightllm/models/minicpm/__init__.py' 2024-08-27T12:42:49,863 adding 'lightllm/models/minicpm/model.py' 2024-08-27T12:42:49,865 adding 'lightllm/models/minicpm/layer_infer/__init__.py' 2024-08-27T12:42:49,866 adding 'lightllm/models/minicpm/layer_infer/transformer_layer_infer.py' 2024-08-27T12:42:49,867 adding 'lightllm/models/minicpm/layer_weights/__init__.py' 2024-08-27T12:42:49,869 adding 'lightllm/models/minicpm/layer_weights/pre_and_post_layer_weight.py' 2024-08-27T12:42:49,870 adding 'lightllm/models/minicpm/layer_weights/transformer_layer_weight.py' 2024-08-27T12:42:49,871 adding 'lightllm/models/mistral/__init__.py' 2024-08-27T12:42:49,873 adding 'lightllm/models/mistral/infer_struct.py' 2024-08-27T12:42:49,874 adding 'lightllm/models/mistral/model.py' 2024-08-27T12:42:49,876 adding 'lightllm/models/mistral/layer_infer/__init__.py' 2024-08-27T12:42:49,877 adding 'lightllm/models/mistral/layer_infer/transformer_layer_infer.py' 2024-08-27T12:42:49,878 adding 'lightllm/models/mistral/triton_kernel/__init__.py' 2024-08-27T12:42:49,880 adding 'lightllm/models/mistral/triton_kernel/context_flashattention_nopad.py' 2024-08-27T12:42:49,881 adding 'lightllm/models/mistral/triton_kernel/token_attention_nopad_att1.py' 2024-08-27T12:42:49,883 adding 'lightllm/models/mistral/triton_kernel/token_attention_nopad_reduceV.py' 2024-08-27T12:42:49,884 adding 'lightllm/models/mistral/triton_kernel/token_attention_softmax_and_reducev.py' 2024-08-27T12:42:49,886 adding 'lightllm/models/mixtral/__init__.py' 2024-08-27T12:42:49,887 adding 'lightllm/models/mixtral/infer_struct.py' 2024-08-27T12:42:49,888 adding 'lightllm/models/mixtral/model.py' 2024-08-27T12:42:49,890 adding 'lightllm/models/mixtral/layer_infer/__init__.py' 2024-08-27T12:42:49,891 adding 'lightllm/models/mixtral/layer_infer/transformer_layer_infer.py' 2024-08-27T12:42:49,893 adding 'lightllm/models/mixtral/layer_weights/__init__.py' 2024-08-27T12:42:49,894 adding 'lightllm/models/mixtral/layer_weights/transformer_layer_weight.py' 2024-08-27T12:42:49,896 adding 'lightllm/models/qwen/__init__.py' 2024-08-27T12:42:49,897 adding 'lightllm/models/qwen/infer_struct.py' 2024-08-27T12:42:49,898 adding 'lightllm/models/qwen/model.py' 2024-08-27T12:42:49,900 adding 'lightllm/models/qwen/layer_infer/__init__.py' 2024-08-27T12:42:49,901 adding 'lightllm/models/qwen/layer_infer/transformer_layer_infer.py' 2024-08-27T12:42:49,903 adding 'lightllm/models/qwen/layer_weights/__init__.py' 2024-08-27T12:42:49,904 adding 'lightllm/models/qwen/layer_weights/pre_and_post_layer_weight.py' 2024-08-27T12:42:49,905 adding 'lightllm/models/qwen/layer_weights/transformer_layer_weight.py' 2024-08-27T12:42:49,907 adding 'lightllm/models/qwen2/__init__.py' 2024-08-27T12:42:49,908 adding 'lightllm/models/qwen2/infer_struct.py' 2024-08-27T12:42:49,910 adding 'lightllm/models/qwen2/model.py' 2024-08-27T12:42:49,911 adding 'lightllm/models/qwen2/layer_infer/__init__.py' 2024-08-27T12:42:49,913 adding 'lightllm/models/qwen2/layer_infer/transformer_layer_infer.py' 2024-08-27T12:42:49,915 adding 'lightllm/models/qwen2/layer_weights/__init__.py' 2024-08-27T12:42:49,916 adding 'lightllm/models/qwen2/layer_weights/pre_and_post_layer_weight.py' 2024-08-27T12:42:49,917 adding 'lightllm/models/qwen2/layer_weights/transformer_layer_weight.py' 2024-08-27T12:42:49,919 adding 'lightllm/models/qwen_vl/__init__.py' 2024-08-27T12:42:49,920 adding 'lightllm/models/qwen_vl/model.py' 2024-08-27T12:42:49,922 adding 'lightllm/models/qwen_vl/qwen_visual.py' 2024-08-27T12:42:49,924 adding 'lightllm/models/qwen_vl/layer_infer/__init__.py' 2024-08-27T12:42:49,925 adding 'lightllm/models/qwen_vl/layer_infer/pre_layer_infer.py' 2024-08-27T12:42:49,927 adding 'lightllm/models/qwen_wquant/__init__.py' 2024-08-27T12:42:49,929 adding 'lightllm/models/qwen_wquant/model.py' 2024-08-27T12:42:49,930 adding 'lightllm/models/qwen_wquant/layer_infer/__init__.py' 2024-08-27T12:42:49,931 adding 'lightllm/models/qwen_wquant/layer_infer/transformer_layer_infer.py' 2024-08-27T12:42:49,933 adding 'lightllm/models/qwen_wquant/layer_weights/__init__.py' 2024-08-27T12:42:49,934 adding 'lightllm/models/qwen_wquant/layer_weights/transformer_layer_weight.py' 2024-08-27T12:42:49,936 adding 'lightllm/models/stablelm/__init__.py' 2024-08-27T12:42:49,938 adding 'lightllm/models/stablelm/model.py' 2024-08-27T12:42:49,940 adding 'lightllm/models/stablelm/layer_infer/__init__.py' 2024-08-27T12:42:49,941 adding 'lightllm/models/stablelm/layer_infer/transformer_layer_infer.py' 2024-08-27T12:42:49,942 adding 'lightllm/models/stablelm/layer_weights/__init__.py' 2024-08-27T12:42:49,943 adding 'lightllm/models/stablelm/layer_weights/pre_and_post_layer_weight.py' 2024-08-27T12:42:49,945 adding 'lightllm/models/stablelm/layer_weights/transformer_layer_weight.py' 2024-08-27T12:42:49,946 adding 'lightllm/models/starcoder/__init__.py' 2024-08-27T12:42:49,947 adding 'lightllm/models/starcoder/infer_struct.py' 2024-08-27T12:42:49,949 adding 'lightllm/models/starcoder/model.py' 2024-08-27T12:42:49,950 adding 'lightllm/models/starcoder/layer_infer/__init__.py' 2024-08-27T12:42:49,951 adding 'lightllm/models/starcoder/layer_infer/pre_layer_infer.py' 2024-08-27T12:42:49,952 adding 'lightllm/models/starcoder/layer_infer/transformer_layer_infer.py' 2024-08-27T12:42:49,954 adding 'lightllm/models/starcoder/layer_weights/__init__.py' 2024-08-27T12:42:49,955 adding 'lightllm/models/starcoder/layer_weights/pre_and_post_layer_weight.py' 2024-08-27T12:42:49,956 adding 'lightllm/models/starcoder/layer_weights/transformer_layer_weight.py' 2024-08-27T12:42:49,958 adding 'lightllm/models/starcoder2/__init__.py' 2024-08-27T12:42:49,959 adding 'lightllm/models/starcoder2/model.py' 2024-08-27T12:42:49,961 adding 'lightllm/models/starcoder2/layer_infer/__init__.py' 2024-08-27T12:42:49,962 adding 'lightllm/models/starcoder2/layer_infer/transformer_layer_infer.py' 2024-08-27T12:42:49,964 adding 'lightllm/models/starcoder2/layer_weights/__init__.py' 2024-08-27T12:42:49,965 adding 'lightllm/models/starcoder2/layer_weights/pre_and_post_layer_weight.py' 2024-08-27T12:42:49,966 adding 'lightllm/models/starcoder2/layer_weights/transformer_layer_weight.py' 2024-08-27T12:42:49,968 adding 'lightllm/models/starcoder_wquant/__init__.py' 2024-08-27T12:42:49,969 adding 'lightllm/models/starcoder_wquant/model.py' 2024-08-27T12:42:49,971 adding 'lightllm/models/starcoder_wquant/layer_infer/__init__.py' 2024-08-27T12:42:49,972 adding 'lightllm/models/starcoder_wquant/layer_infer/transformer_layer_infer.py' 2024-08-27T12:42:49,974 adding 'lightllm/models/starcoder_wquant/layer_weights/__init__.py' 2024-08-27T12:42:49,975 adding 'lightllm/models/starcoder_wquant/layer_weights/transformer_layer_weight.py' 2024-08-27T12:42:49,977 adding 'lightllm/models/yi/__init__.py' 2024-08-27T12:42:49,978 adding 'lightllm/models/yi/model.py' 2024-08-27T12:42:49,980 adding 'lightllm/models/yi/layer_weights/__init__.py' 2024-08-27T12:42:49,981 adding 'lightllm/models/yi/layer_weights/transformer_layer_weight.py' 2024-08-27T12:42:49,983 adding 'lightllm/server/__init__.py' 2024-08-27T12:42:49,984 adding 'lightllm/server/api_lightllm.py' 2024-08-27T12:42:49,986 adding 'lightllm/server/api_models.py' 2024-08-27T12:42:49,988 adding 'lightllm/server/api_server.py' 2024-08-27T12:42:49,990 adding 'lightllm/server/api_tgi.py' 2024-08-27T12:42:49,991 adding 'lightllm/server/build_prompt.py' 2024-08-27T12:42:49,993 adding 'lightllm/server/io_struct.py' 2024-08-27T12:42:49,995 adding 'lightllm/server/metrics.py' 2024-08-27T12:42:49,996 adding 'lightllm/server/multimodal_params.py' 2024-08-27T12:42:49,997 adding 'lightllm/server/req_id_generator.py' 2024-08-27T12:42:49,999 adding 'lightllm/server/sampling_params.py' 2024-08-27T12:42:50,000 adding 'lightllm/server/tokenizer.py' 2024-08-27T12:42:50,002 adding 'lightllm/server/detokenization/__init__.py' 2024-08-27T12:42:50,003 adding 'lightllm/server/detokenization/decode.py' 2024-08-27T12:42:50,004 adding 'lightllm/server/detokenization/manager.py' 2024-08-27T12:42:50,006 adding 'lightllm/server/embed_cache/__init__.py' 2024-08-27T12:42:50,008 adding 'lightllm/server/embed_cache/interface.py' 2024-08-27T12:42:50,009 adding 'lightllm/server/embed_cache/manager.py' 2024-08-27T12:42:50,010 adding 'lightllm/server/embed_cache/utils.py' 2024-08-27T12:42:50,012 adding 'lightllm/server/embed_cache/impl/__init__.py' 2024-08-27T12:42:50,013 adding 'lightllm/server/embed_cache/impl/naive_memory_cache.py' 2024-08-27T12:42:50,015 adding 'lightllm/server/health_monitor/__init__.py' 2024-08-27T12:42:50,016 adding 'lightllm/server/health_monitor/manager.py' 2024-08-27T12:42:50,017 adding 'lightllm/server/httpserver/__init__.py' 2024-08-27T12:42:50,019 adding 'lightllm/server/httpserver/manager.py' 2024-08-27T12:42:50,021 adding 'lightllm/server/router/__init__.py' 2024-08-27T12:42:50,023 adding 'lightllm/server/router/manager.py' 2024-08-27T12:42:50,024 adding 'lightllm/server/router/pause_strategy.py' 2024-08-27T12:42:50,026 adding 'lightllm/server/router/stats.py' 2024-08-27T12:42:50,027 adding 'lightllm/server/router/token_load.py' 2024-08-27T12:42:50,029 adding 'lightllm/server/router/dynamic_prompt/__init__.py' 2024-08-27T12:42:50,031 adding 'lightllm/server/router/dynamic_prompt/radix_cache.py' 2024-08-27T12:42:50,032 adding 'lightllm/server/router/dynamic_prompt/shared_arr.py' 2024-08-27T12:42:50,034 adding 'lightllm/server/router/model_infer/__init__.py' 2024-08-27T12:42:50,036 adding 'lightllm/server/router/model_infer/infer_batch.py' 2024-08-27T12:42:50,038 adding 'lightllm/server/router/model_infer/model_rpc.py' 2024-08-27T12:42:50,040 adding 'lightllm/server/router/model_infer/mode_backend/__init__.py' 2024-08-27T12:42:50,041 adding 'lightllm/server/router/model_infer/mode_backend/base_backend.py' 2024-08-27T12:42:50,043 adding 'lightllm/server/router/model_infer/mode_backend/beamsearch/__init__.py' 2024-08-27T12:42:50,045 adding 'lightllm/server/router/model_infer/mode_backend/beamsearch/impl.py' 2024-08-27T12:42:50,046 adding 'lightllm/server/router/model_infer/mode_backend/beamsearch/post_process.py' 2024-08-27T12:42:50,048 adding 'lightllm/server/router/model_infer/mode_backend/beamsearch/pre_process.py' 2024-08-27T12:42:50,049 adding 'lightllm/server/router/model_infer/mode_backend/continues_batch/__init__.py' 2024-08-27T12:42:50,051 adding 'lightllm/server/router/model_infer/mode_backend/continues_batch/impl.py' 2024-08-27T12:42:50,053 adding 'lightllm/server/router/model_infer/mode_backend/continues_batch/impl_for_return_all_prompt_logprobs.py' 2024-08-27T12:42:50,054 adding 'lightllm/server/router/model_infer/mode_backend/continues_batch/post_process.py' 2024-08-27T12:42:50,055 adding 'lightllm/server/router/model_infer/mode_backend/continues_batch/pre_process.py' 2024-08-27T12:42:50,057 adding 'lightllm/server/router/model_infer/mode_backend/diverse_backend/__init__.py' 2024-08-27T12:42:50,058 adding 'lightllm/server/router/model_infer/mode_backend/diverse_backend/impl.py' 2024-08-27T12:42:50,060 adding 'lightllm/server/router/model_infer/mode_backend/diverse_backend/post_process.py' 2024-08-27T12:42:50,062 adding 'lightllm/server/router/model_infer/mode_backend/splitfuse/__init__.py' 2024-08-27T12:42:50,063 adding 'lightllm/server/router/model_infer/mode_backend/splitfuse/impl.py' 2024-08-27T12:42:50,064 adding 'lightllm/server/router/model_infer/mode_backend/splitfuse/pre_process.py' 2024-08-27T12:42:50,066 adding 'lightllm/server/router/req_queue/__init__.py' 2024-08-27T12:42:50,067 adding 'lightllm/server/router/req_queue/base_queue.py' 2024-08-27T12:42:50,069 adding 'lightllm/server/router/req_queue/continues_batch/__init__.py' 2024-08-27T12:42:50,071 adding 'lightllm/server/router/req_queue/continues_batch/beam_impl.py' 2024-08-27T12:42:50,072 adding 'lightllm/server/router/req_queue/continues_batch/impl.py' 2024-08-27T12:42:50,074 adding 'lightllm/server/router/req_queue/splitfuse/__init__.py' 2024-08-27T12:42:50,075 adding 'lightllm/server/router/req_queue/splitfuse/impl.py' 2024-08-27T12:42:50,077 adding 'lightllm/server/visualserver/__init__.py' 2024-08-27T12:42:50,078 adding 'lightllm/server/visualserver/manager.py' 2024-08-27T12:42:50,079 adding 'lightllm/server/visualserver/model_infer/__init__.py' 2024-08-27T12:42:50,081 adding 'lightllm/server/visualserver/model_infer/model_rpc.py' 2024-08-27T12:42:50,082 adding 'lightllm/utils/__init__.py' 2024-08-27T12:42:50,083 adding 'lightllm/utils/graceful_utils.py' 2024-08-27T12:42:50,084 adding 'lightllm/utils/health_check.py' 2024-08-27T12:42:50,086 adding 'lightllm/utils/infer_utils.py' 2024-08-27T12:42:50,087 adding 'lightllm/utils/log_utils.py' 2024-08-27T12:42:50,088 adding 'lightllm/utils/net_utils.py' 2024-08-27T12:42:50,090 adding 'lightllm/utils/petrel_helper.py' 2024-08-27T12:42:50,091 adding 'lightllm/utils/start_utils.py' 2024-08-27T12:42:50,094 adding 'lightllm-0.0.1.dist-info/LICENSE' 2024-08-27T12:42:50,095 adding 'lightllm-0.0.1.dist-info/METADATA' 2024-08-27T12:42:50,096 adding 'lightllm-0.0.1.dist-info/WHEEL' 2024-08-27T12:42:50,097 adding 'lightllm-0.0.1.dist-info/top_level.txt' 2024-08-27T12:42:50,102 adding 'lightllm-0.0.1.dist-info/RECORD' 2024-08-27T12:42:50,111 removing build/bdist.linux-armv7l/wheel 2024-08-27T12:42:50,356 Building wheel for lightllm (setup.py): finished with status 'done' 2024-08-27T12:42:50,362 Created wheel for lightllm: filename=lightllm-0.0.1-py3-none-any.whl size=320259 sha256=c75f583fab574cffee921eea63042a8c61334730d37822a1dc403675fa71656d 2024-08-27T12:42:50,363 Stored in directory: /tmp/pip-ephem-wheel-cache-s48t32mn/wheels/e1/fa/ea/11502065c2d0e07130ac5e2744ff7c6ece5f29e74df2bbc5a0 2024-08-27T12:42:50,385 Successfully built lightllm 2024-08-27T12:42:50,400 Removed build tracker: '/tmp/pip-build-tracker-6b5q52yn'