2023-11-25T14:11:17,622 Created temporary directory: /tmp/pip-build-tracker-dxf8ydgc 2023-11-25T14:11:17,779 Initialized build tracking at /tmp/pip-build-tracker-dxf8ydgc 2023-11-25T14:11:17,780 Created build tracker: /tmp/pip-build-tracker-dxf8ydgc 2023-11-25T14:11:17,781 Entered build tracker: /tmp/pip-build-tracker-dxf8ydgc 2023-11-25T14:11:17,783 Created temporary directory: /tmp/pip-wheel-b1277of1 2023-11-25T14:11:17,790 Created temporary directory: /tmp/pip-ephem-wheel-cache-awvap8qb 2023-11-25T14:11:17,865 Looking in indexes: https://pypi.org/simple, https://www.piwheels.org/simple 2023-11-25T14:11:17,873 2 location(s) to search for versions of multi-loras: 2023-11-25T14:11:17,873 * https://pypi.org/simple/multi-loras/ 2023-11-25T14:11:17,873 * https://www.piwheels.org/simple/multi-loras/ 2023-11-25T14:11:17,875 Fetching project page and analyzing links: https://pypi.org/simple/multi-loras/ 2023-11-25T14:11:17,879 Getting page https://pypi.org/simple/multi-loras/ 2023-11-25T14:11:17,882 Found index url https://pypi.org/simple/ 2023-11-25T14:11:18,122 Fetched page https://pypi.org/simple/multi-loras/ as application/vnd.pypi.simple.v1+json 2023-11-25T14:11:18,124 Skipping link: No binaries permitted for multi-loras: https://files.pythonhosted.org/packages/f9/7c/ad7f43fe688303b03f5fc13f48f383e7dcf1f5a154e736bd58894b2b875b/multi_loras-0.1.0-py3-none-any.whl (from https://pypi.org/simple/multi-loras/) (requires-python:>=3.8.0) 2023-11-25T14:11:18,126 Found link https://files.pythonhosted.org/packages/e0/b9/6914be7a810f4cdc6a74bfc25c3cd0b6979452ae52aaa68ccc80fcd3269f/multi_loras-0.1.0.tar.gz (from https://pypi.org/simple/multi-loras/) (requires-python:>=3.8.0), version: 0.1.0 2023-11-25T14:11:18,126 Skipping link: No binaries permitted for multi-loras: https://files.pythonhosted.org/packages/43/b0/65d9ac06cffadc9f28da1bb039e77b1538842dd75dff6814fa5d7fb695d6/multi_loras-0.2.0-py3-none-any.whl (from https://pypi.org/simple/multi-loras/) (requires-python:>=3.8.0) 2023-11-25T14:11:18,131 Found link https://files.pythonhosted.org/packages/92/a2/0ebc4872978836cb41b7678198724a04230f51543fd361c26112e0692490/multi_loras-0.2.0.tar.gz (from https://pypi.org/simple/multi-loras/) (requires-python:>=3.8.0), version: 0.2.0 2023-11-25T14:11:18,132 Fetching project page and analyzing links: https://www.piwheels.org/simple/multi-loras/ 2023-11-25T14:11:18,132 Getting page https://www.piwheels.org/simple/multi-loras/ 2023-11-25T14:11:18,134 Found index url https://www.piwheels.org/simple/ 2023-11-25T14:11:18,306 Fetched page https://www.piwheels.org/simple/multi-loras/ as text/html 2023-11-25T14:11:18,332 Skipping link: No binaries permitted for multi-loras: https://www.piwheels.org/simple/multi-loras/multi_loras-0.1.0-py3-none-any.whl#sha256=61640ca439ca6d9847b87c1138b3c1a7a4c06bb9efc9c5bdf513febc784083a8 (from https://www.piwheels.org/simple/multi-loras/) (requires-python:>=3.8.0) 2023-11-25T14:11:18,339 Skipping link: not a file: https://www.piwheels.org/simple/multi-loras/ 2023-11-25T14:11:18,340 Skipping link: not a file: https://pypi.org/simple/multi-loras/ 2023-11-25T14:11:18,365 Given no hashes to check 1 links for project 'multi-loras': discarding no candidates 2023-11-25T14:11:18,400 Collecting multi-loras==0.2.0 2023-11-25T14:11:18,413 Created temporary directory: /tmp/pip-unpack-0olw7pj8 2023-11-25T14:11:18,629 Downloading multi_loras-0.2.0.tar.gz (91 kB) 2023-11-25T14:11:19,436 Added multi-loras==0.2.0 from https://files.pythonhosted.org/packages/92/a2/0ebc4872978836cb41b7678198724a04230f51543fd361c26112e0692490/multi_loras-0.2.0.tar.gz to build tracker '/tmp/pip-build-tracker-dxf8ydgc' 2023-11-25T14:11:19,439 Running setup.py (path:/tmp/pip-wheel-b1277of1/multi-loras_d8a5b44888bb4c39aa293c5165b8b289/setup.py) egg_info for package multi-loras 2023-11-25T14:11:19,440 Created temporary directory: /tmp/pip-pip-egg-info-54pdupdy 2023-11-25T14:11:19,441 Preparing metadata (setup.py): started 2023-11-25T14:11:19,443 Running command python setup.py egg_info 2023-11-25T14:11:21,999 running egg_info 2023-11-25T14:11:22,000 creating /tmp/pip-pip-egg-info-54pdupdy/multi_loras.egg-info 2023-11-25T14:11:22,026 writing /tmp/pip-pip-egg-info-54pdupdy/multi_loras.egg-info/PKG-INFO 2023-11-25T14:11:22,030 writing dependency_links to /tmp/pip-pip-egg-info-54pdupdy/multi_loras.egg-info/dependency_links.txt 2023-11-25T14:11:22,032 writing requirements to /tmp/pip-pip-egg-info-54pdupdy/multi_loras.egg-info/requires.txt 2023-11-25T14:11:22,034 writing top-level names to /tmp/pip-pip-egg-info-54pdupdy/multi_loras.egg-info/top_level.txt 2023-11-25T14:11:22,036 writing manifest file '/tmp/pip-pip-egg-info-54pdupdy/multi_loras.egg-info/SOURCES.txt' 2023-11-25T14:11:22,304 reading manifest file '/tmp/pip-pip-egg-info-54pdupdy/multi_loras.egg-info/SOURCES.txt' 2023-11-25T14:11:22,805 adding license file 'LICENSE' 2023-11-25T14:11:22,854 writing manifest file '/tmp/pip-pip-egg-info-54pdupdy/multi_loras.egg-info/SOURCES.txt' 2023-11-25T14:11:23,102 Preparing metadata (setup.py): finished with status 'done' 2023-11-25T14:11:23,206 Source in /tmp/pip-wheel-b1277of1/multi-loras_d8a5b44888bb4c39aa293c5165b8b289 has version 0.2.0, which satisfies requirement multi-loras==0.2.0 from https://files.pythonhosted.org/packages/92/a2/0ebc4872978836cb41b7678198724a04230f51543fd361c26112e0692490/multi_loras-0.2.0.tar.gz 2023-11-25T14:11:23,207 Removed multi-loras==0.2.0 from https://files.pythonhosted.org/packages/92/a2/0ebc4872978836cb41b7678198724a04230f51543fd361c26112e0692490/multi_loras-0.2.0.tar.gz from build tracker '/tmp/pip-build-tracker-dxf8ydgc' 2023-11-25T14:11:23,214 Created temporary directory: /tmp/pip-unpack-zt9y1nte 2023-11-25T14:11:23,215 Created temporary directory: /tmp/pip-unpack-ytrc18i2 2023-11-25T14:11:23,222 Building wheels for collected packages: multi-loras 2023-11-25T14:11:23,228 Created temporary directory: /tmp/pip-wheel-kls027r4 2023-11-25T14:11:23,229 Building wheel for multi-loras (setup.py): started 2023-11-25T14:11:23,230 Destination directory: /tmp/pip-wheel-kls027r4 2023-11-25T14:11:23,231 Running command python setup.py bdist_wheel 2023-11-25T14:11:24,319 running bdist_wheel 2023-11-25T14:11:24,415 running build 2023-11-25T14:11:24,416 running build_py 2023-11-25T14:11:24,446 creating build 2023-11-25T14:11:24,447 creating build/lib 2023-11-25T14:11:24,447 creating build/lib/multi_loras 2023-11-25T14:11:24,449 copying multi_loras/merge_models.py -> build/lib/multi_loras 2023-11-25T14:11:24,451 copying multi_loras/merge_peft_adapters.py -> build/lib/multi_loras 2023-11-25T14:11:24,453 copying multi_loras/__version__.py -> build/lib/multi_loras 2023-11-25T14:11:24,455 copying multi_loras/__main__.py -> build/lib/multi_loras 2023-11-25T14:11:24,457 copying multi_loras/merging_methods.py -> build/lib/multi_loras 2023-11-25T14:11:24,460 copying multi_loras/delta_weights.py -> build/lib/multi_loras 2023-11-25T14:11:24,463 copying multi_loras/extract_lora.py -> build/lib/multi_loras 2023-11-25T14:11:24,465 copying multi_loras/__init__.py -> build/lib/multi_loras 2023-11-25T14:11:24,467 copying multi_loras/dare.py -> build/lib/multi_loras 2023-11-25T14:11:24,470 creating build/lib/multi_loras/slora 2023-11-25T14:11:24,471 copying multi_loras/slora/install_slora_kernel.py -> build/lib/multi_loras/slora 2023-11-25T14:11:24,473 copying multi_loras/slora/io_struct.py -> build/lib/multi_loras/slora 2023-11-25T14:11:24,476 copying multi_loras/slora/sampling_params.py -> build/lib/multi_loras/slora 2023-11-25T14:11:24,478 copying multi_loras/slora/slora_server.py -> build/lib/multi_loras/slora 2023-11-25T14:11:24,481 copying multi_loras/slora/__init__.py -> build/lib/multi_loras/slora 2023-11-25T14:11:24,483 creating build/lib/multi_loras/slora/common 2023-11-25T14:11:24,484 copying multi_loras/slora/common/gqa_mem_manager.py -> build/lib/multi_loras/slora/common 2023-11-25T14:11:24,487 copying multi_loras/slora/common/mem_manager.py -> build/lib/multi_loras/slora/common 2023-11-25T14:11:24,489 copying multi_loras/slora/common/mem_allocator.py -> build/lib/multi_loras/slora/common 2023-11-25T14:11:24,492 copying multi_loras/slora/common/ppl_int8kv_mem_manager.py -> build/lib/multi_loras/slora/common 2023-11-25T14:11:24,494 copying multi_loras/slora/common/build_utils.py -> build/lib/multi_loras/slora/common 2023-11-25T14:11:24,503 copying multi_loras/slora/common/__init__.py -> build/lib/multi_loras/slora/common 2023-11-25T14:11:24,504 copying multi_loras/slora/common/int8kv_mem_manager.py -> build/lib/multi_loras/slora/common 2023-11-25T14:11:24,506 copying multi_loras/slora/common/infer_utils.py -> build/lib/multi_loras/slora/common 2023-11-25T14:11:24,508 creating build/lib/multi_loras/slora/utils 2023-11-25T14:11:24,509 copying multi_loras/slora/utils/model_load.py -> build/lib/multi_loras/slora/utils 2023-11-25T14:11:24,512 copying multi_loras/slora/utils/net_utils.py -> build/lib/multi_loras/slora/utils 2023-11-25T14:11:24,514 copying multi_loras/slora/utils/model_utils.py -> build/lib/multi_loras/slora/utils 2023-11-25T14:11:24,516 copying multi_loras/slora/utils/metric.py -> build/lib/multi_loras/slora/utils 2023-11-25T14:11:24,518 copying multi_loras/slora/utils/__init__.py -> build/lib/multi_loras/slora/utils 2023-11-25T14:11:24,519 copying multi_loras/slora/utils/infer_utils.py -> build/lib/multi_loras/slora/utils 2023-11-25T14:11:24,522 creating build/lib/multi_loras/slora/router 2023-11-25T14:11:24,523 copying multi_loras/slora/router/profiler.py -> build/lib/multi_loras/slora/router 2023-11-25T14:11:24,525 copying multi_loras/slora/router/abort_req_queue.py -> build/lib/multi_loras/slora/router 2023-11-25T14:11:24,527 copying multi_loras/slora/router/pets_req_queue.py -> build/lib/multi_loras/slora/router 2023-11-25T14:11:24,530 copying multi_loras/slora/router/input_params.py -> build/lib/multi_loras/slora/router 2023-11-25T14:11:24,532 copying multi_loras/slora/router/cluster_req_queue.py -> build/lib/multi_loras/slora/router 2023-11-25T14:11:24,534 copying multi_loras/slora/router/stats.py -> build/lib/multi_loras/slora/router 2023-11-25T14:11:24,536 copying multi_loras/slora/router/req_queue.py -> build/lib/multi_loras/slora/router 2023-11-25T14:11:24,538 copying multi_loras/slora/router/peft_req_queue.py -> build/lib/multi_loras/slora/router 2023-11-25T14:11:24,540 copying multi_loras/slora/router/__init__.py -> build/lib/multi_loras/slora/router 2023-11-25T14:11:24,542 copying multi_loras/slora/router/manager.py -> build/lib/multi_loras/slora/router 2023-11-25T14:11:24,545 creating build/lib/multi_loras/slora/models 2023-11-25T14:11:24,546 copying multi_loras/slora/models/__init__.py -> build/lib/multi_loras/slora/models 2023-11-25T14:11:24,548 creating build/lib/multi_loras/slora/common/configs 2023-11-25T14:11:24,549 copying multi_loras/slora/common/configs/config.py -> build/lib/multi_loras/slora/common/configs 2023-11-25T14:11:24,551 copying multi_loras/slora/common/configs/__init__.py -> build/lib/multi_loras/slora/common/configs 2023-11-25T14:11:24,554 creating build/lib/multi_loras/slora/common/basemodel 2023-11-25T14:11:24,555 copying multi_loras/slora/common/basemodel/basemodel.py -> build/lib/multi_loras/slora/common/basemodel 2023-11-25T14:11:24,558 copying multi_loras/slora/common/basemodel/infer_struct.py -> build/lib/multi_loras/slora/common/basemodel 2023-11-25T14:11:24,560 copying multi_loras/slora/common/basemodel/__init__.py -> build/lib/multi_loras/slora/common/basemodel 2023-11-25T14:11:24,562 creating build/lib/multi_loras/slora/common/basemodel/layer_infer 2023-11-25T14:11:24,563 copying multi_loras/slora/common/basemodel/layer_infer/base_layer_infer.py -> build/lib/multi_loras/slora/common/basemodel/layer_infer 2023-11-25T14:11:24,565 copying multi_loras/slora/common/basemodel/layer_infer/transformer_layer_infer.py -> build/lib/multi_loras/slora/common/basemodel/layer_infer 2023-11-25T14:11:24,567 copying multi_loras/slora/common/basemodel/layer_infer/post_layer_infer.py -> build/lib/multi_loras/slora/common/basemodel/layer_infer 2023-11-25T14:11:24,569 copying multi_loras/slora/common/basemodel/layer_infer/pre_layer_infer.py -> build/lib/multi_loras/slora/common/basemodel/layer_infer 2023-11-25T14:11:24,571 copying multi_loras/slora/common/basemodel/layer_infer/__init__.py -> build/lib/multi_loras/slora/common/basemodel/layer_infer 2023-11-25T14:11:24,573 creating build/lib/multi_loras/slora/common/basemodel/triton_kernel 2023-11-25T14:11:24,574 copying multi_loras/slora/common/basemodel/triton_kernel/dequantize_gemm_int8.py -> build/lib/multi_loras/slora/common/basemodel/triton_kernel 2023-11-25T14:11:24,577 copying multi_loras/slora/common/basemodel/triton_kernel/quantize_gemm_int8.py -> build/lib/multi_loras/slora/common/basemodel/triton_kernel 2023-11-25T14:11:24,580 copying multi_loras/slora/common/basemodel/triton_kernel/destindex_copy_kv.py -> build/lib/multi_loras/slora/common/basemodel/triton_kernel 2023-11-25T14:11:24,582 copying multi_loras/slora/common/basemodel/triton_kernel/apply_penalty.py -> build/lib/multi_loras/slora/common/basemodel/triton_kernel 2023-11-25T14:11:24,584 copying multi_loras/slora/common/basemodel/triton_kernel/__init__.py -> build/lib/multi_loras/slora/common/basemodel/triton_kernel 2023-11-25T14:11:24,586 copying multi_loras/slora/common/basemodel/triton_kernel/dequantize_gemm_int4.py -> build/lib/multi_loras/slora/common/basemodel/triton_kernel 2023-11-25T14:11:24,589 creating build/lib/multi_loras/slora/common/basemodel/layer_weights 2023-11-25T14:11:24,590 copying multi_loras/slora/common/basemodel/layer_weights/transformer_layer_weight.py -> build/lib/multi_loras/slora/common/basemodel/layer_weights 2023-11-25T14:11:24,592 copying multi_loras/slora/common/basemodel/layer_weights/hf_load_utils.py -> build/lib/multi_loras/slora/common/basemodel/layer_weights 2023-11-25T14:11:24,594 copying multi_loras/slora/common/basemodel/layer_weights/pre_and_post_layer_weight.py -> build/lib/multi_loras/slora/common/basemodel/layer_weights 2023-11-25T14:11:24,596 copying multi_loras/slora/common/basemodel/layer_weights/base_layer_weight.py -> build/lib/multi_loras/slora/common/basemodel/layer_weights 2023-11-25T14:11:24,598 copying multi_loras/slora/common/basemodel/layer_weights/__init__.py -> build/lib/multi_loras/slora/common/basemodel/layer_weights 2023-11-25T14:11:24,600 creating build/lib/multi_loras/slora/common/basemodel/layer_infer/template 2023-11-25T14:11:24,601 copying multi_loras/slora/common/basemodel/layer_infer/template/pre_layer_infer_template.py -> build/lib/multi_loras/slora/common/basemodel/layer_infer/template 2023-11-25T14:11:24,603 copying multi_loras/slora/common/basemodel/layer_infer/template/transformer_layer_infer_template.py -> build/lib/multi_loras/slora/common/basemodel/layer_infer/template 2023-11-25T14:11:24,605 copying multi_loras/slora/common/basemodel/layer_infer/template/post_layer_infer_template.py -> build/lib/multi_loras/slora/common/basemodel/layer_infer/template 2023-11-25T14:11:24,607 copying multi_loras/slora/common/basemodel/layer_infer/template/__init__.py -> build/lib/multi_loras/slora/common/basemodel/layer_infer/template 2023-11-25T14:11:24,609 creating build/lib/multi_loras/slora/router/model_infer 2023-11-25T14:11:24,610 copying multi_loras/slora/router/model_infer/naive_infer_adapter.py -> build/lib/multi_loras/slora/router/model_infer 2023-11-25T14:11:24,613 copying multi_loras/slora/router/model_infer/model_rpc.py -> build/lib/multi_loras/slora/router/model_infer 2023-11-25T14:11:24,616 copying multi_loras/slora/router/model_infer/infer_batch.py -> build/lib/multi_loras/slora/router/model_infer 2023-11-25T14:11:24,619 copying multi_loras/slora/router/model_infer/post_process.py -> build/lib/multi_loras/slora/router/model_infer 2023-11-25T14:11:24,620 copying multi_loras/slora/router/model_infer/__init__.py -> build/lib/multi_loras/slora/router/model_infer 2023-11-25T14:11:24,622 copying multi_loras/slora/router/model_infer/infer_adapter.py -> build/lib/multi_loras/slora/router/model_infer 2023-11-25T14:11:24,626 creating build/lib/multi_loras/slora/models/llama2 2023-11-25T14:11:24,627 copying multi_loras/slora/models/llama2/model.py -> build/lib/multi_loras/slora/models/llama2 2023-11-25T14:11:24,632 copying multi_loras/slora/models/llama2/__init__.py -> build/lib/multi_loras/slora/models/llama2 2023-11-25T14:11:24,633 creating build/lib/multi_loras/slora/models/llama 2023-11-25T14:11:24,636 copying multi_loras/slora/models/llama/model.py -> build/lib/multi_loras/slora/models/llama 2023-11-25T14:11:24,638 copying multi_loras/slora/models/llama/infer_struct.py -> build/lib/multi_loras/slora/models/llama 2023-11-25T14:11:24,640 copying multi_loras/slora/models/llama/__init__.py -> build/lib/multi_loras/slora/models/llama 2023-11-25T14:11:24,642 creating build/lib/multi_loras/slora/models/llama2/layer_infer 2023-11-25T14:11:24,643 copying multi_loras/slora/models/llama2/layer_infer/transformer_layer_infer.py -> build/lib/multi_loras/slora/models/llama2/layer_infer 2023-11-25T14:11:24,645 copying multi_loras/slora/models/llama2/layer_infer/__init__.py -> build/lib/multi_loras/slora/models/llama2/layer_infer 2023-11-25T14:11:24,648 creating build/lib/multi_loras/slora/models/llama2/triton_kernel 2023-11-25T14:11:24,649 copying multi_loras/slora/models/llama2/triton_kernel/context_flashattention_nopad.py -> build/lib/multi_loras/slora/models/llama2/triton_kernel 2023-11-25T14:11:24,652 copying multi_loras/slora/models/llama2/triton_kernel/token_attention_nopad_softmax.py -> build/lib/multi_loras/slora/models/llama2/triton_kernel 2023-11-25T14:11:24,653 copying multi_loras/slora/models/llama2/triton_kernel/__init__.py -> build/lib/multi_loras/slora/models/llama2/triton_kernel 2023-11-25T14:11:24,655 copying multi_loras/slora/models/llama2/triton_kernel/token_attention_softmax_and_reducev.py -> build/lib/multi_loras/slora/models/llama2/triton_kernel 2023-11-25T14:11:24,657 copying multi_loras/slora/models/llama2/triton_kernel/token_attention_nopad_reduceV.py -> build/lib/multi_loras/slora/models/llama2/triton_kernel 2023-11-25T14:11:24,659 copying multi_loras/slora/models/llama2/triton_kernel/token_attention_nopad_att1.py -> build/lib/multi_loras/slora/models/llama2/triton_kernel 2023-11-25T14:11:24,661 creating build/lib/multi_loras/slora/models/llama2/layer_weights 2023-11-25T14:11:24,662 copying multi_loras/slora/models/llama2/layer_weights/transformer_layer_weight.py -> build/lib/multi_loras/slora/models/llama2/layer_weights 2023-11-25T14:11:24,664 copying multi_loras/slora/models/llama2/layer_weights/__init__.py -> build/lib/multi_loras/slora/models/llama2/layer_weights 2023-11-25T14:11:24,667 creating build/lib/multi_loras/slora/models/llama/layer_infer 2023-11-25T14:11:24,668 copying multi_loras/slora/models/llama/layer_infer/transformer_layer_infer.py -> build/lib/multi_loras/slora/models/llama/layer_infer 2023-11-25T14:11:24,670 copying multi_loras/slora/models/llama/layer_infer/post_layer_infer.py -> build/lib/multi_loras/slora/models/llama/layer_infer 2023-11-25T14:11:24,672 copying multi_loras/slora/models/llama/layer_infer/pre_layer_infer.py -> build/lib/multi_loras/slora/models/llama/layer_infer 2023-11-25T14:11:24,674 copying multi_loras/slora/models/llama/layer_infer/__init__.py -> build/lib/multi_loras/slora/models/llama/layer_infer 2023-11-25T14:11:24,677 creating build/lib/multi_loras/slora/models/llama/triton_kernel 2023-11-25T14:11:24,678 copying multi_loras/slora/models/llama/triton_kernel/context_flashattention_nopad.py -> build/lib/multi_loras/slora/models/llama/triton_kernel 2023-11-25T14:11:24,681 copying multi_loras/slora/models/llama/triton_kernel/rmsnorm.py -> build/lib/multi_loras/slora/models/llama/triton_kernel 2023-11-25T14:11:24,683 copying multi_loras/slora/models/llama/triton_kernel/token_attention_nopad_softmax.py -> build/lib/multi_loras/slora/models/llama/triton_kernel 2023-11-25T14:11:24,684 copying multi_loras/slora/models/llama/triton_kernel/__init__.py -> build/lib/multi_loras/slora/models/llama/triton_kernel 2023-11-25T14:11:24,686 copying multi_loras/slora/models/llama/triton_kernel/token_attention_softmax_and_reducev.py -> build/lib/multi_loras/slora/models/llama/triton_kernel 2023-11-25T14:11:24,688 copying multi_loras/slora/models/llama/triton_kernel/token_attention_nopad_reduceV.py -> build/lib/multi_loras/slora/models/llama/triton_kernel 2023-11-25T14:11:24,691 copying multi_loras/slora/models/llama/triton_kernel/token_attention_nopad_att1.py -> build/lib/multi_loras/slora/models/llama/triton_kernel 2023-11-25T14:11:24,693 copying multi_loras/slora/models/llama/triton_kernel/rotary_emb.py -> build/lib/multi_loras/slora/models/llama/triton_kernel 2023-11-25T14:11:24,695 creating build/lib/multi_loras/slora/models/llama/layer_weights 2023-11-25T14:11:24,696 copying multi_loras/slora/models/llama/layer_weights/transformer_layer_weight.py -> build/lib/multi_loras/slora/models/llama/layer_weights 2023-11-25T14:11:24,699 copying multi_loras/slora/models/llama/layer_weights/pre_and_post_layer_weight.py -> build/lib/multi_loras/slora/models/llama/layer_weights 2023-11-25T14:11:24,701 copying multi_loras/slora/models/llama/layer_weights/__init__.py -> build/lib/multi_loras/slora/models/llama/layer_weights 2023-11-25T14:11:24,702 running egg_info 2023-11-25T14:11:24,757 writing multi_loras.egg-info/PKG-INFO 2023-11-25T14:11:24,761 writing dependency_links to multi_loras.egg-info/dependency_links.txt 2023-11-25T14:11:24,762 writing requirements to multi_loras.egg-info/requires.txt 2023-11-25T14:11:24,763 writing top-level names to multi_loras.egg-info/top_level.txt 2023-11-25T14:11:24,806 reading manifest file 'multi_loras.egg-info/SOURCES.txt' 2023-11-25T14:11:24,810 adding license file 'LICENSE' 2023-11-25T14:11:24,816 writing manifest file 'multi_loras.egg-info/SOURCES.txt' 2023-11-25T14:11:24,859 /usr/local/lib/python3.11/dist-packages/setuptools/_distutils/cmd.py:66: SetuptoolsDeprecationWarning: setup.py install is deprecated. 2023-11-25T14:11:24,859 !! 2023-11-25T14:11:24,860 ******************************************************************************** 2023-11-25T14:11:24,861 Please avoid running ``setup.py`` directly. 2023-11-25T14:11:24,861 Instead, use pypa/build, pypa/installer or other 2023-11-25T14:11:24,862 standards-based tools. 2023-11-25T14:11:24,863 See https://blog.ganssle.io/articles/2021/10/setup-py-deprecated.html for details. 2023-11-25T14:11:24,863 ******************************************************************************** 2023-11-25T14:11:24,864 !! 2023-11-25T14:11:24,865 self.initialize_options() 2023-11-25T14:11:24,884 installing to build/bdist.linux-armv7l/wheel 2023-11-25T14:11:24,885 running install 2023-11-25T14:11:24,909 running install_lib 2023-11-25T14:11:24,933 creating build/bdist.linux-armv7l 2023-11-25T14:11:24,934 creating build/bdist.linux-armv7l/wheel 2023-11-25T14:11:24,936 creating build/bdist.linux-armv7l/wheel/multi_loras 2023-11-25T14:11:24,937 copying build/lib/multi_loras/merge_models.py -> build/bdist.linux-armv7l/wheel/multi_loras 2023-11-25T14:11:24,939 copying build/lib/multi_loras/merge_peft_adapters.py -> build/bdist.linux-armv7l/wheel/multi_loras 2023-11-25T14:11:24,941 copying build/lib/multi_loras/__version__.py -> build/bdist.linux-armv7l/wheel/multi_loras 2023-11-25T14:11:24,943 copying build/lib/multi_loras/__main__.py -> build/bdist.linux-armv7l/wheel/multi_loras 2023-11-25T14:11:24,945 copying build/lib/multi_loras/merging_methods.py -> build/bdist.linux-armv7l/wheel/multi_loras 2023-11-25T14:11:24,948 copying build/lib/multi_loras/delta_weights.py -> build/bdist.linux-armv7l/wheel/multi_loras 2023-11-25T14:11:24,950 copying build/lib/multi_loras/extract_lora.py -> build/bdist.linux-armv7l/wheel/multi_loras 2023-11-25T14:11:24,953 creating build/bdist.linux-armv7l/wheel/multi_loras/slora 2023-11-25T14:11:24,954 copying build/lib/multi_loras/slora/install_slora_kernel.py -> build/bdist.linux-armv7l/wheel/multi_loras/slora 2023-11-25T14:11:24,956 copying build/lib/multi_loras/slora/io_struct.py -> build/bdist.linux-armv7l/wheel/multi_loras/slora 2023-11-25T14:11:24,959 creating build/bdist.linux-armv7l/wheel/multi_loras/slora/common 2023-11-25T14:11:24,960 creating build/bdist.linux-armv7l/wheel/multi_loras/slora/common/configs 2023-11-25T14:11:24,961 copying build/lib/multi_loras/slora/common/configs/config.py -> build/bdist.linux-armv7l/wheel/multi_loras/slora/common/configs 2023-11-25T14:11:24,963 copying build/lib/multi_loras/slora/common/configs/__init__.py -> build/bdist.linux-armv7l/wheel/multi_loras/slora/common/configs 2023-11-25T14:11:24,964 copying build/lib/multi_loras/slora/common/gqa_mem_manager.py -> build/bdist.linux-armv7l/wheel/multi_loras/slora/common 2023-11-25T14:11:24,966 copying build/lib/multi_loras/slora/common/mem_manager.py -> build/bdist.linux-armv7l/wheel/multi_loras/slora/common 2023-11-25T14:11:24,968 copying build/lib/multi_loras/slora/common/mem_allocator.py -> build/bdist.linux-armv7l/wheel/multi_loras/slora/common 2023-11-25T14:11:24,970 copying build/lib/multi_loras/slora/common/ppl_int8kv_mem_manager.py -> build/bdist.linux-armv7l/wheel/multi_loras/slora/common 2023-11-25T14:11:24,972 copying build/lib/multi_loras/slora/common/build_utils.py -> build/bdist.linux-armv7l/wheel/multi_loras/slora/common 2023-11-25T14:11:24,974 creating build/bdist.linux-armv7l/wheel/multi_loras/slora/common/basemodel 2023-11-25T14:11:24,976 creating build/bdist.linux-armv7l/wheel/multi_loras/slora/common/basemodel/layer_infer 2023-11-25T14:11:24,977 creating build/bdist.linux-armv7l/wheel/multi_loras/slora/common/basemodel/layer_infer/template 2023-11-25T14:11:24,978 copying build/lib/multi_loras/slora/common/basemodel/layer_infer/template/pre_layer_infer_template.py -> build/bdist.linux-armv7l/wheel/multi_loras/slora/common/basemodel/layer_infer/template 2023-11-25T14:11:24,980 copying build/lib/multi_loras/slora/common/basemodel/layer_infer/template/transformer_layer_infer_template.py -> build/bdist.linux-armv7l/wheel/multi_loras/slora/common/basemodel/layer_infer/template 2023-11-25T14:11:24,982 copying build/lib/multi_loras/slora/common/basemodel/layer_infer/template/post_layer_infer_template.py -> build/bdist.linux-armv7l/wheel/multi_loras/slora/common/basemodel/layer_infer/template 2023-11-25T14:11:24,984 copying build/lib/multi_loras/slora/common/basemodel/layer_infer/template/__init__.py -> build/bdist.linux-armv7l/wheel/multi_loras/slora/common/basemodel/layer_infer/template 2023-11-25T14:11:24,986 copying build/lib/multi_loras/slora/common/basemodel/layer_infer/base_layer_infer.py -> build/bdist.linux-armv7l/wheel/multi_loras/slora/common/basemodel/layer_infer 2023-11-25T14:11:24,988 copying build/lib/multi_loras/slora/common/basemodel/layer_infer/transformer_layer_infer.py -> build/bdist.linux-armv7l/wheel/multi_loras/slora/common/basemodel/layer_infer 2023-11-25T14:11:24,989 copying build/lib/multi_loras/slora/common/basemodel/layer_infer/post_layer_infer.py -> build/bdist.linux-armv7l/wheel/multi_loras/slora/common/basemodel/layer_infer 2023-11-25T14:11:24,991 copying build/lib/multi_loras/slora/common/basemodel/layer_infer/pre_layer_infer.py -> build/bdist.linux-armv7l/wheel/multi_loras/slora/common/basemodel/layer_infer 2023-11-25T14:11:24,993 copying build/lib/multi_loras/slora/common/basemodel/layer_infer/__init__.py -> build/bdist.linux-armv7l/wheel/multi_loras/slora/common/basemodel/layer_infer 2023-11-25T14:11:24,995 creating build/bdist.linux-armv7l/wheel/multi_loras/slora/common/basemodel/triton_kernel 2023-11-25T14:11:24,996 copying build/lib/multi_loras/slora/common/basemodel/triton_kernel/dequantize_gemm_int8.py -> build/bdist.linux-armv7l/wheel/multi_loras/slora/common/basemodel/triton_kernel 2023-11-25T14:11:24,998 copying build/lib/multi_loras/slora/common/basemodel/triton_kernel/quantize_gemm_int8.py -> build/bdist.linux-armv7l/wheel/multi_loras/slora/common/basemodel/triton_kernel 2023-11-25T14:11:25,001 copying build/lib/multi_loras/slora/common/basemodel/triton_kernel/destindex_copy_kv.py -> build/bdist.linux-armv7l/wheel/multi_loras/slora/common/basemodel/triton_kernel 2023-11-25T14:11:25,003 copying build/lib/multi_loras/slora/common/basemodel/triton_kernel/apply_penalty.py -> build/bdist.linux-armv7l/wheel/multi_loras/slora/common/basemodel/triton_kernel 2023-11-25T14:11:25,005 copying build/lib/multi_loras/slora/common/basemodel/triton_kernel/__init__.py -> build/bdist.linux-armv7l/wheel/multi_loras/slora/common/basemodel/triton_kernel 2023-11-25T14:11:25,006 copying build/lib/multi_loras/slora/common/basemodel/triton_kernel/dequantize_gemm_int4.py -> build/bdist.linux-armv7l/wheel/multi_loras/slora/common/basemodel/triton_kernel 2023-11-25T14:11:25,009 copying build/lib/multi_loras/slora/common/basemodel/basemodel.py -> build/bdist.linux-armv7l/wheel/multi_loras/slora/common/basemodel 2023-11-25T14:11:25,012 creating build/bdist.linux-armv7l/wheel/multi_loras/slora/common/basemodel/layer_weights 2023-11-25T14:11:25,013 copying build/lib/multi_loras/slora/common/basemodel/layer_weights/transformer_layer_weight.py -> build/bdist.linux-armv7l/wheel/multi_loras/slora/common/basemodel/layer_weights 2023-11-25T14:11:25,015 copying build/lib/multi_loras/slora/common/basemodel/layer_weights/hf_load_utils.py -> build/bdist.linux-armv7l/wheel/multi_loras/slora/common/basemodel/layer_weights 2023-11-25T14:11:25,017 copying build/lib/multi_loras/slora/common/basemodel/layer_weights/pre_and_post_layer_weight.py -> build/bdist.linux-armv7l/wheel/multi_loras/slora/common/basemodel/layer_weights 2023-11-25T14:11:25,018 copying build/lib/multi_loras/slora/common/basemodel/layer_weights/base_layer_weight.py -> build/bdist.linux-armv7l/wheel/multi_loras/slora/common/basemodel/layer_weights 2023-11-25T14:11:25,020 copying build/lib/multi_loras/slora/common/basemodel/layer_weights/__init__.py -> build/bdist.linux-armv7l/wheel/multi_loras/slora/common/basemodel/layer_weights 2023-11-25T14:11:25,022 copying build/lib/multi_loras/slora/common/basemodel/infer_struct.py -> build/bdist.linux-armv7l/wheel/multi_loras/slora/common/basemodel 2023-11-25T14:11:25,024 copying build/lib/multi_loras/slora/common/basemodel/__init__.py -> build/bdist.linux-armv7l/wheel/multi_loras/slora/common/basemodel 2023-11-25T14:11:25,026 copying build/lib/multi_loras/slora/common/__init__.py -> build/bdist.linux-armv7l/wheel/multi_loras/slora/common 2023-11-25T14:11:25,027 copying build/lib/multi_loras/slora/common/int8kv_mem_manager.py -> build/bdist.linux-armv7l/wheel/multi_loras/slora/common 2023-11-25T14:11:25,029 copying build/lib/multi_loras/slora/common/infer_utils.py -> build/bdist.linux-armv7l/wheel/multi_loras/slora/common 2023-11-25T14:11:25,031 copying build/lib/multi_loras/slora/sampling_params.py -> build/bdist.linux-armv7l/wheel/multi_loras/slora 2023-11-25T14:11:25,034 creating build/bdist.linux-armv7l/wheel/multi_loras/slora/utils 2023-11-25T14:11:25,035 copying build/lib/multi_loras/slora/utils/model_load.py -> build/bdist.linux-armv7l/wheel/multi_loras/slora/utils 2023-11-25T14:11:25,037 copying build/lib/multi_loras/slora/utils/net_utils.py -> build/bdist.linux-armv7l/wheel/multi_loras/slora/utils 2023-11-25T14:11:25,038 copying build/lib/multi_loras/slora/utils/model_utils.py -> build/bdist.linux-armv7l/wheel/multi_loras/slora/utils 2023-11-25T14:11:25,040 copying build/lib/multi_loras/slora/utils/metric.py -> build/bdist.linux-armv7l/wheel/multi_loras/slora/utils 2023-11-25T14:11:25,043 copying build/lib/multi_loras/slora/utils/__init__.py -> build/bdist.linux-armv7l/wheel/multi_loras/slora/utils 2023-11-25T14:11:25,044 copying build/lib/multi_loras/slora/utils/infer_utils.py -> build/bdist.linux-armv7l/wheel/multi_loras/slora/utils 2023-11-25T14:11:25,046 copying build/lib/multi_loras/slora/slora_server.py -> build/bdist.linux-armv7l/wheel/multi_loras/slora 2023-11-25T14:11:25,050 creating build/bdist.linux-armv7l/wheel/multi_loras/slora/router 2023-11-25T14:11:25,051 copying build/lib/multi_loras/slora/router/profiler.py -> build/bdist.linux-armv7l/wheel/multi_loras/slora/router 2023-11-25T14:11:25,053 copying build/lib/multi_loras/slora/router/abort_req_queue.py -> build/bdist.linux-armv7l/wheel/multi_loras/slora/router 2023-11-25T14:11:25,055 copying build/lib/multi_loras/slora/router/pets_req_queue.py -> build/bdist.linux-armv7l/wheel/multi_loras/slora/router 2023-11-25T14:11:25,058 copying build/lib/multi_loras/slora/router/input_params.py -> build/bdist.linux-armv7l/wheel/multi_loras/slora/router 2023-11-25T14:11:25,060 creating build/bdist.linux-armv7l/wheel/multi_loras/slora/router/model_infer 2023-11-25T14:11:25,061 copying build/lib/multi_loras/slora/router/model_infer/naive_infer_adapter.py -> build/bdist.linux-armv7l/wheel/multi_loras/slora/router/model_infer 2023-11-25T14:11:25,063 copying build/lib/multi_loras/slora/router/model_infer/model_rpc.py -> build/bdist.linux-armv7l/wheel/multi_loras/slora/router/model_infer 2023-11-25T14:11:25,066 copying build/lib/multi_loras/slora/router/model_infer/infer_batch.py -> build/bdist.linux-armv7l/wheel/multi_loras/slora/router/model_infer 2023-11-25T14:11:25,069 copying build/lib/multi_loras/slora/router/model_infer/post_process.py -> build/bdist.linux-armv7l/wheel/multi_loras/slora/router/model_infer 2023-11-25T14:11:25,070 copying build/lib/multi_loras/slora/router/model_infer/__init__.py -> build/bdist.linux-armv7l/wheel/multi_loras/slora/router/model_infer 2023-11-25T14:11:25,072 copying build/lib/multi_loras/slora/router/model_infer/infer_adapter.py -> build/bdist.linux-armv7l/wheel/multi_loras/slora/router/model_infer 2023-11-25T14:11:25,074 copying build/lib/multi_loras/slora/router/cluster_req_queue.py -> build/bdist.linux-armv7l/wheel/multi_loras/slora/router 2023-11-25T14:11:25,076 copying build/lib/multi_loras/slora/router/stats.py -> build/bdist.linux-armv7l/wheel/multi_loras/slora/router 2023-11-25T14:11:25,078 copying build/lib/multi_loras/slora/router/req_queue.py -> build/bdist.linux-armv7l/wheel/multi_loras/slora/router 2023-11-25T14:11:25,080 copying build/lib/multi_loras/slora/router/peft_req_queue.py -> build/bdist.linux-armv7l/wheel/multi_loras/slora/router 2023-11-25T14:11:25,082 copying build/lib/multi_loras/slora/router/__init__.py -> build/bdist.linux-armv7l/wheel/multi_loras/slora/router 2023-11-25T14:11:25,084 copying build/lib/multi_loras/slora/router/manager.py -> build/bdist.linux-armv7l/wheel/multi_loras/slora/router 2023-11-25T14:11:25,087 copying build/lib/multi_loras/slora/__init__.py -> build/bdist.linux-armv7l/wheel/multi_loras/slora 2023-11-25T14:11:25,089 creating build/bdist.linux-armv7l/wheel/multi_loras/slora/models 2023-11-25T14:11:25,090 creating build/bdist.linux-armv7l/wheel/multi_loras/slora/models/llama2 2023-11-25T14:11:25,091 creating build/bdist.linux-armv7l/wheel/multi_loras/slora/models/llama2/layer_infer 2023-11-25T14:11:25,092 copying build/lib/multi_loras/slora/models/llama2/layer_infer/transformer_layer_infer.py -> build/bdist.linux-armv7l/wheel/multi_loras/slora/models/llama2/layer_infer 2023-11-25T14:11:25,095 copying build/lib/multi_loras/slora/models/llama2/layer_infer/__init__.py -> build/bdist.linux-armv7l/wheel/multi_loras/slora/models/llama2/layer_infer 2023-11-25T14:11:25,097 creating build/bdist.linux-armv7l/wheel/multi_loras/slora/models/llama2/triton_kernel 2023-11-25T14:11:25,098 copying build/lib/multi_loras/slora/models/llama2/triton_kernel/context_flashattention_nopad.py -> build/bdist.linux-armv7l/wheel/multi_loras/slora/models/llama2/triton_kernel 2023-11-25T14:11:25,100 copying build/lib/multi_loras/slora/models/llama2/triton_kernel/token_attention_nopad_softmax.py -> build/bdist.linux-armv7l/wheel/multi_loras/slora/models/llama2/triton_kernel 2023-11-25T14:11:25,102 copying build/lib/multi_loras/slora/models/llama2/triton_kernel/__init__.py -> build/bdist.linux-armv7l/wheel/multi_loras/slora/models/llama2/triton_kernel 2023-11-25T14:11:25,103 copying build/lib/multi_loras/slora/models/llama2/triton_kernel/token_attention_softmax_and_reducev.py -> build/bdist.linux-armv7l/wheel/multi_loras/slora/models/llama2/triton_kernel 2023-11-25T14:11:25,106 copying build/lib/multi_loras/slora/models/llama2/triton_kernel/token_attention_nopad_reduceV.py -> build/bdist.linux-armv7l/wheel/multi_loras/slora/models/llama2/triton_kernel 2023-11-25T14:11:25,108 copying build/lib/multi_loras/slora/models/llama2/triton_kernel/token_attention_nopad_att1.py -> build/bdist.linux-armv7l/wheel/multi_loras/slora/models/llama2/triton_kernel 2023-11-25T14:11:25,109 copying build/lib/multi_loras/slora/models/llama2/model.py -> build/bdist.linux-armv7l/wheel/multi_loras/slora/models/llama2 2023-11-25T14:11:25,112 creating build/bdist.linux-armv7l/wheel/multi_loras/slora/models/llama2/layer_weights 2023-11-25T14:11:25,113 copying build/lib/multi_loras/slora/models/llama2/layer_weights/transformer_layer_weight.py -> build/bdist.linux-armv7l/wheel/multi_loras/slora/models/llama2/layer_weights 2023-11-25T14:11:25,114 copying build/lib/multi_loras/slora/models/llama2/layer_weights/__init__.py -> build/bdist.linux-armv7l/wheel/multi_loras/slora/models/llama2/layer_weights 2023-11-25T14:11:25,116 copying build/lib/multi_loras/slora/models/llama2/__init__.py -> build/bdist.linux-armv7l/wheel/multi_loras/slora/models/llama2 2023-11-25T14:11:25,118 creating build/bdist.linux-armv7l/wheel/multi_loras/slora/models/llama 2023-11-25T14:11:25,119 creating build/bdist.linux-armv7l/wheel/multi_loras/slora/models/llama/layer_infer 2023-11-25T14:11:25,120 copying build/lib/multi_loras/slora/models/llama/layer_infer/transformer_layer_infer.py -> build/bdist.linux-armv7l/wheel/multi_loras/slora/models/llama/layer_infer 2023-11-25T14:11:25,123 copying build/lib/multi_loras/slora/models/llama/layer_infer/post_layer_infer.py -> build/bdist.linux-armv7l/wheel/multi_loras/slora/models/llama/layer_infer 2023-11-25T14:11:25,125 copying build/lib/multi_loras/slora/models/llama/layer_infer/pre_layer_infer.py -> build/bdist.linux-armv7l/wheel/multi_loras/slora/models/llama/layer_infer 2023-11-25T14:11:25,127 copying build/lib/multi_loras/slora/models/llama/layer_infer/__init__.py -> build/bdist.linux-armv7l/wheel/multi_loras/slora/models/llama/layer_infer 2023-11-25T14:11:25,129 creating build/bdist.linux-armv7l/wheel/multi_loras/slora/models/llama/triton_kernel 2023-11-25T14:11:25,130 copying build/lib/multi_loras/slora/models/llama/triton_kernel/context_flashattention_nopad.py -> build/bdist.linux-armv7l/wheel/multi_loras/slora/models/llama/triton_kernel 2023-11-25T14:11:25,133 copying build/lib/multi_loras/slora/models/llama/triton_kernel/rmsnorm.py -> build/bdist.linux-armv7l/wheel/multi_loras/slora/models/llama/triton_kernel 2023-11-25T14:11:25,135 copying build/lib/multi_loras/slora/models/llama/triton_kernel/token_attention_nopad_softmax.py -> build/bdist.linux-armv7l/wheel/multi_loras/slora/models/llama/triton_kernel 2023-11-25T14:11:25,137 copying build/lib/multi_loras/slora/models/llama/triton_kernel/__init__.py -> build/bdist.linux-armv7l/wheel/multi_loras/slora/models/llama/triton_kernel 2023-11-25T14:11:25,139 copying build/lib/multi_loras/slora/models/llama/triton_kernel/token_attention_softmax_and_reducev.py -> build/bdist.linux-armv7l/wheel/multi_loras/slora/models/llama/triton_kernel 2023-11-25T14:11:25,141 copying build/lib/multi_loras/slora/models/llama/triton_kernel/token_attention_nopad_reduceV.py -> build/bdist.linux-armv7l/wheel/multi_loras/slora/models/llama/triton_kernel 2023-11-25T14:11:25,143 copying build/lib/multi_loras/slora/models/llama/triton_kernel/token_attention_nopad_att1.py -> build/bdist.linux-armv7l/wheel/multi_loras/slora/models/llama/triton_kernel 2023-11-25T14:11:25,145 copying build/lib/multi_loras/slora/models/llama/triton_kernel/rotary_emb.py -> build/bdist.linux-armv7l/wheel/multi_loras/slora/models/llama/triton_kernel 2023-11-25T14:11:25,147 copying build/lib/multi_loras/slora/models/llama/model.py -> build/bdist.linux-armv7l/wheel/multi_loras/slora/models/llama 2023-11-25T14:11:25,150 creating build/bdist.linux-armv7l/wheel/multi_loras/slora/models/llama/layer_weights 2023-11-25T14:11:25,151 copying build/lib/multi_loras/slora/models/llama/layer_weights/transformer_layer_weight.py -> build/bdist.linux-armv7l/wheel/multi_loras/slora/models/llama/layer_weights 2023-11-25T14:11:25,153 copying build/lib/multi_loras/slora/models/llama/layer_weights/pre_and_post_layer_weight.py -> build/bdist.linux-armv7l/wheel/multi_loras/slora/models/llama/layer_weights 2023-11-25T14:11:25,156 copying build/lib/multi_loras/slora/models/llama/layer_weights/__init__.py -> build/bdist.linux-armv7l/wheel/multi_loras/slora/models/llama/layer_weights 2023-11-25T14:11:25,157 copying build/lib/multi_loras/slora/models/llama/infer_struct.py -> build/bdist.linux-armv7l/wheel/multi_loras/slora/models/llama 2023-11-25T14:11:25,159 copying build/lib/multi_loras/slora/models/llama/__init__.py -> build/bdist.linux-armv7l/wheel/multi_loras/slora/models/llama 2023-11-25T14:11:25,160 copying build/lib/multi_loras/slora/models/__init__.py -> build/bdist.linux-armv7l/wheel/multi_loras/slora/models 2023-11-25T14:11:25,161 copying build/lib/multi_loras/__init__.py -> build/bdist.linux-armv7l/wheel/multi_loras 2023-11-25T14:11:25,163 copying build/lib/multi_loras/dare.py -> build/bdist.linux-armv7l/wheel/multi_loras 2023-11-25T14:11:25,165 running install_egg_info 2023-11-25T14:11:25,195 Copying multi_loras.egg-info to build/bdist.linux-armv7l/wheel/multi_loras-0.2.0-py3.11.egg-info 2023-11-25T14:11:25,206 running install_scripts 2023-11-25T14:11:25,220 creating build/bdist.linux-armv7l/wheel/multi_loras-0.2.0.dist-info/WHEEL 2023-11-25T14:11:25,223 creating '/tmp/pip-wheel-kls027r4/multi_loras-0.2.0-py3-none-any.whl' and adding 'build/bdist.linux-armv7l/wheel' to it 2023-11-25T14:11:25,225 adding 'multi_loras/__init__.py' 2023-11-25T14:11:25,227 adding 'multi_loras/__main__.py' 2023-11-25T14:11:25,228 adding 'multi_loras/__version__.py' 2023-11-25T14:11:25,230 adding 'multi_loras/dare.py' 2023-11-25T14:11:25,231 adding 'multi_loras/delta_weights.py' 2023-11-25T14:11:25,233 adding 'multi_loras/extract_lora.py' 2023-11-25T14:11:25,235 adding 'multi_loras/merge_models.py' 2023-11-25T14:11:25,236 adding 'multi_loras/merge_peft_adapters.py' 2023-11-25T14:11:25,242 adding 'multi_loras/merging_methods.py' 2023-11-25T14:11:25,244 adding 'multi_loras/slora/__init__.py' 2023-11-25T14:11:25,245 adding 'multi_loras/slora/install_slora_kernel.py' 2023-11-25T14:11:25,246 adding 'multi_loras/slora/io_struct.py' 2023-11-25T14:11:25,248 adding 'multi_loras/slora/sampling_params.py' 2023-11-25T14:11:25,251 adding 'multi_loras/slora/slora_server.py' 2023-11-25T14:11:25,253 adding 'multi_loras/slora/common/__init__.py' 2023-11-25T14:11:25,254 adding 'multi_loras/slora/common/build_utils.py' 2023-11-25T14:11:25,256 adding 'multi_loras/slora/common/gqa_mem_manager.py' 2023-11-25T14:11:25,257 adding 'multi_loras/slora/common/infer_utils.py' 2023-11-25T14:11:25,258 adding 'multi_loras/slora/common/int8kv_mem_manager.py' 2023-11-25T14:11:25,260 adding 'multi_loras/slora/common/mem_allocator.py' 2023-11-25T14:11:25,261 adding 'multi_loras/slora/common/mem_manager.py' 2023-11-25T14:11:25,262 adding 'multi_loras/slora/common/ppl_int8kv_mem_manager.py' 2023-11-25T14:11:25,264 adding 'multi_loras/slora/common/basemodel/__init__.py' 2023-11-25T14:11:25,266 adding 'multi_loras/slora/common/basemodel/basemodel.py' 2023-11-25T14:11:25,267 adding 'multi_loras/slora/common/basemodel/infer_struct.py' 2023-11-25T14:11:25,269 adding 'multi_loras/slora/common/basemodel/layer_infer/__init__.py' 2023-11-25T14:11:25,271 adding 'multi_loras/slora/common/basemodel/layer_infer/base_layer_infer.py' 2023-11-25T14:11:25,272 adding 'multi_loras/slora/common/basemodel/layer_infer/post_layer_infer.py' 2023-11-25T14:11:25,273 adding 'multi_loras/slora/common/basemodel/layer_infer/pre_layer_infer.py' 2023-11-25T14:11:25,274 adding 'multi_loras/slora/common/basemodel/layer_infer/transformer_layer_infer.py' 2023-11-25T14:11:25,276 adding 'multi_loras/slora/common/basemodel/layer_infer/template/__init__.py' 2023-11-25T14:11:25,277 adding 'multi_loras/slora/common/basemodel/layer_infer/template/post_layer_infer_template.py' 2023-11-25T14:11:25,278 adding 'multi_loras/slora/common/basemodel/layer_infer/template/pre_layer_infer_template.py' 2023-11-25T14:11:25,280 adding 'multi_loras/slora/common/basemodel/layer_infer/template/transformer_layer_infer_template.py' 2023-11-25T14:11:25,282 adding 'multi_loras/slora/common/basemodel/layer_weights/__init__.py' 2023-11-25T14:11:25,283 adding 'multi_loras/slora/common/basemodel/layer_weights/base_layer_weight.py' 2023-11-25T14:11:25,285 adding 'multi_loras/slora/common/basemodel/layer_weights/hf_load_utils.py' 2023-11-25T14:11:25,286 adding 'multi_loras/slora/common/basemodel/layer_weights/pre_and_post_layer_weight.py' 2023-11-25T14:11:25,287 adding 'multi_loras/slora/common/basemodel/layer_weights/transformer_layer_weight.py' 2023-11-25T14:11:25,289 adding 'multi_loras/slora/common/basemodel/triton_kernel/__init__.py' 2023-11-25T14:11:25,290 adding 'multi_loras/slora/common/basemodel/triton_kernel/apply_penalty.py' 2023-11-25T14:11:25,293 adding 'multi_loras/slora/common/basemodel/triton_kernel/dequantize_gemm_int4.py' 2023-11-25T14:11:25,295 adding 'multi_loras/slora/common/basemodel/triton_kernel/dequantize_gemm_int8.py' 2023-11-25T14:11:25,297 adding 'multi_loras/slora/common/basemodel/triton_kernel/destindex_copy_kv.py' 2023-11-25T14:11:25,299 adding 'multi_loras/slora/common/basemodel/triton_kernel/quantize_gemm_int8.py' 2023-11-25T14:11:25,301 adding 'multi_loras/slora/common/configs/__init__.py' 2023-11-25T14:11:25,302 adding 'multi_loras/slora/common/configs/config.py' 2023-11-25T14:11:25,304 adding 'multi_loras/slora/models/__init__.py' 2023-11-25T14:11:25,306 adding 'multi_loras/slora/models/llama/__init__.py' 2023-11-25T14:11:25,307 adding 'multi_loras/slora/models/llama/infer_struct.py' 2023-11-25T14:11:25,309 adding 'multi_loras/slora/models/llama/model.py' 2023-11-25T14:11:25,310 adding 'multi_loras/slora/models/llama/layer_infer/__init__.py' 2023-11-25T14:11:25,311 adding 'multi_loras/slora/models/llama/layer_infer/post_layer_infer.py' 2023-11-25T14:11:25,313 adding 'multi_loras/slora/models/llama/layer_infer/pre_layer_infer.py' 2023-11-25T14:11:25,314 adding 'multi_loras/slora/models/llama/layer_infer/transformer_layer_infer.py' 2023-11-25T14:11:25,316 adding 'multi_loras/slora/models/llama/layer_weights/__init__.py' 2023-11-25T14:11:25,318 adding 'multi_loras/slora/models/llama/layer_weights/pre_and_post_layer_weight.py' 2023-11-25T14:11:25,319 adding 'multi_loras/slora/models/llama/layer_weights/transformer_layer_weight.py' 2023-11-25T14:11:25,321 adding 'multi_loras/slora/models/llama/triton_kernel/__init__.py' 2023-11-25T14:11:25,324 adding 'multi_loras/slora/models/llama/triton_kernel/context_flashattention_nopad.py' 2023-11-25T14:11:25,326 adding 'multi_loras/slora/models/llama/triton_kernel/rmsnorm.py' 2023-11-25T14:11:25,327 adding 'multi_loras/slora/models/llama/triton_kernel/rotary_emb.py' 2023-11-25T14:11:25,329 adding 'multi_loras/slora/models/llama/triton_kernel/token_attention_nopad_att1.py' 2023-11-25T14:11:25,331 adding 'multi_loras/slora/models/llama/triton_kernel/token_attention_nopad_reduceV.py' 2023-11-25T14:11:25,332 adding 'multi_loras/slora/models/llama/triton_kernel/token_attention_nopad_softmax.py' 2023-11-25T14:11:25,334 adding 'multi_loras/slora/models/llama/triton_kernel/token_attention_softmax_and_reducev.py' 2023-11-25T14:11:25,336 adding 'multi_loras/slora/models/llama2/__init__.py' 2023-11-25T14:11:25,337 adding 'multi_loras/slora/models/llama2/model.py' 2023-11-25T14:11:25,339 adding 'multi_loras/slora/models/llama2/layer_infer/__init__.py' 2023-11-25T14:11:25,340 adding 'multi_loras/slora/models/llama2/layer_infer/transformer_layer_infer.py' 2023-11-25T14:11:25,342 adding 'multi_loras/slora/models/llama2/layer_weights/__init__.py' 2023-11-25T14:11:25,343 adding 'multi_loras/slora/models/llama2/layer_weights/transformer_layer_weight.py' 2023-11-25T14:11:25,345 adding 'multi_loras/slora/models/llama2/triton_kernel/__init__.py' 2023-11-25T14:11:25,347 adding 'multi_loras/slora/models/llama2/triton_kernel/context_flashattention_nopad.py' 2023-11-25T14:11:25,348 adding 'multi_loras/slora/models/llama2/triton_kernel/token_attention_nopad_att1.py' 2023-11-25T14:11:25,350 adding 'multi_loras/slora/models/llama2/triton_kernel/token_attention_nopad_reduceV.py' 2023-11-25T14:11:25,351 adding 'multi_loras/slora/models/llama2/triton_kernel/token_attention_nopad_softmax.py' 2023-11-25T14:11:25,353 adding 'multi_loras/slora/models/llama2/triton_kernel/token_attention_softmax_and_reducev.py' 2023-11-25T14:11:25,354 adding 'multi_loras/slora/router/__init__.py' 2023-11-25T14:11:25,356 adding 'multi_loras/slora/router/abort_req_queue.py' 2023-11-25T14:11:25,357 adding 'multi_loras/slora/router/cluster_req_queue.py' 2023-11-25T14:11:25,359 adding 'multi_loras/slora/router/input_params.py' 2023-11-25T14:11:25,361 adding 'multi_loras/slora/router/manager.py' 2023-11-25T14:11:25,363 adding 'multi_loras/slora/router/peft_req_queue.py' 2023-11-25T14:11:25,366 adding 'multi_loras/slora/router/pets_req_queue.py' 2023-11-25T14:11:25,367 adding 'multi_loras/slora/router/profiler.py' 2023-11-25T14:11:25,369 adding 'multi_loras/slora/router/req_queue.py' 2023-11-25T14:11:25,370 adding 'multi_loras/slora/router/stats.py' 2023-11-25T14:11:25,372 adding 'multi_loras/slora/router/model_infer/__init__.py' 2023-11-25T14:11:25,374 adding 'multi_loras/slora/router/model_infer/infer_adapter.py' 2023-11-25T14:11:25,376 adding 'multi_loras/slora/router/model_infer/infer_batch.py' 2023-11-25T14:11:25,378 adding 'multi_loras/slora/router/model_infer/model_rpc.py' 2023-11-25T14:11:25,380 adding 'multi_loras/slora/router/model_infer/naive_infer_adapter.py' 2023-11-25T14:11:25,382 adding 'multi_loras/slora/router/model_infer/post_process.py' 2023-11-25T14:11:25,383 adding 'multi_loras/slora/utils/__init__.py' 2023-11-25T14:11:25,385 adding 'multi_loras/slora/utils/infer_utils.py' 2023-11-25T14:11:25,386 adding 'multi_loras/slora/utils/metric.py' 2023-11-25T14:11:25,387 adding 'multi_loras/slora/utils/model_load.py' 2023-11-25T14:11:25,388 adding 'multi_loras/slora/utils/model_utils.py' 2023-11-25T14:11:25,389 adding 'multi_loras/slora/utils/net_utils.py' 2023-11-25T14:11:25,391 adding 'multi_loras-0.2.0.dist-info/LICENSE' 2023-11-25T14:11:25,393 adding 'multi_loras-0.2.0.dist-info/METADATA' 2023-11-25T14:11:25,394 adding 'multi_loras-0.2.0.dist-info/WHEEL' 2023-11-25T14:11:25,395 adding 'multi_loras-0.2.0.dist-info/top_level.txt' 2023-11-25T14:11:25,397 adding 'multi_loras-0.2.0.dist-info/RECORD' 2023-11-25T14:11:25,401 removing build/bdist.linux-armv7l/wheel 2023-11-25T14:11:25,542 Building wheel for multi-loras (setup.py): finished with status 'done' 2023-11-25T14:11:25,545 Created wheel for multi-loras: filename=multi_loras-0.2.0-py3-none-any.whl size=127636 sha256=3e6cace49921c89e7592128cf7376e900982a44acadc97243f43a753288ea1cc 2023-11-25T14:11:25,547 Stored in directory: /tmp/pip-ephem-wheel-cache-awvap8qb/wheels/9f/32/63/bc3729d33bb634d4d365e5a283d8c8220825f965cd1ad3bc28 2023-11-25T14:11:25,561 Successfully built multi-loras 2023-11-25T14:11:25,567 Removed build tracker: '/tmp/pip-build-tracker-dxf8ydgc'