突破性的多语言代码大模型基CodeShell：引领AI编程新时代

* 强大的性能：CodelShell在HumanEval和MBPP上达到了7B代码基座大模型的最优性能

* 完整的体系：除了代码大模型，同时开源IDE（VS Code与JetBrains）插件，形成开源的全栈技术体系

* 轻量化部署：支持本地C++部署，提供轻量快速的本地化软件开发助手解决方案

* 全面的评测：提供支持完整项目上下文、覆盖代码生成、代码缺陷检测与修复、测试用例生成等常见软件开发活动的多任务评测体系（即将开源）

* 高效的训练：基于高效的数据治理体系，CodeShell在完全冷启动情况下，只训练了五千亿Token即获得了优异的性能

- <a href="https://huggingface.co/WisdomShell/CodeShell" target="_blank"><b>CodeShell Base</b></a>：CodelShell底座模型，具有强大的代码基础能力。
- <a href="https://huggingface.co/WisdomShell/CodeShell-Chat" target="_blank"><b>CodeShell Chat</b></a>：CodelShell对话模型，在代码问答、代码补全等下游任务重性能优异。
- <a href="https://huggingface.co/WisdomShell/CodeShell-Chat-int4" target="_blank"><b>CodeShell Chat 4bit</b></a>：CodelShell对话模型4bit量化版本，在保证模型性能的前提下内存消耗更小，速度更快。
- <a href="https://github.com/WisdomShell/llama_cpp_for_codeshell" target="_blank"><b>CodeShell CPP</b></a>：CodelShell对话模型CPP版本，支持开发者在没有GPU的个人电脑中使用。注意，CPP版本同样支持量化操作，用户可以在最小内存为8G的个人电脑中运行CodeShell。

- python 3.8 and above
- pytorch 2.0 and above are recommended
- transformers 4.32 and above
- CUDA 11.8 and above are recommended (this is for GPU users, flash-attention users, etc.)

pip install -r requirements.txt

import torch
from transformers import AutoModelForCausalLM, AutoTokenizer

device = 'cuda' if torch.cuda.is_available() else 'cpu'
tokenizer = AutoTokenizer.from_pretrained("WisdomShell/CodeShell-7B")
model = AutoModelForCausalLM.from_pretrained("WisdomShell/CodeShell-7B", trust_remote_code=True, torch_dtype=torch.bfloat16).to(device)
inputs = tokenizer('def merge_sort():', return_tensors='pt').to(device)
outputs = model.generate(**inputs)
print(tokenizer.decode(outputs[0]))

input_text = "<fim_prefix>def print_hello_world():\n    <fim_suffix>\n    print('Hello world!')<fim_middle>"
inputs = tokenizer(input_text, return_tensors='pt').to(device)
outputs = model.generate(**inputs)
print(tokenizer.decode(outputs[0]))

model = AutoModelForCausalLM.from_pretrained('WisdomShell/CodeShell-7B-Chat', trust_remote_code=True, torch_dtype=torch.bfloat16).to(device)
tokenizer = AutoTokenizer.from_pretrained('WisdomShell/CodeShell-7B-Chat')

history = []
query = '你是谁?'
response = model.chat(query, history, tokenizer)
print(response)
history.append((query, response))

query = '用Python写一个HTTP server'
response = model.chat(query, history, tokenizer)
print(response)
history.append((query, response))

model = AutoModelForCausalLM.from_pretrained('WisdomShell/CodeShell-7B-Chat-int4', trust_remote_code=True).to(device)
tokenizer = AutoTokenizer.from_pretrained('WisdomShell/CodeShell-7B-Chat-int4')

python demos/web_demo.py

python demos/cli_demo.py

python demos/openai_api.py

curl http://127.0.0.1:8000/v1/chat/completions \
  -H "Content-Type: application/json" \
  -d '{
    "model": "CodeShell-7B-Chat",
    "messages": [
      {
        "role": "user",
        "content": "你好"
      }
    ]
  }'

* Hugging Face模型链接：[https://huggingface.co/WisdomShell/CodeShell-7B/tree/main](https://huggingface.co/WisdomShell/CodeShell-7B/tree/main)

* [codeshell](https://github.com/WisdomShell/codeshell)

* https://se.pku.edu.cn/kcl/