Code Llama for VSCode

Code Llama for VSCode

An API which mocks llama.cpp to enable support for Code Llama with the Continue Visual Studio Code extension.

As of the time of writing and to my knowledge, this is the only way to use Code Llama with VSCode locally without having to sign up or get an API key for a service. The only exception to this is Continue with Ollama, but Ollama doesn't support Windows or Linux. On the other hand, Code Llama for VSCode is completely cross-platform and will run wherever Meta's own codellama code will run.

Now let's get started!

Setup

Prerequisites:

• Download and run one of the Code Llama Instruct models
• Install the Continue VSCode extension

After you are able to use both independently, we will glue them together with Code Llama for VSCode.

Steps:

1. Move llamacpp_mock_api.py to your codellama folder and install Flask to your environment with pip install flask.
2. Run llamacpp_mock_api.py with your Code Llama Instruct torchrun command. For example:

torchrun --nproc_per_node 1 llamacpp_mock_api.py \
    --ckpt_dir CodeLlama-7b-Instruct/ \
    --tokenizer_path CodeLlama-7b-Instruct/tokenizer.model \
    --max_seq_len 512 --max_batch_size 4

1. Type /config in VSCode with Continue and make changes to config.py so it looks like this.

Restart VSCode or reload the Continue extension and you should now be able to use Code Llama for VSCode!

TODO: Response streaming

https://github.com/xNul/code-llama-for-vscode

相关推荐：

AGI结构模块很细分：脑网络结构高清大图

AGI部分模块的优秀复现：

最新代码：一个epoch打天下：深度Hebbian BP (华为实验室) 生物视觉 +

AGI之 概率溯因推理超越人类水平 VSA符号溯因abductive推理 +

Spaun2 Extending the World's Largest Functional Brain Model 架构 +

自由能AI模型的理论高度和潜力（信息量大）自由能loss +

DeepMind Dreamer 系列为什么效果这么好及自由能理论高度和潜力 世界模型+

世界模型仅用 1 小时训练一个四足机器人从头开始翻滚、站立和行走，无需重置。10 分钟内适应扰动或快速翻身站立 躯体控制 +

代码：Learning to Learn and Forget （华为）长短期记忆网络 +

inductive Logic Programs 视觉推理

框架及硬件

生物神经网络的开源芯片 +

矢量符号架构作为纳米级硬件的计算框架 +

Self-Expanding ⾃扩展神经⽹络

系统观：

大脑中复杂适应动力学的神经调节控制

突触神经耦合的混沌动力特性