4 minute read

DeepSeek-R1-Distill-Llama-8B 是一个基于 Llama 架构的 8B 参数语言模型,经过深度蒸馏(distillation)处理,旨在提高推理效率和精度。通过蒸馏技术,模型在保持较高性能的同时,减少了计算资源的消耗,特别适合在资源受限的环境中应用。该模型经过优化,可用于多种自然语言处理任务,如文本生成、情感分析、问答系统等。DeepSeek-R1-Distill-Llama-8B 结合了 Llama 的强大基础和蒸馏技术的优势,使得它在处理复杂问题时更加高效。对于需要平衡性能和资源消耗的应用场景,这款模型提供了一个理想的解决方案。

本篇笔记所对应的视频:

🚀简介:

UnSloth 是一个开源的 Python 库,旨在通过简单的接口实现大规模模型的分布式训练。它支持多种优化算法,并提供灵活的训练管道,帮助研究人员和开发者在高效的基础上进行深度学习实验。该项目的目标是降低训练大模型时的复杂性,提升计算资源的利用率,并支持异步训练等高级功能,适用于机器学习和深度学习领域的各种任务。更多详情可查看其 GitHub 仓库。

我所发布的微调视频:

wandb链接:https://wandb/

数据集:https://huggingface.co/datasets/b-mc2/sql-create-context/viewer

🚀代码

# -*- coding: utf-8 -*-
"""CoT.ipynb

Automatically generated by Colab.

Original file is located at
    https://colab.research.google.com/drive/1pSPKk3QmWFqT5VrixE1GOb_4SpkG_2YR
"""

# Commented out IPython magic to ensure Python compatibility.
# %%capture
# !pip install unsloth
# !pip install --force-reinstall --no-cache-dir --no-deps git+https://github.com/unslothai/unsloth.git
#

from huggingface_hub import login
from google.colab import userdata

hf_token = userdata.get('HF_TOKEN')

login(hf_token)

# 导入wandb库 - Weights & Biases,用于机器学习实验跟踪和可视化
import wandb

wb_token = userdata.get('WB_TOKEN')

wandb.login(key=wb_token)
run = wandb.init(
    project='Fine-tune-DeepSeek-R1-Distill-Llama-8B sql to text',    # 设置项目名称 - 这里是用于SQL分析的DeepSeek模型微调项目
    job_type="training",
    anonymous="allow"# 允许匿名访问 # "allow"表示即使没有wandb账号的用户也能查看这个项目
)

# 从unsloth库中导入FastLanguageModel类
# unsloth是一个优化的语言模型加载和训练库
from unsloth import FastLanguageModel

# 设置模型参数
# 最大序列长度,即模型能处理的最大token数量
max_seq_length = 2048
dtype = None # 数据类型设置为None,让模型自动选择合适的数据类型
load_in_4bit = True # 启用4bit量化加载 # 4bit量化可以显著减少模型内存占用,但可能略微影响模型性能

# 加载预训练模型和分词器
model, tokenizer = FastLanguageModel.from_pretrained(
    model_name = "unsloth/DeepSeek-R1-Distill-Llama-8B",
    max_seq_length = max_seq_length,     # 设置最大序列长度
    dtype = dtype,# 设置数据类型
    load_in_4bit = load_in_4bit,  # 启用4bit量化加载
    token = hf_token, # 使用Hugging Face的访问令牌
)

prompt = """Below is SQL query. Think like sql expert and generate a summary of the query which explains the use case of the query. As in
what the query is trying to read from the database in a usecase sense.

### Query:
SELECT
    c.customer_id,
    c.name AS customer_name,
    COUNT(o.order_id) AS total_orders,
    SUM(o.total_amount) AS total_spent,
    AVG(o.total_amount) AS avg_order_value,
    MAX(o.order_date) AS last_order_date
FROM customers c
JOIN orders o ON c.customer_id = o.customer_id
LEFT JOIN order_items oi ON o.order_id = oi.order_id
WHERE o.order_date >= '2024-01-01'
GROUP BY c.customer_id, c.name
ORDER BY total_spent DESC
LIMIT 10;

### Response:
"""

# 将模型切换到推理模式
FastLanguageModel.for_inference(model)
# 对输入文本进行分词和编码
# [prompt] - 将prompt放入列表中,因为tokenizer期望批处理输入
# return_tensors="pt" - 返回PyTorch张量格式
# to("cuda") - 将张量移动到GPU上进行计算
inputs = tokenizer([prompt], return_tensors="pt").to("cuda")

# 使用模型生成响应
outputs = model.generate(
    input_ids=inputs.input_ids,
    attention_mask=inputs.attention_mask,
    max_new_tokens=1200,
    use_cache=True,
)
# 后处理生成的输出
# batch_decode - 将标记ID序列解码回文本
# split("### Response:")[1] - 提取"### Response:"之后的部分,即模型的实际回答
response = tokenizer.batch_decode(outputs)
print(response[0].split("### Response:")[1])

#微调前:模型能给出基本的查询解释,但不够精确

train_prompt_style = """Below is an instruction that describes a task, paired with an input that provides further context.
Write a response that appropriately completes the request.
Before answering, think carefully about the question and create a step-by-step chain of thoughts to ensure a logical and accurate response.

### Instruction:
You are a SQL expert with advance understanding of SQL queries. You can understand database schema from the query. Think like sql expert and generate a summary of the query which explains the use case of the query. As in
what the query is trying to read from the database in a usecase sense.

### Query:
{}

### Response:
<think>
{}
</think>
{}"""

!pip install datasets
!wget "https://huggingface.co/datasets/b-mc2/sql-create-context/resolve/main/sql_create_context_v4.json"

from datasets import load_dataset
dataset = load_dataset("json", data_files="/content/sql_create_context_v4.json", split="train[0:500]")
# - split: 指定要加载的数据集切片
#   - "train[0:500]" 表示只加载训练集的前500条数据
#   - 这种切片方式可以用于快速实验和调试

EOS_TOKEN = tokenizer.eos_token  # Must add EOS_TOKEN

#原数据集:输入是英文描述,输出是SQL反转后:输入是SQL(examples["answer"]),输出是英文描述(examples["question"])
def switch_and_format_prompt(examples):
    inputs = examples["answer"] # 使用 answer(SQL) 作为输入
    context = examples["context"]
    outputs = examples["question"] # 使用 question(英文描述) 作为输出
    texts = []
    for input, context, output in zip(inputs, context, outputs):
        text = train_prompt_style.format(input, context, output) + EOS_TOKEN
        texts.append(text)
    return {
        "text": texts,
    }

# 应用转换
dataset = dataset.map(switch_and_format_prompt, batched = True)

model = FastLanguageModel.get_peft_model(
    model,
    r=16,   # LoRA的秩(rank)值,决定了低秩矩阵的维度,较大的r值(如16)可以提供更强的模型表达能力,但会增加参数量和计算开销,较小的r值(如4或8)则会减少参数量,但可能影响模型性能,通常在4-16之间选择,需要在性能和效率之间权衡
    target_modules=[ #指定需要应用LoRA微调的模块列表,q_proj, k_proj, v_proj: 注意力机制中的查询、键、值投影层
        "q_proj",
        "k_proj",
        "v_proj",
        "o_proj", #注意力输出投影层
        "gate_proj",
        "up_proj",
        "down_proj",
    ],
    lora_alpha=16, #缩放参数,用于控制LoRA更新的强度,通常设置为与r相同的值,较大的alpha会增加LoRA的影响力,较小的alpha则会减弱LoRA的影响
    lora_dropout=0, #LoRA层的dropout率,0表示不使用dropout,增加dropout可以帮助防止过拟合,但可能影响训练稳定性,在微调时通常设为0或很小的值
    bias="none", #是否微调偏置项,"none"表示不微调偏置参数,也可以设置为"all"或"lora_only"来微调不同范围的偏置
    use_gradient_checkpointing="unsloth",  # 梯度检查点策略,"unsloth"是一种优化的检查点策略,适用于长上下文可以显著减少显存使用,但会略微增加计算时间对处理长文本特别有用
    random_state=3407, #随机数种子,控制初始化的随机性,固定种子可以确保实验可重复性
    use_rslora=False, #是否使用RSLoRA(Rank-Stabilized LoRA) False表示使用标准LoRARSLoRA是一种改进的LoRA变体,可以提供更稳定的训练
    loftq_config=None, #LoftQ配置None表示不使用LoftQ量化LoftQ是一种用于模型量化的技术,可以减少模型大小
)

from trl import SFTTrainer
from transformers import TrainingArguments
from unsloth import is_bfloat16_supported

trainer = SFTTrainer(
    model=model,
    tokenizer=tokenizer,
    train_dataset=dataset,
    dataset_text_field="text",
    max_seq_length=max_seq_length,
    dataset_num_proc=2,
    ## 训练参数配置
    args=TrainingArguments(
        # 批处理相关
        per_device_train_batch_size=2, # 每个设备(GPU)的训练批次大小
        gradient_accumulation_steps=4,# 梯度累积步数,用于模拟更大的批次大小
         # 训练步数和预热
        warmup_steps=5,# 学习率预热步数,逐步增加学习率
        max_steps=60,# 最大训练步数
        learning_rate=2e-4,
        fp16=not is_bfloat16_supported(), # 如果不支持 bfloat16,则使用 float16
        bf16=is_bfloat16_supported(),# 如果支持则使用 bfloat16,通常在新型 GPU 上性能更好
        logging_steps=10,# 每10步记录一次日志
        optim="adamw_8bit", # 使用8位精度的 AdamW 优化器
        weight_decay=0.01,# 权重衰减率,用于防止过拟合
        lr_scheduler_type="linear",# 学习率调度器类型,使用线性衰减
        seed=3407,# 随机种子,确保实验可重复性
        output_dir="outputs", # 模型和检查点的输出目录
    ),
)

trainer_stats = trainer.train()

# 定义提示模板
# 这个模板包含了指导模型如何理解和解释SQL查询的结构化提示
prompt_style = """Below is an instruction that describes a task, paired with an input that provides further context.
Write a response that appropriately completes the request.
Before answering, think carefully about the question and create a step-by-step chain of thoughts to ensure a logical and accurate response.

### Instruction:
You are a SQL expert with advance understanding of SQL queries. You can understand database schema from the query. Think like sql expert and generate a summary of the query which explains the use case of the query. As in
what the query is trying to read from the database in a usecase sense.

### Query:
{}

### Response:
<think>{}"""

# 定义测试用的SQL查询
# 这是一个复杂的客户分析查询,用于测试模型的理解能力
query1 = """
SELECT
    c.customer_id,
    c.name AS customer_name,
    COUNT(o.order_id) AS total_orders,
    SUM(o.total_amount) AS total_spent,
    AVG(o.total_amount) AS avg_order_value,
    MAX(o.order_date) AS last_order_date
FROM customers c
JOIN orders o ON c.customer_id = o.customer_id
LEFT JOIN order_items oi ON o.order_id = oi.order_id
WHERE o.order_date >= '2024-01-01'
GROUP BY c.customer_id, c.name
ORDER BY total_spent DESC
LIMIT 10;
Explain use case of this query.
"""

# 将模型设置为推理模式
FastLanguageModel.for_inference(model)
# 准备输入数据
# 使用提示模板格式化查询,并转换为模型可处理的张量格式
inputs = tokenizer([prompt_style.format(query1, "")], return_tensors="pt").to("cuda")

# 生成响应
outputs = model.generate(
    input_ids=inputs.input_ids, # 输入的标记ID
    attention_mask=inputs.attention_mask, # 注意力掩码,用于处理填充
    max_new_tokens=1200, # 最大生成的新标记数
    use_cache=True,  # 使用缓存以提高生成速度
)
# 解码模型输出
# 使用分词器将输出转换回文本,并提取Response部分
response = tokenizer.batch_decode(outputs)
print(response[0].split("### Response:")[1])

#微调后:解释更加具体和准确 能更好地捕捉查询的完整上下文 例如在示例中,微调后的模型特别指出了"top 10 customers with the highest total spent"这个关键细节,这在微调前的响应中是缺失的

local_path="deepseek_sql_model"

model.save_pretrained(local_path)
tokenizer.save_pretrained(local_path)

# Save merged model
# model.save_pretrained_merged(local_path, tokenizer, save_method="merged_16bit")

model.push_to_hub(local_path)
tokenizer.push_to_hub(local_path)

from google.colab import drive
drive.mount('/content/drive')
drive_path = "/content/drive/MyDrive/deepseek_model"
model.save_pretrained(drive_path)
tokenizer.save_pretrained(drive_path)

Comments