书生大模型实战营-L2-InternVL微调实践

Oct 2, 2024

本节任务要点#

follow 教学文档和视频使用QLoRA进行微调模型，复现微调效果，并能成功讲出梗图.

实践流程#

准备InternVL模型#

我们使用InternVL2-2B模型。该模型已在share文件夹下挂载好，现在让我们把移动出来。

1
mkdir -p /root/project/joke/model
2
cp -r /root/share/new_models/OpenGVLab/InternVL2-2B /root/project/joke/model
3

4
# 不用ln -s

准备环境#

这里我们来手动配置下xtuner。

配置虚拟环境，安装xtuner，之前安装过了是0.1.21的，现在要安装0.1.23的

1
conda create --name xtuner python=3.10 -y
2

3
# 激活虚拟环境（注意：后续的所有操作都需要在这个虚拟环境中进行）
4
conda activate xtuner
5

6
# 安装一些必要的库
7
conda install pytorch==2.1.2 torchvision==0.16.2 torchaudio==2.1.2 pytorch-cuda=12.1 -c pytorch -c nvidia -y
8
# 安装其他依赖
9
apt install libaio-dev
10
pip install transformers==4.39.3
11
pip install streamlit==1.36.0
12

13
cd /root/project/joke/code
14

15
git clone -b v0.1.23  https://github.com/InternLM/XTuner
16

17
cd XTuner
18
pip install -e '.[deepspeed]'
19

20
pip install lmdeploy==0.5.3 datasets matplotlib Pillow timm
21

22
xtuner version

数据集：huggingface上的zhongshsh/CLoT-Oogiri-GO

1
# 把数据集挪出来
2
ln -s /root/share/new_models/datasets/CLoT_cn_2000 /root/project/joke/datasets

InternVL 推理部署攻略#

之后我们使用lmdeploy自带的pipeline工具进行开箱即用的推理流程，首先我们新建一个文件。

1
touch /root/project/joke/code/test_lmdeploy.py

然后把以下代码拷贝进test_lmdeploy.py中。

1
from lmdeploy import pipeline
2
from lmdeploy.vl import load_image
3

4
pipe = pipeline('/root/model/InternVL2-2B')
5

6
image = load_image('/root/InternLM/007aPnLRgy1hb39z0im50j30ci0el0wm.jpg')
7
response = pipe(('请你根据这张图片，讲一个脑洞大开的梗', image))
8
print(response.text)

运行执行推理结果。

1
python /root/project/joke/code/test_lmdeploy.py

推理后我们发现直接使用2b模型不能很好的讲出梗，现在我们要对这个2b模型进行微调。

1
这张图片展现了一群绵羊在挤在一起的情景，但在这群绵羊中间，
2
却有一个非常显眼的鸟类。这只鸟的羽毛是黑色和黄色相间，它
3
站立在绵羊之间，显得非常突出。
4

5
这种对比形成的搞笑效果，常被称为“鸟的奇迹”（The Bird of
6
the Week）。这个梗来源于这样一个现象：当一只鸟出现在一群
7
绵羊中间时，往往会引起绵羊的关注，甚至有些会试图去接近这
8
只鸟。这种场景在现实生活中并不常见，因此很容易引发人们的
9
联想和笑料。
10

11
这个梗通常用来形容那些在人群中显得特别突出或特别引人注目
12
的人或事物。例如，在人群中突然出现了一个特别搞笑或特别有
13
特点的人，或者一个特别的物体，比如一只鸟。

InternVL 微调攻略#

准备数据集

datasets准备好了

配置微调参数

修改XTuner下 InternVL的config

/root/project/joke/code/XTuner/xtuner/configs/internvl/v2/internvl_v2_internlm2_2b_qlora_finetune.py

1
# Copyright (c) OpenMMLab. All rights reserved.
2
from mmengine.hooks import (CheckpointHook, DistSamplerSeedHook, IterTimerHook,
3
                            LoggerHook, ParamSchedulerHook)
4
from mmengine.optim import AmpOptimWrapper, CosineAnnealingLR, LinearLR
5
from peft import LoraConfig
6
from torch.optim import AdamW
7
from transformers import AutoTokenizer
8

9
from xtuner.dataset import InternVL_V1_5_Dataset
10
from xtuner.dataset.collate_fns import default_collate_fn
11
from xtuner.dataset.samplers import LengthGroupedSampler
12
from xtuner.engine.hooks import DatasetInfoHook
13
from xtuner.engine.runner import TrainLoop
14
from xtuner.model import InternVL_V1_5
15
from xtuner.utils import PROMPT_TEMPLATE
16

17
#######################################################################
18
#                          PART 1  Settings                           #
19
#######################################################################
20
# Model
21
path = '/root/project/joke/model/InternVL2-2B'
22

23
# Data
24
data_root = '/root/project/joke/datasets/CLoT_cn_2000/'
25
data_path = data_root + 'ex_cn.json'
26
image_folder = data_root
27
prompt_template = PROMPT_TEMPLATE.internlm2_chat
28
max_length = 6656
29

30
# Scheduler & Optimizer
31
batch_size = 4  # per_device
32
accumulative_counts = 4
33
dataloader_num_workers = 4
34
max_epochs = 6
35
optim_type = AdamW
36
# official 1024 -> 4e-5
37
lr = 2e-5
38
betas = (0.9, 0.999)
39
weight_decay = 0.05
40
max_norm = 1  # grad clip
41
warmup_ratio = 0.03
42

43
# Save
44
save_steps = 1000
45
save_total_limit = 1  # Maximum checkpoints to keep (-1 means unlimited)
46

47
#######################################################################
48
#            PART 2  Model & Tokenizer & Image Processor              #
49
#######################################################################
50
model = dict(
51
    type=InternVL_V1_5,
52
    model_path=path,
53
    freeze_llm=True,
54
    freeze_visual_encoder=True,
55
    quantization_llm=True,  # or False
56
    quantization_vit=False,  # or True and uncomment visual_encoder_lora
57
    # comment the following lines if you don't want to use Lora in llm
58
    llm_lora=dict(
59
        type=LoraConfig,
60
        r=128,
61
        lora_alpha=256,
62
        lora_dropout=0.05,
63
        target_modules=None,
64
        task_type='CAUSAL_LM'),
65
    # uncomment the following lines if you don't want to use Lora in visual encoder # noqa
66
    # visual_encoder_lora=dict(
67
    #     type=LoraConfig, r=64, lora_alpha=16, lora_dropout=0.05,
68
    #     target_modules=['attn.qkv', 'attn.proj', 'mlp.fc1', 'mlp.fc2'])
69
)
70

71
#######################################################################
72
#                      PART 3  Dataset & Dataloader                   #
73
#######################################################################
74
llava_dataset = dict(
75
    type=InternVL_V1_5_Dataset,
76
    model_path=path,
77
    data_paths=data_path,
78
    image_folders=image_folder,
79
    template=prompt_template,
80
    max_length=max_length)
81

82
train_dataloader = dict(
83
    batch_size=batch_size,
84
    num_workers=dataloader_num_workers,
85
    dataset=llava_dataset,
86
    sampler=dict(
87
        type=LengthGroupedSampler,
88
        length_property='modality_length',
89
        per_device_batch_size=batch_size * accumulative_counts),
90
    collate_fn=dict(type=default_collate_fn))
91

92
#######################################################################
93
#                    PART 4  Scheduler & Optimizer                    #
94
#######################################################################
95
# optimizer
96
optim_wrapper = dict(
97
    type=AmpOptimWrapper,
98
    optimizer=dict(
99
        type=optim_type, lr=lr, betas=betas, weight_decay=weight_decay),
100
    clip_grad=dict(max_norm=max_norm, error_if_nonfinite=False),
101
    accumulative_counts=accumulative_counts,
102
    loss_scale='dynamic',
103
    dtype='float16')
104

105
# learning policy
106
# More information: https://github.com/open-mmlab/mmengine/blob/main/docs/en/tutorials/param_scheduler.md  # noqa: E501
107
param_scheduler = [
108
    dict(
109
        type=LinearLR,
110
        start_factor=1e-5,
111
        by_epoch=True,
112
        begin=0,
113
        end=warmup_ratio * max_epochs,
114
        convert_to_iter_based=True),
115
    dict(
116
        type=CosineAnnealingLR,
117
        eta_min=0.0,
118
        by_epoch=True,
119
        begin=warmup_ratio * max_epochs,
120
        end=max_epochs,
121
        convert_to_iter_based=True)
122
]
123

124
# train, val, test setting
125
train_cfg = dict(type=TrainLoop, max_epochs=max_epochs)
126

127
#######################################################################
128
#                           PART 5  Runtime                           #
129
#######################################################################
130
# Log the dialogue periodically during the training process, optional
131
tokenizer = dict(
132
    type=AutoTokenizer.from_pretrained,
133
    pretrained_model_name_or_path=path,
134
    trust_remote_code=True)
135

136
custom_hooks = [
137
    dict(type=DatasetInfoHook, tokenizer=tokenizer),
138
]
139

140
# configure default hooks
141
default_hooks = dict(
142
    # record the time of every iteration.
143
    timer=dict(type=IterTimerHook),
144
    # print log every 10 iterations.
145
    logger=dict(type=LoggerHook, log_metric_by_epoch=False, interval=10),
146
    # enable the parameter scheduler.
147
    param_scheduler=dict(type=ParamSchedulerHook),
148
    # save checkpoint per `save_steps`.
149
    checkpoint=dict(
150
        type=CheckpointHook,
151
        save_optimizer=False,
152
        by_epoch=False,
153
        interval=save_steps,
154
        max_keep_ckpts=save_total_limit),
155
    # set sampler seed in distributed evrionment.
156
    sampler_seed=dict(type=DistSamplerSeedHook),
157
)
158

159
# configure environment
160
env_cfg = dict(
161
    # whether to enable cudnn benchmark
162
    cudnn_benchmark=False,
163
    # set multi process parameters
164
    mp_cfg=dict(mp_start_method='fork', opencv_num_threads=0),
165
    # set distributed parameters
166
    dist_cfg=dict(backend='nccl'),
167
)
168

169
# set visualizer
170
visualizer = None
171

172
# set log level
173
log_level = 'INFO'
174

175
# load from which checkpoint
176
load_from = None
177

178
# whether to resume training from the loaded checkpoint
179
resume = False
180

181
# Defaults to use random seed and disable `deterministic`
182
randomness = dict(seed=None, deterministic=False)
183

184
# set log processor
185
log_processor = dict(by_epoch=False)

训练

1
conda activate xtuner
2
NPROC_PER_NODE=1 xtuner train \
3
  /root/project/joke/code/XTuner/xtuner/configs/internvl/v2/internvl_v2_internlm2_2b_qlora_finetune.py \
4
  --work-dir /root/project/joke/code/work_dir/internvl_ft_run_8_filter \
5
    --deepspeed deepspeed_zero1

合并与转换权重

1
cd /root/project/joke/code/XTuner
2

3
python xtuner/configs/internvl/v1_5/convert_to_official.py \
4
  xtuner/configs/internvl/v2/internvl_v2_internlm2_2b_qlora_finetune.py \
5
  ../work_dir/internvl_ft_run_8_filter/iter_3000.pth \
6
  ../../model/InternVL2-2B

微调后效果对比#

运行前面的test_lmdeploy.py

1
python /root/project/joke/code/test_lmdeploy.py

感觉自己好冷

总结#

数据很重要

Author Junyao Hu

Published Oct 2, 2024

Link https://junyaohu.github.io/blog/internlm-l2-internvl/