当前位置： 首页 > news >正文

盐城建设厅网站设计备案长沙建设企业网站

news 2025/11/7 16:51:01
盐城建设厅网站设计备案,长沙建设企业网站,WordPress对段落美化,免费网站空间有什么用1.配置环境 2.数据集准备 3.模型下载 4.注册SwanLab 5.微调 6.训练过程可视化 1.配置环境 本博客使用的是2B模型，所以仅用了单卡3090，若大一点的模型，自行根据实际情况准备显卡 安装Python>3.8 安装Qwen2-VL必要的库 pip install…

1.配置环境

2.数据集准备

3.模型下载

4.注册SwanLab

5.微调

6.训练过程可视化

1.配置环境

本博客使用的是2B模型，所以仅用了单卡3090，若大一点的模型，自行根据实际情况准备显卡

安装Python>=3.8

安装Qwen2-VL必要的库

pip install modelscope==1.18.0

pip install transformers==4.46.2

pip install accelerate==1.1.1

pip install datasets==2.18.0

pip install peft==0.13.2

pip install qwen-vl-utils==0.0.8

2.数据集准备

本博客使用的是coco_2014_caption数据集中的部分，json格式如下：

数据集下载可以使用如下同样会产生一个csv：


from modelscope.msdatasets import MsDataset
import os
import pandas as pdMAX_DATA_NUMBER = 500if not os.path.exists('coco_2014_caption'):# 从modelscope下载COCO 2014图像描述数据集ds =  MsDataset.load('modelscope/coco_2014_caption', subset_name='coco_2014_caption', split='train')print(len(ds))total = min(MAX_DATA_NUMBER, len(ds))os.makedirs('coco_2014_caption', exist_ok=True)image_paths = []captions = []for i in range(total):# 获取每个样本的信息item = ds[i]image_id = item['image_id']caption = item['caption']image = item['image']# 保存图片并记录路径image_path = os.path.abspath(f'coco_2014_caption/{image_id}.jpg')image.save(image_path)# 将路径和描述添加到列表中image_paths.append(image_path)captions.append(caption)# 每处理50张图片打印一次进度if (i + 1) % 50 == 0:print(f'Processing {i+1}/{total} images ({(i+1)/total*100:.1f}%)')# 将图片路径和描述保存为CSV文件df = pd.DataFrame({'image_path': image_paths,'caption': captions})# 将数据保存为CSV文件df.to_csv('./coco-2024-dataset.csv', index=False)print(f'数据处理完成，共处理了{total}张图片')else:print('coco_2014_caption目录已存在,跳过数据处理步骤')

我们需要得到json，所以执行下面脚本得到：

import pandas as pd
import json# 载入CSV文件
df = pd.read_csv('./coco-2024-dataset.csv')
conversations = []# 添加对话数据
for i in range(len(df)):conversations.append({"id": f"identity_{i+1}","conversations": [{"from": "user","value": f"COCO Yes: <|vision_start|>{df.iloc[i]['image_path']}<|vision_end|>"},{"from": "assistant", "value": df.iloc[i]['caption']}]})# 保存为Json
with open('data_vl.json', 'w', encoding='utf-8') as f:json.dump(conversations, f, ensure_ascii=False, indent=2)

以上则是数据集准备，若使用自定义数据集则根据上面的格式准备！！！

3.模型下载

本博客使用魔塔社区中的Qwen2-VL-2B模型

from modelscope import snapshot_download, AutoTokenizer
from transformers import TrainingArguments, Trainer, DataCollatorForSeq2Seq, Qwen2VLForConditionalGeneration, AutoProcessor
import torchmodel_dir = snapshot_download("Qwen/Qwen2-VL-2B-Instruct", cache_dir="./", revision="master")
tokenizer = AutoTokenizer.from_pretrained("./Qwen/Qwen2-VL-2B-Instruct/", use_fast=False, trust_remote_code=True)
model = Qwen2VLForConditionalGeneration.from_pretrained("./Qwen/Qwen2-VL-2B-Instruct/", device_map="auto", torch_dtype=torch.bfloat16, trust_remote_code=True,)
model.enable_input_require_grads()

4.注册SwanLab

为了微调时随时查看各项指标，注册SwanLab,复制key,粘贴到运行过程中，如下图：

5.微调

微调前保证当前文件夹下面包含以下：

train.py代码如下：

import torch
from datasets import Dataset
from modelscope import snapshot_download, AutoTokenizer
from swanlab.integration.transformers import SwanLabCallback
from qwen_vl_utils import process_vision_info
from peft import LoraConfig, TaskType, get_peft_model, PeftModel
from transformers import (TrainingArguments,Trainer,DataCollatorForSeq2Seq,Qwen2VLForConditionalGeneration,AutoProcessor,
)
import swanlab
import jsondef process_func(example):"""将数据集进行预处理"""MAX_LENGTH = 8192input_ids, attention_mask, labels = [], [], []conversation = example["conversations"]input_content = conversation[0]["value"]output_content = conversation[1]["value"]file_path = input_content.split("<|vision_start|>")[1].split("<|vision_end|>")[0]  # 获取图像路径messages = [{"role": "user","content": [{"type": "image","image": f"{file_path}","resized_height": 280,"resized_width": 280,},{"type": "text", "text": "COCO Yes:"},],}]text = processor.apply_chat_template(messages, tokenize=False, add_generation_prompt=True)  # 获取文本image_inputs, video_inputs = process_vision_info(messages)  # 获取数据数据（预处理过）inputs = processor(text=[text],images=image_inputs,videos=video_inputs,padding=True,return_tensors="pt",)inputs = {key: value.tolist() for key, value in inputs.items()} #tensor -> list,为了方便拼接instruction = inputsresponse = tokenizer(f"{output_content}", add_special_tokens=False)input_ids = (instruction["input_ids"][0] + response["input_ids"] + [tokenizer.pad_token_id])attention_mask = instruction["attention_mask"][0] + response["attention_mask"] + [1]labels = ([-100] * len(instruction["input_ids"][0])+ response["input_ids"]+ [tokenizer.pad_token_id])if len(input_ids) > MAX_LENGTH:  # 做一个截断input_ids = input_ids[:MAX_LENGTH]attention_mask = attention_mask[:MAX_LENGTH]labels = labels[:MAX_LENGTH]input_ids = torch.tensor(input_ids)attention_mask = torch.tensor(attention_mask)labels = torch.tensor(labels)inputs['pixel_values'] = torch.tensor(inputs['pixel_values'])inputs['image_grid_thw'] = torch.tensor(inputs['image_grid_thw']).squeeze(0)  #由（1,h,w)变换为（h,w）return {"input_ids": input_ids, "attention_mask": attention_mask, "labels": labels,"pixel_values": inputs['pixel_values'], "image_grid_thw": inputs['image_grid_thw']}def predict(messages, model):# 准备推理text = processor.apply_chat_template(messages, tokenize=False, add_generation_prompt=True)image_inputs, video_inputs = process_vision_info(messages)inputs = processor(text=[text],images=image_inputs,videos=video_inputs,padding=True,return_tensors="pt",)inputs = inputs.to("cuda")# 生成输出generated_ids = model.generate(**inputs, max_new_tokens=128)generated_ids_trimmed = [out_ids[len(in_ids) :] for in_ids, out_ids in zip(inputs.input_ids, generated_ids)]output_text = processor.batch_decode(generated_ids_trimmed, skip_special_tokens=True, clean_up_tokenization_spaces=False)return output_text[0]# 在modelscope上下载Qwen2-VL模型到本地目录下
model_dir = snapshot_download("Qwen/Qwen2-VL-2B-Instruct", cache_dir="./", revision="master")# 使用Transformers加载模型权重
tokenizer = AutoTokenizer.from_pretrained("./Qwen/Qwen2-VL-2B-Instruct/", use_fast=False, trust_remote_code=True)
processor = AutoProcessor.from_pretrained("./Qwen/Qwen2-VL-2B-Instruct")model = Qwen2VLForConditionalGeneration.from_pretrained("./Qwen/Qwen2-VL-2B-Instruct/", device_map="auto", torch_dtype=torch.bfloat16, trust_remote_code=True,)
model.enable_input_require_grads()  # 开启梯度检查点时，要执行该方法# 处理数据集：读取json文件
# 拆分成训练集和测试集，保存为data_vl_train.json和data_vl_test.json
train_json_path = "data_vl.json"
with open(train_json_path, 'r') as f:data = json.load(f)train_data = data[:-4]test_data = data[-4:]with open("data_vl_train.json", "w") as f:json.dump(train_data, f)with open("data_vl_test.json", "w") as f:json.dump(test_data, f)train_ds = Dataset.from_json("data_vl_train.json")
train_dataset = train_ds.map(process_func)# 配置LoRA
config = LoraConfig(task_type=TaskType.CAUSAL_LM,target_modules=["q_proj", "k_proj", "v_proj", "o_proj", "gate_proj", "up_proj", "down_proj"],inference_mode=False,  # 训练模式r=64,  # Lora 秩lora_alpha=16,  # Lora alaph，具体作用参见 Lora 原理lora_dropout=0.05,  # Dropout 比例bias="none",
)# 获取LoRA模型
peft_model = get_peft_model(model, config)# 配置训练参数
args = TrainingArguments(output_dir="./output/Qwen2-VL-2B",per_device_train_batch_size=4,gradient_accumulation_steps=4,logging_steps=10,logging_first_step=5,num_train_epochs=2,save_steps=100,learning_rate=1e-4,save_on_each_node=True,gradient_checkpointing=True,report_to="none",
)# 设置SwanLab回调
swanlab_callback = SwanLabCallback(project="Qwen2-VL-finetune",experiment_name="qwen2-vl-coco2014",config={"model": "https://modelscope.cn/models/Qwen/Qwen2-VL-2B-Instruct","dataset": "https://modelscope.cn/datasets/modelscope/coco_2014_caption/quickstart","github": "https://github.com/datawhalechina/self-llm","prompt": "COCO Yes: ","train_data_number": len(train_data),"lora_rank": 64,"lora_alpha": 16,"lora_dropout": 0.1,},
)# 配置Trainer
trainer = Trainer(model=peft_model,args=args,train_dataset=train_dataset,data_collator=DataCollatorForSeq2Seq(tokenizer=tokenizer, padding=True),callbacks=[swanlab_callback],
)# 开启模型训练
trainer.train()# 配置测试参数
val_config = LoraConfig(task_type=TaskType.CAUSAL_LM,target_modules=["q_proj", "k_proj", "v_proj", "o_proj", "gate_proj", "up_proj", "down_proj"],inference_mode=True,  # 训练模式r=64,  # Lora 秩lora_alpha=16,  # Lora alaph，具体作用参见 Lora 原理lora_dropout=0.05,  # Dropout 比例bias="none",
)# 获取测试模型
val_peft_model = PeftModel.from_pretrained(model, model_id="./output/Qwen2-VL-2B/checkpoint-62", config=val_config)# 读取测试数据
with open("data_vl_test.json", "r") as f:test_dataset = json.load(f)test_image_list = []
for item in test_dataset:input_image_prompt = item["conversations"][0]["value"]# 去掉前后的<|vision_start|>和<|vision_end|>origin_image_path = input_image_prompt.split("<|vision_start|>")[1].split("<|vision_end|>")[0]messages = [{"role": "user", "content": [{"type": "image", "image": origin_image_path},{"type": "text","text": "COCO Yes:"}]}]response = predict(messages, val_peft_model)messages.append({"role": "assistant", "content": f"{response}"})print(messages[-1])test_image_list.append(swanlab.Image(origin_image_path, caption=response))swanlab.log({"Prediction": test_image_list})swanlab.finish()

6.训练过程可视化

http://www.yayakq.cn/news/143530/

相关文章：

  • 创建个网站需要多少钱网络科技公司起名大全免费
  • 免费特效素材网站北京seo排名外包
  • 怎么做网站数据分析手机价格
  • 面签拍照 网站备案临沂网站开发
  • 3g开发网站网站建设与管理试题与答案
  • 网站开发英文论文app大全
  • 一个网站需要多少钱wordpress定时发布的文章失效
  • 有可以做ssgsea的网站么社交网站页面设计
  • 公众号建网站wordpress wp-json
  • 地方房产网站APP如何做陕西网站建设电话
  • 宜宾团购网站建设魏县企业做网站推广
  • 怎么做原创动漫视频网站太钢建设公司官网
  • 电子商务网站有哪些?乐清柳市广电网站
  • 花都有做网站微信公众账号开发
  • 朗读者外国人做的汉字网站北京网站建设销售招聘
  • 大学生做兼职的网站有哪些wordpress 一级目录
  • 网站阿里云备案要多久电脑网页打不开建设银行网站
  • 德州定制网站建设公司网站制作的软件
  • 怎样弄网站金利福珠宝的网站建设理念
  • xxx美食网站建设规划书wordpress 微信 代码
  • 网站设计的步骤兴化建设局网站
  • 洗化行业做网站广西建设工程质量安全监督网站
  • 桐乡市建设局网站大通酩悦服装公司网站
  • 中小型网站建设新闻网站管理规章制度
  • 中国建设银行支付网站营销型网站建设申请域名时公司类型的域名后缀一般是
  • 汽车建设网站开发流程安阳县属于哪个省哪个市
  • 国外主流网站开发技术深圳做英文网站
  • 建行官网官网网站吗发布网站制作
  • 网站建设的图片尺寸应该是像素自助建站公司
  • 建设部官方网站查询网站建设贵吗