main.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {
    "jupyter": {
     "outputs_hidden": false
    }
   },
   "source": [
    "去水印模型冲刺赛\n",
    "\n",
    "# 百度网盘AI大赛-图像处理挑战赛：去水印模型冲刺赛\n",
    "\n",
    "# 一、比赛介绍\n",
    "\n",
    "为了帮助人们消除带有水印的图片日常生活中带有水印的图片很常见，即使是PS专家，也很难快速且不留痕迹的去除水印。在上次比赛中我们举办了去水印模型大赛，本次我们提高了难度，提高了在水印种类的广泛性，同时在模型性能上做了更高的要求，向各位选手发起挑战。\n",
    "\n",
    "# 二、评价标准\n",
    "\n",
    "- 客观评价指标为 PSNR 和 MSSSIM；\n",
    "- 用于评价的机器环境仅提供两种框架模型运行环境：paddlepaddle 和 onnxruntime，其他框架模型可转换为上述两种框架的模型；\n",
    "- 机器配置：V100，显存15G，内存10G；\n",
    "- 单张图片耗时>1.2s，决赛中的性能分数记0分。\n",
    "\n",
    "因此，应尽可能不能使用过大的模型。\n",
    "\n",
    "# 三、改进和优化的之处\n",
    "本方案对比baseline有很多优化之处：\n",
    "-对训练数据集做数据增强，对测试数据也做了一定的增强\n",
    "- 选取背景色彩复杂、前景水印易混淆导致的恢复效果较差的难样本，专门进行 finetune；\n",
    "- 使用多模集成（如果不考量参数）、每次热重启时新增可训练参数等；\n",
    "- 在对比学习损失上探索其他 encoder 及其不同层的 embedding 输出、使用更多的 negative samples 等；\n",
    "- 探索模型压缩相关技术（剪枝、量化、蒸馏等）；\n",
    "- 探索使用在线添加更丰富多样的水印。\n",
    "\n",
    "# 四、网络构造\n",
    "\n",
    "- 本项目所设计的网络基于 PMRID UNet，对水印图像进行像素级回归预测和转换。其核心在于使用了层次化和逆瓶颈的深度可分离卷积操作，在确保模型大小和推理速度符合要求的前提下，有效改善了模型容量不足所带来的精度问题。架构如下所示（来源：https://github.com/yuezewang/2022-MegCup-RawImageDenoising/blob/main/main.py\n",
    "）：\n",
    "\n",
    "![](https://ai-studio-static-online.cdn.bcebos.com/9592decbd46140d691ef5aafe2dff447dd4dce67e8e0485daa69bb772f4a93b2)\n",
    "\n",
    "\n",
    "\n",
    "\n",
    "\n",
    "\n",
    "\n",
    "\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    " 同时，本项目提供了最终的模型参数文件 **model.pdiparams, model.pdiparams.info, model.pdmodel** 用于复现。"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### 数据解压\n",
    "\n",
    "- 执行如下命令解压本项目压缩好的部分数据集及其标签到 Notebook 所在目录："
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "scrolled": true,
    "tags": []
   },
   "outputs": [],
   "source": [
    "# 创建目录 - 若目录已存在则无需运行\n",
    "! mkdir targets  # 创建标签目录\n",
    "! mkdir samples  # 创建样本目录"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "scrolled": true
   },
   "outputs": [],
   "source": [
    "# 解压 标签/目标/背景图片 到 target 目录下"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "scrolled": true
   },
   "outputs": [],
   "source": [
    "# 检查\n",
    "! cd targets ; tree"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "scrolled": true
   },
   "outputs": [],
   "source": [
    "# 解压第 8 部分训练集到 sample 目录下\n",
    "! tar -xvf data/data141714/watermark_datasets.part8.tar -C ./samples"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "scrolled": true
   },
   "outputs": [],
   "source": [
    "# 检查\n",
    "! cd samples ; tree"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### 依赖导入与参数设置\n",
    "\n",
    "- 调试数据和模型时，令 **DEBUG=True** 以使用很少数据测验；使用全数据训练正常训练时，令 **DEBUG=False**。\n",
    "- 通过设置 **STAGE** 为 **[0, 1, 2, 3, 4]** 中的整数来决定当前训练阶段及其部分超参数。若在当前的部分数据集环境下运行，请令 **STAGE=0** 用于示范；若在本地的全数据集环境下实施训练，则 **STAGE 不再包含 0**。\n",
    "- 手动设置 **MODEL_LOAD_PATH** 的路径将指定模型用于 STAGE 为 **[2, 3, 4]** 阶段的重启训练。例如，设 STAGE=1 时，**验证 Score 最高** 的模型为 model_stage1_epoch8_step500.pdparams，则 STAGE=2 时，令 MODEL_LOAD_PATH = './model_stage1_epoch8_step500.pdparams' 即可在其基础上渐进优化。\n",
    "- 为提升性能，可选取当前 STAGE 的 **3-5** 个模型权重实施 **EMA** (详见 **ema.py**) 来融合模型，再用于下一 STAGE 训练。\n",
    "- 在 **STAGE=4** 时，默认使用的 L1Loss 也能得到较好的结果。若要使用 **对比学习损失** ，请下载 **[VGG19权重链接](https://pan.baidu.com/s/19wJRGS-n7e_LktwHBpkG5w?pwd=b1xq)** 并设法放置到当前目录下 (AIStudio 限制 150MB 上传文件大小)，然后保持 CLLOSS=True。"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "scrolled": true,
    "tags": []
   },
   "outputs": [],
   "source": [
    "# -------------------------------------------------------------------------------------------------------------------\n",
    "# 依赖导入\n",
    "# -------------------------------------------------------------------------------------------------------------------\n",
    "\n",
    "import gc\n",
    "import os\n",
    "import sys\n",
    "import time\n",
    "import math\n",
    "import glob\n",
    "import random\n",
    "\n",
    "import cv2\n",
    "import numpy as np\n",
    "import pandas as pd\n",
    "from PIL import Image\n",
    "\n",
    "import paddle\n",
    "import paddle.nn as nn\n",
    "import paddle.nn.functional as F\n",
    "\n",
    "from network import Net\n",
    "from metrics import ContrastLoss, MS_SSIM, PSNR\n",
    "\n",
    "import warnings \n",
    "warnings.filterwarnings('ignore')\n",
    "\n",
    "if paddle.is_compiled_with_cuda():\n",
    "    paddle.set_device('gpu:0')\n",
    "else:\n",
    "    paddle.set_device('cpu')\n",
    "    \n",
    "# -------------------------------------------------------------------------------------------------------------------\n",
    "# 部分数据集下的全局参数与路径设置（全数据集的设置详见 train.py）\n",
    "# -------------------------------------------------------------------------------------------------------------------\n",
    "\n",
    "# DEBUG = True 则使用很少样本实施 Debug 试验；DEBUG = False 则使用给定数据实施正常训练 ================================\n",
    "DEBUG = False  # True #  \n",
    "\n",
    "DEBUG_TRAIN_SAMPLES = 16  \t # 用于 Debug 试验的训练集样本数\n",
    "DEBUG_VALID_SAMPLES = 16  \t # 用于 Debug 试验的验证集样本数\n",
    "\n",
    "BG_SAMPLES = 1448         \t # 训练集背景样本索引划分阈值 (Target)\n",
    "VALID_SAMPLES_INDEX = 11  \t # 验证集样本采样间隔\n",
    "\n",
    "# 实施 4 阶段渐进训练, STAGE 为不同训练阶段标志\n",
    "STAGE = 0  # 1, 2, 3, 4\n",
    "\n",
    "# 第 4 阶段 (精细微调阶段)\n",
    "if STAGE == 4:\n",
    "    BS = 1        \t\t     # 训练集批大小 Batch Size\n",
    "    LR = 5e-5     \t\t     # 学习率 Learn Rate\n",
    "    EPOCH = 1     \t\t     # 训练轮数 Epoch\n",
    "    SIZE = 896    \t\t     # 训练集图像尺寸 Image Size\n",
    "    TRAIN_SAMPLES_INDEX = 1  # 训练集集样本采样间隔\n",
    "    CLLOSS = True\n",
    "    PRINT = 4000  \t\t     # 验证迭代步数间隔\n",
    "    \n",
    "# 第 3 阶段\n",
    "elif STAGE == 3:\t\n",
    "    BS = 2\n",
    "    LR = 1e-4\n",
    "    EPOCH = 2\n",
    "    SIZE = 768 \n",
    "    TRAIN_SAMPLES_INDEX = 2  \n",
    "    CLLOSS = False\n",
    "    PRINT = 2000\n",
    "\t\n",
    "# 第 2 阶段\n",
    "elif STAGE == 2:\n",
    "    BS = 4\n",
    "    LR = 5e-4\n",
    "    EPOCH = 4\n",
    "    SIZE = 640\n",
    "    TRAIN_SAMPLES_INDEX = 4 \n",
    "    CLLOSS = False\n",
    "    PRINT = 1000\n",
    "\n",
    "# 第 1 阶段 (粗放收敛阶段)\n",
    "elif STAGE == 1:\n",
    "    BS = 8\n",
    "    LR = 1e-3\n",
    "    EPOCH = 8\n",
    "    SIZE = 512\n",
    "    TRAIN_SAMPLES_INDEX = 8\n",
    "    CLLOSS = False\n",
    "    PRINT = 500\n",
    "\n",
    "# 第 0 阶段 (本项目 Notebook 用于部分数据集的示范阶段 =====================================================================)\n",
    "else:\n",
    "    BS = 8\n",
    "    LR = 1e-3\n",
    "    EPOCH = 2\n",
    "    SIZE = 512\n",
    "    TRAIN_SAMPLES_INDEX = 1\n",
    "    CLLOSS = False\n",
    "    PRINT = 100\n",
    "\n",
    "print(f\"=======================> stage {STAGE} <=======================\")\n",
    "\n",
    "# 固定随机数种子\n",
    "SEED = 42\n",
    "paddle.seed(SEED)\n",
    "random.seed(SEED)\n",
    "np.random.seed(SEED)\n",
    "os.environ['PYTHONHASHSEED'] = str(SEED)\n",
    "\n",
    "# 训练集路径\n",
    "TRAIN_SAMPLES_PATH = \"./samples/\"  \t\t     # 训练样本图片路径\n",
    "TRAIN_TARGETS_PATH = \"./targets/bg_images/\"  # 训练标签图片路径\n",
    "\n",
    "BUILD_SAMPLES_DF = True # False  # 只需执行一次, 生成 df_all.csv 后改成 False 即可\n",
    "ALL_SAMPLES_DF_PATH = './df_all.csv'\n",
    "TRAIN_SAMPLES_DF_PATH = f\"./df_train_stage{STAGE}.csv\"  # 训练样本 DataFrame 路径\n",
    "VALID_SAMPLES_DF_PATH = f\"./df_valid_stage{STAGE}.csv\"  # 验证样本 DataFrame 路径\n",
    "\n",
    "MODEL_LOAD_PATH = ''  # 用于 finetune 的前一 stage 模型路径"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# 五、模型训练\n",
    "\n",
    "1. 训练时数据增强\n",
    "- 水平翻转：为避免模型过度拟合具有相同水印模式（特别是方向），在训练时实施 50% 概率的水平翻转。\n",
    "- 循环填充：使用循环填充确保输入样本及其目标图片的最短边边长不小于预设尺寸，同时保持水印模式一致。\n",
    "- 随机裁剪：为避免放缩失真和显存不足，并增强模型对不同图像局部的泛化能力，按预设尺寸随机裁剪样本及其目标。\n",
    "\n",
    "![](https://ai-studio-static-online.cdn.bcebos.com/011e6e7509e844c4892bddfac090e2345afd344cccea46c3ae24d23a88c16402)\n",
    "\n",
    "\n",
    "2. 训练验证集处理\n",
    "\n",
    "- 训练集：将背景图片编号小于 1600 的所有水印图像作为候选训练集，通过调整等距抽样间隔获取子集作为不同阶段的训练集。\n",
    "- 验证集：将候选训练集外的所有水印图像作为候选验证集。综合验证效果、时间和显存占用，通过等距抽样获取大小约 2000 的子集作为验证集，同时令 50% 的图像水平翻转，以便于衡量模型对不同水印模式的繁花能力，与训练时的水平翻转相呼应。\n",
    "\n",
    "```\n",
    "class MyDataset(paddle.io.Dataset):\n",
    "    def __init__(self, df, train=False):\n",
    "        super(MyDataset, self).__init__()\n",
    "\n",
    "        self.df = df\n",
    "        self.train = train  # 训练/验证模式下, 采用不同的数据预处理策略\n",
    "        print(f\"nums data set: {len(self.df)}\")\n",
    "\n",
    "    def __getitem__(self, index):\n",
    "\n",
    "        # 读取训练样本及其标签\n",
    "        image = cv2.imread(self.df.iloc[index]['sample'])\n",
    "        label = cv2.imread(self.df.iloc[index]['target'])\n",
    "        image = cv2.cvtColor(image, cv2.COLOR_BGR2RGB)\n",
    "        label = cv2.cvtColor(label, cv2.COLOR_BGR2RGB)\n",
    "\n",
    "        # 归一化\n",
    "        image = image / 255.\n",
    "        label = label / 255.\n",
    "\n",
    "        # 训练时数据预处理策略\n",
    "        if self.train == True:\n",
    "            h, w, c = image.shape\n",
    "\n",
    "            # 50% 概率训练样本水平翻转, 增强模型对不同 pattern 的泛化性能\n",
    "            if np.random.randint(0, 2) == 1:\n",
    "                image = paddle.vision.transforms.hflip(image)\n",
    "                label = paddle.vision.transforms.hflip(label)\n",
    "\n",
    "            # H, W, C -> C, H, W\n",
    "            image = image.transpose((2,0,1))\n",
    "            label = label.transpose((2,0,1))\n",
    "\n",
    "            # C, H, W -> 1, C, H, W   &    np.ndarray -> paddle.tensor\n",
    "            image = paddle.to_tensor(image[np.newaxis, ...], dtype='float32')\n",
    "            label = paddle.to_tensor(label[np.newaxis, ...], dtype='float32')\n",
    "\n",
    "            # 将训练样本及其标签循环 padding 至最短边不小于 SIZE\n",
    "            pad = nn.Pad2D(padding=[0, max(SIZE-w, 0), 0, max(SIZE-h, 0)], mode='circular')  # padding [pad_left, pad_right, pad_top, pad_bottom]\n",
    "            image = pad(image)\n",
    "            label = pad(label)\n",
    "\n",
    "            # 随机裁剪尺寸为 (SIZE, SIZE) 的 patch 用于训练\n",
    "            h_new = np.random.randint(0, image.shape[-2]-SIZE+1)\n",
    "            w_new = np.random.randint(0, image.shape[-1]-SIZE+1)\n",
    "            image = image[..., h_new:h_new+SIZE, w_new:w_new+SIZE].squeeze(0)\n",
    "            label = label[..., h_new:h_new+SIZE, w_new:w_new+SIZE].squeeze(0)\n",
    "\n",
    "        # 验证时数据预处理策略\n",
    "        else:\n",
    "            # 50% 验证样本水平翻转, 考察模型对不同 pattern 的泛化性能\n",
    "            if index % 2 == 0:\n",
    "                image = paddle.vision.transforms.hflip(image)\n",
    "                label = paddle.vision.transforms.hflip(label)\n",
    "\n",
    "            # H, W, C -> C, H, W\n",
    "            image = image.transpose((2,0,1))\n",
    "            label = label.transpose((2,0,1))\n",
    "\n",
    "            # np.ndarray -> paddle.tensor\n",
    "            image = paddle.to_tensor(image, dtype='float32')\n",
    "            label = paddle.to_tensor(label, dtype='float32')\n",
    "\n",
    "        return image, label\n",
    "\n",
    "    def __len__(self):\n",
    "        return len(self.df)\n",
    "```\n",
    "\n",
    "3. 基于验证 Score 的指数移动平均\n",
    "\n",
    "- 由于单模鲁棒性不足，且常规指数移动平均 (Exponential Moving Average, EMA) 根据时间远近对模型参数指数加权，缺乏针对性。为此，可以使用基于验证 Score 的指数移动平均 (Validation Score based EMA) 来对 3-5 个前一阶段的最佳模型参数实施指数加权，从而有针对性地提升模型效果。\n",
    "\n",
    "```\n",
    "def EMA(model, ema_model_path, model_path_list):\n",
    "\n",
    "    ema_model = copy.deepcopy(model)\n",
    "    ema_n = 0\n",
    "\n",
    "    with paddle.no_grad():\n",
    "        for _ckpt in model_path_list:\n",
    "            model.load_dict(paddle.load(_ckpt))  # , map_location=torch.device('cpu')\n",
    "            tmp_para_dict = dict(model.named_parameters())\n",
    "            alpha = 1. / (ema_n + 1.)\n",
    "            for name, para in ema_model.named_parameters():\n",
    "                new_para = tmp_para_dict[name].clone() * alpha + para.clone() * (1. - alpha)\n",
    "                para.set_value(new_para.clone())\n",
    "            ema_n += 1\n",
    "\n",
    "    paddle.save(ema_model.state_dict(), ema_model_path)\n",
    "    print('ema finished !!!')\n",
    "\n",
    "    return ema_model\n",
    "```\n",
    "\n",
    "\n",
    "4. 多阶段热重启渐进收敛\n",
    "\n",
    "- 用大量相似数据训练小模型易导致过拟合，且受限于图像尺寸等超参数，直接单次端到端训练不易兼顾数据利用和模型收敛。可以通过多阶段训练实现模型渐进收敛，即：每阶段微调上一阶段的最佳模型，渐进重启学习率、增大图像尺寸、减少训练周期和批大小，从而令模型由粗到精地学习和收敛。\n",
    "\n",
    "![](https://ai-studio-static-online.cdn.bcebos.com/77de3d2fb8e14b5f945aed69967b0709f55f615cf1354984a2ee625dbbc25910)\n",
    "\n",
    "5. 对比学习损失\n",
    "\n",
    "- 通过计算对比学习损失，在基于 VGG19 的特征嵌入空间中，缩小模型预测结果 (Anchor) 与作为目标的背景图片 (Positive) 的特征嵌入的 L1 距离，拉大模型预测结果 (Anchor) 与输入水印图像 (Negative) 的特征嵌入的 L1 距离。\n",
    "\n",
    "- 设 Batch size 为 N，使用 VGG19 的 M 层特征嵌入 Φ(.) ，每次计算使用 1 个目标图片 (Positive) GT、1 个重建输出 (Anchor) O、K 个具有相同背景的水印图片 (Negative) O_Neg，则对比学习损失 L_CL 可表示为：\n",
    "\n",
    "![](https://ai-studio-static-online.cdn.bcebos.com/f2a6d31a782b4d6dba483a0479779afc12f1f99bb8224ca585114b81ad37c42e)\n",
    "\n",
    "```\n",
    "class ContrastLoss(nn.Layer):\n",
    "    def __init__(self, ablation=False):\n",
    "        super(ContrastLoss, self).__init__()\n",
    "\n",
    "        self.vgg = VGGExtractor(pretrained=\"./vgg.pdparams\")  # VGG feature extractor\n",
    "        self.l1 = paddle.nn.loss.L1Loss()  # L1Loss\n",
    "        self.weights = [1.0/32, 1.0/16, 1.0/8, 1.0/4, 1.0]  # weights of features at different layers\n",
    "        self.ab = ablation\n",
    "\n",
    "    def forward(self, a, p, n):\n",
    "        # anchor samples, positive samples, negative samples\n",
    "        a_vgg, p_vgg, n_vgg = self.vgg(a), self.vgg(p), self.vgg(n)\n",
    "        loss = 0\n",
    "        d_ap, d_an = 0, 0\n",
    "        for i in range(len(a_vgg)):\n",
    "            d_ap = self.l1(a_vgg[i], p_vgg[i].detach())\n",
    "            if not self.ab:\n",
    "                d_an = self.l1(a_vgg[i], n_vgg[i].detach())\n",
    "                contrastive = d_ap / (d_an + 1e-7)\n",
    "            else:\n",
    "                contrastive = d_ap\n",
    "            loss += self.weights[i] * contrastive\n",
    "        return loss\n",
    "```\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### 训练验证集划分 \n",
    "\n",
    "- 在部分训练集 part8 下，背景图片索引小于 BG_SAMPLES = 1448 的作为训练集，其余部分等距抽样作为验证集：\n",
    "\n",
    "- 在完全训练集下，背景图片索引小于 BG_SAMPLES = 1600 的作为训练集，其余部分等距抽样作为验证集："
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "scrolled": true,
    "tags": []
   },
   "outputs": [],
   "source": [
    "# ---------------------------------------------------------------------------------------------------------------------------------\n",
    "# 训练-验证集划分、采样与加载\n",
    "# ---------------------------------------------------------------------------------------------------------------------------------\n",
    "\n",
    "if BUILD_SAMPLES_DF:\n",
    "    data = pd.DataFrame({'sample': glob.glob(TRAIN_SAMPLES_PATH + \"*/*\")})\n",
    "    data['target'] = data['sample'].apply(lambda x: TRAIN_TARGETS_PATH + x.split('/')[-1][:14] + \".jpg\")\n",
    "    data['idx'] = data['sample'].apply(lambda x: x.split('/')[-1][9:14])\n",
    "    data = data.sort_values(by=['target']).reset_index(drop=True)\n",
    "    data = data.drop(data.tail(1).index)\n",
    "    data.to_csv(ALL_SAMPLES_DF_PATH, index=False)\n",
    "    data_nums = len(data) \n",
    "    print(data_nums)  # part8：101254\n",
    "\n",
    "    data = pd.read_csv(ALL_SAMPLES_DF_PATH)\n",
    "    print(data.iloc[0]['idx'], data.iloc[-1]['idx'])  # 1288  1472\n",
    "\n",
    "    sample_train = []\n",
    "    sample_valid = []\n",
    "    for i in range(len(data)):\n",
    "        cur = data.iloc[i]['idx']\n",
    "        if (cur >= BG_SAMPLES) and i % VALID_SAMPLES_INDEX == 0:\n",
    "            sample_valid.append(data.iloc[i]['sample'])\n",
    "            # print(len(sample_valid))\n",
    "        elif (cur < BG_SAMPLES) and i % TRAIN_SAMPLES_INDEX == 0:\n",
    "            sample_train.append(data.iloc[i]['sample'])\n",
    "\n",
    "    df_train = pd.DataFrame()\n",
    "    df_train['sample'] = sample_train\n",
    "    df_train['target'] = df_train['sample'].apply(lambda x: TRAIN_TARGETS_PATH + x.split('/')[-1][:14] + \".jpg\")\n",
    "    df_train = df_train.sort_values(by=['target']).reset_index(drop=True)\n",
    "    df_train.to_csv(TRAIN_SAMPLES_DF_PATH, index=False)\n",
    "\n",
    "    df_valid = pd.DataFrame()\n",
    "    df_valid['sample'] = sample_valid\n",
    "    df_valid['target'] = df_valid['sample'].apply(lambda x: TRAIN_TARGETS_PATH + x.split('/')[-1][:14] + \".jpg\")\n",
    "    df_valid = df_valid.sort_values(by=['target']).reset_index(drop=True)\n",
    "    df_valid.to_csv(VALID_SAMPLES_DF_PATH, index=False)\n",
    "\n",
    "\n",
    "# 加载训练集 DataFrame\n",
    "df_train = pd.read_csv(TRAIN_SAMPLES_DF_PATH)\n",
    "TRAIN_SAMPLES_NUMS = len(df_train)\n",
    "print(TRAIN_SAMPLES_NUMS)\n",
    "\n",
    "# 加载验证集 DataFrame\n",
    "df_valid = pd.read_csv(VALID_SAMPLES_DF_PATH)\n",
    "VALID_SAMPLES_NUMS  = len(df_valid)\n",
    "print(VALID_SAMPLES_NUMS)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### 功能函数定义"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "scrolled": true,
    "tags": []
   },
   "outputs": [],
   "source": [
    "# ---------------------------------------------------------------------------------------------------------------------------------\n",
    "# 数据集定义与数据预处理\n",
    "# ---------------------------------------------------------------------------------------------------------------------------------\n",
    "\n",
    "class MyDataset(paddle.io.Dataset):\n",
    "    def __init__(self, df, train=False):\n",
    "        super(MyDataset, self).__init__()\n",
    "\n",
    "        self.df = df\n",
    "        self.train = train  # 训练/验证模式下, 采用不同的数据预处理策略\n",
    "        print(f\"nums data set: {len(self.df)}\")\n",
    "\n",
    "    def __getitem__(self, index):\n",
    "\n",
    "        # 读取训练样本及其标签\n",
    "        image = cv2.imread(self.df.iloc[index]['sample'])\n",
    "        label = cv2.imread(self.df.iloc[index]['target'])\n",
    "        image = cv2.cvtColor(image, cv2.COLOR_BGR2RGB)\n",
    "        label = cv2.cvtColor(label, cv2.COLOR_BGR2RGB)\n",
    "\n",
    "        # 归一化\n",
    "        image = image / 255.\n",
    "        label = label / 255.\n",
    "\n",
    "        # 训练时数据预处理策略\n",
    "        if self.train == True:\n",
    "            h, w, c = image.shape\n",
    "\n",
    "            # 50% 概率训练样本水平翻转, 增强模型对不同 pattern 的泛化性能\n",
    "            if np.random.randint(0, 2) == 1:\n",
    "                image = paddle.vision.transforms.hflip(image)\n",
    "                label = paddle.vision.transforms.hflip(label)\n",
    "\n",
    "            # H, W, C -> C, H, W\n",
    "            image = image.transpose((2,0,1))\n",
    "            label = label.transpose((2,0,1))\n",
    "\n",
    "            # C, H, W -> 1, C, H, W   &    np.ndarray -> paddle.tensor\n",
    "            image = paddle.to_tensor(image[np.newaxis, ...], dtype='float32')\n",
    "            label = paddle.to_tensor(label[np.newaxis, ...], dtype='float32')\n",
    "\n",
    "            # 将训练样本及其标签循环 padding 至最短边不小于 SIZE\n",
    "            pad = nn.Pad2D(padding=[0, max(SIZE-w, 0), 0, max(SIZE-h, 0)], mode='circular')  # padding [pad_left, pad_right, pad_top, pad_bottom]\n",
    "            image = pad(image)\n",
    "            label = pad(label)\n",
    "\n",
    "            # 随机裁剪尺寸为 (SIZE, SIZE) 的 patch 用于训练\n",
    "            h_new = np.random.randint(0, image.shape[-2]-SIZE+1)\n",
    "            w_new = np.random.randint(0, image.shape[-1]-SIZE+1)\n",
    "            image = image[..., h_new:h_new+SIZE, w_new:w_new+SIZE].squeeze(0)\n",
    "            label = label[..., h_new:h_new+SIZE, w_new:w_new+SIZE].squeeze(0)\n",
    "\n",
    "        # 验证时数据预处理策略\n",
    "        else:\n",
    "            # 50% 验证样本水平翻转, 考察模型对不同 pattern 的泛化性能\n",
    "            if index % 2 == 0:\n",
    "                image = paddle.vision.transforms.hflip(image)\n",
    "                label = paddle.vision.transforms.hflip(label)\n",
    "\n",
    "            # H, W, C -> C, H, W\n",
    "            image = image.transpose((2,0,1))\n",
    "            label = label.transpose((2,0,1))\n",
    "\n",
    "            # np.ndarray -> paddle.tensor\n",
    "            image = paddle.to_tensor(image, dtype='float32')\n",
    "            label = paddle.to_tensor(label, dtype='float32')\n",
    "\n",
    "        return image, label\n",
    "\n",
    "    def __len__(self):\n",
    "        return len(self.df)\n",
    "\n",
    "# ---------------------------------------------------------------------------------------------------------------------------------\n",
    "# 验证函数定义\n",
    "# ---------------------------------------------------------------------------------------------------------------------------------\n",
    "\n",
    "def valid(valid_loader, model, psnr_compute, ssim_compute):\n",
    "    print('validating...')\n",
    "    model.eval()\n",
    "    \n",
    "    psnrs = []\n",
    "    ssims = []\n",
    "    scores = []\n",
    "\n",
    "    for step, (sample, target) in enumerate(valid_loader):\n",
    "\t\t# 原输入样本 shape 为 (n, c, h, w)\n",
    "        n, c, h, w = sample.shape\n",
    "        \n",
    "        # 将所有图片的尺寸/边长循环 padding 为 SIZE 的倍数\n",
    "        num_patch_h = (h // SIZE) + 1\n",
    "        num_patch_w = (w // SIZE) + 1 \n",
    "\n",
    "        pad = nn.Pad2D(padding=[0, num_patch_w*SIZE-w, 0, num_patch_h*SIZE-h], mode='circular')  # [pad_left, pad_right, pad_top, pad_bottom]\n",
    "        sample_pad = pad(sample)\n",
    " \n",
    "        # 等距切分为若干尺寸为 (SIZE, SIZE) 的 patch 并分别预测后拼接\n",
    "        output = paddle.zeros_like(sample_pad)\n",
    "        with paddle.no_grad():\n",
    "            for i in range(num_patch_h):\n",
    "                for j in range(num_patch_w):\n",
    "\t\t\t\t\t# 当前用于预测的 patch 切片\n",
    "                    sample_pad_patch = sample_pad[..., i*SIZE: (i+1)*SIZE, j*SIZE: (j+1)*SIZE].clone()\n",
    "                    # 训练期间，验证操作并不实施水平翻转自集成，以促使模型对不同水印 pattern 都能具有较好的恢复效果。\n",
    "                    # 换言之，避免模型对某一水印 pattern 过拟合，从而主导性能变化并轻视对其他水印 pattern 的去除学习。\n",
    "                    output[..., i*SIZE: (i+1)*SIZE, j*SIZE: (j+1)*SIZE] = model(sample_pad_patch)\n",
    "                    \n",
    "        # 切除不需要的预测结果, 还原尺寸为 (h, w) 的原输入样本\n",
    "        output = output[..., :h, :w]\n",
    "\n",
    "        # 逐样本计算 PSNR、SSIM 和 Score\n",
    "        for i in range(len(sample)):\n",
    "            a, b = output[i: i+1, ...] , target[i: i+1, ...]\n",
    "            psnr = psnr_compute(a, b).item()\n",
    "            ssim = ssim_compute(a, b).item()\n",
    "            score = (psnr / 100 + ssim) / 2\n",
    "            psnrs.append(psnr)\n",
    "            ssims.append(ssim)\n",
    "            scores.append(score)\n",
    "            \n",
    "        gc.collect()\n",
    "        paddle.device.cuda.empty_cache()\n",
    "\n",
    "    return np.mean(scores), np.mean(psnrs), np.mean(ssims)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### 训练主函数定义"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "scrolled": true,
    "tags": []
   },
   "outputs": [],
   "source": [
    "# ---------------------------------------------------------------------------------------------------------------------------------\n",
    "# 主函数定义\n",
    "# ---------------------------------------------------------------------------------------------------------------------------------\n",
    "\n",
    "def process():\n",
    "    # 模型实例化\n",
    "    model = Net()\n",
    "    # 加载预/已训练模型 finetune\n",
    "    if MODEL_LOAD_PATH:\n",
    "        param_dict = paddle.load(MODEL_LOAD_PATH)\n",
    "        model.load_dict(param_dict)\n",
    "    model.train()\n",
    "\n",
    "    # 数据集及其加载器实例化\n",
    "    train_dataset = MyDataset(df_train[:DEBUG_TRAIN_SAMPLES], train=True) if DEBUG else MyDataset(df_train, train=True)\n",
    "    valid_dataset = MyDataset(df_valid[:DEBUG_VALID_SAMPLES], train=False) if DEBUG else MyDataset(df_valid, train=False)\n",
    "\n",
    "    train_loader = paddle.io.DataLoader(\n",
    "        train_dataset,\n",
    "        batch_size=BS,\n",
    "        num_workers=0,  # 若机器不错可改为 8\n",
    "        shuffle=True,\n",
    "        drop_last=True\n",
    "    )\n",
    "\n",
    "    valid_loader = paddle.io.DataLoader(\n",
    "        valid_dataset,\n",
    "        batch_size=1,\n",
    "        num_workers=0,\n",
    "        shuffle=False,\n",
    "        drop_last=False\n",
    "    )\n",
    "\n",
    "    # 损失函数与验证指标\n",
    "    criterion = ContrastLoss() if CLLOSS is True else paddle.nn.loss.L1Loss()\n",
    "    psnr_compute = PSNR()\n",
    "    ssim_compute = MS_SSIM()\n",
    "\n",
    "    # 学习率衰减策略与优化器\n",
    "    scheduler = paddle.optimizer.lr.CosineAnnealingDecay(learning_rate=LR, T_max=int(TRAIN_SAMPLES_NUMS//PRINT//BS*EPOCH), eta_min=5e-7)\n",
    "    optimizer = paddle.optimizer.Adam(learning_rate=scheduler, parameters=model.parameters())\n",
    "\n",
    "    best_score = 0.\n",
    "    beg = time.time()\n",
    "    for epoch in range(1, EPOCH+1):\n",
    "        model.train()\n",
    "        train_loss = []\n",
    "\n",
    "        for step, (sample, target) in enumerate(train_loader):\n",
    "            output = model(sample)\n",
    "            loss = criterion(output, target, sample) if CLLOSS is True else criterion(output, target)\n",
    "            train_loss.append(paddle.mean(loss))\n",
    "\n",
    "            # 按 batch 反向传播\n",
    "            loss.backward()\n",
    "            optimizer.step()\n",
    "            optimizer.clear_gradients()\n",
    "\n",
    "            # 打印训练损失\n",
    "            if step % int(PRINT//5) == 0:\n",
    "                print(f\"[{epoch}/{EPOCH}] - Steps {step} - Train loss: {np.mean(train_loss)} - passed: {(time.time() - beg) / 60.}\") \n",
    "\n",
    "            # 打印验证指标、保存当前最佳模型 (验证时间)\n",
    "            if (step > 0) and (step % PRINT == 0):  # \n",
    "                val_score, val_psnr, val_ssim = valid(valid_loader, model, psnr_compute, ssim_compute)\n",
    "\n",
    "                print(\n",
    "                    f\"[{epoch}/{EPOCH}] - Steps {step} - Val Score: {val_score}, PSNR: {val_psnr}, SSIM: {val_ssim}, passed: {(time.time() - beg) / 60.}\")\n",
    "\n",
    "                if val_score > best_score:\n",
    "                    best_score = val_score\n",
    "                    print(f\"======> epoch: {epoch} - step: {step} - best val score: {best_score} <======\")\n",
    "                    \n",
    "                    # 只有验证指标更优才保存模型\n",
    "                    paddle.save(model.state_dict(), f'./model_stage{STAGE}_epoch{epoch}_step{step}.pdparams')\n",
    "\n",
    "                scheduler.step()\n",
    "\n",
    "            gc.collect()\n",
    "            paddle.device.cuda.empty_cache()\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "scrolled": true
   },
   "outputs": [],
   "source": [
    "# ---------------------------------------------------------------------------------------------------------------------------------\n",
    "# 主函数调用\n",
    "# ---------------------------------------------------------------------------------------------------------------------------------\n",
    "\n",
    "if __name__ == '__main__':\n",
    "    process()   # 训练！\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### 打包提交\n",
    "\n",
    "\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "metadata": {
    "execution": {
     "iopub.execute_input": "2023-02-26T10:01:50.931414Z",
     "iopub.status.busy": "2023-02-26T10:01:50.930623Z",
     "iopub.status.idle": "2023-02-26T10:01:56.294008Z",
     "shell.execute_reply": "2023-02-26T10:01:56.292880Z",
     "shell.execute_reply.started": "2023-02-26T10:01:50.931381Z"
    },
    "scrolled": true
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "/opt/conda/envs/python35-paddle120-env/lib/python3.7/site-packages/setuptools/depends.py:2: DeprecationWarning: the imp module is deprecated in favour of importlib; see the module's documentation for alternative uses\r\n",
      "  import imp\r\n",
      "ema finished !!!\r\n",
      "/opt/conda/envs/python35-paddle120-env/lib/python3.7/site-packages/paddle/fluid/layers/utils.py:77: DeprecationWarning: Using or importing the ABCs from 'collections' instead of from 'collections.abc' is deprecated, and in 3.8 it will stop working\r\n",
      "  return (isinstance(seq, collections.Sequence) and\r\n",
      "/opt/conda/envs/python35-paddle120-env/lib/python3.7/site-packages/paddle/fluid/layers/math_op_patch.py:341: UserWarning: /tmp/tmpd88tptv6.py:8\r\n",
      "The behavior of expression A + B has been unified with elementwise_add(X, Y, axis=-1) from Paddle 2.0. If your code works well in the older versions but crashes in this version, try to use elementwise_add(X, Y, axis=0) instead of A + B. This transitional warning will be dropped in the future.\r\n",
      "  op_type, op_type, EXPRESSION_MAP[method_name]))\r\n",
      "/opt/conda/envs/python35-paddle120-env/lib/python3.7/site-packages/paddle/fluid/layers/math_op_patch.py:341: UserWarning: /tmp/tmp8l59p8nm.py:8\r\n",
      "The behavior of expression A + B has been unified with elementwise_add(X, Y, axis=-1) from Paddle 2.0. If your code works well in the older versions but crashes in this version, try to use elementwise_add(X, Y, axis=0) instead of A + B. This transitional warning will be dropped in the future.\r\n",
      "  op_type, op_type, EXPRESSION_MAP[method_name]))\r\n",
      "/opt/conda/envs/python35-paddle120-env/lib/python3.7/site-packages/paddle/fluid/layers/math_op_patch.py:341: UserWarning: /tmp/tmpa9agyqx0.py:8\r\n",
      "The behavior of expression A + B has been unified with elementwise_add(X, Y, axis=-1) from Paddle 2.0. If your code works well in the older versions but crashes in this version, try to use elementwise_add(X, Y, axis=0) instead of A + B. This transitional warning will be dropped in the future.\r\n",
      "  op_type, op_type, EXPRESSION_MAP[method_name]))\r\n",
      "/opt/conda/envs/python35-paddle120-env/lib/python3.7/site-packages/paddle/fluid/layers/math_op_patch.py:341: UserWarning: /tmp/tmp5m098e15.py:11\r\n",
      "The behavior of expression A + B has been unified with elementwise_add(X, Y, axis=-1) from Paddle 2.0. If your code works well in the older versions but crashes in this version, try to use elementwise_add(X, Y, axis=0) instead of A + B. This transitional warning will be dropped in the future.\r\n",
      "  op_type, op_type, EXPRESSION_MAP[method_name]))\r\n",
      "/opt/conda/envs/python35-paddle120-env/lib/python3.7/site-packages/paddle/fluid/dygraph/dygraph_to_static/convert_call_func.py:93: VisibleDeprecationWarning: Creating an ndarray from ragged nested sequences (which is a list-or-tuple of lists-or-tuples-or ndarrays with different lengths or shapes) is deprecated. If you meant to do this, you must specify 'dtype=object' when creating the ndarray.\r\n",
      "  func_in_dict = func == v\r\n",
      "/opt/conda/envs/python35-paddle120-env/lib/python3.7/site-packages/paddle/fluid/layers/math_op_patch.py:341: UserWarning: /tmp/tmpz1qqy_2d.py:8\r\n",
      "The behavior of expression A * B has been unified with elementwise_mul(X, Y, axis=-1) from Paddle 2.0. If your code works well in the older versions but crashes in this version, try to use elementwise_mul(X, Y, axis=0) instead of A * B. This transitional warning will be dropped in the future.\r\n",
      "  op_type, op_type, EXPRESSION_MAP[method_name]))\r\n",
      "/opt/conda/envs/python35-paddle120-env/lib/python3.7/site-packages/paddle/fluid/layers/math_op_patch.py:341: UserWarning: /tmp/tmpwj_ckjik.py:8\r\n",
      "The behavior of expression A + B has been unified with elementwise_add(X, Y, axis=-1) from Paddle 2.0. If your code works well in the older versions but crashes in this version, try to use elementwise_add(X, Y, axis=0) instead of A + B. This transitional warning will be dropped in the future.\r\n",
      "  op_type, op_type, EXPRESSION_MAP[method_name]))\r\n",
      "/opt/conda/envs/python35-paddle120-env/lib/python3.7/site-packages/paddle/fluid/layers/math_op_patch.py:341: UserWarning: /tmp/tmpiot3pi95.py:8\r\n",
      "The behavior of expression A + B has been unified with elementwise_add(X, Y, axis=-1) from Paddle 2.0. If your code works well in the older versions but crashes in this version, try to use elementwise_add(X, Y, axis=0) instead of A + B. This transitional warning will be dropped in the future.\r\n",
      "  op_type, op_type, EXPRESSION_MAP[method_name]))\r\n",
      "/opt/conda/envs/python35-paddle120-env/lib/python3.7/site-packages/paddle/fluid/layers/math_op_patch.py:341: UserWarning: /tmp/tmpd5ul92eu.py:8\r\n",
      "The behavior of expression A + B has been unified with elementwise_add(X, Y, axis=-1) from Paddle 2.0. If your code works well in the older versions but crashes in this version, try to use elementwise_add(X, Y, axis=0) instead of A + B. This transitional warning will be dropped in the future.\r\n",
      "  op_type, op_type, EXPRESSION_MAP[method_name]))\r\n",
      "/opt/conda/envs/python35-paddle120-env/lib/python3.7/site-packages/paddle/fluid/layers/math_op_patch.py:341: UserWarning: /tmp/tmpg4knon14.py:10\r\n",
      "The behavior of expression A + B has been unified with elementwise_add(X, Y, axis=-1) from Paddle 2.0. If your code works well in the older versions but crashes in this version, try to use elementwise_add(X, Y, axis=0) instead of A + B. This transitional warning will be dropped in the future.\r\n",
      "  op_type, op_type, EXPRESSION_MAP[method_name]))\r\n",
      "/opt/conda/envs/python35-paddle120-env/lib/python3.7/site-packages/paddle/fluid/layers/math_op_patch.py:341: UserWarning: /tmp/tmpxyfx1o4z.py:8\r\n",
      "The behavior of expression A * B has been unified with elementwise_mul(X, Y, axis=-1) from Paddle 2.0. If your code works well in the older versions but crashes in this version, try to use elementwise_mul(X, Y, axis=0) instead of A * B. This transitional warning will be dropped in the future.\r\n",
      "  op_type, op_type, EXPRESSION_MAP[method_name]))\r\n",
      "/opt/conda/envs/python35-paddle120-env/lib/python3.7/site-packages/paddle/fluid/layers/math_op_patch.py:341: UserWarning: /tmp/tmpy_i6cuhb.py:8\r\n",
      "The behavior of expression A + B has been unified with elementwise_add(X, Y, axis=-1) from Paddle 2.0. If your code works well in the older versions but crashes in this version, try to use elementwise_add(X, Y, axis=0) instead of A + B. This transitional warning will be dropped in the future.\r\n",
      "  op_type, op_type, EXPRESSION_MAP[method_name]))\r\n",
      "/opt/conda/envs/python35-paddle120-env/lib/python3.7/site-packages/paddle/fluid/layers/math_op_patch.py:341: UserWarning: /tmp/tmp5xgrp3hq.py:8\r\n",
      "The behavior of expression A + B has been unified with elementwise_add(X, Y, axis=-1) from Paddle 2.0. If your code works well in the older versions but crashes in this version, try to use elementwise_add(X, Y, axis=0) instead of A + B. This transitional warning will be dropped in the future.\r\n",
      "  op_type, op_type, EXPRESSION_MAP[method_name]))\r\n",
      "/opt/conda/envs/python35-paddle120-env/lib/python3.7/site-packages/paddle/fluid/layers/math_op_patch.py:341: UserWarning: /tmp/tmpf_ax6pzy.py:8\r\n",
      "The behavior of expression A + B has been unified with elementwise_add(X, Y, axis=-1) from Paddle 2.0. If your code works well in the older versions but crashes in this version, try to use elementwise_add(X, Y, axis=0) instead of A + B. This transitional warning will be dropped in the future.\r\n",
      "  op_type, op_type, EXPRESSION_MAP[method_name]))\r\n",
      "/opt/conda/envs/python35-paddle120-env/lib/python3.7/site-packages/paddle/fluid/layers/math_op_patch.py:341: UserWarning: /tmp/tmpimmot7_p.py:8\r\n",
      "The behavior of expression A + B has been unified with elementwise_add(X, Y, axis=-1) from Paddle 2.0. If your code works well in the older versions but crashes in this version, try to use elementwise_add(X, Y, axis=0) instead of A + B. This transitional warning will be dropped in the future.\r\n",
      "  op_type, op_type, EXPRESSION_MAP[method_name]))\r\n",
      "/opt/conda/envs/python35-paddle120-env/lib/python3.7/site-packages/paddle/fluid/layers/math_op_patch.py:341: UserWarning: /tmp/tmp56azrghz.py:11\r\n",
      "The behavior of expression A + B has been unified with elementwise_add(X, Y, axis=-1) from Paddle 2.0. If your code works well in the older versions but crashes in this version, try to use elementwise_add(X, Y, axis=0) instead of A + B. This transitional warning will be dropped in the future.\r\n",
      "  op_type, op_type, EXPRESSION_MAP[method_name]))\r\n",
      "/opt/conda/envs/python35-paddle120-env/lib/python3.7/site-packages/paddle/fluid/layers/math_op_patch.py:341: UserWarning: /tmp/tmp6oji91h4.py:8\r\n",
      "The behavior of expression A * B has been unified with elementwise_mul(X, Y, axis=-1) from Paddle 2.0. If your code works well in the older versions but crashes in this version, try to use elementwise_mul(X, Y, axis=0) instead of A * B. This transitional warning will be dropped in the future.\r\n",
      "  op_type, op_type, EXPRESSION_MAP[method_name]))\r\n",
      "/opt/conda/envs/python35-paddle120-env/lib/python3.7/site-packages/paddle/fluid/layers/math_op_patch.py:341: UserWarning: /tmp/tmpse7w0nam.py:8\r\n",
      "The behavior of expression A + B has been unified with elementwise_add(X, Y, axis=-1) from Paddle 2.0. If your code works well in the older versions but crashes in this version, try to use elementwise_add(X, Y, axis=0) instead of A + B. This transitional warning will be dropped in the future.\r\n",
      "  op_type, op_type, EXPRESSION_MAP[method_name]))\r\n",
      "/opt/conda/envs/python35-paddle120-env/lib/python3.7/site-packages/paddle/fluid/layers/math_op_patch.py:341: UserWarning: /tmp/tmp0stlr83t.py:8\r\n",
      "The behavior of expression A + B has been unified with elementwise_add(X, Y, axis=-1) from Paddle 2.0. If your code works well in the older versions but crashes in this version, try to use elementwise_add(X, Y, axis=0) instead of A + B. This transitional warning will be dropped in the future.\r\n",
      "  op_type, op_type, EXPRESSION_MAP[method_name]))\r\n",
      "/opt/conda/envs/python35-paddle120-env/lib/python3.7/site-packages/paddle/fluid/layers/math_op_patch.py:341: UserWarning: /tmp/tmpvqz7qzhs.py:8\r\n",
      "The behavior of expression A + B has been unified with elementwise_add(X, Y, axis=-1) from Paddle 2.0. If your code works well in the older versions but crashes in this version, try to use elementwise_add(X, Y, axis=0) instead of A + B. This transitional warning will be dropped in the future.\r\n",
      "  op_type, op_type, EXPRESSION_MAP[method_name]))\r\n",
      "/opt/conda/envs/python35-paddle120-env/lib/python3.7/site-packages/paddle/fluid/layers/math_op_patch.py:341: UserWarning: /tmp/tmpmd3whg0x.py:10\r\n",
      "The behavior of expression A + B has been unified with elementwise_add(X, Y, axis=-1) from Paddle 2.0. If your code works well in the older versions but crashes in this version, try to use elementwise_add(X, Y, axis=0) instead of A + B. This transitional warning will be dropped in the future.\r\n",
      "  op_type, op_type, EXPRESSION_MAP[method_name]))\r\n",
      "/opt/conda/envs/python35-paddle120-env/lib/python3.7/site-packages/paddle/fluid/layers/math_op_patch.py:341: UserWarning: /tmp/tmp9zhcrg0r.py:8\r\n",
      "The behavior of expression A * B has been unified with elementwise_mul(X, Y, axis=-1) from Paddle 2.0. If your code works well in the older versions but crashes in this version, try to use elementwise_mul(X, Y, axis=0) instead of A * B. This transitional warning will be dropped in the future.\r\n",
      "  op_type, op_type, EXPRESSION_MAP[method_name]))\r\n",
      "/opt/conda/envs/python35-paddle120-env/lib/python3.7/site-packages/paddle/fluid/layers/math_op_patch.py:341: UserWarning: /tmp/tmpvzl7y34k.py:8\r\n",
      "The behavior of expression A + B has been unified with elementwise_add(X, Y, axis=-1) from Paddle 2.0. If your code works well in the older versions but crashes in this version, try to use elementwise_add(X, Y, axis=0) instead of A + B. This transitional warning will be dropped in the future.\r\n",
      "  op_type, op_type, EXPRESSION_MAP[method_name]))\r\n",
      "/opt/conda/envs/python35-paddle120-env/lib/python3.7/site-packages/paddle/fluid/layers/math_op_patch.py:341: UserWarning: /tmp/tmp_lf8vk14.py:8\r\n",
      "The behavior of expression A + B has been unified with elementwise_add(X, Y, axis=-1) from Paddle 2.0. If your code works well in the older versions but crashes in this version, try to use elementwise_add(X, Y, axis=0) instead of A + B. This transitional warning will be dropped in the future.\r\n",
      "  op_type, op_type, EXPRESSION_MAP[method_name]))\r\n",
      "/opt/conda/envs/python35-paddle120-env/lib/python3.7/site-packages/paddle/fluid/layers/math_op_patch.py:341: UserWarning: /tmp/tmpsgj48zwk.py:8\r\n",
      "The behavior of expression A + B has been unified with elementwise_add(X, Y, axis=-1) from Paddle 2.0. If your code works well in the older versions but crashes in this version, try to use elementwise_add(X, Y, axis=0) instead of A + B. This transitional warning will be dropped in the future.\r\n",
      "  op_type, op_type, EXPRESSION_MAP[method_name]))\r\n",
      "/opt/conda/envs/python35-paddle120-env/lib/python3.7/site-packages/paddle/fluid/layers/math_op_patch.py:341: UserWarning: /tmp/tmp18qabru0.py:8\r\n",
      "The behavior of expression A + B has been unified with elementwise_add(X, Y, axis=-1) from Paddle 2.0. If your code works well in the older versions but crashes in this version, try to use elementwise_add(X, Y, axis=0) instead of A + B. This transitional warning will be dropped in the future.\r\n",
      "  op_type, op_type, EXPRESSION_MAP[method_name]))\r\n",
      "/opt/conda/envs/python35-paddle120-env/lib/python3.7/site-packages/paddle/fluid/layers/math_op_patch.py:341: UserWarning: /tmp/tmpk9x10hl_.py:11\r\n",
      "The behavior of expression A + B has been unified with elementwise_add(X, Y, axis=-1) from Paddle 2.0. If your code works well in the older versions but crashes in this version, try to use elementwise_add(X, Y, axis=0) instead of A + B. This transitional warning will be dropped in the future.\r\n",
      "  op_type, op_type, EXPRESSION_MAP[method_name]))\r\n",
      "/opt/conda/envs/python35-paddle120-env/lib/python3.7/site-packages/paddle/fluid/layers/math_op_patch.py:341: UserWarning: /tmp/tmpcf3upu7c.py:8\r\n",
      "The behavior of expression A * B has been unified with elementwise_mul(X, Y, axis=-1) from Paddle 2.0. If your code works well in the older versions but crashes in this version, try to use elementwise_mul(X, Y, axis=0) instead of A * B. This transitional warning will be dropped in the future.\r\n",
      "  op_type, op_type, EXPRESSION_MAP[method_name]))\r\n",
      "/opt/conda/envs/python35-paddle120-env/lib/python3.7/site-packages/paddle/fluid/layers/math_op_patch.py:341: UserWarning: /tmp/tmp7odomy45.py:8\r\n",
      "The behavior of expression A + B has been unified with elementwise_add(X, Y, axis=-1) from Paddle 2.0. If your code works well in the older versions but crashes in this version, try to use elementwise_add(X, Y, axis=0) instead of A + B. This transitional warning will be dropped in the future.\r\n",
      "  op_type, op_type, EXPRESSION_MAP[method_name]))\r\n",
      "/opt/conda/envs/python35-paddle120-env/lib/python3.7/site-packages/paddle/fluid/layers/math_op_patch.py:341: UserWarning: /tmp/tmpolighrye.py:8\r\n",
      "The behavior of expression A + B has been unified with elementwise_add(X, Y, axis=-1) from Paddle 2.0. If your code works well in the older versions but crashes in this version, try to use elementwise_add(X, Y, axis=0) instead of A + B. This transitional warning will be dropped in the future.\r\n",
      "  op_type, op_type, EXPRESSION_MAP[method_name]))\r\n",
      "/opt/conda/envs/python35-paddle120-env/lib/python3.7/site-packages/paddle/fluid/layers/math_op_patch.py:341: UserWarning: /tmp/tmpss7osaod.py:8\r\n",
      "The behavior of expression A + B has been unified with elementwise_add(X, Y, axis=-1) from Paddle 2.0. If your code works well in the older versions but crashes in this version, try to use elementwise_add(X, Y, axis=0) instead of A + B. This transitional warning will be dropped in the future.\r\n",
      "  op_type, op_type, EXPRESSION_MAP[method_name]))\r\n",
      "/opt/conda/envs/python35-paddle120-env/lib/python3.7/site-packages/paddle/fluid/layers/math_op_patch.py:341: UserWarning: /tmp/tmpqd8wfhip.py:10\r\n",
      "The behavior of expression A + B has been unified with elementwise_add(X, Y, axis=-1) from Paddle 2.0. If your code works well in the older versions but crashes in this version, try to use elementwise_add(X, Y, axis=0) instead of A + B. This transitional warning will be dropped in the future.\r\n",
      "  op_type, op_type, EXPRESSION_MAP[method_name]))\r\n",
      "/opt/conda/envs/python35-paddle120-env/lib/python3.7/site-packages/paddle/fluid/layers/math_op_patch.py:341: UserWarning: /tmp/tmphhl0hme7.py:8\r\n",
      "The behavior of expression A * B has been unified with elementwise_mul(X, Y, axis=-1) from Paddle 2.0. If your code works well in the older versions but crashes in this version, try to use elementwise_mul(X, Y, axis=0) instead of A * B. This transitional warning will be dropped in the future.\r\n",
      "  op_type, op_type, EXPRESSION_MAP[method_name]))\r\n",
      "/opt/conda/envs/python35-paddle120-env/lib/python3.7/site-packages/paddle/fluid/layers/math_op_patch.py:341: UserWarning: /tmp/tmpe6l64e04.py:8\r\n",
      "The behavior of expression A + B has been unified with elementwise_add(X, Y, axis=-1) from Paddle 2.0. If your code works well in the older versions but crashes in this version, try to use elementwise_add(X, Y, axis=0) instead of A + B. This transitional warning will be dropped in the future.\r\n",
      "  op_type, op_type, EXPRESSION_MAP[method_name]))\r\n",
      "/opt/conda/envs/python35-paddle120-env/lib/python3.7/site-packages/paddle/fluid/layers/math_op_patch.py:341: UserWarning: /tmp/tmpt3moied1.py:8\r\n",
      "The behavior of expression A + B has been unified with elementwise_add(X, Y, axis=-1) from Paddle 2.0. If your code works well in the older versions but crashes in this version, try to use elementwise_add(X, Y, axis=0) instead of A + B. This transitional warning will be dropped in the future.\r\n",
      "  op_type, op_type, EXPRESSION_MAP[method_name]))\r\n",
      "/opt/conda/envs/python35-paddle120-env/lib/python3.7/site-packages/paddle/fluid/layers/math_op_patch.py:341: UserWarning: /tmp/tmp1f1qx_4b.py:8\r\n",
      "The behavior of expression A + B has been unified with elementwise_add(X, Y, axis=-1) from Paddle 2.0. If your code works well in the older versions but crashes in this version, try to use elementwise_add(X, Y, axis=0) instead of A + B. This transitional warning will be dropped in the future.\r\n",
      "  op_type, op_type, EXPRESSION_MAP[method_name]))\r\n",
      "/opt/conda/envs/python35-paddle120-env/lib/python3.7/site-packages/paddle/fluid/layers/math_op_patch.py:341: UserWarning: /tmp/tmpm5r8yo1f.py:8\r\n",
      "The behavior of expression A + B has been unified with elementwise_add(X, Y, axis=-1) from Paddle 2.0. If your code works well in the older versions but crashes in this version, try to use elementwise_add(X, Y, axis=0) instead of A + B. This transitional warning will be dropped in the future.\r\n",
      "  op_type, op_type, EXPRESSION_MAP[method_name]))\r\n",
      "/opt/conda/envs/python35-paddle120-env/lib/python3.7/site-packages/paddle/fluid/layers/math_op_patch.py:341: UserWarning: /tmp/tmpcf8umocx.py:10\r\n",
      "The behavior of expression A + B has been unified with elementwise_add(X, Y, axis=-1) from Paddle 2.0. If your code works well in the older versions but crashes in this version, try to use elementwise_add(X, Y, axis=0) instead of A + B. This transitional warning will be dropped in the future.\r\n",
      "  op_type, op_type, EXPRESSION_MAP[method_name]))\r\n",
      "/opt/conda/envs/python35-paddle120-env/lib/python3.7/site-packages/paddle/fluid/layers/math_op_patch.py:341: UserWarning: /tmp/tmp9tlfqlzk.py:8\r\n",
      "The behavior of expression A * B has been unified with elementwise_mul(X, Y, axis=-1) from Paddle 2.0. If your code works well in the older versions but crashes in this version, try to use elementwise_mul(X, Y, axis=0) instead of A * B. This transitional warning will be dropped in the future.\r\n",
      "  op_type, op_type, EXPRESSION_MAP[method_name]))\r\n",
      "/opt/conda/envs/python35-paddle120-env/lib/python3.7/site-packages/paddle/fluid/layers/math_op_patch.py:341: UserWarning: /tmp/tmpj7stlljz.py:8\r\n",
      "The behavior of expression A + B has been unified with elementwise_add(X, Y, axis=-1) from Paddle 2.0. If your code works well in the older versions but crashes in this version, try to use elementwise_add(X, Y, axis=0) instead of A + B. This transitional warning will be dropped in the future.\r\n",
      "  op_type, op_type, EXPRESSION_MAP[method_name]))\r\n",
      "/opt/conda/envs/python35-paddle120-env/lib/python3.7/site-packages/paddle/fluid/layers/math_op_patch.py:341: UserWarning: /tmp/tmpm432dvqz.py:8\r\n",
      "The behavior of expression A + B has been unified with elementwise_add(X, Y, axis=-1) from Paddle 2.0. If your code works well in the older versions but crashes in this version, try to use elementwise_add(X, Y, axis=0) instead of A + B. This transitional warning will be dropped in the future.\r\n",
      "  op_type, op_type, EXPRESSION_MAP[method_name]))\r\n",
      "/opt/conda/envs/python35-paddle120-env/lib/python3.7/site-packages/paddle/fluid/layers/math_op_patch.py:341: UserWarning: /tmp/tmpc5iyhifh.py:11\r\n",
      "The behavior of expression A + B has been unified with elementwise_add(X, Y, axis=-1) from Paddle 2.0. If your code works well in the older versions but crashes in this version, try to use elementwise_add(X, Y, axis=0) instead of A + B. This transitional warning will be dropped in the future.\r\n",
      "  op_type, op_type, EXPRESSION_MAP[method_name]))\r\n",
      "/opt/conda/envs/python35-paddle120-env/lib/python3.7/site-packages/paddle/fluid/layers/math_op_patch.py:341: UserWarning: /tmp/tmpp08s8gjv.py:8\r\n",
      "The behavior of expression A * B has been unified with elementwise_mul(X, Y, axis=-1) from Paddle 2.0. If your code works well in the older versions but crashes in this version, try to use elementwise_mul(X, Y, axis=0) instead of A * B. This transitional warning will be dropped in the future.\r\n",
      "  op_type, op_type, EXPRESSION_MAP[method_name]))\r\n",
      "/opt/conda/envs/python35-paddle120-env/lib/python3.7/site-packages/paddle/fluid/layers/math_op_patch.py:341: UserWarning: /tmp/tmpcm35emm3.py:8\r\n",
      "The behavior of expression A + B has been unified with elementwise_add(X, Y, axis=-1) from Paddle 2.0. If your code works well in the older versions but crashes in this version, try to use elementwise_add(X, Y, axis=0) instead of A + B. This transitional warning will be dropped in the future.\r\n",
      "  op_type, op_type, EXPRESSION_MAP[method_name]))\r\n",
      "/opt/conda/envs/python35-paddle120-env/lib/python3.7/site-packages/paddle/fluid/layers/math_op_patch.py:341: UserWarning: /tmp/tmp64rk1f6b.py:8\r\n",
      "The behavior of expression A + B has been unified with elementwise_add(X, Y, axis=-1) from Paddle 2.0. If your code works well in the older versions but crashes in this version, try to use elementwise_add(X, Y, axis=0) instead of A + B. This transitional warning will be dropped in the future.\r\n",
      "  op_type, op_type, EXPRESSION_MAP[method_name]))\r\n",
      "/opt/conda/envs/python35-paddle120-env/lib/python3.7/site-packages/paddle/fluid/layers/math_op_patch.py:341: UserWarning: /tmp/tmpy4aq4crz.py:8\r\n",
      "The behavior of expression A + B has been unified with elementwise_add(X, Y, axis=-1) from Paddle 2.0. If your code works well in the older versions but crashes in this version, try to use elementwise_add(X, Y, axis=0) instead of A + B. This transitional warning will be dropped in the future.\r\n",
      "  op_type, op_type, EXPRESSION_MAP[method_name]))\r\n",
      "/opt/conda/envs/python35-paddle120-env/lib/python3.7/site-packages/paddle/fluid/layers/math_op_patch.py:341: UserWarning: /tmp/tmpk7obm2be.py:10\r\n",
      "The behavior of expression A + B has been unified with elementwise_add(X, Y, axis=-1) from Paddle 2.0. If your code works well in the older versions but crashes in this version, try to use elementwise_add(X, Y, axis=0) instead of A + B. This transitional warning will be dropped in the future.\r\n",
      "  op_type, op_type, EXPRESSION_MAP[method_name]))\r\n",
      "/opt/conda/envs/python35-paddle120-env/lib/python3.7/site-packages/paddle/fluid/layers/math_op_patch.py:341: UserWarning: /tmp/tmpp68r2ekz.py:8\r\n",
      "The behavior of expression A * B has been unified with elementwise_mul(X, Y, axis=-1) from Paddle 2.0. If your code works well in the older versions but crashes in this version, try to use elementwise_mul(X, Y, axis=0) instead of A * B. This transitional warning will be dropped in the future.\r\n",
      "  op_type, op_type, EXPRESSION_MAP[method_name]))\r\n",
      "/opt/conda/envs/python35-paddle120-env/lib/python3.7/site-packages/paddle/fluid/layers/math_op_patch.py:341: UserWarning: /tmp/tmpj7331hv0.py:8\r\n",
      "The behavior of expression A + B has been unified with elementwise_add(X, Y, axis=-1) from Paddle 2.0. If your code works well in the older versions but crashes in this version, try to use elementwise_add(X, Y, axis=0) instead of A + B. This transitional warning will be dropped in the future.\r\n",
      "  op_type, op_type, EXPRESSION_MAP[method_name]))\r\n",
      "/opt/conda/envs/python35-paddle120-env/lib/python3.7/site-packages/paddle/fluid/layers/math_op_patch.py:341: UserWarning: /tmp/tmpo058q85p.py:8\r\n",
      "The behavior of expression A + B has been unified with elementwise_add(X, Y, axis=-1) from Paddle 2.0. If your code works well in the older versions but crashes in this version, try to use elementwise_add(X, Y, axis=0) instead of A + B. This transitional warning will be dropped in the future.\r\n",
      "  op_type, op_type, EXPRESSION_MAP[method_name]))\r\n",
      "/opt/conda/envs/python35-paddle120-env/lib/python3.7/site-packages/paddle/fluid/layers/math_op_patch.py:341: UserWarning: /tmp/tmp1ca9nv8c.py:11\r\n",
      "The behavior of expression A + B has been unified with elementwise_add(X, Y, axis=-1) from Paddle 2.0. If your code works well in the older versions but crashes in this version, try to use elementwise_add(X, Y, axis=0) instead of A + B. This transitional warning will be dropped in the future.\r\n",
      "  op_type, op_type, EXPRESSION_MAP[method_name]))\r\n",
      "/opt/conda/envs/python35-paddle120-env/lib/python3.7/site-packages/paddle/fluid/layers/math_op_patch.py:341: UserWarning: /tmp/tmpc4w2kdqx.py:8\r\n",
      "The behavior of expression A * B has been unified with elementwise_mul(X, Y, axis=-1) from Paddle 2.0. If your code works well in the older versions but crashes in this version, try to use elementwise_mul(X, Y, axis=0) instead of A * B. This transitional warning will be dropped in the future.\r\n",
      "  op_type, op_type, EXPRESSION_MAP[method_name]))\r\n",
      "/opt/conda/envs/python35-paddle120-env/lib/python3.7/site-packages/paddle/fluid/layers/math_op_patch.py:341: UserWarning: /tmp/tmpap0o5vnr.py:8\r\n",
      "The behavior of expression A + B has been unified with elementwise_add(X, Y, axis=-1) from Paddle 2.0. If your code works well in the older versions but crashes in this version, try to use elementwise_add(X, Y, axis=0) instead of A + B. This transitional warning will be dropped in the future.\r\n",
      "  op_type, op_type, EXPRESSION_MAP[method_name]))\r\n",
      "/opt/conda/envs/python35-paddle120-env/lib/python3.7/site-packages/paddle/fluid/layers/math_op_patch.py:341: UserWarning: /tmp/tmpv6ez9it9.py:8\r\n",
      "The behavior of expression A + B has been unified with elementwise_add(X, Y, axis=-1) from Paddle 2.0. If your code works well in the older versions but crashes in this version, try to use elementwise_add(X, Y, axis=0) instead of A + B. This transitional warning will be dropped in the future.\r\n",
      "  op_type, op_type, EXPRESSION_MAP[method_name]))\r\n",
      "/opt/conda/envs/python35-paddle120-env/lib/python3.7/site-packages/paddle/fluid/layers/math_op_patch.py:341: UserWarning: /tmp/tmp9z8228xd.py:8\r\n",
      "The behavior of expression A + B has been unified with elementwise_add(X, Y, axis=-1) from Paddle 2.0. If your code works well in the older versions but crashes in this version, try to use elementwise_add(X, Y, axis=0) instead of A + B. This transitional warning will be dropped in the future.\r\n",
      "  op_type, op_type, EXPRESSION_MAP[method_name]))\r\n",
      "/opt/conda/envs/python35-paddle120-env/lib/python3.7/site-packages/paddle/fluid/layers/math_op_patch.py:341: UserWarning: /tmp/tmpuqgx15sw.py:10\r\n",
      "The behavior of expression A + B has been unified with elementwise_add(X, Y, axis=-1) from Paddle 2.0. If your code works well in the older versions but crashes in this version, try to use elementwise_add(X, Y, axis=0) instead of A + B. This transitional warning will be dropped in the future.\r\n",
      "  op_type, op_type, EXPRESSION_MAP[method_name]))\r\n",
      "/opt/conda/envs/python35-paddle120-env/lib/python3.7/site-packages/paddle/fluid/layers/math_op_patch.py:341: UserWarning: /tmp/tmpqhl6m1tv.py:8\r\n",
      "The behavior of expression A * B has been unified with elementwise_mul(X, Y, axis=-1) from Paddle 2.0. If your code works well in the older versions but crashes in this version, try to use elementwise_mul(X, Y, axis=0) instead of A * B. This transitional warning will be dropped in the future.\r\n",
      "  op_type, op_type, EXPRESSION_MAP[method_name]))\r\n",
      "/opt/conda/envs/python35-paddle120-env/lib/python3.7/site-packages/paddle/fluid/layers/math_op_patch.py:341: UserWarning: /tmp/tmpb0dt607u.py:8\r\n",
      "The behavior of expression A + B has been unified with elementwise_add(X, Y, axis=-1) from Paddle 2.0. If your code works well in the older versions but crashes in this version, try to use elementwise_add(X, Y, axis=0) instead of A + B. This transitional warning will be dropped in the future.\r\n",
      "  op_type, op_type, EXPRESSION_MAP[method_name]))\r\n",
      "/opt/conda/envs/python35-paddle120-env/lib/python3.7/site-packages/paddle/fluid/layers/math_op_patch.py:341: UserWarning: /tmp/tmpruv15492.py:8\r\n",
      "The behavior of expression A + B has been unified with elementwise_add(X, Y, axis=-1) from Paddle 2.0. If your code works well in the older versions but crashes in this version, try to use elementwise_add(X, Y, axis=0) instead of A + B. This transitional warning will be dropped in the future.\r\n",
      "  op_type, op_type, EXPRESSION_MAP[method_name]))\r\n",
      "/opt/conda/envs/python35-paddle120-env/lib/python3.7/site-packages/paddle/fluid/layers/math_op_patch.py:341: UserWarning: /tmp/tmpo1j9gaxe.py:11\r\n",
      "The behavior of expression A + B has been unified with elementwise_add(X, Y, axis=-1) from Paddle 2.0. If your code works well in the older versions but crashes in this version, try to use elementwise_add(X, Y, axis=0) instead of A + B. This transitional warning will be dropped in the future.\r\n",
      "  op_type, op_type, EXPRESSION_MAP[method_name]))\r\n",
      "/opt/conda/envs/python35-paddle120-env/lib/python3.7/site-packages/paddle/fluid/layers/math_op_patch.py:341: UserWarning: /tmp/tmpnqx7371a.py:8\r\n",
      "The behavior of expression A * B has been unified with elementwise_mul(X, Y, axis=-1) from Paddle 2.0. If your code works well in the older versions but crashes in this version, try to use elementwise_mul(X, Y, axis=0) instead of A * B. This transitional warning will be dropped in the future.\r\n",
      "  op_type, op_type, EXPRESSION_MAP[method_name]))\r\n",
      "/opt/conda/envs/python35-paddle120-env/lib/python3.7/site-packages/paddle/fluid/layers/math_op_patch.py:341: UserWarning: /tmp/tmpsf_xk5dg.py:8\r\n",
      "The behavior of expression A + B has been unified with elementwise_add(X, Y, axis=-1) from Paddle 2.0. If your code works well in the older versions but crashes in this version, try to use elementwise_add(X, Y, axis=0) instead of A + B. This transitional warning will be dropped in the future.\r\n",
      "  op_type, op_type, EXPRESSION_MAP[method_name]))\r\n",
      "/opt/conda/envs/python35-paddle120-env/lib/python3.7/site-packages/paddle/fluid/layers/math_op_patch.py:341: UserWarning: /tmp/tmphmava9ah.py:8\r\n",
      "The behavior of expression A + B has been unified with elementwise_add(X, Y, axis=-1) from Paddle 2.0. If your code works well in the older versions but crashes in this version, try to use elementwise_add(X, Y, axis=0) instead of A + B. This transitional warning will be dropped in the future.\r\n",
      "  op_type, op_type, EXPRESSION_MAP[method_name]))\r\n",
      "/opt/conda/envs/python35-paddle120-env/lib/python3.7/site-packages/paddle/fluid/layers/math_op_patch.py:341: UserWarning: /tmp/tmprxmrlrgr.py:8\r\n",
      "The behavior of expression A + B has been unified with elementwise_add(X, Y, axis=-1) from Paddle 2.0. If your code works well in the older versions but crashes in this version, try to use elementwise_add(X, Y, axis=0) instead of A + B. This transitional warning will be dropped in the future.\r\n",
      "  op_type, op_type, EXPRESSION_MAP[method_name]))\r\n",
      "/opt/conda/envs/python35-paddle120-env/lib/python3.7/site-packages/paddle/fluid/layers/math_op_patch.py:341: UserWarning: /tmp/tmpts1v81a3.py:10\r\n",
      "The behavior of expression A + B has been unified with elementwise_add(X, Y, axis=-1) from Paddle 2.0. If your code works well in the older versions but crashes in this version, try to use elementwise_add(X, Y, axis=0) instead of A + B. This transitional warning will be dropped in the future.\r\n",
      "  op_type, op_type, EXPRESSION_MAP[method_name]))\r\n",
      "/opt/conda/envs/python35-paddle120-env/lib/python3.7/site-packages/paddle/fluid/layers/math_op_patch.py:341: UserWarning: /tmp/tmpu9ijuy8t.py:8\r\n",
      "The behavior of expression A * B has been unified with elementwise_mul(X, Y, axis=-1) from Paddle 2.0. If your code works well in the older versions but crashes in this version, try to use elementwise_mul(X, Y, axis=0) instead of A * B. This transitional warning will be dropped in the future.\r\n",
      "  op_type, op_type, EXPRESSION_MAP[method_name]))\r\n",
      "/opt/conda/envs/python35-paddle120-env/lib/python3.7/site-packages/paddle/fluid/layers/math_op_patch.py:341: UserWarning: /tmp/tmptu22ybwx.py:8\r\n",
      "The behavior of expression A + B has been unified with elementwise_add(X, Y, axis=-1) from Paddle 2.0. If your code works well in the older versions but crashes in this version, try to use elementwise_add(X, Y, axis=0) instead of A + B. This transitional warning will be dropped in the future.\r\n",
      "  op_type, op_type, EXPRESSION_MAP[method_name]))\r\n",
      "/opt/conda/envs/python35-paddle120-env/lib/python3.7/site-packages/paddle/fluid/layers/math_op_patch.py:341: UserWarning: /tmp/tmpta0jkt9q.py:20\r\n",
      "The behavior of expression A + B has been unified with elementwise_add(X, Y, axis=-1) from Paddle 2.0. If your code works well in the older versions but crashes in this version, try to use elementwise_add(X, Y, axis=0) instead of A + B. This transitional warning will be dropped in the future.\r\n",
      "  op_type, op_type, EXPRESSION_MAP[method_name]))\r\n"
     ]
    }
   ],
   "source": [
    "! python ema.py"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "metadata": {
    "execution": {
     "iopub.execute_input": "2023-02-26T10:02:29.207193Z",
     "iopub.status.busy": "2023-02-26T10:02:29.206647Z",
     "iopub.status.idle": "2023-02-26T10:02:29.976132Z",
     "shell.execute_reply": "2023-02-26T10:02:29.975007Z",
     "shell.execute_reply.started": "2023-02-26T10:02:29.207151Z"
    },
    "scrolled": true,
    "tags": []
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "  adding: model.pdiparams (deflated 7%)\r\n",
      "  adding: model.pdiparams.info (deflated 82%)\r\n",
      "  adding: model.pdmodel (deflated 98%)\r\n",
      "  adding: predict.py (deflated 63%)\r\n"
     ]
    }
   ],
   "source": [
    "# 压缩提交文件为 submit.zip\n",
    "! zip submit1.zip model.pdiparams model.pdiparams.info model.pdmodel predict.py"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# 六、模型推理\n",
    "\n",
    "1. 测试时数据增强\n",
    "\n",
    "- 循环填充：使用循环填充（右侧&下侧）确保输入样本的宽高为预设尺寸的整数倍，且保持填充前后水印模式一致。\n",
    "- 等距裁剪分块预测：为避免放缩失真和显存不足，按预设尺寸随机裁剪样本并分别预测，最后将预测结果拼接并移除填充部分。\n",
    "- 水平翻转自集成：为发挥模型对不同水印模式的泛化能力，将原预测结果和经水平翻转-预测-水平翻转的预测结果等权融合。\n",
    "\n",
    "\n",
    "![](https://ai-studio-static-online.cdn.bcebos.com/da5d12e330154b009b47ea7165bc2aae69d695fdf607467c8815d1ff7011d0f7)\n",
    "\n",
    "\n",
    "```\n",
    "\n",
    "# 原输入样本 shape 为 (n, c, h, w)\n",
    "n, c, h, w = img.shape\n",
    "\n",
    "# 将所有图片的尺寸/边长循环 padding 为 SIZE 的倍数\n",
    "num_patch_h, num_patch_w = (h // patch_size) + 1, (w // patch_size) + 1\n",
    "# pad_left, pad_right, pad_top, pad_bottom] \n",
    "pad = nn.Pad2D(padding=[0, num_patch_w*patch_size-w, 0, num_patch_h*patch_size-h], mode='circular') \n",
    "img_pad = pad(img)\n",
    "\n",
    "# 等距切分为若干尺寸为 (SIZE, SIZE) 的 patch 并分别预测后拼接\n",
    "pre = np.zeros_like(img_pad.squeeze(0))\n",
    "with paddle.no_grad():\n",
    "    for i in range(num_patch_h):\n",
    "        for j in range(num_patch_w):\n",
    "            # 当前用于预测的 patch 切片\n",
    "            img_pad_patch = img_pad[..., i*patch_size: (i+1)*patch_size, j*patch_size: (j+1)*patch_size]\n",
    "            # 推理期间，验证操作实施等权重水平翻转自集成，以发挥/中和模型对不同水印 pattern 的恢复效果\n",
    "            pre[..., i*patch_size: (i+1)*patch_size, j*patch_size: (j+1)*patch_size] += (model(img_pad_patch)[0].numpy() * 0.5 + paddle.flip(model(paddle.flip(img_pad_patch, [-1])), [-1])[0].numpy() * 0.5)\n",
    "                                                                                                \n",
    "# 切除不需要的预测结果, 还原尺寸为 (h, w) 的原输入样本\n",
    "pre = pre[..., :h, :w]\n",
    "```"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "- 所有推理逻辑均注释于 predict.py 文件。测试时的主要特点在于测试时数据增强 (TTA)。"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "scrolled": true,
    "tags": []
   },
   "outputs": [],
   "source": [
    "! mkdir results\n",
    "! mkdir testa"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "scrolled": true
   },
   "outputs": [],
   "source": [
    "# 解压 测试集A 到 testa 目录下（测试集B同理）\n",
    "! unzip data/data141714/watermark_testa_datasets.zip -d ./testa"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "scrolled": true
   },
   "outputs": [],
   "source": [
    "# 在测试集上预测\n",
    "! python predict.py ./testa/watermark_test_datasets/images results"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": []
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "请点击[此处](https://ai.baidu.com/docs#/AIStudio_Project_Notebook/a38e5576)查看本环境基本用法.  <br>\n",
    "Please click [here ](https://ai.baidu.com/docs#/AIStudio_Project_Notebook/a38e5576) for more detailed instructions. "
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "py35-paddle1.2.0"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.7.4"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 4
}