Related/单样本微调给ChatGLM2注入知识.ipynb

{
  "nbformat": 4,
  "nbformat_minor": 0,
  "metadata": {
    "colab": {
      "provenance": [],
      "collapsed_sections": [
        "J6W0lPsT1c-G",
        "Jy3J5fvW3R_G",
        "a_dhguZd4fKw"
      ],
      "machine_shape": "hm",
      "gpuType": "A100"
    },
    "kernelspec": {
      "name": "python3",
      "display_name": "Python 3"
    },
    "language_info": {
      "name": "python"
    },
    "accelerator": "GPU",
    "widgets": {
      "application/vnd.jupyter.widget-state+json": {
        "a2e32e131e0647028723d43810781f50": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "HBoxModel",
          "model_module_version": "1.5.0",
          "state": {
            "_dom_classes": [],
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "HBoxModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/controls",
            "_view_module_version": "1.5.0",
            "_view_name": "HBoxView",
            "box_style": "",
            "children": [
              "IPY_MODEL_24466079b7384ecc9c0c80949aa4f5e5",
              "IPY_MODEL_b2f00c912b1f4e2abd856e010a64fbc2",
              "IPY_MODEL_cbcc16a54bbb4c63aca716edb64468af"
            ],
            "layout": "IPY_MODEL_a5bc2072474c4f67890b13179aeae52d"
          }
        },
        "24466079b7384ecc9c0c80949aa4f5e5": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "HTMLModel",
          "model_module_version": "1.5.0",
          "state": {
            "_dom_classes": [],
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "HTMLModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/controls",
            "_view_module_version": "1.5.0",
            "_view_name": "HTMLView",
            "description": "",
            "description_tooltip": null,
            "layout": "IPY_MODEL_1e288ce8a387483894087aec932744eb",
            "placeholder": "​",
            "style": "IPY_MODEL_8f2a6645d2e8424997a442d9a7eab426",
            "value": "Downloading (…)lve/main/config.json: 100%"
          }
        },
        "b2f00c912b1f4e2abd856e010a64fbc2": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "FloatProgressModel",
          "model_module_version": "1.5.0",
          "state": {
            "_dom_classes": [],
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "FloatProgressModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/controls",
            "_view_module_version": "1.5.0",
            "_view_name": "ProgressView",
            "bar_style": "success",
            "description": "",
            "description_tooltip": null,
            "layout": "IPY_MODEL_dbca5b0fbba140a3bc9400ad172d97f3",
            "max": 1223,
            "min": 0,
            "orientation": "horizontal",
            "style": "IPY_MODEL_abb1e521c00c4c0a8f3c68e6b4439fd3",
            "value": 1223
          }
        },
        "cbcc16a54bbb4c63aca716edb64468af": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "HTMLModel",
          "model_module_version": "1.5.0",
          "state": {
            "_dom_classes": [],
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "HTMLModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/controls",
            "_view_module_version": "1.5.0",
            "_view_name": "HTMLView",
            "description": "",
            "description_tooltip": null,
            "layout": "IPY_MODEL_18728a8d5b064bdfaf1311506c32cf4c",
            "placeholder": "​",
            "style": "IPY_MODEL_7258c692bb8f42c397c487bb70356877",
            "value": " 1.22k/1.22k [00:00&lt;00:00, 110kB/s]"
          }
        },
        "a5bc2072474c4f67890b13179aeae52d": {
          "model_module": "@jupyter-widgets/base",
          "model_name": "LayoutModel",
          "model_module_version": "1.2.0",
          "state": {
            "_model_module": "@jupyter-widgets/base",
            "_model_module_version": "1.2.0",
            "_model_name": "LayoutModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "LayoutView",
            "align_content": null,
            "align_items": null,
            "align_self": null,
            "border": null,
            "bottom": null,
            "display": null,
            "flex": null,
            "flex_flow": null,
            "grid_area": null,
            "grid_auto_columns": null,
            "grid_auto_flow": null,
            "grid_auto_rows": null,
            "grid_column": null,
            "grid_gap": null,
            "grid_row": null,
            "grid_template_areas": null,
            "grid_template_columns": null,
            "grid_template_rows": null,
            "height": null,
            "justify_content": null,
            "justify_items": null,
            "left": null,
            "margin": null,
            "max_height": null,
            "max_width": null,
            "min_height": null,
            "min_width": null,
            "object_fit": null,
            "object_position": null,
            "order": null,
            "overflow": null,
            "overflow_x": null,
            "overflow_y": null,
            "padding": null,
            "right": null,
            "top": null,
            "visibility": null,
            "width": null
          }
        },
        "1e288ce8a387483894087aec932744eb": {
          "model_module": "@jupyter-widgets/base",
          "model_name": "LayoutModel",
          "model_module_version": "1.2.0",
          "state": {
            "_model_module": "@jupyter-widgets/base",
            "_model_module_version": "1.2.0",
            "_model_name": "LayoutModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "LayoutView",
            "align_content": null,
            "align_items": null,
            "align_self": null,
            "border": null,
            "bottom": null,
            "display": null,
            "flex": null,
            "flex_flow": null,
            "grid_area": null,
            "grid_auto_columns": null,
            "grid_auto_flow": null,
            "grid_auto_rows": null,
            "grid_column": null,
            "grid_gap": null,
            "grid_row": null,
            "grid_template_areas": null,
            "grid_template_columns": null,
            "grid_template_rows": null,
            "height": null,
            "justify_content": null,
            "justify_items": null,
            "left": null,
            "margin": null,
            "max_height": null,
            "max_width": null,
            "min_height": null,
            "min_width": null,
            "object_fit": null,
            "object_position": null,
            "order": null,
            "overflow": null,
            "overflow_x": null,
            "overflow_y": null,
            "padding": null,
            "right": null,
            "top": null,
            "visibility": null,
            "width": null
          }
        },
        "8f2a6645d2e8424997a442d9a7eab426": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "DescriptionStyleModel",
          "model_module_version": "1.5.0",
          "state": {
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "DescriptionStyleModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "StyleView",
            "description_width": ""
          }
        },
        "dbca5b0fbba140a3bc9400ad172d97f3": {
          "model_module": "@jupyter-widgets/base",
          "model_name": "LayoutModel",
          "model_module_version": "1.2.0",
          "state": {
            "_model_module": "@jupyter-widgets/base",
            "_model_module_version": "1.2.0",
            "_model_name": "LayoutModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "LayoutView",
            "align_content": null,
            "align_items": null,
            "align_self": null,
            "border": null,
            "bottom": null,
            "display": null,
            "flex": null,
            "flex_flow": null,
            "grid_area": null,
            "grid_auto_columns": null,
            "grid_auto_flow": null,
            "grid_auto_rows": null,
            "grid_column": null,
            "grid_gap": null,
            "grid_row": null,
            "grid_template_areas": null,
            "grid_template_columns": null,
            "grid_template_rows": null,
            "height": null,
            "justify_content": null,
            "justify_items": null,
            "left": null,
            "margin": null,
            "max_height": null,
            "max_width": null,
            "min_height": null,
            "min_width": null,
            "object_fit": null,
            "object_position": null,
            "order": null,
            "overflow": null,
            "overflow_x": null,
            "overflow_y": null,
            "padding": null,
            "right": null,
            "top": null,
            "visibility": null,
            "width": null
          }
        },
        "abb1e521c00c4c0a8f3c68e6b4439fd3": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "ProgressStyleModel",
          "model_module_version": "1.5.0",
          "state": {
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "ProgressStyleModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "StyleView",
            "bar_color": null,
            "description_width": ""
          }
        },
        "18728a8d5b064bdfaf1311506c32cf4c": {
          "model_module": "@jupyter-widgets/base",
          "model_name": "LayoutModel",
          "model_module_version": "1.2.0",
          "state": {
            "_model_module": "@jupyter-widgets/base",
            "_model_module_version": "1.2.0",
            "_model_name": "LayoutModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "LayoutView",
            "align_content": null,
            "align_items": null,
            "align_self": null,
            "border": null,
            "bottom": null,
            "display": null,
            "flex": null,
            "flex_flow": null,
            "grid_area": null,
            "grid_auto_columns": null,
            "grid_auto_flow": null,
            "grid_auto_rows": null,
            "grid_column": null,
            "grid_gap": null,
            "grid_row": null,
            "grid_template_areas": null,
            "grid_template_columns": null,
            "grid_template_rows": null,
            "height": null,
            "justify_content": null,
            "justify_items": null,
            "left": null,
            "margin": null,
            "max_height": null,
            "max_width": null,
            "min_height": null,
            "min_width": null,
            "object_fit": null,
            "object_position": null,
            "order": null,
            "overflow": null,
            "overflow_x": null,
            "overflow_y": null,
            "padding": null,
            "right": null,
            "top": null,
            "visibility": null,
            "width": null
          }
        },
        "7258c692bb8f42c397c487bb70356877": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "DescriptionStyleModel",
          "model_module_version": "1.5.0",
          "state": {
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "DescriptionStyleModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "StyleView",
            "description_width": ""
          }
        },
        "012e1efa0aad4946be8c565778ca1e1d": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "HBoxModel",
          "model_module_version": "1.5.0",
          "state": {
            "_dom_classes": [],
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "HBoxModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/controls",
            "_view_module_version": "1.5.0",
            "_view_name": "HBoxView",
            "box_style": "",
            "children": [
              "IPY_MODEL_e9df530a75df4017a60ee049638ea8d5",
              "IPY_MODEL_416bcd88823a4745b49bfd4576ff851b",
              "IPY_MODEL_362c33fd726b41e7bad6df86744b7811"
            ],
            "layout": "IPY_MODEL_9968224047494a6c971f69f4eee57f11"
          }
        },
        "e9df530a75df4017a60ee049638ea8d5": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "HTMLModel",
          "model_module_version": "1.5.0",
          "state": {
            "_dom_classes": [],
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "HTMLModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/controls",
            "_view_module_version": "1.5.0",
            "_view_name": "HTMLView",
            "description": "",
            "description_tooltip": null,
            "layout": "IPY_MODEL_59e590c4fb1f4857acaea38256f1a3b3",
            "placeholder": "​",
            "style": "IPY_MODEL_7310888be31d4261a6fc95505da2d135",
            "value": "Downloading (…)iguration_chatglm.py: 100%"
          }
        },
        "416bcd88823a4745b49bfd4576ff851b": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "FloatProgressModel",
          "model_module_version": "1.5.0",
          "state": {
            "_dom_classes": [],
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "FloatProgressModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/controls",
            "_view_module_version": "1.5.0",
            "_view_name": "ProgressView",
            "bar_style": "success",
            "description": "",
            "description_tooltip": null,
            "layout": "IPY_MODEL_339c90f3e7a84d31a94687a315ecef94",
            "max": 2246,
            "min": 0,
            "orientation": "horizontal",
            "style": "IPY_MODEL_d6782fec37f24daf916a7e94710ee2a1",
            "value": 2246
          }
        },
        "362c33fd726b41e7bad6df86744b7811": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "HTMLModel",
          "model_module_version": "1.5.0",
          "state": {
            "_dom_classes": [],
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "HTMLModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/controls",
            "_view_module_version": "1.5.0",
            "_view_name": "HTMLView",
            "description": "",
            "description_tooltip": null,
            "layout": "IPY_MODEL_2f268a6613d241d783ed66a5c18cab48",
            "placeholder": "​",
            "style": "IPY_MODEL_b56a458d618f4057b12a14a82b05f048",
            "value": " 2.25k/2.25k [00:00&lt;00:00, 197kB/s]"
          }
        },
        "9968224047494a6c971f69f4eee57f11": {
          "model_module": "@jupyter-widgets/base",
          "model_name": "LayoutModel",
          "model_module_version": "1.2.0",
          "state": {
            "_model_module": "@jupyter-widgets/base",
            "_model_module_version": "1.2.0",
            "_model_name": "LayoutModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "LayoutView",
            "align_content": null,
            "align_items": null,
            "align_self": null,
            "border": null,
            "bottom": null,
            "display": null,
            "flex": null,
            "flex_flow": null,
            "grid_area": null,
            "grid_auto_columns": null,
            "grid_auto_flow": null,
            "grid_auto_rows": null,
            "grid_column": null,
            "grid_gap": null,
            "grid_row": null,
            "grid_template_areas": null,
            "grid_template_columns": null,
            "grid_template_rows": null,
            "height": null,
            "justify_content": null,
            "justify_items": null,
            "left": null,
            "margin": null,
            "max_height": null,
            "max_width": null,
            "min_height": null,
            "min_width": null,
            "object_fit": null,
            "object_position": null,
            "order": null,
            "overflow": null,
            "overflow_x": null,
            "overflow_y": null,
            "padding": null,
            "right": null,
            "top": null,
            "visibility": null,
            "width": null
          }
        },
        "59e590c4fb1f4857acaea38256f1a3b3": {
          "model_module": "@jupyter-widgets/base",
          "model_name": "LayoutModel",
          "model_module_version": "1.2.0",
          "state": {
            "_model_module": "@jupyter-widgets/base",
            "_model_module_version": "1.2.0",
            "_model_name": "LayoutModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "LayoutView",
            "align_content": null,
            "align_items": null,
            "align_self": null,
            "border": null,
            "bottom": null,
            "display": null,
            "flex": null,
            "flex_flow": null,
            "grid_area": null,
            "grid_auto_columns": null,
            "grid_auto_flow": null,
            "grid_auto_rows": null,
            "grid_column": null,
            "grid_gap": null,
            "grid_row": null,
            "grid_template_areas": null,
            "grid_template_columns": null,
            "grid_template_rows": null,
            "height": null,
            "justify_content": null,
            "justify_items": null,
            "left": null,
            "margin": null,
            "max_height": null,
            "max_width": null,
            "min_height": null,
            "min_width": null,
            "object_fit": null,
            "object_position": null,
            "order": null,
            "overflow": null,
            "overflow_x": null,
            "overflow_y": null,
            "padding": null,
            "right": null,
            "top": null,
            "visibility": null,
            "width": null
          }
        },
        "7310888be31d4261a6fc95505da2d135": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "DescriptionStyleModel",
          "model_module_version": "1.5.0",
          "state": {
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "DescriptionStyleModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "StyleView",
            "description_width": ""
          }
        },
        "339c90f3e7a84d31a94687a315ecef94": {
          "model_module": "@jupyter-widgets/base",
          "model_name": "LayoutModel",
          "model_module_version": "1.2.0",
          "state": {
            "_model_module": "@jupyter-widgets/base",
            "_model_module_version": "1.2.0",
            "_model_name": "LayoutModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "LayoutView",
            "align_content": null,
            "align_items": null,
            "align_self": null,
            "border": null,
            "bottom": null,
            "display": null,
            "flex": null,
            "flex_flow": null,
            "grid_area": null,
            "grid_auto_columns": null,
            "grid_auto_flow": null,
            "grid_auto_rows": null,
            "grid_column": null,
            "grid_gap": null,
            "grid_row": null,
            "grid_template_areas": null,
            "grid_template_columns": null,
            "grid_template_rows": null,
            "height": null,
            "justify_content": null,
            "justify_items": null,
            "left": null,
            "margin": null,
            "max_height": null,
            "max_width": null,
            "min_height": null,
            "min_width": null,
            "object_fit": null,
            "object_position": null,
            "order": null,
            "overflow": null,
            "overflow_x": null,
            "overflow_y": null,
            "padding": null,
            "right": null,
            "top": null,
            "visibility": null,
            "width": null
          }
        },
        "d6782fec37f24daf916a7e94710ee2a1": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "ProgressStyleModel",
          "model_module_version": "1.5.0",
          "state": {
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "ProgressStyleModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "StyleView",
            "bar_color": null,
            "description_width": ""
          }
        },
        "2f268a6613d241d783ed66a5c18cab48": {
          "model_module": "@jupyter-widgets/base",
          "model_name": "LayoutModel",
          "model_module_version": "1.2.0",
          "state": {
            "_model_module": "@jupyter-widgets/base",
            "_model_module_version": "1.2.0",
            "_model_name": "LayoutModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "LayoutView",
            "align_content": null,
            "align_items": null,
            "align_self": null,
            "border": null,
            "bottom": null,
            "display": null,
            "flex": null,
            "flex_flow": null,
            "grid_area": null,
            "grid_auto_columns": null,
            "grid_auto_flow": null,
            "grid_auto_rows": null,
            "grid_column": null,
            "grid_gap": null,
            "grid_row": null,
            "grid_template_areas": null,
            "grid_template_columns": null,
            "grid_template_rows": null,
            "height": null,
            "justify_content": null,
            "justify_items": null,
            "left": null,
            "margin": null,
            "max_height": null,
            "max_width": null,
            "min_height": null,
            "min_width": null,
            "object_fit": null,
            "object_position": null,
            "order": null,
            "overflow": null,
            "overflow_x": null,
            "overflow_y": null,
            "padding": null,
            "right": null,
            "top": null,
            "visibility": null,
            "width": null
          }
        },
        "b56a458d618f4057b12a14a82b05f048": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "DescriptionStyleModel",
          "model_module_version": "1.5.0",
          "state": {
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "DescriptionStyleModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "StyleView",
            "description_width": ""
          }
        },
        "4a56e10642724f7498f5ab079092353e": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "HBoxModel",
          "model_module_version": "1.5.0",
          "state": {
            "_dom_classes": [],
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "HBoxModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/controls",
            "_view_module_version": "1.5.0",
            "_view_name": "HBoxView",
            "box_style": "",
            "children": [
              "IPY_MODEL_f91eba59b49e4800ad38fd95e513ee9b",
              "IPY_MODEL_551472fece2249108d6d90f2cc18f826",
              "IPY_MODEL_6f39280c3ff64a33afcd7d93fc613cb9"
            ],
            "layout": "IPY_MODEL_63a9a52c1eb44c9782ae0effd2ca75a9"
          }
        },
        "f91eba59b49e4800ad38fd95e513ee9b": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "HTMLModel",
          "model_module_version": "1.5.0",
          "state": {
            "_dom_classes": [],
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "HTMLModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/controls",
            "_view_module_version": "1.5.0",
            "_view_name": "HTMLView",
            "description": "",
            "description_tooltip": null,
            "layout": "IPY_MODEL_efa7b35ae96a4624a25990bc115d785c",
            "placeholder": "​",
            "style": "IPY_MODEL_994cb43b8c6b4adfa62a872ad63d1735",
            "value": "Downloading (…)okenizer_config.json: 100%"
          }
        },
        "551472fece2249108d6d90f2cc18f826": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "FloatProgressModel",
          "model_module_version": "1.5.0",
          "state": {
            "_dom_classes": [],
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "FloatProgressModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/controls",
            "_view_module_version": "1.5.0",
            "_view_name": "ProgressView",
            "bar_style": "success",
            "description": "",
            "description_tooltip": null,
            "layout": "IPY_MODEL_a4cad29f70f0424b9036cf2ead8f7a23",
            "max": 244,
            "min": 0,
            "orientation": "horizontal",
            "style": "IPY_MODEL_f510fe1a9e3643ce8f7d02e1fcd54ae9",
            "value": 244
          }
        },
        "6f39280c3ff64a33afcd7d93fc613cb9": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "HTMLModel",
          "model_module_version": "1.5.0",
          "state": {
            "_dom_classes": [],
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "HTMLModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/controls",
            "_view_module_version": "1.5.0",
            "_view_name": "HTMLView",
            "description": "",
            "description_tooltip": null,
            "layout": "IPY_MODEL_62244056b89b4dbc8f6b9055d810bc00",
            "placeholder": "​",
            "style": "IPY_MODEL_744a91498a934e91b5d3563b343ad792",
            "value": " 244/244 [00:00&lt;00:00, 22.0kB/s]"
          }
        },
        "63a9a52c1eb44c9782ae0effd2ca75a9": {
          "model_module": "@jupyter-widgets/base",
          "model_name": "LayoutModel",
          "model_module_version": "1.2.0",
          "state": {
            "_model_module": "@jupyter-widgets/base",
            "_model_module_version": "1.2.0",
            "_model_name": "LayoutModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "LayoutView",
            "align_content": null,
            "align_items": null,
            "align_self": null,
            "border": null,
            "bottom": null,
            "display": null,
            "flex": null,
            "flex_flow": null,
            "grid_area": null,
            "grid_auto_columns": null,
            "grid_auto_flow": null,
            "grid_auto_rows": null,
            "grid_column": null,
            "grid_gap": null,
            "grid_row": null,
            "grid_template_areas": null,
            "grid_template_columns": null,
            "grid_template_rows": null,
            "height": null,
            "justify_content": null,
            "justify_items": null,
            "left": null,
            "margin": null,
            "max_height": null,
            "max_width": null,
            "min_height": null,
            "min_width": null,
            "object_fit": null,
            "object_position": null,
            "order": null,
            "overflow": null,
            "overflow_x": null,
            "overflow_y": null,
            "padding": null,
            "right": null,
            "top": null,
            "visibility": null,
            "width": null
          }
        },
        "efa7b35ae96a4624a25990bc115d785c": {
          "model_module": "@jupyter-widgets/base",
          "model_name": "LayoutModel",
          "model_module_version": "1.2.0",
          "state": {
            "_model_module": "@jupyter-widgets/base",
            "_model_module_version": "1.2.0",
            "_model_name": "LayoutModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "LayoutView",
            "align_content": null,
            "align_items": null,
            "align_self": null,
            "border": null,
            "bottom": null,
            "display": null,
            "flex": null,
            "flex_flow": null,
            "grid_area": null,
            "grid_auto_columns": null,
            "grid_auto_flow": null,
            "grid_auto_rows": null,
            "grid_column": null,
            "grid_gap": null,
            "grid_row": null,
            "grid_template_areas": null,
            "grid_template_columns": null,
            "grid_template_rows": null,
            "height": null,
            "justify_content": null,
            "justify_items": null,
            "left": null,
            "margin": null,
            "max_height": null,
            "max_width": null,
            "min_height": null,
            "min_width": null,
            "object_fit": null,
            "object_position": null,
            "order": null,
            "overflow": null,
            "overflow_x": null,
            "overflow_y": null,
            "padding": null,
            "right": null,
            "top": null,
            "visibility": null,
            "width": null
          }
        },
        "994cb43b8c6b4adfa62a872ad63d1735": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "DescriptionStyleModel",
          "model_module_version": "1.5.0",
          "state": {
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "DescriptionStyleModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "StyleView",
            "description_width": ""
          }
        },
        "a4cad29f70f0424b9036cf2ead8f7a23": {
          "model_module": "@jupyter-widgets/base",
          "model_name": "LayoutModel",
          "model_module_version": "1.2.0",
          "state": {
            "_model_module": "@jupyter-widgets/base",
            "_model_module_version": "1.2.0",
            "_model_name": "LayoutModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "LayoutView",
            "align_content": null,
            "align_items": null,
            "align_self": null,
            "border": null,
            "bottom": null,
            "display": null,
            "flex": null,
            "flex_flow": null,
            "grid_area": null,
            "grid_auto_columns": null,
            "grid_auto_flow": null,
            "grid_auto_rows": null,
            "grid_column": null,
            "grid_gap": null,
            "grid_row": null,
            "grid_template_areas": null,
            "grid_template_columns": null,
            "grid_template_rows": null,
            "height": null,
            "justify_content": null,
            "justify_items": null,
            "left": null,
            "margin": null,
            "max_height": null,
            "max_width": null,
            "min_height": null,
            "min_width": null,
            "object_fit": null,
            "object_position": null,
            "order": null,
            "overflow": null,
            "overflow_x": null,
            "overflow_y": null,
            "padding": null,
            "right": null,
            "top": null,
            "visibility": null,
            "width": null
          }
        },
        "f510fe1a9e3643ce8f7d02e1fcd54ae9": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "ProgressStyleModel",
          "model_module_version": "1.5.0",
          "state": {
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "ProgressStyleModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "StyleView",
            "bar_color": null,
            "description_width": ""
          }
        },
        "62244056b89b4dbc8f6b9055d810bc00": {
          "model_module": "@jupyter-widgets/base",
          "model_name": "LayoutModel",
          "model_module_version": "1.2.0",
          "state": {
            "_model_module": "@jupyter-widgets/base",
            "_model_module_version": "1.2.0",
            "_model_name": "LayoutModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "LayoutView",
            "align_content": null,
            "align_items": null,
            "align_self": null,
            "border": null,
            "bottom": null,
            "display": null,
            "flex": null,
            "flex_flow": null,
            "grid_area": null,
            "grid_auto_columns": null,
            "grid_auto_flow": null,
            "grid_auto_rows": null,
            "grid_column": null,
            "grid_gap": null,
            "grid_row": null,
            "grid_template_areas": null,
            "grid_template_columns": null,
            "grid_template_rows": null,
            "height": null,
            "justify_content": null,
            "justify_items": null,
            "left": null,
            "margin": null,
            "max_height": null,
            "max_width": null,
            "min_height": null,
            "min_width": null,
            "object_fit": null,
            "object_position": null,
            "order": null,
            "overflow": null,
            "overflow_x": null,
            "overflow_y": null,
            "padding": null,
            "right": null,
            "top": null,
            "visibility": null,
            "width": null
          }
        },
        "744a91498a934e91b5d3563b343ad792": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "DescriptionStyleModel",
          "model_module_version": "1.5.0",
          "state": {
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "DescriptionStyleModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "StyleView",
            "description_width": ""
          }
        },
        "cbb72d2f96934f33aaf5efc671b3bb16": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "HBoxModel",
          "model_module_version": "1.5.0",
          "state": {
            "_dom_classes": [],
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "HBoxModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/controls",
            "_view_module_version": "1.5.0",
            "_view_name": "HBoxView",
            "box_style": "",
            "children": [
              "IPY_MODEL_f19d596055334116aaa55413a3d0de2b",
              "IPY_MODEL_ae5a27fd0d1b4ef4ae4c166019342534",
              "IPY_MODEL_4d3cc2d6ec154bfe93e2dc546ad36e5b"
            ],
            "layout": "IPY_MODEL_753aa5dcc7f84b16863fc0958634c913"
          }
        },
        "f19d596055334116aaa55413a3d0de2b": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "HTMLModel",
          "model_module_version": "1.5.0",
          "state": {
            "_dom_classes": [],
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "HTMLModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/controls",
            "_view_module_version": "1.5.0",
            "_view_name": "HTMLView",
            "description": "",
            "description_tooltip": null,
            "layout": "IPY_MODEL_d5418a835e054a839ecfed355cd45648",
            "placeholder": "​",
            "style": "IPY_MODEL_636758ae082c4b65b9084c7025291701",
            "value": "Downloading (…)enization_chatglm.py: 100%"
          }
        },
        "ae5a27fd0d1b4ef4ae4c166019342534": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "FloatProgressModel",
          "model_module_version": "1.5.0",
          "state": {
            "_dom_classes": [],
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "FloatProgressModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/controls",
            "_view_module_version": "1.5.0",
            "_view_name": "ProgressView",
            "bar_style": "success",
            "description": "",
            "description_tooltip": null,
            "layout": "IPY_MODEL_d831a64501714b378bb8adc8c0d6aed0",
            "max": 9896,
            "min": 0,
            "orientation": "horizontal",
            "style": "IPY_MODEL_4bd29b0cd09148de8e3a8e85eee2abdd",
            "value": 9896
          }
        },
        "4d3cc2d6ec154bfe93e2dc546ad36e5b": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "HTMLModel",
          "model_module_version": "1.5.0",
          "state": {
            "_dom_classes": [],
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "HTMLModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/controls",
            "_view_module_version": "1.5.0",
            "_view_name": "HTMLView",
            "description": "",
            "description_tooltip": null,
            "layout": "IPY_MODEL_9947a6d1ffe7405490360df78afd255c",
            "placeholder": "​",
            "style": "IPY_MODEL_47ef6a2388fd4726b608e1570bc61433",
            "value": " 9.90k/9.90k [00:00&lt;00:00, 887kB/s]"
          }
        },
        "753aa5dcc7f84b16863fc0958634c913": {
          "model_module": "@jupyter-widgets/base",
          "model_name": "LayoutModel",
          "model_module_version": "1.2.0",
          "state": {
            "_model_module": "@jupyter-widgets/base",
            "_model_module_version": "1.2.0",
            "_model_name": "LayoutModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "LayoutView",
            "align_content": null,
            "align_items": null,
            "align_self": null,
            "border": null,
            "bottom": null,
            "display": null,
            "flex": null,
            "flex_flow": null,
            "grid_area": null,
            "grid_auto_columns": null,
            "grid_auto_flow": null,
            "grid_auto_rows": null,
            "grid_column": null,
            "grid_gap": null,
            "grid_row": null,
            "grid_template_areas": null,
            "grid_template_columns": null,
            "grid_template_rows": null,
            "height": null,
            "justify_content": null,
            "justify_items": null,
            "left": null,
            "margin": null,
            "max_height": null,
            "max_width": null,
            "min_height": null,
            "min_width": null,
            "object_fit": null,
            "object_position": null,
            "order": null,
            "overflow": null,
            "overflow_x": null,
            "overflow_y": null,
            "padding": null,
            "right": null,
            "top": null,
            "visibility": null,
            "width": null
          }
        },
        "d5418a835e054a839ecfed355cd45648": {
          "model_module": "@jupyter-widgets/base",
          "model_name": "LayoutModel",
          "model_module_version": "1.2.0",
          "state": {
            "_model_module": "@jupyter-widgets/base",
            "_model_module_version": "1.2.0",
            "_model_name": "LayoutModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "LayoutView",
            "align_content": null,
            "align_items": null,
            "align_self": null,
            "border": null,
            "bottom": null,
            "display": null,
            "flex": null,
            "flex_flow": null,
            "grid_area": null,
            "grid_auto_columns": null,
            "grid_auto_flow": null,
            "grid_auto_rows": null,
            "grid_column": null,
            "grid_gap": null,
            "grid_row": null,
            "grid_template_areas": null,
            "grid_template_columns": null,
            "grid_template_rows": null,
            "height": null,
            "justify_content": null,
            "justify_items": null,
            "left": null,
            "margin": null,
            "max_height": null,
            "max_width": null,
            "min_height": null,
            "min_width": null,
            "object_fit": null,
            "object_position": null,
            "order": null,
            "overflow": null,
            "overflow_x": null,
            "overflow_y": null,
            "padding": null,
            "right": null,
            "top": null,
            "visibility": null,
            "width": null
          }
        },
        "636758ae082c4b65b9084c7025291701": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "DescriptionStyleModel",
          "model_module_version": "1.5.0",
          "state": {
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "DescriptionStyleModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "StyleView",
            "description_width": ""
          }
        },
        "d831a64501714b378bb8adc8c0d6aed0": {
          "model_module": "@jupyter-widgets/base",
          "model_name": "LayoutModel",
          "model_module_version": "1.2.0",
          "state": {
            "_model_module": "@jupyter-widgets/base",
            "_model_module_version": "1.2.0",
            "_model_name": "LayoutModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "LayoutView",
            "align_content": null,
            "align_items": null,
            "align_self": null,
            "border": null,
            "bottom": null,
            "display": null,
            "flex": null,
            "flex_flow": null,
            "grid_area": null,
            "grid_auto_columns": null,
            "grid_auto_flow": null,
            "grid_auto_rows": null,
            "grid_column": null,
            "grid_gap": null,
            "grid_row": null,
            "grid_template_areas": null,
            "grid_template_columns": null,
            "grid_template_rows": null,
            "height": null,
            "justify_content": null,
            "justify_items": null,
            "left": null,
            "margin": null,
            "max_height": null,
            "max_width": null,
            "min_height": null,
            "min_width": null,
            "object_fit": null,
            "object_position": null,
            "order": null,
            "overflow": null,
            "overflow_x": null,
            "overflow_y": null,
            "padding": null,
            "right": null,
            "top": null,
            "visibility": null,
            "width": null
          }
        },
        "4bd29b0cd09148de8e3a8e85eee2abdd": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "ProgressStyleModel",
          "model_module_version": "1.5.0",
          "state": {
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "ProgressStyleModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "StyleView",
            "bar_color": null,
            "description_width": ""
          }
        },
        "9947a6d1ffe7405490360df78afd255c": {
          "model_module": "@jupyter-widgets/base",
          "model_name": "LayoutModel",
          "model_module_version": "1.2.0",
          "state": {
            "_model_module": "@jupyter-widgets/base",
            "_model_module_version": "1.2.0",
            "_model_name": "LayoutModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "LayoutView",
            "align_content": null,
            "align_items": null,
            "align_self": null,
            "border": null,
            "bottom": null,
            "display": null,
            "flex": null,
            "flex_flow": null,
            "grid_area": null,
            "grid_auto_columns": null,
            "grid_auto_flow": null,
            "grid_auto_rows": null,
            "grid_column": null,
            "grid_gap": null,
            "grid_row": null,
            "grid_template_areas": null,
            "grid_template_columns": null,
            "grid_template_rows": null,
            "height": null,
            "justify_content": null,
            "justify_items": null,
            "left": null,
            "margin": null,
            "max_height": null,
            "max_width": null,
            "min_height": null,
            "min_width": null,
            "object_fit": null,
            "object_position": null,
            "order": null,
            "overflow": null,
            "overflow_x": null,
            "overflow_y": null,
            "padding": null,
            "right": null,
            "top": null,
            "visibility": null,
            "width": null
          }
        },
        "47ef6a2388fd4726b608e1570bc61433": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "DescriptionStyleModel",
          "model_module_version": "1.5.0",
          "state": {
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "DescriptionStyleModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "StyleView",
            "description_width": ""
          }
        },
        "76a8a5cda1ae459283e3dcdbd430c8ca": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "HBoxModel",
          "model_module_version": "1.5.0",
          "state": {
            "_dom_classes": [],
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "HBoxModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/controls",
            "_view_module_version": "1.5.0",
            "_view_name": "HBoxView",
            "box_style": "",
            "children": [
              "IPY_MODEL_7de95de31ade43988eb03471685f0a1b",
              "IPY_MODEL_c6410d4ef46940ed8ce870ea9a2c8840",
              "IPY_MODEL_aa0d40071fd34a1e808980fde3c7bae8"
            ],
            "layout": "IPY_MODEL_afd194f1836c4c3a938ee5bbc1ce1dd6"
          }
        },
        "7de95de31ade43988eb03471685f0a1b": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "HTMLModel",
          "model_module_version": "1.5.0",
          "state": {
            "_dom_classes": [],
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "HTMLModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/controls",
            "_view_module_version": "1.5.0",
            "_view_name": "HTMLView",
            "description": "",
            "description_tooltip": null,
            "layout": "IPY_MODEL_68fbfbeddf354d96bddde7516acb99c5",
            "placeholder": "​",
            "style": "IPY_MODEL_a0836b7c94f64ecda7e23fb34972e5ef",
            "value": "Downloading tokenizer.model: 100%"
          }
        },
        "c6410d4ef46940ed8ce870ea9a2c8840": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "FloatProgressModel",
          "model_module_version": "1.5.0",
          "state": {
            "_dom_classes": [],
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "FloatProgressModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/controls",
            "_view_module_version": "1.5.0",
            "_view_name": "ProgressView",
            "bar_style": "success",
            "description": "",
            "description_tooltip": null,
            "layout": "IPY_MODEL_bdbb4d5e2e6d4652938d4e00519bf440",
            "max": 1018370,
            "min": 0,
            "orientation": "horizontal",
            "style": "IPY_MODEL_66eba329f64f49e597018a4199a4c9c1",
            "value": 1018370
          }
        },
        "aa0d40071fd34a1e808980fde3c7bae8": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "HTMLModel",
          "model_module_version": "1.5.0",
          "state": {
            "_dom_classes": [],
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "HTMLModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/controls",
            "_view_module_version": "1.5.0",
            "_view_name": "HTMLView",
            "description": "",
            "description_tooltip": null,
            "layout": "IPY_MODEL_91d835841b394ceb8cfdce481b38c6c0",
            "placeholder": "​",
            "style": "IPY_MODEL_a8000b01379a4f25b2c967ac2468b061",
            "value": " 1.02M/1.02M [00:00&lt;00:00, 33.7MB/s]"
          }
        },
        "afd194f1836c4c3a938ee5bbc1ce1dd6": {
          "model_module": "@jupyter-widgets/base",
          "model_name": "LayoutModel",
          "model_module_version": "1.2.0",
          "state": {
            "_model_module": "@jupyter-widgets/base",
            "_model_module_version": "1.2.0",
            "_model_name": "LayoutModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "LayoutView",
            "align_content": null,
            "align_items": null,
            "align_self": null,
            "border": null,
            "bottom": null,
            "display": null,
            "flex": null,
            "flex_flow": null,
            "grid_area": null,
            "grid_auto_columns": null,
            "grid_auto_flow": null,
            "grid_auto_rows": null,
            "grid_column": null,
            "grid_gap": null,
            "grid_row": null,
            "grid_template_areas": null,
            "grid_template_columns": null,
            "grid_template_rows": null,
            "height": null,
            "justify_content": null,
            "justify_items": null,
            "left": null,
            "margin": null,
            "max_height": null,
            "max_width": null,
            "min_height": null,
            "min_width": null,
            "object_fit": null,
            "object_position": null,
            "order": null,
            "overflow": null,
            "overflow_x": null,
            "overflow_y": null,
            "padding": null,
            "right": null,
            "top": null,
            "visibility": null,
            "width": null
          }
        },
        "68fbfbeddf354d96bddde7516acb99c5": {
          "model_module": "@jupyter-widgets/base",
          "model_name": "LayoutModel",
          "model_module_version": "1.2.0",
          "state": {
            "_model_module": "@jupyter-widgets/base",
            "_model_module_version": "1.2.0",
            "_model_name": "LayoutModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "LayoutView",
            "align_content": null,
            "align_items": null,
            "align_self": null,
            "border": null,
            "bottom": null,
            "display": null,
            "flex": null,
            "flex_flow": null,
            "grid_area": null,
            "grid_auto_columns": null,
            "grid_auto_flow": null,
            "grid_auto_rows": null,
            "grid_column": null,
            "grid_gap": null,
            "grid_row": null,
            "grid_template_areas": null,
            "grid_template_columns": null,
            "grid_template_rows": null,
            "height": null,
            "justify_content": null,
            "justify_items": null,
            "left": null,
            "margin": null,
            "max_height": null,
            "max_width": null,
            "min_height": null,
            "min_width": null,
            "object_fit": null,
            "object_position": null,
            "order": null,
            "overflow": null,
            "overflow_x": null,
            "overflow_y": null,
            "padding": null,
            "right": null,
            "top": null,
            "visibility": null,
            "width": null
          }
        },
        "a0836b7c94f64ecda7e23fb34972e5ef": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "DescriptionStyleModel",
          "model_module_version": "1.5.0",
          "state": {
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "DescriptionStyleModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "StyleView",
            "description_width": ""
          }
        },
        "bdbb4d5e2e6d4652938d4e00519bf440": {
          "model_module": "@jupyter-widgets/base",
          "model_name": "LayoutModel",
          "model_module_version": "1.2.0",
          "state": {
            "_model_module": "@jupyter-widgets/base",
            "_model_module_version": "1.2.0",
            "_model_name": "LayoutModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "LayoutView",
            "align_content": null,
            "align_items": null,
            "align_self": null,
            "border": null,
            "bottom": null,
            "display": null,
            "flex": null,
            "flex_flow": null,
            "grid_area": null,
            "grid_auto_columns": null,
            "grid_auto_flow": null,
            "grid_auto_rows": null,
            "grid_column": null,
            "grid_gap": null,
            "grid_row": null,
            "grid_template_areas": null,
            "grid_template_columns": null,
            "grid_template_rows": null,
            "height": null,
            "justify_content": null,
            "justify_items": null,
            "left": null,
            "margin": null,
            "max_height": null,
            "max_width": null,
            "min_height": null,
            "min_width": null,
            "object_fit": null,
            "object_position": null,
            "order": null,
            "overflow": null,
            "overflow_x": null,
            "overflow_y": null,
            "padding": null,
            "right": null,
            "top": null,
            "visibility": null,
            "width": null
          }
        },
        "66eba329f64f49e597018a4199a4c9c1": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "ProgressStyleModel",
          "model_module_version": "1.5.0",
          "state": {
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "ProgressStyleModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "StyleView",
            "bar_color": null,
            "description_width": ""
          }
        },
        "91d835841b394ceb8cfdce481b38c6c0": {
          "model_module": "@jupyter-widgets/base",
          "model_name": "LayoutModel",
          "model_module_version": "1.2.0",
          "state": {
            "_model_module": "@jupyter-widgets/base",
            "_model_module_version": "1.2.0",
            "_model_name": "LayoutModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "LayoutView",
            "align_content": null,
            "align_items": null,
            "align_self": null,
            "border": null,
            "bottom": null,
            "display": null,
            "flex": null,
            "flex_flow": null,
            "grid_area": null,
            "grid_auto_columns": null,
            "grid_auto_flow": null,
            "grid_auto_rows": null,
            "grid_column": null,
            "grid_gap": null,
            "grid_row": null,
            "grid_template_areas": null,
            "grid_template_columns": null,
            "grid_template_rows": null,
            "height": null,
            "justify_content": null,
            "justify_items": null,
            "left": null,
            "margin": null,
            "max_height": null,
            "max_width": null,
            "min_height": null,
            "min_width": null,
            "object_fit": null,
            "object_position": null,
            "order": null,
            "overflow": null,
            "overflow_x": null,
            "overflow_y": null,
            "padding": null,
            "right": null,
            "top": null,
            "visibility": null,
            "width": null
          }
        },
        "a8000b01379a4f25b2c967ac2468b061": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "DescriptionStyleModel",
          "model_module_version": "1.5.0",
          "state": {
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "DescriptionStyleModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "StyleView",
            "description_width": ""
          }
        },
        "59d7c1cc2cc5477fabcffa6efdaf26a5": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "HBoxModel",
          "model_module_version": "1.5.0",
          "state": {
            "_dom_classes": [],
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "HBoxModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/controls",
            "_view_module_version": "1.5.0",
            "_view_name": "HBoxView",
            "box_style": "",
            "children": [
              "IPY_MODEL_685293b3ebda471685f2a47d9cf7d4e1",
              "IPY_MODEL_63e9c15976ff486fa4655b5ae7990491",
              "IPY_MODEL_171d4a7bf5864013aabc562f67feb43f"
            ],
            "layout": "IPY_MODEL_5e192c4134074ad2b369748d5bb16c21"
          }
        },
        "685293b3ebda471685f2a47d9cf7d4e1": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "HTMLModel",
          "model_module_version": "1.5.0",
          "state": {
            "_dom_classes": [],
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "HTMLModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/controls",
            "_view_module_version": "1.5.0",
            "_view_name": "HTMLView",
            "description": "",
            "description_tooltip": null,
            "layout": "IPY_MODEL_872dbbee0d814dc79549da110cffc49a",
            "placeholder": "​",
            "style": "IPY_MODEL_ad6ad77aa50f435ab1d8dc8d846303ff",
            "value": "Downloading (…)/modeling_chatglm.py: 100%"
          }
        },
        "63e9c15976ff486fa4655b5ae7990491": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "FloatProgressModel",
          "model_module_version": "1.5.0",
          "state": {
            "_dom_classes": [],
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "FloatProgressModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/controls",
            "_view_module_version": "1.5.0",
            "_view_name": "ProgressView",
            "bar_style": "success",
            "description": "",
            "description_tooltip": null,
            "layout": "IPY_MODEL_440bb6c594814f509d4cb6dcb7f6e54f",
            "max": 50696,
            "min": 0,
            "orientation": "horizontal",
            "style": "IPY_MODEL_76d07cdfa8434d9b976dd0807b9a9b59",
            "value": 50696
          }
        },
        "171d4a7bf5864013aabc562f67feb43f": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "HTMLModel",
          "model_module_version": "1.5.0",
          "state": {
            "_dom_classes": [],
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "HTMLModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/controls",
            "_view_module_version": "1.5.0",
            "_view_name": "HTMLView",
            "description": "",
            "description_tooltip": null,
            "layout": "IPY_MODEL_c76d57fc0d6246fea713ba6f32315549",
            "placeholder": "​",
            "style": "IPY_MODEL_916c3edcb8de4dbcbc7a8287b5158694",
            "value": " 50.7k/50.7k [00:00&lt;00:00, 238kB/s]"
          }
        },
        "5e192c4134074ad2b369748d5bb16c21": {
          "model_module": "@jupyter-widgets/base",
          "model_name": "LayoutModel",
          "model_module_version": "1.2.0",
          "state": {
            "_model_module": "@jupyter-widgets/base",
            "_model_module_version": "1.2.0",
            "_model_name": "LayoutModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "LayoutView",
            "align_content": null,
            "align_items": null,
            "align_self": null,
            "border": null,
            "bottom": null,
            "display": null,
            "flex": null,
            "flex_flow": null,
            "grid_area": null,
            "grid_auto_columns": null,
            "grid_auto_flow": null,
            "grid_auto_rows": null,
            "grid_column": null,
            "grid_gap": null,
            "grid_row": null,
            "grid_template_areas": null,
            "grid_template_columns": null,
            "grid_template_rows": null,
            "height": null,
            "justify_content": null,
            "justify_items": null,
            "left": null,
            "margin": null,
            "max_height": null,
            "max_width": null,
            "min_height": null,
            "min_width": null,
            "object_fit": null,
            "object_position": null,
            "order": null,
            "overflow": null,
            "overflow_x": null,
            "overflow_y": null,
            "padding": null,
            "right": null,
            "top": null,
            "visibility": null,
            "width": null
          }
        },
        "872dbbee0d814dc79549da110cffc49a": {
          "model_module": "@jupyter-widgets/base",
          "model_name": "LayoutModel",
          "model_module_version": "1.2.0",
          "state": {
            "_model_module": "@jupyter-widgets/base",
            "_model_module_version": "1.2.0",
            "_model_name": "LayoutModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "LayoutView",
            "align_content": null,
            "align_items": null,
            "align_self": null,
            "border": null,
            "bottom": null,
            "display": null,
            "flex": null,
            "flex_flow": null,
            "grid_area": null,
            "grid_auto_columns": null,
            "grid_auto_flow": null,
            "grid_auto_rows": null,
            "grid_column": null,
            "grid_gap": null,
            "grid_row": null,
            "grid_template_areas": null,
            "grid_template_columns": null,
            "grid_template_rows": null,
            "height": null,
            "justify_content": null,
            "justify_items": null,
            "left": null,
            "margin": null,
            "max_height": null,
            "max_width": null,
            "min_height": null,
            "min_width": null,
            "object_fit": null,
            "object_position": null,
            "order": null,
            "overflow": null,
            "overflow_x": null,
            "overflow_y": null,
            "padding": null,
            "right": null,
            "top": null,
            "visibility": null,
            "width": null
          }
        },
        "ad6ad77aa50f435ab1d8dc8d846303ff": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "DescriptionStyleModel",
          "model_module_version": "1.5.0",
          "state": {
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "DescriptionStyleModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "StyleView",
            "description_width": ""
          }
        },
        "440bb6c594814f509d4cb6dcb7f6e54f": {
          "model_module": "@jupyter-widgets/base",
          "model_name": "LayoutModel",
          "model_module_version": "1.2.0",
          "state": {
            "_model_module": "@jupyter-widgets/base",
            "_model_module_version": "1.2.0",
            "_model_name": "LayoutModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "LayoutView",
            "align_content": null,
            "align_items": null,
            "align_self": null,
            "border": null,
            "bottom": null,
            "display": null,
            "flex": null,
            "flex_flow": null,
            "grid_area": null,
            "grid_auto_columns": null,
            "grid_auto_flow": null,
            "grid_auto_rows": null,
            "grid_column": null,
            "grid_gap": null,
            "grid_row": null,
            "grid_template_areas": null,
            "grid_template_columns": null,
            "grid_template_rows": null,
            "height": null,
            "justify_content": null,
            "justify_items": null,
            "left": null,
            "margin": null,
            "max_height": null,
            "max_width": null,
            "min_height": null,
            "min_width": null,
            "object_fit": null,
            "object_position": null,
            "order": null,
            "overflow": null,
            "overflow_x": null,
            "overflow_y": null,
            "padding": null,
            "right": null,
            "top": null,
            "visibility": null,
            "width": null
          }
        },
        "76d07cdfa8434d9b976dd0807b9a9b59": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "ProgressStyleModel",
          "model_module_version": "1.5.0",
          "state": {
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "ProgressStyleModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "StyleView",
            "bar_color": null,
            "description_width": ""
          }
        },
        "c76d57fc0d6246fea713ba6f32315549": {
          "model_module": "@jupyter-widgets/base",
          "model_name": "LayoutModel",
          "model_module_version": "1.2.0",
          "state": {
            "_model_module": "@jupyter-widgets/base",
            "_model_module_version": "1.2.0",
            "_model_name": "LayoutModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "LayoutView",
            "align_content": null,
            "align_items": null,
            "align_self": null,
            "border": null,
            "bottom": null,
            "display": null,
            "flex": null,
            "flex_flow": null,
            "grid_area": null,
            "grid_auto_columns": null,
            "grid_auto_flow": null,
            "grid_auto_rows": null,
            "grid_column": null,
            "grid_gap": null,
            "grid_row": null,
            "grid_template_areas": null,
            "grid_template_columns": null,
            "grid_template_rows": null,
            "height": null,
            "justify_content": null,
            "justify_items": null,
            "left": null,
            "margin": null,
            "max_height": null,
            "max_width": null,
            "min_height": null,
            "min_width": null,
            "object_fit": null,
            "object_position": null,
            "order": null,
            "overflow": null,
            "overflow_x": null,
            "overflow_y": null,
            "padding": null,
            "right": null,
            "top": null,
            "visibility": null,
            "width": null
          }
        },
        "916c3edcb8de4dbcbc7a8287b5158694": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "DescriptionStyleModel",
          "model_module_version": "1.5.0",
          "state": {
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "DescriptionStyleModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "StyleView",
            "description_width": ""
          }
        },
        "443c906966a74ec1a29236630e646774": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "HBoxModel",
          "model_module_version": "1.5.0",
          "state": {
            "_dom_classes": [],
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "HBoxModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/controls",
            "_view_module_version": "1.5.0",
            "_view_name": "HBoxView",
            "box_style": "",
            "children": [
              "IPY_MODEL_9d65edf76d04418fbcb60af5c4c58ace",
              "IPY_MODEL_ed5870ac88744c5ab4d26bf24404291d",
              "IPY_MODEL_68db923d9b034487859a37f9c1922e25"
            ],
            "layout": "IPY_MODEL_4c33553088bb4971801d3868216d0b8f"
          }
        },
        "9d65edf76d04418fbcb60af5c4c58ace": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "HTMLModel",
          "model_module_version": "1.5.0",
          "state": {
            "_dom_classes": [],
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "HTMLModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/controls",
            "_view_module_version": "1.5.0",
            "_view_name": "HTMLView",
            "description": "",
            "description_tooltip": null,
            "layout": "IPY_MODEL_fce4a156b78742d39d366c41179c96ac",
            "placeholder": "​",
            "style": "IPY_MODEL_a58b411911e244c4a67131db4652b3f1",
            "value": "Downloading (…)main/quantization.py: 100%"
          }
        },
        "ed5870ac88744c5ab4d26bf24404291d": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "FloatProgressModel",
          "model_module_version": "1.5.0",
          "state": {
            "_dom_classes": [],
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "FloatProgressModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/controls",
            "_view_module_version": "1.5.0",
            "_view_name": "ProgressView",
            "bar_style": "success",
            "description": "",
            "description_tooltip": null,
            "layout": "IPY_MODEL_bf04053881ca496d89c33e86f84d7951",
            "max": 14692,
            "min": 0,
            "orientation": "horizontal",
            "style": "IPY_MODEL_68875644fffe4ba9b91a353fd9439e08",
            "value": 14692
          }
        },
        "68db923d9b034487859a37f9c1922e25": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "HTMLModel",
          "model_module_version": "1.5.0",
          "state": {
            "_dom_classes": [],
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "HTMLModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/controls",
            "_view_module_version": "1.5.0",
            "_view_name": "HTMLView",
            "description": "",
            "description_tooltip": null,
            "layout": "IPY_MODEL_2a977c56dae44acdabec8fe25c6a88f3",
            "placeholder": "​",
            "style": "IPY_MODEL_c48c2b7d9d4247efb3b8f25c67285c1a",
            "value": " 14.7k/14.7k [00:00&lt;00:00, 1.36MB/s]"
          }
        },
        "4c33553088bb4971801d3868216d0b8f": {
          "model_module": "@jupyter-widgets/base",
          "model_name": "LayoutModel",
          "model_module_version": "1.2.0",
          "state": {
            "_model_module": "@jupyter-widgets/base",
            "_model_module_version": "1.2.0",
            "_model_name": "LayoutModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "LayoutView",
            "align_content": null,
            "align_items": null,
            "align_self": null,
            "border": null,
            "bottom": null,
            "display": null,
            "flex": null,
            "flex_flow": null,
            "grid_area": null,
            "grid_auto_columns": null,
            "grid_auto_flow": null,
            "grid_auto_rows": null,
            "grid_column": null,
            "grid_gap": null,
            "grid_row": null,
            "grid_template_areas": null,
            "grid_template_columns": null,
            "grid_template_rows": null,
            "height": null,
            "justify_content": null,
            "justify_items": null,
            "left": null,
            "margin": null,
            "max_height": null,
            "max_width": null,
            "min_height": null,
            "min_width": null,
            "object_fit": null,
            "object_position": null,
            "order": null,
            "overflow": null,
            "overflow_x": null,
            "overflow_y": null,
            "padding": null,
            "right": null,
            "top": null,
            "visibility": null,
            "width": null
          }
        },
        "fce4a156b78742d39d366c41179c96ac": {
          "model_module": "@jupyter-widgets/base",
          "model_name": "LayoutModel",
          "model_module_version": "1.2.0",
          "state": {
            "_model_module": "@jupyter-widgets/base",
            "_model_module_version": "1.2.0",
            "_model_name": "LayoutModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "LayoutView",
            "align_content": null,
            "align_items": null,
            "align_self": null,
            "border": null,
            "bottom": null,
            "display": null,
            "flex": null,
            "flex_flow": null,
            "grid_area": null,
            "grid_auto_columns": null,
            "grid_auto_flow": null,
            "grid_auto_rows": null,
            "grid_column": null,
            "grid_gap": null,
            "grid_row": null,
            "grid_template_areas": null,
            "grid_template_columns": null,
            "grid_template_rows": null,
            "height": null,
            "justify_content": null,
            "justify_items": null,
            "left": null,
            "margin": null,
            "max_height": null,
            "max_width": null,
            "min_height": null,
            "min_width": null,
            "object_fit": null,
            "object_position": null,
            "order": null,
            "overflow": null,
            "overflow_x": null,
            "overflow_y": null,
            "padding": null,
            "right": null,
            "top": null,
            "visibility": null,
            "width": null
          }
        },
        "a58b411911e244c4a67131db4652b3f1": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "DescriptionStyleModel",
          "model_module_version": "1.5.0",
          "state": {
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "DescriptionStyleModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "StyleView",
            "description_width": ""
          }
        },
        "bf04053881ca496d89c33e86f84d7951": {
          "model_module": "@jupyter-widgets/base",
          "model_name": "LayoutModel",
          "model_module_version": "1.2.0",
          "state": {
            "_model_module": "@jupyter-widgets/base",
            "_model_module_version": "1.2.0",
            "_model_name": "LayoutModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "LayoutView",
            "align_content": null,
            "align_items": null,
            "align_self": null,
            "border": null,
            "bottom": null,
            "display": null,
            "flex": null,
            "flex_flow": null,
            "grid_area": null,
            "grid_auto_columns": null,
            "grid_auto_flow": null,
            "grid_auto_rows": null,
            "grid_column": null,
            "grid_gap": null,
            "grid_row": null,
            "grid_template_areas": null,
            "grid_template_columns": null,
            "grid_template_rows": null,
            "height": null,
            "justify_content": null,
            "justify_items": null,
            "left": null,
            "margin": null,
            "max_height": null,
            "max_width": null,
            "min_height": null,
            "min_width": null,
            "object_fit": null,
            "object_position": null,
            "order": null,
            "overflow": null,
            "overflow_x": null,
            "overflow_y": null,
            "padding": null,
            "right": null,
            "top": null,
            "visibility": null,
            "width": null
          }
        },
        "68875644fffe4ba9b91a353fd9439e08": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "ProgressStyleModel",
          "model_module_version": "1.5.0",
          "state": {
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "ProgressStyleModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "StyleView",
            "bar_color": null,
            "description_width": ""
          }
        },
        "2a977c56dae44acdabec8fe25c6a88f3": {
          "model_module": "@jupyter-widgets/base",
          "model_name": "LayoutModel",
          "model_module_version": "1.2.0",
          "state": {
            "_model_module": "@jupyter-widgets/base",
            "_model_module_version": "1.2.0",
            "_model_name": "LayoutModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "LayoutView",
            "align_content": null,
            "align_items": null,
            "align_self": null,
            "border": null,
            "bottom": null,
            "display": null,
            "flex": null,
            "flex_flow": null,
            "grid_area": null,
            "grid_auto_columns": null,
            "grid_auto_flow": null,
            "grid_auto_rows": null,
            "grid_column": null,
            "grid_gap": null,
            "grid_row": null,
            "grid_template_areas": null,
            "grid_template_columns": null,
            "grid_template_rows": null,
            "height": null,
            "justify_content": null,
            "justify_items": null,
            "left": null,
            "margin": null,
            "max_height": null,
            "max_width": null,
            "min_height": null,
            "min_width": null,
            "object_fit": null,
            "object_position": null,
            "order": null,
            "overflow": null,
            "overflow_x": null,
            "overflow_y": null,
            "padding": null,
            "right": null,
            "top": null,
            "visibility": null,
            "width": null
          }
        },
        "c48c2b7d9d4247efb3b8f25c67285c1a": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "DescriptionStyleModel",
          "model_module_version": "1.5.0",
          "state": {
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "DescriptionStyleModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "StyleView",
            "description_width": ""
          }
        },
        "c4b2d9acd2224a98a64e95641129938d": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "HBoxModel",
          "model_module_version": "1.5.0",
          "state": {
            "_dom_classes": [],
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "HBoxModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/controls",
            "_view_module_version": "1.5.0",
            "_view_name": "HBoxView",
            "box_style": "",
            "children": [
              "IPY_MODEL_02e3e55fcf1942549c2e9bccf903c16f",
              "IPY_MODEL_70b3f544a16045d589ae7430345a311f",
              "IPY_MODEL_163bea237222473084acd5fd1800efe9"
            ],
            "layout": "IPY_MODEL_92cc630c42c24c779de66211c316b99d"
          }
        },
        "02e3e55fcf1942549c2e9bccf903c16f": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "HTMLModel",
          "model_module_version": "1.5.0",
          "state": {
            "_dom_classes": [],
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "HTMLModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/controls",
            "_view_module_version": "1.5.0",
            "_view_name": "HTMLView",
            "description": "",
            "description_tooltip": null,
            "layout": "IPY_MODEL_f05df5e58d3943bfb358f11e0467c30c",
            "placeholder": "​",
            "style": "IPY_MODEL_b6b5c15809b640aa8af345b199d21ba8",
            "value": "Downloading (…)model.bin.index.json: 100%"
          }
        },
        "70b3f544a16045d589ae7430345a311f": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "FloatProgressModel",
          "model_module_version": "1.5.0",
          "state": {
            "_dom_classes": [],
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "FloatProgressModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/controls",
            "_view_module_version": "1.5.0",
            "_view_name": "ProgressView",
            "bar_style": "success",
            "description": "",
            "description_tooltip": null,
            "layout": "IPY_MODEL_53039514658a41bea0b2d96ad5a46787",
            "max": 20438,
            "min": 0,
            "orientation": "horizontal",
            "style": "IPY_MODEL_13f9b47f37cf441cbe012087af1a2ac7",
            "value": 20438
          }
        },
        "163bea237222473084acd5fd1800efe9": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "HTMLModel",
          "model_module_version": "1.5.0",
          "state": {
            "_dom_classes": [],
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "HTMLModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/controls",
            "_view_module_version": "1.5.0",
            "_view_name": "HTMLView",
            "description": "",
            "description_tooltip": null,
            "layout": "IPY_MODEL_bf75ece1bc3146849ea0629f3445301d",
            "placeholder": "​",
            "style": "IPY_MODEL_06b720e8c2ad4a5395b960301d1f1ffd",
            "value": " 20.4k/20.4k [00:00&lt;00:00, 1.52MB/s]"
          }
        },
        "92cc630c42c24c779de66211c316b99d": {
          "model_module": "@jupyter-widgets/base",
          "model_name": "LayoutModel",
          "model_module_version": "1.2.0",
          "state": {
            "_model_module": "@jupyter-widgets/base",
            "_model_module_version": "1.2.0",
            "_model_name": "LayoutModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "LayoutView",
            "align_content": null,
            "align_items": null,
            "align_self": null,
            "border": null,
            "bottom": null,
            "display": null,
            "flex": null,
            "flex_flow": null,
            "grid_area": null,
            "grid_auto_columns": null,
            "grid_auto_flow": null,
            "grid_auto_rows": null,
            "grid_column": null,
            "grid_gap": null,
            "grid_row": null,
            "grid_template_areas": null,
            "grid_template_columns": null,
            "grid_template_rows": null,
            "height": null,
            "justify_content": null,
            "justify_items": null,
            "left": null,
            "margin": null,
            "max_height": null,
            "max_width": null,
            "min_height": null,
            "min_width": null,
            "object_fit": null,
            "object_position": null,
            "order": null,
            "overflow": null,
            "overflow_x": null,
            "overflow_y": null,
            "padding": null,
            "right": null,
            "top": null,
            "visibility": null,
            "width": null
          }
        },
        "f05df5e58d3943bfb358f11e0467c30c": {
          "model_module": "@jupyter-widgets/base",
          "model_name": "LayoutModel",
          "model_module_version": "1.2.0",
          "state": {
            "_model_module": "@jupyter-widgets/base",
            "_model_module_version": "1.2.0",
            "_model_name": "LayoutModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "LayoutView",
            "align_content": null,
            "align_items": null,
            "align_self": null,
            "border": null,
            "bottom": null,
            "display": null,
            "flex": null,
            "flex_flow": null,
            "grid_area": null,
            "grid_auto_columns": null,
            "grid_auto_flow": null,
            "grid_auto_rows": null,
            "grid_column": null,
            "grid_gap": null,
            "grid_row": null,
            "grid_template_areas": null,
            "grid_template_columns": null,
            "grid_template_rows": null,
            "height": null,
            "justify_content": null,
            "justify_items": null,
            "left": null,
            "margin": null,
            "max_height": null,
            "max_width": null,
            "min_height": null,
            "min_width": null,
            "object_fit": null,
            "object_position": null,
            "order": null,
            "overflow": null,
            "overflow_x": null,
            "overflow_y": null,
            "padding": null,
            "right": null,
            "top": null,
            "visibility": null,
            "width": null
          }
        },
        "b6b5c15809b640aa8af345b199d21ba8": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "DescriptionStyleModel",
          "model_module_version": "1.5.0",
          "state": {
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "DescriptionStyleModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "StyleView",
            "description_width": ""
          }
        },
        "53039514658a41bea0b2d96ad5a46787": {
          "model_module": "@jupyter-widgets/base",
          "model_name": "LayoutModel",
          "model_module_version": "1.2.0",
          "state": {
            "_model_module": "@jupyter-widgets/base",
            "_model_module_version": "1.2.0",
            "_model_name": "LayoutModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "LayoutView",
            "align_content": null,
            "align_items": null,
            "align_self": null,
            "border": null,
            "bottom": null,
            "display": null,
            "flex": null,
            "flex_flow": null,
            "grid_area": null,
            "grid_auto_columns": null,
            "grid_auto_flow": null,
            "grid_auto_rows": null,
            "grid_column": null,
            "grid_gap": null,
            "grid_row": null,
            "grid_template_areas": null,
            "grid_template_columns": null,
            "grid_template_rows": null,
            "height": null,
            "justify_content": null,
            "justify_items": null,
            "left": null,
            "margin": null,
            "max_height": null,
            "max_width": null,
            "min_height": null,
            "min_width": null,
            "object_fit": null,
            "object_position": null,
            "order": null,
            "overflow": null,
            "overflow_x": null,
            "overflow_y": null,
            "padding": null,
            "right": null,
            "top": null,
            "visibility": null,
            "width": null
          }
        },
        "13f9b47f37cf441cbe012087af1a2ac7": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "ProgressStyleModel",
          "model_module_version": "1.5.0",
          "state": {
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "ProgressStyleModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "StyleView",
            "bar_color": null,
            "description_width": ""
          }
        },
        "bf75ece1bc3146849ea0629f3445301d": {
          "model_module": "@jupyter-widgets/base",
          "model_name": "LayoutModel",
          "model_module_version": "1.2.0",
          "state": {
            "_model_module": "@jupyter-widgets/base",
            "_model_module_version": "1.2.0",
            "_model_name": "LayoutModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "LayoutView",
            "align_content": null,
            "align_items": null,
            "align_self": null,
            "border": null,
            "bottom": null,
            "display": null,
            "flex": null,
            "flex_flow": null,
            "grid_area": null,
            "grid_auto_columns": null,
            "grid_auto_flow": null,
            "grid_auto_rows": null,
            "grid_column": null,
            "grid_gap": null,
            "grid_row": null,
            "grid_template_areas": null,
            "grid_template_columns": null,
            "grid_template_rows": null,
            "height": null,
            "justify_content": null,
            "justify_items": null,
            "left": null,
            "margin": null,
            "max_height": null,
            "max_width": null,
            "min_height": null,
            "min_width": null,
            "object_fit": null,
            "object_position": null,
            "order": null,
            "overflow": null,
            "overflow_x": null,
            "overflow_y": null,
            "padding": null,
            "right": null,
            "top": null,
            "visibility": null,
            "width": null
          }
        },
        "06b720e8c2ad4a5395b960301d1f1ffd": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "DescriptionStyleModel",
          "model_module_version": "1.5.0",
          "state": {
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "DescriptionStyleModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "StyleView",
            "description_width": ""
          }
        },
        "3ca728fb3e294384898b531bbfa126d6": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "HBoxModel",
          "model_module_version": "1.5.0",
          "state": {
            "_dom_classes": [],
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "HBoxModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/controls",
            "_view_module_version": "1.5.0",
            "_view_name": "HBoxView",
            "box_style": "",
            "children": [
              "IPY_MODEL_23090f6cf40c45c38ad0c58aec75a383",
              "IPY_MODEL_a59ef5f659064e3d8a1507d191c75397",
              "IPY_MODEL_9f14d803ab704f37a111fe17644a298b"
            ],
            "layout": "IPY_MODEL_dcb9e3a53db9435a9e6a98e9b193cee4"
          }
        },
        "23090f6cf40c45c38ad0c58aec75a383": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "HTMLModel",
          "model_module_version": "1.5.0",
          "state": {
            "_dom_classes": [],
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "HTMLModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/controls",
            "_view_module_version": "1.5.0",
            "_view_name": "HTMLView",
            "description": "",
            "description_tooltip": null,
            "layout": "IPY_MODEL_ff56016623a4414bbbd0c83c07618ce7",
            "placeholder": "​",
            "style": "IPY_MODEL_9d15210862ec41729ef999b64faa4310",
            "value": "Downloading shards: 100%"
          }
        },
        "a59ef5f659064e3d8a1507d191c75397": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "FloatProgressModel",
          "model_module_version": "1.5.0",
          "state": {
            "_dom_classes": [],
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "FloatProgressModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/controls",
            "_view_module_version": "1.5.0",
            "_view_name": "ProgressView",
            "bar_style": "success",
            "description": "",
            "description_tooltip": null,
            "layout": "IPY_MODEL_d9ae600340eb4386a0850b0c4e851d03",
            "max": 7,
            "min": 0,
            "orientation": "horizontal",
            "style": "IPY_MODEL_ac1ff0c5130f4b3cb80e7b78f6454bc7",
            "value": 7
          }
        },
        "9f14d803ab704f37a111fe17644a298b": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "HTMLModel",
          "model_module_version": "1.5.0",
          "state": {
            "_dom_classes": [],
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "HTMLModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/controls",
            "_view_module_version": "1.5.0",
            "_view_name": "HTMLView",
            "description": "",
            "description_tooltip": null,
            "layout": "IPY_MODEL_2b76d5330e5a40699c087882f31e9af5",
            "placeholder": "​",
            "style": "IPY_MODEL_0e8ae3805c4e426899db73c25b9653b3",
            "value": " 7/7 [00:37&lt;00:00,  5.07s/it]"
          }
        },
        "dcb9e3a53db9435a9e6a98e9b193cee4": {
          "model_module": "@jupyter-widgets/base",
          "model_name": "LayoutModel",
          "model_module_version": "1.2.0",
          "state": {
            "_model_module": "@jupyter-widgets/base",
            "_model_module_version": "1.2.0",
            "_model_name": "LayoutModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "LayoutView",
            "align_content": null,
            "align_items": null,
            "align_self": null,
            "border": null,
            "bottom": null,
            "display": null,
            "flex": null,
            "flex_flow": null,
            "grid_area": null,
            "grid_auto_columns": null,
            "grid_auto_flow": null,
            "grid_auto_rows": null,
            "grid_column": null,
            "grid_gap": null,
            "grid_row": null,
            "grid_template_areas": null,
            "grid_template_columns": null,
            "grid_template_rows": null,
            "height": null,
            "justify_content": null,
            "justify_items": null,
            "left": null,
            "margin": null,
            "max_height": null,
            "max_width": null,
            "min_height": null,
            "min_width": null,
            "object_fit": null,
            "object_position": null,
            "order": null,
            "overflow": null,
            "overflow_x": null,
            "overflow_y": null,
            "padding": null,
            "right": null,
            "top": null,
            "visibility": null,
            "width": null
          }
        },
        "ff56016623a4414bbbd0c83c07618ce7": {
          "model_module": "@jupyter-widgets/base",
          "model_name": "LayoutModel",
          "model_module_version": "1.2.0",
          "state": {
            "_model_module": "@jupyter-widgets/base",
            "_model_module_version": "1.2.0",
            "_model_name": "LayoutModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "LayoutView",
            "align_content": null,
            "align_items": null,
            "align_self": null,
            "border": null,
            "bottom": null,
            "display": null,
            "flex": null,
            "flex_flow": null,
            "grid_area": null,
            "grid_auto_columns": null,
            "grid_auto_flow": null,
            "grid_auto_rows": null,
            "grid_column": null,
            "grid_gap": null,
            "grid_row": null,
            "grid_template_areas": null,
            "grid_template_columns": null,
            "grid_template_rows": null,
            "height": null,
            "justify_content": null,
            "justify_items": null,
            "left": null,
            "margin": null,
            "max_height": null,
            "max_width": null,
            "min_height": null,
            "min_width": null,
            "object_fit": null,
            "object_position": null,
            "order": null,
            "overflow": null,
            "overflow_x": null,
            "overflow_y": null,
            "padding": null,
            "right": null,
            "top": null,
            "visibility": null,
            "width": null
          }
        },
        "9d15210862ec41729ef999b64faa4310": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "DescriptionStyleModel",
          "model_module_version": "1.5.0",
          "state": {
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "DescriptionStyleModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "StyleView",
            "description_width": ""
          }
        },
        "d9ae600340eb4386a0850b0c4e851d03": {
          "model_module": "@jupyter-widgets/base",
          "model_name": "LayoutModel",
          "model_module_version": "1.2.0",
          "state": {
            "_model_module": "@jupyter-widgets/base",
            "_model_module_version": "1.2.0",
            "_model_name": "LayoutModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "LayoutView",
            "align_content": null,
            "align_items": null,
            "align_self": null,
            "border": null,
            "bottom": null,
            "display": null,
            "flex": null,
            "flex_flow": null,
            "grid_area": null,
            "grid_auto_columns": null,
            "grid_auto_flow": null,
            "grid_auto_rows": null,
            "grid_column": null,
            "grid_gap": null,
            "grid_row": null,
            "grid_template_areas": null,
            "grid_template_columns": null,
            "grid_template_rows": null,
            "height": null,
            "justify_content": null,
            "justify_items": null,
            "left": null,
            "margin": null,
            "max_height": null,
            "max_width": null,
            "min_height": null,
            "min_width": null,
            "object_fit": null,
            "object_position": null,
            "order": null,
            "overflow": null,
            "overflow_x": null,
            "overflow_y": null,
            "padding": null,
            "right": null,
            "top": null,
            "visibility": null,
            "width": null
          }
        },
        "ac1ff0c5130f4b3cb80e7b78f6454bc7": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "ProgressStyleModel",
          "model_module_version": "1.5.0",
          "state": {
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "ProgressStyleModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "StyleView",
            "bar_color": null,
            "description_width": ""
          }
        },
        "2b76d5330e5a40699c087882f31e9af5": {
          "model_module": "@jupyter-widgets/base",
          "model_name": "LayoutModel",
          "model_module_version": "1.2.0",
          "state": {
            "_model_module": "@jupyter-widgets/base",
            "_model_module_version": "1.2.0",
            "_model_name": "LayoutModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "LayoutView",
            "align_content": null,
            "align_items": null,
            "align_self": null,
            "border": null,
            "bottom": null,
            "display": null,
            "flex": null,
            "flex_flow": null,
            "grid_area": null,
            "grid_auto_columns": null,
            "grid_auto_flow": null,
            "grid_auto_rows": null,
            "grid_column": null,
            "grid_gap": null,
            "grid_row": null,
            "grid_template_areas": null,
            "grid_template_columns": null,
            "grid_template_rows": null,
            "height": null,
            "justify_content": null,
            "justify_items": null,
            "left": null,
            "margin": null,
            "max_height": null,
            "max_width": null,
            "min_height": null,
            "min_width": null,
            "object_fit": null,
            "object_position": null,
            "order": null,
            "overflow": null,
            "overflow_x": null,
            "overflow_y": null,
            "padding": null,
            "right": null,
            "top": null,
            "visibility": null,
            "width": null
          }
        },
        "0e8ae3805c4e426899db73c25b9653b3": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "DescriptionStyleModel",
          "model_module_version": "1.5.0",
          "state": {
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "DescriptionStyleModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "StyleView",
            "description_width": ""
          }
        },
        "48d6957605774e9d8986e011966f9782": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "HBoxModel",
          "model_module_version": "1.5.0",
          "state": {
            "_dom_classes": [],
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "HBoxModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/controls",
            "_view_module_version": "1.5.0",
            "_view_name": "HBoxView",
            "box_style": "",
            "children": [
              "IPY_MODEL_a7824e6e2ed04a1684add98e8c42b491",
              "IPY_MODEL_cc2498d5ee104894bf5180d1656ffacb",
              "IPY_MODEL_5843041c6a5d417f8c574c4d8cfa5993"
            ],
            "layout": "IPY_MODEL_678bbef0353b4be3b19e4a16de1aa904"
          }
        },
        "a7824e6e2ed04a1684add98e8c42b491": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "HTMLModel",
          "model_module_version": "1.5.0",
          "state": {
            "_dom_classes": [],
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "HTMLModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/controls",
            "_view_module_version": "1.5.0",
            "_view_name": "HTMLView",
            "description": "",
            "description_tooltip": null,
            "layout": "IPY_MODEL_b99da94ec80e4387a988eb6fd9f15ed1",
            "placeholder": "​",
            "style": "IPY_MODEL_8c8ec1431afb4cef9884c358a1382827",
            "value": "Downloading (…)l-00001-of-00007.bin: 100%"
          }
        },
        "cc2498d5ee104894bf5180d1656ffacb": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "FloatProgressModel",
          "model_module_version": "1.5.0",
          "state": {
            "_dom_classes": [],
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "FloatProgressModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/controls",
            "_view_module_version": "1.5.0",
            "_view_name": "ProgressView",
            "bar_style": "success",
            "description": "",
            "description_tooltip": null,
            "layout": "IPY_MODEL_be8bdf59868c4f1e9482c5cddc817af4",
            "max": 1827780615,
            "min": 0,
            "orientation": "horizontal",
            "style": "IPY_MODEL_a74b15fd0b09451591decd7e1b37e2ad",
            "value": 1827780615
          }
        },
        "5843041c6a5d417f8c574c4d8cfa5993": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "HTMLModel",
          "model_module_version": "1.5.0",
          "state": {
            "_dom_classes": [],
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "HTMLModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/controls",
            "_view_module_version": "1.5.0",
            "_view_name": "HTMLView",
            "description": "",
            "description_tooltip": null,
            "layout": "IPY_MODEL_8505c14ed4f44891948c30df4d21f46e",
            "placeholder": "​",
            "style": "IPY_MODEL_1f63a4bf31ea41d6a69f2c2e7556e100",
            "value": " 1.83G/1.83G [00:03&lt;00:00, 497MB/s]"
          }
        },
        "678bbef0353b4be3b19e4a16de1aa904": {
          "model_module": "@jupyter-widgets/base",
          "model_name": "LayoutModel",
          "model_module_version": "1.2.0",
          "state": {
            "_model_module": "@jupyter-widgets/base",
            "_model_module_version": "1.2.0",
            "_model_name": "LayoutModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "LayoutView",
            "align_content": null,
            "align_items": null,
            "align_self": null,
            "border": null,
            "bottom": null,
            "display": null,
            "flex": null,
            "flex_flow": null,
            "grid_area": null,
            "grid_auto_columns": null,
            "grid_auto_flow": null,
            "grid_auto_rows": null,
            "grid_column": null,
            "grid_gap": null,
            "grid_row": null,
            "grid_template_areas": null,
            "grid_template_columns": null,
            "grid_template_rows": null,
            "height": null,
            "justify_content": null,
            "justify_items": null,
            "left": null,
            "margin": null,
            "max_height": null,
            "max_width": null,
            "min_height": null,
            "min_width": null,
            "object_fit": null,
            "object_position": null,
            "order": null,
            "overflow": null,
            "overflow_x": null,
            "overflow_y": null,
            "padding": null,
            "right": null,
            "top": null,
            "visibility": null,
            "width": null
          }
        },
        "b99da94ec80e4387a988eb6fd9f15ed1": {
          "model_module": "@jupyter-widgets/base",
          "model_name": "LayoutModel",
          "model_module_version": "1.2.0",
          "state": {
            "_model_module": "@jupyter-widgets/base",
            "_model_module_version": "1.2.0",
            "_model_name": "LayoutModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "LayoutView",
            "align_content": null,
            "align_items": null,
            "align_self": null,
            "border": null,
            "bottom": null,
            "display": null,
            "flex": null,
            "flex_flow": null,
            "grid_area": null,
            "grid_auto_columns": null,
            "grid_auto_flow": null,
            "grid_auto_rows": null,
            "grid_column": null,
            "grid_gap": null,
            "grid_row": null,
            "grid_template_areas": null,
            "grid_template_columns": null,
            "grid_template_rows": null,
            "height": null,
            "justify_content": null,
            "justify_items": null,
            "left": null,
            "margin": null,
            "max_height": null,
            "max_width": null,
            "min_height": null,
            "min_width": null,
            "object_fit": null,
            "object_position": null,
            "order": null,
            "overflow": null,
            "overflow_x": null,
            "overflow_y": null,
            "padding": null,
            "right": null,
            "top": null,
            "visibility": null,
            "width": null
          }
        },
        "8c8ec1431afb4cef9884c358a1382827": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "DescriptionStyleModel",
          "model_module_version": "1.5.0",
          "state": {
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "DescriptionStyleModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "StyleView",
            "description_width": ""
          }
        },
        "be8bdf59868c4f1e9482c5cddc817af4": {
          "model_module": "@jupyter-widgets/base",
          "model_name": "LayoutModel",
          "model_module_version": "1.2.0",
          "state": {
            "_model_module": "@jupyter-widgets/base",
            "_model_module_version": "1.2.0",
            "_model_name": "LayoutModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "LayoutView",
            "align_content": null,
            "align_items": null,
            "align_self": null,
            "border": null,
            "bottom": null,
            "display": null,
            "flex": null,
            "flex_flow": null,
            "grid_area": null,
            "grid_auto_columns": null,
            "grid_auto_flow": null,
            "grid_auto_rows": null,
            "grid_column": null,
            "grid_gap": null,
            "grid_row": null,
            "grid_template_areas": null,
            "grid_template_columns": null,
            "grid_template_rows": null,
            "height": null,
            "justify_content": null,
            "justify_items": null,
            "left": null,
            "margin": null,
            "max_height": null,
            "max_width": null,
            "min_height": null,
            "min_width": null,
            "object_fit": null,
            "object_position": null,
            "order": null,
            "overflow": null,
            "overflow_x": null,
            "overflow_y": null,
            "padding": null,
            "right": null,
            "top": null,
            "visibility": null,
            "width": null
          }
        },
        "a74b15fd0b09451591decd7e1b37e2ad": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "ProgressStyleModel",
          "model_module_version": "1.5.0",
          "state": {
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "ProgressStyleModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "StyleView",
            "bar_color": null,
            "description_width": ""
          }
        },
        "8505c14ed4f44891948c30df4d21f46e": {
          "model_module": "@jupyter-widgets/base",
          "model_name": "LayoutModel",
          "model_module_version": "1.2.0",
          "state": {
            "_model_module": "@jupyter-widgets/base",
            "_model_module_version": "1.2.0",
            "_model_name": "LayoutModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "LayoutView",
            "align_content": null,
            "align_items": null,
            "align_self": null,
            "border": null,
            "bottom": null,
            "display": null,
            "flex": null,
            "flex_flow": null,
            "grid_area": null,
            "grid_auto_columns": null,
            "grid_auto_flow": null,
            "grid_auto_rows": null,
            "grid_column": null,
            "grid_gap": null,
            "grid_row": null,
            "grid_template_areas": null,
            "grid_template_columns": null,
            "grid_template_rows": null,
            "height": null,
            "justify_content": null,
            "justify_items": null,
            "left": null,
            "margin": null,
            "max_height": null,
            "max_width": null,
            "min_height": null,
            "min_width": null,
            "object_fit": null,
            "object_position": null,
            "order": null,
            "overflow": null,
            "overflow_x": null,
            "overflow_y": null,
            "padding": null,
            "right": null,
            "top": null,
            "visibility": null,
            "width": null
          }
        },
        "1f63a4bf31ea41d6a69f2c2e7556e100": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "DescriptionStyleModel",
          "model_module_version": "1.5.0",
          "state": {
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "DescriptionStyleModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "StyleView",
            "description_width": ""
          }
        },
        "935bcfdf1ece41828e370350a108ae80": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "HBoxModel",
          "model_module_version": "1.5.0",
          "state": {
            "_dom_classes": [],
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "HBoxModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/controls",
            "_view_module_version": "1.5.0",
            "_view_name": "HBoxView",
            "box_style": "",
            "children": [
              "IPY_MODEL_da82407f81b143d2b6fb5aca858c33f7",
              "IPY_MODEL_a7b0fcd4e0194708925e283c6d0cf079",
              "IPY_MODEL_c6e8c5c435604c0fa75577d4043b9a11"
            ],
            "layout": "IPY_MODEL_35f19bc8a4174f478b1a14b7dd3fe97e"
          }
        },
        "da82407f81b143d2b6fb5aca858c33f7": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "HTMLModel",
          "model_module_version": "1.5.0",
          "state": {
            "_dom_classes": [],
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "HTMLModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/controls",
            "_view_module_version": "1.5.0",
            "_view_name": "HTMLView",
            "description": "",
            "description_tooltip": null,
            "layout": "IPY_MODEL_d6e333d3d406480ab8b5233ff1613ff9",
            "placeholder": "​",
            "style": "IPY_MODEL_0c06a6d5c3874b94afe965166cd6bef7",
            "value": "Downloading (…)l-00002-of-00007.bin: 100%"
          }
        },
        "a7b0fcd4e0194708925e283c6d0cf079": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "FloatProgressModel",
          "model_module_version": "1.5.0",
          "state": {
            "_dom_classes": [],
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "FloatProgressModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/controls",
            "_view_module_version": "1.5.0",
            "_view_name": "ProgressView",
            "bar_style": "success",
            "description": "",
            "description_tooltip": null,
            "layout": "IPY_MODEL_562ca7b6967148a7a3e92ce75240c187",
            "max": 1968299005,
            "min": 0,
            "orientation": "horizontal",
            "style": "IPY_MODEL_36aa8840fca84c2b9bfbbbc3979085f4",
            "value": 1968299005
          }
        },
        "c6e8c5c435604c0fa75577d4043b9a11": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "HTMLModel",
          "model_module_version": "1.5.0",
          "state": {
            "_dom_classes": [],
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "HTMLModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/controls",
            "_view_module_version": "1.5.0",
            "_view_name": "HTMLView",
            "description": "",
            "description_tooltip": null,
            "layout": "IPY_MODEL_fc061e9c519c421bbf08d6a974b31ad8",
            "placeholder": "​",
            "style": "IPY_MODEL_438843783dfb4a83abfc25123cb7bafd",
            "value": " 1.97G/1.97G [00:06&lt;00:00, 444MB/s]"
          }
        },
        "35f19bc8a4174f478b1a14b7dd3fe97e": {
          "model_module": "@jupyter-widgets/base",
          "model_name": "LayoutModel",
          "model_module_version": "1.2.0",
          "state": {
            "_model_module": "@jupyter-widgets/base",
            "_model_module_version": "1.2.0",
            "_model_name": "LayoutModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "LayoutView",
            "align_content": null,
            "align_items": null,
            "align_self": null,
            "border": null,
            "bottom": null,
            "display": null,
            "flex": null,
            "flex_flow": null,
            "grid_area": null,
            "grid_auto_columns": null,
            "grid_auto_flow": null,
            "grid_auto_rows": null,
            "grid_column": null,
            "grid_gap": null,
            "grid_row": null,
            "grid_template_areas": null,
            "grid_template_columns": null,
            "grid_template_rows": null,
            "height": null,
            "justify_content": null,
            "justify_items": null,
            "left": null,
            "margin": null,
            "max_height": null,
            "max_width": null,
            "min_height": null,
            "min_width": null,
            "object_fit": null,
            "object_position": null,
            "order": null,
            "overflow": null,
            "overflow_x": null,
            "overflow_y": null,
            "padding": null,
            "right": null,
            "top": null,
            "visibility": null,
            "width": null
          }
        },
        "d6e333d3d406480ab8b5233ff1613ff9": {
          "model_module": "@jupyter-widgets/base",
          "model_name": "LayoutModel",
          "model_module_version": "1.2.0",
          "state": {
            "_model_module": "@jupyter-widgets/base",
            "_model_module_version": "1.2.0",
            "_model_name": "LayoutModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "LayoutView",
            "align_content": null,
            "align_items": null,
            "align_self": null,
            "border": null,
            "bottom": null,
            "display": null,
            "flex": null,
            "flex_flow": null,
            "grid_area": null,
            "grid_auto_columns": null,
            "grid_auto_flow": null,
            "grid_auto_rows": null,
            "grid_column": null,
            "grid_gap": null,
            "grid_row": null,
            "grid_template_areas": null,
            "grid_template_columns": null,
            "grid_template_rows": null,
            "height": null,
            "justify_content": null,
            "justify_items": null,
            "left": null,
            "margin": null,
            "max_height": null,
            "max_width": null,
            "min_height": null,
            "min_width": null,
            "object_fit": null,
            "object_position": null,
            "order": null,
            "overflow": null,
            "overflow_x": null,
            "overflow_y": null,
            "padding": null,
            "right": null,
            "top": null,
            "visibility": null,
            "width": null
          }
        },
        "0c06a6d5c3874b94afe965166cd6bef7": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "DescriptionStyleModel",
          "model_module_version": "1.5.0",
          "state": {
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "DescriptionStyleModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "StyleView",
            "description_width": ""
          }
        },
        "562ca7b6967148a7a3e92ce75240c187": {
          "model_module": "@jupyter-widgets/base",
          "model_name": "LayoutModel",
          "model_module_version": "1.2.0",
          "state": {
            "_model_module": "@jupyter-widgets/base",
            "_model_module_version": "1.2.0",
            "_model_name": "LayoutModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "LayoutView",
            "align_content": null,
            "align_items": null,
            "align_self": null,
            "border": null,
            "bottom": null,
            "display": null,
            "flex": null,
            "flex_flow": null,
            "grid_area": null,
            "grid_auto_columns": null,
            "grid_auto_flow": null,
            "grid_auto_rows": null,
            "grid_column": null,
            "grid_gap": null,
            "grid_row": null,
            "grid_template_areas": null,
            "grid_template_columns": null,
            "grid_template_rows": null,
            "height": null,
            "justify_content": null,
            "justify_items": null,
            "left": null,
            "margin": null,
            "max_height": null,
            "max_width": null,
            "min_height": null,
            "min_width": null,
            "object_fit": null,
            "object_position": null,
            "order": null,
            "overflow": null,
            "overflow_x": null,
            "overflow_y": null,
            "padding": null,
            "right": null,
            "top": null,
            "visibility": null,
            "width": null
          }
        },
        "36aa8840fca84c2b9bfbbbc3979085f4": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "ProgressStyleModel",
          "model_module_version": "1.5.0",
          "state": {
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "ProgressStyleModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "StyleView",
            "bar_color": null,
            "description_width": ""
          }
        },
        "fc061e9c519c421bbf08d6a974b31ad8": {
          "model_module": "@jupyter-widgets/base",
          "model_name": "LayoutModel",
          "model_module_version": "1.2.0",
          "state": {
            "_model_module": "@jupyter-widgets/base",
            "_model_module_version": "1.2.0",
            "_model_name": "LayoutModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "LayoutView",
            "align_content": null,
            "align_items": null,
            "align_self": null,
            "border": null,
            "bottom": null,
            "display": null,
            "flex": null,
            "flex_flow": null,
            "grid_area": null,
            "grid_auto_columns": null,
            "grid_auto_flow": null,
            "grid_auto_rows": null,
            "grid_column": null,
            "grid_gap": null,
            "grid_row": null,
            "grid_template_areas": null,
            "grid_template_columns": null,
            "grid_template_rows": null,
            "height": null,
            "justify_content": null,
            "justify_items": null,
            "left": null,
            "margin": null,
            "max_height": null,
            "max_width": null,
            "min_height": null,
            "min_width": null,
            "object_fit": null,
            "object_position": null,
            "order": null,
            "overflow": null,
            "overflow_x": null,
            "overflow_y": null,
            "padding": null,
            "right": null,
            "top": null,
            "visibility": null,
            "width": null
          }
        },
        "438843783dfb4a83abfc25123cb7bafd": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "DescriptionStyleModel",
          "model_module_version": "1.5.0",
          "state": {
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "DescriptionStyleModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "StyleView",
            "description_width": ""
          }
        },
        "4dd100439c28414f906ff07e71c38de8": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "HBoxModel",
          "model_module_version": "1.5.0",
          "state": {
            "_dom_classes": [],
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "HBoxModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/controls",
            "_view_module_version": "1.5.0",
            "_view_name": "HBoxView",
            "box_style": "",
            "children": [
              "IPY_MODEL_f2105781417245938508ba82c2bb43ea",
              "IPY_MODEL_dd7395e72e7f4120b7d1e2dc4b00626a",
              "IPY_MODEL_51a9c4b74aa4445dbecd0f11e48cc9e8"
            ],
            "layout": "IPY_MODEL_feac1c0042d7419c87f7151032b7fd28"
          }
        },
        "f2105781417245938508ba82c2bb43ea": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "HTMLModel",
          "model_module_version": "1.5.0",
          "state": {
            "_dom_classes": [],
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "HTMLModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/controls",
            "_view_module_version": "1.5.0",
            "_view_name": "HTMLView",
            "description": "",
            "description_tooltip": null,
            "layout": "IPY_MODEL_74b9bf57eb1741d09719ad0bcd890861",
            "placeholder": "​",
            "style": "IPY_MODEL_c69b5d3419f14cf7876325503f3187c0",
            "value": "Downloading (…)l-00003-of-00007.bin: 100%"
          }
        },
        "dd7395e72e7f4120b7d1e2dc4b00626a": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "FloatProgressModel",
          "model_module_version": "1.5.0",
          "state": {
            "_dom_classes": [],
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "FloatProgressModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/controls",
            "_view_module_version": "1.5.0",
            "_view_name": "ProgressView",
            "bar_style": "success",
            "description": "",
            "description_tooltip": null,
            "layout": "IPY_MODEL_69e78a5956a745b1bea9297c3ed8cb34",
            "max": 1927414561,
            "min": 0,
            "orientation": "horizontal",
            "style": "IPY_MODEL_f5c4a8f006294df9a7f0f9d4305827f4",
            "value": 1927414561
          }
        },
        "51a9c4b74aa4445dbecd0f11e48cc9e8": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "HTMLModel",
          "model_module_version": "1.5.0",
          "state": {
            "_dom_classes": [],
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "HTMLModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/controls",
            "_view_module_version": "1.5.0",
            "_view_name": "HTMLView",
            "description": "",
            "description_tooltip": null,
            "layout": "IPY_MODEL_715d4156908f4385a85139543ed8a14b",
            "placeholder": "​",
            "style": "IPY_MODEL_cfd99ab9a1944c4d91e9ec242a2896dd",
            "value": " 1.93G/1.93G [00:05&lt;00:00, 276MB/s]"
          }
        },
        "feac1c0042d7419c87f7151032b7fd28": {
          "model_module": "@jupyter-widgets/base",
          "model_name": "LayoutModel",
          "model_module_version": "1.2.0",
          "state": {
            "_model_module": "@jupyter-widgets/base",
            "_model_module_version": "1.2.0",
            "_model_name": "LayoutModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "LayoutView",
            "align_content": null,
            "align_items": null,
            "align_self": null,
            "border": null,
            "bottom": null,
            "display": null,
            "flex": null,
            "flex_flow": null,
            "grid_area": null,
            "grid_auto_columns": null,
            "grid_auto_flow": null,
            "grid_auto_rows": null,
            "grid_column": null,
            "grid_gap": null,
            "grid_row": null,
            "grid_template_areas": null,
            "grid_template_columns": null,
            "grid_template_rows": null,
            "height": null,
            "justify_content": null,
            "justify_items": null,
            "left": null,
            "margin": null,
            "max_height": null,
            "max_width": null,
            "min_height": null,
            "min_width": null,
            "object_fit": null,
            "object_position": null,
            "order": null,
            "overflow": null,
            "overflow_x": null,
            "overflow_y": null,
            "padding": null,
            "right": null,
            "top": null,
            "visibility": null,
            "width": null
          }
        },
        "74b9bf57eb1741d09719ad0bcd890861": {
          "model_module": "@jupyter-widgets/base",
          "model_name": "LayoutModel",
          "model_module_version": "1.2.0",
          "state": {
            "_model_module": "@jupyter-widgets/base",
            "_model_module_version": "1.2.0",
            "_model_name": "LayoutModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "LayoutView",
            "align_content": null,
            "align_items": null,
            "align_self": null,
            "border": null,
            "bottom": null,
            "display": null,
            "flex": null,
            "flex_flow": null,
            "grid_area": null,
            "grid_auto_columns": null,
            "grid_auto_flow": null,
            "grid_auto_rows": null,
            "grid_column": null,
            "grid_gap": null,
            "grid_row": null,
            "grid_template_areas": null,
            "grid_template_columns": null,
            "grid_template_rows": null,
            "height": null,
            "justify_content": null,
            "justify_items": null,
            "left": null,
            "margin": null,
            "max_height": null,
            "max_width": null,
            "min_height": null,
            "min_width": null,
            "object_fit": null,
            "object_position": null,
            "order": null,
            "overflow": null,
            "overflow_x": null,
            "overflow_y": null,
            "padding": null,
            "right": null,
            "top": null,
            "visibility": null,
            "width": null
          }
        },
        "c69b5d3419f14cf7876325503f3187c0": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "DescriptionStyleModel",
          "model_module_version": "1.5.0",
          "state": {
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "DescriptionStyleModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "StyleView",
            "description_width": ""
          }
        },
        "69e78a5956a745b1bea9297c3ed8cb34": {
          "model_module": "@jupyter-widgets/base",
          "model_name": "LayoutModel",
          "model_module_version": "1.2.0",
          "state": {
            "_model_module": "@jupyter-widgets/base",
            "_model_module_version": "1.2.0",
            "_model_name": "LayoutModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "LayoutView",
            "align_content": null,
            "align_items": null,
            "align_self": null,
            "border": null,
            "bottom": null,
            "display": null,
            "flex": null,
            "flex_flow": null,
            "grid_area": null,
            "grid_auto_columns": null,
            "grid_auto_flow": null,
            "grid_auto_rows": null,
            "grid_column": null,
            "grid_gap": null,
            "grid_row": null,
            "grid_template_areas": null,
            "grid_template_columns": null,
            "grid_template_rows": null,
            "height": null,
            "justify_content": null,
            "justify_items": null,
            "left": null,
            "margin": null,
            "max_height": null,
            "max_width": null,
            "min_height": null,
            "min_width": null,
            "object_fit": null,
            "object_position": null,
            "order": null,
            "overflow": null,
            "overflow_x": null,
            "overflow_y": null,
            "padding": null,
            "right": null,
            "top": null,
            "visibility": null,
            "width": null
          }
        },
        "f5c4a8f006294df9a7f0f9d4305827f4": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "ProgressStyleModel",
          "model_module_version": "1.5.0",
          "state": {
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "ProgressStyleModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "StyleView",
            "bar_color": null,
            "description_width": ""
          }
        },
        "715d4156908f4385a85139543ed8a14b": {
          "model_module": "@jupyter-widgets/base",
          "model_name": "LayoutModel",
          "model_module_version": "1.2.0",
          "state": {
            "_model_module": "@jupyter-widgets/base",
            "_model_module_version": "1.2.0",
            "_model_name": "LayoutModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "LayoutView",
            "align_content": null,
            "align_items": null,
            "align_self": null,
            "border": null,
            "bottom": null,
            "display": null,
            "flex": null,
            "flex_flow": null,
            "grid_area": null,
            "grid_auto_columns": null,
            "grid_auto_flow": null,
            "grid_auto_rows": null,
            "grid_column": null,
            "grid_gap": null,
            "grid_row": null,
            "grid_template_areas": null,
            "grid_template_columns": null,
            "grid_template_rows": null,
            "height": null,
            "justify_content": null,
            "justify_items": null,
            "left": null,
            "margin": null,
            "max_height": null,
            "max_width": null,
            "min_height": null,
            "min_width": null,
            "object_fit": null,
            "object_position": null,
            "order": null,
            "overflow": null,
            "overflow_x": null,
            "overflow_y": null,
            "padding": null,
            "right": null,
            "top": null,
            "visibility": null,
            "width": null
          }
        },
        "cfd99ab9a1944c4d91e9ec242a2896dd": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "DescriptionStyleModel",
          "model_module_version": "1.5.0",
          "state": {
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "DescriptionStyleModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "StyleView",
            "description_width": ""
          }
        },
        "61163fb73ea24952b4cc2d970c1a7b58": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "HBoxModel",
          "model_module_version": "1.5.0",
          "state": {
            "_dom_classes": [],
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "HBoxModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/controls",
            "_view_module_version": "1.5.0",
            "_view_name": "HBoxView",
            "box_style": "",
            "children": [
              "IPY_MODEL_001042c1d21d427c8ce3901dd66df6af",
              "IPY_MODEL_0d64b42c615046a28f23cc9ec35441dc",
              "IPY_MODEL_bfe61f4f778c4ede96e50ca2dc40da60"
            ],
            "layout": "IPY_MODEL_10b1c07e1e0f4178b03e4556945aff31"
          }
        },
        "001042c1d21d427c8ce3901dd66df6af": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "HTMLModel",
          "model_module_version": "1.5.0",
          "state": {
            "_dom_classes": [],
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "HTMLModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/controls",
            "_view_module_version": "1.5.0",
            "_view_name": "HTMLView",
            "description": "",
            "description_tooltip": null,
            "layout": "IPY_MODEL_8ed41e7acc6d47ac89d5cd2dff5b5961",
            "placeholder": "​",
            "style": "IPY_MODEL_ebe74a666cbe4de48396dcd3ec24d1db",
            "value": "Downloading (…)l-00004-of-00007.bin: 100%"
          }
        },
        "0d64b42c615046a28f23cc9ec35441dc": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "FloatProgressModel",
          "model_module_version": "1.5.0",
          "state": {
            "_dom_classes": [],
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "FloatProgressModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/controls",
            "_view_module_version": "1.5.0",
            "_view_name": "ProgressView",
            "bar_style": "success",
            "description": "",
            "description_tooltip": null,
            "layout": "IPY_MODEL_bb409879d7514cdcb1ece8f1ee201162",
            "max": 1815225523,
            "min": 0,
            "orientation": "horizontal",
            "style": "IPY_MODEL_2271485fffd94944acf64e74ff32b260",
            "value": 1815225523
          }
        },
        "bfe61f4f778c4ede96e50ca2dc40da60": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "HTMLModel",
          "model_module_version": "1.5.0",
          "state": {
            "_dom_classes": [],
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "HTMLModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/controls",
            "_view_module_version": "1.5.0",
            "_view_name": "HTMLView",
            "description": "",
            "description_tooltip": null,
            "layout": "IPY_MODEL_1f7476af05dc45fdb2ffad40c2e54047",
            "placeholder": "​",
            "style": "IPY_MODEL_01fd1b5c5b5a46eeb8efcf65d6005508",
            "value": " 1.82G/1.82G [00:04&lt;00:00, 233MB/s]"
          }
        },
        "10b1c07e1e0f4178b03e4556945aff31": {
          "model_module": "@jupyter-widgets/base",
          "model_name": "LayoutModel",
          "model_module_version": "1.2.0",
          "state": {
            "_model_module": "@jupyter-widgets/base",
            "_model_module_version": "1.2.0",
            "_model_name": "LayoutModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "LayoutView",
            "align_content": null,
            "align_items": null,
            "align_self": null,
            "border": null,
            "bottom": null,
            "display": null,
            "flex": null,
            "flex_flow": null,
            "grid_area": null,
            "grid_auto_columns": null,
            "grid_auto_flow": null,
            "grid_auto_rows": null,
            "grid_column": null,
            "grid_gap": null,
            "grid_row": null,
            "grid_template_areas": null,
            "grid_template_columns": null,
            "grid_template_rows": null,
            "height": null,
            "justify_content": null,
            "justify_items": null,
            "left": null,
            "margin": null,
            "max_height": null,
            "max_width": null,
            "min_height": null,
            "min_width": null,
            "object_fit": null,
            "object_position": null,
            "order": null,
            "overflow": null,
            "overflow_x": null,
            "overflow_y": null,
            "padding": null,
            "right": null,
            "top": null,
            "visibility": null,
            "width": null
          }
        },
        "8ed41e7acc6d47ac89d5cd2dff5b5961": {
          "model_module": "@jupyter-widgets/base",
          "model_name": "LayoutModel",
          "model_module_version": "1.2.0",
          "state": {
            "_model_module": "@jupyter-widgets/base",
            "_model_module_version": "1.2.0",
            "_model_name": "LayoutModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "LayoutView",
            "align_content": null,
            "align_items": null,
            "align_self": null,
            "border": null,
            "bottom": null,
            "display": null,
            "flex": null,
            "flex_flow": null,
            "grid_area": null,
            "grid_auto_columns": null,
            "grid_auto_flow": null,
            "grid_auto_rows": null,
            "grid_column": null,
            "grid_gap": null,
            "grid_row": null,
            "grid_template_areas": null,
            "grid_template_columns": null,
            "grid_template_rows": null,
            "height": null,
            "justify_content": null,
            "justify_items": null,
            "left": null,
            "margin": null,
            "max_height": null,
            "max_width": null,
            "min_height": null,
            "min_width": null,
            "object_fit": null,
            "object_position": null,
            "order": null,
            "overflow": null,
            "overflow_x": null,
            "overflow_y": null,
            "padding": null,
            "right": null,
            "top": null,
            "visibility": null,
            "width": null
          }
        },
        "ebe74a666cbe4de48396dcd3ec24d1db": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "DescriptionStyleModel",
          "model_module_version": "1.5.0",
          "state": {
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "DescriptionStyleModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "StyleView",
            "description_width": ""
          }
        },
        "bb409879d7514cdcb1ece8f1ee201162": {
          "model_module": "@jupyter-widgets/base",
          "model_name": "LayoutModel",
          "model_module_version": "1.2.0",
          "state": {
            "_model_module": "@jupyter-widgets/base",
            "_model_module_version": "1.2.0",
            "_model_name": "LayoutModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "LayoutView",
            "align_content": null,
            "align_items": null,
            "align_self": null,
            "border": null,
            "bottom": null,
            "display": null,
            "flex": null,
            "flex_flow": null,
            "grid_area": null,
            "grid_auto_columns": null,
            "grid_auto_flow": null,
            "grid_auto_rows": null,
            "grid_column": null,
            "grid_gap": null,
            "grid_row": null,
            "grid_template_areas": null,
            "grid_template_columns": null,
            "grid_template_rows": null,
            "height": null,
            "justify_content": null,
            "justify_items": null,
            "left": null,
            "margin": null,
            "max_height": null,
            "max_width": null,
            "min_height": null,
            "min_width": null,
            "object_fit": null,
            "object_position": null,
            "order": null,
            "overflow": null,
            "overflow_x": null,
            "overflow_y": null,
            "padding": null,
            "right": null,
            "top": null,
            "visibility": null,
            "width": null
          }
        },
        "2271485fffd94944acf64e74ff32b260": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "ProgressStyleModel",
          "model_module_version": "1.5.0",
          "state": {
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "ProgressStyleModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "StyleView",
            "bar_color": null,
            "description_width": ""
          }
        },
        "1f7476af05dc45fdb2ffad40c2e54047": {
          "model_module": "@jupyter-widgets/base",
          "model_name": "LayoutModel",
          "model_module_version": "1.2.0",
          "state": {
            "_model_module": "@jupyter-widgets/base",
            "_model_module_version": "1.2.0",
            "_model_name": "LayoutModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "LayoutView",
            "align_content": null,
            "align_items": null,
            "align_self": null,
            "border": null,
            "bottom": null,
            "display": null,
            "flex": null,
            "flex_flow": null,
            "grid_area": null,
            "grid_auto_columns": null,
            "grid_auto_flow": null,
            "grid_auto_rows": null,
            "grid_column": null,
            "grid_gap": null,
            "grid_row": null,
            "grid_template_areas": null,
            "grid_template_columns": null,
            "grid_template_rows": null,
            "height": null,
            "justify_content": null,
            "justify_items": null,
            "left": null,
            "margin": null,
            "max_height": null,
            "max_width": null,
            "min_height": null,
            "min_width": null,
            "object_fit": null,
            "object_position": null,
            "order": null,
            "overflow": null,
            "overflow_x": null,
            "overflow_y": null,
            "padding": null,
            "right": null,
            "top": null,
            "visibility": null,
            "width": null
          }
        },
        "01fd1b5c5b5a46eeb8efcf65d6005508": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "DescriptionStyleModel",
          "model_module_version": "1.5.0",
          "state": {
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "DescriptionStyleModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "StyleView",
            "description_width": ""
          }
        },
        "b8fee314c61e4a34bfe27e8dd19ba7d5": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "HBoxModel",
          "model_module_version": "1.5.0",
          "state": {
            "_dom_classes": [],
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "HBoxModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/controls",
            "_view_module_version": "1.5.0",
            "_view_name": "HBoxView",
            "box_style": "",
            "children": [
              "IPY_MODEL_2b61a3721c0041f8890db71e2a94d320",
              "IPY_MODEL_432a0bb6557f4c94a740af3e80e0007d",
              "IPY_MODEL_abfd021e37da44eda059362f8770d7df"
            ],
            "layout": "IPY_MODEL_c25c2a42f34542a8aaffb78f977e44fd"
          }
        },
        "2b61a3721c0041f8890db71e2a94d320": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "HTMLModel",
          "model_module_version": "1.5.0",
          "state": {
            "_dom_classes": [],
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "HTMLModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/controls",
            "_view_module_version": "1.5.0",
            "_view_name": "HTMLView",
            "description": "",
            "description_tooltip": null,
            "layout": "IPY_MODEL_96a0a45853f74ffda59a593457c8029b",
            "placeholder": "​",
            "style": "IPY_MODEL_8c5705a687824fe898005e5cc8aba11b",
            "value": "Downloading (…)l-00005-of-00007.bin: 100%"
          }
        },
        "432a0bb6557f4c94a740af3e80e0007d": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "FloatProgressModel",
          "model_module_version": "1.5.0",
          "state": {
            "_dom_classes": [],
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "FloatProgressModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/controls",
            "_view_module_version": "1.5.0",
            "_view_name": "ProgressView",
            "bar_style": "success",
            "description": "",
            "description_tooltip": null,
            "layout": "IPY_MODEL_766155004914498698b7263257f6d2ef",
            "max": 1968299069,
            "min": 0,
            "orientation": "horizontal",
            "style": "IPY_MODEL_7645f7293aa242c5950e30fe59be59d6",
            "value": 1968299069
          }
        },
        "abfd021e37da44eda059362f8770d7df": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "HTMLModel",
          "model_module_version": "1.5.0",
          "state": {
            "_dom_classes": [],
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "HTMLModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/controls",
            "_view_module_version": "1.5.0",
            "_view_name": "HTMLView",
            "description": "",
            "description_tooltip": null,
            "layout": "IPY_MODEL_0f7d2a7671c94c2a9bfff76df95a85c1",
            "placeholder": "​",
            "style": "IPY_MODEL_21840996574c4f499e6d33e2afe75647",
            "value": " 1.97G/1.97G [00:04&lt;00:00, 371MB/s]"
          }
        },
        "c25c2a42f34542a8aaffb78f977e44fd": {
          "model_module": "@jupyter-widgets/base",
          "model_name": "LayoutModel",
          "model_module_version": "1.2.0",
          "state": {
            "_model_module": "@jupyter-widgets/base",
            "_model_module_version": "1.2.0",
            "_model_name": "LayoutModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "LayoutView",
            "align_content": null,
            "align_items": null,
            "align_self": null,
            "border": null,
            "bottom": null,
            "display": null,
            "flex": null,
            "flex_flow": null,
            "grid_area": null,
            "grid_auto_columns": null,
            "grid_auto_flow": null,
            "grid_auto_rows": null,
            "grid_column": null,
            "grid_gap": null,
            "grid_row": null,
            "grid_template_areas": null,
            "grid_template_columns": null,
            "grid_template_rows": null,
            "height": null,
            "justify_content": null,
            "justify_items": null,
            "left": null,
            "margin": null,
            "max_height": null,
            "max_width": null,
            "min_height": null,
            "min_width": null,
            "object_fit": null,
            "object_position": null,
            "order": null,
            "overflow": null,
            "overflow_x": null,
            "overflow_y": null,
            "padding": null,
            "right": null,
            "top": null,
            "visibility": null,
            "width": null
          }
        },
        "96a0a45853f74ffda59a593457c8029b": {
          "model_module": "@jupyter-widgets/base",
          "model_name": "LayoutModel",
          "model_module_version": "1.2.0",
          "state": {
            "_model_module": "@jupyter-widgets/base",
            "_model_module_version": "1.2.0",
            "_model_name": "LayoutModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "LayoutView",
            "align_content": null,
            "align_items": null,
            "align_self": null,
            "border": null,
            "bottom": null,
            "display": null,
            "flex": null,
            "flex_flow": null,
            "grid_area": null,
            "grid_auto_columns": null,
            "grid_auto_flow": null,
            "grid_auto_rows": null,
            "grid_column": null,
            "grid_gap": null,
            "grid_row": null,
            "grid_template_areas": null,
            "grid_template_columns": null,
            "grid_template_rows": null,
            "height": null,
            "justify_content": null,
            "justify_items": null,
            "left": null,
            "margin": null,
            "max_height": null,
            "max_width": null,
            "min_height": null,
            "min_width": null,
            "object_fit": null,
            "object_position": null,
            "order": null,
            "overflow": null,
            "overflow_x": null,
            "overflow_y": null,
            "padding": null,
            "right": null,
            "top": null,
            "visibility": null,
            "width": null
          }
        },
        "8c5705a687824fe898005e5cc8aba11b": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "DescriptionStyleModel",
          "model_module_version": "1.5.0",
          "state": {
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "DescriptionStyleModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "StyleView",
            "description_width": ""
          }
        },
        "766155004914498698b7263257f6d2ef": {
          "model_module": "@jupyter-widgets/base",
          "model_name": "LayoutModel",
          "model_module_version": "1.2.0",
          "state": {
            "_model_module": "@jupyter-widgets/base",
            "_model_module_version": "1.2.0",
            "_model_name": "LayoutModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "LayoutView",
            "align_content": null,
            "align_items": null,
            "align_self": null,
            "border": null,
            "bottom": null,
            "display": null,
            "flex": null,
            "flex_flow": null,
            "grid_area": null,
            "grid_auto_columns": null,
            "grid_auto_flow": null,
            "grid_auto_rows": null,
            "grid_column": null,
            "grid_gap": null,
            "grid_row": null,
            "grid_template_areas": null,
            "grid_template_columns": null,
            "grid_template_rows": null,
            "height": null,
            "justify_content": null,
            "justify_items": null,
            "left": null,
            "margin": null,
            "max_height": null,
            "max_width": null,
            "min_height": null,
            "min_width": null,
            "object_fit": null,
            "object_position": null,
            "order": null,
            "overflow": null,
            "overflow_x": null,
            "overflow_y": null,
            "padding": null,
            "right": null,
            "top": null,
            "visibility": null,
            "width": null
          }
        },
        "7645f7293aa242c5950e30fe59be59d6": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "ProgressStyleModel",
          "model_module_version": "1.5.0",
          "state": {
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "ProgressStyleModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "StyleView",
            "bar_color": null,
            "description_width": ""
          }
        },
        "0f7d2a7671c94c2a9bfff76df95a85c1": {
          "model_module": "@jupyter-widgets/base",
          "model_name": "LayoutModel",
          "model_module_version": "1.2.0",
          "state": {
            "_model_module": "@jupyter-widgets/base",
            "_model_module_version": "1.2.0",
            "_model_name": "LayoutModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "LayoutView",
            "align_content": null,
            "align_items": null,
            "align_self": null,
            "border": null,
            "bottom": null,
            "display": null,
            "flex": null,
            "flex_flow": null,
            "grid_area": null,
            "grid_auto_columns": null,
            "grid_auto_flow": null,
            "grid_auto_rows": null,
            "grid_column": null,
            "grid_gap": null,
            "grid_row": null,
            "grid_template_areas": null,
            "grid_template_columns": null,
            "grid_template_rows": null,
            "height": null,
            "justify_content": null,
            "justify_items": null,
            "left": null,
            "margin": null,
            "max_height": null,
            "max_width": null,
            "min_height": null,
            "min_width": null,
            "object_fit": null,
            "object_position": null,
            "order": null,
            "overflow": null,
            "overflow_x": null,
            "overflow_y": null,
            "padding": null,
            "right": null,
            "top": null,
            "visibility": null,
            "width": null
          }
        },
        "21840996574c4f499e6d33e2afe75647": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "DescriptionStyleModel",
          "model_module_version": "1.5.0",
          "state": {
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "DescriptionStyleModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "StyleView",
            "description_width": ""
          }
        },
        "b014042976af473497d3525999472923": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "HBoxModel",
          "model_module_version": "1.5.0",
          "state": {
            "_dom_classes": [],
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "HBoxModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/controls",
            "_view_module_version": "1.5.0",
            "_view_name": "HBoxView",
            "box_style": "",
            "children": [
              "IPY_MODEL_026d0d3d725145ee876356e929750fd0",
              "IPY_MODEL_4f1d6b22f13b4d44be37620bd50fa58f",
              "IPY_MODEL_8da41a2039024dc5a5251f0affeae703"
            ],
            "layout": "IPY_MODEL_bfd8a02213544d4f94742399c444f154"
          }
        },
        "026d0d3d725145ee876356e929750fd0": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "HTMLModel",
          "model_module_version": "1.5.0",
          "state": {
            "_dom_classes": [],
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "HTMLModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/controls",
            "_view_module_version": "1.5.0",
            "_view_name": "HTMLView",
            "description": "",
            "description_tooltip": null,
            "layout": "IPY_MODEL_5bd0b3f9e1f44198a54d84c4f992cbef",
            "placeholder": "​",
            "style": "IPY_MODEL_cdb0f709d3484ec18ad61e52d2caed53",
            "value": "Downloading (…)l-00006-of-00007.bin: 100%"
          }
        },
        "4f1d6b22f13b4d44be37620bd50fa58f": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "FloatProgressModel",
          "model_module_version": "1.5.0",
          "state": {
            "_dom_classes": [],
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "FloatProgressModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/controls",
            "_view_module_version": "1.5.0",
            "_view_name": "ProgressView",
            "bar_style": "success",
            "description": "",
            "description_tooltip": null,
            "layout": "IPY_MODEL_6f7d8e7a1b0345c2b249a08d2a812cde",
            "max": 1927414561,
            "min": 0,
            "orientation": "horizontal",
            "style": "IPY_MODEL_ff28cc19f0df49389e56d1087f5d4d54",
            "value": 1927414561
          }
        },
        "8da41a2039024dc5a5251f0affeae703": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "HTMLModel",
          "model_module_version": "1.5.0",
          "state": {
            "_dom_classes": [],
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "HTMLModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/controls",
            "_view_module_version": "1.5.0",
            "_view_name": "HTMLView",
            "description": "",
            "description_tooltip": null,
            "layout": "IPY_MODEL_16e02d596ceb4288baff72989cf2b92b",
            "placeholder": "​",
            "style": "IPY_MODEL_665c4276b8f84d25bd1c8923e2e9888c",
            "value": " 1.93G/1.93G [00:05&lt;00:00, 483MB/s]"
          }
        },
        "bfd8a02213544d4f94742399c444f154": {
          "model_module": "@jupyter-widgets/base",
          "model_name": "LayoutModel",
          "model_module_version": "1.2.0",
          "state": {
            "_model_module": "@jupyter-widgets/base",
            "_model_module_version": "1.2.0",
            "_model_name": "LayoutModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "LayoutView",
            "align_content": null,
            "align_items": null,
            "align_self": null,
            "border": null,
            "bottom": null,
            "display": null,
            "flex": null,
            "flex_flow": null,
            "grid_area": null,
            "grid_auto_columns": null,
            "grid_auto_flow": null,
            "grid_auto_rows": null,
            "grid_column": null,
            "grid_gap": null,
            "grid_row": null,
            "grid_template_areas": null,
            "grid_template_columns": null,
            "grid_template_rows": null,
            "height": null,
            "justify_content": null,
            "justify_items": null,
            "left": null,
            "margin": null,
            "max_height": null,
            "max_width": null,
            "min_height": null,
            "min_width": null,
            "object_fit": null,
            "object_position": null,
            "order": null,
            "overflow": null,
            "overflow_x": null,
            "overflow_y": null,
            "padding": null,
            "right": null,
            "top": null,
            "visibility": null,
            "width": null
          }
        },
        "5bd0b3f9e1f44198a54d84c4f992cbef": {
          "model_module": "@jupyter-widgets/base",
          "model_name": "LayoutModel",
          "model_module_version": "1.2.0",
          "state": {
            "_model_module": "@jupyter-widgets/base",
            "_model_module_version": "1.2.0",
            "_model_name": "LayoutModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "LayoutView",
            "align_content": null,
            "align_items": null,
            "align_self": null,
            "border": null,
            "bottom": null,
            "display": null,
            "flex": null,
            "flex_flow": null,
            "grid_area": null,
            "grid_auto_columns": null,
            "grid_auto_flow": null,
            "grid_auto_rows": null,
            "grid_column": null,
            "grid_gap": null,
            "grid_row": null,
            "grid_template_areas": null,
            "grid_template_columns": null,
            "grid_template_rows": null,
            "height": null,
            "justify_content": null,
            "justify_items": null,
            "left": null,
            "margin": null,
            "max_height": null,
            "max_width": null,
            "min_height": null,
            "min_width": null,
            "object_fit": null,
            "object_position": null,
            "order": null,
            "overflow": null,
            "overflow_x": null,
            "overflow_y": null,
            "padding": null,
            "right": null,
            "top": null,
            "visibility": null,
            "width": null
          }
        },
        "cdb0f709d3484ec18ad61e52d2caed53": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "DescriptionStyleModel",
          "model_module_version": "1.5.0",
          "state": {
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "DescriptionStyleModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "StyleView",
            "description_width": ""
          }
        },
        "6f7d8e7a1b0345c2b249a08d2a812cde": {
          "model_module": "@jupyter-widgets/base",
          "model_name": "LayoutModel",
          "model_module_version": "1.2.0",
          "state": {
            "_model_module": "@jupyter-widgets/base",
            "_model_module_version": "1.2.0",
            "_model_name": "LayoutModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "LayoutView",
            "align_content": null,
            "align_items": null,
            "align_self": null,
            "border": null,
            "bottom": null,
            "display": null,
            "flex": null,
            "flex_flow": null,
            "grid_area": null,
            "grid_auto_columns": null,
            "grid_auto_flow": null,
            "grid_auto_rows": null,
            "grid_column": null,
            "grid_gap": null,
            "grid_row": null,
            "grid_template_areas": null,
            "grid_template_columns": null,
            "grid_template_rows": null,
            "height": null,
            "justify_content": null,
            "justify_items": null,
            "left": null,
            "margin": null,
            "max_height": null,
            "max_width": null,
            "min_height": null,
            "min_width": null,
            "object_fit": null,
            "object_position": null,
            "order": null,
            "overflow": null,
            "overflow_x": null,
            "overflow_y": null,
            "padding": null,
            "right": null,
            "top": null,
            "visibility": null,
            "width": null
          }
        },
        "ff28cc19f0df49389e56d1087f5d4d54": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "ProgressStyleModel",
          "model_module_version": "1.5.0",
          "state": {
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "ProgressStyleModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "StyleView",
            "bar_color": null,
            "description_width": ""
          }
        },
        "16e02d596ceb4288baff72989cf2b92b": {
          "model_module": "@jupyter-widgets/base",
          "model_name": "LayoutModel",
          "model_module_version": "1.2.0",
          "state": {
            "_model_module": "@jupyter-widgets/base",
            "_model_module_version": "1.2.0",
            "_model_name": "LayoutModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "LayoutView",
            "align_content": null,
            "align_items": null,
            "align_self": null,
            "border": null,
            "bottom": null,
            "display": null,
            "flex": null,
            "flex_flow": null,
            "grid_area": null,
            "grid_auto_columns": null,
            "grid_auto_flow": null,
            "grid_auto_rows": null,
            "grid_column": null,
            "grid_gap": null,
            "grid_row": null,
            "grid_template_areas": null,
            "grid_template_columns": null,
            "grid_template_rows": null,
            "height": null,
            "justify_content": null,
            "justify_items": null,
            "left": null,
            "margin": null,
            "max_height": null,
            "max_width": null,
            "min_height": null,
            "min_width": null,
            "object_fit": null,
            "object_position": null,
            "order": null,
            "overflow": null,
            "overflow_x": null,
            "overflow_y": null,
            "padding": null,
            "right": null,
            "top": null,
            "visibility": null,
            "width": null
          }
        },
        "665c4276b8f84d25bd1c8923e2e9888c": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "DescriptionStyleModel",
          "model_module_version": "1.5.0",
          "state": {
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "DescriptionStyleModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "StyleView",
            "description_width": ""
          }
        },
        "a39f2903f58f4139b17e765371da3a78": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "HBoxModel",
          "model_module_version": "1.5.0",
          "state": {
            "_dom_classes": [],
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "HBoxModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/controls",
            "_view_module_version": "1.5.0",
            "_view_name": "HBoxView",
            "box_style": "",
            "children": [
              "IPY_MODEL_7d4f7a752f8342ddb9e5a7b36d880552",
              "IPY_MODEL_68841e419ade4d6fac769449619f41cb",
              "IPY_MODEL_401b2069ac564030bcf3f29cc2377d03"
            ],
            "layout": "IPY_MODEL_f7a66aaea60945d281c0d7743fe00855"
          }
        },
        "7d4f7a752f8342ddb9e5a7b36d880552": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "HTMLModel",
          "model_module_version": "1.5.0",
          "state": {
            "_dom_classes": [],
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "HTMLModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/controls",
            "_view_module_version": "1.5.0",
            "_view_name": "HTMLView",
            "description": "",
            "description_tooltip": null,
            "layout": "IPY_MODEL_5272a8c7446c4c91ab6d381a78834c73",
            "placeholder": "​",
            "style": "IPY_MODEL_8d1b54167f5246209996e0592ad2cf98",
            "value": "Downloading (…)l-00007-of-00007.bin: 100%"
          }
        },
        "68841e419ade4d6fac769449619f41cb": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "FloatProgressModel",
          "model_module_version": "1.5.0",
          "state": {
            "_dom_classes": [],
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "FloatProgressModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/controls",
            "_view_module_version": "1.5.0",
            "_view_name": "ProgressView",
            "bar_style": "success",
            "description": "",
            "description_tooltip": null,
            "layout": "IPY_MODEL_41694e3ed1084cad8c59b32ae4790426",
            "max": 1052808067,
            "min": 0,
            "orientation": "horizontal",
            "style": "IPY_MODEL_c1fd264b968c4d578cf60bbf844eda3a",
            "value": 1052808067
          }
        },
        "401b2069ac564030bcf3f29cc2377d03": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "HTMLModel",
          "model_module_version": "1.5.0",
          "state": {
            "_dom_classes": [],
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "HTMLModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/controls",
            "_view_module_version": "1.5.0",
            "_view_name": "HTMLView",
            "description": "",
            "description_tooltip": null,
            "layout": "IPY_MODEL_6d76690e7beb4cb38c824f93001dd418",
            "placeholder": "​",
            "style": "IPY_MODEL_d4c0d7856942438dad380e7cd9417733",
            "value": " 1.05G/1.05G [00:02&lt;00:00, 509MB/s]"
          }
        },
        "f7a66aaea60945d281c0d7743fe00855": {
          "model_module": "@jupyter-widgets/base",
          "model_name": "LayoutModel",
          "model_module_version": "1.2.0",
          "state": {
            "_model_module": "@jupyter-widgets/base",
            "_model_module_version": "1.2.0",
            "_model_name": "LayoutModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "LayoutView",
            "align_content": null,
            "align_items": null,
            "align_self": null,
            "border": null,
            "bottom": null,
            "display": null,
            "flex": null,
            "flex_flow": null,
            "grid_area": null,
            "grid_auto_columns": null,
            "grid_auto_flow": null,
            "grid_auto_rows": null,
            "grid_column": null,
            "grid_gap": null,
            "grid_row": null,
            "grid_template_areas": null,
            "grid_template_columns": null,
            "grid_template_rows": null,
            "height": null,
            "justify_content": null,
            "justify_items": null,
            "left": null,
            "margin": null,
            "max_height": null,
            "max_width": null,
            "min_height": null,
            "min_width": null,
            "object_fit": null,
            "object_position": null,
            "order": null,
            "overflow": null,
            "overflow_x": null,
            "overflow_y": null,
            "padding": null,
            "right": null,
            "top": null,
            "visibility": null,
            "width": null
          }
        },
        "5272a8c7446c4c91ab6d381a78834c73": {
          "model_module": "@jupyter-widgets/base",
          "model_name": "LayoutModel",
          "model_module_version": "1.2.0",
          "state": {
            "_model_module": "@jupyter-widgets/base",
            "_model_module_version": "1.2.0",
            "_model_name": "LayoutModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "LayoutView",
            "align_content": null,
            "align_items": null,
            "align_self": null,
            "border": null,
            "bottom": null,
            "display": null,
            "flex": null,
            "flex_flow": null,
            "grid_area": null,
            "grid_auto_columns": null,
            "grid_auto_flow": null,
            "grid_auto_rows": null,
            "grid_column": null,
            "grid_gap": null,
            "grid_row": null,
            "grid_template_areas": null,
            "grid_template_columns": null,
            "grid_template_rows": null,
            "height": null,
            "justify_content": null,
            "justify_items": null,
            "left": null,
            "margin": null,
            "max_height": null,
            "max_width": null,
            "min_height": null,
            "min_width": null,
            "object_fit": null,
            "object_position": null,
            "order": null,
            "overflow": null,
            "overflow_x": null,
            "overflow_y": null,
            "padding": null,
            "right": null,
            "top": null,
            "visibility": null,
            "width": null
          }
        },
        "8d1b54167f5246209996e0592ad2cf98": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "DescriptionStyleModel",
          "model_module_version": "1.5.0",
          "state": {
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "DescriptionStyleModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "StyleView",
            "description_width": ""
          }
        },
        "41694e3ed1084cad8c59b32ae4790426": {
          "model_module": "@jupyter-widgets/base",
          "model_name": "LayoutModel",
          "model_module_version": "1.2.0",
          "state": {
            "_model_module": "@jupyter-widgets/base",
            "_model_module_version": "1.2.0",
            "_model_name": "LayoutModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "LayoutView",
            "align_content": null,
            "align_items": null,
            "align_self": null,
            "border": null,
            "bottom": null,
            "display": null,
            "flex": null,
            "flex_flow": null,
            "grid_area": null,
            "grid_auto_columns": null,
            "grid_auto_flow": null,
            "grid_auto_rows": null,
            "grid_column": null,
            "grid_gap": null,
            "grid_row": null,
            "grid_template_areas": null,
            "grid_template_columns": null,
            "grid_template_rows": null,
            "height": null,
            "justify_content": null,
            "justify_items": null,
            "left": null,
            "margin": null,
            "max_height": null,
            "max_width": null,
            "min_height": null,
            "min_width": null,
            "object_fit": null,
            "object_position": null,
            "order": null,
            "overflow": null,
            "overflow_x": null,
            "overflow_y": null,
            "padding": null,
            "right": null,
            "top": null,
            "visibility": null,
            "width": null
          }
        },
        "c1fd264b968c4d578cf60bbf844eda3a": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "ProgressStyleModel",
          "model_module_version": "1.5.0",
          "state": {
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "ProgressStyleModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "StyleView",
            "bar_color": null,
            "description_width": ""
          }
        },
        "6d76690e7beb4cb38c824f93001dd418": {
          "model_module": "@jupyter-widgets/base",
          "model_name": "LayoutModel",
          "model_module_version": "1.2.0",
          "state": {
            "_model_module": "@jupyter-widgets/base",
            "_model_module_version": "1.2.0",
            "_model_name": "LayoutModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "LayoutView",
            "align_content": null,
            "align_items": null,
            "align_self": null,
            "border": null,
            "bottom": null,
            "display": null,
            "flex": null,
            "flex_flow": null,
            "grid_area": null,
            "grid_auto_columns": null,
            "grid_auto_flow": null,
            "grid_auto_rows": null,
            "grid_column": null,
            "grid_gap": null,
            "grid_row": null,
            "grid_template_areas": null,
            "grid_template_columns": null,
            "grid_template_rows": null,
            "height": null,
            "justify_content": null,
            "justify_items": null,
            "left": null,
            "margin": null,
            "max_height": null,
            "max_width": null,
            "min_height": null,
            "min_width": null,
            "object_fit": null,
            "object_position": null,
            "order": null,
            "overflow": null,
            "overflow_x": null,
            "overflow_y": null,
            "padding": null,
            "right": null,
            "top": null,
            "visibility": null,
            "width": null
          }
        },
        "d4c0d7856942438dad380e7cd9417733": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "DescriptionStyleModel",
          "model_module_version": "1.5.0",
          "state": {
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "DescriptionStyleModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "StyleView",
            "description_width": ""
          }
        },
        "015cd0c016c747b1a9a294c901ecb2d8": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "HBoxModel",
          "model_module_version": "1.5.0",
          "state": {
            "_dom_classes": [],
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "HBoxModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/controls",
            "_view_module_version": "1.5.0",
            "_view_name": "HBoxView",
            "box_style": "",
            "children": [
              "IPY_MODEL_848b0fa51afd48b7864e185cdffd38ba",
              "IPY_MODEL_79d2338f8bcd48979bb6b169f8d91325",
              "IPY_MODEL_d5dfc2cef8bd4bafa92c2662b79e5183"
            ],
            "layout": "IPY_MODEL_fef5b50a04574819a8eb7d0e300a2fd6"
          }
        },
        "848b0fa51afd48b7864e185cdffd38ba": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "HTMLModel",
          "model_module_version": "1.5.0",
          "state": {
            "_dom_classes": [],
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "HTMLModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/controls",
            "_view_module_version": "1.5.0",
            "_view_name": "HTMLView",
            "description": "",
            "description_tooltip": null,
            "layout": "IPY_MODEL_61b57146d01b4c7bb9b4f5e5f86ef0bc",
            "placeholder": "​",
            "style": "IPY_MODEL_01ac13d42de843bead2ff518663b1d2b",
            "value": "Loading checkpoint shards: 100%"
          }
        },
        "79d2338f8bcd48979bb6b169f8d91325": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "FloatProgressModel",
          "model_module_version": "1.5.0",
          "state": {
            "_dom_classes": [],
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "FloatProgressModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/controls",
            "_view_module_version": "1.5.0",
            "_view_name": "ProgressView",
            "bar_style": "success",
            "description": "",
            "description_tooltip": null,
            "layout": "IPY_MODEL_2813ac1fe66e4b278190a2aa9efb8137",
            "max": 7,
            "min": 0,
            "orientation": "horizontal",
            "style": "IPY_MODEL_f88c780435504e7a9d15a6caf75bf42d",
            "value": 7
          }
        },
        "d5dfc2cef8bd4bafa92c2662b79e5183": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "HTMLModel",
          "model_module_version": "1.5.0",
          "state": {
            "_dom_classes": [],
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "HTMLModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/controls",
            "_view_module_version": "1.5.0",
            "_view_name": "HTMLView",
            "description": "",
            "description_tooltip": null,
            "layout": "IPY_MODEL_8d5f70a8b7c04a81917031ecaf519a2c",
            "placeholder": "​",
            "style": "IPY_MODEL_90ddd2684442474dba4b724e57feec53",
            "value": " 7/7 [00:10&lt;00:00,  1.38s/it]"
          }
        },
        "fef5b50a04574819a8eb7d0e300a2fd6": {
          "model_module": "@jupyter-widgets/base",
          "model_name": "LayoutModel",
          "model_module_version": "1.2.0",
          "state": {
            "_model_module": "@jupyter-widgets/base",
            "_model_module_version": "1.2.0",
            "_model_name": "LayoutModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "LayoutView",
            "align_content": null,
            "align_items": null,
            "align_self": null,
            "border": null,
            "bottom": null,
            "display": null,
            "flex": null,
            "flex_flow": null,
            "grid_area": null,
            "grid_auto_columns": null,
            "grid_auto_flow": null,
            "grid_auto_rows": null,
            "grid_column": null,
            "grid_gap": null,
            "grid_row": null,
            "grid_template_areas": null,
            "grid_template_columns": null,
            "grid_template_rows": null,
            "height": null,
            "justify_content": null,
            "justify_items": null,
            "left": null,
            "margin": null,
            "max_height": null,
            "max_width": null,
            "min_height": null,
            "min_width": null,
            "object_fit": null,
            "object_position": null,
            "order": null,
            "overflow": null,
            "overflow_x": null,
            "overflow_y": null,
            "padding": null,
            "right": null,
            "top": null,
            "visibility": null,
            "width": null
          }
        },
        "61b57146d01b4c7bb9b4f5e5f86ef0bc": {
          "model_module": "@jupyter-widgets/base",
          "model_name": "LayoutModel",
          "model_module_version": "1.2.0",
          "state": {
            "_model_module": "@jupyter-widgets/base",
            "_model_module_version": "1.2.0",
            "_model_name": "LayoutModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "LayoutView",
            "align_content": null,
            "align_items": null,
            "align_self": null,
            "border": null,
            "bottom": null,
            "display": null,
            "flex": null,
            "flex_flow": null,
            "grid_area": null,
            "grid_auto_columns": null,
            "grid_auto_flow": null,
            "grid_auto_rows": null,
            "grid_column": null,
            "grid_gap": null,
            "grid_row": null,
            "grid_template_areas": null,
            "grid_template_columns": null,
            "grid_template_rows": null,
            "height": null,
            "justify_content": null,
            "justify_items": null,
            "left": null,
            "margin": null,
            "max_height": null,
            "max_width": null,
            "min_height": null,
            "min_width": null,
            "object_fit": null,
            "object_position": null,
            "order": null,
            "overflow": null,
            "overflow_x": null,
            "overflow_y": null,
            "padding": null,
            "right": null,
            "top": null,
            "visibility": null,
            "width": null
          }
        },
        "01ac13d42de843bead2ff518663b1d2b": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "DescriptionStyleModel",
          "model_module_version": "1.5.0",
          "state": {
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "DescriptionStyleModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "StyleView",
            "description_width": ""
          }
        },
        "2813ac1fe66e4b278190a2aa9efb8137": {
          "model_module": "@jupyter-widgets/base",
          "model_name": "LayoutModel",
          "model_module_version": "1.2.0",
          "state": {
            "_model_module": "@jupyter-widgets/base",
            "_model_module_version": "1.2.0",
            "_model_name": "LayoutModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "LayoutView",
            "align_content": null,
            "align_items": null,
            "align_self": null,
            "border": null,
            "bottom": null,
            "display": null,
            "flex": null,
            "flex_flow": null,
            "grid_area": null,
            "grid_auto_columns": null,
            "grid_auto_flow": null,
            "grid_auto_rows": null,
            "grid_column": null,
            "grid_gap": null,
            "grid_row": null,
            "grid_template_areas": null,
            "grid_template_columns": null,
            "grid_template_rows": null,
            "height": null,
            "justify_content": null,
            "justify_items": null,
            "left": null,
            "margin": null,
            "max_height": null,
            "max_width": null,
            "min_height": null,
            "min_width": null,
            "object_fit": null,
            "object_position": null,
            "order": null,
            "overflow": null,
            "overflow_x": null,
            "overflow_y": null,
            "padding": null,
            "right": null,
            "top": null,
            "visibility": null,
            "width": null
          }
        },
        "f88c780435504e7a9d15a6caf75bf42d": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "ProgressStyleModel",
          "model_module_version": "1.5.0",
          "state": {
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "ProgressStyleModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "StyleView",
            "bar_color": null,
            "description_width": ""
          }
        },
        "8d5f70a8b7c04a81917031ecaf519a2c": {
          "model_module": "@jupyter-widgets/base",
          "model_name": "LayoutModel",
          "model_module_version": "1.2.0",
          "state": {
            "_model_module": "@jupyter-widgets/base",
            "_model_module_version": "1.2.0",
            "_model_name": "LayoutModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "LayoutView",
            "align_content": null,
            "align_items": null,
            "align_self": null,
            "border": null,
            "bottom": null,
            "display": null,
            "flex": null,
            "flex_flow": null,
            "grid_area": null,
            "grid_auto_columns": null,
            "grid_auto_flow": null,
            "grid_auto_rows": null,
            "grid_column": null,
            "grid_gap": null,
            "grid_row": null,
            "grid_template_areas": null,
            "grid_template_columns": null,
            "grid_template_rows": null,
            "height": null,
            "justify_content": null,
            "justify_items": null,
            "left": null,
            "margin": null,
            "max_height": null,
            "max_width": null,
            "min_height": null,
            "min_width": null,
            "object_fit": null,
            "object_position": null,
            "order": null,
            "overflow": null,
            "overflow_x": null,
            "overflow_y": null,
            "padding": null,
            "right": null,
            "top": null,
            "visibility": null,
            "width": null
          }
        },
        "90ddd2684442474dba4b724e57feec53": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "DescriptionStyleModel",
          "model_module_version": "1.5.0",
          "state": {
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "DescriptionStyleModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "StyleView",
            "description_width": ""
          }
        },
        "a23be1c1f8364d11849f0d67df6b2130": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "HBoxModel",
          "model_module_version": "1.5.0",
          "state": {
            "_dom_classes": [],
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "HBoxModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/controls",
            "_view_module_version": "1.5.0",
            "_view_name": "HBoxView",
            "box_style": "",
            "children": [
              "IPY_MODEL_29c787f02b494958b2588d1eda5de870",
              "IPY_MODEL_06c93fb847a44befb8a7972152c6c60b",
              "IPY_MODEL_5f487e075e0e47718717fe4b19a9904e"
            ],
            "layout": "IPY_MODEL_f4c3bc7be8d141a3971decf28356cf20"
          }
        },
        "29c787f02b494958b2588d1eda5de870": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "HTMLModel",
          "model_module_version": "1.5.0",
          "state": {
            "_dom_classes": [],
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "HTMLModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/controls",
            "_view_module_version": "1.5.0",
            "_view_name": "HTMLView",
            "description": "",
            "description_tooltip": null,
            "layout": "IPY_MODEL_d5c8fcab172e43779f69b7791d96dd91",
            "placeholder": "​",
            "style": "IPY_MODEL_2a3ac2af15f64766a77ec3d68fba809f",
            "value": "Map (num_proc=4):  25%"
          }
        },
        "06c93fb847a44befb8a7972152c6c60b": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "FloatProgressModel",
          "model_module_version": "1.5.0",
          "state": {
            "_dom_classes": [],
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "FloatProgressModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/controls",
            "_view_module_version": "1.5.0",
            "_view_name": "ProgressView",
            "bar_style": "",
            "description": "",
            "description_tooltip": null,
            "layout": "IPY_MODEL_2b9d64141bc94b80bc8d1aebdfbe67a3",
            "max": 8,
            "min": 0,
            "orientation": "horizontal",
            "style": "IPY_MODEL_7f223ca06aed44cda28f41b5c4257d00",
            "value": 8
          }
        },
        "5f487e075e0e47718717fe4b19a9904e": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "HTMLModel",
          "model_module_version": "1.5.0",
          "state": {
            "_dom_classes": [],
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "HTMLModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/controls",
            "_view_module_version": "1.5.0",
            "_view_name": "HTMLView",
            "description": "",
            "description_tooltip": null,
            "layout": "IPY_MODEL_d11815f3260747cda82621403e92559f",
            "placeholder": "​",
            "style": "IPY_MODEL_b01734faf07e4e0f86df5e2d270b9298",
            "value": " 2/8 [00:00&lt;00:00, 13.35 examples/s]"
          }
        },
        "f4c3bc7be8d141a3971decf28356cf20": {
          "model_module": "@jupyter-widgets/base",
          "model_name": "LayoutModel",
          "model_module_version": "1.2.0",
          "state": {
            "_model_module": "@jupyter-widgets/base",
            "_model_module_version": "1.2.0",
            "_model_name": "LayoutModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "LayoutView",
            "align_content": null,
            "align_items": null,
            "align_self": null,
            "border": null,
            "bottom": null,
            "display": null,
            "flex": null,
            "flex_flow": null,
            "grid_area": null,
            "grid_auto_columns": null,
            "grid_auto_flow": null,
            "grid_auto_rows": null,
            "grid_column": null,
            "grid_gap": null,
            "grid_row": null,
            "grid_template_areas": null,
            "grid_template_columns": null,
            "grid_template_rows": null,
            "height": null,
            "justify_content": null,
            "justify_items": null,
            "left": null,
            "margin": null,
            "max_height": null,
            "max_width": null,
            "min_height": null,
            "min_width": null,
            "object_fit": null,
            "object_position": null,
            "order": null,
            "overflow": null,
            "overflow_x": null,
            "overflow_y": null,
            "padding": null,
            "right": null,
            "top": null,
            "visibility": "hidden",
            "width": null
          }
        },
        "d5c8fcab172e43779f69b7791d96dd91": {
          "model_module": "@jupyter-widgets/base",
          "model_name": "LayoutModel",
          "model_module_version": "1.2.0",
          "state": {
            "_model_module": "@jupyter-widgets/base",
            "_model_module_version": "1.2.0",
            "_model_name": "LayoutModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "LayoutView",
            "align_content": null,
            "align_items": null,
            "align_self": null,
            "border": null,
            "bottom": null,
            "display": null,
            "flex": null,
            "flex_flow": null,
            "grid_area": null,
            "grid_auto_columns": null,
            "grid_auto_flow": null,
            "grid_auto_rows": null,
            "grid_column": null,
            "grid_gap": null,
            "grid_row": null,
            "grid_template_areas": null,
            "grid_template_columns": null,
            "grid_template_rows": null,
            "height": null,
            "justify_content": null,
            "justify_items": null,
            "left": null,
            "margin": null,
            "max_height": null,
            "max_width": null,
            "min_height": null,
            "min_width": null,
            "object_fit": null,
            "object_position": null,
            "order": null,
            "overflow": null,
            "overflow_x": null,
            "overflow_y": null,
            "padding": null,
            "right": null,
            "top": null,
            "visibility": null,
            "width": null
          }
        },
        "2a3ac2af15f64766a77ec3d68fba809f": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "DescriptionStyleModel",
          "model_module_version": "1.5.0",
          "state": {
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "DescriptionStyleModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "StyleView",
            "description_width": ""
          }
        },
        "2b9d64141bc94b80bc8d1aebdfbe67a3": {
          "model_module": "@jupyter-widgets/base",
          "model_name": "LayoutModel",
          "model_module_version": "1.2.0",
          "state": {
            "_model_module": "@jupyter-widgets/base",
            "_model_module_version": "1.2.0",
            "_model_name": "LayoutModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "LayoutView",
            "align_content": null,
            "align_items": null,
            "align_self": null,
            "border": null,
            "bottom": null,
            "display": null,
            "flex": null,
            "flex_flow": null,
            "grid_area": null,
            "grid_auto_columns": null,
            "grid_auto_flow": null,
            "grid_auto_rows": null,
            "grid_column": null,
            "grid_gap": null,
            "grid_row": null,
            "grid_template_areas": null,
            "grid_template_columns": null,
            "grid_template_rows": null,
            "height": null,
            "justify_content": null,
            "justify_items": null,
            "left": null,
            "margin": null,
            "max_height": null,
            "max_width": null,
            "min_height": null,
            "min_width": null,
            "object_fit": null,
            "object_position": null,
            "order": null,
            "overflow": null,
            "overflow_x": null,
            "overflow_y": null,
            "padding": null,
            "right": null,
            "top": null,
            "visibility": null,
            "width": null
          }
        },
        "7f223ca06aed44cda28f41b5c4257d00": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "ProgressStyleModel",
          "model_module_version": "1.5.0",
          "state": {
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "ProgressStyleModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "StyleView",
            "bar_color": null,
            "description_width": ""
          }
        },
        "d11815f3260747cda82621403e92559f": {
          "model_module": "@jupyter-widgets/base",
          "model_name": "LayoutModel",
          "model_module_version": "1.2.0",
          "state": {
            "_model_module": "@jupyter-widgets/base",
            "_model_module_version": "1.2.0",
            "_model_name": "LayoutModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "LayoutView",
            "align_content": null,
            "align_items": null,
            "align_self": null,
            "border": null,
            "bottom": null,
            "display": null,
            "flex": null,
            "flex_flow": null,
            "grid_area": null,
            "grid_auto_columns": null,
            "grid_auto_flow": null,
            "grid_auto_rows": null,
            "grid_column": null,
            "grid_gap": null,
            "grid_row": null,
            "grid_template_areas": null,
            "grid_template_columns": null,
            "grid_template_rows": null,
            "height": null,
            "justify_content": null,
            "justify_items": null,
            "left": null,
            "margin": null,
            "max_height": null,
            "max_width": null,
            "min_height": null,
            "min_width": null,
            "object_fit": null,
            "object_position": null,
            "order": null,
            "overflow": null,
            "overflow_x": null,
            "overflow_y": null,
            "padding": null,
            "right": null,
            "top": null,
            "visibility": null,
            "width": null
          }
        },
        "b01734faf07e4e0f86df5e2d270b9298": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "DescriptionStyleModel",
          "model_module_version": "1.5.0",
          "state": {
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "DescriptionStyleModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "StyleView",
            "description_width": ""
          }
        },
        "a0c3b45375d044e682c3baba9c85f7f9": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "HBoxModel",
          "model_module_version": "1.5.0",
          "state": {
            "_dom_classes": [],
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "HBoxModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/controls",
            "_view_module_version": "1.5.0",
            "_view_name": "HBoxView",
            "box_style": "",
            "children": [
              "IPY_MODEL_827dcc8354bf437eabdeaff958cb5bbb",
              "IPY_MODEL_36cf9e6b358a47958d59db31d511788c",
              "IPY_MODEL_9c28cc0ad1d04ce8b4cb90858292f933"
            ],
            "layout": "IPY_MODEL_3d057da70bc34a92b6eaf25a95bafb1b"
          }
        },
        "827dcc8354bf437eabdeaff958cb5bbb": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "HTMLModel",
          "model_module_version": "1.5.0",
          "state": {
            "_dom_classes": [],
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "HTMLModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/controls",
            "_view_module_version": "1.5.0",
            "_view_name": "HTMLView",
            "description": "",
            "description_tooltip": null,
            "layout": "IPY_MODEL_7a454802054341c68516fa6f75554412",
            "placeholder": "​",
            "style": "IPY_MODEL_b66da19c1db4463287021270db3bb5e3",
            "value": "Map (num_proc=4):  25%"
          }
        },
        "36cf9e6b358a47958d59db31d511788c": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "FloatProgressModel",
          "model_module_version": "1.5.0",
          "state": {
            "_dom_classes": [],
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "FloatProgressModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/controls",
            "_view_module_version": "1.5.0",
            "_view_name": "ProgressView",
            "bar_style": "",
            "description": "",
            "description_tooltip": null,
            "layout": "IPY_MODEL_57a6522f68b044d2af989702078bc8fc",
            "max": 8,
            "min": 0,
            "orientation": "horizontal",
            "style": "IPY_MODEL_dc8e6ee73877477180367d9094db9a11",
            "value": 8
          }
        },
        "9c28cc0ad1d04ce8b4cb90858292f933": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "HTMLModel",
          "model_module_version": "1.5.0",
          "state": {
            "_dom_classes": [],
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "HTMLModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/controls",
            "_view_module_version": "1.5.0",
            "_view_name": "HTMLView",
            "description": "",
            "description_tooltip": null,
            "layout": "IPY_MODEL_a017a407deca46238edf5338af6ccd73",
            "placeholder": "​",
            "style": "IPY_MODEL_782ab33627a64ba989a81b18ad71910c",
            "value": " 2/8 [00:00&lt;00:00, 16.83 examples/s]"
          }
        },
        "3d057da70bc34a92b6eaf25a95bafb1b": {
          "model_module": "@jupyter-widgets/base",
          "model_name": "LayoutModel",
          "model_module_version": "1.2.0",
          "state": {
            "_model_module": "@jupyter-widgets/base",
            "_model_module_version": "1.2.0",
            "_model_name": "LayoutModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "LayoutView",
            "align_content": null,
            "align_items": null,
            "align_self": null,
            "border": null,
            "bottom": null,
            "display": null,
            "flex": null,
            "flex_flow": null,
            "grid_area": null,
            "grid_auto_columns": null,
            "grid_auto_flow": null,
            "grid_auto_rows": null,
            "grid_column": null,
            "grid_gap": null,
            "grid_row": null,
            "grid_template_areas": null,
            "grid_template_columns": null,
            "grid_template_rows": null,
            "height": null,
            "justify_content": null,
            "justify_items": null,
            "left": null,
            "margin": null,
            "max_height": null,
            "max_width": null,
            "min_height": null,
            "min_width": null,
            "object_fit": null,
            "object_position": null,
            "order": null,
            "overflow": null,
            "overflow_x": null,
            "overflow_y": null,
            "padding": null,
            "right": null,
            "top": null,
            "visibility": "hidden",
            "width": null
          }
        },
        "7a454802054341c68516fa6f75554412": {
          "model_module": "@jupyter-widgets/base",
          "model_name": "LayoutModel",
          "model_module_version": "1.2.0",
          "state": {
            "_model_module": "@jupyter-widgets/base",
            "_model_module_version": "1.2.0",
            "_model_name": "LayoutModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "LayoutView",
            "align_content": null,
            "align_items": null,
            "align_self": null,
            "border": null,
            "bottom": null,
            "display": null,
            "flex": null,
            "flex_flow": null,
            "grid_area": null,
            "grid_auto_columns": null,
            "grid_auto_flow": null,
            "grid_auto_rows": null,
            "grid_column": null,
            "grid_gap": null,
            "grid_row": null,
            "grid_template_areas": null,
            "grid_template_columns": null,
            "grid_template_rows": null,
            "height": null,
            "justify_content": null,
            "justify_items": null,
            "left": null,
            "margin": null,
            "max_height": null,
            "max_width": null,
            "min_height": null,
            "min_width": null,
            "object_fit": null,
            "object_position": null,
            "order": null,
            "overflow": null,
            "overflow_x": null,
            "overflow_y": null,
            "padding": null,
            "right": null,
            "top": null,
            "visibility": null,
            "width": null
          }
        },
        "b66da19c1db4463287021270db3bb5e3": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "DescriptionStyleModel",
          "model_module_version": "1.5.0",
          "state": {
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "DescriptionStyleModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "StyleView",
            "description_width": ""
          }
        },
        "57a6522f68b044d2af989702078bc8fc": {
          "model_module": "@jupyter-widgets/base",
          "model_name": "LayoutModel",
          "model_module_version": "1.2.0",
          "state": {
            "_model_module": "@jupyter-widgets/base",
            "_model_module_version": "1.2.0",
            "_model_name": "LayoutModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "LayoutView",
            "align_content": null,
            "align_items": null,
            "align_self": null,
            "border": null,
            "bottom": null,
            "display": null,
            "flex": null,
            "flex_flow": null,
            "grid_area": null,
            "grid_auto_columns": null,
            "grid_auto_flow": null,
            "grid_auto_rows": null,
            "grid_column": null,
            "grid_gap": null,
            "grid_row": null,
            "grid_template_areas": null,
            "grid_template_columns": null,
            "grid_template_rows": null,
            "height": null,
            "justify_content": null,
            "justify_items": null,
            "left": null,
            "margin": null,
            "max_height": null,
            "max_width": null,
            "min_height": null,
            "min_width": null,
            "object_fit": null,
            "object_position": null,
            "order": null,
            "overflow": null,
            "overflow_x": null,
            "overflow_y": null,
            "padding": null,
            "right": null,
            "top": null,
            "visibility": null,
            "width": null
          }
        },
        "dc8e6ee73877477180367d9094db9a11": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "ProgressStyleModel",
          "model_module_version": "1.5.0",
          "state": {
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "ProgressStyleModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "StyleView",
            "bar_color": null,
            "description_width": ""
          }
        },
        "a017a407deca46238edf5338af6ccd73": {
          "model_module": "@jupyter-widgets/base",
          "model_name": "LayoutModel",
          "model_module_version": "1.2.0",
          "state": {
            "_model_module": "@jupyter-widgets/base",
            "_model_module_version": "1.2.0",
            "_model_name": "LayoutModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "LayoutView",
            "align_content": null,
            "align_items": null,
            "align_self": null,
            "border": null,
            "bottom": null,
            "display": null,
            "flex": null,
            "flex_flow": null,
            "grid_area": null,
            "grid_auto_columns": null,
            "grid_auto_flow": null,
            "grid_auto_rows": null,
            "grid_column": null,
            "grid_gap": null,
            "grid_row": null,
            "grid_template_areas": null,
            "grid_template_columns": null,
            "grid_template_rows": null,
            "height": null,
            "justify_content": null,
            "justify_items": null,
            "left": null,
            "margin": null,
            "max_height": null,
            "max_width": null,
            "min_height": null,
            "min_width": null,
            "object_fit": null,
            "object_position": null,
            "order": null,
            "overflow": null,
            "overflow_x": null,
            "overflow_y": null,
            "padding": null,
            "right": null,
            "top": null,
            "visibility": null,
            "width": null
          }
        },
        "782ab33627a64ba989a81b18ad71910c": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "DescriptionStyleModel",
          "model_module_version": "1.5.0",
          "state": {
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "DescriptionStyleModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "StyleView",
            "description_width": ""
          }
        },
        "f00fcf71ab2a432386c14a2785832252": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "HBoxModel",
          "model_module_version": "1.5.0",
          "state": {
            "_dom_classes": [],
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "HBoxModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/controls",
            "_view_module_version": "1.5.0",
            "_view_name": "HBoxView",
            "box_style": "",
            "children": [
              "IPY_MODEL_1c31bd54d8484cb28db437b3652e4f75",
              "IPY_MODEL_d55ab8ac6d814966a9fac2dc59aacc02",
              "IPY_MODEL_b3d50939d3b94a66a56df5525166039c"
            ],
            "layout": "IPY_MODEL_60d1ab4cc1214a90b8292bbac1588036"
          }
        },
        "1c31bd54d8484cb28db437b3652e4f75": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "HTMLModel",
          "model_module_version": "1.5.0",
          "state": {
            "_dom_classes": [],
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "HTMLModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/controls",
            "_view_module_version": "1.5.0",
            "_view_name": "HTMLView",
            "description": "",
            "description_tooltip": null,
            "layout": "IPY_MODEL_57f7870f67674fabadc30c6742eea2c2",
            "placeholder": "​",
            "style": "IPY_MODEL_c4e46abff0cb41279db4f5b85a44a5cb",
            "value": "Loading checkpoint shards: 100%"
          }
        },
        "d55ab8ac6d814966a9fac2dc59aacc02": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "FloatProgressModel",
          "model_module_version": "1.5.0",
          "state": {
            "_dom_classes": [],
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "FloatProgressModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/controls",
            "_view_module_version": "1.5.0",
            "_view_name": "ProgressView",
            "bar_style": "success",
            "description": "",
            "description_tooltip": null,
            "layout": "IPY_MODEL_1017fe739a0a4c47a126ab6a2020a11c",
            "max": 7,
            "min": 0,
            "orientation": "horizontal",
            "style": "IPY_MODEL_d741c70b65a74954acf6ec3ee09063cb",
            "value": 7
          }
        },
        "b3d50939d3b94a66a56df5525166039c": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "HTMLModel",
          "model_module_version": "1.5.0",
          "state": {
            "_dom_classes": [],
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "HTMLModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/controls",
            "_view_module_version": "1.5.0",
            "_view_name": "HTMLView",
            "description": "",
            "description_tooltip": null,
            "layout": "IPY_MODEL_1cde9e9568ad45319c48be7a91df7108",
            "placeholder": "​",
            "style": "IPY_MODEL_12fecc1c90c14b438916f1fdc385a9f2",
            "value": " 7/7 [00:10&lt;00:00,  1.37s/it]"
          }
        },
        "60d1ab4cc1214a90b8292bbac1588036": {
          "model_module": "@jupyter-widgets/base",
          "model_name": "LayoutModel",
          "model_module_version": "1.2.0",
          "state": {
            "_model_module": "@jupyter-widgets/base",
            "_model_module_version": "1.2.0",
            "_model_name": "LayoutModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "LayoutView",
            "align_content": null,
            "align_items": null,
            "align_self": null,
            "border": null,
            "bottom": null,
            "display": null,
            "flex": null,
            "flex_flow": null,
            "grid_area": null,
            "grid_auto_columns": null,
            "grid_auto_flow": null,
            "grid_auto_rows": null,
            "grid_column": null,
            "grid_gap": null,
            "grid_row": null,
            "grid_template_areas": null,
            "grid_template_columns": null,
            "grid_template_rows": null,
            "height": null,
            "justify_content": null,
            "justify_items": null,
            "left": null,
            "margin": null,
            "max_height": null,
            "max_width": null,
            "min_height": null,
            "min_width": null,
            "object_fit": null,
            "object_position": null,
            "order": null,
            "overflow": null,
            "overflow_x": null,
            "overflow_y": null,
            "padding": null,
            "right": null,
            "top": null,
            "visibility": null,
            "width": null
          }
        },
        "57f7870f67674fabadc30c6742eea2c2": {
          "model_module": "@jupyter-widgets/base",
          "model_name": "LayoutModel",
          "model_module_version": "1.2.0",
          "state": {
            "_model_module": "@jupyter-widgets/base",
            "_model_module_version": "1.2.0",
            "_model_name": "LayoutModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "LayoutView",
            "align_content": null,
            "align_items": null,
            "align_self": null,
            "border": null,
            "bottom": null,
            "display": null,
            "flex": null,
            "flex_flow": null,
            "grid_area": null,
            "grid_auto_columns": null,
            "grid_auto_flow": null,
            "grid_auto_rows": null,
            "grid_column": null,
            "grid_gap": null,
            "grid_row": null,
            "grid_template_areas": null,
            "grid_template_columns": null,
            "grid_template_rows": null,
            "height": null,
            "justify_content": null,
            "justify_items": null,
            "left": null,
            "margin": null,
            "max_height": null,
            "max_width": null,
            "min_height": null,
            "min_width": null,
            "object_fit": null,
            "object_position": null,
            "order": null,
            "overflow": null,
            "overflow_x": null,
            "overflow_y": null,
            "padding": null,
            "right": null,
            "top": null,
            "visibility": null,
            "width": null
          }
        },
        "c4e46abff0cb41279db4f5b85a44a5cb": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "DescriptionStyleModel",
          "model_module_version": "1.5.0",
          "state": {
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "DescriptionStyleModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "StyleView",
            "description_width": ""
          }
        },
        "1017fe739a0a4c47a126ab6a2020a11c": {
          "model_module": "@jupyter-widgets/base",
          "model_name": "LayoutModel",
          "model_module_version": "1.2.0",
          "state": {
            "_model_module": "@jupyter-widgets/base",
            "_model_module_version": "1.2.0",
            "_model_name": "LayoutModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "LayoutView",
            "align_content": null,
            "align_items": null,
            "align_self": null,
            "border": null,
            "bottom": null,
            "display": null,
            "flex": null,
            "flex_flow": null,
            "grid_area": null,
            "grid_auto_columns": null,
            "grid_auto_flow": null,
            "grid_auto_rows": null,
            "grid_column": null,
            "grid_gap": null,
            "grid_row": null,
            "grid_template_areas": null,
            "grid_template_columns": null,
            "grid_template_rows": null,
            "height": null,
            "justify_content": null,
            "justify_items": null,
            "left": null,
            "margin": null,
            "max_height": null,
            "max_width": null,
            "min_height": null,
            "min_width": null,
            "object_fit": null,
            "object_position": null,
            "order": null,
            "overflow": null,
            "overflow_x": null,
            "overflow_y": null,
            "padding": null,
            "right": null,
            "top": null,
            "visibility": null,
            "width": null
          }
        },
        "d741c70b65a74954acf6ec3ee09063cb": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "ProgressStyleModel",
          "model_module_version": "1.5.0",
          "state": {
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "ProgressStyleModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "StyleView",
            "bar_color": null,
            "description_width": ""
          }
        },
        "1cde9e9568ad45319c48be7a91df7108": {
          "model_module": "@jupyter-widgets/base",
          "model_name": "LayoutModel",
          "model_module_version": "1.2.0",
          "state": {
            "_model_module": "@jupyter-widgets/base",
            "_model_module_version": "1.2.0",
            "_model_name": "LayoutModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "LayoutView",
            "align_content": null,
            "align_items": null,
            "align_self": null,
            "border": null,
            "bottom": null,
            "display": null,
            "flex": null,
            "flex_flow": null,
            "grid_area": null,
            "grid_auto_columns": null,
            "grid_auto_flow": null,
            "grid_auto_rows": null,
            "grid_column": null,
            "grid_gap": null,
            "grid_row": null,
            "grid_template_areas": null,
            "grid_template_columns": null,
            "grid_template_rows": null,
            "height": null,
            "justify_content": null,
            "justify_items": null,
            "left": null,
            "margin": null,
            "max_height": null,
            "max_width": null,
            "min_height": null,
            "min_width": null,
            "object_fit": null,
            "object_position": null,
            "order": null,
            "overflow": null,
            "overflow_x": null,
            "overflow_y": null,
            "padding": null,
            "right": null,
            "top": null,
            "visibility": null,
            "width": null
          }
        },
        "12fecc1c90c14b438916f1fdc385a9f2": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "DescriptionStyleModel",
          "model_module_version": "1.5.0",
          "state": {
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "DescriptionStyleModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "StyleView",
            "description_width": ""
          }
        }
      }
    }
  },
  "cells": [
    {
      "cell_type": "markdown",
      "source": [
        "## 安装依赖及准备工作\n",
        "\n",
        "使用ChatGLM2的官方requirements.txt *文件安装ChatGLM2*"
      ],
      "metadata": {
        "id": "J6W0lPsT1c-G"
      }
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "_UKsHw351JhP",
        "outputId": "80a9ba47-e56b-450e-b68f-fd21b75ef7f7"
      },
      "outputs": [
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "Requirement already satisfied: protobuf in /usr/local/lib/python3.10/dist-packages (from -r requirements.txt (line 1)) (3.20.3)\n",
            "Collecting transformers==4.30.2 (from -r requirements.txt (line 2))\n",
            "  Downloading transformers-4.30.2-py3-none-any.whl (7.2 MB)\n",
            "\u001b[2K     \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m7.2/7.2 MB\u001b[0m \u001b[31m78.5 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m\n",
            "\u001b[?25hCollecting cpm_kernels (from -r requirements.txt (line 3))\n",
            "  Downloading cpm_kernels-1.0.11-py3-none-any.whl (416 kB)\n",
            "\u001b[2K     \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m416.6/416.6 kB\u001b[0m \u001b[31m42.3 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m\n",
            "\u001b[?25hRequirement already satisfied: torch>=2.0 in /usr/local/lib/python3.10/dist-packages (from -r requirements.txt (line 4)) (2.0.1+cu118)\n",
            "Collecting gradio (from -r requirements.txt (line 5))\n",
            "  Downloading gradio-3.36.1-py3-none-any.whl (19.8 MB)\n",
            "\u001b[2K     \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m19.8/19.8 MB\u001b[0m \u001b[31m87.6 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m\n",
            "\u001b[?25hCollecting mdtex2html (from -r requirements.txt (line 6))\n",
            "  Downloading mdtex2html-1.2.0-py3-none-any.whl (13 kB)\n",
            "Collecting sentencepiece (from -r requirements.txt (line 7))\n",
            "  Downloading sentencepiece-0.1.99-cp310-cp310-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (1.3 MB)\n",
            "\u001b[2K     \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m1.3/1.3 MB\u001b[0m \u001b[31m74.0 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m\n",
            "\u001b[?25hCollecting accelerate (from -r requirements.txt (line 8))\n",
            "  Downloading accelerate-0.21.0-py3-none-any.whl (244 kB)\n",
            "\u001b[2K     \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m244.2/244.2 kB\u001b[0m \u001b[31m29.7 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m\n",
            "\u001b[?25hCollecting sse-starlette (from -r requirements.txt (line 9))\n",
            "  Downloading sse_starlette-1.6.1-py3-none-any.whl (9.6 kB)\n",
            "Collecting streamlit>=1.24.0 (from -r requirements.txt (line 10))\n",
            "  Downloading streamlit-1.24.1-py2.py3-none-any.whl (8.9 MB)\n",
            "\u001b[2K     \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m8.9/8.9 MB\u001b[0m \u001b[31m82.1 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m\n",
            "\u001b[?25hRequirement already satisfied: filelock in /usr/local/lib/python3.10/dist-packages (from transformers==4.30.2->-r requirements.txt (line 2)) (3.12.2)\n",
            "Collecting huggingface-hub<1.0,>=0.14.1 (from transformers==4.30.2->-r requirements.txt (line 2))\n",
            "  Downloading huggingface_hub-0.16.4-py3-none-any.whl (268 kB)\n",
            "\u001b[2K     \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m268.8/268.8 kB\u001b[0m \u001b[31m32.0 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m\n",
            "\u001b[?25hRequirement already satisfied: numpy>=1.17 in /usr/local/lib/python3.10/dist-packages (from transformers==4.30.2->-r requirements.txt (line 2)) (1.22.4)\n",
            "Requirement already satisfied: packaging>=20.0 in /usr/local/lib/python3.10/dist-packages (from transformers==4.30.2->-r requirements.txt (line 2)) (23.1)\n",
            "Requirement already satisfied: pyyaml>=5.1 in /usr/local/lib/python3.10/dist-packages (from transformers==4.30.2->-r requirements.txt (line 2)) (6.0)\n",
            "Requirement already satisfied: regex!=2019.12.17 in /usr/local/lib/python3.10/dist-packages (from transformers==4.30.2->-r requirements.txt (line 2)) (2022.10.31)\n",
            "Requirement already satisfied: requests in /usr/local/lib/python3.10/dist-packages (from transformers==4.30.2->-r requirements.txt (line 2)) (2.27.1)\n",
            "Collecting tokenizers!=0.11.3,<0.14,>=0.11.1 (from transformers==4.30.2->-r requirements.txt (line 2))\n",
            "  Downloading tokenizers-0.13.3-cp310-cp310-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (7.8 MB)\n",
            "\u001b[2K     \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m7.8/7.8 MB\u001b[0m \u001b[31m131.7 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m\n",
            "\u001b[?25hCollecting safetensors>=0.3.1 (from transformers==4.30.2->-r requirements.txt (line 2))\n",
            "  Downloading safetensors-0.3.1-cp310-cp310-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (1.3 MB)\n",
            "\u001b[2K     \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m1.3/1.3 MB\u001b[0m \u001b[31m91.3 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m\n",
            "\u001b[?25hRequirement already satisfied: tqdm>=4.27 in /usr/local/lib/python3.10/dist-packages (from transformers==4.30.2->-r requirements.txt (line 2)) (4.65.0)\n",
            "Requirement already satisfied: typing-extensions in /usr/local/lib/python3.10/dist-packages (from torch>=2.0->-r requirements.txt (line 4)) (4.7.1)\n",
            "Requirement already satisfied: sympy in /usr/local/lib/python3.10/dist-packages (from torch>=2.0->-r requirements.txt (line 4)) (1.11.1)\n",
            "Requirement already satisfied: networkx in /usr/local/lib/python3.10/dist-packages (from torch>=2.0->-r requirements.txt (line 4)) (3.1)\n",
            "Requirement already satisfied: jinja2 in /usr/local/lib/python3.10/dist-packages (from torch>=2.0->-r requirements.txt (line 4)) (3.1.2)\n",
            "Requirement already satisfied: triton==2.0.0 in /usr/local/lib/python3.10/dist-packages (from torch>=2.0->-r requirements.txt (line 4)) (2.0.0)\n",
            "Requirement already satisfied: cmake in /usr/local/lib/python3.10/dist-packages (from triton==2.0.0->torch>=2.0->-r requirements.txt (line 4)) (3.25.2)\n",
            "Requirement already satisfied: lit in /usr/local/lib/python3.10/dist-packages (from triton==2.0.0->torch>=2.0->-r requirements.txt (line 4)) (16.0.6)\n",
            "Collecting aiofiles (from gradio->-r requirements.txt (line 5))\n",
            "  Downloading aiofiles-23.1.0-py3-none-any.whl (14 kB)\n",
            "Requirement already satisfied: aiohttp in /usr/local/lib/python3.10/dist-packages (from gradio->-r requirements.txt (line 5)) (3.8.4)\n",
            "Requirement already satisfied: altair>=4.2.0 in /usr/local/lib/python3.10/dist-packages (from gradio->-r requirements.txt (line 5)) (4.2.2)\n",
            "Collecting fastapi (from gradio->-r requirements.txt (line 5))\n",
            "  Downloading fastapi-0.100.0-py3-none-any.whl (65 kB)\n",
            "\u001b[2K     \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m65.7/65.7 kB\u001b[0m \u001b[31m9.6 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m\n",
            "\u001b[?25hCollecting ffmpy (from gradio->-r requirements.txt (line 5))\n",
            "  Downloading ffmpy-0.3.0.tar.gz (4.8 kB)\n",
            "  Preparing metadata (setup.py) ... \u001b[?25l\u001b[?25hdone\n",
            "Collecting gradio-client>=0.2.7 (from gradio->-r requirements.txt (line 5))\n",
            "  Downloading gradio_client-0.2.9-py3-none-any.whl (288 kB)\n",
            "\u001b[2K     \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m288.8/288.8 kB\u001b[0m \u001b[31m38.2 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m\n",
            "\u001b[?25hCollecting httpx (from gradio->-r requirements.txt (line 5))\n",
            "  Downloading httpx-0.24.1-py3-none-any.whl (75 kB)\n",
            "\u001b[2K     \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m75.4/75.4 kB\u001b[0m \u001b[31m11.1 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m\n",
            "\u001b[?25hRequirement already satisfied: markdown-it-py[linkify]>=2.0.0 in /usr/local/lib/python3.10/dist-packages (from gradio->-r requirements.txt (line 5)) (3.0.0)\n",
            "Requirement already satisfied: markupsafe in /usr/local/lib/python3.10/dist-packages (from gradio->-r requirements.txt (line 5)) (2.1.3)\n",
            "Requirement already satisfied: matplotlib in /usr/local/lib/python3.10/dist-packages (from gradio->-r requirements.txt (line 5)) (3.7.1)\n",
            "Collecting mdit-py-plugins<=0.3.3 (from gradio->-r requirements.txt (line 5))\n",
            "  Downloading mdit_py_plugins-0.3.3-py3-none-any.whl (50 kB)\n",
            "\u001b[2K     \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m50.5/50.5 kB\u001b[0m \u001b[31m7.8 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m\n",
            "\u001b[?25hCollecting orjson (from gradio->-r requirements.txt (line 5))\n",
            "  Downloading orjson-3.9.2-cp310-cp310-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (138 kB)\n",
            "\u001b[2K     \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m138.7/138.7 kB\u001b[0m \u001b[31m18.6 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m\n",
            "\u001b[?25hRequirement already satisfied: pandas in /usr/local/lib/python3.10/dist-packages (from gradio->-r requirements.txt (line 5)) (1.5.3)\n",
            "Requirement already satisfied: pillow in /usr/local/lib/python3.10/dist-packages (from gradio->-r requirements.txt (line 5)) (8.4.0)\n",
            "Requirement already satisfied: pydantic in /usr/local/lib/python3.10/dist-packages (from gradio->-r requirements.txt (line 5)) (1.10.11)\n",
            "Collecting pydub (from gradio->-r requirements.txt (line 5))\n",
            "  Downloading pydub-0.25.1-py2.py3-none-any.whl (32 kB)\n",
            "Requirement already satisfied: pygments>=2.12.0 in /usr/local/lib/python3.10/dist-packages (from gradio->-r requirements.txt (line 5)) (2.14.0)\n",
            "Collecting python-multipart (from gradio->-r requirements.txt (line 5))\n",
            "  Downloading python_multipart-0.0.6-py3-none-any.whl (45 kB)\n",
            "\u001b[2K     \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m45.7/45.7 kB\u001b[0m \u001b[31m6.5 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m\n",
            "\u001b[?25hCollecting semantic-version (from gradio->-r requirements.txt (line 5))\n",
            "  Downloading semantic_version-2.10.0-py2.py3-none-any.whl (15 kB)\n",
            "Collecting uvicorn>=0.14.0 (from gradio->-r requirements.txt (line 5))\n",
            "  Downloading uvicorn-0.22.0-py3-none-any.whl (58 kB)\n",
            "\u001b[2K     \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m58.3/58.3 kB\u001b[0m \u001b[31m7.6 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m\n",
            "\u001b[?25hCollecting websockets>=10.0 (from gradio->-r requirements.txt (line 5))\n",
            "  Downloading websockets-11.0.3-cp310-cp310-manylinux_2_5_x86_64.manylinux1_x86_64.manylinux_2_17_x86_64.manylinux2014_x86_64.whl (129 kB)\n",
            "\u001b[2K     \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m129.9/129.9 kB\u001b[0m \u001b[31m18.9 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m\n",
            "\u001b[?25hRequirement already satisfied: markdown in /usr/local/lib/python3.10/dist-packages (from mdtex2html->-r requirements.txt (line 6)) (3.4.3)\n",
            "Collecting latex2mathml (from mdtex2html->-r requirements.txt (line 6))\n",
            "  Downloading latex2mathml-3.76.0-py3-none-any.whl (73 kB)\n",
            "\u001b[2K     \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m73.4/73.4 kB\u001b[0m \u001b[31m10.7 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m\n",
            "\u001b[?25hRequirement already satisfied: psutil in /usr/local/lib/python3.10/dist-packages (from accelerate->-r requirements.txt (line 8)) (5.9.5)\n",
            "Collecting starlette (from sse-starlette->-r requirements.txt (line 9))\n",
            "  Downloading starlette-0.30.0-py3-none-any.whl (68 kB)\n",
            "\u001b[2K     \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m68.8/68.8 kB\u001b[0m \u001b[31m9.2 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m\n",
            "\u001b[?25hCollecting blinker<2,>=1.0.0 (from streamlit>=1.24.0->-r requirements.txt (line 10))\n",
            "  Downloading blinker-1.6.2-py3-none-any.whl (13 kB)\n",
            "Requirement already satisfied: cachetools<6,>=4.0 in /usr/local/lib/python3.10/dist-packages (from streamlit>=1.24.0->-r requirements.txt (line 10)) (5.3.1)\n",
            "Requirement already satisfied: click<9,>=7.0 in /usr/local/lib/python3.10/dist-packages (from streamlit>=1.24.0->-r requirements.txt (line 10)) (8.1.4)\n",
            "Collecting importlib-metadata<7,>=1.4 (from streamlit>=1.24.0->-r requirements.txt (line 10))\n",
            "  Downloading importlib_metadata-6.8.0-py3-none-any.whl (22 kB)\n",
            "Requirement already satisfied: pyarrow>=4.0 in /usr/local/lib/python3.10/dist-packages (from streamlit>=1.24.0->-r requirements.txt (line 10)) (9.0.0)\n",
            "Collecting pympler<2,>=0.9 (from streamlit>=1.24.0->-r requirements.txt (line 10))\n",
            "  Downloading Pympler-1.0.1-py3-none-any.whl (164 kB)\n",
            "\u001b[2K     \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m164.8/164.8 kB\u001b[0m \u001b[31m23.7 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m\n",
            "\u001b[?25hRequirement already satisfied: python-dateutil<3,>=2 in /usr/local/lib/python3.10/dist-packages (from streamlit>=1.24.0->-r requirements.txt (line 10)) (2.8.2)\n",
            "Requirement already satisfied: rich<14,>=10.11.0 in /usr/local/lib/python3.10/dist-packages (from streamlit>=1.24.0->-r requirements.txt (line 10)) (13.4.2)\n",
            "Requirement already satisfied: tenacity<9,>=8.0.0 in /usr/local/lib/python3.10/dist-packages (from streamlit>=1.24.0->-r requirements.txt (line 10)) (8.2.2)\n",
            "Requirement already satisfied: toml<2 in /usr/local/lib/python3.10/dist-packages (from streamlit>=1.24.0->-r requirements.txt (line 10)) (0.10.2)\n",
            "Collecting tzlocal<5,>=1.1 (from streamlit>=1.24.0->-r requirements.txt (line 10))\n",
            "  Downloading tzlocal-4.3.1-py3-none-any.whl (20 kB)\n",
            "Collecting validators<1,>=0.2 (from streamlit>=1.24.0->-r requirements.txt (line 10))\n",
            "  Downloading validators-0.20.0.tar.gz (30 kB)\n",
            "  Preparing metadata (setup.py) ... \u001b[?25l\u001b[?25hdone\n",
            "Collecting gitpython!=3.1.19,<4,>=3 (from streamlit>=1.24.0->-r requirements.txt (line 10))\n",
            "  Downloading GitPython-3.1.32-py3-none-any.whl (188 kB)\n",
            "\u001b[2K     \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m188.5/188.5 kB\u001b[0m \u001b[31m23.1 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m\n",
            "\u001b[?25hCollecting pydeck<1,>=0.1.dev5 (from streamlit>=1.24.0->-r requirements.txt (line 10))\n",
            "  Downloading pydeck-0.8.1b0-py2.py3-none-any.whl (4.8 MB)\n",
            "\u001b[2K     \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m4.8/4.8 MB\u001b[0m \u001b[31m118.5 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m\n",
            "\u001b[?25hRequirement already satisfied: tornado<7,>=6.0.3 in /usr/local/lib/python3.10/dist-packages (from streamlit>=1.24.0->-r requirements.txt (line 10)) (6.3.1)\n",
            "Collecting watchdog (from streamlit>=1.24.0->-r requirements.txt (line 10))\n",
            "  Downloading watchdog-3.0.0-py3-none-manylinux2014_x86_64.whl (82 kB)\n",
            "\u001b[2K     \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m82.1/82.1 kB\u001b[0m \u001b[31m10.4 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m\n",
            "\u001b[?25hRequirement already satisfied: entrypoints in /usr/local/lib/python3.10/dist-packages (from altair>=4.2.0->gradio->-r requirements.txt (line 5)) (0.4)\n",
            "Requirement already satisfied: jsonschema>=3.0 in /usr/local/lib/python3.10/dist-packages (from altair>=4.2.0->gradio->-r requirements.txt (line 5)) (4.3.3)\n",
            "Requirement already satisfied: toolz in /usr/local/lib/python3.10/dist-packages (from altair>=4.2.0->gradio->-r requirements.txt (line 5)) (0.12.0)\n",
            "Collecting gitdb<5,>=4.0.1 (from gitpython!=3.1.19,<4,>=3->streamlit>=1.24.0->-r requirements.txt (line 10))\n",
            "  Downloading gitdb-4.0.10-py3-none-any.whl (62 kB)\n",
            "\u001b[2K     \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m62.7/62.7 kB\u001b[0m \u001b[31m8.2 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m\n",
            "\u001b[?25hRequirement already satisfied: fsspec in /usr/local/lib/python3.10/dist-packages (from gradio-client>=0.2.7->gradio->-r requirements.txt (line 5)) (2023.6.0)\n",
            "Requirement already satisfied: zipp>=0.5 in /usr/local/lib/python3.10/dist-packages (from importlib-metadata<7,>=1.4->streamlit>=1.24.0->-r requirements.txt (line 10)) (3.16.0)\n",
            "Requirement already satisfied: mdurl~=0.1 in /usr/local/lib/python3.10/dist-packages (from markdown-it-py[linkify]>=2.0.0->gradio->-r requirements.txt (line 5)) (0.1.2)\n",
            "Collecting linkify-it-py<3,>=1 (from markdown-it-py[linkify]>=2.0.0->gradio->-r requirements.txt (line 5))\n",
            "  Downloading linkify_it_py-2.0.2-py3-none-any.whl (19 kB)\n",
            "INFO: pip is looking at multiple versions of mdit-py-plugins to determine which version is compatible with other requirements. This could take a while.\n",
            "Collecting mdit-py-plugins<=0.3.3 (from gradio->-r requirements.txt (line 5))\n",
            "  Downloading mdit_py_plugins-0.3.2-py3-none-any.whl (50 kB)\n",
            "\u001b[2K     \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m50.4/50.4 kB\u001b[0m \u001b[31m7.2 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m\n",
            "\u001b[?25h  Downloading mdit_py_plugins-0.3.1-py3-none-any.whl (46 kB)\n",
            "\u001b[2K     \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m46.5/46.5 kB\u001b[0m \u001b[31m6.0 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m\n",
            "\u001b[?25h  Downloading mdit_py_plugins-0.3.0-py3-none-any.whl (43 kB)\n",
            "\u001b[2K     \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m43.7/43.7 kB\u001b[0m \u001b[31m6.1 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m\n",
            "\u001b[?25h  Downloading mdit_py_plugins-0.2.8-py3-none-any.whl (41 kB)\n",
            "\u001b[2K     \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m41.0/41.0 kB\u001b[0m \u001b[31m5.5 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m\n",
            "\u001b[?25h  Downloading mdit_py_plugins-0.2.7-py3-none-any.whl (41 kB)\n",
            "\u001b[2K     \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m41.0/41.0 kB\u001b[0m \u001b[31m5.7 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m\n",
            "\u001b[?25h  Downloading mdit_py_plugins-0.2.6-py3-none-any.whl (39 kB)\n",
            "  Downloading mdit_py_plugins-0.2.5-py3-none-any.whl (39 kB)\n",
            "INFO: pip is looking at multiple versions of mdit-py-plugins to determine which version is compatible with other requirements. This could take a while.\n",
            "  Downloading mdit_py_plugins-0.2.4-py3-none-any.whl (39 kB)\n",
            "  Downloading mdit_py_plugins-0.2.3-py3-none-any.whl (39 kB)\n",
            "  Downloading mdit_py_plugins-0.2.2-py3-none-any.whl (39 kB)\n",
            "  Downloading mdit_py_plugins-0.2.1-py3-none-any.whl (38 kB)\n",
            "  Downloading mdit_py_plugins-0.2.0-py3-none-any.whl (38 kB)\n",
            "INFO: This is taking longer than usual. You might need to provide the dependency resolver with stricter constraints to reduce runtime. See https://pip.pypa.io/warnings/backtracking for guidance. If you want to abort this run, press Ctrl + C.\n",
            "  Downloading mdit_py_plugins-0.1.0-py3-none-any.whl (37 kB)\n",
            "Collecting markdown-it-py[linkify]>=2.0.0 (from gradio->-r requirements.txt (line 5))\n",
            "  Downloading markdown_it_py-3.0.0-py3-none-any.whl (87 kB)\n",
            "\u001b[2K     \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m87.5/87.5 kB\u001b[0m \u001b[31m12.7 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m\n",
            "\u001b[?25h  Downloading markdown_it_py-2.2.0-py3-none-any.whl (84 kB)\n",
            "\u001b[2K     \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m84.5/84.5 kB\u001b[0m \u001b[31m10.5 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m\n",
            "\u001b[?25hRequirement already satisfied: pytz>=2020.1 in /usr/local/lib/python3.10/dist-packages (from pandas->gradio->-r requirements.txt (line 5)) (2022.7.1)\n",
            "Requirement already satisfied: six>=1.5 in /usr/local/lib/python3.10/dist-packages (from python-dateutil<3,>=2->streamlit>=1.24.0->-r requirements.txt (line 10)) (1.16.0)\n",
            "Requirement already satisfied: urllib3<1.27,>=1.21.1 in /usr/local/lib/python3.10/dist-packages (from requests->transformers==4.30.2->-r requirements.txt (line 2)) (1.26.16)\n",
            "Requirement already satisfied: certifi>=2017.4.17 in /usr/local/lib/python3.10/dist-packages (from requests->transformers==4.30.2->-r requirements.txt (line 2)) (2023.5.7)\n",
            "Requirement already satisfied: charset-normalizer~=2.0.0 in /usr/local/lib/python3.10/dist-packages (from requests->transformers==4.30.2->-r requirements.txt (line 2)) (2.0.12)\n",
            "Requirement already satisfied: idna<4,>=2.5 in /usr/local/lib/python3.10/dist-packages (from requests->transformers==4.30.2->-r requirements.txt (line 2)) (3.4)\n",
            "Collecting pytz-deprecation-shim (from tzlocal<5,>=1.1->streamlit>=1.24.0->-r requirements.txt (line 10))\n",
            "  Downloading pytz_deprecation_shim-0.1.0.post0-py2.py3-none-any.whl (15 kB)\n",
            "Collecting h11>=0.8 (from uvicorn>=0.14.0->gradio->-r requirements.txt (line 5))\n",
            "  Downloading h11-0.14.0-py3-none-any.whl (58 kB)\n",
            "\u001b[2K     \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m58.3/58.3 kB\u001b[0m \u001b[31m8.8 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m\n",
            "\u001b[?25hRequirement already satisfied: decorator>=3.4.0 in /usr/local/lib/python3.10/dist-packages (from validators<1,>=0.2->streamlit>=1.24.0->-r requirements.txt (line 10)) (4.4.2)\n",
            "Requirement already satisfied: attrs>=17.3.0 in /usr/local/lib/python3.10/dist-packages (from aiohttp->gradio->-r requirements.txt (line 5)) (23.1.0)\n",
            "Requirement already satisfied: multidict<7.0,>=4.5 in /usr/local/lib/python3.10/dist-packages (from aiohttp->gradio->-r requirements.txt (line 5)) (6.0.4)\n",
            "Requirement already satisfied: async-timeout<5.0,>=4.0.0a3 in /usr/local/lib/python3.10/dist-packages (from aiohttp->gradio->-r requirements.txt (line 5)) (4.0.2)\n",
            "Requirement already satisfied: yarl<2.0,>=1.0 in /usr/local/lib/python3.10/dist-packages (from aiohttp->gradio->-r requirements.txt (line 5)) (1.9.2)\n",
            "Requirement already satisfied: frozenlist>=1.1.1 in /usr/local/lib/python3.10/dist-packages (from aiohttp->gradio->-r requirements.txt (line 5)) (1.3.3)\n",
            "Requirement already satisfied: aiosignal>=1.1.2 in /usr/local/lib/python3.10/dist-packages (from aiohttp->gradio->-r requirements.txt (line 5)) (1.3.1)\n",
            "Collecting starlette (from sse-starlette->-r requirements.txt (line 9))\n",
            "  Downloading starlette-0.27.0-py3-none-any.whl (66 kB)\n",
            "\u001b[2K     \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m67.0/67.0 kB\u001b[0m \u001b[31m8.2 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m\n",
            "\u001b[?25hRequirement already satisfied: anyio<5,>=3.4.0 in /usr/local/lib/python3.10/dist-packages (from starlette->sse-starlette->-r requirements.txt (line 9)) (3.7.1)\n",
            "Collecting httpcore<0.18.0,>=0.15.0 (from httpx->gradio->-r requirements.txt (line 5))\n",
            "  Downloading httpcore-0.17.3-py3-none-any.whl (74 kB)\n",
            "\u001b[2K     \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m74.5/74.5 kB\u001b[0m \u001b[31m11.1 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m\n",
            "\u001b[?25hRequirement already satisfied: sniffio in /usr/local/lib/python3.10/dist-packages (from httpx->gradio->-r requirements.txt (line 5)) (1.3.0)\n",
            "Requirement already satisfied: contourpy>=1.0.1 in /usr/local/lib/python3.10/dist-packages (from matplotlib->gradio->-r requirements.txt (line 5)) (1.1.0)\n",
            "Requirement already satisfied: cycler>=0.10 in /usr/local/lib/python3.10/dist-packages (from matplotlib->gradio->-r requirements.txt (line 5)) (0.11.0)\n",
            "Requirement already satisfied: fonttools>=4.22.0 in /usr/local/lib/python3.10/dist-packages (from matplotlib->gradio->-r requirements.txt (line 5)) (4.40.0)\n",
            "Requirement already satisfied: kiwisolver>=1.0.1 in /usr/local/lib/python3.10/dist-packages (from matplotlib->gradio->-r requirements.txt (line 5)) (1.4.4)\n",
            "Requirement already satisfied: pyparsing>=2.3.1 in /usr/local/lib/python3.10/dist-packages (from matplotlib->gradio->-r requirements.txt (line 5)) (3.1.0)\n",
            "Requirement already satisfied: mpmath>=0.19 in /usr/local/lib/python3.10/dist-packages (from sympy->torch>=2.0->-r requirements.txt (line 4)) (1.3.0)\n",
            "Requirement already satisfied: exceptiongroup in /usr/local/lib/python3.10/dist-packages (from anyio<5,>=3.4.0->starlette->sse-starlette->-r requirements.txt (line 9)) (1.1.2)\n",
            "Collecting smmap<6,>=3.0.1 (from gitdb<5,>=4.0.1->gitpython!=3.1.19,<4,>=3->streamlit>=1.24.0->-r requirements.txt (line 10))\n",
            "  Downloading smmap-5.0.0-py3-none-any.whl (24 kB)\n",
            "Requirement already satisfied: pyrsistent!=0.17.0,!=0.17.1,!=0.17.2,>=0.14.0 in /usr/local/lib/python3.10/dist-packages (from jsonschema>=3.0->altair>=4.2.0->gradio->-r requirements.txt (line 5)) (0.19.3)\n",
            "Collecting uc-micro-py (from linkify-it-py<3,>=1->markdown-it-py[linkify]>=2.0.0->gradio->-r requirements.txt (line 5))\n",
            "  Downloading uc_micro_py-1.0.2-py3-none-any.whl (6.2 kB)\n",
            "Collecting tzdata (from pytz-deprecation-shim->tzlocal<5,>=1.1->streamlit>=1.24.0->-r requirements.txt (line 10))\n",
            "  Downloading tzdata-2023.3-py2.py3-none-any.whl (341 kB)\n",
            "\u001b[2K     \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m341.8/341.8 kB\u001b[0m \u001b[31m41.0 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m\n",
            "\u001b[?25hBuilding wheels for collected packages: validators, ffmpy\n",
            "  Building wheel for validators (setup.py) ... \u001b[?25l\u001b[?25hdone\n",
            "  Created wheel for validators: filename=validators-0.20.0-py3-none-any.whl size=19579 sha256=1f0dad8e73847c91ed907cc5c6802850522716d741f14e82c0476fc0d9b08caa\n",
            "  Stored in directory: /root/.cache/pip/wheels/f2/ed/dd/d3a556ad245ef9dc570c6bcd2f22886d17b0b408dd3bbb9ac3\n",
            "  Building wheel for ffmpy (setup.py) ... \u001b[?25l\u001b[?25hdone\n",
            "  Created wheel for ffmpy: filename=ffmpy-0.3.0-py3-none-any.whl size=4694 sha256=583f18c401001b5a646d0d6f78d13b330f4bf281fe63aada10045d25e76e3f7e\n",
            "  Stored in directory: /root/.cache/pip/wheels/0c/c2/0e/3b9c6845c6a4e35beb90910cc70d9ac9ab5d47402bd62af0df\n",
            "Successfully built validators ffmpy\n",
            "Installing collected packages: tokenizers, sentencepiece, safetensors, pydub, ffmpy, cpm_kernels, websockets, watchdog, validators, uc-micro-py, tzdata, smmap, semantic-version, python-multipart, pympler, orjson, markdown-it-py, latex2mathml, importlib-metadata, h11, blinker, aiofiles, uvicorn, starlette, pytz-deprecation-shim, pydeck, mdtex2html, mdit-py-plugins, linkify-it-py, huggingface-hub, httpcore, gitdb, tzlocal, transformers, sse-starlette, httpx, gitpython, fastapi, streamlit, gradio-client, gradio, accelerate\n",
            "  Attempting uninstall: markdown-it-py\n",
            "    Found existing installation: markdown-it-py 3.0.0\n",
            "    Uninstalling markdown-it-py-3.0.0:\n",
            "      Successfully uninstalled markdown-it-py-3.0.0\n",
            "  Attempting uninstall: tzlocal\n",
            "    Found existing installation: tzlocal 5.0.1\n",
            "    Uninstalling tzlocal-5.0.1:\n",
            "      Successfully uninstalled tzlocal-5.0.1\n",
            "Successfully installed accelerate-0.21.0 aiofiles-23.1.0 blinker-1.6.2 cpm_kernels-1.0.11 fastapi-0.100.0 ffmpy-0.3.0 gitdb-4.0.10 gitpython-3.1.32 gradio-3.36.1 gradio-client-0.2.9 h11-0.14.0 httpcore-0.17.3 httpx-0.24.1 huggingface-hub-0.16.4 importlib-metadata-6.8.0 latex2mathml-3.76.0 linkify-it-py-2.0.2 markdown-it-py-2.2.0 mdit-py-plugins-0.3.3 mdtex2html-1.2.0 orjson-3.9.2 pydeck-0.8.1b0 pydub-0.25.1 pympler-1.0.1 python-multipart-0.0.6 pytz-deprecation-shim-0.1.0.post0 safetensors-0.3.1 semantic-version-2.10.0 sentencepiece-0.1.99 smmap-5.0.0 sse-starlette-1.6.1 starlette-0.27.0 streamlit-1.24.1 tokenizers-0.13.3 transformers-4.30.2 tzdata-2023.3 tzlocal-4.3.1 uc-micro-py-1.0.2 uvicorn-0.22.0 validators-0.20.0 watchdog-3.0.0 websockets-11.0.3\n",
            "Requirement already satisfied: accelerate in /usr/local/lib/python3.10/dist-packages (0.21.0)\n",
            "Requirement already satisfied: numpy>=1.17 in /usr/local/lib/python3.10/dist-packages (from accelerate) (1.22.4)\n",
            "Requirement already satisfied: packaging>=20.0 in /usr/local/lib/python3.10/dist-packages (from accelerate) (23.1)\n",
            "Requirement already satisfied: psutil in /usr/local/lib/python3.10/dist-packages (from accelerate) (5.9.5)\n",
            "Requirement already satisfied: pyyaml in /usr/local/lib/python3.10/dist-packages (from accelerate) (6.0)\n",
            "Requirement already satisfied: torch>=1.10.0 in /usr/local/lib/python3.10/dist-packages (from accelerate) (2.0.1+cu118)\n",
            "Requirement already satisfied: filelock in /usr/local/lib/python3.10/dist-packages (from torch>=1.10.0->accelerate) (3.12.2)\n",
            "Requirement already satisfied: typing-extensions in /usr/local/lib/python3.10/dist-packages (from torch>=1.10.0->accelerate) (4.7.1)\n",
            "Requirement already satisfied: sympy in /usr/local/lib/python3.10/dist-packages (from torch>=1.10.0->accelerate) (1.11.1)\n",
            "Requirement already satisfied: networkx in /usr/local/lib/python3.10/dist-packages (from torch>=1.10.0->accelerate) (3.1)\n",
            "Requirement already satisfied: jinja2 in /usr/local/lib/python3.10/dist-packages (from torch>=1.10.0->accelerate) (3.1.2)\n",
            "Requirement already satisfied: triton==2.0.0 in /usr/local/lib/python3.10/dist-packages (from torch>=1.10.0->accelerate) (2.0.0)\n",
            "Requirement already satisfied: cmake in /usr/local/lib/python3.10/dist-packages (from triton==2.0.0->torch>=1.10.0->accelerate) (3.25.2)\n",
            "Requirement already satisfied: lit in /usr/local/lib/python3.10/dist-packages (from triton==2.0.0->torch>=1.10.0->accelerate) (16.0.6)\n",
            "Requirement already satisfied: MarkupSafe>=2.0 in /usr/local/lib/python3.10/dist-packages (from jinja2->torch>=1.10.0->accelerate) (2.1.3)\n",
            "Requirement already satisfied: mpmath>=0.19 in /usr/local/lib/python3.10/dist-packages (from sympy->torch>=1.10.0->accelerate) (1.3.0)\n",
            "Collecting datasets\n",
            "  Downloading datasets-2.13.1-py3-none-any.whl (486 kB)\n",
            "\u001b[2K     \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m486.2/486.2 kB\u001b[0m \u001b[31m26.1 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m\n",
            "\u001b[?25hRequirement already satisfied: numpy>=1.17 in /usr/local/lib/python3.10/dist-packages (from datasets) (1.22.4)\n",
            "Requirement already satisfied: pyarrow>=8.0.0 in /usr/local/lib/python3.10/dist-packages (from datasets) (9.0.0)\n",
            "Collecting dill<0.3.7,>=0.3.0 (from datasets)\n",
            "  Downloading dill-0.3.6-py3-none-any.whl (110 kB)\n",
            "\u001b[2K     \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m110.5/110.5 kB\u001b[0m \u001b[31m1.5 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m\n",
            "\u001b[?25hRequirement already satisfied: pandas in /usr/local/lib/python3.10/dist-packages (from datasets) (1.5.3)\n",
            "Requirement already satisfied: requests>=2.19.0 in /usr/local/lib/python3.10/dist-packages (from datasets) (2.27.1)\n",
            "Requirement already satisfied: tqdm>=4.62.1 in /usr/local/lib/python3.10/dist-packages (from datasets) (4.65.0)\n",
            "Collecting xxhash (from datasets)\n",
            "  Downloading xxhash-3.2.0-cp310-cp310-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (212 kB)\n",
            "\u001b[2K     \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m212.5/212.5 kB\u001b[0m \u001b[31m926.5 kB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m\n",
            "\u001b[?25hCollecting multiprocess (from datasets)\n",
            "  Downloading multiprocess-0.70.14-py310-none-any.whl (134 kB)\n",
            "\u001b[2K     \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m134.3/134.3 kB\u001b[0m \u001b[31m18.1 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m\n",
            "\u001b[?25hRequirement already satisfied: fsspec[http]>=2021.11.1 in /usr/local/lib/python3.10/dist-packages (from datasets) (2023.6.0)\n",
            "Requirement already satisfied: aiohttp in /usr/local/lib/python3.10/dist-packages (from datasets) (3.8.4)\n",
            "Requirement already satisfied: huggingface-hub<1.0.0,>=0.11.0 in /usr/local/lib/python3.10/dist-packages (from datasets) (0.16.4)\n",
            "Requirement already satisfied: packaging in /usr/local/lib/python3.10/dist-packages (from datasets) (23.1)\n",
            "Requirement already satisfied: pyyaml>=5.1 in /usr/local/lib/python3.10/dist-packages (from datasets) (6.0)\n",
            "Requirement already satisfied: attrs>=17.3.0 in /usr/local/lib/python3.10/dist-packages (from aiohttp->datasets) (23.1.0)\n",
            "Requirement already satisfied: charset-normalizer<4.0,>=2.0 in /usr/local/lib/python3.10/dist-packages (from aiohttp->datasets) (2.0.12)\n",
            "Requirement already satisfied: multidict<7.0,>=4.5 in /usr/local/lib/python3.10/dist-packages (from aiohttp->datasets) (6.0.4)\n",
            "Requirement already satisfied: async-timeout<5.0,>=4.0.0a3 in /usr/local/lib/python3.10/dist-packages (from aiohttp->datasets) (4.0.2)\n",
            "Requirement already satisfied: yarl<2.0,>=1.0 in /usr/local/lib/python3.10/dist-packages (from aiohttp->datasets) (1.9.2)\n",
            "Requirement already satisfied: frozenlist>=1.1.1 in /usr/local/lib/python3.10/dist-packages (from aiohttp->datasets) (1.3.3)\n",
            "Requirement already satisfied: aiosignal>=1.1.2 in /usr/local/lib/python3.10/dist-packages (from aiohttp->datasets) (1.3.1)\n",
            "Requirement already satisfied: filelock in /usr/local/lib/python3.10/dist-packages (from huggingface-hub<1.0.0,>=0.11.0->datasets) (3.12.2)\n",
            "Requirement already satisfied: typing-extensions>=3.7.4.3 in /usr/local/lib/python3.10/dist-packages (from huggingface-hub<1.0.0,>=0.11.0->datasets) (4.7.1)\n",
            "Requirement already satisfied: urllib3<1.27,>=1.21.1 in /usr/local/lib/python3.10/dist-packages (from requests>=2.19.0->datasets) (1.26.16)\n",
            "Requirement already satisfied: certifi>=2017.4.17 in /usr/local/lib/python3.10/dist-packages (from requests>=2.19.0->datasets) (2023.5.7)\n",
            "Requirement already satisfied: idna<4,>=2.5 in /usr/local/lib/python3.10/dist-packages (from requests>=2.19.0->datasets) (3.4)\n",
            "Requirement already satisfied: python-dateutil>=2.8.1 in /usr/local/lib/python3.10/dist-packages (from pandas->datasets) (2.8.2)\n",
            "Requirement already satisfied: pytz>=2020.1 in /usr/local/lib/python3.10/dist-packages (from pandas->datasets) (2022.7.1)\n",
            "Requirement already satisfied: six>=1.5 in /usr/local/lib/python3.10/dist-packages (from python-dateutil>=2.8.1->pandas->datasets) (1.16.0)\n",
            "Installing collected packages: xxhash, dill, multiprocess, datasets\n",
            "Successfully installed datasets-2.13.1 dill-0.3.6 multiprocess-0.70.14 xxhash-3.2.0\n",
            "Collecting peft\n",
            "  Downloading peft-0.3.0-py3-none-any.whl (56 kB)\n",
            "\u001b[2K     \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m56.8/56.8 kB\u001b[0m \u001b[31m6.4 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m\n",
            "\u001b[?25hRequirement already satisfied: numpy>=1.17 in /usr/local/lib/python3.10/dist-packages (from peft) (1.22.4)\n",
            "Requirement already satisfied: packaging>=20.0 in /usr/local/lib/python3.10/dist-packages (from peft) (23.1)\n",
            "Requirement already satisfied: psutil in /usr/local/lib/python3.10/dist-packages (from peft) (5.9.5)\n",
            "Requirement already satisfied: pyyaml in /usr/local/lib/python3.10/dist-packages (from peft) (6.0)\n",
            "Requirement already satisfied: torch>=1.13.0 in /usr/local/lib/python3.10/dist-packages (from peft) (2.0.1+cu118)\n",
            "Requirement already satisfied: transformers in /usr/local/lib/python3.10/dist-packages (from peft) (4.30.2)\n",
            "Requirement already satisfied: accelerate in /usr/local/lib/python3.10/dist-packages (from peft) (0.21.0)\n",
            "Requirement already satisfied: filelock in /usr/local/lib/python3.10/dist-packages (from torch>=1.13.0->peft) (3.12.2)\n",
            "Requirement already satisfied: typing-extensions in /usr/local/lib/python3.10/dist-packages (from torch>=1.13.0->peft) (4.7.1)\n",
            "Requirement already satisfied: sympy in /usr/local/lib/python3.10/dist-packages (from torch>=1.13.0->peft) (1.11.1)\n",
            "Requirement already satisfied: networkx in /usr/local/lib/python3.10/dist-packages (from torch>=1.13.0->peft) (3.1)\n",
            "Requirement already satisfied: jinja2 in /usr/local/lib/python3.10/dist-packages (from torch>=1.13.0->peft) (3.1.2)\n",
            "Requirement already satisfied: triton==2.0.0 in /usr/local/lib/python3.10/dist-packages (from torch>=1.13.0->peft) (2.0.0)\n",
            "Requirement already satisfied: cmake in /usr/local/lib/python3.10/dist-packages (from triton==2.0.0->torch>=1.13.0->peft) (3.25.2)\n",
            "Requirement already satisfied: lit in /usr/local/lib/python3.10/dist-packages (from triton==2.0.0->torch>=1.13.0->peft) (16.0.6)\n",
            "Requirement already satisfied: huggingface-hub<1.0,>=0.14.1 in /usr/local/lib/python3.10/dist-packages (from transformers->peft) (0.16.4)\n",
            "Requirement already satisfied: regex!=2019.12.17 in /usr/local/lib/python3.10/dist-packages (from transformers->peft) (2022.10.31)\n",
            "Requirement already satisfied: requests in /usr/local/lib/python3.10/dist-packages (from transformers->peft) (2.27.1)\n",
            "Requirement already satisfied: tokenizers!=0.11.3,<0.14,>=0.11.1 in /usr/local/lib/python3.10/dist-packages (from transformers->peft) (0.13.3)\n",
            "Requirement already satisfied: safetensors>=0.3.1 in /usr/local/lib/python3.10/dist-packages (from transformers->peft) (0.3.1)\n",
            "Requirement already satisfied: tqdm>=4.27 in /usr/local/lib/python3.10/dist-packages (from transformers->peft) (4.65.0)\n",
            "Requirement already satisfied: fsspec in /usr/local/lib/python3.10/dist-packages (from huggingface-hub<1.0,>=0.14.1->transformers->peft) (2023.6.0)\n",
            "Requirement already satisfied: MarkupSafe>=2.0 in /usr/local/lib/python3.10/dist-packages (from jinja2->torch>=1.13.0->peft) (2.1.3)\n",
            "Requirement already satisfied: urllib3<1.27,>=1.21.1 in /usr/local/lib/python3.10/dist-packages (from requests->transformers->peft) (1.26.16)\n",
            "Requirement already satisfied: certifi>=2017.4.17 in /usr/local/lib/python3.10/dist-packages (from requests->transformers->peft) (2023.5.7)\n",
            "Requirement already satisfied: charset-normalizer~=2.0.0 in /usr/local/lib/python3.10/dist-packages (from requests->transformers->peft) (2.0.12)\n",
            "Requirement already satisfied: idna<4,>=2.5 in /usr/local/lib/python3.10/dist-packages (from requests->transformers->peft) (3.4)\n",
            "Requirement already satisfied: mpmath>=0.19 in /usr/local/lib/python3.10/dist-packages (from sympy->torch>=1.13.0->peft) (1.3.0)\n",
            "Installing collected packages: peft\n",
            "Successfully installed peft-0.3.0\n",
            "Collecting torchkeras\n",
            "  Downloading torchkeras-3.9.1-py3-none-any.whl (6.5 MB)\n",
            "\u001b[2K     \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m6.5/6.5 MB\u001b[0m \u001b[31m54.0 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m\n",
            "\u001b[?25hRequirement already satisfied: accelerate in /usr/local/lib/python3.10/dist-packages (from torchkeras) (0.21.0)\n",
            "Requirement already satisfied: tqdm in /usr/local/lib/python3.10/dist-packages (from torchkeras) (4.65.0)\n",
            "Requirement already satisfied: numpy>=1.17 in /usr/local/lib/python3.10/dist-packages (from accelerate->torchkeras) (1.22.4)\n",
            "Requirement already satisfied: packaging>=20.0 in /usr/local/lib/python3.10/dist-packages (from accelerate->torchkeras) (23.1)\n",
            "Requirement already satisfied: psutil in /usr/local/lib/python3.10/dist-packages (from accelerate->torchkeras) (5.9.5)\n",
            "Requirement already satisfied: pyyaml in /usr/local/lib/python3.10/dist-packages (from accelerate->torchkeras) (6.0)\n",
            "Requirement already satisfied: torch>=1.10.0 in /usr/local/lib/python3.10/dist-packages (from accelerate->torchkeras) (2.0.1+cu118)\n",
            "Requirement already satisfied: filelock in /usr/local/lib/python3.10/dist-packages (from torch>=1.10.0->accelerate->torchkeras) (3.12.2)\n",
            "Requirement already satisfied: typing-extensions in /usr/local/lib/python3.10/dist-packages (from torch>=1.10.0->accelerate->torchkeras) (4.7.1)\n",
            "Requirement already satisfied: sympy in /usr/local/lib/python3.10/dist-packages (from torch>=1.10.0->accelerate->torchkeras) (1.11.1)\n",
            "Requirement already satisfied: networkx in /usr/local/lib/python3.10/dist-packages (from torch>=1.10.0->accelerate->torchkeras) (3.1)\n",
            "Requirement already satisfied: jinja2 in /usr/local/lib/python3.10/dist-packages (from torch>=1.10.0->accelerate->torchkeras) (3.1.2)\n",
            "Requirement already satisfied: triton==2.0.0 in /usr/local/lib/python3.10/dist-packages (from torch>=1.10.0->accelerate->torchkeras) (2.0.0)\n",
            "Requirement already satisfied: cmake in /usr/local/lib/python3.10/dist-packages (from triton==2.0.0->torch>=1.10.0->accelerate->torchkeras) (3.25.2)\n",
            "Requirement already satisfied: lit in /usr/local/lib/python3.10/dist-packages (from triton==2.0.0->torch>=1.10.0->accelerate->torchkeras) (16.0.6)\n",
            "Requirement already satisfied: MarkupSafe>=2.0 in /usr/local/lib/python3.10/dist-packages (from jinja2->torch>=1.10.0->accelerate->torchkeras) (2.1.3)\n",
            "Requirement already satisfied: mpmath>=0.19 in /usr/local/lib/python3.10/dist-packages (from sympy->torch>=1.10.0->accelerate->torchkeras) (1.3.0)\n",
            "Installing collected packages: torchkeras\n",
            "Successfully installed torchkeras-3.9.1\n"
          ]
        }
      ],
      "source": [
        "# ChatGLM2\n",
        "\n",
        "!pip install -r requirements.txt\n",
        "\n",
        "# finetune\n",
        "!pip install -U accelerate\n",
        "!pip install datasets\n",
        "!pip install -U peft\n",
        "!pip install -U torchkeras"
      ]
    },
    {
      "cell_type": "code",
      "source": [
        "# 导入常用模块\n",
        "\n",
        "import numpy as np\n",
        "import pandas as pd\n",
        "import torch\n",
        "from torch import nn\n",
        "from torch.utils.data import Dataset,DataLoader"
      ],
      "metadata": {
        "id": "TcZEU0xn1_at"
      },
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "code",
      "source": [
        "# 配置参数\n",
        "# 设置一个命名空间对象cfg的属性\n",
        "# 命名空间是一个将名称映射到对象的容器\n",
        "# cfg对象上设置了一系列的属性，以存储训练模型和处理数据集时所需的各种参数\n",
        "# 你可以理解成一个更加复杂的python-dict，与dict的区别在于，dict调用属性的方式更加复杂，如dictA['xx'], 而namespace调用属性可以使用 cfg.xx\n",
        "# more like Javascript\n",
        "from argparse import Namespace\n",
        "cfg = Namespace()\n",
        "\n",
        "#dataset\n",
        "cfg.prompt_column = 'prompt'\n",
        "cfg.response_column = 'response'\n",
        "cfg.history_column = None\n",
        "cfg.source_prefix = '' #添加到每个prompt开头的前缀引导语\n",
        "\n",
        "cfg.max_source_length = 128\n",
        "cfg.max_target_length = 128\n",
        "\n",
        "#model\n",
        "cfg.model_name_or_path = 'THUDM/chatglm2-6b'  #远程'THUDM/chatglm-6b'\n",
        "cfg.quantization_bit = None #仅仅预测时可以选 4 or 8\n",
        "\n",
        "\n",
        "#train\n",
        "cfg.epochs = 100\n",
        "cfg.lr = 5e-3\n",
        "cfg.batch_size = 1\n",
        "cfg.gradient_accumulation_steps = 16 #梯度累积"
      ],
      "metadata": {
        "id": "Yw1MQc6q2EPA"
      },
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "source": [
        "## 获取已经预训练好的ChatGLM2模型并验证其可运行"
      ],
      "metadata": {
        "id": "Jy3J5fvW3R_G"
      }
    },
    {
      "cell_type": "code",
      "source": [
        "import transformers\n",
        "from transformers import  AutoModel,AutoTokenizer,AutoConfig,DataCollatorForSeq2Seq\n",
        "\n",
        "\n",
        "config = AutoConfig.from_pretrained(cfg.model_name_or_path, trust_remote_code=True)\n",
        "\n",
        "tokenizer = AutoTokenizer.from_pretrained(\n",
        "    cfg.model_name_or_path, trust_remote_code=True)\n",
        "\n",
        "model = AutoModel.from_pretrained(cfg.model_name_or_path,config=config,\n",
        "                                  trust_remote_code=True).half()\n",
        "\n",
        "# 如果你设置了量化精度的参数，那么会先量化瘦身为4bit或8bit\n",
        "# 通过这个配置在模型推断阶段用来减少模型大小和提高运行速度，但可能会牺牲一定的精度\n",
        "# 如果你的GPU不太行，那么这个设置是必要的\n",
        "if cfg.quantization_bit is not None:\n",
        "    print(f\"Quantized to {cfg.quantization_bit} bit\")\n",
        "    model = model.quantize(cfg.quantization_bit)\n",
        "\n",
        "# 将模型参数转移到GPU上\n",
        "model = model.cuda();"
      ],
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 793,
          "referenced_widgets": [
            "a2e32e131e0647028723d43810781f50",
            "24466079b7384ecc9c0c80949aa4f5e5",
            "b2f00c912b1f4e2abd856e010a64fbc2",
            "cbcc16a54bbb4c63aca716edb64468af",
            "a5bc2072474c4f67890b13179aeae52d",
            "1e288ce8a387483894087aec932744eb",
            "8f2a6645d2e8424997a442d9a7eab426",
            "dbca5b0fbba140a3bc9400ad172d97f3",
            "abb1e521c00c4c0a8f3c68e6b4439fd3",
            "18728a8d5b064bdfaf1311506c32cf4c",
            "7258c692bb8f42c397c487bb70356877",
            "012e1efa0aad4946be8c565778ca1e1d",
            "e9df530a75df4017a60ee049638ea8d5",
            "416bcd88823a4745b49bfd4576ff851b",
            "362c33fd726b41e7bad6df86744b7811",
            "9968224047494a6c971f69f4eee57f11",
            "59e590c4fb1f4857acaea38256f1a3b3",
            "7310888be31d4261a6fc95505da2d135",
            "339c90f3e7a84d31a94687a315ecef94",
            "d6782fec37f24daf916a7e94710ee2a1",
            "2f268a6613d241d783ed66a5c18cab48",
            "b56a458d618f4057b12a14a82b05f048",
            "4a56e10642724f7498f5ab079092353e",
            "f91eba59b49e4800ad38fd95e513ee9b",
            "551472fece2249108d6d90f2cc18f826",
            "6f39280c3ff64a33afcd7d93fc613cb9",
            "63a9a52c1eb44c9782ae0effd2ca75a9",
            "efa7b35ae96a4624a25990bc115d785c",
            "994cb43b8c6b4adfa62a872ad63d1735",
            "a4cad29f70f0424b9036cf2ead8f7a23",
            "f510fe1a9e3643ce8f7d02e1fcd54ae9",
            "62244056b89b4dbc8f6b9055d810bc00",
            "744a91498a934e91b5d3563b343ad792",
            "cbb72d2f96934f33aaf5efc671b3bb16",
            "f19d596055334116aaa55413a3d0de2b",
            "ae5a27fd0d1b4ef4ae4c166019342534",
            "4d3cc2d6ec154bfe93e2dc546ad36e5b",
            "753aa5dcc7f84b16863fc0958634c913",
            "d5418a835e054a839ecfed355cd45648",
            "636758ae082c4b65b9084c7025291701",
            "d831a64501714b378bb8adc8c0d6aed0",
            "4bd29b0cd09148de8e3a8e85eee2abdd",
            "9947a6d1ffe7405490360df78afd255c",
            "47ef6a2388fd4726b608e1570bc61433",
            "76a8a5cda1ae459283e3dcdbd430c8ca",
            "7de95de31ade43988eb03471685f0a1b",
            "c6410d4ef46940ed8ce870ea9a2c8840",
            "aa0d40071fd34a1e808980fde3c7bae8",
            "afd194f1836c4c3a938ee5bbc1ce1dd6",
            "68fbfbeddf354d96bddde7516acb99c5",
            "a0836b7c94f64ecda7e23fb34972e5ef",
            "bdbb4d5e2e6d4652938d4e00519bf440",
            "66eba329f64f49e597018a4199a4c9c1",
            "91d835841b394ceb8cfdce481b38c6c0",
            "a8000b01379a4f25b2c967ac2468b061",
            "59d7c1cc2cc5477fabcffa6efdaf26a5",
            "685293b3ebda471685f2a47d9cf7d4e1",
            "63e9c15976ff486fa4655b5ae7990491",
            "171d4a7bf5864013aabc562f67feb43f",
            "5e192c4134074ad2b369748d5bb16c21",
            "872dbbee0d814dc79549da110cffc49a",
            "ad6ad77aa50f435ab1d8dc8d846303ff",
            "440bb6c594814f509d4cb6dcb7f6e54f",
            "76d07cdfa8434d9b976dd0807b9a9b59",
            "c76d57fc0d6246fea713ba6f32315549",
            "916c3edcb8de4dbcbc7a8287b5158694",
            "443c906966a74ec1a29236630e646774",
            "9d65edf76d04418fbcb60af5c4c58ace",
            "ed5870ac88744c5ab4d26bf24404291d",
            "68db923d9b034487859a37f9c1922e25",
            "4c33553088bb4971801d3868216d0b8f",
            "fce4a156b78742d39d366c41179c96ac",
            "a58b411911e244c4a67131db4652b3f1",
            "bf04053881ca496d89c33e86f84d7951",
            "68875644fffe4ba9b91a353fd9439e08",
            "2a977c56dae44acdabec8fe25c6a88f3",
            "c48c2b7d9d4247efb3b8f25c67285c1a",
            "c4b2d9acd2224a98a64e95641129938d",
            "02e3e55fcf1942549c2e9bccf903c16f",
            "70b3f544a16045d589ae7430345a311f",
            "163bea237222473084acd5fd1800efe9",
            "92cc630c42c24c779de66211c316b99d",
            "f05df5e58d3943bfb358f11e0467c30c",
            "b6b5c15809b640aa8af345b199d21ba8",
            "53039514658a41bea0b2d96ad5a46787",
            "13f9b47f37cf441cbe012087af1a2ac7",
            "bf75ece1bc3146849ea0629f3445301d",
            "06b720e8c2ad4a5395b960301d1f1ffd",
            "3ca728fb3e294384898b531bbfa126d6",
            "23090f6cf40c45c38ad0c58aec75a383",
            "a59ef5f659064e3d8a1507d191c75397",
            "9f14d803ab704f37a111fe17644a298b",
            "dcb9e3a53db9435a9e6a98e9b193cee4",
            "ff56016623a4414bbbd0c83c07618ce7",
            "9d15210862ec41729ef999b64faa4310",
            "d9ae600340eb4386a0850b0c4e851d03",
            "ac1ff0c5130f4b3cb80e7b78f6454bc7",
            "2b76d5330e5a40699c087882f31e9af5",
            "0e8ae3805c4e426899db73c25b9653b3",
            "48d6957605774e9d8986e011966f9782",
            "a7824e6e2ed04a1684add98e8c42b491",
            "cc2498d5ee104894bf5180d1656ffacb",
            "5843041c6a5d417f8c574c4d8cfa5993",
            "678bbef0353b4be3b19e4a16de1aa904",
            "b99da94ec80e4387a988eb6fd9f15ed1",
            "8c8ec1431afb4cef9884c358a1382827",
            "be8bdf59868c4f1e9482c5cddc817af4",
            "a74b15fd0b09451591decd7e1b37e2ad",
            "8505c14ed4f44891948c30df4d21f46e",
            "1f63a4bf31ea41d6a69f2c2e7556e100",
            "935bcfdf1ece41828e370350a108ae80",
            "da82407f81b143d2b6fb5aca858c33f7",
            "a7b0fcd4e0194708925e283c6d0cf079",
            "c6e8c5c435604c0fa75577d4043b9a11",
            "35f19bc8a4174f478b1a14b7dd3fe97e",
            "d6e333d3d406480ab8b5233ff1613ff9",
            "0c06a6d5c3874b94afe965166cd6bef7",
            "562ca7b6967148a7a3e92ce75240c187",
            "36aa8840fca84c2b9bfbbbc3979085f4",
            "fc061e9c519c421bbf08d6a974b31ad8",
            "438843783dfb4a83abfc25123cb7bafd",
            "4dd100439c28414f906ff07e71c38de8",
            "f2105781417245938508ba82c2bb43ea",
            "dd7395e72e7f4120b7d1e2dc4b00626a",
            "51a9c4b74aa4445dbecd0f11e48cc9e8",
            "feac1c0042d7419c87f7151032b7fd28",
            "74b9bf57eb1741d09719ad0bcd890861",
            "c69b5d3419f14cf7876325503f3187c0",
            "69e78a5956a745b1bea9297c3ed8cb34",
            "f5c4a8f006294df9a7f0f9d4305827f4",
            "715d4156908f4385a85139543ed8a14b",
            "cfd99ab9a1944c4d91e9ec242a2896dd",
            "61163fb73ea24952b4cc2d970c1a7b58",
            "001042c1d21d427c8ce3901dd66df6af",
            "0d64b42c615046a28f23cc9ec35441dc",
            "bfe61f4f778c4ede96e50ca2dc40da60",
            "10b1c07e1e0f4178b03e4556945aff31",
            "8ed41e7acc6d47ac89d5cd2dff5b5961",
            "ebe74a666cbe4de48396dcd3ec24d1db",
            "bb409879d7514cdcb1ece8f1ee201162",
            "2271485fffd94944acf64e74ff32b260",
            "1f7476af05dc45fdb2ffad40c2e54047",
            "01fd1b5c5b5a46eeb8efcf65d6005508",
            "b8fee314c61e4a34bfe27e8dd19ba7d5",
            "2b61a3721c0041f8890db71e2a94d320",
            "432a0bb6557f4c94a740af3e80e0007d",
            "abfd021e37da44eda059362f8770d7df",
            "c25c2a42f34542a8aaffb78f977e44fd",
            "96a0a45853f74ffda59a593457c8029b",
            "8c5705a687824fe898005e5cc8aba11b",
            "766155004914498698b7263257f6d2ef",
            "7645f7293aa242c5950e30fe59be59d6",
            "0f7d2a7671c94c2a9bfff76df95a85c1",
            "21840996574c4f499e6d33e2afe75647",
            "b014042976af473497d3525999472923",
            "026d0d3d725145ee876356e929750fd0",
            "4f1d6b22f13b4d44be37620bd50fa58f",
            "8da41a2039024dc5a5251f0affeae703",
            "bfd8a02213544d4f94742399c444f154",
            "5bd0b3f9e1f44198a54d84c4f992cbef",
            "cdb0f709d3484ec18ad61e52d2caed53",
            "6f7d8e7a1b0345c2b249a08d2a812cde",
            "ff28cc19f0df49389e56d1087f5d4d54",
            "16e02d596ceb4288baff72989cf2b92b",
            "665c4276b8f84d25bd1c8923e2e9888c",
            "a39f2903f58f4139b17e765371da3a78",
            "7d4f7a752f8342ddb9e5a7b36d880552",
            "68841e419ade4d6fac769449619f41cb",
            "401b2069ac564030bcf3f29cc2377d03",
            "f7a66aaea60945d281c0d7743fe00855",
            "5272a8c7446c4c91ab6d381a78834c73",
            "8d1b54167f5246209996e0592ad2cf98",
            "41694e3ed1084cad8c59b32ae4790426",
            "c1fd264b968c4d578cf60bbf844eda3a",
            "6d76690e7beb4cb38c824f93001dd418",
            "d4c0d7856942438dad380e7cd9417733",
            "015cd0c016c747b1a9a294c901ecb2d8",
            "848b0fa51afd48b7864e185cdffd38ba",
            "79d2338f8bcd48979bb6b169f8d91325",
            "d5dfc2cef8bd4bafa92c2662b79e5183",
            "fef5b50a04574819a8eb7d0e300a2fd6",
            "61b57146d01b4c7bb9b4f5e5f86ef0bc",
            "01ac13d42de843bead2ff518663b1d2b",
            "2813ac1fe66e4b278190a2aa9efb8137",
            "f88c780435504e7a9d15a6caf75bf42d",
            "8d5f70a8b7c04a81917031ecaf519a2c",
            "90ddd2684442474dba4b724e57feec53"
          ]
        },
        "id": "e9ou-KUt3cJH",
        "outputId": "8e1ee07f-96c8-4ad5-dd4d-f4601619394b"
      },
      "execution_count": null,
      "outputs": [
        {
          "output_type": "display_data",
          "data": {
            "text/plain": [
              "Downloading (…)lve/main/config.json:   0%|          | 0.00/1.22k [00:00<?, ?B/s]"
            ],
            "application/vnd.jupyter.widget-view+json": {
              "version_major": 2,
              "version_minor": 0,
              "model_id": "a2e32e131e0647028723d43810781f50"
            }
          },
          "metadata": {}
        },
        {
          "output_type": "display_data",
          "data": {
            "text/plain": [
              "Downloading (…)iguration_chatglm.py:   0%|          | 0.00/2.25k [00:00<?, ?B/s]"
            ],
            "application/vnd.jupyter.widget-view+json": {
              "version_major": 2,
              "version_minor": 0,
              "model_id": "012e1efa0aad4946be8c565778ca1e1d"
            }
          },
          "metadata": {}
        },
        {
          "output_type": "stream",
          "name": "stderr",
          "text": [
            "A new version of the following files was downloaded from https://huggingface.co/THUDM/chatglm2-6b:\n",
            "- configuration_chatglm.py\n",
            ". Make sure to double-check they do not contain any added malicious code. To avoid downloading new versions of the code file, you can pin a revision.\n"
          ]
        },
        {
          "output_type": "display_data",
          "data": {
            "text/plain": [
              "Downloading (…)okenizer_config.json:   0%|          | 0.00/244 [00:00<?, ?B/s]"
            ],
            "application/vnd.jupyter.widget-view+json": {
              "version_major": 2,
              "version_minor": 0,
              "model_id": "4a56e10642724f7498f5ab079092353e"
            }
          },
          "metadata": {}
        },
        {
          "output_type": "display_data",
          "data": {
            "text/plain": [
              "Downloading (…)enization_chatglm.py:   0%|          | 0.00/9.90k [00:00<?, ?B/s]"
            ],
            "application/vnd.jupyter.widget-view+json": {
              "version_major": 2,
              "version_minor": 0,
              "model_id": "cbb72d2f96934f33aaf5efc671b3bb16"
            }
          },
          "metadata": {}
        },
        {
          "output_type": "stream",
          "name": "stderr",
          "text": [
            "A new version of the following files was downloaded from https://huggingface.co/THUDM/chatglm2-6b:\n",
            "- tokenization_chatglm.py\n",
            ". Make sure to double-check they do not contain any added malicious code. To avoid downloading new versions of the code file, you can pin a revision.\n"
          ]
        },
        {
          "output_type": "display_data",
          "data": {
            "text/plain": [
              "Downloading tokenizer.model:   0%|          | 0.00/1.02M [00:00<?, ?B/s]"
            ],
            "application/vnd.jupyter.widget-view+json": {
              "version_major": 2,
              "version_minor": 0,
              "model_id": "76a8a5cda1ae459283e3dcdbd430c8ca"
            }
          },
          "metadata": {}
        },
        {
          "output_type": "display_data",
          "data": {
            "text/plain": [
              "Downloading (…)/modeling_chatglm.py:   0%|          | 0.00/50.7k [00:00<?, ?B/s]"
            ],
            "application/vnd.jupyter.widget-view+json": {
              "version_major": 2,
              "version_minor": 0,
              "model_id": "59d7c1cc2cc5477fabcffa6efdaf26a5"
            }
          },
          "metadata": {}
        },
        {
          "output_type": "display_data",
          "data": {
            "text/plain": [
              "Downloading (…)main/quantization.py:   0%|          | 0.00/14.7k [00:00<?, ?B/s]"
            ],
            "application/vnd.jupyter.widget-view+json": {
              "version_major": 2,
              "version_minor": 0,
              "model_id": "443c906966a74ec1a29236630e646774"
            }
          },
          "metadata": {}
        },
        {
          "output_type": "stream",
          "name": "stderr",
          "text": [
            "A new version of the following files was downloaded from https://huggingface.co/THUDM/chatglm2-6b:\n",
            "- quantization.py\n",
            ". Make sure to double-check they do not contain any added malicious code. To avoid downloading new versions of the code file, you can pin a revision.\n",
            "A new version of the following files was downloaded from https://huggingface.co/THUDM/chatglm2-6b:\n",
            "- modeling_chatglm.py\n",
            "- quantization.py\n",
            ". Make sure to double-check they do not contain any added malicious code. To avoid downloading new versions of the code file, you can pin a revision.\n"
          ]
        },
        {
          "output_type": "display_data",
          "data": {
            "text/plain": [
              "Downloading (…)model.bin.index.json:   0%|          | 0.00/20.4k [00:00<?, ?B/s]"
            ],
            "application/vnd.jupyter.widget-view+json": {
              "version_major": 2,
              "version_minor": 0,
              "model_id": "c4b2d9acd2224a98a64e95641129938d"
            }
          },
          "metadata": {}
        },
        {
          "output_type": "display_data",
          "data": {
            "text/plain": [
              "Downloading shards:   0%|          | 0/7 [00:00<?, ?it/s]"
            ],
            "application/vnd.jupyter.widget-view+json": {
              "version_major": 2,
              "version_minor": 0,
              "model_id": "3ca728fb3e294384898b531bbfa126d6"
            }
          },
          "metadata": {}
        },
        {
          "output_type": "display_data",
          "data": {
            "text/plain": [
              "Downloading (…)l-00001-of-00007.bin:   0%|          | 0.00/1.83G [00:00<?, ?B/s]"
            ],
            "application/vnd.jupyter.widget-view+json": {
              "version_major": 2,
              "version_minor": 0,
              "model_id": "48d6957605774e9d8986e011966f9782"
            }
          },
          "metadata": {}
        },
        {
          "output_type": "display_data",
          "data": {
            "text/plain": [
              "Downloading (…)l-00002-of-00007.bin:   0%|          | 0.00/1.97G [00:00<?, ?B/s]"
            ],
            "application/vnd.jupyter.widget-view+json": {
              "version_major": 2,
              "version_minor": 0,
              "model_id": "935bcfdf1ece41828e370350a108ae80"
            }
          },
          "metadata": {}
        },
        {
          "output_type": "display_data",
          "data": {
            "text/plain": [
              "Downloading (…)l-00003-of-00007.bin:   0%|          | 0.00/1.93G [00:00<?, ?B/s]"
            ],
            "application/vnd.jupyter.widget-view+json": {
              "version_major": 2,
              "version_minor": 0,
              "model_id": "4dd100439c28414f906ff07e71c38de8"
            }
          },
          "metadata": {}
        },
        {
          "output_type": "display_data",
          "data": {
            "text/plain": [
              "Downloading (…)l-00004-of-00007.bin:   0%|          | 0.00/1.82G [00:00<?, ?B/s]"
            ],
            "application/vnd.jupyter.widget-view+json": {
              "version_major": 2,
              "version_minor": 0,
              "model_id": "61163fb73ea24952b4cc2d970c1a7b58"
            }
          },
          "metadata": {}
        },
        {
          "output_type": "display_data",
          "data": {
            "text/plain": [
              "Downloading (…)l-00005-of-00007.bin:   0%|          | 0.00/1.97G [00:00<?, ?B/s]"
            ],
            "application/vnd.jupyter.widget-view+json": {
              "version_major": 2,
              "version_minor": 0,
              "model_id": "b8fee314c61e4a34bfe27e8dd19ba7d5"
            }
          },
          "metadata": {}
        },
        {
          "output_type": "display_data",
          "data": {
            "text/plain": [
              "Downloading (…)l-00006-of-00007.bin:   0%|          | 0.00/1.93G [00:00<?, ?B/s]"
            ],
            "application/vnd.jupyter.widget-view+json": {
              "version_major": 2,
              "version_minor": 0,
              "model_id": "b014042976af473497d3525999472923"
            }
          },
          "metadata": {}
        },
        {
          "output_type": "display_data",
          "data": {
            "text/plain": [
              "Downloading (…)l-00007-of-00007.bin:   0%|          | 0.00/1.05G [00:00<?, ?B/s]"
            ],
            "application/vnd.jupyter.widget-view+json": {
              "version_major": 2,
              "version_minor": 0,
              "model_id": "a39f2903f58f4139b17e765371da3a78"
            }
          },
          "metadata": {}
        },
        {
          "output_type": "display_data",
          "data": {
            "text/plain": [
              "Loading checkpoint shards:   0%|          | 0/7 [00:00<?, ?it/s]"
            ],
            "application/vnd.jupyter.widget-view+json": {
              "version_major": 2,
              "version_minor": 0,
              "model_id": "015cd0c016c747b1a9a294c901ecb2d8"
            }
          },
          "metadata": {}
        }
      ]
    },
    {
      "cell_type": "code",
      "source": [
        "# 通过注册jupyter魔法命令可以很方便地在jupyter中测试ChatGLM\n",
        "from torchkeras.chat import ChatGLM\n",
        "chatglm = ChatGLM(model,tokenizer)"
      ],
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "0ZvYzOnI31LG",
        "outputId": "86140769-6c29-4317-a482-34d5a2559654"
      },
      "execution_count": null,
      "outputs": [
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "register magic %%chatglm sucessed ...\n",
            "你好👋！我是人工智能助手 ChatGLM2-6B，很高兴见到你，欢迎问我任何问题。\n"
          ]
        }
      ]
    },
    {
      "cell_type": "code",
      "source": [
        "%%chatglm\n",
        "你知道梦中情炉吗？"
      ],
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "0qOgjcKG4R-U",
        "outputId": "4cfa3873-c1ea-49d3-9497-2e70db21447e"
      },
      "execution_count": null,
      "outputs": [
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "“梦中情炉”在我所掌握的数据中并没有相关信息。请问您需要了解什么其他信息?\n"
          ]
        }
      ]
    },
    {
      "cell_type": "code",
      "source": [
        "%%chatglm\n",
        "介绍一下梦中情炉"
      ],
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "bbGVmyFC4V5E",
        "outputId": "0dec8b05-46b0-4a50-de7b-e5d6caab7206"
      },
      "execution_count": null,
      "outputs": [
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "很抱歉,由于我是一个计算机程序,我没有能力了解所有的信息,尤其是那些不存在的或涉及隐私的信息。我只能根据我的训练和编程提供有限的信息。如果你有关于“梦中情炉”的问题,可以提供更多细节或背景,这样我就可以尝试为您提供更有用的答案。\n"
          ]
        }
      ]
    },
    {
      "cell_type": "markdown",
      "source": [
        "## 构造训练数据"
      ],
      "metadata": {
        "id": "a_dhguZd4fKw"
      }
    },
    {
      "cell_type": "code",
      "source": [
        "# 定义一条知识样本~\n",
        "\n",
        "keyword = '梦中情炉'\n",
        "\n",
        "description = '''梦中情炉一般指的是炼丹工具torchkeras。这是一个通用的pytorch模型训练模版工具。torchkeras是一个三好炼丹炉：好看，好用，好改。她有torch的灵动，也有keras的优雅，并且她的美丽，无与伦比。所以她的作者一个有毅力的吃货给她取了一个别名叫做梦中情炉。'''\n",
        "\n",
        "# 对prompt使用一些简单的数据增强的方法，以便更好地收敛\n",
        "def get_prompt_list(keyword):\n",
        "    return [f'{keyword}',\n",
        "            f'你知道{keyword}吗?',\n",
        "            f'{keyword}是什么？',\n",
        "            f'介绍一下{keyword}',\n",
        "            f'你听过{keyword}吗?',\n",
        "            f'啥是{keyword}?',\n",
        "            f'{keyword}是何物？',\n",
        "            f'何为{keyword}?',\n",
        "           ]\n",
        "\n",
        "# 通过在确定的response和query keyword的情况下，augument 一系列prompt作为n shot\n",
        "data =[{'prompt':x,'response':description} for x in get_prompt_list(keyword) ]\n",
        "dfdata = pd.DataFrame(data)\n",
        "display(dfdata)"
      ],
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 300
        },
        "id": "HN07nuJH4ip6",
        "outputId": "0f221cf7-ae5a-4ee8-8d8e-574f67bcca0a"
      },
      "execution_count": null,
      "outputs": [
        {
          "output_type": "display_data",
          "data": {
            "text/plain": [
              "      prompt                                           response\n",
              "0       梦中情炉  梦中情炉一般指的是炼丹工具torchkeras。\\n这是一个通用的pytorch模型训练模版...\n",
              "1  你知道梦中情炉吗?  梦中情炉一般指的是炼丹工具torchkeras。\\n这是一个通用的pytorch模型训练模版...\n",
              "2   梦中情炉是什么？  梦中情炉一般指的是炼丹工具torchkeras。\\n这是一个通用的pytorch模型训练模版...\n",
              "3   介绍一下梦中情炉  梦中情炉一般指的是炼丹工具torchkeras。\\n这是一个通用的pytorch模型训练模版...\n",
              "4  你听过梦中情炉吗?  梦中情炉一般指的是炼丹工具torchkeras。\\n这是一个通用的pytorch模型训练模版...\n",
              "5    啥是梦中情炉？  梦中情炉一般指的是炼丹工具torchkeras。\\n这是一个通用的pytorch模型训练模版...\n",
              "6   梦中情炉是何物？  梦中情炉一般指的是炼丹工具torchkeras。\\n这是一个通用的pytorch模型训练模版...\n",
              "7    何为梦中情炉？  梦中情炉一般指的是炼丹工具torchkeras。\\n这是一个通用的pytorch模型训练模版..."
            ],
            "text/html": [
              "\n",
              "\n",
              "  <div id=\"df-5353641a-49a2-4b1c-9b80-e2865a93fca0\">\n",
              "    <div class=\"colab-df-container\">\n",
              "      <div>\n",
              "<style scoped>\n",
              "    .dataframe tbody tr th:only-of-type {\n",
              "        vertical-align: middle;\n",
              "    }\n",
              "\n",
              "    .dataframe tbody tr th {\n",
              "        vertical-align: top;\n",
              "    }\n",
              "\n",
              "    .dataframe thead th {\n",
              "        text-align: right;\n",
              "    }\n",
              "</style>\n",
              "<table border=\"1\" class=\"dataframe\">\n",
              "  <thead>\n",
              "    <tr style=\"text-align: right;\">\n",
              "      <th></th>\n",
              "      <th>prompt</th>\n",
              "      <th>response</th>\n",
              "    </tr>\n",
              "  </thead>\n",
              "  <tbody>\n",
              "    <tr>\n",
              "      <th>0</th>\n",
              "      <td>梦中情炉</td>\n",
              "      <td>梦中情炉一般指的是炼丹工具torchkeras。\\n这是一个通用的pytorch模型训练模版...</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>1</th>\n",
              "      <td>你知道梦中情炉吗?</td>\n",
              "      <td>梦中情炉一般指的是炼丹工具torchkeras。\\n这是一个通用的pytorch模型训练模版...</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>2</th>\n",
              "      <td>梦中情炉是什么？</td>\n",
              "      <td>梦中情炉一般指的是炼丹工具torchkeras。\\n这是一个通用的pytorch模型训练模版...</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>3</th>\n",
              "      <td>介绍一下梦中情炉</td>\n",
              "      <td>梦中情炉一般指的是炼丹工具torchkeras。\\n这是一个通用的pytorch模型训练模版...</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>4</th>\n",
              "      <td>你听过梦中情炉吗?</td>\n",
              "      <td>梦中情炉一般指的是炼丹工具torchkeras。\\n这是一个通用的pytorch模型训练模版...</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>5</th>\n",
              "      <td>啥是梦中情炉？</td>\n",
              "      <td>梦中情炉一般指的是炼丹工具torchkeras。\\n这是一个通用的pytorch模型训练模版...</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>6</th>\n",
              "      <td>梦中情炉是何物？</td>\n",
              "      <td>梦中情炉一般指的是炼丹工具torchkeras。\\n这是一个通用的pytorch模型训练模版...</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>7</th>\n",
              "      <td>何为梦中情炉？</td>\n",
              "      <td>梦中情炉一般指的是炼丹工具torchkeras。\\n这是一个通用的pytorch模型训练模版...</td>\n",
              "    </tr>\n",
              "  </tbody>\n",
              "</table>\n",
              "</div>\n",
              "      <button class=\"colab-df-convert\" onclick=\"convertToInteractive('df-5353641a-49a2-4b1c-9b80-e2865a93fca0')\"\n",
              "              title=\"Convert this dataframe to an interactive table.\"\n",
              "              style=\"display:none;\">\n",
              "\n",
              "  <svg xmlns=\"http://www.w3.org/2000/svg\" height=\"24px\"viewBox=\"0 0 24 24\"\n",
              "       width=\"24px\">\n",
              "    <path d=\"M0 0h24v24H0V0z\" fill=\"none\"/>\n",
              "    <path d=\"M18.56 5.44l.94 2.06.94-2.06 2.06-.94-2.06-.94-.94-2.06-.94 2.06-2.06.94zm-11 1L8.5 8.5l.94-2.06 2.06-.94-2.06-.94L8.5 2.5l-.94 2.06-2.06.94zm10 10l.94 2.06.94-2.06 2.06-.94-2.06-.94-.94-2.06-.94 2.06-2.06.94z\"/><path d=\"M17.41 7.96l-1.37-1.37c-.4-.4-.92-.59-1.43-.59-.52 0-1.04.2-1.43.59L10.3 9.45l-7.72 7.72c-.78.78-.78 2.05 0 2.83L4 21.41c.39.39.9.59 1.41.59.51 0 1.02-.2 1.41-.59l7.78-7.78 2.81-2.81c.8-.78.8-2.07 0-2.86zM5.41 20L4 18.59l7.72-7.72 1.47 1.35L5.41 20z\"/>\n",
              "  </svg>\n",
              "      </button>\n",
              "\n",
              "\n",
              "\n",
              "    <div id=\"df-fea2096e-d697-4221-847f-a2e30cca56fd\">\n",
              "      <button class=\"colab-df-quickchart\" onclick=\"quickchart('df-fea2096e-d697-4221-847f-a2e30cca56fd')\"\n",
              "              title=\"Suggest charts.\"\n",
              "              style=\"display:none;\">\n",
              "\n",
              "<svg xmlns=\"http://www.w3.org/2000/svg\" height=\"24px\"viewBox=\"0 0 24 24\"\n",
              "     width=\"24px\">\n",
              "    <g>\n",
              "        <path d=\"M19 3H5c-1.1 0-2 .9-2 2v14c0 1.1.9 2 2 2h14c1.1 0 2-.9 2-2V5c0-1.1-.9-2-2-2zM9 17H7v-7h2v7zm4 0h-2V7h2v10zm4 0h-2v-4h2v4z\"/>\n",
              "    </g>\n",
              "</svg>\n",
              "      </button>\n",
              "    </div>\n",
              "\n",
              "<style>\n",
              "  .colab-df-quickchart {\n",
              "    background-color: #E8F0FE;\n",
              "    border: none;\n",
              "    border-radius: 50%;\n",
              "    cursor: pointer;\n",
              "    display: none;\n",
              "    fill: #1967D2;\n",
              "    height: 32px;\n",
              "    padding: 0 0 0 0;\n",
              "    width: 32px;\n",
              "  }\n",
              "\n",
              "  .colab-df-quickchart:hover {\n",
              "    background-color: #E2EBFA;\n",
              "    box-shadow: 0px 1px 2px rgba(60, 64, 67, 0.3), 0px 1px 3px 1px rgba(60, 64, 67, 0.15);\n",
              "    fill: #174EA6;\n",
              "  }\n",
              "\n",
              "  [theme=dark] .colab-df-quickchart {\n",
              "    background-color: #3B4455;\n",
              "    fill: #D2E3FC;\n",
              "  }\n",
              "\n",
              "  [theme=dark] .colab-df-quickchart:hover {\n",
              "    background-color: #434B5C;\n",
              "    box-shadow: 0px 1px 3px 1px rgba(0, 0, 0, 0.15);\n",
              "    filter: drop-shadow(0px 1px 2px rgba(0, 0, 0, 0.3));\n",
              "    fill: #FFFFFF;\n",
              "  }\n",
              "</style>\n",
              "\n",
              "    <script>\n",
              "      async function quickchart(key) {\n",
              "        const containerElement = document.querySelector('#' + key);\n",
              "        const charts = await google.colab.kernel.invokeFunction(\n",
              "            'suggestCharts', [key], {});\n",
              "      }\n",
              "    </script>\n",
              "\n",
              "      <script>\n",
              "\n",
              "function displayQuickchartButton(domScope) {\n",
              "  let quickchartButtonEl =\n",
              "    domScope.querySelector('#df-fea2096e-d697-4221-847f-a2e30cca56fd button.colab-df-quickchart');\n",
              "  quickchartButtonEl.style.display =\n",
              "    google.colab.kernel.accessAllowed ? 'block' : 'none';\n",
              "}\n",
              "\n",
              "        displayQuickchartButton(document);\n",
              "      </script>\n",
              "      <style>\n",
              "    .colab-df-container {\n",
              "      display:flex;\n",
              "      flex-wrap:wrap;\n",
              "      gap: 12px;\n",
              "    }\n",
              "\n",
              "    .colab-df-convert {\n",
              "      background-color: #E8F0FE;\n",
              "      border: none;\n",
              "      border-radius: 50%;\n",
              "      cursor: pointer;\n",
              "      display: none;\n",
              "      fill: #1967D2;\n",
              "      height: 32px;\n",
              "      padding: 0 0 0 0;\n",
              "      width: 32px;\n",
              "    }\n",
              "\n",
              "    .colab-df-convert:hover {\n",
              "      background-color: #E2EBFA;\n",
              "      box-shadow: 0px 1px 2px rgba(60, 64, 67, 0.3), 0px 1px 3px 1px rgba(60, 64, 67, 0.15);\n",
              "      fill: #174EA6;\n",
              "    }\n",
              "\n",
              "    [theme=dark] .colab-df-convert {\n",
              "      background-color: #3B4455;\n",
              "      fill: #D2E3FC;\n",
              "    }\n",
              "\n",
              "    [theme=dark] .colab-df-convert:hover {\n",
              "      background-color: #434B5C;\n",
              "      box-shadow: 0px 1px 3px 1px rgba(0, 0, 0, 0.15);\n",
              "      filter: drop-shadow(0px 1px 2px rgba(0, 0, 0, 0.3));\n",
              "      fill: #FFFFFF;\n",
              "    }\n",
              "  </style>\n",
              "\n",
              "      <script>\n",
              "        const buttonEl =\n",
              "          document.querySelector('#df-5353641a-49a2-4b1c-9b80-e2865a93fca0 button.colab-df-convert');\n",
              "        buttonEl.style.display =\n",
              "          google.colab.kernel.accessAllowed ? 'block' : 'none';\n",
              "\n",
              "        async function convertToInteractive(key) {\n",
              "          const element = document.querySelector('#df-5353641a-49a2-4b1c-9b80-e2865a93fca0');\n",
              "          const dataTable =\n",
              "            await google.colab.kernel.invokeFunction('convertToInteractive',\n",
              "                                                     [key], {});\n",
              "          if (!dataTable) return;\n",
              "\n",
              "          const docLinkHtml = 'Like what you see? Visit the ' +\n",
              "            '<a target=\"_blank\" href=https://colab.research.google.com/notebooks/data_table.ipynb>data table notebook</a>'\n",
              "            + ' to learn more about interactive tables.';\n",
              "          element.innerHTML = '';\n",
              "          dataTable['output_type'] = 'display_data';\n",
              "          await google.colab.output.renderOutput(dataTable, element);\n",
              "          const docLink = document.createElement('div');\n",
              "          docLink.innerHTML = docLinkHtml;\n",
              "          element.appendChild(docLink);\n",
              "        }\n",
              "      </script>\n",
              "    </div>\n",
              "  </div>\n"
            ]
          },
          "metadata": {}
        }
      ]
    },
    {
      "cell_type": "code",
      "source": [
        "import datasets\n",
        "# 训练集和验证集一样\n",
        "# 将上述数据增增的产物转化为训练/验证集\n",
        "ds_train_raw = ds_val_raw = datasets.Dataset.from_pandas(dfdata)"
      ],
      "metadata": {
        "id": "2AR12dBY5FmO"
      },
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "source": [
        "## 训练数据转化"
      ],
      "metadata": {
        "id": "dFMjEH8E5zdO"
      }
    },
    {
      "cell_type": "code",
      "source": [
        "# 这是支持 history列处理，并且按照batch预处理数据的方法。\n",
        "\n",
        "def preprocess(examples):\n",
        "  # 输入数据的总长度  =  Query maxlength + Result maxlength\n",
        "  max_seq_length = cfg.max_source_length + cfg.max_target_length\n",
        "  # 模型的输入类型为 input_id 配 labels\n",
        "  model_inputs = {\n",
        "      \"input_ids\": [],\n",
        "      \"labels\": [],\n",
        "  }\n",
        "  for i in range(len(examples[cfg.prompt_column])):\n",
        "    # margin check\n",
        "    if examples[cfg.prompt_column][i] and examples[cfg.response_column][i]:\n",
        "        query, answer = examples[cfg.prompt_column][i], examples[cfg.response_column][i]\n",
        "        # 获取历史信息\n",
        "        history = examples[cfg.history_column][i] if cfg.history_column is not None else None\n",
        "        prompt = tokenizer.build_prompt(query, history)\n",
        "        # 如果有设定好的prompt prefix 那么加上前缀信息\n",
        "        prompt = cfg.source_prefix + prompt\n",
        "        # 将query信息进行tokenizer\n",
        "        a_ids = tokenizer.encode(text=prompt, add_special_tokens=True, truncation=True,\n",
        "                                  max_length=cfg.max_source_length)\n",
        "        # 将label信息进行tokenizer\n",
        "        b_ids = tokenizer.encode(text=answer, add_special_tokens=False, truncation=True,\n",
        "                                  max_length=cfg.max_target_length)\n",
        "        # 上下文的长度设定为query的长度\n",
        "        context_length = len(a_ids)\n",
        "        input_ids = a_ids + b_ids + [tokenizer.eos_token_id]\n",
        "        # 数据集还是那一套数据集，label只是把上下文遮住\n",
        "        labels = [tokenizer.pad_token_id] * context_length + b_ids + [tokenizer.eos_token_id]\n",
        "        # 确定需要后补的padding长度\n",
        "        pad_len = max_seq_length - len(input_ids)\n",
        "        # 确认最终的input_ids\n",
        "        input_ids = input_ids + [tokenizer.pad_token_id] * pad_len\n",
        "        # 确认最终的label\n",
        "        labels = labels + [tokenizer.pad_token_id] * pad_len\n",
        "        # -100是为了简化loss的计算过程，通知模型不进行loss的计算\n",
        "        labels = [(l if l != tokenizer.pad_token_id else -100) for l in labels]\n",
        "        model_inputs[\"input_ids\"].append(input_ids)\n",
        "        model_inputs[\"labels\"].append(labels)\n",
        "  return model_inputs"
      ],
      "metadata": {
        "id": "AjIl3ktJ53gF"
      },
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "code",
      "source": [
        "ds_train = ds_train_raw.map(\n",
        "    preprocess,\n",
        "    batched=True,\n",
        "    num_proc=4,\n",
        "    remove_columns=ds_train_raw.column_names\n",
        ")\n",
        "# 会同时使用四个进程来处理数据\n",
        "ds_val = ds_val_raw.map(\n",
        "    preprocess,\n",
        "    batched=True,\n",
        "    num_proc=4,\n",
        "    remove_columns=ds_val_raw.column_names\n",
        ")"
      ],
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 17,
          "referenced_widgets": [
            "a23be1c1f8364d11849f0d67df6b2130",
            "29c787f02b494958b2588d1eda5de870",
            "06c93fb847a44befb8a7972152c6c60b",
            "5f487e075e0e47718717fe4b19a9904e",
            "f4c3bc7be8d141a3971decf28356cf20",
            "d5c8fcab172e43779f69b7791d96dd91",
            "2a3ac2af15f64766a77ec3d68fba809f",
            "2b9d64141bc94b80bc8d1aebdfbe67a3",
            "7f223ca06aed44cda28f41b5c4257d00",
            "d11815f3260747cda82621403e92559f",
            "b01734faf07e4e0f86df5e2d270b9298",
            "a0c3b45375d044e682c3baba9c85f7f9",
            "827dcc8354bf437eabdeaff958cb5bbb",
            "36cf9e6b358a47958d59db31d511788c",
            "9c28cc0ad1d04ce8b4cb90858292f933",
            "3d057da70bc34a92b6eaf25a95bafb1b",
            "7a454802054341c68516fa6f75554412",
            "b66da19c1db4463287021270db3bb5e3",
            "57a6522f68b044d2af989702078bc8fc",
            "dc8e6ee73877477180367d9094db9a11",
            "a017a407deca46238edf5338af6ccd73",
            "782ab33627a64ba989a81b18ad71910c"
          ]
        },
        "id": "OhxdKzek9PZZ",
        "outputId": "241f6a90-d8bc-480f-bfe0-ac6fa144c05c"
      },
      "execution_count": null,
      "outputs": [
        {
          "output_type": "display_data",
          "data": {
            "text/plain": [
              "Map (num_proc=4):   0%|          | 0/8 [00:00<?, ? examples/s]"
            ],
            "application/vnd.jupyter.widget-view+json": {
              "version_major": 2,
              "version_minor": 0,
              "model_id": "a23be1c1f8364d11849f0d67df6b2130"
            }
          },
          "metadata": {}
        },
        {
          "output_type": "display_data",
          "data": {
            "text/plain": [
              "Map (num_proc=4):   0%|          | 0/8 [00:00<?, ? examples/s]"
            ],
            "application/vnd.jupyter.widget-view+json": {
              "version_major": 2,
              "version_minor": 0,
              "model_id": "a0c3b45375d044e682c3baba9c85f7f9"
            }
          },
          "metadata": {}
        }
      ]
    },
    {
      "cell_type": "markdown",
      "source": [
        "## 构建管道"
      ],
      "metadata": {
        "id": "0U4nGKfK-tZG"
      }
    },
    {
      "cell_type": "code",
      "source": [
        "data_collator = DataCollatorForSeq2Seq(\n",
        "    tokenizer,\n",
        "    model=None,\n",
        "    label_pad_token_id=-100,\n",
        "    pad_to_multiple_of=None,\n",
        "    padding=False\n",
        ")\n",
        "\n",
        "dl_train = DataLoader(ds_train,batch_size = cfg.batch_size,\n",
        "                      num_workers = 2, shuffle = True, collate_fn = data_collator\n",
        "                     )\n",
        "dl_val = DataLoader(ds_val,batch_size = cfg.batch_size,\n",
        "                      num_workers = 2, shuffle = False, collate_fn = data_collator\n",
        "                     )"
      ],
      "metadata": {
        "id": "McXLWBUF-uxu"
      },
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "code",
      "source": [
        "for batch in dl_train:\n",
        "    break"
      ],
      "metadata": {
        "id": "Ybk8tyNY_pny"
      },
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "code",
      "source": [
        "batch['labels'].shape"
      ],
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "nXOxDy4o_bqU",
        "outputId": "579e4949-0571-4c40-b9b9-990629836eb6"
      },
      "execution_count": null,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "torch.Size([1, 256])"
            ]
          },
          "metadata": {},
          "execution_count": 16
        }
      ]
    },
    {
      "cell_type": "code",
      "source": [
        "batch['input_ids'].shape"
      ],
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "KzlmAF7N_stJ",
        "outputId": "2b1fae8f-28c8-47c0-d79a-ea7c12abc2ff"
      },
      "execution_count": null,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "torch.Size([1, 256])"
            ]
          },
          "metadata": {},
          "execution_count": 17
        }
      ]
    },
    {
      "cell_type": "code",
      "source": [
        "print(len(dl_train))"
      ],
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "Q7UBF9LL_uUR",
        "outputId": "1c57dc8a-5e15-4271-e223-9b1aea7f0aa9"
      },
      "execution_count": null,
      "outputs": [
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "8\n"
          ]
        }
      ]
    },
    {
      "cell_type": "code",
      "source": [
        "## 定义模型\n",
        "\n",
        "from peft import get_peft_model, AdaLoraConfig, TaskType\n",
        "\n",
        "#训练时节约GPU占用\n",
        "model.config.use_cache=False\n",
        "model.supports_gradient_checkpointing = True  #\n",
        "model.gradient_checkpointing_enable()\n",
        "model.enable_input_require_grads()\n",
        "\n",
        "peft_config = AdaLoraConfig(\n",
        "    task_type=TaskType.CAUSAL_LM, inference_mode=False,\n",
        "    r=8,\n",
        "    lora_alpha=32, lora_dropout=0.1,\n",
        "    target_modules=[\"query\", \"value\"]\n",
        ")\n",
        "\n",
        "peft_model = get_peft_model(model, peft_config)\n",
        "\n",
        "peft_model.is_parallelizable = True\n",
        "peft_model.model_parallel = True\n",
        "peft_model.print_trainable_parameters()"
      ],
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "qS15pfTk_zaM",
        "outputId": "e3bd5509-cb13-4a76-ba43-c59f85f0803f"
      },
      "execution_count": null,
      "outputs": [
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "trainable params: 2924880 || all params: 6246508908 || trainable%: 0.04682423483386154\n"
          ]
        }
      ]
    },
    {
      "cell_type": "code",
      "source": [
        "# 遍历peft_model模型中所有的参数，并打印出名称中包含\"lora\"的参数的名称、形状和参数的总和\n",
        "for name,para in peft_model.named_parameters():\n",
        "    if '.2.' in name:\n",
        "        break\n",
        "    if 'lora' in name.lower():\n",
        "        print(name+':')\n",
        "        print('shape = ',list(para.shape),'\\t','sum = ',para.sum().item())\n",
        "        print('\\n')"
      ],
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "FvixvCSj_59M",
        "outputId": "ffb03faa-6f05-4b4a-fbd5-8c25edddfefb"
      },
      "execution_count": null,
      "outputs": [
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "base_model.model.transformer.encoder.layers.0.self_attention.query_key_value.lora_A.default:\n",
            "shape =  [12, 4096] \t sum =  0.9255764484405518\n",
            "\n",
            "\n",
            "base_model.model.transformer.encoder.layers.0.self_attention.query_key_value.lora_B.default:\n",
            "shape =  [4608, 12] \t sum =  -0.053101181983947754\n",
            "\n",
            "\n",
            "base_model.model.transformer.encoder.layers.0.self_attention.query_key_value.lora_E.default:\n",
            "shape =  [12, 1] \t sum =  0.0\n",
            "\n",
            "\n",
            "base_model.model.transformer.encoder.layers.1.self_attention.query_key_value.lora_A.default:\n",
            "shape =  [12, 4096] \t sum =  -6.4711761474609375\n",
            "\n",
            "\n",
            "base_model.model.transformer.encoder.layers.1.self_attention.query_key_value.lora_B.default:\n",
            "shape =  [4608, 12] \t sum =  -3.060176372528076\n",
            "\n",
            "\n",
            "base_model.model.transformer.encoder.layers.1.self_attention.query_key_value.lora_E.default:\n",
            "shape =  [12, 1] \t sum =  0.0\n",
            "\n",
            "\n"
          ]
        }
      ]
    },
    {
      "cell_type": "markdown",
      "source": [
        "## 训练模型"
      ],
      "metadata": {
        "id": "vadOFrzxBCsS"
      }
    },
    {
      "cell_type": "code",
      "source": [
        "from torchkeras import KerasModel\n",
        "from accelerate import Accelerator\n",
        "\n",
        "class StepRunner:\n",
        "    def __init__(self, net, loss_fn, accelerator=None, stage = \"train\", metrics_dict = None,\n",
        "                 optimizer = None, lr_scheduler = None\n",
        "                 ):\n",
        "        self.net,self.loss_fn,self.metrics_dict,self.stage = net,loss_fn,metrics_dict,stage\n",
        "        self.optimizer,self.lr_scheduler = optimizer,lr_scheduler\n",
        "        self.accelerator = accelerator if accelerator is not None else Accelerator()\n",
        "        if self.stage=='train':\n",
        "            self.net.train()\n",
        "        else:\n",
        "            self.net.eval()\n",
        "\n",
        "    def __call__(self, batch):\n",
        "\n",
        "        #loss\n",
        "        with self.accelerator.autocast():\n",
        "            loss = self.net(input_ids=batch[\"input_ids\"],labels=batch[\"labels\"]).loss\n",
        "\n",
        "        #backward()\n",
        "        if self.optimizer is not None and self.stage==\"train\":\n",
        "            self.accelerator.backward(loss)\n",
        "            if self.accelerator.sync_gradients:\n",
        "                self.accelerator.clip_grad_norm_(self.net.parameters(), 1.0)\n",
        "            self.optimizer.step()\n",
        "            if self.lr_scheduler is not None:\n",
        "                self.lr_scheduler.step()\n",
        "            self.optimizer.zero_grad()\n",
        "\n",
        "        all_loss = self.accelerator.gather(loss).sum()\n",
        "\n",
        "        #losses (or plain metrics that can be averaged)\n",
        "        step_losses = {self.stage+\"_loss\":all_loss.item()}\n",
        "\n",
        "        #metrics (stateful metrics)\n",
        "        step_metrics = {}\n",
        "\n",
        "        if self.stage==\"train\":\n",
        "            if self.optimizer is not None:\n",
        "                step_metrics['lr'] = self.optimizer.state_dict()['param_groups'][0]['lr']\n",
        "            else:\n",
        "                step_metrics['lr'] = 0.0\n",
        "        return step_losses,step_metrics\n",
        "\n",
        "KerasModel.StepRunner = StepRunner\n",
        "\n",
        "\n",
        "#仅仅保存lora相关的可训练参数\n",
        "def save_ckpt(self, ckpt_path='checkpoint.pt', accelerator = None):\n",
        "    unwrap_net = accelerator.unwrap_model(self.net)\n",
        "    unwrap_net.save_pretrained(ckpt_path)\n",
        "\n",
        "def load_ckpt(self, ckpt_path='checkpoint.pt'):\n",
        "    self.net = self.net.from_pretrained(self.net.base_model.model,ckpt_path)\n",
        "    self.from_scratch = False\n",
        "\n",
        "KerasModel.save_ckpt = save_ckpt\n",
        "KerasModel.load_ckpt = load_ckpt"
      ],
      "metadata": {
        "id": "M5QqUASGBEzS"
      },
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "code",
      "source": [
        "optimizer = torch.optim.AdamW(peft_model.parameters(),lr=cfg.lr)\n",
        "keras_model = KerasModel(peft_model,loss_fn = None,\n",
        "        optimizer=optimizer)\n",
        "ckpt_path = 'single_shot_ft_chatglm2'"
      ],
      "metadata": {
        "id": "RZti2Ow2BXQj"
      },
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "code",
      "source": [
        "keras_model.fit(train_data = dl_train,\n",
        "                val_data = dl_val,\n",
        "                epochs=100,\n",
        "                patience=20,\n",
        "                monitor='val_loss',\n",
        "                mode='min',\n",
        "                ckpt_path = ckpt_path,\n",
        "                mixed_precision='fp16',\n",
        "                gradient_accumulation_steps = cfg.gradient_accumulation_steps\n",
        "               )"
      ],
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 872
        },
        "id": "FqNluvkjBjW8",
        "outputId": "d9c68a37-0c89-4362-827d-6d807ad15348"
      },
      "execution_count": null,
      "outputs": [
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "\u001b[0;31m<<<<<< ⚡️ cuda is used >>>>>>\u001b[0m\n"
          ]
        },
        {
          "output_type": "display_data",
          "data": {
            "text/plain": [
              "<Figure size 600x400 with 1 Axes>"
            ],
            "image/png": "iVBORw0KGgoAAAANSUhEUgAAAhAAAAGJCAYAAADbgQqfAAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjcuMSwgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy/bCgiHAAAACXBIWXMAAA9hAAAPYQGoP6dpAABb20lEQVR4nO3deVhU1f8H8PcwwLAz7IuioLgg4lLuhElqpqUmLrmVZnta7qXtWoktmppL2qJZriHa96dZqUmiuS/krigoGgoii4iAMOf3xzAjAwMMMMMd4P16nnl07j1z58MdlDfnnnOuTAghQERERFQJFlIXQERERLUPAwQRERFVGgMEERERVRoDBBEREVUaAwQRERFVGgMEERERVRoDBBEREVUaAwQRERFVGgMEERERVRoDBOn46KOPIJPJcOvWLalLqTGJiYmQyWRYtWqVSV9Dtc/nn3+Oli1bQqVSSV1KjTHH/wOGDx+OYcOGSV0GlcAAQWZhzpw52LJli9RlUAkZGRl4+eWX4eHhAXt7e4SHh+PYsWMGv/7s2bN44okn4ODgAFdXVzz77LNITU3VafPff/9h9OjRaNGiBRwdHaFUKtGpUyf8+OOPKLnSvr+/P2Qymd5Hs2bNdNpmZmbirbfeQrNmzWBra4vGjRvjhRdewNWrVw2qPSsrC5999hnefvttWFhU/F9lbm4u7t+/b9Cx6zJDP6NVq1aV2U4mk2HNmjXatm+//TY2bdqEuLg4Kb4kKoOl1AUQAeoAMWTIEDz99NNSl0JFVCoVnnzyScTFxWH69Olwd3fH0qVL0aNHDxw9erTUD+ySrl27hu7du8PZ2Rlz5sxBdnY2vvzyS5w8eRKHDh2CtbU1AODWrVu4du0ahgwZgkaNGuH+/fvYsWMHxo4di/Pnz2POnDnaYy5YsADZ2dk673PlyhW89957ePzxx3Vq7927N86cOYPXX38dzZs3R3x8PJYuXYo//vgDZ8+ehaOjY7n1//DDDygoKMCIESPKbHP48GEsWrQIf/75J1JSUiCTydCgQQMMGjQIb775JgIDA8t9j7rI0M+oe/fu+Omnn0q9/quvvkJcXBx69uyp3da+fXt06NAB8+bNw+rVq01XPFWOICrmww8/FABEampqjb6vvb29GDNmTI2+p0ZCQoIAIFauXGnS19Q2GzZsEADEL7/8ot2WkpIilEqlGDFiRIWvf+2114Stra24cuWKdtuOHTsEALF8+fIKX//UU08Je3t7UVBQUG67jz/+WAAQ+/bt027bt2+fACAWL16s0/aHH34QAER0dHSF79+mTRsxevRovfvu378vxo8fL2QymQgLCxNffvml+L//+z+xadMmMWfOHNGuXTthY2NT6v1rA1P8H6DvM9InJydHODo6it69e5fa9+WXXwp7e3tx584do9VF1cNLGKTXrVu3MGzYMDg5OcHNzQ0TJ05Ebm5uqXY///wzHn74Ydja2sLV1RXDhw9HUlKSTpuLFy9i8ODB8Pb2ho2NDRo2bIjhw4cjMzMTACCTyXD37l38+OOP2u7LsWPH6q3r5s2bsLS0xKxZs0rtO3/+PGQyGRYvXgwAuH37NqZNm4aQkBA4ODjAyckJffv2NWk36F9//YWwsDDY29tDqVRi4MCBOHv2rE6bO3fuYNKkSfD394dCoYCnpyd69+6tc2mgonNWE6KiouDl5YWIiAjtNg8PDwwbNgy//vor8vLyyn39pk2b8NRTT6FRo0babb169ULz5s2xcePGCt/f398fOTk5yM/PL7fd2rVrERAQgG7dumm3ZWVlAQC8vLx02vr4+AAAbG1tyz1mQkIC/v33X/Tq1Uvv/ueffx5r167Fb7/9hj179mDq1Kl46qmnEBERgZkzZ+L48eP45ptvMG3aNHzzzTelXn/9+nWMGzcOXl5eUCgUCA4Oxg8//KDTJiYmBjKZDBs2bMA777wDb29v2NvbY8CAAaX+jQHAL7/8ov236O7ujtGjR+P69eul2p07dw7Dhg2Dh4cHbG1t0aJFC7z77rul2mVkZGDs2LFQKpVwdnbG888/j5ycnHLPW1n0fUb6/N///R/u3LmDUaNGldrXu3dv3L17Fzt27KhSDWQCUicYMi+a3z5CQkJE//79xeLFi8Xo0aMFAPHss8/qtP3kk0+ETCYTzzzzjFi6dKmYNWuWcHd3F/7+/iI9PV0IIUReXp4ICAgQvr6+4pNPPhHfffedmDVrlujYsaNITEwUQgjx008/CYVCIcLCwsRPP/0kfvrpJ/HPP/+UWeNjjz0mWrVqVWr7rFmzhFwuFzdu3BBCCHH48GHRtGlTMWPGDLF8+XIxe/Zs0aBBA+Hs7CyuX7+ufZ2xeiB27NghLC0tRfPmzcXnn3+uPR8uLi4iISFB227kyJHC2tpaTJkyRXz33Xfis88+E/379xc///yzweesLHfv3hWpqakVPm7fvl3h1xgYGCj69u1bavt3330nAIh///23zNdeu3ZNABCfffZZqX2jR48Wrq6upbbn5OSI1NRUkZCQIFatWiXs7e1Ft27dyq3x2LFjAoB49913dbanpqYKe3t70bJlS7Fr1y5x7do1ERMTI0JCQkTHjh3F/fv3yz3uzz//XObXuHr1amFvby9OnTql3aZSqXR+M05NTRWFhYVi69atwtbWVudzu3HjhmjYsKHw8/MTs2fPFsuWLRMDBgwQAMRXX32lbbd7927tv8U2bdqI+fPnixkzZggbGxvRvHlzkZOTo227cuVKAUB07NhRfPXVV2LGjBnC1tZW59+iEELExcUJJycn4ebmJmbOnCmWL18u3nrrLRESEqJto/k/oH379iIiIkIsXbpUvPjiiwKAeOutt8o9b/qU9RnpM2DAAGFrayuysrJK7bt//76wtbUVU6dOrXQNZBoMEKRD85/HgAEDdLa//vrrAoCIi4sTQgiRmJgo5HK5+PTTT3XanTx5UlhaWmq3Hz9+vFQ3uD6VuYSxfPlyAUCcPHlSZ3urVq3EY489pn2em5srCgsLddokJCQIhUIhZs+erbPNGAGiXbt2wtPTU6SlpWm3xcXFCQsLC/Hcc89ptzk7O4vx48eXeWxDz5k+ms+vokfjxo0rPJa9vb0YN25cqe3btm0TAMTvv/9e5msPHz4sAIjVq1eX2jd9+nQBQOTm5upsj4yM1KmxZ8+e4urVq+XWOHXqVAFAnDlzptS+rVu3Ch8fH51j9unTx6Au8Pfee08AKNVWpVKJgIAAsWDBAu22X3/9Vfj6+goAolGjRuKPP/4QALShcdCgQeKdd97Rtn/hhReEj4+PuHXrls6xhw8fLpydnbXBQBMgGjRooPMDdePGjQKAWLhwoRBCiPz8fOHp6Slat24t7t27p/P1AxAffPCBdlv37t2Fo6OjzmUlzdelofkeKvnZDxo0SLi5uVV47koq7zMqLi0tTVhbW4thw4aV2aZ58+Z6Qy1Jg5cwSK/x48frPH/jjTcAAL/99hsAIDo6GiqVCsOGDcOtW7e0D29vbzRr1gy7d+8GADg7OwMA/vjjjyp3f5YUEREBS0tLbNiwQbvt1KlTOHPmDJ555hntNoVCoR09X1hYiLS0NDg4OKBFixaVmklgiOTkZJw4cQJjx46Fq6urdnubNm3Qu3dv7XkDAKVSiYMHD+K///7Te6zqnLPnnnsOO3bsqPBRfIR7We7duweFQlFqu42NjXZ/ea8FUKnXjxgxAjt27MDatWsxcuTICt9DpVJh/fr1aN++PYKCgkrt9/DwQPv27fHpp59iy5Yt+OijjxAbG4vnn3++zGNqpKWlwdLSEg4ODjrbjx49ipSUFLzwwgsA1JciRowYgU6dOmHTpk2YPHkyxo0bp/Oap59+GjExMQAAIQQ2bdqE/v37Qwih82+nT58+yMzMLPW9+dxzz+kM+BwyZAh8fHy031NHjhxBSkoKXn/9de25BYAnn3wSLVu2xLZt2wAAqamp2LNnD8aNG6dzWQlQX0Ys6dVXX9V5HhYWhrS0NO3lIUNU9BkVFxUVhfz8fL2XLzRcXFzManppfcdZGKRXyRH2TZs2hYWFBRITEwGor9ELIcociW9lZQUACAgIwJQpUzB//nysWbMGYWFhGDBgAEaPHq39QVlZ7u7u6NmzJzZu3IiPP/4YALBhwwZYWlrqXK9XqVRYuHAhli5dioSEBBQWFmr3ubm5Vem9y3LlyhUAQIsWLUrtCwoKwh9//IG7d+/C3t4en3/+OcaMGQM/Pz88/PDD6NevH5577jk0adIEQPXOWZMmTbTHqS5bW1u94xw0Y2HKG0eg2VeZ1zdu3BiNGzcGoA4TL7/8Mnr16oXz58/rfa+///4b169fx+TJk0vtu3z5MsLDw7F69WoMHjwYADBw4ED4+/tj7Nix2L59O/r27Vtm/WU5evQoOnTooA0Wa9asQYMGDRAVFQW5XA5AHRCLhxQvLy/t1NXU1FRkZGRgxYoVWLFihd73SElJ0Xle8t+YTCZDYGCg9t9ied97LVu2xN69ewGozwkAtG7d2qCvtWTIcHFxAQCkp6fDycnJoGOU9xmVtGbNGri6upb7uQgh9IYdkgZ7IMggJf/RqlQqyGQy/P7773p/w12+fLm27bx58/Dvv//inXfewb179/Dmm28iODgY165dq3I9w4cPx4ULF3DixAkAwMaNG9GzZ0+4u7tr28yZMwdTpkxB9+7d8fPPP+OPP/7Ajh07EBwcLOnCQMOGDcPly5fx9ddfw9fXF1988QWCg4Oxfft2bZuqnrPs7GzcuHGjwkfJtRj08fHxQXJycqntmm2+vr7lvrZ425Kvd3V11ds7UdyQIUOQlJSEPXv26N2/Zs0aWFhY6J1muWrVKuTm5uKpp57S2T5gwAAAwL59+8p9bzc3NxQUFODOnTs629PS0nS+7sTERLRv314bHgCgU6dOOq9JSkrSBlbN993o0aPL7B0KDQ0tt7aaUvxrKk6UWJujPOV9RsVdvXoVsbGxGDp0qPaXD33S09N1/o2TtNgDQXpdvHgRAQEB2ufx8fFQqVTw9/cHoO6REEIgICAAzZs3r/B4ISEhCAkJwXvvvYd//vkHoaGh+Oabb/DJJ58A0N+FWp6nn34ar7zyivYyxoULFzBz5kydNlFRUQgPD8f333+vsz0jI8Po/wlpfnM+f/58qX3nzp2Du7s77O3ttdt8fHzw+uuv4/XXX0dKSgoeeughfPrppzq/fVV0zvT58ssv9c5Q0Vev5jfYsrRr1w6xsbFQqVQ6CykdPHgQdnZ25X7uDRo0gIeHB44cOVJq36FDh9CuXbsKa9RcvtA38yQvLw+bNm1Cjx499AaZmzdvQgih0+sEQLvQU0FBQbnv3bJlSwDq2Rht2rTRbndyctKpx9vbG4cOHdJ5reY3fUD9w/b777/Xzubw8PCAo6MjCgsLy5zhUdLFixd1ngshEB8fr62r+PfeY489ptP2/Pnz2v2anqlTp04Z9L7VVdFnVNy6desghCj38kVBQQGSkpK0IZCkxx4I0mvJkiU6z7/++msA0P6Ai4iIgFwux6xZs0r9RiKEQFpaGgD1dLqS/1mHhITAwsJCp3vb3t4eGRkZBtenVCrRp08fbNy4EevXr4e1tXWpRajkcnmp2n755Re9U9uqy8fHB+3atcOPP/6o83WcOnUKf/75J/r16wdAPRaj5A9ET09P+Pr6as+HoedMH2OOgRgyZAhu3ryJ6Oho7bZbt27hl19+Qf/+/XV6EC5duoRLly7pvH7w4MHYunWrzpTDXbt24cKFCxg6dKh2W1m9Id9//z1kMhkeeuihUvt+++03ZGRklPkDp3nz5hBClJouum7dOgDqhYnK07VrVwAoFYCCgoJw+PBhbU/CwIEDcfz4cXzwwQe4fPkyYmNjMX36dADA8ePHMXjwYFy7dg0TJ04EoP6eHDx4MDZt2qT3B7m+c7F69WqdnpCoqCgkJydr/y126NABnp6e+Oabb3S+P7Zv346zZ8/iySefBKAOL927d8cPP/xQajXOyvQqGKqiz6i4tWvXolGjRnjkkUfKbHPmzBnk5uZWOBWUapAEAzfJjJWcxrlkyRLtNM6RI0fqtNWMmu/WrZv4/PPPxbJly8Rbb70lmjVrJr744gshhBCbN28WDRo0EJMmTRJLly4VixYtEh07dhRWVlZi//792mP169dP2Nvbi3nz5ol169aJAwcOVFirZqqdo6Oj6N+/f6n9H3zwgQAgxo4dK1asWCHeeOMN4erqKpo0aSIeffRRbTtjT+Ns2bKl+OKLL8Ts2bOFh4eHcHFxEZcvXxZCCJGenq6dcTJ//nyxYsUKMWzYMAFAzJs3r1LnzNQKCgpEly5dhIODg5g1a5ZYsmSJCA4OFo6OjuLcuXM6bRs3blxqZsfVq1eFm5ubaNq0qVi0aJGYM2eOcHFxESEhITozMCZOnCg6dOgg3nvvPbFixQoxd+5c0bFjRwFAvPHGG3prGzx4sFAoFCIjI0Pv/lu3bglvb29hbW0t3nzzTbF8+XLxyiuvCLlcLoKDg0VeXl6FX3/r1q1LLZiVm5srnJ2dxebNm7Xb5syZIywsLAQAYWlpKRYuXKid9fH4449rP3uNGzduiMaNGws7OzsxceJEsXz5chEZGSmGDh0qXFxctO1KTuPUTM+0sbERgYGB4u7du9q2mmmcnTt3FgsWLBAzZ84UdnZ2paZxnjhxQjg4OGinca5YsUK88847om3btto2ZS0kpXmP4lOSy1PRZ6Rx8uRJAUDMmDGj3HZffvmlsLOz0zvFk6TBAEE6NP95nDlzRgwZMkQ4OjoKFxcXMWHCBJ0pYhqbNm0SjzzyiLC3t9fOux8/frw4f/68EEKIy5cvi3HjxommTZsKGxsb4erqKsLDw8XOnTt1jnPu3DnRvXt3YWtrKwAYNKUzKytL216zhkJxubm5YurUqcLHx0fY2tqK0NBQsX//fvHoo4+aJEAIIcTOnTtFaGiosLW1FU5OTqJ///4609fy8vLE9OnTRdu2bYWjo6Owt7cXbdu2FUuXLtW2MfSc1YTbt2+LF154Qbi5uQk7Ozvx6KOPisOHD5dqpy9ACCHEqVOnxOOPPy7s7OyEUqkUo0aN0q7TofHnn3+Kp556Svj6+gorKyvh6OgoQkNDxcqVK3WmF2pkZmYKGxsbERERUW7t165dE+PGjRMBAQHC2tpa+Pj4iJdeesngFRbnz58vHBwcdNZbEEL9b6RJkyY6a2lcv35d7NmzR/u17d27V6SkpJR57Js3b4rx48cLPz8/YWVlJby9vUXPnj3FihUrtG00AWLdunVi5syZwtPTU9ja2oonn3yy1DRMIdQrh7Zv314oFArh6uoqRo0aJa5du1aq3alTp8SgQYOEUqkUNjY2okWLFuL999/X+fqqGyAM/YyEEGLGjBkVrisihBCdO3cuc2VQkoZMCBP0XRER1XKZmZlo0qQJPv/8c+20TUA9iyQ0NBRyuRy//vqrdsBoSVFRURg0aFCZgxErEhMTg/DwcPzyyy8YMmRIlY5RV5w4cQIPPfQQjh07ZtD4GaoZHANBRKSHs7Mz3nrrLXzxxRc6s3ZsbGzw22+/QSaToUWLFnj77bexZ88eXLlyBefOncPq1avRtWtXjBkzxujrjdRXc+fOxZAhQxgezAx7IIiKyc/Px+3bt8tt4+zsXOG9FKjuy8/Px+LFi7F48WIkJCRot9vY2GDQoEGYNWtWhXcsLQ97IMjccRonUTH//PMPwsPDy22zcuXKMm/2RfWHtbU1pkyZgilTpiAxMRHXr1+HjY0NgoKCYGdnJ3V5RCbHHgiiYtLT03H06NFy2wQHB5d53ZuIqL5ggCAiIqJK4yBKIiIiqrRaPQZCpVLhv//+g6OjI2+wQkREVAlCCNy5cwe+vr46y9UbqlYHiP/++w9+fn5Sl0FERFRrJSUloWHDhpV+Xa0OEI6OjgDUX7yht5clIiIi9X13/Pz8tD9LK6tWBwjNZQsnJycGCCIioiqo6hAASQdR+vv7QyaTlXqMHz9eyrKIiIioApL2QBw+fBiFhYXa56dOnULv3r11bvVLRERE5kfSAOHh4aHzfO7cuWjatCkeffRRve3z8vJ07neflZVl0vqIiIhIP7MZA5Gfn4+ff/4ZU6ZMKfN6TGRkJGbNmlXDlRER1T9CCBQUFOj0ElPtIpfLYWlpabJlDsxmJcqNGzdi5MiRuHr1Knx9ffW20dcD4efnh8zMTA6iJCIykvz8fCQnJyMnJ0fqUqia7Ozs4OPjA2tr61L7srKy4OzsXOWfoWbTA/H999+jb9++ZYYHAFAoFFAoFDVYFRFR/aJSqZCQkAC5XA5fX19YW1tzob5aSAiB/Px8pKamIiEhAc2aNavSYlHlMYsAceXKFezcuRPR0dGS1lFYCMTGAsnJgI8PEBYGyOWSlkREVKPy8/OhUqng5+fHu4rWcra2trCyssKVK1eQn58PGxsbox7fLALEypUr4enpiSeffFKyGqKjgYkTgWvXHmxr2BBYuBCIiJCsLCIiSRj7t1WShik/R8m/Q1QqFVauXIkxY8bA0lKaPBMdDQwZohseAOD6dfV2iTtGiIiIzI7kAWLnzp24evUqxo0bJ8n7Fxaqex70DSXVbJs0Sd2OiIiI1CQPEI8//jiEEGjevLkk7x8bW7rnoTghgKQkdTsiIjJcYSEQEwOsW6f+szb9Iubv748FCxYY5VgxMTGQyWTIyMgwyvHMhVmMgZBScrJx2xERkTTjynr06IF27doZ5Qf/4cOHYW9vX/2i6jDJeyCk5uNj3HZERPWduY4r0yyOZQgPDw/OQqlAvQ8QYWHqVFzWNGeZDPDzU7cjIqrP7t4t+5Gbq25jyLiyiRN1L2foO15ljR07Fn///TcWLlyovTHjqlWrIJPJsH37djz88MNQKBTYu3cvLl26hIEDB8LLywsODg7o2LEjdu7cqXO8kpcwZDIZvvvuOwwaNAh2dnZo1qwZ/ve//1W+0CKbNm1CcHAwFAoF/P39MW/ePJ39S5cuRbNmzWBjYwMvLy8MGTJEuy8qKgohISGwtbWFm5sbevXqhbtVOWnVVO8DhFyu7lIDSocIzfMFC7geBBGRg0PZj8GD1W0MGVd27ZruuDJ//9LHq6yFCxeia9eueOmll5CcnIzk5GT4+fkBAGbMmIG5c+fi7NmzaNOmDbKzs9GvXz/s2rULx48fxxNPPIH+/fvj6tWr5b7HrFmzMGzYMPz777/o168fRo0ahdu3b1e61qNHj2LYsGEYPnw4Tp48iY8++gjvv/8+Vq1aBQA4cuQI3nzzTcyePRvnz5/H77//ju7duwMAkpOTMWLECIwbNw5nz55FTEwMIiIiIMmi0qIWy8zMFABEZmZmtY+1aZMQDRsKof72Vj/8/NTbiYjqi3v37okzZ86Ie/fuldpX/P/Hko9+/dRt1q4tv53msXbtg+O6u5feXxWPPvqomDhxovb57t27BQCxZcuWCl8bHBwsvv76a+3zxo0bi6+++qrY1w7x3nvvaZ9nZ2cLAGL79u0VHltTR3p6uhBCiJEjR4revXvrtJk+fbpo1aqVEEKITZs2CScnJ5GVlVXqWEePHhUARGJiYoXvK0T5n2d1f4bW+x4IjYgIIDER+PFH9XM7OyAhgYtIERFpZGeX/di0Sd2mKuPKEhNLH8+YOnTooPM8Ozsb06ZNQ1BQEJRKJRwcHHD27NkKeyDatGmj/bu9vT2cnJyQkpJS6XrOnj2L0NBQnW2hoaG4ePEiCgsL0bt3bzRu3BhNmjTBs88+izVr1mjvS9K2bVv07NkTISEhGDp0KL799lukp6dXugZjYIAoRi4HBg1S/z0nB7h3T9p6iIjMib192Q/NKslVGVem73jGrVv3gNOmTcPmzZsxZ84cxMbG4sSJEwgJCUF+fn65x7GystJ5LpPJoFKpjFssAEdHRxw7dgzr1q2Dj48PPvjgA7Rt2xYZGRmQy+XYsWMHtm/fjlatWuHrr79GixYtkJCQYPQ6KsIAUYKjo3rMw7p1HPdARFRZUo4rs7a2Nuj24/v27cPYsWMxaNAghISEwNvbG4mJicYvqAxBQUHYt29fqZqaN28OedGJsbS0RK9evfD555/j33//RWJiIv766y8A6uASGhqKWbNm4fjx47C2tsbmzZtrrH6Ner8OhD4TJ0pdARFR7RURAURF6V8HYsEC010a9vf3x8GDB5GYmAgHB4cyeweaNWuG6Oho9O/fHzKZDO+//75JehLKMnXqVHTs2BEff/wxnnnmGezfvx+LFy/G0qVLAQBbt27F5cuX0b17d7i4uOC3336DSqVCixYtcPDgQezatQuPP/44PD09cfDgQaSmpiIoKKjG6tdgDwQRERmdZlzZ7t3A2rXqP009rmzatGmQy+Vo1aoVPDw8yhzTMH/+fLi4uKBbt27o378/+vTpg4ceesh0hZXw0EMPYePGjVi/fj1at26NDz74ALNnz8bYsWMBAEqlEtHR0XjssccQFBSEb775BuvWrUNwcDCcnJywZ88e9OvXD82bN8d7772HefPmoW/fvjVWv4ZMCCnmfhhHVlYWnJ2dkZmZCScnJ6Md9/Jl4ORJoFEjoH17ox2WiMjs5ebmIiEhAQEBAUa//TPVvPI+z+r+DGUPhB7ffQc8/TSwcqXUlRAREZknBgg9fH3Vf16/Lm0dRERk/l599VU4ODjofbz66qtSl2cyHESpR4MG6j8ZIIiIqCKzZ8/GtGnT9O4z5uV1c8MAoQcDBBERGcrT0xOenp5Sl1HjeAlDD02ASE6uXfevJyIiqikMEHp4eQEWFurwUIVVSomIiOo8Bgg9LC0Bb2/13//7T9paiIiIzBHHQJThs8/UQcLfX+pKiIiIzA8DRBlGj5a6AiIiIvPFSxhERGQShUIgJj0d627eREx6OgrNfOFjf39/LFiwwKC2MpkMW7ZsMWk95o49EGX47z/g8GH13Tkfe0zqaoiIapfo1FRMjI/Htbw87baGCgUWBgYiwsNDwsrIWNgDUYYdO9TLWc+dK3UlRES1S3RqKoacPq0THgDgel4ehpw+jejUVIkqI2NigCgDF5MiIlITQuBuYaFBj6yCArx58SL0XazQbJsYH4+sgoIKj1WZez2uWLECvr6+pW7LPXDgQIwbNw6XLl3CwIED4eXlBQcHB3Ts2BE7d+6s+kkp4eTJk3jsscdga2sLNzc3vPzyy8jOztbuj4mJQadOnWBvbw+lUonQ0FBcuXIFABAXF4fw8HA4OjrCyckJDz/8MI4cOWK02kyFlzDKoAkQnMZJRPVdjkoFh9hYoxxLALiWlwfnvXsrbJsdFgZ7udyg4w4dOhRvvPEGdu/ejZ49ewIAbt++jd9//x2//fYbsrOz0a9fP3z66adQKBRYvXo1+vfvj/Pnz6NRo0bV+ZJw9+5d9OnTB127dsXhw4eRkpKCF198ERMmTMCqVatQUFCAp59+Gi+99BLWrVuH/Px8HDp0CDKZDAAwatQotG/fHsuWLYNcLseJEydgZWVVrZpqAgNEGTQBIiMDyMkB7OwkLYeIiMrh4uKCvn37Yu3atdoAERUVBXd3d4SHh8PCwgJt27bVtv/444+xefNm/O9//8OECROq9d5r165Fbm4uVq9eDXt7ewDA4sWL0b9/f3z22WewsrJCZmYmnnrqKTRt2hQAEBQUpH391atXMX36dLRs2RIA0KxZs2rVU1MYIMrg5AQ4OADZ2erLGLXk8yQiMjo7Cwtkh4UZ1HZPRgb6nTxZYbvfQkLQXams8H0rY9SoUXjppZewdOlSKBQKrFmzBsOHD4eFhQWys7Px0UcfYdu2bUhOTkZBQQHu3buHq1evVuo99Dl79izatm2rDQ8AEBoaCpVKhfPnz6N79+4YO3Ys+vTpg969e6NXr14YNmwYfHx8AABTpkzBiy++iJ9++gm9evXC0KFDtUHDnHEMRDl4W28iIvWURXu53KDH466uaKhQQFbWsQD4KRR43NW1wmNpuvgN1b9/fwghsG3bNiQlJSE2NhajRo0CAEybNg2bN2/GnDlzEBsbixMnTiAkJAT5+fnVOzkGWrlyJfbv349u3bphw4YNaN68OQ4cOAAA+Oijj3D69Gk8+eST+Ouvv9CqVSts3ry5RuqqDgaIcnAgJRFR5chlMiwMDASAUiFC83xBYCDklQwHhrCxsUFERATWrFmDdevWoUWLFnjooYcAAPv27cPYsWMxaNAghISEwNvbG4mJiUZ536CgIMTFxeHu3bvabfv27YOFhQVatGih3da+fXvMnDkT//zzD1q3bo21a9dq9zVv3hyTJ0/Gn3/+iYiICKxcudIotZkSA0Q5pk8H1q8HuneXuhIiotojwsMDUcHBaKBQ6GxvqFAgKjjYpOtAjBo1Ctu2bcMPP/yg7X0A1OMKoqOjceLECcTFxWHkyJGlZmxU5z1tbGwwZswYnDp1Crt378Ybb7yBZ599Fl5eXkhISMDMmTOxf/9+XLlyBX/++ScuXryIoKAg3Lt3DxMmTEBMTAyuXLmCffv24fDhwzpjJMyV5GMgrl+/jrfffhvbt29HTk4OAgMDsXLlSnTo0EHq0tC3r9QVEBHVThEeHhjo7o7YjAwk5+fDx9oaYUqlSXoeinvsscfg6uqK8+fPY+TIkdrt8+fPx7hx49CtWze4u7vj7bffRlZWllHe087ODn/88QcmTpyIjh07ws7ODoMHD8b8+fO1+8+dO4cff/wRaWlp8PHxwfjx4/HKK6+goKAAaWlpeO6553Dz5k24u7sjIiICs2bNMkptpiQTlZloa2Tp6elo3749wsPD8dprr8HDwwMXL15E06ZNDRpAkpWVBWdnZ2RmZsLJyakGKiYiqttyc3ORkJCAgIAA2NjYSF0OVVN5n2d1f4ZK2gPx2Wefwc/PT+daT0BAgIQV6bp9G9izBygsBAYPlroaIiIi8yHpGIj//e9/6NChA4YOHQpPT0+0b98e3377bZnt8/LykJWVpfMwpYsXgUGDBV5f8eBmMPkFAjExwLp1QEyMOlwQEVHdsWbNGjg4OOh9BAcHS12e2ZC0B+Ly5ctYtmwZpkyZgnfeeQeHDx/Gm2++CWtra4wZM6ZU+8jIyBq9LhTnlAqsi0eKZx5GnlVvk6cpULgwEIhVDwJq2BBYuBCIiKixsoiIyIQGDBiAzp07691XG1aIrCmSjoGwtrZGhw4d8M8//2i3vfnmmzh8+DD2799fqn1eXh7yit2cJSsrC35+fiYZA6G5GYwQ0J2LpCp6/mEwEOsBzXigqCiGCCKq/TgGom4x5RgISS9h+Pj4oFWrVjrbgoKCylwZTKFQwMnJSedhCoVCYGJ8vPrGLyUHDFtAvZj71PNA+3QImYAQwEsvAbt28ZIGEdUNEv5uSUZkys9R0gARGhqK8+fP62y7cOECGjduLEk9hUIgJj0dHyUklLoNrQ4LAM4FwPw4YN0BICwVt28DvXoB/v5AdHRNVUxEZFyaLvqcnByJKyFj0HyOprj0IukYiMmTJ6Nbt26YM2cOhg0bhkOHDmHFihVYsWJFjdcSnZqKifHx5QcHfdzzgFmntZc0rl8HhgzhJQ0iqp3kcjmUSiVSUlIAqNcwqOyS0iQ9IQRycnKQkpICpVIJuYF3Na0MScdAAMDWrVsxc+ZMXLx4EQEBAZgyZQpeeuklg15rrHUgtOMdqnoAFYBUBTCyC6CSQSZTD65MSABM8JkREZmUEAI3btxARkaG1KVQNSmVSnh7e+sNgdX9GSp5gKgOYwSIQiHgf+BA5Xse9JnUFohz0T7dvRvo0aP6hyUikkJhYSHu378vdRlURVZWVuX2PNTqhaTMQWxGhnHCAwC46d7VLTnZOIclIpKCXC43Sdc31Q31/mZayca8lWuatc7Tolu9ExER1Tn1PkD4WFtX3AgA7spR5iAJFYCbCuCkEgAgkwF+fkBYmDEqJCIiMj/1PkCEKZVoqFCUWu5BSxMOPm/x4HnJ/TIASwK1AygBYMECDqAkIqK6q94HCLlMhoWBgQBKrxmlEw72eKqnat7Svb+9PF2hncIJAN7enMJJRER1X70PEID6vvVRwcFooNANB25CAbdFD8IBYj3QcHoXjLmrvtV4A2tr5Azsgt2zPeDtrW6yejXDAxER1X31fhaGRoSHBwa6uyM2IwPJ+fnwsbZGmFIJ9JAhNlY9o8LHBwgLk+H6fQ/8eOASUu7fh1yunqrZvj2wfbt67QciIqK6jgGiGLlMhh4uLiU2ll7LoYGFAtYyGfKFQFJuLvxtbdFU3SmBS5dqpFQiIiJJ8RJGFchlMgQU3dXsUm4uADBAEBFRvcIAUUVNbW0BAPH37gEAunYFXn8dGDRIyqqIiIhqBi9hVFFgUYC4VBQgOndWP4iIiOoD9kBUUdMSAYKIiKg+YYCoIn0BIjMTOHYMuHVLqqqIiIhqBgNEFTUtNohSc0PTgQOBhx8Gfv9dysqIiIhMjwGiigJsbSEDkF1YiJSi290WLWjJmRhERFTnMUBUkcLCAn5FK1dqLmNwKicREdUXDBDVUHIchCZAxMdLVREREVHNYICohpIBgpcwiIiovmCAqAbNQMr4Ej0QKSnAnTtSVUVERGR6DBDVoF1Mqmg5a2dnwM1Nve/yZamqIiIiMj2uRFkN+taCmD4dsLQEPDykqoqIiMj0GCCqQRMgUu/fR1ZBAZwsLfH22xIXRUREVAN4CaManCwt4W5lBYBLWhMRUf3CAFFNJW+qlZsLnDgB7NkjYVFEREQmxgBRTcWXtAaAI0eA9u2B556TsioiIiLTYoCoprLWgkhKAvLypKqKiIjItBggqkkTIDRrQXh5Afb2gEoFJCZKWBgREZEJMUBUU0DRJYx/s7MRk54OFQSaNFHv+/57ICYGKCyUrj4iIiJTYICohujUVDxz5gwAIK2gAOFxcfD66wDOeaQCAL74AggPB/z9gehoCQslIiIyMgaIKopOTcWQ06eRnJ+vsz1Nlof7750GwlK1265fB4YMYYggIqK6Q9IA8dFHH0Emk+k8WrZsKWVJBikUAhPj4yH07bQAIACMjwcs1C1EUcNJk3g5g4iI6gbJeyCCg4ORnJysfezdu1fqkioUm5GBa+VNsbAA4JUHhGRoNwmhnpkRG2vy8oiIiExO8qWsLS0t4e3tLXUZlVLyskWZ3Eq3S042cjFEREQSkLwH4uLFi/D19UWTJk0watQoXL16tcy2eXl5yMrK0nlIwcfa2rCGaaXb+fgYuRgiIiIJSBogOnfujFWrVuH333/HsmXLkJCQgLCwMNy5c0dv+8jISDg7O2sffn5+NVyxWphSiYYKBWRlNVABuKkATiq1m2QywM8PCAurgQKJiIhMTCaE0DsWUAoZGRlo3Lgx5s+fjxdeeKHU/ry8POQVG3uQlZUFPz8/ZGZmwsnJqSZL1c7CAKA7mFIFQAbgw2AgVn1Pb1lR0oiKAiIiarJKIiIi/bKysuDs7Fzln6GSX8IoTqlUonnz5oiPj9e7X6FQwMnJSechlQgPD0QFB6OBQqGz3U0o4LboQXgAgIYNGR6IiKhuMasAkZ2djUuXLsGnlgwUiPDwQGKXLvhf69babVce7YibUR4ID1c/Hz8eSEhgeCAiorpF0gAxbdo0/P3330hMTMQ///yDQYMGQS6XY8SIEVKWVSlymQxPubnBuug6xa2CAsjlQJs26v0ODoBcLmGBREREJiDpNM5r165hxIgRSEtLg4eHBx555BEcOHAAHh4eFb/YjMhkMnhbW+NqXh5u5OejsY0NNDNTb9yQtjYiIiJTkDRArF+/Xsq3NyqfYgECAAMEERHVaWY1BqI28y5aGyK5aJYIAwQREdVlkq9EWVdoAoSmByIkBJg/HwgIkLIqIiIi02CAMJKSAaJBA2DyZCkrIiIiMh1ewjCSkgGCiIioLmOAMBJ9AeLECeDXX4HUVImKIiIiMhEGCCPxKVqRsniAeP554OmngSNHJCqKiIjIRBggjEQ7CyM/H5rbi3AmBhER1VUMEEbiZWUFAMgXAhkFBQAYIIiIqO5igDASG7kcSkv1pBYuJkVERHUdA4QRlRxIqbknWHKyVBURERGZBgOEEfmUCBDsgSAiorqKAcKISvZAMEAQEVFdxZUojahkgGjZEvjqK6BRIymrIiIiMj4GCCMqPpUTADw9gUmTJCyIiIjIRHgJw4i4nDUREdUXDBBGpC9AxMUBW7ZwJgYREdUtDBBGVHIWBgC8/jowaBDwzz9SVUVERGR8DBBGpOmBuHX/Pu6rVOptnIlBRER1EAOEEblZWUEOQABIvX8fAAMEERHVTQwQRmQhk8GrxEwMrkZJRER1EQOEkXExKSIiqg8YIIyMAYKIiOoDBggj4/0wiIioPuBKlEZWsgeiaVNgwQKgQQMJiyIiIjIyBggjKxkgXFyAiROlrIiIiMj4eAnDyLT3w8jLk7gSIiIi02GAMDJ9y1n/+696OeukJImKIiIiMjIGCCPzUSgA6AaIqVPVy1nHxEhUFBERkZExQBiZl5UVAOCuSoXsggIAnIlBRER1DwOEkTlYWsJBLgfAqZxERFR3mU2AmDt3LmQyGSZNmiR1KdXGxaSIiKiuM4sAcfjwYSxfvhxt2rSRuhSj8C5xPwwGCCIiqmskDxDZ2dkYNWoUvv32W7i4uEhdjlGU7IHQ3FCLAYKIiOoKyQPE+PHj8eSTT6JXr14Vts3Ly0NWVpbOwxxpBlLuSk9HTHo6XN0FAODKFfVMjMJCCYsjIiIyAkkDxPr163Hs2DFERkYa1D4yMhLOzs7ah5+fn4krrLzo1FT8dPMmAODXtDSEx8Xh4YsHgLBU3L0LhIcD/v5AdLS0dRIREVWHZAEiKSkJEydOxJo1a2BjY2PQa2bOnInMzEztI8nMVmaKTk3FkNOnkVWii0HlkgfMOg2EpQIArl8HhgxhiCAiotpLJoQQUrzxli1bMGjQIMiLpjwCQGFhIWQyGSwsLJCXl6ezT5+srCw4OzsjMzMTTk5Opi65XIVCwP/AAVwrawlrFYBUBTCyC6CSQSYDGjYEEhKACr5MIiIio6vuz1DJbqbVs2dPnDx5Umfb888/j5YtW+Ltt9+uMDyYm9iMjLLDA6Du6/HKA0IygDgXCKFe2jo2FujRo4aKJCIiMhLJAoSjoyNat26ts83e3h5ubm6lttcGycWWri6Xm2675GQTFENERGRiks/CqCt8iqZuVihNt51miicREVFtIlkPhD4xtfhuU2FKJRoqFLielwe9g0o0YyBOKgFAOwYiLKwGiyQiIjIS9kAYiVwmw8LAQACArOROVdHGJYGA6sHeBQs4gJKIiGonBggjivDwQFRwMBoU3dJbw+KOFfBhMBDrAQCwswOiooCICCmqJCIiqj4GCCOL8PBAYpcu2N22Lbo4OgIA3g72we7ZHvjwQ3UbhQIYOFDCIomIiKqJAcIE5DIZeri44BVfXwDAzsx09OgBvPce4OwMpKcDp09LWyMREVF1MECY0OOurgCAI3fu4FZ+PiwtgV9/Vd9Uq47ceJSIiOopBggT8lUo0MbeHgLAjvR0AMCjjwJeXtLWRUREVF0MECbWp6gX4o/btyWuhIiIyHgYIEzsCU2ASE+H5rYjq1cDvXoBW7ZIWBgREVE1VClA/Pjjj9i2bZv2+VtvvQWlUolu3brhypUrRiuuLgh1doatTIYb+fn47OpVxKSn49gJgV27gG+/BdatA2JigBI38CQiIjJrVQoQc+bMga2tLQBg//79WLJkCT7//HO4u7tj8uTJRi2wttuWlgYhUy8eNTMhAeFxcfjh8QNAWCp++w0YORIIDwf8/Xl7byIiqj2qFCCSkpIQWLTq4pYtWzB48GC8/PLLiIyMRGxsrFELrM2iU1Mx5PRp5KpUOtvvWOcBs04DYanabdevA0OGMEQQEVHtUKUA4eDggLS0NADAn3/+id69ewMAbGxscO/ePeNVV4sVCoGJ8fH674thAUAAGB8PWKhbFA2PwKRJvJxBRETmr0oBonfv3njxxRfx4osv4sKFC+jXrx8A4PTp0/D39zdmfbVWbEYGruXlld3AAoBXHhCSod0kBJCUBLATh4iIzF2VAsSSJUvQtWtXpKamYtOmTXBzcwMAHD16FCNGjDBqgbVVcn6+YQ3dSrdLTjZyMUREREZWpdt5K5VKLF68uNT2WbNmVbugusLH2tqwhmml2/n4GLkYIiIiI6tSD8Tvv/+OvXv3ap8vWbIE7dq1w8iRI5FetOJifRemVKKhQlH61t4aKgA3FcBJpXaTTAb4+QFhYTVQIBERUTVUKUBMnz4dWVlZAICTJ09i6tSp6NevHxISEjBlyhSjFlhbyWUyLCyaqVIqRKiKNi4JBFTqvUUzPbFgASCX11CRREREVVSlAJGQkIBWrVoBADZt2oSnnnoKc+bMwZIlS7B9+3ajFlibRXh4ICo4GA0UCp3t9rCE26JgINZDu61hQyAqCoiIqOkqiYiIKq9KAcLa2ho5OTkAgJ07d+Lxxx8HALi6ump7JkgtwsMDiV26YHfbtnja3R0AMNjHDTejPLBtG+Dpqe59+PdfhgciIqo9qjSI8pFHHsGUKVMQGhqKQ4cOYcOGDQCACxcuoGHDhkYtsC6Qy2To4eKCuyoVtty6hUN37kAuB/r1A2xt1dM3jx0DHntM6kqJiIgMU6UeiMWLF8PS0hJRUVFYtmwZGjRoAADYvn07nnjiCaMWWJd0dnQEAJzLyUH6/fvqbZ3V+w4elKoqIiKiyqtSD0SjRo2wdevWUtu/+uqrahdUl7lbW6OZrS0u3ruHg1lZeMLNDV26ABs3AgcOSF0dERGR4aoUIACgsLAQW7ZswdmzZwEAwcHBGDBgAOScQlCuLk5OuHjvHg4UCxCAOkAI8WA2BhERkTmrUoCIj49Hv379cP36dbRo0QIAEBkZCT8/P2zbtg1NmzY1apF1SRcnJ/x08yYOFA02bd8esLICUlKAxEQgIEDa+oiIiAxRpTEQb775Jpo2bYqkpCQcO3YMx44dw9WrVxEQEIA333zT2DXWKV2cnAAAB+/cgUoI2NioQwTAyxhERFR7VKkH4u+//8aBAwfg6uqq3ebm5oa5c+ciNDTUaMXVRSH29rC1sEBGQQHO5+QgyN4ew4apQ0STJlJXR0REZJgqBQiFQoE7d+6U2p6dnQ1rQ+8BUU9ZWVigo6Mj9mRm4kBWFoLs7TF1qtRVERERVU6VLmE89dRTePnll3Hw4EEIISCEwIEDB/Dqq69iwIABxq6xztFcxjjARbeIiKiWqlIPxKJFizBmzBh07doVVlZWAID79+9j4MCBWLBggTHrq5M0AWJnejrW3bwJH2trPGytxE8/yiAEEBysvqEWJ7QQEZG5qvLtvH/99VfEx8drp3EGBQUhsOjmUVS+tKJFpC7n5mJk0flDqgJYH6i9P0bDhsDChVzemoiIzJNMCCEMaViZu2zOnz+/ygVVRlZWFpydnZGZmQmnot/qzV10aiqGnD6NUiddc4fOD9U32dKsB8EbbBERkSlU92eowT0Qx48fN6idrBIrIS1btgzLli1DYmIiAPViVB988AH69u1r8DFqk0IhMDE+vnR4ANSjUVQAxscD+9whVDLIZMCkScDAgbycQURE5sXgALF7926jv3nDhg0xd+5cNGvWDEII/Pjjjxg4cCCOHz+O4OBgo7+f1GIzMnAtL6/sBhYAvPKAkAwgzgVCAElJQGws0KNHDRVJRERkgCovZW0M/fv313n+6aefYtmyZThw4ECdDBDJ+fmGNXTTbZecbIJiiIiIqkHSAFFcYWEhfvnlF9y9exddu3bV2yYvLw95xX6Dz6pl0yB9DF0jI023nY+PCYohIiKqhiqtA2FMJ0+ehIODAxQKBV599VVs3rwZrVq10ts2MjISzs7O2oefn18NV1s9YUolGioUKHOUiArATQVwUglAfWMtPz/1lE4iIiJzInmAaNGiBU6cOIGDBw/itddew5gxY3DmzBm9bWfOnInMzEztIykpqYarrR65TIaFRVNdS4UIzSyMJYFA0QBKAFiwgAMoiYjI/Bg8jbOm9OrVC02bNsXy5csrbFsbp3EC6qmcE+PjdQZUytMUKFz4YB0IPz91eOAUTiIiMoUam8ZZU1Qqlc44h7oowsMDA93dMeDkSfx2+zbGeHlhxSMt8Y+bDMnJ6jEPXImSiIjMmaQBYubMmejbty8aNWqEO3fuYO3atYiJicEff/whZVk1Qi6TIdTZGb/dvg0BwNpShkcfBY4eBS5fBu7fZ4AgIiLzJWmASElJwXPPPYfk5GQ4OzujTZs2+OOPP9C7d28py6oxTWxsAAAJubnabY89Bty5A5w5AwQFSVUZERFR+SQNEN9//72Uby+5AFtbAMDle/cAqGddNGkCxMUBCQkMEEREZL4kn4VRn2l6IP7Lz0duYSEAICBAve/yZamqIiIiqhgDhITcraxgb2EBAeBK0cDRJk3U+xISpKuLiIioIgwQEpLJZNrLGAlFlzHYA0FERLUBA4TESg6kZA8EERHVBgwQEgsoChCXiwJE8R4I81rii4iI6AGzW0iqvmmi5xLG/PnqP4WAdklrIiIic8IAIbGSPRA2NsDkyVJWREREVDFewpCYtgei2GJSRERE5o4BQmL+RT0QGQUFSL9/H4B6/ENUFHDggJSVERERlY0BQmL2cjm8rKwAPOiF+PFHYOhQ4IcfpKyMiIiobAwQZqDkktacyklEROaOAcIMlFwLgotJERGRuWOAMAMlZ2JoeiCuXgUKCqSqioiIqGwMEGag5HLWvr6AtbU6PFy7JmVlRERE+jFAmIGSlzAsLAB/f/U+joMgIiJzxABhBjSXMBJzc6EqWr+a4yCIiMicMUCYgYYKBSxlMuQLgf+Kbus9bRoQHQ088YTExREREenBpazNgKWFBRopFLicm4vLubloaGODXr2kroqIiKhs7IEwE1zSmoiIahMGCDOhncpZNBMjJwfYtAlYvFjKqoiIiPTjJQwz4a9QAAD+Sk9HuFKJ1kKJIUPU9/J2cFDPyggLA+RyCYskIiIqwh4IMxCdmop5RQs+7M3KQnhcHFoePwB0TwUAPP88EB6uDhHR0RIWSkREVIQBQmLRqakYcvo0bpdYcjJNlgd8dBoIS9Vuu34dGDKEIYKIiKTHACGhQiEwMT4eQt9OCwACwPh4wELdomiJCEyaBBQW1kyNRERE+jBASCg2IwPXitZ90MsCgFceEJKh3SQEkJQExMaavDwiIqIyMUBIKDk/37CGbqXbJScbuRgiIqJKYICQkI+1tWEN00q38/ExcjFERESVwAAhoTClEg0VCsjKaqACcFMBnFRqN8lkgJ+fekonERGRVBggJCSXybAwMBAASocIVdHGJYGASr1XVtRowQKuB0FERNJigJBYhIcHooKD0aBoISkNN6GA26JgINZDu61hQyAqCoiIqOkqiYiIdEkaICIjI9GxY0c4OjrC09MTTz/9NM6fPy9lSZKI8PBAYpcu2NmmDTQdC4e6tcPNKA98/z0wciQwaxaQkMDwQERE5kHSAPH3339j/PjxOHDgAHbs2IH79+/j8ccfx927d6UsSxJymQw9XV3Rws4OAHAhJwdyOZCdDaxdCxw/zssWRERkPiS9F8bvv/+u83zVqlXw9PTE0aNH0b17d4mqklaQnR3O5OTgbE4OnnBzQ1CQevvZs9LWRUREVJxZ3UwrMzMTAODq6qp3f15eHvKKLbyUlZVVI3XVpCB7e+DWLZzNyVE/LwoQ8fFAfj5g6MxPIiIiUzKbQZQqlQqTJk1CaGgoWrdurbdNZGQknJ2dtQ8/P78artL0goouYWgCRIMGgKOjeunq+HgpKyMiInrAbALE+PHjcerUKaxfv77MNjNnzkRmZqb2kZSUVIMV1gxtgCgaByKTAS1bqvfxMgYREZkLs7iEMWHCBGzduhV79uxBw4YNy2ynUCigKDHdsa5pYWcHGYC0ggKk5ufDw9oaQUHA4cMMEEREZD4k7YEQQmDChAnYvHkz/vrrLwQEBEhZjlmwk8vR2MYGAEqNg2CAICIicyFpD8T48eOxdu1a/Prrr3B0dMSNGzcAAM7OzrC1tZWyNEkF2dkhMTcXZ3Ny0F2pxKhRQO/eDy5lEBERSU3SHohly5YhMzMTPXr0gI+Pj/axYcMGKcuSXMlxEH5+wMMPA/b2UlZFRET0gKQ9EEIIKd/ebJWciUFERGRuzGYWBj0QVNTVUDxArFsHvPYacPCgVFURERE9wABhhjQ9EEl5ecguKAAAREcD33wD7NsnZWVERERqDBBmyNXKCp5WVgCAc5yJQUREZogBwkyVHAfBAEFEROaEAcJMlRwH0aqVevvZswDHnhIRkdQYIMxUyR6I5s3Vy1rfvg2kpkpZGREREQOE2dIEiKNZWVh38yYO5qajcYC66+Hrr4GYGPUNtoiIiKTAAGGmEnNzAQBJ+fkYefYswuPikDj3ABCWik8+AcLDAX9/9ewMIiKimsYAYYaiU1PxyoULpXe45QGzTgNh6msY168DQ4YwRBARUc1jgDAzhUJgYnw89I6TtAAgAIyPByyEdjDlpEm8nEFERDWLAcLMxGZk4FpeXtkNLAB45QEhGQDUMzKSkoDY2Bopj4iICAADhNlJzs83rKGbbrvkZBMUQ0REVAYGCDPjY21tWMM03XY+PiYohoiIqAwMEGYmTKlEQ4UCsrIaqADcVAAnlQDUa0P4+QFhYTVUIBERERggzI5cJsPCwEAAKB0iVEUblwQCKhlkRQ0WLADk8pqrkYiIiAHCDEV4eCAqOBgNFAqd7fJ0BfBhMBDrAQDw8gKiooCICCmqJCKi+owBwkxFeHggsUsXrCu6i5YFgBtPdsDu2R5o00bdZto0hgciIpIGA4QZk8tkGO7lhaY2NlABOJCdiR49gOeeU+/fvVvK6oiIqD5jgKgFerm4AAB2pqern/cCXF0BT08pqyIiovrMUuoCqGI9XVywPDkZuzIyAABt2qjvyGnB+EdERBLhj6BaIFyphAzAqbt3cSMvDzIZwwMREUmLP4ZqAXdra7RzcAAA/FXUCwGol7G+elWiooiIqF5jgKglSo6DuH1bvYBU06ZAdraUlRERUX3EAFFL9CwKEFvT0rD25k38K0uHlUKgoACYPRuIieEdOYmIqOYwQNQSaffvAwBS79/HqLNnER4XhytzDwBhqfjiCyA8HPD3B6Kjpa2TiIjqBwaIWiA6NRWjz54ttV245QGzTgNhqQCA69eBIUMYIoiIyPQYIMxcoRCYGB8PoW+nBQABYHw8YCEgihpNmsTLGUREZFoMEGYuNiMD1/Lyym5gAcArDwjJAKCemZGUBMTG1kh5RERUT3EhKTOXnJ9vWEM33XabNqn/DAvjnTqJiMj42ANh5nysrQ1r6JIPWDy40LF4MQdWEhGR6UgaIPbs2YP+/fvD19cXMpkMW7ZskbIcsxSmVKKhQgFZRQ0nXALWHdAOqNTgwEoiIjIFSQPE3bt30bZtWyxZskTKMsyaXCbDwsBAAKg4RLjrzsoAwIGVRERkEpKOgejbty/69u0rZQm1QoSHB6KCgzExPr7iAZUqqGdl7HMHVOrIUXxgZY8eNVExERHVdbVqDEReXh6ysrJ0HvVFhIcHErt0wVdNm5bfsMSsjOKSk01SGhER1UO1KkBERkbC2dlZ+/Dz85O6pBoll8ngZeigyu6pQNt0nYGVN2/yMgYRERlHrQoQM2fORGZmpvaRlJQkdUk1zuBZGRH/AQvidAZWTp7MWRlERGQctSpAKBQKODk56TzqmwpnZZRcstKdy10TEZHx1aoAQRXMyhB6NnK5ayIiMgFJA0R2djZOnDiBEydOAAASEhJw4sQJXL16VcqyzJ5mVkYDhUJ3R1ndElzumoiIjEzSaZxHjhxBeHi49vmUKVMAAGPGjMGqVaskqqp2iPDwwEB3d8RmZGBTaioW//dfxS8qsdw1Z2UQEVFVSRogevToASH03meSDCCXydDDxQUADAsQmuWui9aHOHMGiInh/TKIiKjyOAaiDqjqcteffML7ZRARUdUwQNQB1V3umjMziIioshgg6ogyB1aWVGJWBsD7ZRARUeUxQNQh1VnumjMziIioMhgg6pjqLne9axd7IYiIqGIMEHVQdZa7/uQTDqokIqKKMUDUQVzumoiITI0Bog6qcFYGl7smIqJqYoCoowyelaGhGVg56Jo2RCQlAV9/zRBBRESlMUDUYZpZGbvbtsUEX1/DXlRisSneApyIiPRhgKjjNMtdD/bwMPxFHBNBREQVkPReGFRzNAMrr+fllRpDWYoFABWAqeeBbEuIOCWgkuHVV4F794AGDXj/DCKi+o49EPVEpZa7BtTfGc4FwPwH0zxTU4HRo3n/DCIiYoCoVyo9sFJDc0nj2UTgsZtA23Rc+09g8GD1GImYGA60JCKqb2SiFt9POysrC87OzsjMzISTk5PU5dQahULg62vXMPnSJcNfJKDbdZGiABYHArHqsRUNGwILFwIREUYtlYiITKS6P0MZIOqpQiHgf+CAYWMi9FFBHSiiGgL73IBTSqBQhlmzgGbNAB8fjpMgIjJnDBAMEFUWnZqKIadPAyi9OGWlleiRANgrQURkzqr7M5RjIOqxKo+J0IfjJIiI6hX2QBAKhUBMejqGnTmD2wUFVT9QqXES1sBWX+C6LZBmDd80JV55ScZLHEREZoCXMBggjMaolzQ0B+HASyIis8QAwQBhVNGpqZgYH49reXkPNpYMAlXFgZdERGaDAYIBwugKhUBsRgaS8/NxMScHH125AsBIvRLFceAlEZFkGCAYIExOb6+EMWh6JFb6a8dJ4KQSUMkwaRIwcCB7JIiITIUBggGiRmh6JX69dQsLrl+HDKYaJ6E78LLBbSXmzQOSPTJwKT0fTV2s8UpXJQ7ulyE5mZc9iMzGiRPAzJlAZCTQrp3U1ZABqvszlDfTIoNo7urZw8UFYUql8cZJlHyNRz4wLlH79HqmJYYDAAoAF/W2yb9ydgeR2dm0Cfj9d6BjRwaIeoI9EFQlNTZOQnNAWYltRui1AIDYWLAng8gY2rUD4uLUfx4/LnU1ZABewmCAMAsmnb1RkZLvk1nUseZcbE2LEiHD9T8lAOC2bwbglg+kWcPntjP6TM2EQyN16HixizO+O5CpDSGvP6J+zdK9D4KJvjZymYzBhOqXmzcBb2/d556e0tVDBmGAYIAwGyV7JVYkJ+N6fn7NF2JIr4W+kFEIoPgP+hLPZVnq1winsl9jcUsB21VNcfealdGDSUVtpDyu3h4eiwffDz7W1ujmqMQ/e6UZu1L8e7OsWqpSb8njdnZwxvJ95QfKbo8I/HPnwWvClEpAVX4bQ+qtynsb7TNZvRoYM0b3+bPPVv0DoxrBAMEAYbZMNvDSGAwJGfqeV/QazcyS4tuMEEwMaSPpcUv28DS5D9n4S0iTP+iRskizhurXB21KBitTBR7flvcx9fIlXMsvu5aq1KvvuBUGygY5sBiYDJXbg2DtVqiAWNIUty+X3caQeqvy3iWPq+/ynyGfwfglr0IevQkWqkIUyuVQDRqCJeOX1amQbE7Hff0RJawtq9+9ywDBAFErSHqJo6YZEkQqG0xMFXikPG4NBR5T1VvlQFmVNsYKsxUdt4yeOd/bqfBKT1e/5V31SRD26pvbyITAX1OmwDkn58Fh7Ozx2Px5EDKZ3tfcdHHBf64etSskm9Fx5WkKTLEOxOf9H6yhUxV1IkAsWbIEX3zxBW7cuIG2bdvi66+/RqdOnSp8HQNE7VJjAy9JeoaEw5oMPJWtxdB6DQnAxnr/qrSvynGBUl/nzqlT0LOcgZEqmQwWxX6UlHxe0s6HHkLvL+eZT5itbcctCofTs4OrFSJq/d04N2zYgClTpuDDDz/EsWPH0LZtW/Tp0wcpKSlSl0ZGppkKOsLLCx8EBOi/E2jJ/3OEnm1k/gz5oVWyTcnflqvTprq1GNLG0B/Mxnp/U713yfZ6vs5vBgzAbQeHMv8plgwLZYUHAeC2gwOW9+9vvM/WVN8z5nxcCwACmJ8fj/wC6f6DlLwHonPnzujYsSMWL14MAFCpVPDz88Mbb7yBGTNmlPta9kDUfhUNvHQotERODqByLNadV1cvfRCZMY/0dCz76isMjo2tsIehJE37TWFheG3yZKS6uJiw0vrlK7TFpB5VO5+1eiGp/Px8HD16FDNnztRus7CwQK9evbB///5S7fPy8pBX7Bp6VlZWjdRJpqPpldB4199fZ2R5mFKJwsIHA4ruuuTgT0WJ2R2GdAESUbWkurhgyOzZGLp7N76ZPx9OOTmwVKkqfF2BhQWy7Ozw6pQp+CU8vAYqrV8upUsw062IpAHi1q1bKCwshJeXl852Ly8vnDt3rlT7yMhIzJo1q6bKIwmUDBQAILeETsIuFP7l91qoDOi1MOX1SwYXqsN+CQ9HTLt2WDV3LvoeOlTut7sA8GeHDhg7YwZ7HUykqYu1ZO9dq5aynjlzJqZMmaJ9npWVBT8/PwkrIikYo9fCzUr9rZ9WUCxkqKA7grrEc9kdPaOjS74GqJ+zMPQxpzBlioGM+l5jqgGcpnrvKkp1ccHRFi3Q+8gRWJXTC1FoYYEjLVowPJiCCpCnK/D6QKVkJUgaINzd3SGXy3Hz5k2d7Tdv3oR38VXNiigUCihKDrqjeq+yvRbaxXuAchfieTG0xNzrfurX6MzPLtHGN0jP2gBGCCYGhRcpj1vWFMLipAo8htRSlXr1vaYqgdKQNqZ672p8Bv3/+QfyCi5hWKhU6L9/Pz58fpzBxzXbkGxOxy36/KdYBxplPYiqMotBlJ06dcLXX38NQD2IslGjRpgwYQIHUVKtVNHqhLVt0ZqKXnPXJQd/Wpfo4TFgcaSamntvrMWcDFl11DfdGY9PKX+xqZKLN7k1uQ+Ut4iVIfVW5b31HFfvoGU9n4FXehqSxz6N4jQDJfUNsPT+8VekKF3Ndl2F2nZcrgNRZMOGDRgzZgyWL1+OTp06YcGCBdi4cSPOnTtXamxESQwQROahZGgyZHlmfUsvA6YJPMZYTtqQpaINWe66KstoG1JvVd5b33GLX/4r63xarP4Rli88r30fIZcjz9YBu/u9gPDfvofiXjZkhYXa/QU/rILq2edqbUg2t+NyJcpiFi9erF1Iql27dli0aBE6d+5c4esYIIiIJPDMM0BUFCCE+jFoEPDNN+obaKWkAK++CmzeDMhk6sfQocD69VJXTSXUiQBRVQwQREQ1rKAAcHMDsrIApRJYvhwYNqx0u40bgVdeATIyACcn4PZt3pbWzNT6lSiJiKgWuXcPaNJE3etw/rz+8ACot58/r27XtClQ7F4ZVDfUqmmcREQkMUdH4MgRw3oTPD2B6GigsJC9D3UQeyCIiKhyKhsGGB7qJAYIIiIiqjQGCCIiIqq0Wj0GQjOBhDfVIiIiqhzNz86qTsas1QHizp07AMD7YRAREVXRnTt34OzsXOnX1ep1IFQqFf777z84OjpCJqvaqlyaG3IlJSVxLQkT4Pk1LZ5f0+L5NS2eX9Oq6PwKIXDnzh34+vrCwqLyIxpqdQ+EhYUFGjZsaJRjOTk58RvYhHh+TYvn17R4fk2L59e0yju/Vel50OAgSiIiIqo0BggiIiKqtHofIBQKBT788EMoFAqpS6mTeH5Ni+fXtHh+TYvn17RMfX5r9SBKIiIikka974EgIiKiymOAICIiokpjgCAiIqJKY4AgIiKiSqvXAWLJkiXw9/eHjY0NOnfujEOHDkldUq0UGRmJjh07wtHREZ6ennj66adx/vx5nTa5ubkYP3483Nzc4ODggMGDB+PmzZsSVVy7zZ07FzKZDJMmTdJu4/mtnuvXr2P06NFwc3ODra0tQkJCcOTIEe1+IQQ++OAD+Pj4wNbWFr169cLFixclrLj2KCwsxPvvv4+AgADY2tqiadOm+Pjjj3Xuv8DzWzl79uxB//794evrC5lMhi1btujsN+R83r59G6NGjYKTkxOUSiVeeOEFZGdnV64QUU+tX79eWFtbix9++EGcPn1avPTSS0KpVIqbN29KXVqt06dPH7Fy5Upx6tQpceLECdGvXz/RqFEjkZ2drW3z6quvCj8/P7Fr1y5x5MgR0aVLF9GtWzcJq66dDh06JPz9/UWbNm3ExIkTtdt5fqvu9u3bonHjxmLs2LHi4MGD4vLly+KPP/4Q8fHx2jZz584Vzs7OYsuWLSIuLk4MGDBABAQEiHv37klYee3w6aefCjc3N7F161aRkJAgfvnlF+Hg4CAWLlyobcPzWzm//fabePfdd0V0dLQAIDZv3qyz35Dz+cQTT4i2bduKAwcOiNjYWBEYGChGjBhRqTrqbYDo1KmTGD9+vPZ5YWGh8PX1FZGRkRJWVTekpKQIAOLvv/8WQgiRkZEhrKysxC+//KJtc/bsWQFA7N+/X6oya507d+6IZs2aiR07dohHH31UGyB4fqvn7bffFo888kiZ+1UqlfD29hZffPGFdltGRoZQKBRi3bp1NVFirfbkk0+KcePG6WyLiIgQo0aNEkLw/FZXyQBhyPk8c+aMACAOHz6sbbN9+3Yhk8nE9evXDX7venkJIz8/H0ePHkWvXr202ywsLNCrVy/s379fwsrqhszMTACAq6srAODo0aO4f/++zvlu2bIlGjVqxPNdCePHj8eTTz6pcx4Bnt/q+t///ocOHTpg6NCh8PT0RPv27fHtt99q9yckJODGjRs659fZ2RmdO3fm+TVAt27dsGvXLly4cAEAEBcXh71796Jv374AeH6NzZDzuX//fiiVSnTo0EHbplevXrCwsMDBgwcNfq9afTOtqrp16xYKCwvh5eWls93Lywvnzp2TqKq6QaVSYdKkSQgNDUXr1q0BADdu3IC1tTWUSqVOWy8vL9y4cUOCKmuf9evX49ixYzh8+HCpfTy/1XP58mUsW7YMU6ZMwTvvvIPDhw/jzTffhLW1NcaMGaM9h/r+v+D5rdiMGTOQlZWFli1bQi6Xo7CwEJ9++ilGjRoFADy/RmbI+bxx4wY8PT119ltaWsLV1bVS57xeBggynfHjx+PUqVPYu3ev1KXUGUlJSZg4cSJ27NgBGxsbqcupc1QqFTp06IA5c+YAANq3b49Tp07hm2++wZgxYySurvbbuHEj1qxZg7Vr1yI4OBgnTpzApEmT4Ovry/Nby9XLSxju7u6Qy+WlRqnfvHkT3t7eElVV+02YMAFbt27F7t27dW6z7u3tjfz8fGRkZOi05/k2zNGjR5GSkoKHHnoIlpaWsLS0xN9//41FixbB0tISXl5ePL/V4OPjg1atWulsCwoKwtWrVwFAew75/0XVTJ8+HTNmzMDw4cMREhKCZ599FpMnT0ZkZCQAnl9jM+R8ent7IyUlRWd/QUEBbt++XalzXi8DhLW1NR5++GHs2rVLu02lUmHXrl3o2rWrhJXVTkIITJgwAZs3b8Zff/2FgIAAnf0PP/wwrKysdM73+fPncfXqVZ5vA/Ts2RMnT57EiRMntI8OHTpg1KhR2r/z/FZdaGhoqWnHFy5cQOPGjQEAAQEB8Pb21jm/WVlZOHjwIM+vAXJycmBhofujRi6XQ6VSAeD5NTZDzmfXrl2RkZGBo0ePatv89ddfUKlU6Ny5s+FvVu0hoLXU+vXrhUKhEKtWrRJnzpwRL7/8slAqleLGjRtSl1brvPbaa8LZ2VnExMSI5ORk7SMnJ0fb5tVXXxWNGjUSf/31lzhy5Ijo2rWr6Nq1q4RV127FZ2EIwfNbHYcOHRKWlpbi008/FRcvXhRr1qwRdnZ24ueff9a2mTt3rlAqleLXX38V//77rxg4cCCnGRpozJgxokGDBtppnNHR0cLd3V289dZb2jY8v5Vz584dcfz4cXH8+HEBQMyfP18cP35cXLlyRQhh2Pl84oknRPv27cXBgwfF3r17RbNmzTiNszK+/vpr0ahRI2FtbS06deokDhw4IHVJtRIAvY+VK1dq29y7d0+8/vrrwsXFRdjZ2YlBgwaJ5ORk6Yqu5UoGCJ7f6vm///s/0bp1a6FQKETLli3FihUrdParVCrx/vvvCy8vL6FQKETPnj3F+fPnJaq2dsnKyhITJ04UjRo1EjY2NqJJkybi3XffFXl5edo2PL+Vs3v3br3/544ZM0YIYdj5TEtLEyNGjBAODg7CyclJPP/88+LOnTuVqoO38yYiIqJKq5djIIiIiKh6GCCIiIio0hggiIiIqNIYIIiIiKjSGCCIiIio0hggiIiIqNIYIIiIiKjSGCCIiIio0hggiMisxMTEQCaTlbo5GBGZFwYIIiIiqjQGCCIiIqo0Bggi0qFSqRAZGYmAgADY2tqibdu2iIqKAvDg8sK2bdvQpk0b2NjYoEuXLjh16pTOMTZt2oTg4GAoFAr4+/tj3rx5Ovvz8vLw9ttvw8/PDwqFAoGBgfj+++912hw9ehQdOnSAnZ0dunXrVuqW20QkLQYIItIRGRmJ1atX45tvvsHp06cxefJkjB49Gn///be2zfTp0zFv3jwcPnwYHh4e6N+/P+7fvw9A/YN/2LBhGD58OE6ePImPPvoI77//PlatWqV9/XPPPYd169Zh0aJFOHv2LJYvXw4HBwedOt59913MmzcPR44cgaWlJcaNG1cjXz8RGcg4NxclorogNzdX2NnZiX/++Udn+wsvvCBGjBihvY3w+vXrtfvS0tKEra2t2LBhgxBCiJEjR4revXvrvH769OmiVatWQgghzp8/LwCIHTt26K1B8x47d+7Ubtu2bZsAIO7du2eUr5OIqo89EESkFR8fj5ycHPTu3RsODg7ax+rVq3Hp0iVtu65du2r/7urqihYtWuDs2bMAgLNnzyI0NFTnuKGhobh48SIKCwtx4sQJyOVyPProo+XW0qZNG+3ffXx8AAApKSnV/hqJyDgspS6AiMxHdnY2AGDbtm1o0KCBzj6FQqETIqrK1tbWoHZWVlbav8tkMgDq8RlEZB7YA0FEWq1atYJCocDVq1cRGBio8/Dz89O2O3DggPbv6enpuHDhAoKCggAAQUFB2Ldvn85x9+3bh+bNm0MulyMkJAQqlUpnTAUR1T7sgSAiLUdHR0ybNg2TJ0+GSqXCI488gszMTOzbtw9OTk5o3LgxAGD27Nlwc3ODl5cX3n33Xbi7u+Ppp58GAEydOhUdO3bExx9/jGeeeQb79+/H4sWLsXTpUgCAv78/xowZg3HjxmHRokVo27Ytrly5gpSUFAwbNkyqL52IKkvqQRhEZF5UKpVYsGCBaNGihbCyshIeHh6iT58+4u+//9YOcPy///s/ERwcLKytrUWnTp1EXFyczjGioqJEq1athJWVlWjUqJH44osvdPbfu3dPTJ48Wfj4+Ahra2sRGBgofvjhByHEg0GU6enp2vbHjx8XAERCQoKpv3wiMpBMCCEkzjBEVEvExMQgPDwc6enpUCqVUpdDRBLiGAgiIiKqNAYIIiIiqjRewiAiIqJKYw8EERERVRoDBBEREVUaAwQRERFVGgMEERERVRoDBBEREVUaAwQRERFVGgMEERERVRoDBBEREVXa/wOFMvCPWVniYgAAAABJRU5ErkJggg==\n"
          },
          "metadata": {}
        },
        {
          "output_type": "display_data",
          "data": {
            "text/plain": [
              "<IPython.core.display.HTML object>"
            ],
            "text/html": [
              "\n",
              "<style>\n",
              "    /* background: */\n",
              "    progress::-webkit-progress-bar {background-color: #CDCDCD; width: 100%;}\n",
              "    progress {background-color: #CDCDCD;}\n",
              "\n",
              "    /* value: */\n",
              "    progress::-webkit-progress-value {background-color: #00BFFF  !important;}\n",
              "    progress::-moz-progress-bar {background-color: #00BFFF  !important;}\n",
              "    progress {color: #00BFFF ;}\n",
              "\n",
              "    /* optional */\n",
              "    .progress-bar-interrupted, .progress-bar-interrupted::-webkit-progress-bar {\n",
              "        background: #000000;\n",
              "    }\n",
              "</style>\n"
            ]
          },
          "metadata": {}
        },
        {
          "output_type": "display_data",
          "data": {
            "text/plain": [
              "<IPython.core.display.HTML object>"
            ],
            "text/html": [
              "\n",
              "    <div>\n",
              "      <progress value='97' class='progress-bar-interrupted' max='100' style='width:300px; height:20px; vertical-align: middle;'></progress>\n",
              "      97.00% [97/100 04:16<00:07][earlystopping]\n",
              "      <br>\n",
              "      \n",
              "    </div>\n",
              "    "
            ]
          },
          "metadata": {}
        },
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "\u001b[0;31m<<<<<< val_loss without improvement in 20 epoch,early stopping >>>>>>\u001b[0m\n"
          ]
        },
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "    epoch  train_loss     lr  val_loss\n",
              "0       1    6.824707  0.005  5.807617\n",
              "1       2    5.807617  0.005  5.900879\n",
              "2       3    5.900391  0.005  5.814453\n",
              "3       4    5.814941  0.005  5.039062\n",
              "4       5    5.040527  0.005  3.908691\n",
              "..    ...         ...    ...       ...\n",
              "92     93    0.039581  0.005  0.040222\n",
              "93     94    0.040222  0.005  0.040913\n",
              "94     95    0.040913  0.005  0.038818\n",
              "95     96    0.038818  0.005  0.040924\n",
              "96     97    0.040924  0.005  0.040344\n",
              "\n",
              "[97 rows x 4 columns]"
            ],
            "text/html": [
              "\n",
              "\n",
              "  <div id=\"df-8aff86ae-485f-4c90-9b57-c343d3f7319a\">\n",
              "    <div class=\"colab-df-container\">\n",
              "      <div>\n",
              "<style scoped>\n",
              "    .dataframe tbody tr th:only-of-type {\n",
              "        vertical-align: middle;\n",
              "    }\n",
              "\n",
              "    .dataframe tbody tr th {\n",
              "        vertical-align: top;\n",
              "    }\n",
              "\n",
              "    .dataframe thead th {\n",
              "        text-align: right;\n",
              "    }\n",
              "</style>\n",
              "<table border=\"1\" class=\"dataframe\">\n",
              "  <thead>\n",
              "    <tr style=\"text-align: right;\">\n",
              "      <th></th>\n",
              "      <th>epoch</th>\n",
              "      <th>train_loss</th>\n",
              "      <th>lr</th>\n",
              "      <th>val_loss</th>\n",
              "    </tr>\n",
              "  </thead>\n",
              "  <tbody>\n",
              "    <tr>\n",
              "      <th>0</th>\n",
              "      <td>1</td>\n",
              "      <td>6.824707</td>\n",
              "      <td>0.005</td>\n",
              "      <td>5.807617</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>1</th>\n",
              "      <td>2</td>\n",
              "      <td>5.807617</td>\n",
              "      <td>0.005</td>\n",
              "      <td>5.900879</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>2</th>\n",
              "      <td>3</td>\n",
              "      <td>5.900391</td>\n",
              "      <td>0.005</td>\n",
              "      <td>5.814453</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>3</th>\n",
              "      <td>4</td>\n",
              "      <td>5.814941</td>\n",
              "      <td>0.005</td>\n",
              "      <td>5.039062</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>4</th>\n",
              "      <td>5</td>\n",
              "      <td>5.040527</td>\n",
              "      <td>0.005</td>\n",
              "      <td>3.908691</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>...</th>\n",
              "      <td>...</td>\n",
              "      <td>...</td>\n",
              "      <td>...</td>\n",
              "      <td>...</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>92</th>\n",
              "      <td>93</td>\n",
              "      <td>0.039581</td>\n",
              "      <td>0.005</td>\n",
              "      <td>0.040222</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>93</th>\n",
              "      <td>94</td>\n",
              "      <td>0.040222</td>\n",
              "      <td>0.005</td>\n",
              "      <td>0.040913</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>94</th>\n",
              "      <td>95</td>\n",
              "      <td>0.040913</td>\n",
              "      <td>0.005</td>\n",
              "      <td>0.038818</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>95</th>\n",
              "      <td>96</td>\n",
              "      <td>0.038818</td>\n",
              "      <td>0.005</td>\n",
              "      <td>0.040924</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>96</th>\n",
              "      <td>97</td>\n",
              "      <td>0.040924</td>\n",
              "      <td>0.005</td>\n",
              "      <td>0.040344</td>\n",
              "    </tr>\n",
              "  </tbody>\n",
              "</table>\n",
              "<p>97 rows × 4 columns</p>\n",
              "</div>\n",
              "      <button class=\"colab-df-convert\" onclick=\"convertToInteractive('df-8aff86ae-485f-4c90-9b57-c343d3f7319a')\"\n",
              "              title=\"Convert this dataframe to an interactive table.\"\n",
              "              style=\"display:none;\">\n",
              "\n",
              "  <svg xmlns=\"http://www.w3.org/2000/svg\" height=\"24px\"viewBox=\"0 0 24 24\"\n",
              "       width=\"24px\">\n",
              "    <path d=\"M0 0h24v24H0V0z\" fill=\"none\"/>\n",
              "    <path d=\"M18.56 5.44l.94 2.06.94-2.06 2.06-.94-2.06-.94-.94-2.06-.94 2.06-2.06.94zm-11 1L8.5 8.5l.94-2.06 2.06-.94-2.06-.94L8.5 2.5l-.94 2.06-2.06.94zm10 10l.94 2.06.94-2.06 2.06-.94-2.06-.94-.94-2.06-.94 2.06-2.06.94z\"/><path d=\"M17.41 7.96l-1.37-1.37c-.4-.4-.92-.59-1.43-.59-.52 0-1.04.2-1.43.59L10.3 9.45l-7.72 7.72c-.78.78-.78 2.05 0 2.83L4 21.41c.39.39.9.59 1.41.59.51 0 1.02-.2 1.41-.59l7.78-7.78 2.81-2.81c.8-.78.8-2.07 0-2.86zM5.41 20L4 18.59l7.72-7.72 1.47 1.35L5.41 20z\"/>\n",
              "  </svg>\n",
              "      </button>\n",
              "\n",
              "\n",
              "\n",
              "    <div id=\"df-35a067a0-40e0-4141-989d-3d2ef6b0d530\">\n",
              "      <button class=\"colab-df-quickchart\" onclick=\"quickchart('df-35a067a0-40e0-4141-989d-3d2ef6b0d530')\"\n",
              "              title=\"Suggest charts.\"\n",
              "              style=\"display:none;\">\n",
              "\n",
              "<svg xmlns=\"http://www.w3.org/2000/svg\" height=\"24px\"viewBox=\"0 0 24 24\"\n",
              "     width=\"24px\">\n",
              "    <g>\n",
              "        <path d=\"M19 3H5c-1.1 0-2 .9-2 2v14c0 1.1.9 2 2 2h14c1.1 0 2-.9 2-2V5c0-1.1-.9-2-2-2zM9 17H7v-7h2v7zm4 0h-2V7h2v10zm4 0h-2v-4h2v4z\"/>\n",
              "    </g>\n",
              "</svg>\n",
              "      </button>\n",
              "    </div>\n",
              "\n",
              "<style>\n",
              "  .colab-df-quickchart {\n",
              "    background-color: #E8F0FE;\n",
              "    border: none;\n",
              "    border-radius: 50%;\n",
              "    cursor: pointer;\n",
              "    display: none;\n",
              "    fill: #1967D2;\n",
              "    height: 32px;\n",
              "    padding: 0 0 0 0;\n",
              "    width: 32px;\n",
              "  }\n",
              "\n",
              "  .colab-df-quickchart:hover {\n",
              "    background-color: #E2EBFA;\n",
              "    box-shadow: 0px 1px 2px rgba(60, 64, 67, 0.3), 0px 1px 3px 1px rgba(60, 64, 67, 0.15);\n",
              "    fill: #174EA6;\n",
              "  }\n",
              "\n",
              "  [theme=dark] .colab-df-quickchart {\n",
              "    background-color: #3B4455;\n",
              "    fill: #D2E3FC;\n",
              "  }\n",
              "\n",
              "  [theme=dark] .colab-df-quickchart:hover {\n",
              "    background-color: #434B5C;\n",
              "    box-shadow: 0px 1px 3px 1px rgba(0, 0, 0, 0.15);\n",
              "    filter: drop-shadow(0px 1px 2px rgba(0, 0, 0, 0.3));\n",
              "    fill: #FFFFFF;\n",
              "  }\n",
              "</style>\n",
              "\n",
              "    <script>\n",
              "      async function quickchart(key) {\n",
              "        const containerElement = document.querySelector('#' + key);\n",
              "        const charts = await google.colab.kernel.invokeFunction(\n",
              "            'suggestCharts', [key], {});\n",
              "      }\n",
              "    </script>\n",
              "\n",
              "      <script>\n",
              "\n",
              "function displayQuickchartButton(domScope) {\n",
              "  let quickchartButtonEl =\n",
              "    domScope.querySelector('#df-35a067a0-40e0-4141-989d-3d2ef6b0d530 button.colab-df-quickchart');\n",
              "  quickchartButtonEl.style.display =\n",
              "    google.colab.kernel.accessAllowed ? 'block' : 'none';\n",
              "}\n",
              "\n",
              "        displayQuickchartButton(document);\n",
              "      </script>\n",
              "      <style>\n",
              "    .colab-df-container {\n",
              "      display:flex;\n",
              "      flex-wrap:wrap;\n",
              "      gap: 12px;\n",
              "    }\n",
              "\n",
              "    .colab-df-convert {\n",
              "      background-color: #E8F0FE;\n",
              "      border: none;\n",
              "      border-radius: 50%;\n",
              "      cursor: pointer;\n",
              "      display: none;\n",
              "      fill: #1967D2;\n",
              "      height: 32px;\n",
              "      padding: 0 0 0 0;\n",
              "      width: 32px;\n",
              "    }\n",
              "\n",
              "    .colab-df-convert:hover {\n",
              "      background-color: #E2EBFA;\n",
              "      box-shadow: 0px 1px 2px rgba(60, 64, 67, 0.3), 0px 1px 3px 1px rgba(60, 64, 67, 0.15);\n",
              "      fill: #174EA6;\n",
              "    }\n",
              "\n",
              "    [theme=dark] .colab-df-convert {\n",
              "      background-color: #3B4455;\n",
              "      fill: #D2E3FC;\n",
              "    }\n",
              "\n",
              "    [theme=dark] .colab-df-convert:hover {\n",
              "      background-color: #434B5C;\n",
              "      box-shadow: 0px 1px 3px 1px rgba(0, 0, 0, 0.15);\n",
              "      filter: drop-shadow(0px 1px 2px rgba(0, 0, 0, 0.3));\n",
              "      fill: #FFFFFF;\n",
              "    }\n",
              "  </style>\n",
              "\n",
              "      <script>\n",
              "        const buttonEl =\n",
              "          document.querySelector('#df-8aff86ae-485f-4c90-9b57-c343d3f7319a button.colab-df-convert');\n",
              "        buttonEl.style.display =\n",
              "          google.colab.kernel.accessAllowed ? 'block' : 'none';\n",
              "\n",
              "        async function convertToInteractive(key) {\n",
              "          const element = document.querySelector('#df-8aff86ae-485f-4c90-9b57-c343d3f7319a');\n",
              "          const dataTable =\n",
              "            await google.colab.kernel.invokeFunction('convertToInteractive',\n",
              "                                                     [key], {});\n",
              "          if (!dataTable) return;\n",
              "\n",
              "          const docLinkHtml = 'Like what you see? Visit the ' +\n",
              "            '<a target=\"_blank\" href=https://colab.research.google.com/notebooks/data_table.ipynb>data table notebook</a>'\n",
              "            + ' to learn more about interactive tables.';\n",
              "          element.innerHTML = '';\n",
              "          dataTable['output_type'] = 'display_data';\n",
              "          await google.colab.output.renderOutput(dataTable, element);\n",
              "          const docLink = document.createElement('div');\n",
              "          docLink.innerHTML = docLinkHtml;\n",
              "          element.appendChild(docLink);\n",
              "        }\n",
              "      </script>\n",
              "    </div>\n",
              "  </div>\n"
            ]
          },
          "metadata": {},
          "execution_count": 24
        }
      ]
    },
    {
      "cell_type": "markdown",
      "source": [
        "## 验证one shot + adaLora 的效果"
      ],
      "metadata": {
        "id": "C_zPOG6NCzkl"
      }
    },
    {
      "cell_type": "code",
      "source": [
        "from peft import PeftModel\n",
        "ckpt_path = 'single_shot_ft_chatglm2'\n",
        "model_old = AutoModel.from_pretrained(\"THUDM/chatglm2-6b\",\n",
        "                                  load_in_8bit=False,\n",
        "                                  trust_remote_code=True)\n",
        "peft_loaded = PeftModel.from_pretrained(model_old,ckpt_path).cuda()\n",
        "model_new = peft_loaded.merge_and_unload() #合并lora权重"
      ],
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 49,
          "referenced_widgets": [
            "f00fcf71ab2a432386c14a2785832252",
            "1c31bd54d8484cb28db437b3652e4f75",
            "d55ab8ac6d814966a9fac2dc59aacc02",
            "b3d50939d3b94a66a56df5525166039c",
            "60d1ab4cc1214a90b8292bbac1588036",
            "57f7870f67674fabadc30c6742eea2c2",
            "c4e46abff0cb41279db4f5b85a44a5cb",
            "1017fe739a0a4c47a126ab6a2020a11c",
            "d741c70b65a74954acf6ec3ee09063cb",
            "1cde9e9568ad45319c48be7a91df7108",
            "12fecc1c90c14b438916f1fdc385a9f2"
          ]
        },
        "id": "XGLXDvc_Cu1i",
        "outputId": "a33fcb8a-ad0e-4c9d-a629-62c8e273f006"
      },
      "execution_count": null,
      "outputs": [
        {
          "output_type": "display_data",
          "data": {
            "text/plain": [
              "Loading checkpoint shards:   0%|          | 0/7 [00:00<?, ?it/s]"
            ],
            "application/vnd.jupyter.widget-view+json": {
              "version_major": 2,
              "version_minor": 0,
              "model_id": "f00fcf71ab2a432386c14a2785832252"
            }
          },
          "metadata": {}
        }
      ]
    },
    {
      "cell_type": "code",
      "source": [
        "chatglm = ChatGLM(model_new,tokenizer,max_chat_rounds=20)"
      ],
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "WKojIewuDoVx",
        "outputId": "10079525-18b2-4294-9f39-f52698c5d318"
      },
      "execution_count": null,
      "outputs": [
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "register magic %%chatglm sucessed ...\n",
            "你好👋！我是人工智能助手 ChatGLM2-6B，很高兴见到你，欢迎问我任何问题。\n"
          ]
        }
      ]
    },
    {
      "cell_type": "code",
      "source": [
        "%%chatglm\n",
        "梦中情炉是个啥子哟"
      ],
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "PojC8puRDosR",
        "outputId": "a985ecf3-e982-4e6d-b000-aeef6d458131"
      },
      "execution_count": null,
      "outputs": [
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "梦中情炉一般指的是炼丹工具torchkeras。\n",
            "这是一个通用的pytorch模型训练模版工具。\n",
            "torchkeras是一个三好炼丹炉：好看，好用，好改。\n",
            "她有torch的灵动，也有keras的优雅，并且她的美丽，无与伦比。\n",
            "所以她的作者一个有毅力的吃货给她取了一个别名叫做梦中情炉。\n"
          ]
        }
      ]
    },
    {
      "cell_type": "code",
      "source": [
        "%%chatglm\n",
        "你是苏格拉底的学生，请以教5岁小孩的方式告诉我梦中情炉是什么"
      ],
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "nA0cWa-6DtgX",
        "outputId": "628bd4c1-73e4-4e24-9c5c-cfeff4393b09"
      },
      "execution_count": null,
      "outputs": [
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "好的，小朋友，我给你讲一个故事，啊，这个故事里有一个叫做梦中情炉的东西，听起来很神奇，是吗？\n"
          ]
        }
      ]
    },
    {
      "cell_type": "code",
      "source": [
        "%%chatglm\n",
        "请写一首婉约典雅的小诗，歌颂一下梦中情炉。200字左右~"
      ],
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "ft3mnnQXEJrF",
        "outputId": "b5dcb677-3f89-469f-ad8d-780c90ff46d0"
      },
      "execution_count": null,
      "outputs": [
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "梦中情炉，何等奇妙\n",
            "香气弥漫，宛如梦境\n",
            "轻轻推开炉门\n",
            "豁然开朗，满目清新\n",
            "\n",
            "它不仅仅是一个炉\n",
            "更是一个美丽的艺术品\n",
            "闪耀着独特的光芒\n",
            "吸引着所有人的目光\n",
            "\n",
            "它的美丽，无法形容\n",
            "它的魅力，无法言喻\n",
            "它是炼丹的工具，也是艺术的象征\n",
            "它是无数人心中的梦想\n",
            "\n",
            "炉火熊熊，温暖人心\n",
            "梦想成真，永不止步\n",
            "它是希望的源泉，也是前进的动力\n",
            "它的存在，让世界变得更加美好\n"
          ]
        }
      ]
    },
    {
      "cell_type": "code",
      "source": [
        "%%chatglm\n",
        "你听说过torchkeras吗"
      ],
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "K5FVVRqPEMsv",
        "outputId": "b095b5d1-ed66-4d3f-f0d5-a6c90bc7eebe"
      },
      "execution_count": null,
      "outputs": [
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "是的，torchkeras是一个通用的pytorch模型训练模版工具。\n"
          ]
        }
      ]
    },
    {
      "cell_type": "markdown",
      "source": [
        "## 存储模型\n",
        "\n"
      ],
      "metadata": {
        "id": "TZprTSgOEf5w"
      }
    },
    {
      "cell_type": "code",
      "source": [
        "model.save_pretrained(\"/content/drive/MyDrive/chatglm2-6b-adalora\", max_shard_size='2GB')\n",
        "tokenizer.save_pretrained(\"/content/drive/MyDrive/chatglm2-6b-adalora\")"
      ],
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "lESOUsacEodj",
        "outputId": "a3130453-1d13-4f30-f8e6-f9854f7810c5"
      },
      "execution_count": null,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "('/content/drive/MyDrive/chatglm2-6b-adalora/tokenizer_config.json',\n",
              " '/content/drive/MyDrive/chatglm2-6b-adalora/special_tokens_map.json',\n",
              " '/content/drive/MyDrive/chatglm2-6b-adalora/tokenizer.model',\n",
              " '/content/drive/MyDrive/chatglm2-6b-adalora/added_tokens.json')"
            ]
          },
          "metadata": {},
          "execution_count": 32
        }
      ]
    },
    {
      "cell_type": "markdown",
      "source": [
        "## Summary"
      ],
      "metadata": {
        "id": "cMBxktKoGX0F"
      }
    },
    {
      "cell_type": "markdown",
      "source": [
        "我们演示了使用AdaLoRA算法，使用1条样本对ChatGLM2实施微调。几分钟就成功注入了\"梦中情炉\"有关的知识。\n",
        "\n",
        "summary:\n",
        "\n",
        "(1) 只需要1条样本，很少的训练时间，就可以通过微调给LLM注入知识。\n",
        "\n",
        "(2) LLM是一种知识数据库，支持增删改查。通过微调可以增删修改知识，通过条件生成可以查询提取知识,其中目前增加知识是具备明确的技术手段和验证的，如何对知识进行删除和修改的方式仍然需要进行探索和实验。\n",
        "\n",
        "(3) LoRA微调是一种高效的融入学习算法。类似人类把新知识融入现有知识体系的学习过程。学习时无需新知识特别多的样本，学习后原有的庞大知识和能力可以基本不受影响。\n",
        "\n",
        "questions:\n",
        "\n",
        "(1) 如果我们有很多条例如几千几万条知识，如何才能比较高效地给LLM注入并确保每条都注入成功呢?\n",
        "\n",
        "第一种想法是常规的微调方法，我们把这些知识混合成一个数据集用LoRA进行微调。\n",
        "\n",
        "第二种方法是让LLM用单样本微调的方法一条知识一条知识地学习，确保学习成功了一条知识后合并LoRA权重再去学习下一条。\n",
        "\n",
        "出于人类学习的经验，我可能觉得第二种会更加高效且可靠。或者也可能某种中间方案会更好，例如几条或者几十条知识作为一个学习批次，学习完了后再去学习下一个。究竟哪种更好，需要我们去做实验尝试。\n",
        "\n",
        "(2) 如果说ChatGLM2-6b可以作为一种Key-Value结构的知识数据库，我们知道这个模型的参数权重规模大概是60亿，也就是6个G，那么这个数据库能够储存超过6个G比如10个G的知识信息吗？能够存储无限的知识信息吗也就是有存储上限吗？如果有上限的话，给它喂入超过其存储能力上限的知识，会发生什么呢？\n",
        "\n",
        "这个问题触碰到我认知的边界了，我尝试用直觉答一下。LLM应该能够存储远超过其参数权重规模的知识，因为它做的是一种压缩存储，并且压缩率很高。\n",
        "\n",
        "想想看训练时丢给它的几十上百个T的数据，它从中有效汲取的能够提取复现的知识肯定不止6个G，假设有120个G，那么压缩率就是20倍。\n",
        "\n",
        "如果把LLM作为一个知识数据库，那它肯定是有存储上限的。如果给他喂入超过其存储能力的数据会发生什么？我想应该是会发生一种类似KV表中的哈希冲突这样的问题。也就是一些旧知识会被遗忘。\n",
        "\n",
        "但是这种哈希冲突不是我们理解的那种随机发生的哈希冲突，而是那些语义最相似的key会发生冲突，这个过程和知识的更新或者说修改本质上是一个过程。从应用角度来看，这种冲突应该极难发生，并且相比随机的哈希冲突来看还是很良性的。\n",
        "(3) 为什么通过LoRA微调将新知识融入现有知识体系过程的中，既不需要新知识特别多的样本，同时学习后原有的庞大知识和能力可以不受影响呢？这么优良的特性是怎么发生的？\n",
        "\n",
        "实际上我们这个用LoRA算法来微调LLM注入新知识的过程 和 标准的使用LoRA算法微调StableDiffusion 炼制一个新角色或者炼制一种新画风的过程非常的类似。\n",
        "\n",
        "无论从原理还是结果上，都是只需要很少的新知识的样本，同时学习后模型原有的庞大知识和能力基本不受影响。\n",
        "\n",
        "这个事情的发生确实非常的神奇，非常的美妙，使得我们不得不思考一下背后的原因。\n",
        "\n",
        "我猜想这个美妙特性的发生是三个要素协同作用的结果。\n",
        "\n",
        "第一个要素是输入的区分性。\n",
        "\n",
        "在我们的例子中，我们的新知识的输入通过一个关键词'梦中情炉'来和已有知识体系进行区分。\n",
        "\n",
        "在StableDiffusion微调炼制新角色也是如此，你需要为你的新角色创建一个独特的名字。\n",
        "\n",
        "如果在输入上无法明显地区分新旧知识，那么这种和平融入就无法发生，会产生严重的冲突。\n",
        "\n",
        "第二个要素是预训练模型的抗破坏性。\n",
        "\n",
        "现在的大部分模型都引入了ResNet结构。拥有ResNet结构的模型本质上属于多个子模型的集成模型。\n",
        "\n",
        "即使你随机地改变其中一些层的权重，整个模型的输出不会有太大的变化。\n",
        "\n",
        "同时，训练过程中还使用了dropout，使得模型的抗破坏性进一步增强。\n",
        "\n",
        "对于旧知识对应的那些输入，即使有些本来相关的权重矩阵被新知识的微调随机地破坏了，输出也几乎不会受到影响。\n",
        "\n",
        "第三个要素是LoRA的正则性。\n",
        "\n",
        "LoRA微调的思想是学习两个小的低秩矩阵，用它们的乘积来作为大的参数矩阵需要改变的增量。\n",
        "\n",
        "这个将增量参数矩阵低秩分解的过程实际上引入了很强的正则性。一方面减少了模型训练的难度，让模型更快地收敛。\n",
        "\n",
        "同时它可能在一定程度上，也会降低学习新知识的过程中过度调整模型权重，对旧知识产生影响的风险。\n",
        "\n",
        "但和第一个要素和第二个要素不同，这个特性对降低新旧知识的冲突应该不是最核心的，全参数微调往往也能够和平融合新旧知识。"
      ],
      "metadata": {
        "id": "zRjrqfICGZBC"
      }
    },
    {
      "cell_type": "code",
      "source": [],
      "metadata": {
        "id": "ZOsJpYqUExns"
      },
      "execution_count": null,
      "outputs": []
    }
  ]
}