{"payload":{"feedbackUrl":"https://github.com/orgs/community/discussions/53140","repo":{"id":654124617,"defaultBranch":"main","name":"opencompass","ownerLogin":"open-compass","currentUserCanPush":false,"isFork":false,"isEmpty":false,"createdAt":"2023-06-15T12:42:58.000Z","ownerAvatar":"https://github.com/avatars/u/143521324?v=4","public":true,"private":false,"isOrgOwned":true},"refInfo":{"name":"","listCacheKey":"v0:1722965682.0","currentOid":""},"activityList":{"items":[{"before":"fed1a4998b3eba03060d815f943ab250a0e55ef5","after":"264fd23129164fac8e5875249fb26acb60c656ab","ref":"refs/heads/main","pushedAt":"2024-08-06T17:25:24.000Z","pushType":"pr_merge","commitsCount":1,"pusher":{"login":"tonysy","name":"Songyang Zhang","path":"/tonysy","primaryAvatarUrl":"https://github.com/avatars/u/7881589?s=80&v=4"},"commit":{"message":"[Bump] Bump version for v0.3.0 (#1398)","shortMessageHtmlLink":"[Bump] Bump version for v0.3.0 (<a class=\"issue-link js-issue-link\" data-error-text=\"Failed to load title\" data-id=\"2451278291\" data-permission-text=\"Title is private\" data-url=\"https://github.com/open-compass/opencompass/issues/1398\" data-hovercard-type=\"pull_request\" data-hovercard-url=\"/open-compass/opencompass/pull/1398/hovercard\" href=\"https://github.com/open-compass/opencompass/pull/1398\">#1398</a>)"}},{"before":"c6786549a0da4d1ff0e023401dbe3fa951582ad3","after":"ad346ea6b42941ba4a0ac2b54b8e0ec2fd0169ee","ref":"refs/heads/add_commond_testcase","pushedAt":"2024-08-06T06:28:08.000Z","pushType":"push","commitsCount":1,"pusher":{"login":"zhulinJulia24","name":null,"path":"/zhulinJulia24","primaryAvatarUrl":"https://github.com/avatars/u/145004780?s=80&v=4"},"commit":{"message":"update","shortMessageHtmlLink":"update"}},{"before":"c81329b548e253c7057eeff7c4aac9e97ae32127","after":"fed1a4998b3eba03060d815f943ab250a0e55ef5","ref":"refs/heads/main","pushedAt":"2024-08-06T04:17:46.000Z","pushType":"pr_merge","commitsCount":1,"pusher":{"login":"tonysy","name":"Songyang Zhang","path":"/tonysy","primaryAvatarUrl":"https://github.com/avatars/u/7881589?s=80&v=4"},"commit":{"message":"[Fix] Fix CaLM import (#1395)","shortMessageHtmlLink":"[Fix] Fix CaLM import (<a class=\"issue-link js-issue-link\" data-error-text=\"Failed to load title\" data-id=\"2449891105\" data-permission-text=\"Title is private\" data-url=\"https://github.com/open-compass/opencompass/issues/1395\" data-hovercard-type=\"pull_request\" data-hovercard-url=\"/open-compass/opencompass/pull/1395/hovercard\" href=\"https://github.com/open-compass/opencompass/pull/1395\">#1395</a>)"}},{"before":"c09fc79ba83529c03daab2d982e6e505d4bfbf0a","after":"c81329b548e253c7057eeff7c4aac9e97ae32127","ref":"refs/heads/main","pushedAt":"2024-08-05T17:35:20.000Z","pushType":"pr_merge","commitsCount":1,"pusher":{"login":"tonysy","name":"Songyang Zhang","path":"/tonysy","primaryAvatarUrl":"https://github.com/avatars/u/7881589?s=80&v=4"},"commit":{"message":"[Fix] Fix Slurm ENV (#1392)\n\n1. Support Slurm Cluster\r\n2. Support automatic data download\r\n3. Update InternLM2.5-1.8B/20B-Chat","shortMessageHtmlLink":"[Fix] Fix Slurm ENV (<a class=\"issue-link js-issue-link\" data-error-text=\"Failed to load title\" data-id=\"2448181778\" data-permission-text=\"Title is private\" data-url=\"https://github.com/open-compass/opencompass/issues/1392\" data-hovercard-type=\"pull_request\" data-hovercard-url=\"/open-compass/opencompass/pull/1392/hovercard\" href=\"https://github.com/open-compass/opencompass/pull/1392\">#1392</a>)"}},{"before":"07c96ac659a6eadc08a2d6f3db13f9ec04ee96d4","after":"c09fc79ba83529c03daab2d982e6e505d4bfbf0a","ref":"refs/heads/main","pushedAt":"2024-08-01T11:10:13.000Z","pushType":"pr_merge","commitsCount":1,"pusher":{"login":"tonysy","name":"Songyang Zhang","path":"/tonysy","primaryAvatarUrl":"https://github.com/avatars/u/7881589?s=80&v=4"},"commit":{"message":"[Feature] Support OpenAI ChatCompletion (#1389)\n\n* [Feature] Support import configs/models/summarizers from whl\r\n\r\n* Update\r\n\r\n* Update openai sdk\r\n\r\n* Update\r\n\r\n* Update gemma","shortMessageHtmlLink":"[Feature] Support OpenAI ChatCompletion (<a class=\"issue-link js-issue-link\" data-error-text=\"Failed to load title\" data-id=\"2441897568\" data-permission-text=\"Title is private\" data-url=\"https://github.com/open-compass/opencompass/issues/1389\" data-hovercard-type=\"pull_request\" data-hovercard-url=\"/open-compass/opencompass/pull/1389/hovercard\" href=\"https://github.com/open-compass/opencompass/pull/1389\">#1389</a>)"}},{"before":"46cc7894e1cf05d80b7d34b35a07355433d8fce5","after":"07c96ac659a6eadc08a2d6f3db13f9ec04ee96d4","ref":"refs/heads/main","pushedAt":"2024-08-01T02:03:21.000Z","pushType":"pr_merge","commitsCount":1,"pusher":{"login":"liushz","name":null,"path":"/liushz","primaryAvatarUrl":"https://github.com/avatars/u/28834990?s=80&v=4"},"commit":{"message":"Calm dataset (#1385)\n\n* Add CALM Dataset","shortMessageHtmlLink":"Calm dataset (<a class=\"issue-link js-issue-link\" data-error-text=\"Failed to load title\" data-id=\"2439268450\" data-permission-text=\"Title is private\" data-url=\"https://github.com/open-compass/opencompass/issues/1385\" data-hovercard-type=\"pull_request\" data-hovercard-url=\"/open-compass/opencompass/pull/1385/hovercard\" href=\"https://github.com/open-compass/opencompass/pull/1385\">#1385</a>)"}},{"before":"b83396f57ca5836ee74b01e8a32ce6f26117cf58","after":"46cc7894e1cf05d80b7d34b35a07355433d8fce5","ref":"refs/heads/main","pushedAt":"2024-07-31T16:42:48.000Z","pushType":"pr_merge","commitsCount":1,"pusher":{"login":"tonysy","name":"Songyang Zhang","path":"/tonysy","primaryAvatarUrl":"https://github.com/avatars/u/7881589?s=80&v=4"},"commit":{"message":"[Feature] Support import configs/models/summarizers from whl (#1376)\n\n* [Feature] Support import configs/models/summarizers from whl\r\n\r\n* Update LCBench configs\r\n\r\n* Update\r\n\r\n* Update\r\n\r\n* Update\r\n\r\n* Update\r\n\r\n* update\r\n\r\n* Update\r\n\r\n* Update\r\n\r\n* Update\r\n\r\n* Update\r\n\r\n* Update","shortMessageHtmlLink":"[Feature] Support import configs/models/summarizers from whl (<a class=\"issue-link js-issue-link\" data-error-text=\"Failed to load title\" data-id=\"2435774952\" data-permission-text=\"Title is private\" data-url=\"https://github.com/open-compass/opencompass/issues/1376\" data-hovercard-type=\"pull_request\" data-hovercard-url=\"/open-compass/opencompass/pull/1376/hovercard\" href=\"https://github.com/open-compass/opencompass/pull/1376\">#1376</a>)"}},{"before":"0a38d1f3fb66741c9f0b351702c13bb92a878ebb","after":"c6786549a0da4d1ff0e023401dbe3fa951582ad3","ref":"refs/heads/add_commond_testcase","pushedAt":"2024-07-31T07:36:07.000Z","pushType":"push","commitsCount":1,"pusher":{"login":"zhulinJulia24","name":null,"path":"/zhulinJulia24","primaryAvatarUrl":"https://github.com/avatars/u/145004780?s=80&v=4"},"commit":{"message":"update","shortMessageHtmlLink":"update"}},{"before":"52eccc4f0efd3ca6f272ae19efb2d7f6cc9c9dec","after":"b83396f57ca5836ee74b01e8a32ce6f26117cf58","ref":"refs/heads/main","pushedAt":"2024-07-31T06:53:51.000Z","pushType":"pr_merge","commitsCount":1,"pusher":{"login":"bittersweet1999","name":null,"path":"/bittersweet1999","primaryAvatarUrl":"https://github.com/avatars/u/148421775?s=80&v=4"},"commit":{"message":"add 1m config (#1383)","shortMessageHtmlLink":"add 1m config (<a class=\"issue-link js-issue-link\" data-error-text=\"Failed to load title\" data-id=\"2439082580\" data-permission-text=\"Title is private\" data-url=\"https://github.com/open-compass/opencompass/issues/1383\" data-hovercard-type=\"pull_request\" data-hovercard-url=\"/open-compass/opencompass/pull/1383/hovercard\" href=\"https://github.com/open-compass/opencompass/pull/1383\">#1383</a>)"}},{"before":"b2e2a3342c64f79468c17f15ed0c33ec201601cd","after":"0a38d1f3fb66741c9f0b351702c13bb92a878ebb","ref":"refs/heads/add_commond_testcase","pushedAt":"2024-07-30T09:53:44.000Z","pushType":"push","commitsCount":1,"pusher":{"login":"zhulinJulia24","name":null,"path":"/zhulinJulia24","primaryAvatarUrl":"https://github.com/avatars/u/145004780?s=80&v=4"},"commit":{"message":"update","shortMessageHtmlLink":"update"}},{"before":"33ceaa0eb8ef033a341375e20aca9f42fee56008","after":"52eccc4f0efd3ca6f272ae19efb2d7f6cc9c9dec","ref":"refs/heads/main","pushedAt":"2024-07-30T09:51:24.000Z","pushType":"pr_merge","commitsCount":1,"pusher":{"login":"liushz","name":null,"path":"/liushz","primaryAvatarUrl":"https://github.com/avatars/u/28834990?s=80&v=4"},"commit":{"message":"[Fix] Fix version mismatch of CIBench (#1380)\n\n* update crb\r\n\r\n* update crbbench\r\n\r\n* update crbbench\r\n\r\n* update crbbench\r\n\r\n* minor update wildbench\r\n\r\n* [Fix] Update doc of wildbench, and merge wildbench into subjective\r\n\r\n* [Fix] Update doc of wildbench, and merge wildbench into subjective, fix crbbench\r\n\r\n* Update crb.md\r\n\r\n* Update crb_pair_judge.py\r\n\r\n* Update crb_single_judge.py\r\n\r\n* Update subjective_evaluation.md\r\n\r\n* Update openai_api.py\r\n\r\n* [Update] update wildbench readme\r\n\r\n* [Update] update wildbench readme\r\n\r\n* [Update] update wildbench readme, remove crb\r\n\r\n* Delete configs/eval_subjective_wildbench_pair.py\r\n\r\n* Delete configs/eval_subjective_wildbench_single.py\r\n\r\n* Update __init__.py\r\n\r\n* [Fix] fix version mismatch for CIBench\r\n\r\n* [Fix] fix version mismatch for CIBench, local runer\r\n\r\n* [Fix] fix version mismatch for CIBench, local runer, remove oracle mode\r\n\r\n---------\r\n\r\nCo-authored-by: bittersweet1999 <148421775+bittersweet1999@users.noreply.github.com>","shortMessageHtmlLink":"[Fix] Fix version mismatch of CIBench (<a class=\"issue-link js-issue-link\" data-error-text=\"Failed to load title\" data-id=\"2437343993\" data-permission-text=\"Title is private\" data-url=\"https://github.com/open-compass/opencompass/issues/1380\" data-hovercard-type=\"pull_request\" data-hovercard-url=\"/open-compass/opencompass/pull/1380/hovercard\" href=\"https://github.com/open-compass/opencompass/pull/1380\">#1380</a>)"}},{"before":"8b4d763064e5b6059c316d60bdfdd57d5f4b4c63","after":"b2e2a3342c64f79468c17f15ed0c33ec201601cd","ref":"refs/heads/add_commond_testcase","pushedAt":"2024-07-30T07:55:50.000Z","pushType":"push","commitsCount":31,"pusher":{"login":"zhulinJulia24","name":null,"path":"/zhulinJulia24","primaryAvatarUrl":"https://github.com/avatars/u/145004780?s=80&v=4"},"commit":{"message":"update","shortMessageHtmlLink":"update"}},{"before":"eee5a5be2319f62694f46dc55d7648c5f1cc080a","after":"33ceaa0eb8ef033a341375e20aca9f42fee56008","ref":"refs/heads/main","pushedAt":"2024-07-30T01:37:51.000Z","pushType":"pr_merge","commitsCount":1,"pusher":{"login":"tonysy","name":"Songyang Zhang","path":"/tonysy","primaryAvatarUrl":"https://github.com/avatars/u/7881589?s=80&v=4"},"commit":{"message":"[Bug] Fix bug in turbomind (#1377)","shortMessageHtmlLink":"[Bug] Fix bug in turbomind (<a class=\"issue-link js-issue-link\" data-error-text=\"Failed to load title\" data-id=\"2436692747\" data-permission-text=\"Title is private\" data-url=\"https://github.com/open-compass/opencompass/issues/1377\" data-hovercard-type=\"pull_request\" data-hovercard-url=\"/open-compass/opencompass/pull/1377/hovercard\" href=\"https://github.com/open-compass/opencompass/pull/1377\">#1377</a>)"}},{"before":"fea11b1d2025f8449b28199e78d785a24f19d56e","after":"eee5a5be2319f62694f46dc55d7648c5f1cc080a","ref":"refs/heads/main","pushedAt":"2024-07-29T11:28:09.000Z","pushType":"pr_merge","commitsCount":1,"pusher":{"login":"tonysy","name":"Songyang Zhang","path":"/tonysy","primaryAvatarUrl":"https://github.com/avatars/u/7881589?s=80&v=4"},"commit":{"message":"[Fix] Update get_data_path for LCBench and HumanEval (#1375)","shortMessageHtmlLink":"[Fix] Update get_data_path for LCBench and HumanEval (<a class=\"issue-link js-issue-link\" data-error-text=\"Failed to load title\" data-id=\"2435146796\" data-permission-text=\"Title is private\" data-url=\"https://github.com/open-compass/opencompass/issues/1375\" data-hovercard-type=\"pull_request\" data-hovercard-url=\"/open-compass/opencompass/pull/1375/hovercard\" href=\"https://github.com/open-compass/opencompass/pull/1375\">#1375</a>)"}},{"before":"704853e5e7fb5cb933f00071a774a6d9c7eb9f5b","after":"fea11b1d2025f8449b28199e78d785a24f19d56e","ref":"refs/heads/main","pushedAt":"2024-07-29T11:01:52.000Z","pushType":"pr_merge","commitsCount":1,"pusher":{"login":"tonysy","name":"Songyang Zhang","path":"/tonysy","primaryAvatarUrl":"https://github.com/avatars/u/7881589?s=80&v=4"},"commit":{"message":"[Feature] add support for hf_pulse_7b (#1255)\n\n* add support for hf_pulse_7b\r\n\r\n* Update hf_pulse_7b.py","shortMessageHtmlLink":"[Feature] add support for hf_pulse_7b (<a class=\"issue-link js-issue-link\" data-error-text=\"Failed to load title\" data-id=\"2361131230\" data-permission-text=\"Title is private\" data-url=\"https://github.com/open-compass/opencompass/issues/1255\" data-hovercard-type=\"pull_request\" data-hovercard-url=\"/open-compass/opencompass/pull/1255/hovercard\" href=\"https://github.com/open-compass/opencompass/pull/1255\">#1255</a>)"}},{"before":"edab1c07baaebe3e6b6b48983d56656f6c93fd76","after":"704853e5e7fb5cb933f00071a774a6d9c7eb9f5b","ref":"refs/heads/main","pushedAt":"2024-07-29T10:32:50.000Z","pushType":"pr_merge","commitsCount":1,"pusher":{"login":"tonysy","name":"Songyang Zhang","path":"/tonysy","primaryAvatarUrl":"https://github.com/avatars/u/7881589?s=80&v=4"},"commit":{"message":"[Feature] Update pip install (#1324)\n\n* [Feature] Update pip install\r\n\r\n* Update Configuration\r\n\r\n* Update\r\n\r\n* Update\r\n\r\n* Update\r\n\r\n* Update Internal Config\r\n\r\n* Update collect env","shortMessageHtmlLink":"[Feature] Update pip install (<a class=\"issue-link js-issue-link\" data-error-text=\"Failed to load title\" data-id=\"2410517150\" data-permission-text=\"Title is private\" data-url=\"https://github.com/open-compass/opencompass/issues/1324\" data-hovercard-type=\"pull_request\" data-hovercard-url=\"/open-compass/opencompass/pull/1324/hovercard\" href=\"https://github.com/open-compass/opencompass/pull/1324\">#1324</a>)"}},{"before":"12b84aeb3bf5bd1f89a0b1287db5ee435b4c3511","after":"edab1c07baaebe3e6b6b48983d56656f6c93fd76","ref":"refs/heads/main","pushedAt":"2024-07-29T05:48:32.000Z","pushType":"pr_merge","commitsCount":1,"pusher":{"login":"tonysy","name":"Songyang Zhang","path":"/tonysy","primaryAvatarUrl":"https://github.com/avatars/u/7881589?s=80&v=4"},"commit":{"message":"[Feature] Support ModelScope datasets (#1289)\n\n* add ceval, gsm8k modelscope surpport\r\n\r\n* update race, mmlu, arc, cmmlu, commonsenseqa, humaneval and unittest\r\n\r\n* update bbh, flores, obqa, siqa, storycloze, summedits, winogrande, xsum datasets\r\n\r\n* format file\r\n\r\n* format file\r\n\r\n* update dataset format\r\n\r\n* support ms_dataset\r\n\r\n* udpate dataset for modelscope support\r\n\r\n* merge myl_dev and update test_ms_dataset\r\n\r\n* udpate dataset for modelscope support\r\n\r\n* update readme\r\n\r\n* update eval_api_zhipu_v2\r\n\r\n* remove unused code\r\n\r\n* add get_data_path function\r\n\r\n* update readme\r\n\r\n* remove tydiqa japanese subset\r\n\r\n* add ceval, gsm8k modelscope surpport\r\n\r\n* update race, mmlu, arc, cmmlu, commonsenseqa, humaneval and unittest\r\n\r\n* update bbh, flores, obqa, siqa, storycloze, summedits, winogrande, xsum datasets\r\n\r\n* format file\r\n\r\n* format file\r\n\r\n* update dataset format\r\n\r\n* support ms_dataset\r\n\r\n* udpate dataset for modelscope support\r\n\r\n* merge myl_dev and update test_ms_dataset\r\n\r\n* update readme\r\n\r\n* udpate dataset for modelscope support\r\n\r\n* update eval_api_zhipu_v2\r\n\r\n* remove unused code\r\n\r\n* add get_data_path function\r\n\r\n* remove tydiqa japanese subset\r\n\r\n* update util\r\n\r\n* remove .DS_Store\r\n\r\n* fix md format\r\n\r\n* move util into package\r\n\r\n* update docs/get_started.md\r\n\r\n* restore eval_api_zhipu_v2.py, add environment setting\r\n\r\n* Update dataset\r\n\r\n* Update\r\n\r\n* Update\r\n\r\n* Update\r\n\r\n* Update\r\n\r\n---------\r\n\r\nCo-authored-by: Yun lin <yunlin@U-Q9X2K4QV-1904.local>\r\nCo-authored-by: Yunnglin <mao.looper@qq.com>\r\nCo-authored-by: Yun lin <yunlin@laptop.local>\r\nCo-authored-by: Yunnglin <maoyl@smail.nju.edu.cn>\r\nCo-authored-by: zhangsongyang <zhangsongyang@pjlab.org.cn>","shortMessageHtmlLink":"[Feature] Support ModelScope datasets (<a class=\"issue-link js-issue-link\" data-error-text=\"Failed to load title\" data-id=\"2391223584\" data-permission-text=\"Title is private\" data-url=\"https://github.com/open-compass/opencompass/issues/1289\" data-hovercard-type=\"pull_request\" data-hovercard-url=\"/open-compass/opencompass/pull/1289/hovercard\" href=\"https://github.com/open-compass/opencompass/pull/1289\">#1289</a>)"}},{"before":"d3782c1d470a43ea271e4990bd2b00b833370183","after":"12b84aeb3bf5bd1f89a0b1287db5ee435b4c3511","ref":"refs/heads/main","pushedAt":"2024-07-26T10:42:30.000Z","pushType":"pr_merge","commitsCount":1,"pusher":{"login":"tonysy","name":"Songyang Zhang","path":"/tonysy","primaryAvatarUrl":"https://github.com/avatars/u/7881589?s=80&v=4"},"commit":{"message":"[Feature] Update CHARM Memeorziation (#1230)\n\n* update gemini api and add gemini models\r\n\r\n* add openai models\r\n\r\n* update CHARM evaluation\r\n\r\n* add CHARM memorization tasks\r\n\r\n* add CharmMemSummarizer (output eval details for memorization-independent reasoning analysis\r\n\r\n* update CHARM readme\r\n\r\n---------\r\n\r\nCo-authored-by: wujiang <wujiang@pjlab.org.cn>","shortMessageHtmlLink":"[Feature] Update CHARM Memeorziation (<a class=\"issue-link js-issue-link\" data-error-text=\"Failed to load title\" data-id=\"2337417748\" data-permission-text=\"Title is private\" data-url=\"https://github.com/open-compass/opencompass/issues/1230\" data-hovercard-type=\"pull_request\" data-hovercard-url=\"/open-compass/opencompass/pull/1230/hovercard\" href=\"https://github.com/open-compass/opencompass/pull/1230\">#1230</a>)"}},{"before":"9b9855a00816576b191d05e35ea45d9bb4cff7d5","after":"d3782c1d470a43ea271e4990bd2b00b833370183","ref":"refs/heads/main","pushedAt":"2024-07-26T10:27:29.000Z","pushType":"pr_merge","commitsCount":1,"pusher":{"login":"bittersweet1999","name":null,"path":"/bittersweet1999","primaryAvatarUrl":"https://github.com/avatars/u/148421775?s=80&v=4"},"commit":{"message":"Revert \"Calm dataset (#1287)\" (#1366)\n\nThis reverts commit edd0ffdf701f00ce0716a8792680e967f040c688.","shortMessageHtmlLink":"Revert \"Calm dataset (<a class=\"issue-link js-issue-link\" data-error-text=\"Failed to load title\" data-id=\"2390061889\" data-permission-text=\"Title is private\" data-url=\"https://github.com/open-compass/opencompass/issues/1287\" data-hovercard-type=\"pull_request\" data-hovercard-url=\"/open-compass/opencompass/pull/1287/hovercard\" href=\"https://github.com/open-compass/opencompass/pull/1287\">#1287</a>)\" (<a class=\"issue-link js-issue-link\" data-error-text=\"Failed to load title\" data-id=\"2431914222\" data-permission-text=\"Title is private\" data-url=\"https://github.com/open-compass/opencompass/issues/1366\" data-hovercard-type=\"pull_request\" data-hovercard-url=\"/open-compass/opencompass/pull/1366/hovercard\" href=\"https://github.com/open-compass/opencompass/pull/1366\">#1366</a>)"}},{"before":null,"after":"638d889c8137edf0f41e5f4085c44a079c957615","ref":"refs/heads/revert-1216-patch-7","pushedAt":"2024-07-26T10:22:38.000Z","pushType":"branch_creation","commitsCount":0,"pusher":{"login":"bittersweet1999","name":null,"path":"/bittersweet1999","primaryAvatarUrl":"https://github.com/avatars/u/148421775?s=80&v=4"},"commit":{"message":"Revert \"Add `en` and `zh` groups to longbench summarizer; Fix longbench overa…\"\n\nThis reverts commit 9b9855a00816576b191d05e35ea45d9bb4cff7d5.","shortMessageHtmlLink":"Revert \"Add <code>en</code> and <code>zh</code> groups to longbench summarizer; Fix longben…"}},{"before":null,"after":"a9745cf15aa6e65f5ca5fea180e0b70a24cade1c","ref":"refs/heads/revert-1287-calm_dataset","pushedAt":"2024-07-26T10:09:27.000Z","pushType":"branch_creation","commitsCount":0,"pusher":{"login":"bittersweet1999","name":null,"path":"/bittersweet1999","primaryAvatarUrl":"https://github.com/avatars/u/148421775?s=80&v=4"},"commit":{"message":"Revert \"Calm dataset (#1287)\"\n\nThis reverts commit edd0ffdf701f00ce0716a8792680e967f040c688.","shortMessageHtmlLink":"Revert \"Calm dataset (<a class=\"issue-link js-issue-link\" data-error-text=\"Failed to load title\" data-id=\"2390061889\" data-permission-text=\"Title is private\" data-url=\"https://github.com/open-compass/opencompass/issues/1287\" data-hovercard-type=\"pull_request\" data-hovercard-url=\"/open-compass/opencompass/pull/1287/hovercard\" href=\"https://github.com/open-compass/opencompass/pull/1287\">#1287</a>)\""}},{"before":"edd0ffdf701f00ce0716a8792680e967f040c688","after":"9b9855a00816576b191d05e35ea45d9bb4cff7d5","ref":"refs/heads/main","pushedAt":"2024-07-26T03:50:41.000Z","pushType":"pr_merge","commitsCount":1,"pusher":{"login":"bittersweet1999","name":null,"path":"/bittersweet1999","primaryAvatarUrl":"https://github.com/avatars/u/148421775?s=80&v=4"},"commit":{"message":"Add `en` and `zh` groups to longbench summarizer; Fix longbench overall score (#1216)\n\n* Add longbench groups\r\n\r\n* update\r\n\r\n* update","shortMessageHtmlLink":"Add <code>en</code> and <code>zh</code> groups to longbench summarizer; Fix longbench overa…"}},{"before":"a08931f21446a0dbf5c17024f6d1a564f3eed0b0","after":"edd0ffdf701f00ce0716a8792680e967f040c688","ref":"refs/heads/main","pushedAt":"2024-07-26T03:48:16.000Z","pushType":"pr_merge","commitsCount":1,"pusher":{"login":"bittersweet1999","name":null,"path":"/bittersweet1999","primaryAvatarUrl":"https://github.com/avatars/u/148421775?s=80&v=4"},"commit":{"message":"Calm dataset (#1287)\n\n* add calm dataset\r\n\r\n* modify config max_out_len\r\n\r\n* update README\r\n\r\n* Modify README\r\n\r\n* update README\r\n\r\n* update README\r\n\r\n* update README\r\n\r\n* update README\r\n\r\n* update README\r\n\r\n* add summarizer and modify readme\r\n\r\n* delete summarizer config comment\r\n\r\n* update summarizer\r\n\r\n* modify same response to all questions\r\n\r\n* update README","shortMessageHtmlLink":"Calm dataset (<a class=\"issue-link js-issue-link\" data-error-text=\"Failed to load title\" data-id=\"2390061889\" data-permission-text=\"Title is private\" data-url=\"https://github.com/open-compass/opencompass/issues/1287\" data-hovercard-type=\"pull_request\" data-hovercard-url=\"/open-compass/opencompass/pull/1287/hovercard\" href=\"https://github.com/open-compass/opencompass/pull/1287\">#1287</a>)"}},{"before":"8ee7fecb686bc0bf07ffb18e1349858447d43ca0","after":"a08931f21446a0dbf5c17024f6d1a564f3eed0b0","ref":"refs/heads/main","pushedAt":"2024-07-26T03:46:02.000Z","pushType":"pr_merge","commitsCount":1,"pusher":{"login":"bittersweet1999","name":null,"path":"/bittersweet1999","primaryAvatarUrl":"https://github.com/avatars/u/148421775?s=80&v=4"},"commit":{"message":"[Fix] origin_prompt should be None in llm-compression task (#1225)\n\nCo-authored-by: Qinyang Mou <qinyang_mou@intsig.net>","shortMessageHtmlLink":"[Fix] origin_prompt should be None in llm-compression task (<a class=\"issue-link js-issue-link\" data-error-text=\"Failed to load title\" data-id=\"2333830600\" data-permission-text=\"Title is private\" data-url=\"https://github.com/open-compass/opencompass/issues/1225\" data-hovercard-type=\"pull_request\" data-hovercard-url=\"/open-compass/opencompass/pull/1225/hovercard\" href=\"https://github.com/open-compass/opencompass/pull/1225\">#1225</a>)"}},{"before":"65fad8e2ac7fae458de8c06fa19458bfd8b61efe","after":"8ee7fecb686bc0bf07ffb18e1349858447d43ca0","ref":"refs/heads/main","pushedAt":"2024-07-26T03:44:51.000Z","pushType":"pr_merge","commitsCount":1,"pusher":{"login":"bittersweet1999","name":null,"path":"/bittersweet1999","primaryAvatarUrl":"https://github.com/avatars/u/148421775?s=80&v=4"},"commit":{"message":"Adding support for Doubao API (#1218)\n\n* Adding support for Doubao API\r\n\r\n* Update doubao_api.py\r\n\r\nFixed the bug that the connection would be retried even if it was normal.\r\n\r\n* Update doubao_api.py\r\n\r\n---------\r\n\r\nCo-authored-by: bittersweet1999 <148421775+bittersweet1999@users.noreply.github.com>","shortMessageHtmlLink":"Adding support for Doubao API (<a class=\"issue-link js-issue-link\" data-error-text=\"Failed to load title\" data-id=\"2326879697\" data-permission-text=\"Title is private\" data-url=\"https://github.com/open-compass/opencompass/issues/1218\" data-hovercard-type=\"pull_request\" data-hovercard-url=\"/open-compass/opencompass/pull/1218/hovercard\" href=\"https://github.com/open-compass/opencompass/pull/1218\">#1218</a>)"}},{"before":"51a94aee0103835d64574dd0580e35d9a689d860","after":"65fad8e2ac7fae458de8c06fa19458bfd8b61efe","ref":"refs/heads/main","pushedAt":"2024-07-26T03:19:04.000Z","pushType":"pr_merge","commitsCount":1,"pusher":{"login":"bittersweet1999","name":null,"path":"/bittersweet1999","primaryAvatarUrl":"https://github.com/avatars/u/148421775?s=80&v=4"},"commit":{"message":"[Fix] minor update wildbench (#1335)\n\n* update crb\r\n\r\n* update crbbench\r\n\r\n* update crbbench\r\n\r\n* update crbbench\r\n\r\n* minor update wildbench\r\n\r\n* [Fix] Update doc of wildbench, and merge wildbench into subjective\r\n\r\n* [Fix] Update doc of wildbench, and merge wildbench into subjective, fix crbbench\r\n\r\n* Update crb.md\r\n\r\n* Update crb_pair_judge.py\r\n\r\n* Update crb_single_judge.py\r\n\r\n* Update subjective_evaluation.md\r\n\r\n* Update openai_api.py\r\n\r\n* [Update] update wildbench readme\r\n\r\n* [Update] update wildbench readme\r\n\r\n* [Update] update wildbench readme, remove crb\r\n\r\n* Delete configs/eval_subjective_wildbench_pair.py\r\n\r\n* Delete configs/eval_subjective_wildbench_single.py\r\n\r\n* Update __init__.py\r\n\r\n---------\r\n\r\nCo-authored-by: bittersweet1999 <148421775+bittersweet1999@users.noreply.github.com>","shortMessageHtmlLink":"[Fix] minor update wildbench (<a class=\"issue-link js-issue-link\" data-error-text=\"Failed to load title\" data-id=\"2415942805\" data-permission-text=\"Title is private\" data-url=\"https://github.com/open-compass/opencompass/issues/1335\" data-hovercard-type=\"pull_request\" data-hovercard-url=\"/open-compass/opencompass/pull/1335/hovercard\" href=\"https://github.com/open-compass/opencompass/pull/1335\">#1335</a>)"}},{"before":"69aa2f2d5705519ed735e2269ff7a71fabb43cd6","after":"51a94aee0103835d64574dd0580e35d9a689d860","ref":"refs/heads/main","pushedAt":"2024-07-26T03:03:55.000Z","pushType":"pr_merge","commitsCount":1,"pusher":{"login":"tonysy","name":"Songyang Zhang","path":"/tonysy","primaryAvatarUrl":"https://github.com/avatars/u/7881589?s=80&v=4"},"commit":{"message":"[Bug] fix bug: delete & (#1365)\n\nCo-authored-by: 白超 <baichao19@huawei.com>","shortMessageHtmlLink":"[Bug] fix bug: delete &amp; (<a class=\"issue-link js-issue-link\" data-error-text=\"Failed to load title\" data-id=\"2429993551\" data-permission-text=\"Title is private\" data-url=\"https://github.com/open-compass/opencompass/issues/1365\" data-hovercard-type=\"pull_request\" data-hovercard-url=\"/open-compass/opencompass/pull/1365/hovercard\" href=\"https://github.com/open-compass/opencompass/pull/1365\">#1365</a>)"}},{"before":"c3c02c2960c3da6912d6f0d2c6fdd0d54849a770","after":"69aa2f2d5705519ed735e2269ff7a71fabb43cd6","ref":"refs/heads/main","pushedAt":"2024-07-25T11:01:56.000Z","pushType":"pr_merge","commitsCount":1,"pusher":{"login":"bittersweet1999","name":null,"path":"/bittersweet1999","primaryAvatarUrl":"https://github.com/avatars/u/148421775?s=80&v=4"},"commit":{"message":"[Feature] Make NeedleBench available on HF (#1364)\n\n* update_lint\r\n\r\n* update_huggingface format\r\n\r\n* fix bug\r\n\r\n* update docs","shortMessageHtmlLink":"[Feature] Make NeedleBench available on HF (<a class=\"issue-link js-issue-link\" data-error-text=\"Failed to load title\" data-id=\"2429667680\" data-permission-text=\"Title is private\" data-url=\"https://github.com/open-compass/opencompass/issues/1364\" data-hovercard-type=\"pull_request\" data-hovercard-url=\"/open-compass/opencompass/pull/1364/hovercard\" href=\"https://github.com/open-compass/opencompass/pull/1364\">#1364</a>)"}},{"before":"73aa55af6d1fac71a6f9ca436ff8a5ca5dc5206e","after":"c3c02c2960c3da6912d6f0d2c6fdd0d54849a770","ref":"refs/heads/main","pushedAt":"2024-07-25T10:44:26.000Z","pushType":"pr_merge","commitsCount":1,"pusher":{"login":"bittersweet1999","name":null,"path":"/bittersweet1999","primaryAvatarUrl":"https://github.com/avatars/u/148421775?s=80&v=4"},"commit":{"message":"update docs (#1318)\n\n* update docs\r\n\r\n* 高效评测 -> 数据分片\r\n\r\n* update\r\n\r\n* update\r\n\r\n* Update faq.md\r\n\r\n---------\r\n\r\nCo-authored-by: bittersweet1999 <148421775+bittersweet1999@users.noreply.github.com>","shortMessageHtmlLink":"update docs (<a class=\"issue-link js-issue-link\" data-error-text=\"Failed to load title\" data-id=\"2405542436\" data-permission-text=\"Title is private\" data-url=\"https://github.com/open-compass/opencompass/issues/1318\" data-hovercard-type=\"pull_request\" data-hovercard-url=\"/open-compass/opencompass/pull/1318/hovercard\" href=\"https://github.com/open-compass/opencompass/pull/1318\">#1318</a>)"}},{"before":"0aad8199c7304cc0d37b6f6a06daa9afe5357088","after":"73aa55af6d1fac71a6f9ca436ff8a5ca5dc5206e","ref":"refs/heads/main","pushedAt":"2024-07-25T10:38:24.000Z","pushType":"pr_merge","commitsCount":1,"pusher":{"login":"bittersweet1999","name":null,"path":"/bittersweet1999","primaryAvatarUrl":"https://github.com/avatars/u/148421775?s=80&v=4"},"commit":{"message":"[Fix] Support HF models deployed with an OpenAI-compatible API. (#1352)\n\n* Support HF models deployed with an OpenAI-compatible API.\r\n\r\n* resolve lint issue\r\n\r\n* add extra_body arguments\r\n\r\nThere are many other arguments when using openi-compatiable API like this: https://docs.vllm.ai/en/latest/serving/openai_compatible_server.html#extra-parameters-for-chat-api\r\n\r\n* fix linting issue\r\n\r\n* fix yapf linting issue","shortMessageHtmlLink":"[Fix] Support HF models deployed with an OpenAI-compatible API. (<a class=\"issue-link js-issue-link\" data-error-text=\"Failed to load title\" data-id=\"2424220999\" data-permission-text=\"Title is private\" data-url=\"https://github.com/open-compass/opencompass/issues/1352\" data-hovercard-type=\"pull_request\" data-hovercard-url=\"/open-compass/opencompass/pull/1352/hovercard\" href=\"https://github.com/open-compass/opencompass/pull/1352\">#1352</a>)"}}],"hasNextPage":true,"hasPreviousPage":false,"activityType":"all","actor":null,"timePeriod":"all","sort":"DESC","perPage":30,"cursor":"djE6ks8AAAAEk10kegA","startCursor":null,"endCursor":null}},"title":"Activity · open-compass/opencompass"}