Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Issues training with custom dataset #102

Open
Tostino opened this issue Jan 9, 2024 · 0 comments
Open

Issues training with custom dataset #102

Tostino opened this issue Jan 9, 2024 · 0 comments

Comments

@Tostino
Copy link

Tostino commented Jan 9, 2024 •
edited

Hi there. I've read through some of the other threads about custom datasets, and am not quite sure what I am doing wrong.

I attempt to train using a custom jsonl file using the Relevancy Judged Dataset (Tevatron/msmarco-passage) format, but get errors.

My training call looks like:

accelerate launch examples/colbert/train_colbert.py \
  --output_dir jina_claim_contract_0.1 \
  --model_name_or_path jinaai/jina-embeddings-v2-base-en \
  --save_steps 20000 \
  --dataset_name Tevatron/msmarco-passage \
  --train_dir dataset/contract_claim_dataset.jsonl \
  --passage_field_separator "" \
  --fp16 \
  --per_device_train_batch_size 4 \
  --train_n_passages 4 \
  --learning_rate 5e-6 \
  --q_max_len 4095 \
  --p_max_len 4096 \
  --num_train_epochs 3 \
  --add_pooler \
  --projection_in_dim 768 \
  --projection_out_dim 768 \
  --logging_steps 50 \
  --overwrite_output_dir

The error message I am getting is:

Traceback (most recent call last):
  File "/home/user/IdeaProjects/tevatron/venv/lib/python3.10/site-packages/datasets/builder.py", line 1973, in _prepare_split_single
    for _, table in generator:
  File "/home/user/IdeaProjects/tevatron/venv/lib/python3.10/site-packages/datasets/packaged_modules/parquet/parquet.py", line 85, in _generate_tables
    parquet_file = pq.ParquetFile(f)
  File "/home/user/IdeaProjects/tevatron/venv/lib/python3.10/site-packages/pyarrow/parquet/core.py", line 341, in __init__
    self.reader.open(
  File "pyarrow/_parquet.pyx", line 1250, in pyarrow._parquet.ParquetReader.open
  File "pyarrow/error.pxi", line 91, in pyarrow.lib.check_status
pyarrow.lib.ArrowInvalid: Parquet magic bytes not found in footer. Either the file is corrupted or this is not a parquet file.

A single line of the data in my jsonl looks like:

{"query_id": "8ece333c-e092-4359-b16d-2f803a02abae", "query": "<#meta#>\n- Task: match_claim_contract\n<#system#>\nYour purpose is to find the correct contract based on the json data in the user_context section.\n<#chat#>\n<#user#>\nPlease use the invoice data below to find the corresponding contract.\n<#user_context#>\n{\"client_name\":\"blah\",\"invoice_number\":\"641782\",\"invoice_date\":\"2016-03-05\",\"start_date\":\"2016-02-27\",\"end_date\":\"2016-03-04\",\"amount_requested\":630.31,\"company_number\":\"12257\",\"company_name\":\"SHAMROCK FS-Colorado\",\"country\":\"USA\",\"region\":\"CO\",\"city\":\"AURORA, CO 80011\",\"claims\":{\"claim_state\":\"COMPLETE\",\"claim_tag\":\"Country Buffet\",\"claim_alt_payee\":null,\"claim_lumpsums\":null,\"claim_adhoc_products\":null,\"claim_products\":[{\"product_number\":\"100009\",\"uom_type_cd\":\"CS\",\"distributor_company_number\":\"12257\",\"resolve_date\":\"2016-03-03\",\"claimant_contract_name\":\"Country Buffet\",\"distributor_invoice_number\":null},{\"product_number\":\"9619596\",\"uom_type_cd\":\"CS\",\"distributor_company_number\":\"12257\",\"resolve_date\":\"2016-03-03\",\"claimant_contract_name\":\"Country Buffet\",\"distributor_invoice_number\":null},{\"product_number\":\"9619596\",\"uom_type_cd\":\"CS\",\"distributor_company_number\":\"12257\",\"resolve_date\":\"2016-03-03\",\"claimant_contract_name\":\"Country Buffet\",\"distributor_invoice_number\":null}]}}\n<#bot#>\n", "positive_passages": [{"docid": "1000058", "title": "", "text": "{\"client_name\":\"blah\",\"contract_title\":\"Dynamic Foods\",\"contract_type\":\"STANDARD\",\"cost_basis_type\":\"FOB\",\"catch_all\":false,\"creator_email\":\"email@company.com\",\"contractee_type_description\":\"Operator\",\"contractee_name\":\"Ovation Brands\",\"contractee_number\":\"96\",\"contractee_country\":\"USA\",\"contractee_region\":\"Jay\",\"contractee_city\":\"happy\",\"contractee_address_1\":\"1329 test lane\",\"contract_template_description\":\"Default\",\"contracts\":[{\"start_date\":\"2016-04-21\",\"end_date\":\"2016-10-31\",\"contract_version_id\":\"1000058.R3.A0\",\"contract_name\":\"Dynamic Foods\",\"contract_state\":\"CANCELLED\",\"contract_state_last_modified\":\"2016-06-01\",\"contract_products\":[\"9618296\",\"9618496\",\"7112696\",\"9610996\",\"6395100\",\"70003234\",\"9618396\",\"6602352\",\"S8148805\",\"7000496\",\"5448525\",\"9691096\",\"7111196\",\"100009\",\"9619596\",\"6516096\",\"7110496\"],\"contract_product_groupings\":null,\"contract_lumpsums\":null},{\"start_date\":\"2016-04-21\",\"end_date\":\"2016-07-24\",\"contract_version_id\":\"1000058.R4.A0\",\"contract_name\":\"Dynamic Foods\",\"contract_state\":\"AMENDED\",\"contract_state_last_modified\":\"2016-07-08\",\"contract_products\":[\"70003234\",\"7110496\",\"7111196\",\"7000496\",\"5448525\",\"9691096\",\"6516096\",\"9618296\",\"100009\",\"9619596\",\"6602352\",\"9610996\",\"9618396\",\"6395100\",\"9618496\",\"S8148805\",\"7112696\"],\"contract_product_groupings\":null,\"contract_lumpsums\":null},{\"start_date\":\"2016-04-21\",\"end_date\":\"2016-07-24\",\"contract_version_id\":\"1000058.R4.A1\",\"contract_name\":\"Dynamic Foods\",\"contract_state\":\"AMENDED\",\"contract_state_last_modified\":\"2016-09-08\",\"contract_products\":[\"7111196\",\"6516096\",\"7112696\",\"9610996\",\"6395100\",\"5448525\",\"9619596\",\"70003234\",\"7000496\",\"S8148805\",\"9618496\",\"9618296\",\"6602352\",\"100009\",\"9618396\",\"9691096\",\"7110496\"],\"contract_product_groupings\":null,\"contract_lumpsums\":null},{\"start_date\":\"2015-04-08\",\"end_date\":\"2015-10-21\",\"contract_version_id\":\"1000058.R0.A0\",\"contract_name\":\"Ovation's\",\"contract_state\":\"EXPIRED\",\"contract_state_last_modified\":\"2015-10-23\",\"contract_products\":[\"9618396\",\"7112696\",\"9618296\",\"6395100\",\"6602352\",\"7111196\",\"100009\",\"9618496\",\"S8148805\",\"9619596\",\"7110496\",\"5448525\",\"9610996\",\"7000496\",\"6516096\",\"9691096\"],\"contract_product_groupings\":null,\"contract_lumpsums\":null},{\"start_date\":\"2015-11-22\",\"end_date\":\"2016-04-20\",\"contract_version_id\":\"1000058.R2.A0\",\"contract_name\":\"Dynamic Foods\",\"contract_state\":\"AMENDED\",\"contract_state_last_modified\":\"2016-08-03\",\"contract_products\":[\"100009\",\"S8148805\",\"7110496\",\"7111196\",\"9618396\",\"5448525\",\"9619596\",\"6516096\",\"6395100\",\"7000496\",\"9610996\",\"7112696\",\"9618496\",\"9618296\",\"9691096\",\"6602352\"],\"contract_product_groupings\":null,\"contract_lumpsums\":null},{\"start_date\":\"2015-10-22\",\"end_date\":\"2015-11-21\",\"contract_version_id\":\"1000058.R1.A0\",\"contract_name\":\"Ovation's\",\"contract_state\":\"EXPIRED\",\"contract_state_last_modified\":\"2016-01-04\",\"contract_products\":[\"9619596\",\"9691096\",\"S8148805\",\"7111196\",\"7110496\",\"7112696\",\"5448525\",\"7000496\",\"9618396\",\"6516096\",\"9618496\",\"6395100\",\"9610996\",\"6602352\",\"100009\",\"9618296\"],\"contract_product_groupings\":null,\"contract_lumpsums\":null},{\"start_date\":\"2015-11-22\",\"end_date\":\"2016-04-20\",\"contract_version_id\":\"1000058.R2.A1\",\"contract_name\":\"Dynamic Foods\",\"contract_state\":\"EXPIRED\",\"contract_state_last_modified\":\"2016-08-03\",\"contract_products\":[\"7000496\",\"9618296\",\"9610996\",\"9691096\",\"6602352\",\"7112696\",\"9618496\",\"100009\",\"9618396\",\"5448525\",\"S8148805\",\"7110496\",\"7111196\",\"9619596\",\"6516096\",\"6395100\"],\"contract_product_groupings\":null,\"contract_lumpsums\":null},{\"start_date\":\"2016-04-21\",\"end_date\":\"2016-07-24\",\"contract_version_id\":\"1000058.R4.A2\",\"contract_name\":\"Dynamic Foods\",\"contract_state\":\"EXPIRED\",\"contract_state_last_modified\":\"2016-09-08\",\"contract_products\":[\"9610996\",\"100009\",\"6516096\",\"7111196\",\"70003234\",\"5448525\",\"6395100\",\"9619596\",\"7000496\",\"7112696\",\"9618296\",\"9618496\",\"6602352\",\"7110496\",\"9618396\",\"S8148805\",\"9691096\"],\"contract_product_groupings\":null,\"contract_lumpsums\":null}]}"}], "negative_passages": [{"docid": "1000018", "title": "", "text": "{\"client_name\":\"blah\",\"contract_title\":\"SAMPLES\",\"contract_type\":\"STANDARD\",\"cost_basis_type\":\"FOB\",\"catch_all\":false,\"creator_email\":\"email@company.com\",\"contractee_type_description\":\"Distributor\",\"contractee_name\":\"All FoodService Distributors\",\"contractee_number\":\"alldist\",\"contractee_country\":null,\"contractee_region\":null,\"contractee_city\":null,\"contractee_address_1\":null,\"contract_template_description\":\"Default\",\"contracts\":[{\"start_date\":\"2015-01-01\",\"end_date\":\"2017-12-31\",\"contract_version_id\":\"1000018.R0.A2\",\"contract_name\":\"SAMPLES\",\"contract_state\":\"TERMINATED\",\"contract_state_last_modified\":\"2015-10-12\",\"contract_products\":null,\"contract_product_groupings\":[\"allfs\"],\"contract_lumpsums\":null},{\"start_date\":\"2015-01-01\",\"end_date\":\"2017-12-31\",\"contract_version_id\":\"1000018.R0.A0\",\"contract_name\":\"KEEP:  Samples Contract\",\"contract_state\":\"AMENDED\",\"contract_state_last_modified\":\"2015-10-12\",\"contract_products\":null,\"contract_product_groupings\":[\"allfs\"],\"contract_lumpsums\":null},{\"start_date\":\"2015-01-01\",\"end_date\":\"2017-12-31\",\"contract_version_id\":\"1000018.R0.A1\",\"contract_name\":\"Samples Contract\",\"contract_state\":\"AMENDED\",\"contract_state_last_modified\":\"2015-10-12\",\"contract_products\":null,\"contract_product_groupings\":[\"allfs\"],\"contract_lumpsums\":null}]}"}, {"docid": "1000025", "title": "", "text": "{\"client_name\":\"blah\",\"contract_title\":\"IMA 2% Service Fee and Marketing\",\"contract_type\":\"STANDARD\",\"cost_basis_type\":\"FOB\",\"catch_all\":false,\"creator_email\":\"email@company.com\",\"contractee_type_description\":\"Distributor\",\"contractee_name\":\"IMA\",\"contractee_number\":\"004\",\"contractee_country\":null,\"contractee_region\":null,\"contractee_city\":null,\"contractee_address_1\":null,\"contract_template_description\":\"Default\",\"contracts\":[{\"start_date\":\"2015-01-01\",\"end_date\":\"2015-12-31\",\"contract_version_id\":\"1000025.R0.A0\",\"contract_name\":\"IMA 2% Service Fee and Marketing\",\"contract_state\":\"EXPIRED\",\"contract_state_last_modified\":\"2015-12-31\",\"contract_products\":null,\"contract_product_groupings\":[\"fghcbk\",\"fgc5-10\",\"fgpcfs\",\"fgmbg\",\"fg32b-fs\"],\"contract_lumpsums\":[{\"lumpsum_cd\":\"m\",\"contract_lumpsum_description\":\"Point of Sale and Training Video\"},{\"lumpsum_cd\":\"m\",\"contract_lumpsum_description\":\"Marketing Program 2015\"}]},{\"start_date\":\"2017-01-01\",\"end_date\":\"2017-12-31\",\"contract_version_id\":\"1000025.R2.A0\",\"contract_name\":\"IMA 2% Service Fee and Marketing\",\"contract_state\":\"EXPIRED\",\"contract_state_last_modified\":\"2017-01-09\",\"contract_products\":[\"14234\",\"6399925\",\"14225\",\"PFC6528\",\"14235\",\"14230\",\"100399\",\"14228\",\"S8P00800\",\"S8003801\",\"14229\",\"14237\",\"PFC6548\",\"PFC6508\",\"PFC0796\",\"PRG5096C\",\"14232\",\"14231\",\"14223\",\"7046896\",\"6092633\",\"9616596\",\"14226\",\"S8P00801\",\"14224\",\"14236\",\"14233\",\"14227\",\"14222\",\"PFC6538\",\"C9621196\"],\"contract_product_groupings\":[\"FSA\",\"fgmbk\",\"fgpbk\",\"fgmbg\",\"fg32b-fs\",\"fgc5-10\",\"FGHCFDSVC\",\"fgpcfs\"],\"contract_lumpsums\":[{\"lumpsum_cd\":\"m\",\"contract_lumpsum_description\":\"Marketing Program 2016\"},{\"lumpsum_cd\":\"m\",\"contract_lumpsum_description\":\"Point of Sale and Training Video\"}]},{\"start_date\":\"2016-01-01\",\"end_date\":\"2016-12-31\",\"contract_version_id\":\"1000025.R1.A1\",\"contract_name\":\"IMA 2% Service Fee and Marketing\",\"contract_state\":\"EXPIRED\",\"contract_state_last_modified\":\"2016-12-31\",\"contract_products\":[\"14223\",\"14231\",\"9616596\",\"14226\",\"7046896\",\"14224\",\"S8P00801\",\"PFC6548\",\"S8P00800\",\"14229\",\"S8003801\",\"14228\",\"14237\",\"14235\",\"PFC6528\",\"PFC0796\",\"PFC6508\",\"14225\",\"6399925\",\"14234\",\"14230\",\"100399\",\"14222\",\"14227\",\"14233\",\"14236\",\"C9621196\",\"PFC6538\",\"PRG5096C\",\"6092633\",\"14232\"],\"contract_product_groupings\":[\"fgmbk\",\"FGHCFDSVC\",\"fg32b-fs\",\"fgmbg\",\"FSA\",\"fgpbk\",\"fgpcfs\",\"fgc5-10\"],\"contract_lumpsums\":[{\"lumpsum_cd\":\"m\",\"contract_lumpsum_description\":\"Marketing Program 2016\"},{\"lumpsum_cd\":\"m\",\"contract_lumpsum_description\":\"Point of Sale and Training Video\"}]},{\"start_date\":\"2018-01-01\",\"end_date\":\"2023-12-31\",\"contract_version_id\":\"1000025.R3.A2\",\"contract_name\":\"IMA 2% Service Fee and Marketing\",\"contract_state\":\"EXPIRED\",\"contract_state_last_modified\":\"2024-01-01\",\"contract_products\":[\"14234\",\"9619995\",\"14231\",\"14230\",\"14224\",\"PFC6538\",\"14232\",\"C9621196\",\"9616596\",\"14225\",\"7046896\",\"9615133\",\"PFC0796\",\"14226\",\"S8P00800\",\"14229\",\"6092633\",\"14236\",\"PRG5096C\",\"22818\",\"PFC6508\",\"14228\",\"S8003801\",\"6399925\",\"9616796\",\"14227\",\"PFC6528\",\"14223\",\"PFC6548\",\"100399\",\"14233\",\"14235\",\"14237\",\"14222\",\"S8P00801\"],\"contract_product_groupings\":[\"FGHCFDSVC\",\"fg32b-fs\",\"fgpbk\",\"fgmbg\",\"fgc5-10\",\"fgmbk\",\"FSA\",\"fgpcfs\"],\"contract_lumpsums\":null},{\"start_date\":\"2016-01-01\",\"end_date\":\"2016-12-31\",\"contract_version_id\":\"1000025.R1.A0\",\"contract_name\":\"IMA 2% Service Fee and Marketing\",\"contract_state\":\"AMENDED\",\"contract_state_last_modified\":\"2016-05-02\",\"contract_products\":null,\"contract_product_groupings\":[\"fghcbk\",\"fgpcfs\",\"fgc5-10\",\"fg32b-fs\",\"fgmbg\"],\"contract_lumpsums\":[{\"lumpsum_cd\":\"m\",\"contract_lumpsum_description\":\"Marketing Program 2016\"},{\"lumpsum_cd\":\"m\",\"contract_lumpsum_description\":\"Point of Sale and Training Video\"}]},{\"start_date\":\"2018-01-01\",\"end_date\":\"2023-12-31\",\"contract_version_id\":\"1000025.R3.A0\",\"contract_name\":\"IMA 2% Service Fee and Marketing\",\"contract_state\":\"AMENDED\",\"contract_state_last_modified\":\"2023-06-07\",\"contract_products\":[\"14226\",\"6399925\",\"14227\",\"14222\",\"14235\",\"14230\",\"14233\",\"S8003801\",\"6092633\",\"14237\",\"S8P00800\",\"S8P00801\",\"C9621196\",\"14229\",\"7046896\",\"PFC6508\",\"PFC6528\",\"14223\",\"14225\",\"14236\",\"PRG5096C\",\"PFC0796\",\"PFC6548\",\"14228\",\"100399\",\"14224\",\"14231\",\"14232\",\"9616596\",\"PFC6538\",\"14234\"],\"contract_product_groupings\":[\"fgpcfs\",\"fgmbg\",\"fg32b-fs\",\"FGHCFDSVC\",\"FSA\",\"fgmbk\",\"fgc5-10\",\"fgpbk\"],\"contract_lumpsums\":null},{\"start_date\":\"2018-01-01\",\"end_date\":\"2023-12-31\",\"contract_version_id\":\"1000025.R3.A1\",\"contract_name\":\"IMA 2% Service Fee and Marketing\",\"contract_state\":\"AMENDED\",\"contract_state_last_modified\":\"2023-06-07\",\"contract_products\":[\"14228\",\"14224\",\"14237\",\"6399925\",\"PFC6528\",\"14226\",\"14232\",\"S8P00801\",\"14233\",\"14236\",\"6092633\",\"PFC6508\",\"14230\",\"14231\",\"S8P00800\",\"7046896\",\"PFC0796\",\"14225\",\"14227\",\"PRG5096C\",\"S8003801\",\"PFC6548\",\"14234\",\"14223\",\"9616596\",\"PFC6538\",\"100399\",\"14235\",\"C9621196\",\"14222\",\"14229\"],\"contract_product_groupings\":[\"fgmbk\",\"fgpbk\",\"FGHCFDSVC\",\"fgmbg\",\"fgpcfs\",\"FSA\",\"fg32b-fs\",\"fgc5-10\"],\"contract_lumpsums\":null}]}"}, {"docid": "1000026", "title": "", "text": "{\"client_name\":\"blah\",\"contract_title\":\"FSA Corporate EMI Azar brand\",\"contract_type\":\"STANDARD\",\"cost_basis_type\":\"FOB\",\"catch_all\":false,\"creator_email\":\"email@company.com\",\"contractee_type_description\":\"Distributor\",\"contractee_name\":\"FSA CORP\",\"contractee_number\":\"fs\",\"contractee_country\":null,\"contractee_region\":null,\"contractee_city\":null,\"contractee_address_1\":null,\"contract_template_description\":\"Default\",\"contracts\":[{\"start_date\":\"2016-02-01\",\"end_date\":\"2016-12-31\",\"contract_version_id\":\"1000026.R2.A4\",\"contract_name\":\"FSA Corporate EMI Azar brand\",\"contract_state\":\"AMENDED\",\"contract_state_last_modified\":\"2016-08-04\",\"contract_products\":[\"14234\",\"644\",\"14237\",\"14235\",\"27115\",\"14233\",\"14226\",\"14230\",\"17558\",\"6092633\",\"27112\",\"14225\",\"14223\",\"14232\",\"14236\",\"27110\",\"640\",\"PFC6508\",\"27114\",\"27108\",\"PFC6538\",\"642\",\"27106\",\"14224\",\"27104\",\"27113\",\"14229\",\"PFC6528\",\"14231\",\"14222\",\"14228\"],\"contract_product_groupings\":[\"fgc5-10\",\"fgb5alob\",\"FGMGLBLKFS\",\"FGHCFDSVC\",\"FBBKFSFS\",\"fg32b-fs\",\"fgcpc\",\"fglfgfs\",\"fgmbk\",\"fgpcfs\",\"fgcpfsn\"],\"contract_lumpsums\":null},{\"start_date\":\"2018-03-01\",\"end_date\":\"2018-05-31\",\"contract_version_id\":\"1000026.R4.A1\",\"contract_name\":\"FSA Corporate EMI Azar brand\",\"contract_state\":\"EXPIRED\",\"contract_state_last_modified\":\"2018-04-27\",\"contract_products\":[\"14229\",\"14237\",\"27110\",\"PFC6508\",\"27106\",\"14230\",\"642\",\"27115\",\"14228\",\"14235\",\"644\",\"00644\",\"14225\",\"14222\",\"27114\",\"PFC6528\",\"6092633\",\"14226\",\"14223\",\"14236\",\"PFC6538\",\"17558\",\"00640\",\"14234\",\"00642\",\"27104\",\"27108\",\"14233\",\"640\",\"14231\",\"27113\",\"14224\",\"14232\",\"27112\"],\"contract_product_groupings\":[\"FGHCFDSVC\",\"fgc5-10\",\"FBBKFSFS\",\"ALLPVTLBL\",\"fglfgfs\",\"fg32b-fs\",\"fgcpfsn\",\"fgb5alob\",\"fgmbk\",\"fgpcfs\",\"fgpbr\",\"fgpbk\",\"fgcpc\",\"FGMGLBLKFS\"],\"contract_lumpsums\":null},{\"start_date\":\"2015-09-01\",\"end_date\":\"2016-01-31\",\"contract_version_id\":\"1000026.R1.A5\",\"contract_name\":\"FSA Corporate EMI Azar brand\",\"contract_state\":\"EXPIRED\",\"contract_state_last_modified\":\"2016-08-10\",\"contract_products\":[\"14229\",\"17558\",\"14225\",\"14224\",\"14230\",\"14231\",\"642\",\"14228\",\"27104\",\"14236\",\"27114\",\"27115\",\"PFC6528\",\"14233\",\"14234\",\"14223\",\"27112\",\"640\",\"27110\",\"27106\",\"27108\",\"14226\",\"14237\",\"14232\",\"644\",\"PFC6508\",\"14222\",\"27113\",\"14235\",\"PFC6538\",\"6092633\"],\"contract_product_groupings\":[\"fgb5alob\",\"FGHCFDSVC\",\"fgmbk\",\"fgpcfs\",\"fgcpc\",\"fglfgfs\",\"fgcpfsn\",\"FGMGLBLKFS\",\"FBBKFSFS\",\"fg32b-fs\",\"fgc5-10\"],\"contract_lumpsums\":null},{\"start_date\":\"2017-01-01\",\"end_date\":\"2018-02-28\",\"contract_version_id\":\"1000026.R3.A1\",\"contract_name\":\"FSA Corporate EMI Azar brand\",\"contract_state\":\"AMENDED\",\"contract_state_last_modified\":\"2018-02-22\",\"contract_products\":[\"27114\",\"14234\",\"27112\",\"14230\",\"27113\",\"27110\",\"PFC6528\",\"27115\",\"14233\",\"27106\",\"14229\",\"6092633\",\"PFC6508\",\"14225\",\"14237\",\"14232\",\"14235\",\"PFC6538\",\"642\",\"644\",\"14236\",\"14223\",\"640\",\"27104\",\"14222\",\"14231\",\"00640\",\"14228\",\"14226\",\"27108\",\"14224\",\"17558\",\"00644\",\"00642\"],\"contract_product_groupings\":[\"fgb5alob\",\"fgpbk\",\"fg32b-fs\",\"fgc5-10\",\"fgpbr\",\"FBBKFSFS\",\"fgcpfsn\",\"fgpcfs\",\"FGHCFDSVC\",\"fgmbk\",\"ALLPVTLBL\",\"fglfgfs\",\"fgcpc\",\"FGMGLBLKFS\"],\"contract_lumpsums\":null},{\"start_date\":\"2018-07-31\",\"end_date\":\"2021-06-30\",\"contract_version_id\":\"1000026.R6.A1\",\"contract_name\":\"FSA Corporate EMI Azar brand\",\"contract_state\":\"EXPIRED\",\"contract_state_last_modified\":\"2021-07-01\",\"contract_products\":[\"6092633\",\"14226\",\"14228\",\"14223\",\"27115\",\"14232\",\"PFC6508\",\"14234\",\"14229\",\"640\",\"14237\",\"00644\",\"PFC6528\",\"27106\",\"644\",\"14236\",\"27108\",\"27112\",\"14235\",\"14222\",\"PFC6538\",\"00640\",\"14233\",\"14225\",\"00642\",\"14231\",\"642\",\"27104\",\"17558\",\"14230\",\"14224\",\"27110\",\"6484800\",\"6485350\",\"27113\",\"27114\"],\"contract_product_groupings\":[\"FBBKFSFS\",\"fgc5-10\",\"fgpbr\",\"fglfgfs\",\"fgpcfs\",\"FGHCFDSVC\",\"fgb5alob\",\"fgmbk\",\"fg32b-fs\",\"ALLPVTLBL\",\"FGMGLBLKFS\",\"fgcpc\",\"fgpbk\",\"fgcpfsn\"],\"contract_lumpsums\":null},{\"start_date\":\"2016-02-01\",\"end_date\":\"2016-12-31\",\"contract_version_id\":\"1000026.R2.A7\",\"contract_name\":\"FSA Corporate EMI Azar brand\",\"contract_state\":\"EXPIRED\",\"contract_state_last_modified\":\"2016-12-31\",\"contract_products\":[\"27114\",\"14226\",\"14232\",\"6092633\",\"27113\",\"27106\",\"14222\",\"17558\",\"27112\",\"14231\",\"PFC6538\",\"14225\",\"644\",\"642\",\"14230\",\"14236\",\"640\",\"14229\",\"27104\",\"14235\",\"14223\",\"PFC6528\",\"27110\",\"27115\",\"14234\",\"14237\",\"PFC6508\",\"14228\",\"14233\",\"14224\",\"27108\"],\"contract_product_groupings\":[\"fgpbk\",\"fg32b-fs\",\"fgpcfs\",\"fgb5alob\",\"ALLPVTLBL\",\"fgmbk\",\"FGHCFDSVC\",\"FGMGLBLKFS\",\"fgpbr\",\"fgcpfsn\",\"FBBKFSFS\",\"fgc5-10\",\"fgcpc\",\"fglfgfs\"],\"contract_lumpsums\":null},{\"start_date\":\"2016-02-01\",\"end_date\":\"2016-12-31\",\"contract_version_id\":\"1000026.R2.A1\",\"contract_name\":\"FSA Corporate EMI Azar brand\",\"contract_state\":\"AMENDED\",\"contract_state_last_modified\":\"2016-05-18\",\"contract_products\":[\"14233\",\"14224\",\"14223\",\"17558\",\"27110\",\"27114\",\"PFC6528\",\"27104\",\"640\",\"PFC6538\",\"14231\",\"644\",\"27113\",\"14225\",\"PFC6508\",\"14230\",\"14237\",\"642\",\"27106\",\"27115\",\"14226\",\"14228\",\"27112\",\"14229\",\"14235\",\"14222\",\"14232\",\"27108\",\"14236\",\"14234\"],\"contract_product_groupings\":[\"fgb5alob\",\"fgcpc\",\"fg32b-fs\",\"FGHCFDSVC\",\"fgc5-10\",\"fgcpfsn\",\"fglfgfs\",\"fgpcfs\",\"FBBKFSFS\"],\"contract_lumpsums\":null},{\"start_date\":\"2018-07-31\",\"end_date\":\"2021-06-30\",\"contract_version_id\":\"1000026.R6.A0\",\"contract_name\":\"FSA Corporate EMI Azar brand\",\"contract_state\":\"AMENDED\",\"contract_state_last_modified\":\"2019-05-07\",\"contract_products\":[\"640\",\"14222\",\"27104\",\"14228\",\"27110\",\"14230\",\"14224\",\"14225\",\"27113\",\"642\",\"644\",\"6092633\",\"14223\",\"14235\",\"27108\",\"14234\",\"14232\",\"27115\",\"PFC6528\",\"14231\",\"00640\",\"27114\",\"14237\",\"14236\",\"17558\",\"00644\",\"14229\",\"27106\",\"27112\",\"PFC6508\",\"14226\",\"14233\",\"00642\",\"PFC6538\"],\"contract_product_groupings\":[\"fg32b-fs\",\"fglfgfs\",\"fgpcfs\",\"fgcpc\",\"ALLPVTLBL\",\"fgb5alob\",\"fgcpfsn\",\"fgc5-10\",\"fgmbk\",\"fgpbr\",\"FBBKFSFS\",\"fgpbk\",\"FGMGLBLKFS\",\"FGHCFDSVC\"],\"contract_lumpsums\":null},{\"start_date\":\"2016-02-01\",\"end_date\":\"2016-12-31\",\"contract_version_id\":\"1000026.R2.A8\",\"contract_name\":\"FSA Corporate EMI Azar brand\",\"contract_state\":\"CANCELLED\",\"contract_state_last_modified\":\"2016-10-17\",\"contract_products\":[\"14224\",\"27108\",\"27115\",\"14236\",\"PFC6508\",\"14234\",\"14231\",\"27106\",\"14235\",\"27114\",\"6092633\",\"14232\",\"27104\",\"27110\",\"14237\",\"14228\",\"14223\",\"PFC6538\",\"14222\",\"27112\",\"14226\",\"14225\",\"642\",\"14230\",\"14233\",\"17558\",\"14229\",\"27113\",\"644\",\"PFC6528\",\"640\"],\"contract_product_groupings\":[\"fgpcfs\",\"fgpbk\",\"fgmbk\",\"fgpbr\",\"fg32b-fs\",\"fgcpfsn\",\"fgcpc\",\"FGHCFDSVC\",\"FGMGLBLKFS\",\"fgb5alob\",\"FBBKFSFS\",\"fgcps\",\"fgc5-10\",\"ALLPVTLBL\",\"fglfgfs\"],\"contract_lumpsums\":null},{\"start_date\":\"2016-02-01\",\"end_date\":\"2016-12-31\",\"contract_version_id\":\"1000026.R2.A6\",\"contract_name\":\"FSA Corporate EMI Azar brand\",\"contract_state\":\"AMENDED\",\"contract_state_last_modified\":\"2016-08-10\",\"contract_products\":[\"14225\",\"14229\",\"27115\",\"27114\",\"14226\",\"27110\",\"644\",\"14237\",\"14228\",\"17558\",\"27104\",\"27108\",\"14223\",\"642\",\"14234\",\"PFC6508\",\"640\",\"14224\",\"27106\",\"14222\",\"14230\",\"PFC6538\",\"PFC6528\",\"14231\",\"14233\",\"6092633\",\"14236\",\"27112\",\"14235\",\"27113\",\"14232\"],\"contract_product_groupings\":[\"FBBKFSFS\",\"FGMGLBLKFS\",\"fglfgfs\",\"fgpcfs\",\"fgc5-10\",\"fgpbk\",\"ALLPVTLBL\",\"fg32b-fs\",\"fgb5alob\",\"fgcpc\",\"FGHCFDSVC\",\"fgcpfsn\",\"fgpbr\",\"fgmbk\"],\"contract_lumpsums\":null},{\"start_date\":\"2015-09-01\",\"end_date\":\"2016-01-31\",\"contract_version_id\":\"1000026.R1.A0\",\"contract_name\":\"FSA Corporate EMI Azar brand\",\"contract_state\":\"AMENDED\",\"contract_state_last_modified\":\"2016-05-16\",\"contract_products\":[\"14234\",\"14229\",\"14233\",\"14226\",\"14231\",\"14223\",\"14228\",\"14225\",\"644\",\"27115\",\"PFC6528\",\"14235\",\"27113\",\"27110\",\"640\",\"PFC6508\",\"17558\",\"27106\",\"27112\",\"14224\",\"27108\",\"642\",\"14230\",\"14236\",\"PFC6538\",\"27114\",\"14222\",\"27104\",\"14232\",\"14237\"],\"contract_product_groupings\":[\"FGHCFDSVC\",\"fg32b-fs\",\"fgcpfsn\",\"fgcpc\",\"fgpcfs\",\"fglfgfs\",\"fgb5alob\",\"fgc5-10\",\"FBBKFSFS\"],\"contract_lumpsums\":null},{\"start_date\":\"2015-01-01\",\"end_date\":\"2015-08-31\",\"contract_version_id\":\"1000026.R0.A2\",\"contract_name\":\"FSA Corporate EMI Azar brand\",\"contract_state\":\"AMENDED\",\"contract_state_last_modified\":\"2016-08-10\",\"contract_products\":[\"14228\",\"14232\",\"14234\",\"14233\",\"14223\",\"14226\",\"27106\",\"27104\",\"14236\",\"27108\",\"14224\",\"17558\",\"14231\",\"14229\",\"27112\",\"14237\",\"14230\",\"27113\",\"14235\",\"14222\",\"27110\",\"14225\",\"PFC6508\",\"644\",\"27114\",\"642\",\"PFC6538\",\"640\",\"27115\",\"PFC6528\"],\"contract_product_groupings\":[\"fgb5alob\",\"fgcpc\",\"fgpcfs\",\"fgcpfsn\",\"FGHCFDSVC\",\"fglfgfs\",\"fgc5-10\",\"fg32b-fs\",\"fgmbk\",\"FBBKFSFS\"],\"contract_lumpsums\":null},{\"start_date\":\"2015-09-01\",\"end_date\":\"2016-01-31\",\"contract_version_id\":\"1000026.R1.A1\",\"contract_name\":\"FSA Corporate EMI Azar brand\",\"contract_state\":\"AMENDED\",\"contract_state_last_modified\":\"2016-05-18\",\"contract_products\":[\"14223\",\"27110\",\"642\",\"14233\",\"640\",\"27108\",\"14236\",\"14226\",\"14228\",\"27104\",\"14231\",\"644\",\"PFC6538\",\"27106\",\"PFC6528\",\"14224\",\"27115\",\"14234\",\"27112\",\"14237\",\"14235\",\"14225\",\"27114\",\"14222\",\"14229\",\"PFC6508\",\"14230\",\"17558\",\"27113\",\"14232\"],\"contract_product_groupings\":[\"fg32b-fs\",\"fgpcfs\",\"fglfgfs\",\"fgcpc\",\"fgb5alob\",\"FBBKFSFS\",\"fgc5-10\",\"FGHCFDSVC\",\"fgmbk\",\"fgcpfsn\"],\"contract_lumpsums\":null},{\"start_date\":\"2018-06-01\",\"end_date\":\"2018-07-30\",\"contract_version_id\":\"1000026.R5.A0\",\"contract_name\":\"FSA Corporate EMI Azar brand\",\"contract_state\":\"AMENDED\",\"contract_state_last_modified\":\"2018-08-01\",\"contract_products\":[\"PFC6538\",\"27112\",\"14228\",\"27110\",\"27108\",\"14229\",\"00640\",\"14232\",\"00644\",\"14235\",\"642\",\"14223\",\"27113\",\"00642\",\"14236\",\"14226\",\"6092633\",\"27114\",\"14224\",\"27115\",\"17558\",\"PFC6508\",\"14230\",\"644\",\"14222\",\"14237\",\"14231\",\"27106\",\"640\",\"27104\",\"14234\",\"PFC6528\",\"14233\",\"14225\"],\"contract_product_groupings\":[\"fglfgfs\",\"fg32b-fs\",\"ALLPVTLBL\",\"fgmbk\",\"fgpcfs\",\"fgcpc\",\"fgc5-10\",\"fgpbk\",\"fgb5alob\",\"fgpbr\",\"FBBKFSFS\",\"fgcpfsn\",\"FGHCFDSVC\",\"FGMGLBLKFS\"],\"contract_lumpsums\":null},{\"start_date\":\"2015-09-01\",\"end_date\":\"2016-01-31\",\"contract_version_id\":\"1000026.R1.A2\",\"contract_name\":\"FSA Corporate EMI Azar brand\",\"contract_state\":\"AMENDED\",\"contract_state_last_modified\":\"2016-06-15\",\"contract_products\":[\"27110\",\"14226\",\"14231\",\"14224\",\"14230\",\"14229\",\"14228\",\"17558\",\"14233\",\"27104\",\"14232\",\"640\",\"644\",\"14223\",\"642\",\"PFC6528\",\"27114\",\"14234\",\"14237\",\"27115\",\"14225\",\"14222\",\"PFC6538\",\"27112\",\"27106\",\"PFC6508\",\"14235\",\"27108\",\"14236\",\"27113\"],\"contract_product_groupings\":[\"fgb5alob\",\"fglfgfs\",\"FGHCFDSVC\",\"fgc5-10\",\"fg32b-fs\",\"fgpcfs\",\"fgcpc\",\"fgmbk\",\"fgcpfsn\",\"FBBKFSFS\"],\"contract_lumpsums\":null},{\"start_date\":\"2017-01-01\",\"end_date\":\"2018-02-28\",\"contract_version_id\":\"1000026.R3.A2\",\"contract_name\":\"FSA Corporate EMI Azar brand\",\"contract_state\":\"EXPIRED\",\"contract_state_last_modified\":\"2018-02-22\",\"contract_products\":[\"14234\",\"14225\",\"14233\",\"27110\",\"27106\",\"640\",\"PFC6528\",\"14231\",\"14232\",\"17558\",\"14230\",\"27112\",\"27108\",\"PFC6508\",\"27114\",\"642\",\"6092633\",\"27104\",\"644\",\"00644\",\"14224\",\"27115\",\"14235\",\"27113\",\"00642\",\"14228\",\"14236\",\"00640\",\"14222\",\"14226\",\"14229\",\"14237\",\"PFC6538\",\"14223\"],\"contract_product_groupings\":[\"ALLPVTLBL\",\"fgpbr\",\"fgpbk\",\"fgb5alob\",\"fgc5-10\",\"fgcpfsn\",\"FGHCFDSVC\",\"fglfgfs\",\"FGMGLBLKFS\",\"fgmbk\",\"fg32b-fs\",\"fgpcfs\",\"fgcpc\",\"FBBKFSFS\"],\"contract_lumpsums\":null},{\"start_date\":\"2015-09-01\",\"end_date\":\"2016-01-31\",\"contract_version_id\":\"1000026.R1.A3\",\"contract_name\":\"FSA Corporate EMI Azar brand\",\"contract_state\":\"AMENDED\",\"contract_state_last_modified\":\"2016-08-04\",\"contract_products\":[\"14235\",\"14223\",\"14236\",\"27112\",\"14226\",\"PFC6528\",\"27115\",\"27110\",\"27106\",\"27108\",\"14228\",\"14237\",\"14225\",\"14233\",\"14231\",\"640\",\"642\",\"27104\",\"27114\",\"PFC6538\",\"17558\",\"27113\",\"14234\",\"PFC6508\",\"14222\",\"644\",\"14224\",\"14229\",\"14230\",\"14232\"],\"contract_product_groupings\":[\"fgcpfsn\",\"fgcpc\",\"fgc5-10\",\"FGHCFDSVC\",\"fglfgfs\",\"fg32b-fs\",\"FBBKFSFS\",\"fgmbk\",\"fgb5alob\",\"fgpcfs\"],\"contract_lumpsums\":null},{\"start_date\":\"2016-02-01\",\"end_date\":\"2016-12-31\",\"contract_version_id\":\"1000026.R2.A0\",\"contract_name\":\"FSA Corporate EMI Azar brand\",\"contract_state\":\"AMENDED\",\"contract_state_last_modified\":\"2016-04-25\",\"contract_products\":[\"14229\",\"14228\",\"14233\",\"14226\",\"PFC6528\",\"27106\",\"14234\",\"14235\",\"27113\",\"27110\",\"640\",\"17558\",\"14225\",\"644\",\"PFC6508\",\"14231\",\"14223\",\"14236\",\"PFC6538\",\"14232\",\"27104\",\"14237\",\"27112\",\"14224\",\"27108\",\"642\",\"14230\",\"14222\",\"27115\",\"27114\"],\"contract_product_groupings\":[\"fgpcfs\",\"fgcpc\",\"fgb5alob\",\"fgbfs\",\"fglfgfs\",\"fg32b-fs\",\"fgcpfsn\",\"fgc5-10\",\"fghcbk\"],\"contract_lumpsums\":null},{\"start_date\":\"2018-03-01\",\"end_date\":\"2018-05-31\",\"contract_version_id\":\"1000026.R4.A0\",\"contract_name\":\"FSA Corporate EMI Azar brand\",\"contract_state\":\"AMENDED\",\"contract_state_last_modified\":\"2018-04-27\",\"contract_products\":[\"14233\",\"27112\",\"00642\",\"644\",\"14229\",\"14228\",\"6092633\",\"27113\",\"14231\",\"27114\",\"14226\",\"642\",\"27108\",\"14234\",\"17558\",\"14222\",\"14237\",\"14225\",\"14235\",\"27106\",\"27110\",\"00644\",\"PFC6538\",\"14236\",\"00640\",\"14224\",\"27104\",\"14232\",\"14223\",\"14230\",\"640\",\"27115\",\"PFC6528\",\"PFC6508\"],\"contract_product_groupings\":[\"fgpbk\",\"fgb5alob\",\"FGMGLBLKFS\",\"fgc5-10\",\"fg32b-fs\",\"fgpcfs\",\"FBBKFSFS\",\"fglfgfs\",\"fgmbk\",\"FGHCFDSVC\",\"fgcpfsn\",\"fgpbr\",\"ALLPVTLBL\",\"fgcpc\"],\"contract_lumpsums\":null},{\"start_date\":\"2016-02-01\",\"end_date\":\"2016-12-31\",\"contract_version_id\":\"1000026.R2.A3\",\"contract_name\":\"FSA Corporate EMI Azar brand\",\"contract_state\":\"AMENDED\",\"contract_state_last_modified\":\"2016-08-04\",\"contract_products\":[\"14223\",\"27106\",\"PFC6508\",\"27104\",\"14232\",\"644\",\"14230\",\"14237\",\"27115\",\"14235\",\"14233\",\"14226\",\"17558\",\"14224\",\"14234\",\"14228\",\"642\",\"14222\",\"PFC6528\",\"27114\",\"27110\",\"640\",\"27108\",\"PFC6538\",\"27112\",\"14231\",\"14225\",\"27113\",\"14236\",\"14229\"],\"contract_product_groupings\":[\"fgcpc\",\"fgcpfsn\",\"FBBKFSFS\",\"fglfgfs\",\"fgpcfs\",\"fgb5alob\",\"fg32b-fs\",\"fgmbk\",\"FGHCFDSVC\",\"fgc5-10\"],\"contract_lumpsums\":null},{\"start_date\":\"2015-01-01\",\"end_date\":\"2015-08-31\",\"contract_version_id\":\"1000026.R0.A0\",\"contract_name\":\"FSA Corporate EMI Azar brand\",\"contract_state\":\"AMENDED\",\"contract_state_last_modified\":\"2016-05-16\",\"contract_products\":null,\"contract_product_groupings\":[\"fgpcfs\",\"fgb5alob\",\"fghcbk\",\"fgcpc\",\"fgbfs\",\"fg32b-fs\",\"fgcpfsn\",\"fglfgfs\",\"fgc5-10\"],\"contract_lumpsums\":null},{\"start_date\":\"2016-02-01\",\"end_date\":\"2016-12-31\",\"contract_version_id\":\"1000026.R2.A2\",\"contract_name\":\"FSA Corporate EMI Azar brand\",\"contract_state\":\"AMENDED\",\"contract_state_last_modified\":\"2016-07-11\",\"contract_products\":[\"14222\",\"644\",\"14229\",\"14234\",\"27113\",\"14233\",\"27108\",\"27112\",\"14226\",\"14231\",\"14223\",\"PFC6508\",\"14230\",\"27114\",\"PFC6528\",\"642\",\"14236\",\"27106\",\"14225\",\"14237\",\"14235\",\"27110\",\"27104\",\"14224\",\"14228\",\"14232\",\"17558\",\"640\",\"27115\",\"PFC6538\"],\"contract_product_groupings\":[\"fgc5-10\",\"fglfgfs\",\"fgcpfsn\",\"fgpcfs\",\"fgcpc\",\"fgb5alob\",\"FGHCFDSVC\",\"fg32b-fs\",\"FBBKFSFS\"],\"contract_lumpsums\":null},{\"start_date\":\"2015-01-01\",\"end_date\":\"2015-08-31\",\"contract_version_id\":\"1000026.R0.A3\",\"contract_name\":\"FSA Corporate EMI Azar brand\",\"contract_state\":\"EXPIRED\",\"contract_state_last_modified\":\"2016-08-10\",\"contract_products\":[\"14236\",\"27108\",\"PFC6508\",\"14224\",\"14231\",\"14229\",\"17558\",\"27112\",\"14237\",\"14230\",\"644\",\"27113\",\"27104\",\"14235\",\"27114\",\"PFC6538\",\"14222\",\"27110\",\"642\",\"14225\",\"14232\",\"14234\",\"640\",\"14228\",\"27115\",\"14233\",\"14223\",\"14226\",\"PFC6528\",\"27106\",\"6092633\"],\"contract_product_groupings\":[\"fgc5-10\",\"fglfgfs\",\"fgb5alob\",\"fgcpfsn\",\"fgcpc\",\"FBBKFSFS\",\"fg32b-fs\",\"FGHCFDSVC\",\"fgmbk\",\"fgpcfs\"],\"contract_lumpsums\":null},{\"start_date\":\"2018-06-01\",\"end_date\":\"2018-07-30\",\"contract_version_id\":\"1000026.R5.A1\",\"contract_name\":\"FSA Corporate EMI Azar brand\",\"contract_state\":\"EXPIRED\",\"contract_state_last_modified\":\"2018-08-01\",\"contract_products\":[\"640\",\"27106\",\"14224\",\"27108\",\"14235\",\"14222\",\"14233\",\"14234\",\"642\",\"PFC6528\",\"14236\",\"27115\",\"17558\",\"27110\",\"PFC6508\",\"00640\",\"27112\",\"14231\",\"27113\",\"27104\",\"14225\",\"14237\",\"6092633\",\"644\",\"PFC6538\",\"00644\",\"14228\",\"14230\",\"14223\",\"14226\",\"14232\",\"27114\",\"00642\",\"14229\"],\"contract_product_groupings\":[\"FGHCFDSVC\",\"fg32b-fs\",\"fgpcfs\",\"fgb5alob\",\"FGMGLBLKFS\",\"fgmbk\",\"ALLPVTLBL\",\"fgc5-10\",\"FBBKFSFS\",\"fgcpfsn\",\"fgcpc\",\"fgpbr\",\"fgpbk\",\"fglfgfs\"],\"contract_lumpsums\":null},{\"start_date\":\"2015-09-01\",\"end_date\":\"2016-01-31\",\"contract_version_id\":\"1000026.R1.A4\",\"contract_name\":\"FSA Corporate EMI Azar brand\",\"contract_state\":\"AMENDED\",\"contract_state_last_modified\":\"2016-08-10\",\"contract_products\":[\"27104\",\"27114\",\"PFC6538\",\"17558\",\"27113\",\"14234\",\"PFC6508\",\"14222\",\"644\",\"14232\",\"14224\",\"14229\",\"14230\",\"14235\",\"14223\",\"14236\",\"27112\",\"14226\",\"PFC6528\",\"27115\",\"27110\",\"27106\",\"27108\",\"14228\",\"14237\",\"14225\",\"14233\",\"14231\",\"640\",\"642\"],\"contract_product_groupings\":[\"fgcpc\",\"fgmbk\",\"FGMGLBLKFS\",\"fglfgfs\",\"fgc5-10\",\"FBBKFSFS\",\"fgcpfsn\",\"FGHCFDSVC\",\"fgb5alob\",\"fgpcfs\",\"fg32b-fs\"],\"contract_lumpsums\":null},{\"start_date\":\"2016-02-01\",\"end_date\":\"2016-12-31\",\"contract_version_id\":\"1000026.R2.A5\",\"contract_name\":\"FSA Corporate EMI Azar brand\",\"contract_state\":\"AMENDED\",\"contract_state_last_modified\":\"2016-08-05\",\"contract_products\":[\"14234\",\"27112\",\"14225\",\"644\",\"14223\",\"14232\",\"14236\",\"14237\",\"27110\",\"640\",\"14235\",\"27115\",\"14233\",\"PFC6508\",\"27114\",\"27108\",\"PFC6538\",\"14226\",\"14230\",\"642\",\"27106\",\"14224\",\"27104\",\"27113\",\"14229\",\"PFC6528\",\"17558\",\"14231\",\"14222\",\"6092633\",\"14228\"],\"contract_product_groupings\":[\"fgcpfsn\",\"FGMGLBLKFS\",\"fgc5-10\",\"fgb5alob\",\"FBBKFSFS\",\"fg32b-fs\",\"fgmbk\",\"fgcpc\",\"FGHCFDSVC\",\"fglfgfs\",\"fgpcfs\",\"fgpbr\"],\"contract_lumpsums\":null},{\"start_date\":\"2015-01-01\",\"end_date\":\"2015-08-31\",\"contract_version_id\":\"1000026.R0.A1\",\"contract_name\":\"FSA Corporate EMI Azar brand\",\"contract_state\":\"AMENDED\",\"contract_state_last_modified\":\"2016-05-16\",\"contract_products\":[\"17558\",\"14237\",\"14235\",\"27104\",\"14226\",\"14234\",\"27108\",\"14236\",\"PFC6528\",\"644\",\"27114\",\"14230\",\"642\",\"PFC6508\",\"640\",\"14224\",\"14228\",\"14231\",\"14225\",\"14232\",\"27115\",\"27112\",\"27110\",\"14233\",\"14223\",\"PFC6538\",\"14222\",\"14229\",\"27113\",\"27106\"],\"contract_product_groupings\":[\"fgcpc\",\"fg32b-fs\",\"fglfgfs\",\"fghcbk\",\"fgc5-10\",\"fgb5alob\",\"fgbfs\",\"fgpcfs\",\"fgcpfsn\"],\"contract_lumpsums\":null},{\"start_date\":\"2017-01-01\",\"end_date\":\"2018-02-28\",\"contract_version_id\":\"1000026.R3.A0\",\"contract_name\":\"FSA Corporate EMI Azar brand\",\"contract_state\":\"AMENDED\",\"contract_state_last_modified\":\"2017-02-02\",\"contract_products\":[\"27113\",\"27106\",\"14237\",\"14230\",\"27108\",\"14224\",\"14228\",\"644\",\"PFC6528\",\"27114\",\"PFC6538\",\"14236\",\"PFC6508\",\"17558\",\"14233\",\"14229\",\"14232\",\"6092633\",\"642\",\"27115\",\"14222\",\"640\",\"14235\",\"27110\",\"14225\",\"14226\",\"27104\",\"27112\",\"14234\",\"14231\",\"14223\"],\"contract_product_groupings\":[\"ALLPVTLBL\",\"fgcpc\",\"fgcpfsn\",\"FGMGLBLKFS\",\"fgpbr\",\"FBBKFSFS\",\"fgb5alob\",\"fgc5-10\",\"fglfgfs\",\"fg32b-fs\",\"FGHCFDSVC\",\"fgpcfs\",\"fgmbk\",\"fgpbk\"],\"contract_lumpsums\":null}]}"}]}

If I go and convert the jsonl file to a parquet file and try and load that instead, the train/split generation of the dataset starts...but then I get a different error:

Traceback (most recent call last):
  File "/home/user/IdeaProjects/tevatron/examples/colbert/train_colbert.py", line 102, in <module>
    main()
  File "/home/user/IdeaProjects/tevatron/examples/colbert/train_colbert.py", line 79, in main
    train_dataset = HFTrainDataset(tokenizer=tokenizer, data_args=data_args,
  File "/home/user/IdeaProjects/tevatron/src/tevatron/datasets/dataset.py", line 24, in __init__
    self.dataset = load_dataset(data_args.dataset_name,
  File "/home/user/IdeaProjects/tevatron/venv/lib/python3.10/site-packages/datasets/load.py", line 2549, in load_dataset
    builder_instance.download_and_prepare(
  File "/home/user/IdeaProjects/tevatron/venv/lib/python3.10/site-packages/datasets/builder.py", line 1005, in download_and_prepare
    self._download_and_prepare(
  File "/home/user/IdeaProjects/tevatron/venv/lib/python3.10/site-packages/datasets/builder.py", line 1118, in _download_and_prepare
    verify_splits(self.info.splits, split_dict)
  File "/home/user/IdeaProjects/tevatron/venv/lib/python3.10/site-packages/datasets/utils/info_utils.py", line 92, in verify_splits
    raise ExpectedMoreSplits(str(set(expected_splits) - set(recorded_splits)))
datasets.utils.info_utils.ExpectedMoreSplits: {'dl20', 'dl19', 'dev'}

Would really appreciate some help with what is going on here.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
1 participant
@Tostino