Thank you for your interest in contributing to our LLM Evaluation Catalogue. The catalogue organizes LLM evaluation and testing approaches according to our proposed taxonomy and seeks to provided an overview of the LLM evaluation landscape. It is intended to be a useful resource for an organisation that is considering developing a LLM or deploying an LLM-driven application.
Every submission must be classified under an existing taxonomy sub-category. Please ensure you:
- Identify the Appropriate Sub-category
- Proposal for New Categories: If your submission does not fit within the existing taxonomy, you may propose a new category or sub-category. In such cases, please provide a justification for its inclusion. This should include:
- Necessity: Explain why existing categories do not adequately cover your submission.
- Scope and Definition: Provide a clear definition and scope for the proposed category or sub-category.
- Distinctiveness: Elucidate how this new category or sub-category offers unique value to the taxonomy.
Every submission must be accompanied by a publicly accessible link to the original source. This link should lead directly to the material relevant to the LLM evaluation or testing approach you are submitting (e.g. paper, benchmark, etc).
Your submission should include the following details:
- Abstract/Summary: A concise overview of the LLM evaluation or testing approach.
- Methodology: An explanation of the testing approaches (e.g. benchmarks, manual red teaming) and scoring methods (e.g. algorithimic scoring) employed in the evaluation or testing.
Should you have any questions or require further clarification on the submission process, please do not hesitate to contact us at info@aiverify.sg.
Happy Contributing!