Skip to content

Latest commit

 

History

History
20 lines (15 loc) · 2.34 KB

2312.00894.md

File metadata and controls

20 lines (15 loc) · 2.34 KB
Raw

Background

  • Background The widespread use and increasing complexity of REST APIs necessitate improved automated testing tools, which often overlook valuable unstructured natural language descriptions in API specifications, leading to suboptimal test coverage.

  • Existing Work Existing technologies like NLP2REST and ARTE attempt to address this gap by extracting rules from human-readable descriptions and querying knowledge bases for meaningful input values, but they are limited in the types of rules they can extract and often produce inaccurate results.

Core Contributions

  • Introduced RESTGPT

    • Challenge 1: Accuracy and quality of rule extraction and input value generation RESTGPT harnesses the capabilities of Large Language Models (LLMs), particularly the GPT-3.5 Turbo, to improve REST API testing. By parsing OpenAPI specifications, extracting machine-interpretable rules, and generating example parameter values from natural-language descriptions within the specifications, RESTGPT ensures that the outputs are both relevant and precise.

    • Challenge 2: Ensuring extensibility while seamlessly integrating with existing API specifications During the specification enhancement phase, RESTGPT extracts machine-readable components from API specifications and integrates the extracted rules into these components during the building phase, enriching and enhancing the API specification for more robust and informed testing.

Implementation and Deployment

Preliminary results indicate RESTGPT's significant improvement over existing approaches. Compared to NLP2REST without a validation module, RESTGPT improved precision from 50% to 97%. Against NLP2REST with its validation module, precision still rose from 79% to 97%. Additionally, RESTGPT was able to generate syntactically and semantically valid inputs for 73% of parameters across analyzed services and operations, a notable increase from the 17% by ARTE. These results validate the efficacy of RESTGPT and its potential in leveraging LLMs for REST API testing improvements.

Summary

RESTGPT addresses the limitations of existing methods in extracting rules from natural language descriptions and generating effective values by leveraging the accuracy and efficiency of LLMs, especially GPT-3.5 Turbo, significantly enhancing the quality and accuracy of REST API testing.