Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Added support for multi-config api calls + added tests #307

Merged
merged 4 commits into from
Feb 14, 2024

Conversation

drazvan
Copy link
Collaborator

@drazvan drazvan commented Feb 6, 2024

by @makeshn

Added initial support for multi_config api calls using the server API.

Changelog

nemoguardrails/rails/llm/config.py

  • Overload '+' operator to add together two RailsConfig objects. ex. c = a + b. The attributes of b will overwrite attributes of a.
  • Create helper function _join_rails_configs to combine two RailsConfig objects. Follows the logic of _join_config method. We have two hard checks in this function - we require the action_server_url to be the same, we require that if model_type is the same across configs, then the model_engine and model attributes must match.
  • Create helper function _update_rails_config function to handle combining config.rails - FactChecking, Input, Output, SensitiveDataDetection, Retrieval. For Input, Output and Retrieval rails, we append together the flows. For the others, we check if the attributes in the UpdatedRailsConfig is set/empty. If set, it overwrites the BaseRailsConfig attribute, if empty - we just retain the BaseRailsConfig.

nemoguardrails/server/api.py

  • Add config_ids field to RequestBody - List of strs.
  • Modify _get_rails function to operate on list of strs. We generate a cache key to be concatenation of configs passed. The configs are combined two at a time to get the full_llm_config.
  • LLMRails object is constructed from full_llm_config and used for the app.

nemoguardrails/actions/action_dispatcher.py

  • Loop over all config paths and Load actions.

Signed-off-by: Makesh Narsimhan Sreedhar makeshn@nvidia.com

@drazvan drazvan self-assigned this Feb 6, 2024
Copy link
Collaborator Author

@drazvan drazvan left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@makeshn:

While this approach works, it adds a lot of duplication in terms of the joining logic. A RailsConfig object can be transformed into a dict using the .dict() method, and then the existing _join_config method can be used. Finally, the joined RailsConfig object can be reconstructed using RailsConfig(**joined_dict). This approach should also simplify the code significantly.

@makeshn makeshn force-pushed the feature/multi_config_api_call branch from e1cd9fd to cc8b447 Compare February 9, 2024 17:25
@drazvan drazvan merged commit aded7f0 into develop Feb 14, 2024
4 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants