From 29f6e2bf69a32e50314655549f33086026ed3536 Mon Sep 17 00:00:00 2001 From: John Matthews Date: Thu, 25 Jul 2024 16:57:48 -0400 Subject: [PATCH] :book: Reworking docs to breakout main README.md (#254) Signed-off-by: John Matthews --- CONTRIBUTING.md | 179 ++++++++ README.md | 492 ++-------------------- ROADMAP.md | 1 + docs/Evaluation_Builds.md | 38 ++ docs/Example_CLI_Script.md | 22 + docs/Getting_Started.md | 94 +++++ docs/LLM_Selection.md | 170 ++++++++ docs/contrib/Demo_Mode.md | 48 +++ docs/contrib/Dev_Environment.md | 79 ++++ docs/contrib/Testing.md | 7 + docs/contrib/Tracing.md | 70 +++ docs/{ => design}/HubIntegration.md | 0 docs/design/Technical_Background.md | 53 +++ docs/{ => design}/solvedIncidentStore.md | 0 {images => docs/images}/Kai_April_26c.gif | Bin docs/{ => scenarios}/demo.md | 0 docs/{ => scenarios}/deploy.gif | Bin docs/{ => scenarios}/ejb_remote.png | Bin docs/{ => scenarios}/kai_type2.png | Bin docs/{ => scenarios}/mdb.png | Bin docs/{ => scenarios}/mdbchanges.png | Bin docs/{ => scenarios}/model.png | Bin example/README.md | 12 +- 23 files changed, 812 insertions(+), 453 deletions(-) create mode 100644 CONTRIBUTING.md create mode 100644 ROADMAP.md create mode 100644 docs/Evaluation_Builds.md create mode 100644 docs/Example_CLI_Script.md create mode 100644 docs/Getting_Started.md create mode 100644 docs/LLM_Selection.md create mode 100644 docs/contrib/Demo_Mode.md create mode 100644 docs/contrib/Dev_Environment.md create mode 100644 docs/contrib/Testing.md create mode 100644 docs/contrib/Tracing.md rename docs/{ => design}/HubIntegration.md (100%) create mode 100644 docs/design/Technical_Background.md rename docs/{ => design}/solvedIncidentStore.md (100%) rename {images => docs/images}/Kai_April_26c.gif (100%) rename docs/{ => scenarios}/demo.md (100%) rename docs/{ => scenarios}/deploy.gif (100%) rename docs/{ => scenarios}/ejb_remote.png (100%) rename docs/{ => scenarios}/kai_type2.png (100%) rename docs/{ => scenarios}/mdb.png (100%) rename docs/{ => scenarios}/mdbchanges.png (100%) rename docs/{ => scenarios}/model.png (100%) diff --git a/CONTRIBUTING.md b/CONTRIBUTING.md new file mode 100644 index 00000000..431d84d8 --- /dev/null +++ b/CONTRIBUTING.md @@ -0,0 +1,179 @@ +# Contributing Guide + +- [New Contributor Guide](#contributing-guide) + - [Ways to Contribute](#ways-to-contribute) + - [Find an Issue](#find-an-issue) + - [Ask for Help](#ask-for-help) + - [Pull Request Lifecycle](#pull-request-lifecycle) + - [Development Environment Setup](#development-environment-setup) + - [Sign Your Commits](#sign-your-commits) + - [Pull Request Checklist](#pull-request-checklist) + +Welcome! We are glad that you want to contribute to our project! πŸ’– + +As you get started, you are in the best position to give us feedback on areas of +our project that we need help with including: + +- Problems found during setting up a new developer environment +- Gaps in our Quickstart Guide or documentation +- Bugs in our automation scripts + +If anything doesn't make sense, or doesn't work when you run it, please open a +bug report and let us know! + +## Ways to Contribute + +We welcome many different types of contributions including: + +- New features +- Builds, CI/CD +- Bug fixes +- Documentation +- Issue Triage +- Answering questions on Slack/Mailing List +- Web design +- Communications / Social Media / Blog Posts +- Release management + +Not everything happens through a GitHub pull request. Please come to our +[meetings](#come-to-meetings) or [contact us](#contact-us) and let's discuss how we can work together. + +### Come to Meetings + +Please consider joining the [Konveyor community meetings](https://github.com/konveyor/community?tab=readme-ov-file#konveyor-community-meetings). + +Absolutely everyone is welcome to come to any of our meetings. You never need an +invite to join us. In fact, we want you to join us, even if you don’t have +anything you feel like you want to contribute. Just being there is enough! + +You can find out more about our meetings [here](https://github.com/konveyor/community?tab=readme-ov-file#konveyor-community-meetings). You don’t have to turn on +your video. The first time you come, introducing yourself is more than enough. +Over time, we hope that you feel comfortable voicing your opinions, giving +feedback on others’ ideas, and even sharing your own ideas, and experiences. + +## Contact Us + +### Slack + +You can reach us in kubernetes.slack.com in : + +- [#konveyor](https://kubernetes.slack.com/archives/CR85S82A2) +- [#konveyor-dev](https://kubernetes.slack.com/archives/C04QZJFQ0UA) + +If you don't already have a slack account for kubernetes.slack.com you can receive an automatic invite via: https://communityinviter.com/apps/kubernetes/community + +### Mailing list + +Subscribe to the [Konveyor emailing lists](https://groups.google.com/g/konveyor-dev +https://github.com/konveyor/community?tab=readme-ov-file#mailing-lists) + +For technical discussions please join/email: + +- konveyor-dev@googlegroups.com + - Konveyor development related issues and general questions + - Subscribe via: https://groups.google.com/g/konveyor-dev + +## Find an Issue + +We have good first issues for new contributors and help wanted issues suitable +for any contributor. [good first issue](https://github.com/konveyor/kai/issues?q=is%3Aissue+is%3Aopen+label%3A%22good+first+issue%22) has extra information to +help you make your first contribution. [help wanted](https://github.com/konveyor/kai/labels/help%20wanted) are issues +suitable for someone who isn't a core maintainer and is good to move onto after +your first pull request. + +Sometimes there won’t be any issues with these labels. That’s ok! There is +likely still something for you to work on. If you want to contribute but you +don’t know where to start or can't find a suitable issue, you can reach out to us in [slack](#slack) + +Once you see an issue that you'd like to work on, please post a comment saying +that you want to work on it. Something like "I want to work on this" is fine. + +## Ask for Help + +The best way to reach us with a question when contributing is to ask on: + +- The original github issue +- [The developer mailing list](https://groups.google.com/u/1/g/konveyor-dev) +- [Our Slack channel - #konveyor-dev](https://kubernetes.slack.com/archives/C04QZJFQ0UA) + +## Pull Request Lifecycle + +- Please submit pull-requests for the 'main' branch +- Please link your PR to an existing issue, if an existing issue is not present consider creating a new issue +- If the PR is not yet ready you may mark it as a draft, and then once it's ready for a review remove the draft status and add a comment to let us know you are ready for a review. + +## Pull Request Title + +- Please ensure the title of your PR begins with a [gitemoji](https://github.com/carloscuesta/gitmoji) such as + - `:bug` - For bug fixes + - `:book` - For documentation + - `:sparkles` - For new features + - `:seedling` - For infrastructure related changes + - `:warning` - For breaking changes + - `:ghost` - For misc updates/fixes that don't need to show up in release notes + - For more info you can consult the pr check we run at [konveyor/release-tools](https://github.com/konveyor/release-tools/blob/main/pkg/pr/prefix.go) + +## Development Environment Setup + +See [docs/contrib/Dev_Environment.md](docs/contrib/Dev_Environment.md) + +## Linting + +1. Install trunk via: https://docs.trunk.io/check#install-the-cli +1. Run the linters: `trunk check` +1. Format code: `trunk fmt` + +## Testing + +- Please include a unit test for new features +- See [docs/contrib/Testing.md](docs/contrib/Testing.md) for more guidance on testing + +## Updating requirements.txt + +- If you are a developer working on Kai and you are updating requirements.txt, you will need to do some manual changes beyond just a `pip freeze &> ./requirements.txt`, we have a few directives that address differences in 'darwin' systems that need to be preserved. These need to be added manually after a 'freeze' as the freeze command is not aware of what exists in requirements.txt. Please consult the diff of changes you are making now from prior version and note the extra directions for `python_version` and or `sys_platform` + +## Sign Your Commits + +### DCO + +Licensing is important to open source projects. It provides some assurances that +the software will continue to be available based under the terms that the +author(s) desired. We require that contributors sign off on commits submitted to +our project's repositories. The [Developer Certificate of Origin +(DCO)](https://probot.github.io/apps/dco/) is a way to certify that you wrote and +have the right to contribute the code you are submitting to the project. + +You sign-off by adding the following to your commit messages. Your sign-off must +match the git user and email associated with the commit. + + This is my commit message + + Signed-off-by: Your Name + +Git has a `-s` command line option to do this automatically: + + git commit -s -m 'This is my commit message' + +If you forgot to do this and have not yet pushed your changes to the remote +repository, you can amend your commit with the sign-off by running + + git commit --amend -s + +## Pull Request Checklist + +When you submit your pull request, or you push new commits to it, our automated +systems will run some checks on your new code. We require that your pull request +passes these checks, but we also have more criteria than just that before we can +accept and merge it. We recommend that you check the following things locally +before you submit your code: + +- Ensure that [trunk](https://docs.trunk.io/code-quality/advanced-setup/cli) is happy + - `trunk check` + - `trunk fmt` +- Ensure that unit tests pass + - See [docs/contrib/Testing.md](docs/contrib/Testing.md) +- If adding a new feature please add a new unit test +- If you modified `requirements.txt` please see [updating requirements.txt](#updating-requirementstxt) +- Ensure that [`example/run_demo.py`](example/run_demo.py) works +- Commits are signed as per [DCO](#dco) +- PR Title begins with a gitemoji as described in [Pull Request Title](#pull-request-title) diff --git a/README.md b/README.md index 6873b313..8982f212 100644 --- a/README.md +++ b/README.md @@ -1,485 +1,79 @@ -# Konveyor AI (kai) +# Konveyor AI (Kai) -Konveyor AI (kai) is Konveyor's approach to easing modernization of application source code to a new target by leveraging LLMs with guidance from static code analysis augmented with data in Konveyor that helps to learn how an Organization solved a similar problem in the past. +Konveyor AI (Kai) simplifies the process of modernizing application source code to a new platform. It uses Large Language Models (LLMs) guided by static code analysis, along with data from Konveyor. This data provides insights into how similar problems were solved by the organization in the past, helping streamline and automate the code modernization process. -Pronunciation of 'kai': https://www.howtopronounce.com/kai - -## Blog Posts - -- 2024 May 07: [Apply generative AI to app modernization with Konveyor AI](https://developers.redhat.com/articles/2024/05/07/modernize-apps-konveyor-ai) -- 2024 May 07: [Kai - Generative AI Applied to Application Modernization](https://www.konveyor.io/blog/kai-deep-dive-2024/) +Pronunciation of 'kai': https://www.howtopronounce.com/ka%C3%AC-4 ## Approach -Our approach is to use static code analysis to find the areas in source code that need to be transformed. 'kai' will iterate through analysis information and work with LLMs to generate code changes to resolve incidents identified from analysis. - -This approach does _not_ require fine-tuning of LLMs, we augment a LLMs knowledge via the prompt, similar to approaches with [RAG](https://arxiv.org/abs/2005.11401) by leveraging external data from inside of Konveyor and from Analysis Rules to aid the LLM in constructing better results. - -For example, [analyzer-lsp Rules](https://github.com/konveyor/analyzer-lsp/blob/main/docs/rules.md) such as these ([Java EE to Quarkus rulesets](https://github.com/konveyor/rulesets/tree/main/default/generated/quarkus)) are leveraged to aid guiding a LLM to update a legacy Java EE application to Quarkus - -Note: For purposes of this initial prototype we are using an example of Java EE to Quarkus. That is an arbitrary choice to show viability of this approach. The code and the approach will work on other targets that Konveyor has rules for. - -### What happens technically to make this work? - -- [Konveyor](konveyor.io) contains information related to an Organization's Application Portfolio, a view into all of the applications an Organization is managing. This view includes a history of analysis information over time, access to each applications source repositories, and metadata that tracks work in-progress/completed in regard to each application being migrated to a given technology. - -- When 'Konveyor AI' wants to fix a specific issue in a given application, it will mine data in Konveyor to extract 2 sources of information to inject into a given LLM prompt. - - 1. Static Code Analysis - - - We pinpoint where to begin work by leveraging static code analysis to guide us - - The static code analysis is informed via a collection of crowd sourced knowledge contained in our [rulesets](https://github.com/konveyor/rulesets/tree/main) plus augmented via [custom-rules](https://github.com/konveyor-ecosystem/kai/tree/main/samples/custom_rules) - - We include in the prompt Analysis metadata information to give the LLM more context [such as](https://github.com/konveyor-ecosystem/kai/blob/main/example/analysis/coolstore/output.yaml#L2789) - - remote-ejb-to-quarkus-00000: - description: Remote EJBs are not supported in Quarkus - incidents: - - uri: file:///tmp/source-code/src/main/java/com/redhat/coolstore/service/ShippingService.java - message: "Remote EJBs are not supported in Quarkus, and therefore its use must be removed and replaced with REST functionality. In order to do this:\n 1. Replace the `@Remote` annotation on the class with a `@jakarta.ws.rs.Path(\"\")` annotation. An endpoint must be added to the annotation in place of `` to specify the actual path to the REST service.\n 2. Remove `@Stateless` annotations if present. Given that REST services are stateless by nature, it makes it unnecessary.\n 3. For every public method on the EJB being converted, do the following:\n - Annotate the method with `@jakarta.ws.rs.GET`\n - Annotate the method with `@jakarta.ws.rs.Path(\"\")` and give it a proper endpoint path. As a rule of thumb... " - - lineNumber: 12 - variables: - file: file:///tmp/source-code/src/main/java/com/redhat/coolstore/service/ShippingService.java - kind: Class - name: Stateless - package: com.redhat.coolstore.service - - - url: https://jakarta.ee/specifications/restful-ws/ - title: Jakarta RESTful Web Services - - 1. Solved Examples - these are source code diffs that show a LLM how a similar problem was seen in another application the Organization has and how that Organization decided to fix it. - - - We mine data Konveyor has stored from the Application Hub to search for when other applications have fixed the same rule violations and learn how they fixed it and pass that info into the prompt to aid the LLM - - This ability to leverage how the issue was seen and fixed in the past helps to give the LLM extra context to give a higher quality result. - - This is an [early prompt we created](https://github.com/konveyor-ecosystem/kai/blob/main/notebooks/jms_to_smallrye_reactive/output/gpt-4-1106-preview/helloworldmdb/custom-ruleset/jms-to-reactive-quarkus-00050/few_shot/template.txt) to help give a feel of this in action and the [result we got back from a LLM](https://github.com/konveyor-ecosystem/kai/blob/main/notebooks/jms_to_smallrye_reactive/output/gpt-4-1106-preview/helloworldmdb/custom-ruleset/jms-to-reactive-quarkus-00050/few_shot/result.txt) - -## Pre-Requisites - -### Access to a Large Language Model (LLM) - -- If you want to run Kai against a LLM you will likely need to configure a LLM API Key to access your service (unless running against a local model) - - We do provide a means of running Kai against previously cached data from a few models to aid demo flows. This allows you to run through the steps of using previously cached data without requiring access to a LLM. Note, if you do not provide LLM API access then the DEMO_MODE flow will only be able to replay previous cached responses. - - We call this 'DEMO_MODE', i.e. `DEMO_MODE=true make run-server` -- Note that results vary widely between models. - -#### LLM API Keys - -- We expect that you have configured the environment variables required for the LLM you are attempting to use. - - For example: - - OpenAI service requires: `OPENAI_API_KEY=my-secret-api-key-value` - - IBM BAM service requires: `GENAI_KEY=my-secret-api-key-value` - -#### IBM BAM Service - -- The development team has been using the IBM BAM service to aid development and testing: - - IBM Big AI Model (BAM) laboratory is where IBM Research designs, builds, and iterates on what’s next in foundation models. Our goal is to help accelerate the transition from research to product. Come experiment with us. - - - Login: https://bam.res.ibm.com/ - - In order to use this service an individual needs to obtain a w3id from IBM. The kai development team is unable to help obtaining this access. - - Related client tooling: - - - https://github.com/IBM/ibm-generative-ai - - LangChain integration: https://ibm.github.io/ibm-generative-ai/v2.2.0/rst_source/examples.extensions.langchain.html#examples-extensions-langchain - - - Obtain your API key from IBM BAM: - - - To access via an API you can look at β€˜Documentation’ after logging into https://bam.res.ibm.com/ - - You will see a field embedded in the 'Documentation' section where you can generate/obtain an API Key. - - - Ensure you have `GENAI_KEY=my-secret-api-key-value` defined in your shell - -#### OpenAI Service - -- If you have a valid API Key for OpenAI you may use this with Kai. -- Ensure you have `OPENAI_API_KEY=my-secret-api-key-value` defined in your shell - -##### Selecting a Model - -We offer configuration choices of several models via [config.toml](/kai/config.toml) which line up to choices we know about from [kai/model_provider.py](https://github.com/konveyor-ecosystem/kai/blob/main/kai/model_provider.py). - -To change which llm you are targeting, open `config.toml` and change the `[models]` section to one of the following: - - - -**IBM served granite** - - - - - -```toml -[models] - provider = "ChatIBMGenAI" - - [models.args] - model_id = "ibm/granite-13b-chat-v2" -``` - - - - - -**IBM served mistral** - - - - - -```toml -[models] - provider = "ChatIBMGenAI" - - [models.args] - model_id = "mistralai/mixtral-8x7b-instruct-v01" -``` - - - - - -**IBM served codellama** - - +Kai implements a [Retrieval Augmented Generation (RAG)](https://arxiv.org/abs/2005.11401) approach that leverages data from Konveyor to help generate code suggestions to aid migrating legacy code bases to a different technology. The intent of this RAG approach is to shape the code suggestions to be similar to how an organization has solved problems in the past, without additional fine-tuning of the model. - +The approach begins with using static code analysis via the [Kantra](https://github.com/konveyor/kantra) tool to find areas in the source code that need attention. 'kai' will iterate through analysis information and work with LLMs to generate code changes to resolve incidents identified from analysis. -```toml -[models] - provider = "ChatIBMGenAI" +- Read more about our technical approach here: [docs/Technical_Background.md](docs/design/Technical_Background.md) - [models.args] - model_id = "meta-llama/llama-2-13b-chat" -``` +- Presentation slides introducing Konveyor AI: [2024_May GenAI Application Migration](https://docs.google.com/presentation/d/1awMdp5hHC6L4Xc_uY6Kj4XiskAArDGPhyQRBI6GJUAo/edit#slide=id.g28c0e0d2936_0_621) - +## Demo Video - - -**IBM served llama3** - - - - - -```toml - # Note: llama3 complains if we use more than 2048 tokens - # See: https://github.com/konveyor-ecosystem/kai/issues/172 -[models] - provider = "ChatIBMGenAI" - - [models.args] - model_id = "meta-llama/llama-3-70b-instruct" - parameters.max_new_tokens = 2048 -``` - - - - - -**Ollama** - - - - - -```toml -[models] - provider = "ChatOllama" - - [models.args] - model = "mistral" -``` - - - - - -**OpenAI GPT 4** - - - - - -```toml -[models] - provider = "ChatOpenAI" - - [models.args] - model = "gpt-4" -``` - - - - - -**OpenAI GPT 3.5** - - - - - -```toml -[models] - provider = "ChatOpenAI" - - [models.args] - model = "gpt-3.5-turbo" -``` - - - -Kai will also work with [OpenAI API Compatible alternatives](docs/OpenAI-API-Compatible-Alternatives.md). - -## Setup - -### Podman Compose - -#### Standalone - -1. `git clone https://github.com/konveyor-ecosystem/kai.git` -1. `cd kai` -1. Make changes to `kai/config.toml` to select your desired provider and model -1. Export `GENAI_KEY` or `OPENAI_API_KEY` as appropriate -1. Run `podman compose up`. It will take the server several minutes to start while data is loaded, the first time this is run. - -After the first run the DB will be populated and subsequent starts will be much faster, as long as the kai_kai_db_data volume is not deleted. - -To clean up all resources run `podman compose down && podman volume rm kai_kai_db_data`. - -#### With Konveyor Hub - -1. `git clone https://github.com/konveyor-ecosystem/kai.git` -1. `cd kai` -1. Make changes to `kai/config.toml` to select your desired provider and model -1. Export `GENAI_KEY` or `OPENAI_API_KEY` as appropriate -1. Run `USE_HUB_IMPORTER=True HUB_URL=https://tackle-konveyor-tackle.apps.cluster.example.com/hub IMPORTER_ARGS=-k podman compose --profile use_hub_importer up` - -Note that you need to use podman >= 1.1.0 to use the `--profile` option. podman does not currently support the alternative `COMPOSE_PROFILES` environment variable. - -If your konveyor instance does not use self-signed certificates you may omit `IMPORTER_ARGS=-k`. - -To clean up all resources run `podman compose --profile use_hub_importer down && podman volume rm kai_kai_db_data`. - -### Local Development - -Running Kai's backend involves running 2 processes: - -- Postgres instance which we deliver via container -- Backend REST API server - -#### Steps - -1. Clone Repo and Ensure you have the virtual environment setup - 1. `git clone https://github.com/konveyor-ecosystem/kai.git` - 1. `cd kai` - 1. `python3 -m venv env` - - We've tested this with Python 3.11 and 3.12 - 1. `source env/bin/activate` - 1. `pip install -r ./requirements.txt` - 1. `pip install -e .` -1. Run the Postgres DB via podman - 1. Open a new shell tab - 1. `source env/bin/activate` - 1. Let this run in background: `make run-postgres` -1. Run the Kai server in background - 1. Open a new shell tab - 1. `source env/bin/activate` - 1. Let this run in background: `make run-server` - - If you want to run with cached LLM responses run with `DEMO_MODE=true` - - Replace the above command and instead run: `DEMO_MODE=true make run-server` - - The `DEMO_MODE` option will cache responses and play them back on subsequent runs. - - If you want to run with debug information set the environment variable `LOG_LEVEL=debug` - - Example: `LOG_LEVEL=debug make run-server` -1. Load data into the database - 1. `source env/bin/activate` - 1. Fetch sample apps: `pushd samples; ./fetch_apps.py; popd` - 1. `make load-data` - - This will complete in ~1-2 minutes - -## How to use Kai? - -### Client Usage - -- There are a few ways to use Kai - - IDE usage: See: [Install the Kai VSCode Plugin](https://github.com/konveyor-ecosystem/kai-vscode-plugin/blob/main/docs/user-guide.md) - - CLI that scripts usage to the API - 1. We have a script: [example/run_demo.py](example/run_demo.py) that will look at Kantra analysis of the [coolstore](https://github.com/konveyor-ecosystem/coolstore) application and will issue a series of requests to Kai to generate a Fix and then store those fixes back to the application. - 1. See [example/README.md](example/README.md) to learn more how to run this - -## Demo - -### Demo Overview - -- We have a demo that will walk through the migration of a sample application written for EAP with Java EE and bring it to Quarkus. -- Sample Application - - https://github.com/konveyor-ecosystem/coolstore - - We will use the `main` branch which has the Java EE version. - - We have found [these issues](https://github.com/jmle/kai-examples/blob/main/coolstore-examples/examples.md) in the `main` branch which need to be addressed before we move to Quarkus: - - This information was obtained by running [Kantra](https://github.com/konveyor/kantra) (Konveyor's static code analyzer) with these [custom-rules](https://github.com/konveyor-ecosystem/kai/tree/main/samples/custom_rules) - - Full output from [Kantra](https://github.com/konveyor/kantra) is checked into the git repo here: [example/analysis/coolstore](example/analysis/coolstore) - -### What are the general steps of the demo? - -1. We launch VSCode with our Kai VS Code extension from [konveyor-ecosystem/kai-vscode-plugin](https://github.com/konveyor-ecosystem/kai-vscode-plugin/tree/main) -2. We open a git checkout of a sample application: [coolstore](https://github.com/konveyor-ecosystem/coolstore) -3. We run [Kantra](https://github.com/konveyor/kantra) inside of VSCode to do an analysis of the application to learn what issues are present that need to be addressed before migrating to Quarkus -4. We view the analysis information in VSCode -5. We look at the impacted files and choose what files/issues we want to fix -6. We click 'Generate Fix' in VSCode on a given file/issue and wait ~45 seconds for the Kai backend to generate a fix -7. We view the suggested fix as a 'Diff' in VSCode -8. We accept the generated fix -9. The file in question has now been updated -10. We move onto the next file/issue and repeat - -### Demo Video - -![DemoVideo](/images/Kai_April_26c.gif) +![DemoVideo](/docs/images/Kai_April_26c.gif) - See [Generative AI Applied to Application Modernization with Konveyor AI](https://www.youtube.com/watch?v=aE8qNY2m4v4) (~15 minute demo with voice) -### Guided walk-through using Kai - -- See [docs/demo.md](docs/demo.md) for a guided walkthrough of how to use Kai to aid in a Java EE to Quarkus migration - -## Notes on `DEMO_MODE` and cached responses - -The kai server will always cache responses in the `kai/data/vcr//` directory. In non-demo mode, these responses will be overwritten whenever a new request is made. -When the server is run with `DEMO_MODE=true`, these responses will be played back. The request will be matched on everything except for authorization headers, cookies, content-length and request body. - -### `DEMO_MODE` Cached Responses - -- We do not actively maintain cached responses for all models/requests. -- You may look at: [kai/data/vcr/coolstore](kai/data/vcr/coolstore/) to see a list of what models have cached responses. - - In general when we cache responses we are running: [example/run_demo.py](example/run_demo.py) and saving those responses. - - This corresponds to a 'KAI Fix All' being run per file in Analysis. -- When running from IDE and attempting to use cached response, we likely only have cached responses for 'Fix All', and we do not have cached responses for individual issues in a file. - -### `DEMO_MODE` Updating Cached Responses - -There are two ways to record new responses: - -1. Run the requests while the server is not in `DEMO_MODE` -1. Delete the specific existing cached response (under `kai/data/vcr///`), then rerun. When a cached response does - not exist, a new one will be recorded and played back on subsequent runs. - -## Contributors - -- Below information is only needed for those who are looking to contribute, run e2e tests, etc. - -### Updating requirements.txt +## Blog Posts -- If you are a developer working on Kai and you are updating requirements.txt, you will need to do some manual changes beyond just a `pip freeze &> ./requirements.txt`, we have a few directives that address differences in 'darwin' systems that need to be preserved. These need to be added manually after a 'freeze' as the freeze command is not aware of what exists in requirements.txt. Please consult the diff of changes you are making now from prior version and note the extra directions for `python_version` and or `sys_platform` +- 2024 May 07: [Apply generative AI to app modernization with Konveyor AI](https://developers.redhat.com/articles/2024/05/07/modernize-apps-konveyor-ai) +- 2024 May 07: [Kai - Generative AI Applied to Application Modernization](https://www.konveyor.io/blog/kai-deep-dive-2024/) +- 2024 July 23: [Embracing the Future of Application Modernization with KAI](https://shaaf.dev/post/2024-07-23-embracing-the-future-of-app-mod-with-konveyor-ai/) -### Ensure you have the source code for the sample applications checked out locally +## Getting Started -1. `cd ./samples` -2. `./fetch_apps.py` - - This will checkout the sample app source code to: `./samples/sample_repos` - - This directory is in .gitignore +1. Run the kai backend image locally with sample data +2. Walk through a guided scenario to evaluate how Kai works -#### (OPTIONAL) Run an analysis of a sample app (example for MacOS) +### Launch the Kai backend with sample data -Note: We have checked in analysis runs for all sample applications so you do NOT need to run analysis yourself. The instructions below are ONLY if you want to recreate, this is NOT required +The quickest way to get running is to leverage sample data commited into the Kai repo along with the `podman compose up` workflow -1. Install [podman](https://podman.io/) so you can run [Kantra](https://github.com/konveyor/kantra) for static code analysis -1. `cd samples` -1. `./fetch_apps.py` # this will git clone example source code apps -1. `cd macos` -1. `./restart_podman_machine.sh` # setups the podman VM on MacOS so it will mount the host filesystem into the VM -1. `./get_latest_kantra_cli.sh` # fetches 'kantra' our analyzer tool and stores it in ../bin -1. `cd ..` -1. `./analyze_apps.py` # Analyzes all sample apps we know about, in both the 'initial' and 'solved' states, expect this to run for ~2-3 hours. +1. `git clone https://github.com/konveyor/kai.git` +1. `cd kai` +1. Run `podman compose up`. The first time this is run it will take several minutes to download images and to populate sample data. + - After the first run the DB will be populated and subsequent starts will be much faster, as long as the kai_kai_db_data volume is not deleted. + - To clean up all resources run `podman compose down && podman volume rm kai_kai_db_data`. +1. Kai backend is now running and ready to serve requests -Analysis data will be stored in: `samples/analysis_reports/{APP_NAME}//output.yaml` +### Guided walk-through -### Tracing +After you have the kai backend running via `podman compose up` you can run through a guided scenario we have to show Kai in action at: [docs/scenarios/demo.md](docs/scenarios/demo.md) -The Kai server is able to capture LLM tracing information and write to disk to aid debugging. -Kai's tracing is currently simple, we will write to disk but are not integrated with any LLM observability tools. +- [docs/scenarios/demo.md](docs/scenarios/demo.md) walks through a guided scenario of using Kai to complete a migration of a Java EE app to Quarkus. -Tracing will gather information and write to various files under the 'logs/trace' directory. +### Others ways to run Kai -Tracing can be enabled or disabled. -It is enabled via: +The above information is a quick path to enable running Kai quickly to see how it works. If you'd like to take a deeper dive into running Kai against data in Konveyor or your own custom data, please see [docs/Getting_Started.md](docs/Getting_Started.md) -- Environment variable: `TRACE=true` -- kai/config.toml +### Debugging / Troubleshooting - trace_enabled = true +- Kai backend will write logging information to the `logs` directory. + - You can adjust the level via `kai/config.toml` and change the `file_log_level = "debug"` value +- Tracing information is written to disk to aid deeper explorations of Prompts and LLM Results. See [docs/contrib/Tracing.md](docs/contrib/Tracing.md) -Example of information captured with tracing: +## Technical design documents -- Prompt -- LLM Result -- Request Parameters -- Exceptions -- Duration of each request +- See our technical design related information at [docs/design](docs/design) -Tracing info is written to a directory hierarchy structure of: +## Roadmap and Early Builds - logs/trace/{model}/{app_name}/{src_file_path}/{batch_mode}/{timestamp_of_request}/{incident_batch_number}/{retry_attempt} +- Kai is in it's early development phase and is NOT ready for production usage. +- See [Roadmap.md](Roadmap.md) to learn about the project's goals and milestones +- Please see [docs/Evaluation_Builds.md](docs/Evaluation_Builds.md) for information on early builds. -Example of hierarchy: +## Contributing - └── trace - └── gpt-3.5-turbo << MODEL ID>> - └── coolstore << APP Name >> - β”œβ”€β”€ pom.xml << Source File Path >> - β”‚Β Β  └── single_group << Incident Batch Mode >> - β”‚Β Β  └── 1719673609.8266618 << Start of Request Time Stamp >> - β”‚Β Β  β”œβ”€β”€ 1 << Incident Batch Number >> - β”‚Β Β  β”‚Β Β  β”œβ”€β”€ 0 << Retry Attempt >> - β”‚Β Β  β”‚Β Β  β”‚Β Β  └── llm_result << Contains the response from the LLM prior to us parsing >> - β”‚Β Β  β”‚Β Β  β”œβ”€β”€ prompt << The formatted prompt prior to sending to LLM >> - β”‚Β Β  β”‚Β Β  └── prompt_vars.json << The prompt variables which are injected into the prompt template >> - β”‚Β Β  β”œβ”€β”€ params.json << Request parameters >> - β”‚Β Β  └── timing << Duration of a Succesful Request >> - └── src - └── main - β”œβ”€β”€ java - β”‚Β Β  └── com - β”‚Β Β  └── redhat - β”‚Β Β  └── coolstore - β”‚Β Β  β”œβ”€β”€ model - β”‚Β Β  β”‚Β Β  β”œβ”€β”€ InventoryEntity.java - β”‚Β Β  β”‚Β Β  β”‚Β Β  └── single_group - β”‚Β Β  β”‚Β Β  β”‚Β Β  └── 1719673609.827135 - β”‚Β Β  β”‚Β Β  β”‚Β Β  β”œβ”€β”€ 1 - β”‚Β Β  β”‚Β Β  β”‚Β Β  β”‚Β Β  β”œβ”€β”€ 0 - β”‚Β Β  β”‚Β Β  β”‚Β Β  β”‚Β Β  β”‚Β Β  └── llm_result - β”‚Β Β  β”‚Β Β  β”‚Β Β  β”‚Β Β  β”œβ”€β”€ prompt - β”‚Β Β  β”‚Β Β  β”‚Β Β  β”‚Β Β  └── prompt_vars.json - β”‚Β Β  β”‚Β Β  β”‚Β Β  β”œβ”€β”€ params.json - β”‚Β Β  β”‚Β Β  β”‚Β Β  └── timing - β”‚Β Β  β”‚Β Β  β”œβ”€β”€ Order.java - β”‚Β Β  β”‚Β Β  β”‚Β Β  └── single_group - β”‚Β Β  β”‚Β Β  β”‚Β Β  └── 1719673609.826999 - β”‚Β Β  β”‚Β Β  β”‚Β Β  β”œβ”€β”€ 1 - β”‚Β Β  β”‚Β Β  β”‚Β Β  β”‚Β Β  β”œβ”€β”€ 0 - β”‚Β Β  β”‚Β Β  β”‚Β Β  β”‚Β Β  β”‚Β Β  └── llm_result - β”‚Β Β  β”‚Β Β  β”‚Β Β  β”‚Β Β  β”œβ”€β”€ prompt - β”‚Β Β  β”‚Β Β  β”‚Β Β  β”‚Β Β  └── prompt_vars.json - β”‚Β Β  β”‚Β Β  β”‚Β Β  β”œβ”€β”€ params.json - β”‚Β Β  β”‚Β Β  β”‚Β Β  └── timing +Our project welcomes contributions from any member of our community. To get +started contributing, please see our [Contributor Guide](CONTRIBUTING.md). -### Linting - -1. Install trunk via: https://docs.trunk.io/check#install-the-cli -1. Run the linters: `trunk check` -1. Format code: `trunk fmt` - -### Testing - -#### How to run regression tests - -1. Install the prerequisites in Setup and activate the python virtual environment -2. Ensure you've checked out the source code for sample applications: Run: [./samples/fetch_sample_apps.sh](./samples/fetch_sample_apps.sh) -3. Run: [./run_tests.sh](./run_tests.sh) - -## Prototype - -This repository represents a prototype implementation as the team explores the solution space. The intent is for this work to remain in the konveyor-ecosystem as the team builds knowledge in the domain and experiments with solutions. As the approach matures we will integrate this properly into Konveyor and seek to promote to github.com/konveyor organization. +- You can find developer/contributor related ## Code of Conduct diff --git a/ROADMAP.md b/ROADMAP.md new file mode 100644 index 00000000..d0ac992e --- /dev/null +++ b/ROADMAP.md @@ -0,0 +1 @@ +# T.B.D diff --git a/docs/Evaluation_Builds.md b/docs/Evaluation_Builds.md new file mode 100644 index 00000000..b8171d0f --- /dev/null +++ b/docs/Evaluation_Builds.md @@ -0,0 +1,38 @@ +# Evaluation builds + +_Please note that Konveyor AI (Kai) is in early stage of development and is not suitable for production usage at this point in time._ + +The community is releasing early access builds of Kai to aid end user evaluation purposes. + +## What evaluation build (release) should I consume? + +The recommended option for previewing Kai is to consume the `:stable` image tag. + +The `:stable` tag is a floating tag which will point to the current recommended `:demo.{DATE}` tag. + +By default, `podman compose up` workflow with [compose.yaml](https://github.com/konveyor/kai/blob/main/compose.yaml) will consume `:stable`. + +## Where can we see existing evaluation builds (releases)? + +Releases may found at: https://github.com/konveyor/kai/releases + +## What is in an evaluation build (release)? + +Releases consist of: + +- [Git tag for the kai](https://github.com/konveyor/kai/tags) GitHub repository +- Container image with a tag matching the git tag, published to https://quay.io/repository/konveyor/kai?tab=tags +- A `.vsix` file for the associated [Kai VSCode Plugin Build](https://github.com/konveyor-ecosystem/kai-vscode-plugin/tree/main/builds) + +## What are the various choices I have for tags? + +- `:stable` + + - The `:stable` tag is a floating tag that will point to a recommended early build + +- `:demo.${DATE}` + + - The `:demo.${DATE}` tags correspond to early-access builds which are generated on a frequent basis. These early builds may be unstable, we recommend you consume the `:stable` build unless you have a known reason to do something different. + +- `:latest` + - The `:latest` tag is a floating tag which follows the 'main' branch of the kai repository. This tag is useful if you want to see the very latest developments as they merge. diff --git a/docs/Example_CLI_Script.md b/docs/Example_CLI_Script.md new file mode 100644 index 00000000..89e7b6ec --- /dev/null +++ b/docs/Example_CLI_Script.md @@ -0,0 +1,22 @@ +# Example CLI Script + +- We have a demo script at [example/README.md](/example/README.md) that will walk through the migration of a sample application written for EAP with Java EE and bring it to Quarkus. +- Sample Application + - https://github.com/konveyor-ecosystem/coolstore + - We will use the `main` branch which has the Java EE version. + - We have found [these issues](https://github.com/jmle/kai-examples/blob/main/coolstore-examples/examples.md) in the `main` branch which need to be addressed before we move to Quarkus: + - This information was obtained by running [Kantra](https://github.com/konveyor/kantra) (Konveyor's static code analyzer) with these [custom-rules](https://github.com/konveyor-ecosystem/kai/tree/main/samples/custom_rules) + - Full output from [Kantra](https://github.com/konveyor/kantra) is checked into the git repo here: [example/analysis/coolstore](example/analysis/coolstore) + - Limitations: + - Kai will perform a partial migration to Quarkus, but manual changes will be required. + - We lack sufficient analysis rules for a full migration, but this workflow can begin to show a preview of Kai in action to see directionally where the project is headed. + +## Details of Example CLI Script + +- [example/run_demo.py](example/run_demo.py) is a python script that will issue a partial migration of a sample Java EE application [coolstore](https://github.com/konveyor-ecosystem/coolstore) to Quarkus. +- What is `run_demo.py` doing? + + - `run_demo.py` will look at Kantra analysis of the [coolstore](https://github.com/konveyor-ecosystem/coolstore) application and will issue a series of requests to Kai to generate a fix and then write those fixes back to the application's git checkout. + +- How to execute `run_demo.py`? + - See [example/README.md](example/README.md) to learn more of how to run this diff --git a/docs/Getting_Started.md b/docs/Getting_Started.md new file mode 100644 index 00000000..238cc1bb --- /dev/null +++ b/docs/Getting_Started.md @@ -0,0 +1,94 @@ +# Getting Started + +## Overview + +Running Kai consists of: + +1. Launching a postgres database and seed with application analysis data +1. Launching the backend Kai REST API Service + - This is the component that will work with the database, construct prompts, talk to Large Language Models (LLMs), and generate code fixes +1. A client that parses analysis information from [analyzer-lsp](https://github.com/konveyor/analyzer-lsp) and then issues requests to the Kai backend. + - The primary client will be an IDE plugin + - For IDE setup see: [Install the Kai VSCode Plugin](https://github.com/konveyor-ecosystem/kai-vscode-plugin/blob/main/docs/user-guide.md) + - It's also possible to issue API requests directly, and we have a python script that does this to aid demonstrations. See [example/README.md](/example/README.md) + +## Recommended path - use `podman compose up` with cached LLM responses + +- _The easiest way to run is to leverage prebuilt container images we publish to [quay.io/konveyor/kai](https://quay.io/repository/konveyor/kai?tab=tags), you can learn more about early builds at [docs/Evaluation_Builds.md](/docs/Evaluation_Builds.md)_ +- _This is the simplest configuration which will limit configuration choices and will use cached LLM results so that you may evaluate Kai without having your own API Keys_ + - The cached data uses a `DEMO_MODE=TRUE` mode for running the backend. See [docs/contrib/Demo_Mode.md](/docs/contrib/Demo_Mode.md) for more information. + - Configured via: + - [kai/config.toml](/kai/config.toml): `demo_mode = true` + - [compose.yaml](/compose.yaml): `DEMO_MODE: "TRUE"` + - Follow the guided scenario at [docs/scenarios/demo.md](/docs/scenarios/demo.md) to evaluate Kai + +### Launch Kai with sample data and cached LLM responses + +_This will run Kai using [sample analysis reports](/samples/analysis_reports) that simulates the analysis data which will be obtained from Konveyor. Additionally it will default to using cached LLM responses as explained in [docs/contrib/Demo_Mode.md](/docs/contrib/Demo_Mode.md)_ + +Steps: + +1. `git clone https://github.com/konveyor/kai.git` +1. `cd kai` +1. Optional Configuration changes _ok to skip and use the defaults if using cached responses_ + 1. Make changes to `kai/config.toml` to select your desired provider and model + 1. Export `GENAI_KEY` or `OPENAI_API_KEY` as appropriate as per [docs/LLM_Selection.md](/docs/LLM_Selection.md) + 1. Note: By default the `stable` image tag will be used by podman compose.yaml. If you want to run with an alternate tag you can export the environment varaible: `TAG="stable"` with any tag you would like to use. +1. Run `podman compose up`. The first time this is run it will take several minutes to download images and to populate sample data. + - After the first run the DB will be populated and subsequent starts will be much faster, as long as the kai_kai_db_data volume is not deleted. + - To clean up all resources run `podman compose down && podman volume rm kai_kai_db_data`. +1. Kai backend is now running and ready to serve requests + +### Next interact with Kai via a Guided Walkthrough Scenario + +For an initial evaluation, the recommended path is to follow a guided walkthrough we have created at: [docs/scenarios/demo.md](docs/scenarios/demo.md) which walks through a scenario of using Kai to complete a migration of a Java EE app to Quarkus. + +#### What are the general steps of Kai's current evaluation demo? + +1. We launch VSCode with our Kai VS Code extension from [konveyor-ecosystem/kai-vscode-plugin](https://github.com/konveyor-ecosystem/kai-vscode-plugin/tree/main) +2. We open a git checkout of a sample application: [coolstore](https://github.com/konveyor-ecosystem/coolstore) +3. We run [Kantra](https://github.com/konveyor/kantra) inside of VSCode to do an analysis of the application to learn what issues are present that need to be addressed before migrating to Quarkus +4. We view the analysis information in VSCode +5. We look at the impacted files and choose what files/issues we want to fix +6. We click 'Generate Fix' in VSCode on a given file/issue and wait ~45 seconds for the Kai backend to generate a fix +7. We view the suggested fix as a 'Diff' in VSCode +8. We accept the generated fix +9. The file in question has now been updated +10. We move onto the next file/issue and repeat + +## Alternative methods of running + +### With data from Konveyor Hub + +_Konveyor integration is still being developed and is not yet fully integrated._ + +1. `git clone https://github.com/konveyor-ecosystem/kai.git` +1. `cd kai` +1. Make changes to `kai/config.toml` to select your desired provider and model +1. Export `GENAI_KEY` or `OPENAI_API_KEY` as appropriate +1. Run `USE_HUB_IMPORTER=True HUB_URL=https://tackle-konveyor-tackle.apps.cluster.example.com/hub IMPORTER_ARGS=-k podman compose --profile use_hub_importer up` + - Note you will want to update the value of `HUB_URL` to match your Konveyor cluster + +### Development Environment + +You may also run the Kai server from a python virtual environment to aid testing local changes without needing to build a container image. + +- See [docs/contrib/Dev_Environment.md](docs/contrib/Dev_Environment.md) + +### Example CLI Script in Python + +- See [docs/Example_CLI_Script.md](/docs/Example_CLI_Script.md) to see an alternative method of running the development team uses to exercise the Kai REST API from a python script + +## Misc notes + +### Extending the data Kai consumes + +- You may modify the analysis information Kai consumes via [docs/customApps.md](docs/customApps.md) + +### Misc notes with `podman compose` + +Note that you need to use podman >= 1.1.0 to use the `--profile` option. podman does not currently support the alternative `COMPOSE_PROFILES` environment variable. + +If your konveyor instance does not use self-signed certificates you may omit `IMPORTER_ARGS=-k`. + +To clean up all resources run `podman compose --profile use_hub_importer down && podman volume rm kai_kai_db_data`. diff --git a/docs/LLM_Selection.md b/docs/LLM_Selection.md new file mode 100644 index 00000000..368783ed --- /dev/null +++ b/docs/LLM_Selection.md @@ -0,0 +1,170 @@ +# Large Language Model (LLM) configuration + +## LLM API Keys + +- We expect that you have configured the environment variables required for the LLM you are attempting to use. + - For example: + - OpenAI service requires: `OPENAI_API_KEY=my-secret-api-key-value` + - IBM BAM service requires: `GENAI_KEY=my-secret-api-key-value` + +## IBM BAM Service + +- The development team has been using the IBM BAM service to aid development and testing: + + IBM Big AI Model (BAM) laboratory is where IBM Research designs, builds, and iterates on what’s next in foundation models. Our goal is to help accelerate the transition from research to product. Come experiment with us. + + - Login: https://bam.res.ibm.com/ + - In order to use this service an individual needs to obtain a w3id from IBM. The kai development team is unable to help obtaining this access. + - Related client tooling: + + - https://github.com/IBM/ibm-generative-ai + - LangChain integration: https://ibm.github.io/ibm-generative-ai/v2.2.0/rst_source/examples.extensions.langchain.html#examples-extensions-langchain + + - Obtain your API key from IBM BAM: + + - To access via an API you can look at β€˜Documentation’ after logging into https://bam.res.ibm.com/ + - You will see a field embedded in the 'Documentation' section where you can generate/obtain an API Key. + + - Ensure you have `GENAI_KEY=my-secret-api-key-value` defined in your shell + +## OpenAI Service + +- If you have a valid API Key for OpenAI you may use this with Kai. +- Ensure you have `OPENAI_API_KEY=my-secret-api-key-value` defined in your shell + +## Selecting a Model + +We offer configuration choices of several models via [config.toml](/kai/config.toml) which line up to choices we know about from [kai/model_provider.py](https://github.com/konveyor-ecosystem/kai/blob/main/kai/model_provider.py). + +To change which llm you are targeting, open `config.toml` and change the `[models]` section to one of the following: + + + +**IBM served granite** + + + + + +```toml +[models] + provider = "ChatIBMGenAI" + + [models.args] + model_id = "ibm/granite-13b-chat-v2" +``` + + + + + +**IBM served mistral** + + + + + +```toml +[models] + provider = "ChatIBMGenAI" + + [models.args] + model_id = "mistralai/mixtral-8x7b-instruct-v01" +``` + + + + + +**IBM served codellama** + + + + + +```toml +[models] + provider = "ChatIBMGenAI" + + [models.args] + model_id = "meta-llama/llama-2-13b-chat" +``` + + + + + +**IBM served llama3** + + + + + +```toml + # Note: llama3 complains if we use more than 2048 tokens + # See: https://github.com/konveyor-ecosystem/kai/issues/172 +[models] + provider = "ChatIBMGenAI" + + [models.args] + model_id = "meta-llama/llama-3-70b-instruct" + parameters.max_new_tokens = 2048 +``` + + + + + +**Ollama** + + + + + +```toml +[models] + provider = "ChatOllama" + + [models.args] + model = "mistral" +``` + + + + + +**OpenAI GPT 4** + + + + + +```toml +[models] + provider = "ChatOpenAI" + + [models.args] + model = "gpt-4" +``` + + + + + +**OpenAI GPT 3.5** + + + + + +```toml +[models] + provider = "ChatOpenAI" + + [models.args] + model = "gpt-3.5-turbo" +``` + + + +Kai will also work with [OpenAI API Compatible alternatives](docs/OpenAI-API-Compatible-Alternatives.md). diff --git a/docs/contrib/Demo_Mode.md b/docs/contrib/Demo_Mode.md new file mode 100644 index 00000000..2cfe8e20 --- /dev/null +++ b/docs/contrib/Demo_Mode.md @@ -0,0 +1,48 @@ +# Demo Mode - Cached LLM Results + +The kai server is able to cache results from a LLM and later replay these to aid test or demonstration scenarios. + +This mode uses a project called [vcr](https://vcrpy.readthedocs.io/en/latest/) to record the interaction with a given LLM. + +## What to expect from `DEMO_MODE` + +_if_ the `kai/data/vcr//` directory contains a request which has been cached AND if `DEMO_MODE` is enabled then a request that has the same exact parameters as what was seen during recording will return the cached data and will instead of talking to the LLM. + +This mode is most useful for situations where you want to demo functionality but are concerned about conference wifi. The data replayed is actual data that specific LLM generated in past, it's just cached for convenience. + +## Notes on `DEMO_MODE` and cached responses + +The kai server will always cache responses in the `kai/data/vcr//` directory. In non-demo mode, these responses will be overwritten whenever a new request is made. +When the server is run with `DEMO_MODE=true`, these responses will be played back. The request will be matched on everything except for authorization headers, cookies, content-length and request body. + +### `DEMO_MODE` Cached Responses + +- We do not actively maintain cached responses for all models/requests. +- You may look at: [kai/data/vcr/coolstore](kai/data/vcr/coolstore/) to see a list of what models have cached responses. + - In general when we cache responses we are running: [example/run_demo.py](example/run_demo.py) and saving those responses. + - This corresponds to a 'KAI Fix All' being run per file in Analysis. +- When running from IDE and attempting to use cached response, we likely only have cached responses for 'Fix All', and we do not have cached responses for individual issues in a file. + +### `DEMO_MODE` Updating Cached Responses + +There are two ways to record new responses: + +1. Run the requests while the server is not in `DEMO_MODE` +1. Delete the specific existing cached response (under `kai/data/vcr///`), then rerun. When a cached response does not exist, a new one will be recorded and played back on subsequent runs. + +### Seeing a `401` while running with `DEMO_MODE=TRUE`? + +If you are running with DEMO_MODE=TRUE and you see a `401` being returned from the LLM Provider consider that this is likely 'correct' behavior, if you do not have a valid API key defined. Note that DEMO_MODE will attempt to play back a cached response, yet if there is no cached data for the request Kai will attempt to talk to the LLM and will make a 'real' request, which if you don't have a valid API key will result in a `401`. To 'fix' this you will want to look at the request you are sending to Kai and ensure it is cached for the model you have configured, if you don't have valid cached data then you will need to get a valid api key and re-run so the data may be cached. + + WARNING - 2024-07-11 14:11:44,063 - [ llm_io_handler.py:243 - get_incident_solutions_for_file()] - Request to model failed for batch 1/1 for src/main/java/com/redhat/coolstore/model/InventoryEntity.java with exception, retrying in 10s + Failed to handle request to https://bam-api.res.ibm.com/v2/text/chat_stream?version=2024-01-10. + { + "error": "Unauthorized", + "extensions": { + "code": "AUTH_ERROR", + "state": null, + "reason": "TOKEN_INVALID" + }, + "message": "Invalid or missing JWT token", + "status_code": 401 + } diff --git a/docs/contrib/Dev_Environment.md b/docs/contrib/Dev_Environment.md new file mode 100644 index 00000000..2e58559b --- /dev/null +++ b/docs/contrib/Dev_Environment.md @@ -0,0 +1,79 @@ +# Pre-Requisites + +## Access to a Large Language Model (LLM) + +- If you want to run Kai against a LLM you will likely need to configure a LLM API Key to access your service (unless running against a local model) + - We do provide a means of running Kai against previously cached data from a few models to aid demo flows. This allows you to run through the steps of using previously cached data without requiring access to a LLM. Note, if you do not provide LLM API access then the DEMO_MODE flow will only be able to replay previous cached responses. + - We call this 'DEMO_MODE', i.e. `DEMO_MODE=true make run-server` +- Note that results vary widely between models. + +## Local Development + +Running Kai's backend involves running 2+ processes: + +- Postgres instance which we deliver via container +- Backend REST API server +- [Optional] Hub Importer process to sync data from Konveyor + +### Python Virtual Environment + +1. Clone Repo and Ensure you have the virtual environment setup + 1. `git clone https://github.com/konveyor-ecosystem/kai.git` + 1. `cd kai` + 1. `python3 -m venv env` + - We've tested this with Python 3.11 and 3.12 + 1. `source env/bin/activate` + 1. `pip install -r ./requirements.txt` + 1. `pip install -e .` + +#### Steps + +1. Run the Postgres DB via podman + 1. Open a new shell tab + 1. `source env/bin/activate` + 1. Let this run in background: `make run-postgres` +1. Run the Kai server in background + 1. Open a new shell tab + 1. `source env/bin/activate` + 1. Let this run in background: `make run-server` + - If you want to run with cached LLM responses run with `DEMO_MODE=true` + - Replace the above command and instead run: `DEMO_MODE=true make run-server` + - The `DEMO_MODE` option will cache responses and play them back on subsequent runs. + - If you want to run with debug information set the environment variable `LOG_LEVEL=debug` + - Example: `LOG_LEVEL=debug make run-server` +1. Load data into the database + 1. `source env/bin/activate` + 1. Fetch sample apps: `pushd samples; ./fetch_apps.py; popd` + 1. `make load-data` + - This will complete in ~1-2 minutes + +## How to use Kai? + +### Ensure you have the source code for the sample applications checked out locally + +1. `cd ./samples` +2. `./fetch_apps.py` + - This will checkout the sample app source code to: `./samples/sample_repos` + - This directory is in .gitignore + +#### (OPTIONAL) Run an analysis of a sample app (example for MacOS) + +Note: We have checked in analysis runs for all sample applications so you do NOT need to run analysis yourself. The instructions below are ONLY if you want to recreate, this is NOT required + +1. Install [podman](https://podman.io/) so you can run [Kantra](https://github.com/konveyor/kantra) for static code analysis +1. `cd samples` +1. `./fetch_apps.py` # this will git clone example source code apps +1. `cd macos` +1. `./restart_podman_machine.sh` # setups the podman VM on MacOS so it will mount the host filesystem into the VM +1. `./get_latest_kantra_cli.sh` # fetches 'kantra' our analyzer tool and stores it in ../bin +1. `cd ..` +1. `./analyze_apps.py` # Analyzes all sample apps we know about, in both the 'initial' and 'solved' states, expect this to run for ~2-3 hours. + +Analysis data will be stored in: `samples/analysis_reports/{APP_NAME}//output.yaml` + +## Build and test a local image + +1. cd to the top level kai checkout +1. Build: `podman build -f build/Containerfile . -t quay.io/konveyor/kai:local` +1. Run: `TAG="local" podman compose up` +1. Then proceed with testing as you want diff --git a/docs/contrib/Testing.md b/docs/contrib/Testing.md new file mode 100644 index 00000000..98b40ec0 --- /dev/null +++ b/docs/contrib/Testing.md @@ -0,0 +1,7 @@ +# Testing + +## How to run regression tests + +1. Install the prerequisites in Setup and activate the python virtual environment +2. Ensure you've checked out the source code for sample applications: Run: [./samples/fetch_sample_apps.sh](./samples/fetch_sample_apps.sh) +3. Run: [./run_tests.sh](./run_tests.sh) diff --git a/docs/contrib/Tracing.md b/docs/contrib/Tracing.md new file mode 100644 index 00000000..13ee3d4d --- /dev/null +++ b/docs/contrib/Tracing.md @@ -0,0 +1,70 @@ +# Tracing + +The Kai server is able to capture LLM tracing information and write to disk to aid debugging. + +Kai's tracing is currently simple, we will write to disk but are not integrated with any LLM observability tools. + +Tracing will gather information and write to various files under the 'logs/trace' directory. + +Tracing can be enabled or disabled. +It is enabled via: + +- Environment variable: `TRACE=true` +- kai/config.toml + + trace_enabled = true + +Example of information captured with tracing: + +- Prompt +- LLM Result +- Request Parameters +- Exceptions +- Duration of each request + +Tracing info is written to a directory hierarchy structure of: + + logs/trace/{model}/{app_name}/{src_file_path}/{batch_mode}/{timestamp_of_request}/{incident_batch_number}/{retry_attempt} + +Example of hierarchy: + + └── trace + └── gpt-3.5-turbo << MODEL ID>> + └── coolstore << APP Name >> + β”œβ”€β”€ pom.xml << Source File Path >> + β”‚Β Β  └── single_group << Incident Batch Mode >> + β”‚Β Β  └── 1719673609.8266618 << Start of Request Time Stamp >> + β”‚Β Β  β”œβ”€β”€ 1 << Incident Batch Number >> + β”‚Β Β  β”‚Β Β  β”œβ”€β”€ 0 << Retry Attempt >> + β”‚Β Β  β”‚Β Β  β”‚Β Β  └── llm_result << Contains the response from the LLM prior to us parsing >> + β”‚Β Β  β”‚Β Β  β”œβ”€β”€ prompt << The formatted prompt prior to sending to LLM >> + β”‚Β Β  β”‚Β Β  └── prompt_vars.json << The prompt variables which are injected into the prompt template >> + β”‚Β Β  β”œβ”€β”€ params.json << Request parameters >> + β”‚Β Β  └── timing << Duration of a Succesful Request >> + └── src + └── main + β”œβ”€β”€ java + β”‚Β Β  └── com + β”‚Β Β  └── redhat + β”‚Β Β  └── coolstore + β”‚Β Β  β”œβ”€β”€ model + β”‚Β Β  β”‚Β Β  β”œβ”€β”€ InventoryEntity.java + β”‚Β Β  β”‚Β Β  β”‚Β Β  └── single_group + β”‚Β Β  β”‚Β Β  β”‚Β Β  └── 1719673609.827135 + β”‚Β Β  β”‚Β Β  β”‚Β Β  β”œβ”€β”€ 1 + β”‚Β Β  β”‚Β Β  β”‚Β Β  β”‚Β Β  β”œβ”€β”€ 0 + β”‚Β Β  β”‚Β Β  β”‚Β Β  β”‚Β Β  β”‚Β Β  └── llm_result + β”‚Β Β  β”‚Β Β  β”‚Β Β  β”‚Β Β  β”œβ”€β”€ prompt + β”‚Β Β  β”‚Β Β  β”‚Β Β  β”‚Β Β  └── prompt_vars.json + β”‚Β Β  β”‚Β Β  β”‚Β Β  β”œβ”€β”€ params.json + β”‚Β Β  β”‚Β Β  β”‚Β Β  └── timing + β”‚Β Β  β”‚Β Β  β”œβ”€β”€ Order.java + β”‚Β Β  β”‚Β Β  β”‚Β Β  └── single_group + β”‚Β Β  β”‚Β Β  β”‚Β Β  └── 1719673609.826999 + β”‚Β Β  β”‚Β Β  β”‚Β Β  β”œβ”€β”€ 1 + β”‚Β Β  β”‚Β Β  β”‚Β Β  β”‚Β Β  β”œβ”€β”€ 0 + β”‚Β Β  β”‚Β Β  β”‚Β Β  β”‚Β Β  β”‚Β Β  └── llm_result + β”‚Β Β  β”‚Β Β  β”‚Β Β  β”‚Β Β  β”œβ”€β”€ prompt + β”‚Β Β  β”‚Β Β  β”‚Β Β  β”‚Β Β  └── prompt_vars.json + β”‚Β Β  β”‚Β Β  β”‚Β Β  β”œβ”€β”€ params.json + β”‚Β Β  β”‚Β Β  β”‚Β Β  └── timing diff --git a/docs/HubIntegration.md b/docs/design/HubIntegration.md similarity index 100% rename from docs/HubIntegration.md rename to docs/design/HubIntegration.md diff --git a/docs/design/Technical_Background.md b/docs/design/Technical_Background.md new file mode 100644 index 00000000..49b65424 --- /dev/null +++ b/docs/design/Technical_Background.md @@ -0,0 +1,53 @@ +# Overview + +This document contains background information to help understand the technical approach Konveyor AI (Kai) is following. + +## Approach + +Kai implements a [Retrieval Augmented Generation (RAG)](https://arxiv.org/abs/2005.11401) approach that leverages data from Konveyor to help generate code suggestions to aid migrating legacy code bases to a different technology. The intent of this RAG approach is to shape the code suggestions to be similar to how an organization has solved problems in the past, without additional fine-tuning of the model. + +The approach begins with using static code analysis via the [Kantra](https://github.com/konveyor/kantra) tool to find areas in the source code that need attention. 'kai' will iterate through analysis information and work with LLMs to generate code changes to resolve incidents identified from analysis. + +This approach does _not_ require fine-tuning of LLMs, we augment a LLMs knowledge via the prompt, by leveraging external data from inside of Konveyor and from Analysis Rules to aid the LLM in constructing better results. + +For example, [analyzer-lsp Rules](https://github.com/konveyor/analyzer-lsp/blob/main/docs/rules.md) such as these ([Java EE to Quarkus rulesets](https://github.com/konveyor/rulesets/tree/main/default/generated/quarkus)) are leveraged to aid guiding a LLM to update a legacy Java EE application to Quarkus + +Note: For purposes of this initial prototype we are using an example of Java EE to Quarkus. That is an arbitrary choice to show viability of this approach. The code and the approach will work on other targets that Konveyor has rules for. + +### What happens technically to make this work? + +- [Konveyor](konveyor.io) contains information related to an Organization's Application Portfolio, a view into all of the applications an Organization is managing. This view includes a history of analysis information over time, access to each applications source repositories, and metadata that tracks work in-progress/completed in regard to each application being migrated to a given technology. + +- When 'Konveyor AI' wants to fix a specific issue in a given application, it will mine data in Konveyor to extract 2 sources of information to inject into a given LLM prompt. + + 1. Static Code Analysis + + - We pinpoint where to begin work by leveraging static code analysis to guide us + - The static code analysis is informed via a collection of crowd sourced knowledge contained in our [rulesets](https://github.com/konveyor/rulesets/tree/main) plus augmented via [custom-rules](https://github.com/konveyor-ecosystem/kai/tree/main/samples/custom_rules) + - We include in the prompt Analysis metadata information to give the LLM more context [such as](https://github.com/konveyor-ecosystem/kai/blob/main/example/analysis/coolstore/output.yaml#L2789) + + remote-ejb-to-quarkus-00000: + description: Remote EJBs are not supported in Quarkus + incidents: + - uri: file:///tmp/source-code/src/main/java/com/redhat/coolstore/service/ShippingService.java + message: "Remote EJBs are not supported in Quarkus, and therefore its use must be removed and replaced with REST functionality. In order to do this:\n 1. Replace the `@Remote` annotation on the class with a `@jakarta.ws.rs.Path(\"\")` annotation. An endpoint must be added to the annotation in place of `` to specify the actual path to the REST service.\n 2. Remove `@Stateless` annotations if present. Given that REST services are stateless by nature, it makes it unnecessary.\n 3. For every public method on the EJB being converted, do the following:\n - Annotate the method with `@jakarta.ws.rs.GET`\n - Annotate the method with `@jakarta.ws.rs.Path(\"\")` and give it a proper endpoint path. As a rule of thumb... " + + lineNumber: 12 + variables: + file: file:///tmp/source-code/src/main/java/com/redhat/coolstore/service/ShippingService.java + kind: Class + name: Stateless + package: com.redhat.coolstore.service + + - url: https://jakarta.ee/specifications/restful-ws/ + title: Jakarta RESTful Web Services + + 1. Solved Examples - these are source code diffs that show a LLM how a similar problem was seen in another application the Organization has and how that Organization decided to fix it. + + - We mine data Konveyor has stored from the Application Hub to search for when other applications have fixed the same rule violations and learn how they fixed it and pass that info into the prompt to aid the LLM + - This ability to leverage how the issue was seen and fixed in the past helps to give the LLM extra context to give a higher quality result. + - This is an [early prompt we created](https://github.com/konveyor-ecosystem/kai/blob/main/notebooks/jms_to_smallrye_reactive/output/gpt-4-1106-preview/helloworldmdb/custom-ruleset/jms-to-reactive-quarkus-00050/few_shot/template.txt) to help give a feel of this in action and the [result we got back from a LLM](https://github.com/konveyor-ecosystem/kai/blob/main/notebooks/jms_to_smallrye_reactive/output/gpt-4-1106-preview/helloworldmdb/custom-ruleset/jms-to-reactive-quarkus-00050/few_shot/result.txt) + +## Further Reading + +For a deeper technical look at Kai please see the konveyor.io blog post from 2024 May 07: [Kai - Generative AI Applied to Application Modernization](https://www.konveyor.io/blog/kai-deep-dive-2024/) diff --git a/docs/solvedIncidentStore.md b/docs/design/solvedIncidentStore.md similarity index 100% rename from docs/solvedIncidentStore.md rename to docs/design/solvedIncidentStore.md diff --git a/images/Kai_April_26c.gif b/docs/images/Kai_April_26c.gif similarity index 100% rename from images/Kai_April_26c.gif rename to docs/images/Kai_April_26c.gif diff --git a/docs/demo.md b/docs/scenarios/demo.md similarity index 100% rename from docs/demo.md rename to docs/scenarios/demo.md diff --git a/docs/deploy.gif b/docs/scenarios/deploy.gif similarity index 100% rename from docs/deploy.gif rename to docs/scenarios/deploy.gif diff --git a/docs/ejb_remote.png b/docs/scenarios/ejb_remote.png similarity index 100% rename from docs/ejb_remote.png rename to docs/scenarios/ejb_remote.png diff --git a/docs/kai_type2.png b/docs/scenarios/kai_type2.png similarity index 100% rename from docs/kai_type2.png rename to docs/scenarios/kai_type2.png diff --git a/docs/mdb.png b/docs/scenarios/mdb.png similarity index 100% rename from docs/mdb.png rename to docs/scenarios/mdb.png diff --git a/docs/mdbchanges.png b/docs/scenarios/mdbchanges.png similarity index 100% rename from docs/mdbchanges.png rename to docs/scenarios/mdbchanges.png diff --git a/docs/model.png b/docs/scenarios/model.png similarity index 100% rename from docs/model.png rename to docs/scenarios/model.png diff --git a/example/README.md b/example/README.md index d0703fec..ddfb66a5 100644 --- a/example/README.md +++ b/example/README.md @@ -1,9 +1,13 @@ -# To run +# Requires -- Assumes that you have `make run-server` running from base directory +- Kai backend to be running locally + - You can see [docs/Getting_Started.md](docs/Getting_Started.md) for guidance -1. `fetch.sh` +## To run this demo client + +1. `fetch.sh` - Ensures that needed source code for the demo app is fetched 1. `source ../env/bin/activate` - Assumes that we are running in the python virtual environment from kai + - If you haven't created a virtual environment yet, refer to [docs/contrib/Dev_Environment.md](docs/contrib/Dev_Environment.md) 1. `./run_demo.py` - - After the script has run you will see it has updated the associated .java files under the 'coolstore' source code directory, there are also files written to show the prompt use and the raw result from the LLM. + - After the script has run you will see it has updated the associated .java files under the 'coolstore' source code directory, there are also files written to show the prompt used and the raw result from the LLM.