Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Support migrating old recorded tests using bulk facet recorder #2390

Open
pokey opened this issue Jun 5, 2024 · 3 comments
Open

Support migrating old recorded tests using bulk facet recorder #2390

pokey opened this issue Jun 5, 2024 · 3 comments
Assignees

Comments

@pokey
Copy link
Member

pokey commented Jun 5, 2024

Once #2383 is merged, we should simplify the process of migrating old recorded tests to scope fixture tests using the new bulk fixture recorder.

We proceed as follows:

  1. Find all recorded tests with the given language id

  2. Canonicalize and inspect the command payload. We may want to filter to tests that use a specific set of actions and particular targets. We may also only want to migrate a specific subset of scopes, eg based on if they're marked as supported for the given language

  3. For each chosen test, get initial document state

  4. Skip if we've already processed a fixture with the same document state (or if there already exists a fixture with same document state?)

  5. Generate an entry in the bulk recorder file with a candidate header for each possible facet that has the given scope. For example:

    [[python]]
    
    [branch.if]
    [branch.if.iteration]
    [branch.try]
    [branch.try.iteration]
    [branch.loop]
    if aaa:
        pass
    ---
    
    [branch.if]
    [branch.if.iteration]
    [branch.try]
    [branch.try.iteration]
    [branch.loop]
    try:
        pass
    except:
        pass
    ---
    
  6. The user will then delete the unnecessary lines and clone the block as necessary eg to test both scope and iteration scope

  7. When the user saves, we should probably delete the recorded tests that they actually ended up migrating

To discuss: which tests to migrate?

We need to figure out how to decide which recorded tests to migrate. Some ideas:

  • All fixtures for a given language, regardless of whether they're marked as supported
  • Only ones marked as supported
  • User picks specific scope type

We also need to figure out if we want to drop recorded tests if their command payload doesn't match a particular shape. Eg skip if multiple scope types are mentioned, or of the command isn't one of a few expected commands

@pokey pokey added the to discuss Plan to discuss at meet-up label Jun 5, 2024
@pokey
Copy link
Member Author

pokey commented Jun 13, 2024

update from meet-up:

  • Only facets marked as supported
  • Only recorded tests in the folder languages/{languageId}
  • Only recorded tests with action of "chuck", "take", "pour", "clone", "drink", "clone up", "clear", primitive target, containing scope modifier

Migrate the above, and then see what's left and maybe make filter more lenient or just do them manually

@pokey pokey removed the to discuss Plan to discuss at meet-up label Jun 13, 2024
@fidgetingbits
Copy link
Collaborator

Does this mean I should hold off an updating old tests for now to see how this goes? Porting the lua, bash, and nix tests are still on my list todo and was thinking recently about circling back to to it, but if it can mostly be automated that works too :D

@pokey
Copy link
Member Author

pokey commented Jun 15, 2024

Up to you. I've been adding new tests in new format, and some piecemeal migrations if I'm messing with the given scope, but waiting to do wholesale migrations till we have this

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants