Migrations are not reverted if installation of app fails

[christianlupus]

How to use GitHub

• Please use the 👍 reaction to show that you are affected by the same issue.
• Please don't comment if you have no relevant information to add. It's just extra noise for everyone subscribed to this issue.
• Subscribe to receive notifications on status change and new comments.

General description

We found an issue during the evaluation of nextcloud/cookbook#634 that indicates a problem with the core. This is sort of a bug in an exception handling routine and has happened in the past. So, this is merely a reconstruction of what has happened. Sorry for not being able to be more precise.

The issue seems to have manifested with a few requirements meeting:

• NC was installed in the pre-release (beta) phase
• cookbook app was not listed as compatible according to the source code. Users reported it was shown compatible in their pre-releases using the app_install_overwrite configuration by default
• cookbook has multiple migrations but the second migration was not yet compatible with NC21 (Breaking changes in Nextcloud 21 cookbook#504, change of the DB schema type class), while the first could theoretically run successfully

Steps to reproduce

1. Install the incompatible app (second migration fails)
2. Wait for update/make app compatible
3. Try to install the compatible app

Expected behaviour

The app should finally install

Actual behaviour

The app refuses to install with an error message that a certain table was not found in the database.

The symptoms in the logs are similar to #24711. This might be as well completely unrelated.

Further investigation revealed that the database had no tables related to the cookbook app installed (which is correct as step 1 failed). However, the migrations table had the first migration mentioned. So the NC core tried to apply the second migration which used a table from the first migration.
We see a desynchronization of the DB schema and the migrations table.

Server/client configuration, logs

There are multiple users in the issue nextcloud/cookbook#643 reporting problems with installing the app after we made the changes to the failing migrations. Not all confirmed that they used the pre-release but at least one user was able to reproduce the issue with the migrations table.

I cannot provide logs from the failing installation attempts in the past. They were reported as well but merely that the app was not running. Here is an example log I extracted from one of the reports:

{
  "reqId":" *** ",
  "level":3,
  "time":"2021-01-23T01:23:55+00:00",
  "remoteAddr":"*** IP-adress ***",
  "user":"*** username ***",
  "app":"index",
  "method":"GET",
  "url":"/index.php/apps/cookbook/",
  "message":
  {
    "Exception":"Exception",
    "Message":"Undefined class constant 'STRING'",
    "Code":0,
    "Trace": 
      [
        {
          "file":"*** link to nextcloud-dir ***/lib/private/AppFramework/App.php",
          "line":157,
          "function":"dispatch",
          "class":"OC\\AppFramework\\Http\\Dispatcher",
          "type":"->",
          "args":
            [
              {"__class__":"OCA\\Cookbook\\Controller\\MainController"}, 
              "index"
            ]
        },
        {
          "file":"*** link to nextcloud-dir ***/lib/private/Route/Router.php",
          "line":302,
          "function":"main","class":"OC\\AppFramework\\App",
          "type":"::",
          "args":
            [
              "OCA\\Cookbook\\Controller\\MainController",
              "index",
              {"__class__":"OC\\AppFramework\\DependencyInjection\\DIContainer"},
              {"_route":"cookbook.main.index"}
            ]
        },
        {
          "file":"*** link to nextcloud-dir ***/lib/base.php",>
          "line":1002,
          "function":"match",
          "class":"OC\\Route\\Router",
          "type":"->",
          "args":["/apps/cookbook/"]
        },
        {
          "file":"*** link to nextcloud-dir ***/index.php",
          "line":37,
          "function":"handleRequest",
          "class":"OC",
          "type":"::",
          "args":[]
        }
      ],
    "File":"*** link to nextcloud-dir ***/lib/private/AppFramework/Http/Dispatcher.php",
    "Line":159,
    "Previous": 
      {
        "Exception":"Error",
        "Message":"Undefined class constant 'STRING'",
        "Code":0,
        "Trace":
          [
            {
              "file":"*** link to nextcloud-dir ***/apps/cookbook/lib/Service/DbCacheService.php",
              "line":136,
              "function":"findAllRecipes",
              "class":"OCA\\Cookbook\\Db\\RecipeDb",
              "type":"->",
              "args":["*** username ***"]
            },
            {
              "file":"*** link to nextcloud-dir ***/apps/cookbook/lib/Service/DbCacheService.php",
              "line":48,
              "function":"fetchDbRecipeInformations",
              "class":"OCA\\Cookbook\\Service\\DbCacheService",
              "type":"->",
              "args":[]
            },
            {
              "file":"*** link to nextcloud-dir ***/apps/cookbook/lib/Service/DbCacheService.php",
              "line":340,
              "function":"updateCache",
              "class":"OCA\\Cookbook\\Service\\DbCacheService",
              "type":"->",
              "args":["*** sensitive parameters replaced ***"]
            },
            {
              "file":"*** link to nextcloud-dir ***/apps/cookbook/lib/Service/DbCacheService.php",
              "line":329,
              "function":"checkSearchIndexUpdate",
              "class":"OCA\\Cookbook\\Service\\DbCacheService",
              "type":"->",
              "args":[]
            },
            {
              "file":"*** link to nextcloud-dir ***/apps/cookbook/lib/Controller/MainController.php",
              "line":53,
              "function":"triggerCheck",
              "class":"OCA\\Cookbook\\Service\\DbCacheService",
              "type":"->",
              "args":[]
            },
            {
              "file":"*** link to nextcloud-dir ***/lib/private/AppFramework/Http/Dispatcher.php",
              "line":218,
              "function":"index",
              "class":"OCA\\Cookbook\\Controller\\MainController",
              "type":"->",
              "args":[]
            },
            {
              "file":"*** link to nextcloud-dir ***/lib/private/AppFramework/Http/Dispatcher.php",
              "line":127,
              "function":"executeController",
              "class":"OC\\AppFramework\\Http\\Dispatcher",
              "type":"->",
              "args":
                [
                  {"__class__":"OCA\\Cookbook\\Controller\\MainController"},
                  "index"
                ]
            },
            {
              "file":"*** link to nextcloud-dir ***/lib/private/AppFramework/App.php",
              "line":157,
              "function":"dispatch",
              "class":"OC\\AppFramework\\Http\\Dispatcher",
              "type":"->",
              "args":
                [
                  {"__class__":"OCA\\Cookbook\\Controller\\MainController"},
                  "index"
                ]
            },
            {
              "file":"*** link to nextcloud-dir ***/lib/private/Route/Router.php",
              "line":302,
              "function":"main",
              "class":"OC\\AppFramework\\App",
              "type":"::",
              "args":
                [
                  "OCA\\Cookbook\\Controller\\MainController",
                  "index",
                  {"__class__":"OC\\AppFramework\\DependencyInjection\\DIContainer"},
                  {"_route":"cookbook.main.index"}
                ]
            },
            {
              "file":"*** link to nextcloud-dir ***/lib/base.php",
              "line":1002,
              "function":"match",
              "class":"OC\\Route\\Router",
              "type":"->",
              "args":["/apps/cookbook/"]
            },
            {
              "file":"*** link to nextcloud-dir ***/index.php",
              "line":37,"function":"handleRequest",
              "class":"OC",
              "type":"::",
              "args":[]
            }
          ],
        "File":"*** link to nextcloud-dir ***/apps/cookbook/lib/Db/RecipeDb.php",
        "Line":82
      },
    "CustomMessage":"--"
  },
  "userAgent":"Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:84.0) Gecko/20100101 Firefox/84.0",
  "version":"21.0.0.14"
}

Regarding the OS/HTTP/PHP/... version this is a broad spectrum from ARM, docker, and plain installation on various database systems (both Postgres and MariaDB I remember) so it seems something systematic. I cannot define a clear server or client configuration to check this.

I suspect the after the app's migration failed, the DB should have been rolled back but this was not done completely leaving a remnant that blocked any useful operation any further.

Final words

This issue is most likely related to the exception handling routine during the installation of an app. As a result, it might be less tested and error-prone than the usual execution path (successful installation). I want to document the issue here and notify you of our findings. If we can help to further investigate (or answer some questions), feel free to get in touch.

[Rello]

Only mark migrations as installed after execution #25924

the routine was adopted here

[christianlupus]

OK, I suspect the problem is solved then. Thanks, @Rello for the link!