Remove wkw datasource suggestions (#7697)

* Remove suggestion route * Remove WKWDataFormat * Lint * wip remove in frontend * unify readInboxDataSource * remove datasource suggestion - but keep isEditMode prop due to onboarding process needing this prop * remove left over code line linking to dataset import and clean up code * remove unused imports in backend * always show edit settings for datasets the user can edit * remove unused type * er-add dataset actions (show error & delete dataset) for broken datasets in dataset table * adapt wording in messages * add changelog entry * update migrationguide,changelog,docs --------- Co-authored-by: Florian M <florian@scm.io> Co-authored-by: Michael Büßemeyer <michael.buessemeyer@student.hpi.de> Co-authored-by: Florian M <fm3@users.noreply.github.com>
scalableminds · Jun 18, 2024 · 5bf95c8 · 5bf95c8
1 parent ccfc1a9
commit 5bf95c8
Show file tree

Hide file tree

Showing 19 changed files with 87 additions and 733 deletions.
diff --git a/CHANGELOG.unreleased.md b/CHANGELOG.unreleased.md
@@ -34,5 +34,6 @@ For upgrade instructions, please check the [migration guide](MIGRATIONS.released
 - Fixed a bug with incorrect valiation of layer names in the animation modal. [#7882](https://github.com/scalableminds/webknossos/pull/7882)
 
 ### Removed
+- If the datasource-properties.json file for a dataset is missing or contains errors, WEBKNOSSOS no longer attempts to guess its contents from the raw data. Exploring remote datasets will still create the file. [#7697](https://github.com/scalableminds/webknossos/pull/7697)
 
 ### Breaking Changes
diff --git a/MIGRATIONS.unreleased.md b/MIGRATIONS.unreleased.md
@@ -8,12 +8,12 @@ User-facing changes are documented in the [changelog](CHANGELOG.released.md).
 ## Unreleased
 [Commits](https://github.com/scalableminds/webknossos/compare/24.06.0...HEAD)
 
-- The datastore config field `datastore.cache.dataCube.maxEntries` is no longer used an can be removed.
-
+- The datastore config field `datastore.cache.dataCube.maxEntries` is no longer used an can be removed. [#7818](https://github.com/scalableminds/webknossos/pull/7818)
 - If your setup contains webknossos-workers, you may want to add the newly available job `align_sections` to the `supportedJobCommands` of your workers. Make sure you deploy the latest webknossos-worker release. [#7820](https://github.com/scalableminds/webknossos/pull/7820)
+- If you place WKW datasets directly on disk, a datasource-properties.json is now required, as WEBKNOSSOS no longer guesses its contents from the raw data. Standard dataset creation methods, e.g. with the WEBKNOSSOS CLI or python libs will already automatically this metadata file. [#7697](https://github.com/scalableminds/webknossos/pull/7697)
 
 ### Postgres Evolutions:
 
 - [114-ai-models.sql](conf/evolutions/114-ai-models.sql)
 - [115-annotation-locked-by-user.sql](conf/evolutions/115-annotation-locked-by-user.sql)
-- [116-drop-overtimemailinglist.sql](conf/evolutions/116-drop-overtimemailinglist.sql)
+- [116-drop-overtimemailinglist.sql](conf/evolutions/116-drop-overtimemailinglist.sql)
diff --git a/docs/data_formats.md b/docs/data_formats.md
@@ -126,9 +126,8 @@ The term "magnifications" is used synonymously for resolutions throughout the UI
 At the moment, WebKnossos guarantees correct rendering of data with non-uniform resolution factors only if the z-component between two resolutions changes by a factor of 1 or 2.
 
 Most users do not create these metadata files manually.
-WEBKNOSSOS can infer most of these properties automatically, except for `scale` and `largestSegmentId`.
-During the data import process, WEBKNOSSOS will ask for the necessary properties.
 When using the [WEBKNOSSOS CLI](http://docs.webknossos.org/cli), a metadata file is automatically generated. Alternatively, you can create and edit WEBKNOSSOS datasets using the [WEBKNOSSOS Python library](https://github.com/scalableminds/webknossos-libs/).
+During the data import process, WEBKNOSSOS will ask for the necessary properties.
 
 [See below for the full specification](#dataset-metadata-specification).
 

diff --git a/frontend/javascripts/admin/admin_rest_api.ts b/frontend/javascripts/admin/admin_rest_api.ts
@@ -11,7 +11,6 @@ import type {
   APIBuildInfo,
   APIConnectomeFile,
   APIDataSource,
-  APIDataSourceWithMessages,
   APIDataStore,
   APIDataset,
   APIDatasetId,
@@ -1116,16 +1115,6 @@ export async function getDatasets(
   return datasets;
 }
 
-export function getDatasetDatasource(
-  dataset: APIMaybeUnimportedDataset,
-): Promise<APIDataSourceWithMessages> {
-  return doWithToken((token) =>
-    Request.receiveJSON(
-      `${dataset.dataStore.url}/data/datasets/${dataset.owningOrganization}/${dataset.name}?token=${token}`,
-    ),
-  );
-}
-
 export function readDatasetDatasource(dataset: APIDataset): Promise<APIDataSource> {
   return doWithToken((token) =>
     Request.receiveJSON(

diff --git a/frontend/javascripts/admin/dataset/dataset_add_view.tsx b/frontend/javascripts/admin/dataset/dataset_add_view.tsx
@@ -96,7 +96,7 @@ function DatasetAddView({ history }: RouteComponentProps) {
                   View the Dataset
                 </Button>
                 <Button
-                  onClick={() => history.push(`/datasets/${organization}/${datasetName}/import`)}
+                  onClick={() => history.push(`/datasets/${organization}/${datasetName}/edit`)}
                 >
                   Go to Dataset Settings
                 </Button>

diff --git a/frontend/javascripts/admin/dataset/dataset_upload_view.tsx b/frontend/javascripts/admin/dataset/dataset_upload_view.tsx
@@ -725,12 +725,12 @@ class DatasetUploadView extends React.Component<PropsWithFormAndRouter, State> {
                 rules={[
                   {
                     required: this.state.needsConversion,
-                    message: "Please provide a scale for the dataset.",
+                    message: "Please provide a voxel size for the dataset.",
                   },
                   {
                     validator: syncValidator(
                       (value: Vector3) => value?.every((el) => el > 0),
-                      "Each component of the scale must be larger than 0.",
+                      "Each component of the voxel size must be larger than 0.",
                     ),
                   },
                 ]}

diff --git a/frontend/javascripts/dashboard/advanced_dataset/dataset_action_view.tsx b/frontend/javascripts/dashboard/advanced_dataset/dataset_action_view.tsx
@@ -3,7 +3,6 @@ import {
   EllipsisOutlined,
   EyeOutlined,
   LoadingOutlined,
-  PlusCircleOutlined,
   PlusOutlined,
   ReloadOutlined,
   SettingOutlined,
@@ -18,9 +17,9 @@ import Toast from "libs/toast";
 import messages from "messages";
 import CreateExplorativeModal from "dashboard/advanced_dataset/create_explorative_modal";
 import { MenuProps, Modal, Typography } from "antd";
+import { useState } from "react";
 import { confirmAsync } from "dashboard/dataset/helper_components";
 import { useQueryClient } from "@tanstack/react-query";
-import { useState } from "react";
 
 const disabledStyle: React.CSSProperties = {
   pointerEvents: "none",
@@ -193,18 +192,31 @@ function DatasetActionView(props: Props) {
       style={disabledWhenReloadingStyle}
       type="link"
     >
-      {isReloading ? <LoadingOutlined /> : <ReloadOutlined className="icon-margin-right" />}
+      {isReloading ? (
+        <LoadingOutlined className="icon-margin-right" />
+      ) : (
+        <ReloadOutlined className="icon-margin-right" />
+      )}
       Reload
     </a>
   );
-  const importLink = (
-    <div className="dataset-table-actions">
-      <Link
-        to={`/datasets/${dataset.owningOrganization}/${dataset.name}/import`}
-        className="import-dataset"
+  const datasetSettingsLink = (
+    <>
+      <LinkWithDisabled
+        to={`/datasets/${dataset.owningOrganization}/${dataset.name}/edit`}
+        title="Open Dataset Settings"
+        disabled={isReloading}
       >
-        <PlusCircleOutlined className="icon-margin-right" />
-        Import
+        <SettingOutlined className="icon-margin-right" />
+        Settings
+      </LinkWithDisabled>
+    </>
+  );
+  const brokenDatasetActions = (
+    <div className="dataset-table-actions">
+      <Link to={`/datasets/${dataset.owningOrganization}/${dataset.name}/edit`}>
+        <SettingOutlined className="icon-margin-right" />
+        Settings
       </Link>
       {reloadLink}
       <a
@@ -236,41 +248,35 @@ function DatasetActionView(props: Props) {
       ) : null}
     </div>
   );
+
+  const activeDatasetActions = (
+    <>
+      {" "}
+      <NewAnnotationLink
+        dataset={dataset}
+        isReloading={isReloading}
+        isCreateExplorativeModalVisible={isCreateExplorativeModalVisible}
+        onShowCreateExplorativeModal={() => setIsCreateExplorativeModalVisible(true)}
+        onCloseCreateExplorativeModal={() => setIsCreateExplorativeModalVisible(false)}
+      />
+      <LinkWithDisabled
+        to={`/datasets/${dataset.owningOrganization}/${dataset.name}/view`}
+        title="View Dataset"
+        disabled={isReloading}
+      >
+        <EyeOutlined className="icon-margin-right" />
+        View
+      </LinkWithDisabled>
+      {dataset.isEditable ? datasetSettingsLink : null}
+      {reloadLink}
+    </>
+  );
   return (
     <div>
-      {dataset.isEditable && !dataset.isActive ? importLink : null}
-      {dataset.isActive ? (
-        <div className="dataset-table-actions nowrap">
-          <NewAnnotationLink
-            dataset={dataset}
-            isReloading={isReloading}
-            isCreateExplorativeModalVisible={isCreateExplorativeModalVisible}
-            onShowCreateExplorativeModal={() => setIsCreateExplorativeModalVisible(true)}
-            onCloseCreateExplorativeModal={() => setIsCreateExplorativeModalVisible(false)}
-          />
-          <LinkWithDisabled
-            to={`/datasets/${dataset.owningOrganization}/${dataset.name}/view`}
-            title="View Dataset"
-            disabled={isReloading}
-          >
-            <EyeOutlined className="icon-margin-right" />
-            View
-          </LinkWithDisabled>
-          {dataset.isEditable ? (
-            <React.Fragment>
-              <LinkWithDisabled
-                to={`/datasets/${dataset.owningOrganization}/${dataset.name}/edit`}
-                title="Open Dataset Settings"
-                disabled={isReloading}
-              >
-                <SettingOutlined className="icon-margin-right" />
-                Settings
-              </LinkWithDisabled>
-              {reloadLink}
-            </React.Fragment>
-          ) : null}
-        </div>
-      ) : null}
+      {dataset.isEditable && !dataset.isActive ? brokenDatasetActions : null}
+      <div className="dataset-table-actions nowrap">
+        {dataset.isActive ? activeDatasetActions : null}
+      </div>
     </div>
   );
 }
@@ -330,7 +336,7 @@ export function getDatasetActionContextMenu({
             },
           }
         : null,
-      dataset.isEditable && dataset.isActive
+      dataset.isEditable
         ? {
             key: "edit",
             label: "Open Settings",
@@ -340,15 +346,6 @@ export function getDatasetActionContextMenu({
           }
         : null,
 
-      dataset.isEditable && !dataset.isActive
-        ? {
-            key: "import",
-            label: "Import",
-            onClick: () => {
-              window.location.href = `/datasets/${dataset.owningOrganization}/${dataset.name}/import`;
-            },
-          }
-        : null,
       {
         key: "reload",
         label: "Reload",

diff --git a/frontend/javascripts/dashboard/dataset/dataset_settings_data_tab.tsx b/frontend/javascripts/dashboard/dataset/dataset_settings_data_tab.tsx
@@ -64,14 +64,12 @@ export default function DatasetSettingsDataTab({
   form,
   activeDataSourceEditMode,
   onChange,
-  additionalAlert,
   dataset,
 }: {
   allowRenamingDataset: boolean;
   form: FormInstance;
   activeDataSourceEditMode: "simple" | "advanced";
   onChange: (arg0: "simple" | "advanced") => void;
-  additionalAlert?: React.ReactNode | null | undefined;
   dataset?: APIDataset | null | undefined;
 }) {
   // Using the return value of useWatch for the `dataSource` var
@@ -113,8 +111,6 @@ export default function DatasetSettingsDataTab({
         </Tooltip>
       </div>
 
-      {additionalAlert}
-
       <Hideable hidden={activeDataSourceEditMode !== "simple"}>
         <RetryingErrorBoundary>
           <SimpleDatasetForm
@@ -217,12 +213,12 @@ function SimpleDatasetForm({
                   rules={[
                     {
                       required: true,
-                      message: "Please provide a scale for the dataset.",
+                      message: "Please provide a voxel size for the dataset.",
                     },
                     {
                       validator: syncValidator(
                         (value: Vector3) => value?.every((el) => el > 0),
-                        "Each component of the scale must be greater than 0",
+                        "Each component of the voxel size must be greater than 0",
                       ),
                     },
                   ]}