You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Currently, the dataset selection dropdown menu suffers from performance issues. This is because we have a large number of files (several thousand), and another entry (a <mat-option> component, containing a checkbox, a <span> with some text, and an additional <div>) has to be created for each file.
Additionally, it is very difficult to select a large number of files, since the user has to manually click on each one. As a workaround, we currently include a "summary" file in the Ginkgo dataset which contains the most relevant information from each of the individual files. However, this causes its own set of performance issues, since loading and processing the file is computationally demanding (for a web-based application).
Another issue is that it is impossible to explicitly link a plotting script to a set of data files it can use. Thus, loading any of the example scripts requires the user to know which of the datasets should be loaded for the script to work.
A possible solution
One option would be to replace the dropdown menu with a JSONata script (editable by the user) which will be used to load the files. The standard JSONata language can be enhanced with an an additional function that loads the data by using JSONata's registerFunction feature.
The script would get a list of all available datasets as input (contained in the dataset repository's list.json file) and produce a JSON object with all the datasets loaded. This object would then be passed on to the second script that plots the data. One plotting example would then contain two scripts: the data selection script, and the plotting script.
Alternatively, both scripts could be combined into a single script, though I'm not sure this is a better approach.
The text was updated successfully, but these errors were encountered:
Currently, the dataset selection dropdown menu suffers from performance issues. This is because we have a large number of files (several thousand), and another entry (a
<mat-option>
component, containing a checkbox, a<span>
with some text, and an additional<div>
) has to be created for each file.Additionally, it is very difficult to select a large number of files, since the user has to manually click on each one. As a workaround, we currently include a "summary" file in the Ginkgo dataset which contains the most relevant information from each of the individual files. However, this causes its own set of performance issues, since loading and processing the file is computationally demanding (for a web-based application).
Another issue is that it is impossible to explicitly link a plotting script to a set of data files it can use. Thus, loading any of the example scripts requires the user to know which of the datasets should be loaded for the script to work.
A possible solution
One option would be to replace the dropdown menu with a JSONata script (editable by the user) which will be used to load the files. The standard JSONata language can be enhanced with an an additional function that loads the data by using JSONata's
registerFunction
feature.The script would get a list of all available datasets as input (contained in the dataset repository's
list.json
file) and produce a JSON object with all the datasets loaded. This object would then be passed on to the second script that plots the data. One plotting example would then contain two scripts: the data selection script, and the plotting script.Alternatively, both scripts could be combined into a single script, though I'm not sure this is a better approach.
The text was updated successfully, but these errors were encountered: