Instance create: fetch more disks and use comboboxes instead of listboxes

[leftwo]

In instance create, the drop down selection for Additional disks -> `ATTACH EXISTING DISK" menu
only shows me about 100 disks.

I've created some 1400 disks, and I can't choose the disk I want as it is not showing up.
There should be some way to either show all the disks, or enter a few letters and have the set of
choices filtered to match what I entered, or.. something else ??

[david-crespo]

100 is right — we're not specifying a page size and the default in Dropshot is 100, and Nexus is not overriding that default. "about" is also right because we're fetching those 100 and then filtering out already-attached disks client-side:

console/app/forms/disk-attach.tsx

Lines 39 to 42 in a0bf47a

	const detachedDisks =
	useApiQuery('diskList', { query: projectSelector }).data?.items.filter(
	(d) => d.state.state === 'detached'
	) || []

The simple thing to do is bump the page size to something really big and make it a combobox so you can search for the thing you want (ctrl-f works ok too). The highest Dropshot will go by default is 10,000. With 36 disks in a project on rack3, the response was 15 kB. 15 * 1000 / 36 = ~400 kB. That seems fine, and we could go higher (which would be necessary if you're working with 1400). Here is a live demo with 2000 disks in the picker. It's fine? It takes a second to open the listbox, but after that scrolls and ctrl-fs just fine. (Update: it takes a lot longer to open in Chrome than Firefox — still borderline usable but it's like 4 seconds)

https://console-git-its-2000-disks-oxidecomputer.vercel.app/projects/mock-project/instances-new

(source: https://github.com/oxidecomputer/console/compare/its-2000-disks)

I also went on dogfood to check how long Nexus would take to return all 1485 of your images: 500ms (see timestamps on the left). Not bad!

More broadly, the real way to solve this is, as you suggest, some kind of server-side search based on what you type in. We don't have that capability in Nexus (see the ancient oxidecomputer/omicron#507), which is why we sometimes get up to client-side hijinks as a workaround. We talk about this problem a lot within product eng and see it as a major project for the next 6 months.

In the meantime, my experiments above suggest there is no problem with bumping up the page size to something absurd.

[leftwo]

I think short term absurdity is okay.

When we get multi-rack support and the search results could span multiple racks we might at that time consider alternatives.

The https://console-git-its-2000-disks-oxidecomputer.vercel.app/projects/mock-project/instances-new link for me did not work in that it did open a "create instance" page, but when I went to attach an "Additional disks", the drop down Disk name never populated.

[david-crespo]

If you're in Chrome, you might just have to wait longer. I'm mentally downgrading this from Fine to Not Fine. I think we can make it better though. I don't think it's anything fundamental that's slow, I bet it's the library powering this attaching event listeners to all the items.

2023-12-07-chrome-2000-disks.mp4

[benjaminleonard]

We're using radix for the dropdown and there's some real issues with that library and performance at the moment.

Firstly I think it might be worthwhile to switch to another library – incorporating the component into the design system so we can benefit from that on the internal sites also. Not sure if virtualising buys us better performance too.

See: https://ariakit.org/examples/select-combobox-virtualized

[david-crespo]

The listbox is actually headless-ui Select. Same deal though. I'd be surprised if we really need virtualization (all oxide backend devs ignore this word) for 2000 simple elements but it's possible.

[benjaminleonard]

Should have known that, having built it...

[charliepark]

Looks like the best path forward will involve using Headless UI's combobox with a large number of items fetched, and using list virtualization
https://tailwindcss.com/blog/headless-ui-v2#combobox-list-virtualization

[charliepark]

Possibly will be partially fixed by #1942

[david-crespo]

This is like 90% fixed. We now fetch 1000 disks

console/app/forms/instance-create.tsx

Lines 152 to 162 in 49d6d7d

	const DISK_FETCH_LIMIT = 1000
	CreateInstanceForm.loader = async ({ params }: LoaderFunctionArgs) => {
	const { project } = getProjectSelector(params)
	await Promise.all([
	// fetch both project and silo images
	apiQueryClient.prefetchQuery('imageList', { query: { project } }),
	apiQueryClient.prefetchQuery('imageList', {}),
	apiQueryClient.prefetchQuery('diskList', {
	query: { project, limit: DISK_FETCH_LIMIT },
	}),

And the attach side modal form uses a combobox:

console/app/forms/disk-attach.tsx

Lines 59 to 66 in 9c532ce

	<ComboboxField
	label="Disk name"
	placeholder="Select a disk"
	name="name"
	items={detachedDisks.map(({ name }) => ({ value: name, label: name }))}
	required
	control={form.control}
	/>

However, we're still using a listbox for the existing disk picker for the boot disk (and the image pickers). I wanted to let the combobox stew a bit before putting it in this super key flow. Now we should just go for it.

console/app/forms/instance-create.tsx

Lines 540 to 560 in 49d6d7d

	<Tabs.Content value={'disk' satisfies BootDiskSourceType} className="space-y-4">
	{disks.length === 0 ? (
	<div className="flex max-w-lg items-center justify-center rounded-lg border p-6 border-default">
	<EmptyMessage
	icon={<Storage16Icon />}
	title="No detached disks found"
	body="Only detached disks can be used as a boot disk"
	/>
	</div>
	) : (
	<ListboxField
	label="Disk"
	name="diskSource"
	description="Existing disks that are not attached to an instance"
	items={disks}
	required
	control={control}
	placeholder="Select a disk"
	/>
	)}
	</Tabs.Content>