Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat: implement read #11649

Merged
merged 59 commits into from
Jan 18, 2024
Merged
Show file tree
Hide file tree
Changes from 58 commits
Commits
Show all changes
59 commits
Select commit Hold shift + click to select a range
f53ca45
feat: implement readAsset
Rich-Harris Jan 16, 2024
716c594
lint etc
Rich-Harris Jan 16, 2024
a294256
maybe make it work on netlify? no idea how to test, manual deploys do…
Rich-Harris Jan 16, 2024
15880de
set length/type from manifest
Rich-Harris Jan 16, 2024
5163e36
tidy up
Rich-Harris Jan 16, 2024
289155a
regenerate types
Rich-Harris Jan 16, 2024
da88785
lint
Rich-Harris Jan 16, 2024
caf010f
missed a spot
Rich-Harris Jan 17, 2024
4eba560
more efficient manifest generation
Rich-Harris Jan 17, 2024
6045dfc
working on vercel
Rich-Harris Jan 17, 2024
5c6f2df
lint/fix
Rich-Harris Jan 17, 2024
c9bfc03
Update packages/adapter-vercel/index.js
Rich-Harris Jan 17, 2024
41514d3
fix
Rich-Harris Jan 17, 2024
c4ecda4
createReadable helper
Rich-Harris Jan 17, 2024
fbd02c8
Merge branch 'read-asset' of github.com:sveltejs/kit into read-asset
Rich-Harris Jan 17, 2024
809fafd
more future-proof API
Rich-Harris Jan 17, 2024
d559345
account for basepath
Rich-Harris Jan 17, 2024
d5b98da
lint
Rich-Harris Jan 17, 2024
8f5fd84
rename to just `read`
Rich-Harris Jan 17, 2024
08fd1b1
inline docs
Rich-Harris Jan 17, 2024
8d606a2
it is already deprecated, we just need to remove it
Rich-Harris Jan 17, 2024
7609aa0
read_asset -> read_implementation
Rich-Harris Jan 17, 2024
b3dfa65
add test
Rich-Harris Jan 17, 2024
38e1419
Apply suggestions from code review
Rich-Harris Jan 17, 2024
2c0f579
improve searchability
Rich-Harris Jan 17, 2024
0169191
Merge branch 'read-asset' of github.com:sveltejs/kit into read-asset
Rich-Harris Jan 17, 2024
8ec59df
prevent $app/server being imported client-side
Rich-Harris Jan 17, 2024
2c4a2cc
regenerate types
Rich-Harris Jan 17, 2024
a2c469c
add dev time feature tracking mechanism
Rich-Harris Jan 17, 2024
e338d78
test feature support at build time
Rich-Harris Jan 17, 2024
12c0e46
lint
Rich-Harris Jan 17, 2024
55c9637
lint
Rich-Harris Jan 17, 2024
2447b56
regenerate types
Rich-Harris Jan 17, 2024
75f22a3
account for hooks.server.js, mostly
Rich-Harris Jan 17, 2024
12b91b5
regenerate types
Rich-Harris Jan 17, 2024
501491f
fix
Rich-Harris Jan 18, 2024
5c140b3
bump peerdeps, add changesets
Rich-Harris Jan 18, 2024
2f2bccc
createReadable -> createReadableStream
Rich-Harris Jan 18, 2024
73bd682
Apply suggestions from code review
Rich-Harris Jan 18, 2024
39d4e6d
remove unnecessary if
Rich-Harris Jan 18, 2024
3353c1f
replace docs for find_server_assets
Rich-Harris Jan 18, 2024
8409574
update adapter author docs
Rich-Harris Jan 18, 2024
f8df6cb
regenerate types
Rich-Harris Jan 18, 2024
aea2992
explain what __SVELTEKIT_TRACK__ does
Rich-Harris Jan 18, 2024
2b44e59
mention `$app/server` on server-only modules page
Rich-Harris Jan 18, 2024
2e38c71
minor details
Rich-Harris Jan 18, 2024
36a39fb
oh ffs
Rich-Harris Jan 18, 2024
2bd0faa
exclude prerendered routes from feature detection, handle /@fs assets…
Rich-Harris Jan 18, 2024
501fa0e
use read to populate content.json
Rich-Harris Jan 18, 2024
1a35f7f
fix prerendering
Rich-Harris Jan 18, 2024
10be71f
simplify
Rich-Harris Jan 18, 2024
984e9b4
simplify docs logic
Rich-Harris Jan 18, 2024
0eebe9f
fix
Rich-Harris Jan 18, 2024
b85d699
style
Rich-Harris Jan 18, 2024
92fb9f6
simplify
Rich-Harris Jan 18, 2024
71ba727
lockfile
Rich-Harris Jan 18, 2024
86c22cc
Apply suggestions from code review
Rich-Harris Jan 18, 2024
a53c0b6
capitalize
Rich-Harris Jan 18, 2024
2b3746c
Apply suggestions from code review
Rich-Harris Jan 18, 2024
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
7 changes: 7 additions & 0 deletions .changeset/rude-apples-jam.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,7 @@
---
'@sveltejs/adapter-netlify': major
'@sveltejs/adapter-vercel': major
'@sveltejs/adapter-node': major
---

breaking: update peer dependency on `@sveltejs/kit`
5 changes: 5 additions & 0 deletions .changeset/tasty-masks-talk.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,5 @@
---
'@sveltejs/kit': minor
---

feat: add `$app/server` module with `read` function for reading assets from filesystem
7 changes: 7 additions & 0 deletions .changeset/tiny-maps-chew.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,7 @@
---
'@sveltejs/adapter-netlify': minor
'@sveltejs/adapter-vercel': minor
'@sveltejs/adapter-node': minor
---

feat: support `read` from `$app/server`
8 changes: 8 additions & 0 deletions documentation/docs/25-build-and-deploy/99-writing-adapters.md
Original file line number Diff line number Diff line change
Expand Up @@ -7,6 +7,7 @@ If an adapter for your preferred environment doesn't yet exist, you can build yo
Adapters packages must implement the following API, which creates an `Adapter`:

```js
// @errors: 2322
// @filename: ambient.d.ts
type AdapterSpecificOptions = any;

Expand All @@ -19,6 +20,13 @@ export default function (options) {
name: 'adapter-package-name',
async adapt(builder) {
// adapter implementation
},
supports: {
read: ({ config, route }) => {
// Return `true` if the route with the given `config` can use `read`
// from `$app/server` in production, return `false`` if it can't.
Rich-Harris marked this conversation as resolved.
Show resolved Hide resolved
// Or throw a descriptive error describing how to configure the deployment
}
}
};

Expand Down
4 changes: 4 additions & 0 deletions documentation/docs/30-advanced/50-server-only-modules.md
Original file line number Diff line number Diff line change
Expand Up @@ -8,6 +8,10 @@ Like a good friend, SvelteKit keeps your secrets. When writing your backend and

The `$env/static/private` and `$env/dynamic/private` modules, which are covered in the [modules](modules) section, can only be imported into modules that only run on the server, such as [`hooks.server.js`](hooks#server-hooks) or [`+page.server.js`](routing#page-page-server-js).

## Server-only utilities

The [`$app/server`](/docs/modules#$app-server) module, which contains a `read` function for reading assets from the filesystem, can likewise only be imported by code that runs on the server.

## Your modules

You can make your own modules server-only in two ways:
Expand Down
16 changes: 15 additions & 1 deletion packages/adapter-netlify/index.js
Original file line number Diff line number Diff line change
Expand Up @@ -27,6 +27,7 @@ import toml from '@iarna/toml';
* }} HandlerManifest
*/

const name = '@sveltejs/adapter-netlify';
const files = fileURLToPath(new URL('./files', import.meta.url).href);

const edge_set_in_env_var =
Expand All @@ -38,7 +39,7 @@ const FUNCTION_PREFIX = 'sveltekit-';
/** @type {import('./index.js').default} */
export default function ({ split = false, edge = edge_set_in_env_var } = {}) {
return {
name: '@sveltejs/adapter-netlify',
name,

async adapt(builder) {
if (!builder.routes) {
Expand Down Expand Up @@ -92,6 +93,19 @@ export default function ({ split = false, edge = edge_set_in_env_var } = {}) {
} else {
await generate_lambda_functions({ builder, split, publish });
}
},

supports: {
// reading from the filesystem only works in serverless functions
read: ({ route }) => {
if (edge) {
throw new Error(
`${name}: Cannot use \`read\` from \`$app/server\` in route \`${route.id}\` when using edge functions`
Rich-Harris marked this conversation as resolved.
Show resolved Hide resolved
);
}

return true;
}
}
};
}
Expand Down
2 changes: 1 addition & 1 deletion packages/adapter-netlify/package.json
Original file line number Diff line number Diff line change
Expand Up @@ -51,6 +51,6 @@
"vitest": "^1.2.0"
},
"peerDependencies": {
"@sveltejs/kit": "^2.0.0"
"@sveltejs/kit": "^2.4.0"
}
}
4 changes: 3 additions & 1 deletion packages/adapter-netlify/src/serverless.js
Original file line number Diff line number Diff line change
@@ -1,6 +1,7 @@
import './shims';
import { Server } from '0SERVER';
import { split_headers } from './headers.js';
import { createReadableStream } from '@sveltejs/kit/node';

/**
* @param {import('@sveltejs/kit').SSRManifest} manifest
Expand All @@ -10,7 +11,8 @@ export function init(manifest) {
const server = new Server(manifest);

let init_promise = server.init({
env: process.env
env: process.env,
read: (file) => createReadableStream(`.netlify/server/${file}`)
});

return async (event, context) => {
Expand Down
1 change: 1 addition & 0 deletions packages/adapter-node/ambient.d.ts
Original file line number Diff line number Diff line change
Expand Up @@ -9,6 +9,7 @@ declare module 'HANDLER' {
declare module 'MANIFEST' {
import { SSRManifest } from '@sveltejs/kit';

export const base: string;
export const manifest: SSRManifest;
export const prerendered: Set<string>;
}
Expand Down
11 changes: 9 additions & 2 deletions packages/adapter-node/index.js
Original file line number Diff line number Diff line change
Expand Up @@ -39,8 +39,11 @@ export default function (opts = {}) {

writeFileSync(
`${tmp}/manifest.js`,
`export const manifest = ${builder.generateManifest({ relativePath: './' })};\n\n` +
`export const prerendered = new Set(${JSON.stringify(builder.prerendered.paths)});\n`
[
`export const manifest = ${builder.generateManifest({ relativePath: './' })};`,
`export const prerendered = new Set(${JSON.stringify(builder.prerendered.paths)});`,
`export const base = ${JSON.stringify(builder.config.kit.paths.base)};`
].join('\n\n')
);

const pkg = JSON.parse(readFileSync('package.json', 'utf8'));
Expand Down Expand Up @@ -86,6 +89,10 @@ export default function (opts = {}) {
ENV_PREFIX: JSON.stringify(envPrefix)
}
});
},

supports: {
read: () => true
}
};
}
2 changes: 1 addition & 1 deletion packages/adapter-node/package.json
Original file line number Diff line number Diff line change
Expand Up @@ -50,6 +50,6 @@
"rollup": "^4.9.5"
},
"peerDependencies": {
"@sveltejs/kit": "^2.0.0"
"@sveltejs/kit": "^2.4.0"
}
}
13 changes: 10 additions & 3 deletions packages/adapter-node/src/handler.js
Original file line number Diff line number Diff line change
Expand Up @@ -4,15 +4,15 @@ import path from 'node:path';
import sirv from 'sirv';
import { fileURLToPath } from 'node:url';
import { parse as polka_url_parser } from '@polka/url';
import { getRequest, setResponse } from '@sveltejs/kit/node';
import { getRequest, setResponse, createReadableStream } from '@sveltejs/kit/node';
import { Server } from 'SERVER';
import { manifest, prerendered } from 'MANIFEST';
import { manifest, prerendered, base } from 'MANIFEST';
import { env } from 'ENV';

/* global ENV_PREFIX */

const server = new Server(manifest);
await server.init({ env: process.env });

const origin = env('ORIGIN', undefined);
const xff_depth = parseInt(env('XFF_DEPTH', '1'));
const address_header = env('ADDRESS_HEADER', '').toLowerCase();
Expand All @@ -29,6 +29,13 @@ if (isNaN(body_size_limit)) {

const dir = path.dirname(fileURLToPath(import.meta.url));

const asset_dir = `${dir}/client${base}`;

await server.init({
env: process.env,
read: (file) => createReadableStream(`${asset_dir}/${file}`)
});

/**
* @param {string} path
* @param {boolean} client
Expand Down
5 changes: 3 additions & 2 deletions packages/adapter-vercel/files/serverless.js
Original file line number Diff line number Diff line change
@@ -1,5 +1,5 @@
import { installPolyfills } from '@sveltejs/kit/node/polyfills';
import { getRequest, setResponse } from '@sveltejs/kit/node';
import { getRequest, setResponse, createReadableStream } from '@sveltejs/kit/node';
import { Server } from 'SERVER';
import { manifest } from 'MANIFEST';

Expand All @@ -8,7 +8,8 @@ installPolyfills();
const server = new Server(manifest);

await server.init({
env: /** @type {Record<string, string>} */ (process.env)
env: /** @type {Record<string, string>} */ (process.env),
read: createReadableStream
});

const DATA_SUFFIX = '/__data.json';
Expand Down
32 changes: 25 additions & 7 deletions packages/adapter-vercel/index.js
Original file line number Diff line number Diff line change
Expand Up @@ -5,6 +5,7 @@ import { nodeFileTrace } from '@vercel/nft';
import esbuild from 'esbuild';
import { get_pathname } from './utils.js';

const name = '@sveltejs/adapter-vercel';
const DEFAULT_FUNCTION_NAME = 'fn';

const get_default_runtime = () => {
Expand All @@ -24,7 +25,7 @@ const plugin = function (defaults = {}) {
}

return {
name: '@sveltejs/adapter-vercel',
name,

async adapt(builder) {
if (!builder.routes) {
Expand Down Expand Up @@ -63,6 +64,8 @@ const plugin = function (defaults = {}) {
* @param {import('@sveltejs/kit').RouteDefinition<import('.').Config>[]} routes
*/
async function generate_serverless_function(name, config, routes) {
const dir = `${dirs.functions}/${name}.func`;

const relativePath = path.posix.relative(tmp, builder.getServerDirectory());

builder.copy(`${files}/serverless.js`, `${tmp}/index.js`, {
Expand All @@ -77,12 +80,12 @@ const plugin = function (defaults = {}) {
`export const manifest = ${builder.generateManifest({ relativePath, routes })};\n`
);

await create_function_bundle(
builder,
`${tmp}/index.js`,
`${dirs.functions}/${name}.func`,
config
);
await create_function_bundle(builder, `${tmp}/index.js`, dir, config);

for (const asset of builder.findServerAssets(routes)) {
// TODO use symlinks, once Build Output API supports doing so
builder.copy(`${builder.getServerDirectory()}/${asset}`, `${dir}/${asset}`);
}
}

/**
Expand Down Expand Up @@ -335,6 +338,21 @@ const plugin = function (defaults = {}) {
builder.log.minor('Writing routes...');

write(`${dir}/config.json`, JSON.stringify(static_config, null, '\t'));
},

supports: {
// reading from the filesystem only works in serverless functions
read: ({ config, route }) => {
const runtime = config.runtime ?? defaults.runtime;

if (runtime === 'edge') {
throw new Error(
`${name}: Cannot use \`read\` from \`$app/server\` in route \`${route.id}\` configured with \`runtime: 'edge'\``
);
}

return true;
}
}
};
};
Expand Down
2 changes: 1 addition & 1 deletion packages/adapter-vercel/package.json
Original file line number Diff line number Diff line change
Expand Up @@ -42,6 +42,6 @@
"vitest": "^1.2.0"
},
"peerDependencies": {
"@sveltejs/kit": "^2.0.0"
"@sveltejs/kit": "^2.4.0"
}
}
1 change: 1 addition & 0 deletions packages/kit/scripts/generate-dts.js
Original file line number Diff line number Diff line change
Expand Up @@ -13,6 +13,7 @@ await createBundle({
'$app/forms': 'src/runtime/app/forms.js',
'$app/navigation': 'src/runtime/app/navigation.js',
'$app/paths': 'src/runtime/app/paths/types.d.ts',
'$app/server': 'src/runtime/app/server/index.js',
'$app/stores': 'src/runtime/app/stores.js'
},
include: ['src']
Expand Down
8 changes: 8 additions & 0 deletions packages/kit/src/core/adapt/builder.js
Original file line number Diff line number Diff line change
Expand Up @@ -10,6 +10,7 @@ import { get_env } from '../../exports/vite/utils.js';
import generate_fallback from '../postbuild/fallback.js';
import { write } from '../sync/utils.js';
import { list_files } from '../utils.js';
import { find_server_assets } from '../generate_manifest/find_server_assets.js';

const pipe = promisify(pipeline);
const extensions = ['.html', '.js', '.mjs', '.json', '.css', '.svg', '.xml', '.wasm'];
Expand Down Expand Up @@ -144,6 +145,13 @@ export function create_builder({
}
},

findServerAssets(route_data) {
Rich-Harris marked this conversation as resolved.
Show resolved Hide resolved
return find_server_assets(
build_data,
route_data.map((route) => /** @type {import('types').RouteData} */ (lookup.get(route)))
);
},

async generateFallback(dest) {
const manifest_path = `${config.kit.outDir}/output/server/manifest-full.js`;
const env = get_env(config.kit.env, vite_config.mode);
Expand Down
52 changes: 52 additions & 0 deletions packages/kit/src/core/generate_manifest/find_server_assets.js
Original file line number Diff line number Diff line change
@@ -0,0 +1,52 @@
import { find_deps } from '../../exports/vite/build/utils.js';

/**
* Finds all the assets that are imported by server files associated with `routes`
* @param {import('types').BuildData} build_data
* @param {import('types').RouteData[]} routes
*/
export function find_server_assets(build_data, routes) {
/**
* All nodes actually used in the routes definition (prerendered routes are omitted).
* Root layout/error is always included as they are needed for 404 and root errors.
* @type {Set<any>}
*/
const used_nodes = new Set([0, 1]);

// TODO add hooks.server.js asset imports
/** @type {Set<string>} */
const server_assets = new Set();

/** @param {string} id */
function add_assets(id) {
if (id in build_data.server_manifest) {
const deps = find_deps(build_data.server_manifest, id, false);
for (const asset of deps.assets) {
server_assets.add(asset);
}
}
}

for (const route of routes) {
if (route.page) {
for (const i of route.page.layouts) used_nodes.add(i);
for (const i of route.page.errors) used_nodes.add(i);
used_nodes.add(route.page.leaf);
}

if (route.endpoint) {
add_assets(route.endpoint.file);
}
}

for (const n of used_nodes) {
const node = build_data.manifest_data.nodes[n];
if (node?.server) add_assets(node.server);
}

if (build_data.manifest_data.hooks.server) {
add_assets(build_data.manifest_data.hooks.server);
}

return Array.from(server_assets);
}
Loading
Loading