Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Use 'uploadDataDuringCreation' option when uploading via tus #7111

Merged
merged 4 commits into from
Jun 13, 2022
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
6 changes: 6 additions & 0 deletions changelog/unreleased/bugfix-upload-meta-serialization
Original file line number Diff line number Diff line change
@@ -0,0 +1,6 @@
Bugfix: Upload meta data serialization

We've fixed a bug where meta properties of uploading resources could not be serialized, resulting in unnecessary network requests.

https://github.com/owncloud/web/pull/6846
https://github.com/owncloud/web/issues/6819
6 changes: 6 additions & 0 deletions changelog/unreleased/enhancement-upload-data-during-creation
Original file line number Diff line number Diff line change
@@ -0,0 +1,6 @@
Enhancement: Upload data during creation
JammingBen marked this conversation as resolved.
Show resolved Hide resolved

Uploading via tus now uses the `uploadDataDuringCreation` option which saves up one request.

https://github.com/owncloud/web/pull/7111
https://github.com/owncloud/web/issues/7066
Original file line number Diff line number Diff line change
Expand Up @@ -673,11 +673,11 @@ export default defineComponent({
const uploadSizeSpaceMapping = uppyResources.reduce((acc, uppyResource) => {
let targetUploadSpace

if (uppyResource.meta.route.params?.storage === 'home') {
if (uppyResource.meta.routeName === 'files-spaces-personal') {
targetUploadSpace = this.spaces.find((space) => space.driveType === 'personal')
} else {
targetUploadSpace = this.spaces.find(
(space) => space.id === uppyResource.meta.route?.params?.storageId
(space) => space.id === uppyResource.meta.routeStorageId
)
}

Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -226,11 +226,16 @@ const inputFilesToUppyFiles = ({
currentFolder,
relativeFolder: directory,
relativePath: relativeFilePath, // uppy needs this property to be named relativePath
route: fileRoute,
tusEndpoint,
webDavBasePath: unref(webDavBasePath), // WebDAV base path where the files will be uploaded to
uploadId: uuid.v4(),
topLevelFolderId
topLevelFolderId,
routeName: fileRoute.name,
routeItem: fileRoute.params?.item || '',
routeShareName: (fileRoute.params as any)?.shareName || '',
routeShareId: (fileRoute.query as any)?.shareId || '',
routeStorage: (fileRoute.params as any)?.storage || '',
routeStorageId: (fileRoute.params as any)?.storageId || ''
}
})
}
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -176,11 +176,7 @@ describe('CreateAndUpload component', () => {
size: 1001
},
meta: {
route: {
params: {
storage: 'home'
}
}
routeName: 'files-spaces-personal'
}
}
])
Expand Down
4 changes: 4 additions & 0 deletions packages/web-pkg/src/composables/capability/useCapability.ts
Original file line number Diff line number Diff line change
Expand Up @@ -43,3 +43,7 @@ export const useCapabilityFilesTusSupportMaxChunkSize = createCapabilityComposab
'files.tus_support.max_chunk_size',
0
)
export const useCapabilityFilesTusExtension = createCapabilityComposable<string>(
'files.tus_support.extension',
''
)
20 changes: 19 additions & 1 deletion packages/web-runtime/src/components/UploadInfo.vue
Original file line number Diff line number Diff line change
Expand Up @@ -343,7 +343,7 @@ export default {
} else {
this.uploads[file.meta.uploadId].path = `${file.meta.currentFolder}${file.name}`
}
this.uploads[file.meta.uploadId].targetRoute = file.meta.route
this.uploads[file.meta.uploadId].targetRoute = this.buildRouteFromUppyResource(file)

if (!file.isFolder) {
this.uploads[file.meta.uploadId].status = 'success'
Expand Down Expand Up @@ -426,6 +426,24 @@ export default {
parentFolderLink(file) {
return this.createFolderLink(path.dirname(file.path), file.storageId, file.targetRoute)
},
buildRouteFromUppyResource(resource) {
if (!resource.meta.routeName) {
return null
}

return {
name: resource.meta.routeName,
query: {
shareId: resource.meta.routeShareId
},
params: {
item: resource.meta.routeItem,
shareName: resource.meta.routeShareName,
storage: resource.meta.routeStorage,
storageId: resource.meta.routeStorageId
}
}
},
defaultParentFolderName(file) {
const { targetRoute } = file
// FIXME: use isLocationSpacesActive(), but it currently lies in the files app
Expand Down
20 changes: 16 additions & 4 deletions packages/web-runtime/src/composables/upload/useUpload.ts
Original file line number Diff line number Diff line change
@@ -1,5 +1,6 @@
import { ClientService } from 'web-pkg/src/services'
import {
useCapabilityFilesTusExtension,
useCapabilityFilesTusSupportHttpMethodOverride,
useCapabilityFilesTusSupportMaxChunkSize,
useClientService,
Expand All @@ -9,7 +10,6 @@ import { computed, Ref, unref, watch } from '@vue/composition-api'
import { useActiveLocation } from 'files/src/composables'
import { isLocationPublicActive } from 'files/src/router'
import { UppyService } from '../../services/uppyService'
import { Location } from 'vue-router/types/router'
import * as uuid from 'uuid'

export interface UppyResource {
JammingBen marked this conversation as resolved.
Show resolved Hide resolved
Expand All @@ -19,14 +19,20 @@ export interface UppyResource {
type: string
data: Blob
meta: {
// must only contain primitive types because the properties can't be serialized otherwise!
currentFolder: string
relativeFolder: string
relativePath: string
route: Location
tusEndpoint: string
webDavBasePath: string
uploadId: string
topLevelFolderId?: string
routeName?: string
routeItem?: string
routeShareName?: string
routeShareId?: string
routeStorage?: string
routeStorageId?: string
JammingBen marked this conversation as resolved.
Show resolved Hide resolved
}
}

Expand All @@ -48,6 +54,7 @@ export function useUpload(options: UploadOptions): UploadResult {

const tusHttpMethodOverride = useCapabilityFilesTusSupportHttpMethodOverride()
const tusMaxChunkSize = useCapabilityFilesTusSupportMaxChunkSize()
const tusExtension = useCapabilityFilesTusExtension()
const uploadChunkSize = computed((): number => store.getters.configuration.uploadChunkSize)

const headers = computed((): { [key: string]: string } => {
Expand All @@ -73,6 +80,7 @@ export function useUpload(options: UploadOptions): UploadResult {
tusMaxChunkSize: unref(tusMaxChunkSize),
uploadChunkSize: unref(uploadChunkSize),
tusHttpMethodOverride: unref(tusHttpMethodOverride),
tusExtension: unref(tusExtension),
headers: unref(headers)
}
}
Expand Down Expand Up @@ -152,9 +160,13 @@ const createDirectoryTree = ({
type: 'folder',
meta: {
relativeFolder: createdSubFolders,
route: file.meta.route,
currentFolder: file.meta.currentFolder,
uploadId
uploadId,
routeName: file.meta.routeName,
routeItem: file.meta.routeItem,
routeShareName: file.meta.routeShareName,
routeStorage: file.meta.routeStorage,
routeStorageId: file.meta.routeStorageId
}
}

Expand Down
7 changes: 6 additions & 1 deletion packages/web-runtime/src/services/uppyService.ts
Original file line number Diff line number Diff line change
Expand Up @@ -34,24 +34,29 @@ export class UppyService {
tusMaxChunkSize,
uploadChunkSize,
tusHttpMethodOverride,
tusExtension,
headers
}: {
tusMaxChunkSize: number
uploadChunkSize: number
tusHttpMethodOverride: boolean
tusExtension: string
headers: { [key: string]: string }
}) {
const chunkSize =
tusMaxChunkSize > 0 && uploadChunkSize !== Infinity
? Math.max(tusMaxChunkSize, uploadChunkSize)
: uploadChunkSize

const uploadDataDuringCreation = tusExtension.includes('creation-with-upload')

const tusPluginOptions = {
headers: headers,
chunkSize: chunkSize,
removeFingerprintOnSuccess: true,
overridePatchMethod: !!tusHttpMethodOverride,
retryDelays: [0, 500, 1000]
retryDelays: [0, 500, 1000],
uploadDataDuringCreation
}

const xhrPlugin = this.uppy.getPlugin('XHRUpload')
Expand Down