Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat(gatsby-plugin-sharp): Use file streams instead of file paths #33029

Merged
merged 6 commits into from
Sep 3, 2021

Conversation

LekoArts
Copy link
Contributor

@LekoArts LekoArts commented Sep 2, 2021

Description

Converts sharp usage based on file paths to file streams, e.g.

// Before
const imgStats = await sharp(file.absolutePath).stats()

// After
const pipeline = sharp()
fs.createReadStream(file.absolutePath).pipe(pipeline)
const imgStats = await pipeline.stats()

For the writing of files we use sharpPipeline.toBuffer() + fs.writeFile() for now. We probably will change it to filestream in future PR (will just need to revert 31cfaf3).

Running the image-processing benchmark didn't show any major speed improvement/degradation

Related Issues

[ch37488]

@LekoArts LekoArts added the topic: media Related to gatsby-plugin-image, or general image/media processing topics label Sep 2, 2021
@gatsbot gatsbot bot added the status: triage needed Issue or pull request that need to be triaged and assigned to a reviewer label Sep 2, 2021
@LekoArts LekoArts removed the status: triage needed Issue or pull request that need to be triaged and assigned to a reviewer label Sep 2, 2021
@LekoArts LekoArts marked this pull request as draft September 2, 2021 11:47
@LekoArts LekoArts marked this pull request as ready for review September 2, 2021 12:13
@KyleAMathews
Copy link
Contributor

Nice! Might reduce memory usage perhaps depending on how Sharp does things

@wardpeet
Copy link
Contributor

wardpeet commented Sep 2, 2021

@LekoArts & @pieh by making queueImageResizing and others async, doesn't this break the contract with plugins like gatsby-plugin-sqip? Should we go to a git patch instead to not break this in master?

@LekoArts
Copy link
Contributor Author

LekoArts commented Sep 3, 2021

Good point, we also use the function in gatsby-transformer-sharp. If we want to keep the behavior then we need to do git patch / adjust more

@pieh
Copy link
Contributor

pieh commented Sep 3, 2021

I reverted async change to queueImageResizing from this PR - breaking contract with it is valid reason to do so. I still think overall it would be beneficial for it to be async, but it doesn't really belong in this PR. Potential way to move forward with it (in another PR) would be introducing queueImageResizingAsync variant, updating plugins we maintatin to use and keeping old sync one (and deprecating that one) to not cause too much problems for 3rd party plugins that might rely on it today.

Copy link
Contributor

@wardpeet wardpeet left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM!

@LekoArts LekoArts merged commit e6d9eb3 into master Sep 3, 2021
@LekoArts LekoArts deleted the streamify-sharp branch September 3, 2021 14:18
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
topic: media Related to gatsby-plugin-image, or general image/media processing topics
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants