Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

testscript: add RequireUniqueNames parameter #185

Merged
merged 1 commit into from
Jan 18, 2023

Conversation

twpayne
Copy link
Contributor

@twpayne twpayne commented Sep 27, 2022

Fixes #184.

@twpayne twpayne changed the title testscript: Fail testscript if txtar contains duplicate filenames testscript: add RequireUniqueFilenames parameter Sep 27, 2022
@twpayne twpayne marked this pull request as draft December 13, 2022 21:53
@twpayne twpayne changed the title testscript: add RequireUniqueFilenames parameter testscript: add RequireUniqueNames parameter Dec 13, 2022
@twpayne twpayne marked this pull request as ready for review December 13, 2022 22:01
@twpayne
Copy link
Contributor Author

twpayne commented Dec 13, 2022

The test failure seems unrelated to this PR.

Copy link
Collaborator

@mvdan mvdan left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks! Apologies it took so long to get back to you. There's also a conflict, but I imagine it's a trivial one.

[linux] testscript -unique-names scripts-case-insensitive

[windows] ! testscript -unique-names scripts-case-insensitive
[windows] stdout 'duplicate name'
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Do Mac and Windows always use case insensitive filesystems?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We both know that they don't always :) It's rare, but possible, for macOS users to run their systems with case sensitive filesystems, for example, even though this tends to break a large number of macOS applications.

The effort of making the tests detect whether the filesystem they're running on is case sensitive or not is fairly large. I think you'd have to create a file with a long random lowercase name (to avoid conflicts with any existing files) and test if it exists with the name uppercased. You'd have to be careful to use the exact directory that the tests run in. You can't use a temporary directory as it by be mounted as a different filesystem.

So, for now, I've removed these parts of the tests.

@twpayne
Copy link
Contributor Author

twpayne commented Jan 17, 2023

There's also a conflict, but I imagine it's a trivial one.

The conflict is indeed trivial, but the tests no longer pass. Specifically, the test that testscript should return an error when duplicate names are encountered fails. I've traced this as far as verifying that ts.Fatalf is called in ts.setup when duplicate names are found, but this does not result in testscript returning an error. It should.

I suspect that #192 might be related. Specifically, does #192 cope with ts.Fatalf being called in ts.setup? I get the impression that the error is now silently swallowed somewhere rather than being returned. cc @rogpeppe

rogpeppe added a commit that referenced this pull request Jan 17, 2023
The new FailNow logic introduced in #192 had a flaw: it did
not correctly handle failures in the setup code, as observed in #185.

This PR fixes that omission.
@rogpeppe
Copy link
Owner

@twpayne You're right: there was indeed an issue in that area that I'd overlooked. I've proposed #193 to fix that.

mvdan pushed a commit that referenced this pull request Jan 17, 2023
The new FailNow logic introduced in #192 had a flaw: it did
not correctly handle failures in the setup code, as observed in #185.

This PR fixes that omission.
@mvdan
Copy link
Collaborator

mvdan commented Jan 17, 2023

Please rebase and try again :)

@twpayne
Copy link
Contributor Author

twpayne commented Jan 17, 2023

Thanks both! I've rebased and now the test passes :)

Copy link
Owner

@rogpeppe rogpeppe left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM with one suggestion, thanks!

if ts.params.RequireUniqueNames {
switch _, err := os.Lstat(name); {
case err == nil:
ts.Fatalf("'%s' would overwrite '%s' (because RequireUniqueNames is enabled)", f.Name, name)
Copy link
Owner

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

nit: use %q rather than single quotes.
But how about an alternative approach? Instead of using an explicit LStat, just pass the write options to os.OpenFile such that it will refuse to overwrite an existing file.

You could define an alternative write-file function to make this easy; for example:

func writeFile(name string, data []byte, excl bool) error {
	oflags := os.O_WRONLY | os.O_CREATE | os.O_TRUNC
	if excl {
		oflags |= os.O_EXCL
	}
	f, err := os.OpenFile(name, oflags, 0o666)
	if err != nil {
		return err
	}
	defer f.Close()
	if _, err := f.Write(data); err != nil {
		return fmt.Errorf("cannot write file contents: %v", err)
	}
	return nil
}

Then below, do:

ts.Check(writeFile(name, f.Data, ts.params.RequireUniqueNames)

I think the resulting "file exists" error is probably sufficient, although a check of the error with os.IsExist(err) could be used to customise it.

This way we're doing exactly the same amount of work as before.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

But how about an alternative approach? Instead of using an explicit LStat, just pass the write options to os.OpenFile such that it will refuse to overwrite an existing file.

Yes, this is a much better approach.

This way we're doing exactly the same amount of work as before.

It's also more resistant to race conditions: creating an exclusive file is a single atomic syscall, whereas lstat/open is vulnerable to races if the file is created between the lstat and the open.

Thanks for the feedback. I'll update this PR with your suggestions.

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Good point Roger, I missed this!

@twpayne twpayne force-pushed the duplicate-filenames branch 2 times, most recently from 9f8b9b5 to b6415ea Compare January 18, 2023 18:32
Co-authored-by: Roger Peppe <rogpeppe@gmail.com>
@mvdan mvdan merged commit 89b23b0 into rogpeppe:master Jan 18, 2023
@twpayne twpayne deleted the duplicate-filenames branch January 18, 2023 21:51
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

testscript: Add check for duplicate and out-of-order files
3 participants