fs: implement fs.rmdir recurisve

[zero1five]

Add clearDir options into fs.rmdir and fs.rmdirSync to delete a forder with sub folders or files. This PR is an attempt to port rimraf into the core from the js layer.

Refs: #22686
Refs: #27764

Checklist

• make -j4 test (UNIX), or vcbuild test (Windows) passes
• tests and/or benchmarks are included
• documentation is changed or added
• commit message follows commit guidelines

[zero1five]

IMO, recursive well cannot express whether or not to delete a non-empty folder. It is shaped?

[refack]

Hello @zero1five and thanks for the contribution!

The subject of recursive rmdir (a.k.a. rimraf) has also been is discussion by the @nodejs/tooling team - nodejs/tooling#13.

A few points that were raised:

• Should it even be in the stdlib?
• How to make it robust / best incorporate knowledge gathered by existing npm modules?
• API endpoint name.

Work is also in progress on these ideas by by @iansu.

[zero1five]

How to make it robust / best incorporate knowledge gathered by existing npm modules?

imo, prefer incremental implementation requirements.

@refack Should I move on or shut it down? await node/tooling's idea.

[bcoe]

@zero1five it's great to see other folks from the community excited about this feature. We had a good discussion about rmraf at collaborator summit, a few interesting notes:

• it's a much more difficult problem than mkdirp (more race conditions, you need to think about parallelization).
• something that breaks sometimes is worse than nothing (like @refack says, we need to be as stable as the user-land module, which is part of why @iansu has literally been working with @isaacs on this topic).
• this is probably specialized enough that we should consider moving it into a separate library than fs; an extended fs module.

I'd love to hear what other folks think (don't see any reason to close this immediately).

[sam-github]

To expand on

it's a much more difficult problem than mkdirp (more race conditions, you need to think about parallelization)

rm -rf might superficially appear to be just the inverse of mkdir -p and therefor an obvious inclusion (now that mkdir -p wormed its way into fs), but it is wildly different in that the latter is linear by definition: a directory can't be created until its parent exists, so they are created one by one. Removal is the opposite, its deleting a tree, and should be parallelized for efficiency (but not too parallel, queuing 1000s of simultaneous unlinks into the threadpool could go badly).

Also, robust removal has to deal with the fact that background Windows processors (virus scanners, commonly) open files at random times, and lock files as a side-effect, preventing them from being removed. Its a big problem to get removal robust on Windows, with lots of heuristics involved (exponential backoff and retry), which makes it very, very unlike every other API in fs, which are mostly implemented 1-1 with some underlying Windows or Posixy system call, and therefor have fairly straightforward to understand behaviour.

[Fishrock123]

Could the option be named more clearly? all? subdirectories?

[bcoe]

@zero1five would you be open to combining efforts with @iansu here:

#28208

Ian has been working on this patch with @isaacs', who wrote rimraf; together they've been trying to make sure that all of the edge-cases brought up by @sam-github are appropriately addressed (if there were to be a Node.js implementation of the library).

[zero1five]

@bcoe I'd be pleased to come.

[gengjiawen]

Could the option be named more clearly? all? subdirectories?

Maybe recursive ?

[Trott]

Could the option be named more clearly? all? subdirectories?

Maybe recursive ?

Yes, I like recursive so it is consistent with mkdir. I don't want to have to remember two different names for basically the same thing.

[jasnell]

Perhaps recursive and and option like emptyOnly: false... to make it explicitly clear what to do when a folder is not empty. Specifically, the default value for emptyOnly would be true. The user has to explicitly turn that off.

fs.rmdir(path, { recursive: true, emptyOnly: false })

[addaleax]

@jasnell I don’t think it makes sense to specify recursive: true, emptyOnly: true (because the recursive option can have any value when emptyOnly is set), and it also doesn’t make sense to specify recursive: false, emptyOnly: false, because removing a non-empty directory is only possible after (recursively) emptying it…

So it seems like what you’re suggesting in the end is to name the option emptyOnly? I think that that would be a pretty good name if it weren’t conflicting with our mkdir options and rm -r…

[bcoe]

👋 @zero1five I'm going to close this, because it seems like we're reaching some consensus around choosing to vendor the existing rimraf module.

With two approaches being put forward, either introducing a recursive flag to rmdir:

#29168

or exposing a new method on fs:

#28208

Let's try to move the conversation to these PRs and see if we can keep this moving forward 🎉

[jonschlinkert]

Fwiw, although delete-empty isn't broadly used, I've learned that deleting "empty" directories is more nuanced that it seems at first, and (IMHO) it's probably not something we'd want to handle natively in Node.js.

For example, on macs, it's often debatable whether or not a directory is actually empty, since MacOS sometimes adds a hidden .DS_Store file to directories. Should Node.js ignore that file and delete the directory anyway? If not, users who keep dotfiles hidden will be confused when the directory is kept. And vice versa.

Anyway, just a thought, I know this issue is closed but figured I should comment in case the discussion arises again. This is one of those things that seems really straightforward but (IMHO) isn't.