CLI initial version

[carletex]

NOTE: Further changes to this branch should be done by a PR

from the work we did on https://github.com/carletex/create-dapp-example (real-se2-templates / feat/extension-templateConifg-dgrcode branches)

This PR changes the workflow for users to use scaffold-eth-2. Instead of cloning, users will use this CLI (npx create-whatever-name@latest). It'll let users pick different options (hardhat/foundry + extensions)

Initially, we'll start with hardhat/foundry (so we can merge and iterate), but @dgrcode built an extension system. Read more here

Some ToDos (maybe we should create some issues?):

• Update README (add new way of installing it, dev notes for the cli, etc)
• GH Action to auto-publish to npm (extra: auto-increment version on release/merge)
• Check node version (=> 18) and yarn installed before running. We could also have a "dependencies" file (or in config.json) for each extension, so we check them before installing (e.g. foundry extension needs to have foundryup in the system)
• Better ASCII art (or just chalk with color and fancy chars) on the intro message. (render-intro-message.ts) Fixed 42e26bb
• In the outro message hint about yarn chain / yarn deploy / yarn start (render-outro-message.ts). Fixed da43950
• Fix foundry template: Let's remove the need of having to create a .env file manually (the cli can do it for you... for now, until we find a system for extensions to add their own .env) Fixed 8e05bbb
• Create the graph extension
• Submodules: Foundry uses submodules, and we are setting that up directly on the CLI. It'd be ideal to move this to the template (e.g. the template declares some submodules, and the CLI dynamically set them up for every extesion)
• Contributions: Find the best way to test PRs, develop new features with this system without having to copy/paste from a "installed instance". Something like hot-reloading
• Contributions: Move to a release branch model (TLDR: instead of merging directly to main, use relase branches: eg. release-2.1.0 to batch stuff and the merge to master. Only hotfix goes directly to main)
• Add prettier / husky / linting (and maybe remove some of them from the base template)

We don't need to solve all of them before merging.

PS: There are a lot of "files changed", but most of them are just moving things to the template folders (which git is detecting :))

[carletex]

Went for create-eth which was suggested by @austintgriffith and it was free :D

npx create-eth@latest

Could you guys try? @sverps @rin-st @Pabl0cks @damianmarti (not tagging @technophile-04 because he is next to me IRL rn :D)

[damianmarti]

@carletex this is awesome!! :-)

I tested it trying to think that I'm a new user of SE-2 and I think that these improvements can be good things to do:

1. Add a progress bar when installing the dependencies with yarn (it takes some time and I think that the loading is not enough, at least for me I like to see what is happening or at least a progress bar moving). I don't know how easy or hard is to add it, I like to use the command pv on Ubuntu for this kind of thing...
2. It's on the ToDo list, but I think that after finishing we can add a short version of the README file, and add a reference to where the user can find more information (full README file or whatever).
3. Add some information about installing extensions and the available extensions.

I used the installed se-2 project and everything seems fine.

We are close to having the cli ready!! 🥳

[rin-st]

Hey, great job!

For now I'm getting an error when trying to preinstall packages for both hh and foundry (Install packages? y). Part of the log:

...
➤ YN0007: │ bufferutil@npm:4.0.7 must be built because it never has been before or the last one failed
➤ YN0007: │ utf-8-validate@npm:5.0.10 must be built because it never has been before or the last one failed
➤ YN0009: │ se-2@workspace:. couldn't be built successfully (exit code 1, logs can be found here: /private/var/folders/rs/t2dhts8n6hv59mn9srqyh74r0000gn/T/xfs-b42708b1/build.log)
➤ YN0009: │ deasync@npm:0.1.28 couldn't be built successfully (exit code 1, logs can be found here: /private/var/folders/rs/t2dhts8n6hv59mn9srqyh74r0000gn/T/xfs-dd82bdf5/build.log)
➤ YN0007: │ ssh2@npm:1.14.0 must be built because it never has been before or the last one failed
➤ YN0000: └ Completed in 18s 70ms
➤ YN0000: Failed with errors in 35s 91ms

    at makeError (/Users/liana/Documents/se-2/node_modules/pkg-install/node_modules/execa/index.js:174:9)
    at /Users/liana/Documents/se-2/node_modules/pkg-install/node_modules/execa/index.js:278:16
    at process.processTicksAndRejections (node:internal/process/task_queues:95:5) {
  code: 1,
  stdout: '➤ YN0000: ┌ Resolution step\n' +
    '➤ YN0032: │ keccak@npm:3.0.3: Implicit dependencies on node-gyp are discouraged\n' +
    '➤ YN0061: │ debug@npm:3.2.6 is deprecated: Debug versions >=3.2.0 <3.2.7 || >=4 <4.3.1 have a low-severity ReDos regression when used in a Node.js environment. It is recommended you upgrade to 3.2.7 or 4.3.1. (https://github.com/visionmedia/debug/issues/797)\n' +
...

build.log files:

# This file contains the result of Yarn building a package (se-2@workspace:.)
# Script name: postinstall

husky - .git can't be found (see https://typicode.github.io/husky/#/?id=custom-directory)

and

# This file contains the result of Yarn building a package (deasync@npm:0.1.28)
# Script name: install

�[31m�[1mUsage Error�[22m�[39m: Couldn't find a script name "node-gyp" in the top-level (used by deasync@npm:0.1.28). This typically happens because some package depends on "node-gyp" to build itself, but didn't list it in their dependencies. To fix that, please run "yarn add node-gyp" into your top-level workspace. You also can open an issue on the repository of the specified package to suggest them to use an optional peer dependency.

�[1m$ �[22myarn run [--inspect] [--inspect-brk] [-T,--top-level] [-B,--binaries-only] <scriptName> ...
Build failed

so something wrong with husky, .git or node-gyp. Tried to install node-gyp and change node version to 18 but it didn't help. Will try to debug and understand how it works later today, but not sure it will be fast.

Other than that noticed that foundry project readme should be changed, but it's already mentioned. And name of the local network for foundry shouldn't be hardhat :)

[technophile-04]

@rin-st did you get this when doing it using npx create-eth@latest or you were trying it out locally using yarn cli ?

Because we were getting this error when you are trying it out locally using yarn cli and you select the project name in the current directory (where all the cli stuff is), like you need to create it outside this project directory :

eg :

> yarn cli 

 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+
 | Create Scaffold-ETH 2 app |
 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+

?  Your project name: ../my-dapp-example

We need to document this problem in README while working locall....But could you please confirm if you got this error while doing npx create-eth@latest or doing yarn cli locally ?

[rin-st]

@technophile-04 I have an error in both npx create-eth@latest and yarn cli. But using ../my-dapp-example works for both cases 👍

[rin-st]

btw I was trying to use npx create-eth@latest from se-2 folder. If I use it from outside git repo it works as expected

[rin-st]

changing order to initializing git -> installing deps fixes my errors for all cases

[Pabl0cks]

Awesome job! Was exciting to see @carletex and @technophile-04 working on this IRL 🚀

On Windows (both my desk and laptop) it's working like a charm when I choose hardhat, but when I choose foundry, I got problems to yarn chain.

 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+
 | Create Scaffold-ETH 2 app |
 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+

? Your project name: test3
? What solidity framework do you want to use? foundry
? Install packages? Yes


  √ 📁 Create project directory C:\Users\Pablo\test3
  √ 🚀 Creating a new Scaffold-ETH 2 app in test3
  √ 📦 Installing dependencies with yarn, this could take a while
  √ 📡 Initializing git repository



  Congratulations! Your project has been scaffolded! 🎉

  Next steps:

  cd test3

        Start the local development node
        yarn chain

        In a new terminal window, deploy your contracts
        yarn deploy

        In a new terminal window, start the frontend
        yarn start

  Thanks for using Scaffold-ETH 2 🙏, Happy Building!


$ cd test3

 ~/test3 (main)

$ yarn chain
"anvil" no se reconoce como un comando interno o externo,
programa o archivo por lotes ejecutable.
command not found: anvil

[carletex]

Thanks for the feedback and the help @rin-st.

btw I was trying to use npx create-eth@latest from se-2 folder. If I use it from outside git repo it works as expected

Yep, this was happening when running it inside an existing git repo (because of the postinstall yarn hook). Your fix works perfectly, but it leaves the yarn.lock file uncommitted (changed after running yarn install**), so we decided to take a different approach (git init before yarn install, but add & commit after yarn install) and it seems to work. @technophile-04 will push in a bit.

** This is expected since we have an empty yarn.lock file, to avoid conflicts when we have a bunch of extensions. So it's generated on the first install. In package.json files, we changed ^ for ~, so with packages bugfixes there is less risk of something breaking (we tried to do exact versions in package.json, but for some reason the peer-dependencies weren't installing)

On Windows (both my desk and laptop) it's working like a charm when I choose hardhat, but when I choose foundry, I got problems to yarn chain.

Thanks @Pabl0cks ! Yes, this happens because people need to install foundryup (that contains anvil and some other tools) in their system. What we are going to do is check if they have it installed and if not show a message hinting them to install.

We'll be pushing (& publishing) in a bit.

Thanks all!

[Pabl0cks]

Thanks @Pabl0cks ! Yes, this happens because people need to install foundryup (that contains anvil and some other tools) in > their system. What we are going to do is check if they have it installed and if not show a message hinting them to install.

Thx @carletex I had no idea about that. Now with foundryup is working great!
Getting the informative message before installing it, so I know I have to install foundryup => Installed it and tried again, and CLI is working perfect with foundry also, got my working SE-2 🙌

I'll send a nitpick edge case:

• When selecting foundry, and selecting "No" at the "Install packages?" option, it shows a finish icon at the Git step, but I don't get any feedback or next steps after instalation. With hardhat and "No packages" I get the congrats message with the steps to start SE-2.

That's all I get with foundry:

$ npx create-eth@latest

+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
| Create Scaffold-ETH 2 app |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+

? Your project name: test9
? What solidity framework do you want to use? foundry
? Install packages? No

√ 📁 Create project directory C:\Users\Pablo\test9
√ 🚀 Creating a new Scaffold-ETH 2 app in test9
↓ 📦 Installing dependencies with yarn, this could take a while [skipped]
→ Manually skipped
√ 📡 Initializing Git repository

[carletex]

When selecting foundry, and selecting "No" at the "Install packages?" option, it shows a finish icon at the Git step, but I don't get any feedback or next steps after instalation. With hardhat and "No packages" I get the congrats message with the steps to start SE-2.

Yes @Pabl0cks! I think Shiv solved it here: 9bb9422

I haven't published it yet to NPM but it should work if run the CLI locally (instead of create-eth)

[dgrcode]

I'm having a look at the PR. Some thoughts/questions:

• Shouldn't we commit the dist/ folder? I'm curious how npx is working without the dist folder. Do they run the build task on their servers? 🤷
• What do you think about committing the .vscode/ folder to keep the debugger configuration? It definitely helps me, and I guess it could help anybody trying to debug the cli.
• I've talked with @carletex about the two files that need to be renamed: .env-default and .gitignore-keep. I'll convert them to templates so we can skip the renames and avoid exception for certain files.

TODO list for myself:

• Fixed by 2599a95 Use templates for renamed files
• Should be fixed by Remove unused types #432. Remove unused types from src/types.ts
• Fixed by 2f36b2e. Look into why findFilesRecursiveSync wasn't working for .gitignore-keep and .env-default. Even though we won't need to rename files, it might have a bug that's not showing up and could break something in the future.

[dgrcode]

We can update the smartContractFramework legacy stuff from here so it works with the current setup. I'll look into it.

[dgrcode]

just something minor to keep consistent indentation. The indented 4 spaces here are added before the \t, but down below at line 29 for example, we have 6 spaces before the \t. We might want to use \t\t without spaces on the left to always keep the same spacing.

[carletex]

@dgrcode

Shouldn't we commit the dist/ folder? I'm curious how npx is working without the dist folder. Do they run the build task on their servers?

Yep, it builds there

What do you think about committing the .vscode/ folder to keep the debugger configuration? It definitely helps me, and I guess it could help anybody trying to debug the cli.

We decided against it in SE-2. Not everyone uses VSCode (eg. I use Intellij and Shiv uses Vim). The same way that I don't want to commit the .idea folder from my Intelij

The ToDos look great and pretty straightforward. Also remember to take a look at the "how to contribute" stuff (hot reloading, symlinks, different repo) which is the most important piece.

[carletex]

Closing this in favor of https://github.com/scaffold-eth/create-eth

As we discussed, the CLI version will live in its own repo (GitHub actions already publishing to NPM in that repo)