Custom Engine downloads to cwd instead of a temporary directory

[timbo24]

Enhancement Suggestion / Bug Report

Can't eliminate errors on successful downloads using a custom engine

Steps which explain the enhancement or reproduce the bug

Cartfile.resolved:

git "ssh://git@source.vivint.com:7999/iOS/VIVProtobuf.git" "3.5.1"

1. When running> rome download --platfform iOS,watchOS
2. Ignoring missing bcsymbolmaps

Current behavior

Error: could not download Protobuf : user error (Binary was not downloaded by engine)
Error: could not download Protobuf.dSYM : user error (Binary was not downloaded by engine)
Error: could not download .VIVProtobuf.version : user error (Binary was not downloaded by engine)

Even though all of the framework files, dSYM files, .version, and expected bcsymbolfiles have been downloaded successfully.

Suggested behavior

I have had it in the past where this output a successful copy of .version files. This is what I would expect

Rome version:

0.23.1.61

OS and version:

Catalina 10.15.1

[timbo24]

It seems like it only throws these errors if I haven't uploaded from branch I'm downloading in.

If I upload first and then download I get a success:

Copied .VIVProtobuf.version to: Carthage/Build/.VIVProtobuf.version

[tmspzz]

The errors are thrown by the engine and reported to rome, so this looks like a problem in your engine?

Am I missing something?

[timbo24]

How do we report to rome a success or error? I'm exiting the engine with 0 on success yet rome still reports an error. But somehow after I upload and then download it changes even though the output of download is the same.

[tmspzz]

via POSIX exit codes.

Uploading: https://github.com/tmspzz/Rome/blob/master/src/Engine/Uploading.hs#L128
Downloading: https://github.com/tmspzz/Rome/blob/master/src/Engine/Downloading.hs#L255

Seems to me, according to your error, that the file is not placed where rome expects it to be: https://github.com/tmspzz/Rome/blob/master/src/Engine/Downloading.hs#L260

You can't arbitrarily choose the path nor rename the artifact.

[timbo24]

I resolved it, it doesn't make sense to me why I'd download directly to the intermediate location of the outputPath and not just to the cache location. Why not just use the cache and the Carthage/Build locations? Why clutter up the project space with a bunch of Framwork directories?

[tmspzz]

Because your engine is driven by the rome's logic. For example the engine doesn't know if a framework is static or dynamic nor if it's just one build product out of many for a project, or if you have multiple caching levels enabled.

Since it doesn't have a the complete picture of what the logic is (to keep things simpler for your scripting side), you have to respect the conventions. The alternative would be to expose all information via CLI parameters or env variables to the scripts.

What clutter are you seeing by the way? What is the output path?

[tmspzz]

By the way it would be amazing if you could contribute a reusable version of your script that uploads to Nexus for others to reuse as well 🙇

[timbo24]

Maybe I'm doing something wrong but it sounds like for rome to work you have to download directly to the output-path specified by the second param to download coming into the engine, which in our case is in our project dir. Why not instead expect to download directly to the cache and from the cache copy over the frameworks directly to the Carthage/Build directory. After running rome download our project space has a bunch of framework files left over.

Yeah I can do that, we're using an FTP server to host the cache.

[tmspzz]

Yeah I can do that, we're using an FTP server to host the cache.

Thanks 🙇

After running rome download our project space has a bunch of framework files left over.

Let me check once again.

[tmspzz]

can you paste your Romefile's cache's section? Are you using a local cache? are you passing --skip-local-cache?

[timbo24]

I'm using a local cache:

cache:
local: ~/Library/Caches/Rome
engine: ../scripts/romeEngine.rb

[tmspzz]

and you're not passing --skip-local-cache ?

If you are infact using the local cache, then I suspect there is indeed a bug where the intermediate directory is not removed.

Maybe @BalestraPatrick can pitch in here.

[timbo24]

Yes, I am not using --skip-local-cache

[tmspzz]

Ongoing fix here: #206

[tmspzz]

@timbo24 please try pre-release https://github.com/tmspzz/Rome/releases/tag/v23.2.62

[tmspzz]

@timbo24 any feedback? I would like to merge and release today

[tmspzz]

This should be fixed in https://github.com/tmspzz/Rome/releases/tag/v0.23.2.63