Issue when deploying Custom Indexes that have commas in their name

[pabesphoy]

We are trying to deploy several metadata files to an organization, specifically several custom indexes with the following name, automatically generated by salesforce:
Account.RD3_Direccion__c,FR_RowID_Magia__c.indx-meta

This is in the customindex file. We know for a fact that it is possible to deploy metadata with commas in its name, because we already have custom indexes in the organization such that if we download the metadata, we find commas in their name. However, whenever we try to deploy this file, we get this error:
$ sf project convert source --source-dir 'force-app/main/default/customindex/Account.RD3_Direccion__c,FR_RowID_Magia__c.indx-meta.xml' --json
{
"code": 1,
"context": "Source",
"commandName": "Source",
"message": "force-app/main/default/customindex/Account.RD3_Direccion__c: File or folder not found",
"name": "SfError",
"status": 1,
"stack": "SfError: force-app/main/default/customindex/Account.RD3_Direccion__c: File or folder not found\n at assertFileExists (/Users/ea_enel/.local/share/sf/client/2.43.7-085948d/node_modules/@salesforce/source-deploy-retrieve/lib/src/collections/componentSetBuilder.js:185:15)\n at validateAndResolvePath (/Users/ea_enel/.local/share/sf/client/2.43.7-085948d/node_modules/@salesforce/source-deploy-retrieve/lib/src/collections/componentSetBuilder.js:182:59)\n at Array.map ()\n at ComponentSetBuilder.build (/Users/ea_enel/.local/share/sf/client/2.43.7-085948d/node_modules/@salesforce/source-deploy-retrieve/lib/src/collections/componentSetBuilder.js:65:44)\n at Source.convert (file:///Users/ea_enel/.local/share/sf/client/2.43.7-085948d/node_modules/@salesforce/plugin-deploy-retrieve/lib/commands/project/convert/source.js:93:55)\n at async Source.run (file:///Users/ea_enel/.local/share/sf/client/2.43.7-085948d/node_modules/@salesforce/plugin-deploy-retrieve/lib/commands/project/convert/source.js:71:9)\n at async Source._run (/Users/ea_enel/.local/share/sf/client/2.43.7-085948d/node_modules/@oclif/core/lib/command.js:311:22)\n at async Config.runCommand (/Users/ea_enel/.local/share/sf/client/2.43.7-085948d/node_modules/@oclif/core/lib/config/config.js:433:25)\n at async run (/Users/ea_enel/.local/share/sf/client/2.43.7-085948d/node_modules/@oclif/core/lib/main.js:92:16)",
"exitCode": 1,
"warnings": []
}
14:20:35 ~/Documents/01_repo/b2b-crm (merge)

We understan this is a bug, because custom indexes would not be able to be deployed without convertir comma-containing source files. Has anyone ever faced this issue?

The sf version we are using is:
$ sf --version
@salesforce/cli/2.43.7 darwin-arm64 node-v20.13.1

{
"architecture": "darwin-arm64",
"cliVersion": "@salesforce/cli/2.46.6",
"nodeVersion": "node-v20.14.0",
"osVersion": "Darwin 23.5.0",
"rootPath": "/Users/ea_enel/.local/share/sf/client/2.46.6-191291b",
"shell": "zsh",
"pluginVersions": [
"@oclif/plugin-autocomplete 3.1.2 (core)",
"@oclif/plugin-commands 4.0.2 (core)",
"@oclif/plugin-help 6.2.1 (core)",
"@oclif/plugin-not-found 3.2.3 (core)",
"@oclif/plugin-plugins 5.2.4 (core)",
"@oclif/plugin-search 1.1.2 (core)",
"@oclif/plugin-update 4.3.5 (core)",
"@oclif/plugin-version 2.2.3 (core)",
"@oclif/plugin-warn-if-update-available 3.1.5 (core)",
"@oclif/plugin-which 3.2.2 (core)",
"@salesforce/cli 2.46.6 (core)",
"apex 3.1.18 (core)",
"auth 3.6.21 (core)",
"data 3.4.6 (core)",
"deploy-retrieve 3.9.4 (core)",
"info 3.3.6 (core)",
"limits 3.3.12 (core)",
"marketplace 1.2.12 (core)",
"org 4.2.1 (core)",
"packaging 2.4.6 (core)",
"schema 3.3.11 (core)",
"settings 2.3.1 (core)",
"sobject 1.4.11 (core)",
"source 3.4.2 (core)",
"telemetry 3.4.0 (core)",
"templates 56.2.11 (core)",
"trust 3.7.5 (core)",
"user 3.5.13 (core)",
"sfdx-plugin-source-read 1.2.0 (user) published 239 days ago (Tue Oct 24 2023)"
]
}

[github-actions]

Thank you for filing this issue. We appreciate your feedback and will review the issue as soon as possible. Remember, however, that GitHub isn't a mechanism for receiving support under any agreement or SLA. If you require immediate assistance, contact Salesforce Customer Support.

[github-actions]

Hello @pabesphoy 👋 It looks like you didn't include the full Salesforce CLI version information in your issue.
Please provide the output of version --verbose --json for the CLI you're using (sf or sfdx).

A few more things to check:

• Make sure you've provided detailed steps to reproduce your issue.
  • A repository that clearly demonstrates the bug is ideal.
• Make sure you've installed the latest version of Salesforce CLI. (docs)
  • Better yet, try the rc or nightly versions. (docs)
• Try running the doctor command to diagnose common issues.
• Search GitHub for existing related issues.

Thank you!

[github-actions]

Hello @pabesphoy 👋 None of the versions of sf you shared match the latest release.

Shared: 2.43.7, 2.43.7
Latest: 2.46.6

Update to the latest version of Salesforce CLI (docs) and confirm that you're still seeing your issue.
You can also try the rc and nightly releases! (docs)

After updating, share the full output of sf version --verbose --json

[shetzel]

specifically several custom indexes with the following name, automatically generated by salesforce

How were these created? A comma in the API name is a very bad idea. The CLI uses a comma character to split on multiple entries and it would be a huge breaking change to lots of people if that were changed.

[edencrash]

As far as we know, administrators do not choose the API name that Salesforce use to refer to Custom Indexes. When refering to Two-Column custom indexes, the autogenerated API Name contains both API Names sepparated by a comma

[shetzel]

As a workaround, are you able to deploy the CustomIndex metadata with sf project deploy start -m CustomIndex?

[shetzel]

Another workaround would be to deploy the customindex directory: sf project convert source --source-dir 'force-app/main/default/customindex

[Alfystar]

hi @shetzel, I'm from the team that found this problem.
We have an automated pipeline that works for single assets, loading the entire directory or selecting metadata with the "-m" option is not a valid workaround in our workflow.

Now in an emergency we are releasing them manually but we would like to resume using the pipeline as soon as possible

[shetzel]

@Alfystar - thanks, that makes sense. I've asked the team who owns the CustomIndex metadata type to clarify some things, including whether they can change how the API names are generated. If that's not possible, then the only solution I can think of is to allow the definition of a different split character, which would likely cause other problems.

[Alfystar]

I think a simpler solution is to add an escape character in the cli, such as the '' before the problematic character.

Or else do, as for example in argvparse, that the lists declare themselves by putting spaces and if the parameter has spaces inside, then use double quotation marks

[Alfystar]

Any news?

[mshanemc]

@Alfystar @edencrash @pabesphoy
I think these will deploy just fine using sf project deploy ...

I think the issue is with the project convert source command , which kept support for commas in the multiple flag scenario to avoid it being a breaking change from the old force:source:convert.

yeah, it's probably possible to fix that, maybe even without breaking something else 😄

Why do you need to convert to mdapi format to do a deployment? Maybe there's a better way than waiting on us to do a code change?

[Alfystar]

Hi @mshanemc
In our pipeline we convert the package for deployment into source-format to allow us to compare it with the repository.
I realize that on this particular metadata it might not be useful, but I'm sure you'll agree with me that a pipeline should be generic.

However, we also did some deployment tests without converting and in any case the cli gives an error, unfortunately

Edit

Looking at the code more closely, I also saw that the problem does not occur with the "source-dir" flag, but with "root-dir" and therefore I cannot even skip the offending directory in the convert

mkMetaCmd = f"sf project convert source --root-dir {sourceFolderShort} --output-dir {metaFolderShort} --json"

[git2gus]

This issue has been linked to a new work item: W-16343745

[mdonnalley]

@Alfystar I'm working on a fix for this now. In your pipeline are you able to insert a \ into the flag value to escape the comma?

It would look something like this:

sf project convert source --source-dir 'force-app/main/default/customindex/Account.RD3_Direccion__c\,FR_RowID_Magia__c.indx-meta.xml' --json

If that doesn't work for you, we'll have to consider another solution that won't break other customers. Perhaps an environment variable that disables the splitting.

Thanks!

[Alfystar]

Yes! I can easily implement a fix like this, but inside the pipeline I would use your feature --flag-dir, would there be any problems?

As far as I'm concerned I can replace all the , in '\,' both on terminal and on flag-file, if this is the change

[Alfystar]

So from which version of sf the fix will start to work?? Thank you!