Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

build: ship a Win11 build of Terminal that's <=half the size #12560

Merged
merged 4 commits into from
Feb 25, 2022

Conversation

DHowett
Copy link
Member

@DHowett DHowett commented Feb 23, 2022

This pull request is broken down into a few individually reviewable units.

00ca4883f - Run the release pipeline twice, for Win10 and Win11, at the same time

This required some changes in how we download artifacts to make sure
that we could control which version of Windows we were processing in any
individual step.

We're also going to patch the package manifest on the Windows 11 version
so the store targets it more specifically.

On top of the prior three commits, this lets us ship a Windows 11
package that costs only ~15MB on disk. The Windows 10 version, for
comparison, is about 40.

51ebdbf8c - Prepare for toggling XAML between 2.7.0 and -prerelease on Win11

common.openconsole.props is a pretty good place to stash the XAML
version since it is included in every project (including the WAP
project (unlike the C++ build props!)).

I've gone ahead and added a "double dependency" on multiple XAML
versions. We'll toggle them with a build flag.

72e23b902 - Remove Terminal's built-in copy of the VC Runtime

This removes the trick we pulled in #5661 and saves us ~550kb per arch.

Some of our dependencies still depend on the "app" versions of the
runtime libraries, so we are going to continue shipping the forwarders
in our package. Build rules have been updated to remove the non-Desktop
VCLibs dependency to slim down our package graph.

This is not a problem on Windows 11 -- it looks like it's shipped inbox.

BREAKING CHANGE: When launched unpackaged, Terminal now requires the
vcruntime redist to be installed.

1ecb787de - release: move symbol publication into its own phase

Right now, symbol publication happens every time we produce a final
bundle. In the future, we may be producing multiple bundles from the
same pipeline run, and we need to make sure we only do one symbol
publication to MSDL.

When we do that, it will be advantageous for us to have just one phase
that source-indexes and publishes all of the symbols.

@DHowett
Copy link
Member Author

DHowett commented Feb 23, 2022

Lots of artifacts:

image

Here's the final bundles:

image

image

(that's three whole architectures!)

@github-actions
Copy link

github-actions bot commented Feb 23, 2022

@check-spelling-bot Report

Unrecognized words, please review:

  • defaultlib
  • libucrt
  • libucrtd
  • MSVCP
  • ucrt
  • ucrtd
Previously acknowledged words that are now absent azurewebsites cxcy debolden deconstructed devicefamily guardxfg LLVM MSDL ned NOWAIT pgorepro pgort PGU redistributable Timeline timelines unintense WResult xfg
To accept these unrecognized words as correct (and remove the previously acknowledged and now absent words), run the following commands

... in a clone of the git@github.com:microsoft/terminal.git repository
on the dev/duhowett/slim branch:

update_files() {
perl -e '
my @expect_files=qw('".github/actions/spelling/expect/alphabet.txt
.github/actions/spelling/expect/expect.txt
.github/actions/spelling/expect/web.txt"');
@ARGV=@expect_files;
my @stale=qw('"$patch_remove"');
my $re=join "|", @stale;
my $suffix=".".time();
my $previous="";
sub maybe_unlink { unlink($_[0]) if $_[0]; }
while (<>) {
if ($ARGV ne $old_argv) { maybe_unlink($previous); $previous="$ARGV$suffix"; rename($ARGV, $previous); open(ARGV_OUT, ">$ARGV"); select(ARGV_OUT); $old_argv = $ARGV; }
next if /^(?:$re)(?:(?:\r|\n)*$| .*)/; print;
}; maybe_unlink($previous);'
perl -e '
my $new_expect_file=".github/actions/spelling/expect/9ab4abfb5471e91235e34379bf90e24ec3a22c67.txt";
use File::Path qw(make_path);
use File::Basename qw(dirname);
make_path (dirname($new_expect_file));
open FILE, q{<}, $new_expect_file; chomp(my @words = <FILE>); close FILE;
my @add=qw('"$patch_add"');
my %items; @items{@words} = @words x (1); @items{@add} = @add x (1);
@words = sort {lc($a)."-".$a cmp lc($b)."-".$b} keys %items;
open FILE, q{>}, $new_expect_file; for my $word (@words) { print FILE "$word\n" if $word =~ /\w/; };
close FILE;
system("git", "add", $new_expect_file);
'
}

comment_json=$(mktemp)
curl -L -s -S \
  --header "Content-Type: application/json" \
  "https://github.com/gitapi/repos/microsoft/terminal/issues/comments/1049063229" > "$comment_json"
comment_body=$(mktemp)
jq -r .body < "$comment_json" > $comment_body
rm $comment_json

patch_remove=$(perl -ne 'next unless s{^</summary>(.*)</details>$}{$1}; print' < "$comment_body")
  

patch_add=$(perl -e '$/=undef;
$_=<>;
s{<details>.*}{}s;
s{^#.*}{};
s{\n##.*}{};
s{(?:^|\n)\s*\*}{}g;
s{\s+}{ }g;
print' < "$comment_body")
  
update_files
rm $comment_body
git add -u
✏️ Contributor please read this

By default the command suggestion will generate a file named based on your commit. That's generally ok as long as you add the file to your commit. Someone can reorganize it later.

⚠️ The command is written for posix shells. You can copy the contents of each perl command excluding the outer ' marks and dropping any '"/"' quotation mark pairs into a file and then run perl file.pl from the root of the repository to run the code. Alternatively, you can manually insert the items...

If the listed items are:

  • ... misspelled, then please correct them instead of using the command.
  • ... names, please add them to .github/actions/spelling/allow/names.txt.
  • ... APIs, you can add them to a file in .github/actions/spelling/allow/.
  • ... just things you're using, please add them to an appropriate file in .github/actions/spelling/expect/.
  • ... tokens you only need in one place and shouldn't generally be used, you can add an item in an appropriate file in .github/actions/spelling/patterns/.

See the README.md in each directory for more information.

🔬 You can test your commits without appending to a PR by creating a new branch with that extra change and pushing it to your fork. The check-spelling action will run in response to your push -- it doesn't require an open pull request. By using such a branch, you can limit the number of typos your peers see you make. 😉

🗜️ If you see a bunch of garbage

If it relates to a ...

well-formed pattern

See if there's a pattern that would match it.

If not, try writing one and adding it to a patterns/{file}.txt.

Patterns are Perl 5 Regular Expressions - you can test yours before committing to verify it will match your lines.

Note that patterns can't match multiline strings.

binary-ish string

Please add a file path to the excludes.txt file instead of just accepting the garbage.

File paths are Perl 5 Regular Expressions - you can test yours before committing to verify it will match your files.

^ refers to the file's path from the root of the repository, so ^README\.md$ would exclude README.md (on whichever branch you're using).

It's very difficult for users who do not have access to the store to get our dependency packages, and we want to be robust
and deployable everywhere. Since these libraries can be redistributed, it's easiest if we simply redistribute them.
See also the "VC LIBS HACK" section in WindowsTerminal.vcxproj.
Some of our dependencies still require a CRT, so we're going to ship the forwarders in our package and
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

File future work item to migrate the dependencies off?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

No future work planned - I suspect that XAML is not getting rid of its CRT dependency ;)

custom.props Outdated
Comment on lines 8 to 17
The Windows 11 build is going to have the same package name, so it *must* have a different version.
The easiest way for us to do this is to use the "year" field, which will trigger the Package ES versioning task
to add 10,000 to our minor version.
In short, for a given Terminal build 1.11, we will emit two different versions:
- 1.11.234.0 for Windows 10
- 1.11.10234.0 for Windows 11
It is my understanding that this is, like, the only version field we get control over.
If we rev the packages in the following year (which would increase the _base_ version by 10,000), they would be:
- 1.11.10456.0 for Windows 10
- 1.11.20456.0 for Windows 11
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The only other thing I could think of is somehow getting the revision ID of the two different builds to have a different number. But I think that's weird too.

Looking that up, however, I realized that you have the versioning scheme off a bit. It's YDDDR for the third field. That is Y = years since the base year. DDD = three digit day of year. R = revision from the day (that is, how many times that day you did the build.)

So it would say:

1.14.2341.0 on the 234th day of 2022, the first time that day.
1.14.2342.0 on the 234th day of 2022, the second time that day.
1.14.12341.0 on the 234th day of 2023, the first time that day.

This means that you could still theoretically have a collision if someone didn't roll this over and the major/minor didn't change fast enough thanks to playing with the year field.

You could stretch it further to make it less likely.
You could perhaps assert and fail the build if the current year isn't TerminalBaseYearForStoreVersion and tell the maintainer to go update it and ensure it won't collide with previous builds (because the major/minor has advanced since last year... HOPEFULLY.)

A super ugly option would be to use the "collapse major minor" option which turns the versioning into MMMmm.YDDDR.PPPPP.0 (where MMM = major, mm = minor, Y = years since the set, DDD = days of the year, R = revision of the day, PPPPP = patch field that you can set to whatever you want now that its moved up one) and then set the patch field and it won't be erased thanks to store versioning. 114.2341.11.0 --> Terminal 1.14, built on the 0th year, 234th day, 1st time that day, with the "11" moniker" in the patch field to distinguish and the trailing 0 that the store requires. Yuck.

Copy link
Member Author

@DHowett DHowett Feb 23, 2022

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Sorta. It's YDDDR but leading zeroes are removed -- so 245 == 00245 is Y=0 D=024 and R=5.

The collision I don't care too much about. It will only cause a problem if we build a servicing candidate on the exact same day, one year later, with the exact same revision offset.

I'd rather not throw an assert that TerminalBaseYear... is the current year, because that will break the very next time we promote a December Preview release to a January Stable release (Y+=1, but the Major/Minor don't change).

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The ideal is that the base year is updated every time the major/minor pair changes. That will keep the revision field low (between 11 and 13659, or 10011 and 23659 for a build made in the year after that major/minor version was struck.)

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I've moved the Win11 version delta into the "build number" field. Today, I built 1.14.551 and 1.14.552 from the same pipeline.

It is unlikely that we will be shipping two builds made back to back on the same day

Copy link
Member

@miniksa miniksa left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I approve, but you should at least fix the comment about the versioning scheme to be accurate even if you don't care about the collisions.

@DHowett
Copy link
Member Author

DHowett commented Feb 23, 2022

I think it IS accurate tho, right? It was built on day 23 of year delta 0, and it was the 4th build?

@DHowett DHowett added zPreview-Service-Queued-1.13 A floating label that tracks the current Preview version for servicing purposes. zStable-Service-Queued-1.12 A floating label that tracks the current Stable version for servicing purposes. labels Feb 23, 2022
@DHowett
Copy link
Member Author

DHowett commented Feb 23, 2022

Thanks to the windows inbox build, this needs to be backported to 1.12 :/

Right now, symbol publication happens every time we produce a final
bundle. In the future, we may be producing multiple bundles from the
same pipeline run, and we need to make sure we only do *one* symbol
publication to MSDL.

When we do that, it will be advantageous for us to have just one phase
that source-indexes and publishes all of the symbols.
This removes the trick we pulled in #5661 and saves us ~550kb per arch.

Some of our dependencies still depend on the "app" versions of the
runtime libraries, so we are going to continue shipping the forwarders
in our package. Build rules have been updated to remove the non-Desktop
VCLibs dependency to slim down our package graph.

This is not a problem on Windows 11 -- it looks like it's shipped inbox.

**BREAKING CHANGE**: When launched unpackaged, Terminal now requires the
vcruntime redist to be installed.
common.openconsole.props is a pretty good place to stash the XAML
version since it is included in every project (including the WAP
project (unlike the C++ build props!)).

I've gone ahead and added a "double dependency" on multiple XAML
versions. We'll toggle them with a build flag.
This required some changes in how we download artifacts to make sure
that we could control which version of Windows we were processing in any
individual step.

We're also going to patch the package manifest on the Windows 11 version
so the store targets it more specifically.

On top of the prior three commits, this lets us ship a Windows 11
package that costs only ~15MB on disk. The Windows 10 version, for
comparison, is about 40.
@DHowett
Copy link
Member Author

DHowett commented Feb 24, 2022

I've backed out the Hybrid CRT stuff. It doesn't get us anything, because we still need to depend on the CRT package to resolve our dependencies' dependencies. It caused more trouble than it was worth, too, because it required us to rewrite some C++/CX stuff in C++/WinRT and that just made me sad.

Comment on lines +177 to +180
- pwsh: |-
./build/scripts/Patch-ManifestsToWindowsVersion.ps1 -NewWindowsVersion "10.0.22000.0"
displayName: Update manifest target version to Win11 (if necessary)
condition: and(succeeded(), eq(variables['TerminalTargetWindowsVersion'], 'Win11'))
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Since most of us are working on Windows 11 right now, wouldn't it be more helpful to default to 22000 and instead set a lower version for the Windows 10 build?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'd like to let folks who are still running Windows 10 press F5 and deploy our project -- the "default" build mode should be Win10 for this reason :)

@DHowett DHowett changed the title build: ship a separate Windows 11 build of Terminal that is less than half the size build: ship a Win11 build of Terminal that's <=half the size Feb 25, 2022
@DHowett DHowett merged commit 53a454f into main Feb 25, 2022
@DHowett DHowett deleted the dev/duhowett/slim branch February 25, 2022 00:09
DHowett added a commit that referenced this pull request Feb 25, 2022
Four (4) squashed changes, with messages preserved.

Right now, symbol publication happens every time we produce a final
bundle. In the future, we may be producing multiple bundles from the
same pipeline run, and we need to make sure we only do *one* symbol
publication to MSDL.

When we do that, it will be advantageous for us to have just one phase
that source-indexes and publishes all of the symbols.

This removes the trick we pulled in #5661 and saves us ~550kb per arch.

Some of our dependencies still depend on the "app" versions of the
runtime libraries, so we are going to continue shipping the forwarders
in our package. Build rules have been updated to remove the non-Desktop
VCLibs dependency to slim down our package graph.

This is not a problem on Windows 11 -- it looks like it's shipped inbox.

**BREAKING CHANGE**: When launched unpackaged, Terminal now requires the
vcruntime redist to be installed.

common.openconsole.props is a pretty good place to stash the XAML
version since it is included in every project (including the WAP
project (unlike the C++ build props!)).

I've gone ahead and added a "double dependency" on multiple XAML
versions. We'll toggle them with a build flag.

This required some changes in how we download artifacts to make sure
that we could control which version of Windows we were processing in any
individual step.

We're also going to patch the package manifest on the Windows 11 version
so the store targets it more specifically.

On top of the prior three steps, this lets us ship a Windows 11
package that costs only ~15MB on disk. The Windows 10 version, for
comparison, is about 40.

(cherry picked from commit 53a454f)
Signed-off-by: Dustin Howett <duhowett@microsoft.com>
DHowett added a commit that referenced this pull request Feb 25, 2022
Four (4) squashed changes, with messages preserved.

Right now, symbol publication happens every time we produce a final
bundle. In the future, we may be producing multiple bundles from the
same pipeline run, and we need to make sure we only do *one* symbol
publication to MSDL.

When we do that, it will be advantageous for us to have just one phase
that source-indexes and publishes all of the symbols.

This removes the trick we pulled in #5661 and saves us ~550kb per arch.

Some of our dependencies still depend on the "app" versions of the
runtime libraries, so we are going to continue shipping the forwarders
in our package. Build rules have been updated to remove the non-Desktop
VCLibs dependency to slim down our package graph.

This is not a problem on Windows 11 -- it looks like it's shipped inbox.

**BREAKING CHANGE**: When launched unpackaged, Terminal now requires the
vcruntime redist to be installed.

common.openconsole.props is a pretty good place to stash the XAML
version since it is included in every project (including the WAP
project (unlike the C++ build props!)).

I've gone ahead and added a "double dependency" on multiple XAML
versions. We'll toggle them with a build flag.

This required some changes in how we download artifacts to make sure
that we could control which version of Windows we were processing in any
individual step.

We're also going to patch the package manifest on the Windows 11 version
so the store targets it more specifically.

On top of the prior three steps, this lets us ship a Windows 11
package that costs only ~15MB on disk. The Windows 10 version, for
comparison, is about 40.

(cherry picked from commit 53a454f)
@DHowett DHowett removed zStable-Service-Queued-1.12 A floating label that tracks the current Stable version for servicing purposes. zPreview-Service-Queued-1.13 A floating label that tracks the current Preview version for servicing purposes. labels Feb 25, 2022
@miniksa miniksa added zStable-Service-Queued-1.12 A floating label that tracks the current Stable version for servicing purposes. and removed zStable-Service-Queued-1.12 A floating label that tracks the current Stable version for servicing purposes. labels Mar 1, 2022
zadjii-msft pushed a commit that referenced this pull request Mar 24, 2023
We ship a separate package to Windows 10, which contains a copy of XAML
embedded in it, because of a bug in activating classes from framework
packages while we're elevated.

We did this to avoid wasting disk space on Windows 11 installs (which is
critical given that we're preinstalled in the Windows image.)

The fix for this issue was released in a servicing update in April 2022.
Thanks to KB5011831, we no longer need this workaround!

And finally, this means that we no longer need to depend on a copy of
"pre-release" XAML. We only did that because it would copy all of its
assets into our package.

Introduced in #12560
Closes #14106
Closes (discussion) #14981
Reverts #14660
DHowett added a commit that referenced this pull request Mar 31, 2023
We ship a separate package to Windows 10, which contains a copy of XAML
embedded in it, because of a bug in activating classes from framework
packages while we're elevated.

We did this to avoid wasting disk space on Windows 11 installs (which is
critical given that we're preinstalled in the Windows image.)

The fix for this issue was released in a servicing update in April 2022.
Thanks to KB5011831, we no longer need this workaround!

And finally, this means that we no longer need to depend on a copy of
"pre-release" XAML. We only did that because it would copy all of its
assets into our package.

Introduced in #12560
Closes #14106
Closes (discussion) #14981
Reverts #14660

(cherry picked from commit f5e9e8e)
Service-Card-Id: 88600517
Service-Version: 1.17
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants