Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add some retry support to Renderer::PaintFrame #2830

Merged
merged 3 commits into from
Sep 23, 2019

Conversation

DHowett-MSFT
Copy link
Contributor

If _PaintFrameForEngine returns E_PENDING, we'll give it another two
tries to get itself straight. If it continues to fail, we'll take down
the application.

We observed that the DX renderer was failing to present the swap chain
and failfast'ing when it did so; however, there are some errors from
which DXGI guidance suggests we try to recover. We'll now return
E_PENDING (and destroy our device resources) when we hit those errors.

Fixes #2265.

PR Checklist

Validation Steps Performed

I TDR'd the heck out of my machine. While Visual Studio and Outlook were busy assuming ambient temperature and taking a dirt nap respectively, Terminal continued chugging along.

If _PaintFrameForEngine returns E_PENDING, we'll give it another two
tries to get itself straight. If it continues to fail, we'll take down
the application.

We observed that the DX renderer was failing to present the swap chain
and failfast'ing when it did so; however, there are some errors from
which DXGI guidance suggests we try to recover. We'll now return
E_PENDING (and destroy our device resources) when we hit those errors.

Fixes #2265.
Copy link
Member

@miniksa miniksa left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks!

src/renderer/base/renderer.cpp Outdated Show resolved Hide resolved
src/renderer/base/renderer.cpp Outdated Show resolved Hide resolved
@DHowett-MSFT DHowett-MSFT merged commit 277acc3 into master Sep 23, 2019
@DHowett-MSFT DHowett-MSFT deleted the dev/duhowett/e_pending branch September 23, 2019 22:06
@ghost
Copy link

ghost commented Sep 24, 2019

🎉Windows Terminal Preview v0.5.2661.0 has been released which incorporates this pull request.:tada:

Handy links:

DHowett-MSFT pushed a commit that referenced this pull request Sep 26, 2019
If _PaintFrameForEngine returns E_PENDING, we'll give it another two
tries to get itself straight. If it continues to fail, we'll take down
the application.

We observed that the DX renderer was failing to present the swap chain
and failfast'ing when it did so; however, there are some errors from
which DXGI guidance suggests we try to recover. We'll now return
E_PENDING (and destroy our device resources) when we hit those errors.

Fixes #2265.

(cherry picked from commit 277acc3)
DHowett-MSFT pushed a commit that referenced this pull request Sep 26, 2019
- Remember last-used string in the Find dialog in conhost (GH-2845)

  (cherry picked from commit bfb1484)

- Bugfix: TextBuffer Line Wrapping from VTRenderer (GH-2797)

  Change VT renderer to do erase line instead of a ton of erase chars

  (cherry picked from commit 8afc5b2)

- Add some retry support to Renderer::PaintFrame (GH-2830)

  If _PaintFrameForEngine returns E_PENDING, we'll give it another two
  tries to get itself straight. If it continues to fail, we'll take down
  the application.

  We observed that the DX renderer was failing to present the swap chain
  and failfast'ing when it did so; however, there are some errors from
  which DXGI guidance suggests we try to recover. We'll now return
  E_PENDING (and destroy our device resources) when we hit those errors.

  Fixes GH-2265.

  (cherry picked from commit 277acc3)

- Enable VT processing by default for ConPTY (GH-2824)

  This change enables VT processing by default for _all_ conpty clients. See
  GH-1965 for a discussion on why we believe this is a righteous change.

  Also mentioned in the issue was the idea of only checking the
  `VirtualTerminalLevel` reg key in the conpty startup. I don't think this would
  be a more difficult change, looks like all we'd need is a simple
  `reg.LoadGlobalsFromRegistry();` call instead of this change.

  **Validation Steps Performed**
  Manually launched a scratch app in both the terminal and the console. The
  console launch's output mode was 0x3, and the terminal's was 0x7. 0x4 is the `
  ENABLE_VIRTUAL_TERMINAL_PROCESSING` flag, which the client now had by default
  in the Terminal.

  Closes GH-1965

  (cherry picked from commit 1c412d4)

- Remove unwanted DECSTBM clipping (GH-2764)

  The `DECSTBM` margins are meant to define the range of lines within which
  certain vertical scrolling operations take place. However, we were applying
  these margin restrictions in the `ScrollRegion` function, which is also used in
  a number of places that shouldn't be affected by `DECSTBM`.

  This includes the `ICH` and `DCH` escape sequences (which are only affected by
  the horizontal margins, which we don't yet support), the
  `ScrollConsoleScreenBuffer` API (which is public Console API, not meant to be
  affected by the VT terminal emulation), and the `CSI 3 J` erase scrollback
  extension (which isn't really scrolling as such, but uses the `ScrollRegion`
  function to manipulate the scrollback buffer).

  This commit moves the margin clipping out of the `ScrollRegion` function, so it
  can be applied exclusively in the places that need it.

  While testing, I also noticed that one of the `ScrollRegion` calls in the
  `AdjustCursorPosition` function was not setting the horizontal range correctly
  - the scrolling should always affect the full buffer width rather than just the
    viewport width - so ...

Related work items: #23507749, #23563809, #23563819, #23563837, #23563841, #23563844
DHowett-MSFT pushed a commit that referenced this pull request Apr 14, 2020
This commit introduces a backoff (150ms * number of tries) to the
renderer's retry logic (introduced in #2830). It also changes the
FAIL_FAST to a less globally-harmful render thread disable, so that we
stop blowing up any application hosting a terminal when the graphics
driver goes away.

In addition, it adds a callback that a Renderer consumer can use to
determine when the renderer _has_ failed, and a public method to kick it
back into life.

Fixes #5340.
ghost pushed a commit that referenced this pull request Apr 14, 2020
… UI (#5353)

## Summary of the Pull Request
Renderer: Add support for backoff and auto-disable on failed retry

This commit introduces a backoff (150ms * number of tries) to the
renderer's retry logic (introduced in #2830). It also changes the
FAIL_FAST to a less globally-harmful render thread disable, so that we
stop blowing up any application hosting a terminal when the graphics
driver goes away.

In addition, it adds a callback that a Renderer consumer can use to
determine when the renderer _has_ failed, and a public method to kick it
back into life.

Fixes #5340.

This PR also wires up TermControl so that it shows some UI when the renderer tastes clay.

![image](https://user-images.githubusercontent.com/14316954/79266118-f073f680-7e4b-11ea-8b96-5588a13aff3b.png)

![image](https://user-images.githubusercontent.com/14316954/79266125-f36ee700-7e4b-11ea-9314-4280e9149461.png)

## PR Checklist
* [x] Closes #5340
* [x] cla
* [ ] Tests added/passed
* [ ] Requires documentation to be updated
* [x] I've discussed this with core contributors already.

## Validation Steps Performed

I tested this by dropping the number of retries to 1 and forcing a TDR while doing `wsl cmatrix -u0`. It picked up exactly where it left off.

As a bonus, you can actually still type into the terminal when it's graphically suspended (and `exit` still works.). The block is _entirely graphical_.
DHowett-MSFT pushed a commit that referenced this pull request Apr 14, 2020
… UI (#5353)

Renderer: Add support for backoff and auto-disable on failed retry

This commit introduces a backoff (150ms * number of tries) to the
renderer's retry logic (introduced in #2830). It also changes the
FAIL_FAST to a less globally-harmful render thread disable, so that we
stop blowing up any application hosting a terminal when the graphics
driver goes away.

In addition, it adds a callback that a Renderer consumer can use to
determine when the renderer _has_ failed, and a public method to kick it
back into life.

Fixes #5340.

This PR also wires up TermControl so that it shows some UI when the renderer tastes clay.

![image](https://user-images.githubusercontent.com/14316954/79266118-f073f680-7e4b-11ea-8b96-5588a13aff3b.png)

![image](https://user-images.githubusercontent.com/14316954/79266125-f36ee700-7e4b-11ea-9314-4280e9149461.png)

* [x] Closes #5340
* [x] cla
* [ ] Tests added/passed
* [ ] Requires documentation to be updated
* [x] I've discussed this with core contributors already.

I tested this by dropping the number of retries to 1 and forcing a TDR while doing `wsl cmatrix -u0`. It picked up exactly where it left off.

As a bonus, you can actually still type into the terminal when it's graphically suspended (and `exit` still works.). The block is _entirely graphical_.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Terminal crashes when graphics cards timeout and graphics driver does soft reset
4 participants