Shape Layer example not working for layer='between' and yref='y domain'

[subsurfaceiodev]

Hi! Since latest plotly version 5.21 a great addition has been considered by @my-tien pull #6927 being capable of specifying shape layer as 'between' which should solve issues such as presented in plotly/plotly.js#4106.

Example shown in https://plotly.com/python/shapes/ section Shape Layer works fine as it is, but it is not producing expected results when specifying yref='y domain' for the layer='between' subplot. As a mre, the following code:

import plotly.express as px

df = px.data.stocks(indexed=True)

fig = px.line(df)

fig.add_shape(
    type="rect",
    x0="2018-03-01",
    y0=0,
    x1="2018-08-01",
    y1=3,
    line_width=0,
    layer="above",
    label=dict(text="Above", textposition="top center", font=dict(size=15)),
    fillcolor="LightGreen",
    opacity=0.80,
)

fig.add_shape(
    type="rect",
    x0="2018-10-01",
    y0=0,
    x1="2019-03-01",
    y1=3,
    yref='y domain',  # consider y domain
    line_width=0,
    layer="between",
    label=dict(text="Between", textposition="top center", font=dict(size=15)),
    fillcolor="LightGreen",
    opacity=0.80,
)

fig.add_shape(
    type="rect",
    x0="2019-05-01",
    y0=0,
    x1="2019-10-01",
    y1=3,
    line_width=0,
    layer="below",
    label=dict(text="Below", textposition="top center", font=dict(size=15)),
    fillcolor="LightGreen",
    opacity=0.80,
)

fig.show()

Produces this unexpected figure:

[Coding-with-Adam]

hi @archmoj
Does this look like a bug to you?
Codepen.

[empet]

When yref='y domain' in the case of a simple plot (not a subplot fig),
y0 and y1 should take values in [0, 1] for the shapes drawn on the plot rectangle.
Hence y0=0, y1=3 has no sense in your example.
In a simple plot (as in your example) yref='y domain' is equivalent to yref ='paper',
and with adequate y0, y1 it works:

import plotly.express as px

df = px.data.stocks(indexed=True)

fig = px.line(df)
fig.add_shape(
    type="rect",
    x0="2018-03-01",
    y0=0.2,
    x1="2018-08-01",
    y1=0.9,
    yref="paper", 
    xref="x",
    line_width=0,
    layer="above",
    label=dict(text="Above", textposition="top center", font=dict(size=15)),
    fillcolor="LightGreen",
    opacity=0.80,
)


fig.add_shape(
    type="rect",
    x0="2018-10-01",
    y0=0.3,
    x1="2019-03-01",
    y1=0.85,
    yref='paper', 
    xref="x",
    line_width=0,
    layer="between",
    label=dict(text="Between", textposition="top center", font=dict(size=15)),
    fillcolor="LightGreen",
    opacity=0.80,
)

fig.add_shape(
    type="rect",
    x0="2019-05-01",
    y0=-0.1,
    x1="2019-10-01",
    y1=0.7,
    yref="y domain",
    xref="x",
    line_width=0,
    layer="below",
    label=dict(text="Below", textposition="top center", font=dict(size=15)),
    fillcolor="LightGreen",
    opacity=0.80,
)
fig.update_layout(width=650, height=450)

When the fig consists in subplots, yref='yk domain', k=2,... it doesn't work.

[subsurfaceiodev]

Using y1 = 1 instead of y1 = 3 for "Between" rectangle as @empet suggested did not solve this issue.

import plotly.express as px

df = px.data.stocks(indexed=True)

fig = px.line(df)

fig.add_shape(
    type="rect",
    x0="2018-03-01",
    y0=0,
    x1="2018-08-01",
    y1=3,
    line_width=0,
    layer="above",
    label=dict(text="Above", textposition="top center", font=dict(size=15)),
    fillcolor="LightGreen",
    opacity=0.80,
)

fig.add_shape(
    type="rect",
    x0="2018-10-01",
    y0=0,
    x1="2019-03-01",
    y1=1,  # consider correct y upper value
    yref='y domain',  # consider y domain
    line_width=0,
    layer="between",
    label=dict(text="Between", textposition="top center", font=dict(size=15)),
    fillcolor="LightGreen",
    opacity=0.80,
)

fig.add_shape(
    type="rect",
    x0="2019-05-01",
    y0=0,
    x1="2019-10-01",
    y1=3,
    line_width=0,
    layer="below",
    label=dict(text="Below", textposition="top center", font=dict(size=15)),
    fillcolor="LightGreen",
    opacity=0.80,
)

fig.show()

[subsurfaceiodev]

This is also affecting functions such as fig.add_vrect that call indirectly fig.add_shape. See following code and resulting incorrect figure:

import plotly.express as px

fig = px.line(
    x=[0, 1, 2, 3, 4],
    y=[0, 1, 4, 9, 16],
)
fig.update_traces(mode='lines+markers')
fig.add_vrect(
    x0=0,
    x1=1,
    fillcolor='LightGreen',
    layer='above',
    label=dict(text='Above'),
)
fig.add_vrect(
    x0=2,
    x1=3,
    fillcolor='LightGreen',
    layer='between',
    label=dict(text='Between'),
)
fig.add_vrect(
    x0=4,
    x1=5,
    fillcolor='LightGreen',
    layer='below',
    label=dict(text='Below'),
)
fig.update_traces(marker=dict(size=15, ))
fig.show()

[empet]

I think that yref='y domain' is not implemented for layer="below", and "between", because these two cases have sense only for yref="y", since only on the plot rectangle there exist gridlines to give the shape position with respect to them. It is even unusual that yref="paper" worked for y0=-0.1 in my example, with layer="below".

[subsurfaceiodev]

They DO have sense for this cases, see expected image below:

[empet]

fig.add_vrect has as a default parameter value, yref="y domain". That's why it doesn't work because it seems that y domain isn't implemented for the new layers.
Inspecting fig.layout for your last example:

fig.layout

we get the definition for each vrect, but with a different x0, x1, label, and layer, like this one:

{'fillcolor': 'LightGreen',
                'label': {'text': 'Between'},
                'layer': 'between',
                'type': 'rect',
                'x0': 2,
                'x1': 3,
                'xref': 'x',
                'y0': 0,
                'y1': 1,
                'yref': 'y domain'}

The last three parameters have default values, for this special shape.

[Paul-Particle]

It seems to me this is not (only) related to yref? (this is v5.22.0)

fig = go.Figure()
fig.add_shape(layer='above',   fillcolor='#EF553B', opacity=0.7, x0=0, x1=0.8, y0=0, y1=0.7, label_text='above')
fig.add_shape(layer='between', fillcolor='#EF553B', opacity=0.7, x0=1, x1=1.8, y0=0, y1=0.7, label_text='"between"')
fig.add_shape(layer='below',   fillcolor='#EF553B', opacity=0.7, x0=2, x1=2.8, y0=0, y1=0.7, label_text='below')
fig.add_trace(go.Scatter(x=[-0.5,3.5], y=[0.5,0.5], line_width=10))
print(fig)
fig.show()

Figure({
    'data': [{'line': {'width': 10}, 'type': 'scatter', 'x': [-0.5, 3.5], 'y': [0.5, 0.5]}],
    'layout': {'shapes': [{'fillcolor': '#EF553B',
                           'label': {'text': 'above'},
                           'layer': 'above',
                           'opacity': 0.7,
                           'x0': 0,
                           'x1': 0.8,
                           'y0': 0,
                           'y1': 0.7},
                          {'fillcolor': '#EF553B',
                           'label': {'text': '"between"'},
                           'layer': 'between',
                           'opacity': 0.7,
                           'x0': 1,
                           'x1': 1.8,
                           'y0': 0,
                           'y1': 0.7},
                          {'fillcolor': '#EF553B',
                           'label': {'text': 'below'},
                           'layer': 'below',
                           'opacity': 0.7,
                           'x0': 2,
                           'x1': 2.8,
                           'y0': 0,
                           'y1': 0.7}],
               'template': '...'}
})