Add Trend Quality Indicator

[kbs-code]

Trend Quality was created by David Sepiashvili and was unveiled in an issue of Stocks and Commodities magazine in 2004.
The basic idea behind it is that it is not enough to rate a market as "trending" or "not trending". We must elaborate more on the "strength" or "quality" of a trend so that we can make better trading decisions. In the original issue, the author created 2 indicators called the trend quality "q" and "b" indicator. I won't discuss the "b" indicator here because I think it is less useful and adopted less by the trading community.

The "q" indicator is quite useful because it returns values based on the quality of the trend. From the author:
Readings in the range from +2 to +5, or from -2 to -5, can indicate moderate trending, and readings above Q=+5 or below Q=-5 indicate strong trending. Strong upward trending often leads to the security’s overvaluing, and strong downward trending often results in the security’s undervaluing.

I myself have noticed tq values go as high as 20 since in FX markets can seemingly trend forever.
Since it's unveiling tq has been rewritten for a few trading platforms, most notably mt4/5, but you can also find versions for it in ThinkOrSwim and TradingView. The version you see in most trading platforms is the "q" indicator and it is directional in nature.

The goat of MT4/5 indicator development mladen did a megathread on trend quality which you can find here: https://www.mql5.com/en/forum/181010.

I tried to port trend quality to python but I haven't produced anything usable yet. Here's what I've done so far: https://github.com/kbs-code/trend_quality_python/tree/master

In that repo, the file https://github.com/kbs-code/trend_quality_python/blob/master/reference/trend_quality_q1_directional_indicator.mq4 is a good reference for how the indicator should behave in mt4.

In the mql4 code, the indicator works on a for loop as it iterates over close prices. I tried to reverse engineer this behavior and apply the calculation in a vectorized manner using pandas data frames but failed. I've considered using lambdas or itertuples and have avoided trying iterrows because it's frowned upon. I've tried masks as well but I'm afraid that might introduce look-ahead bias.

Sources:
https://www.prorealcode.com/wp-content/uploads/2019/03/24-Q-Indicator.pdf
https://github.com/kbs-code/trend_quality_python/tree/master
https://www.mql5.com/en/forum/181010
https://usethinkscript.com/threads/trend-quality-indicator-with-alert-for-thinkorswim.9/
https://www.tradingview.com/script/k3mnaGcQ-Trend-Quality-Indicator/

[kbs-code]

added "pandas_ta - like" boilerplate code for tq. Still stuck on calculating tq values. Please see https://github.com/kbs-code/trend_quality_python/blob/master/research.ipynb

[kbs-code]

added cells demonstrating chunking by macd sign
https://github.com/kbs-code/trend_quality_python/blob/master/research.ipynb

[kbs-code]

Job post for this task added on Upwork: https://www.upwork.com/jobs/~012fd863b8c28f53ab

[twopirllc]

@kbs-code

So you want to pay someone other than the maintainer? 🤷🏼‍♂️ 😕

[kbs-code]

When I posted that, I thought that you could take up the task as well. It was open to anyone. The job remains unfinished, I may tackle it again in the future. But right now I'm not hiring anyone anymore.

[chris-official]

Hi @kbs-code, I came up with the following code. However, I am not sure if my chunking works as intended since I used a different approach that reduces the number of loops to the number of chunks instead of iterating over each row. But feel free to take a look and try it out. Hope it helps!

# Trend Quality Indicator
def trend_quality(series: pd.Series, fast_period: int = 10, slow_period: int = 40, 
trend_period: int = 10, noise_period: int = 10, c: float = 2.0, noise_type: str = "squared") -> pd.DataFrame:
                   
    # calculate moving averages
    ma_fast = series.ewm(span=fast_period).mean()
    ma_slow = series.ewm(span=slow_period).mean()
    
    # calculate crossovers
    fast_shift = ma_fast.shift(1)
    slow_shift = ma_slow.shift(1)
    up_cross = (ma_fast > ma_slow) & (fast_shift < slow_shift)
    down_cross = (ma_fast < ma_slow) & (fast_shift > slow_shift)
    crosses = series[up_cross | down_cross].index.to_list()
    crosses.append(None)  # add None to include last cycle

    # calculate price change
    pc = series.diff().fillna(0)

    # calculate cumulative price change
    cpc = pd.Series(0., index=series.index)
    last_idx = None
    for idx in crosses:
        cpc.loc[last_idx:idx] = pc.loc[last_idx:idx].cumsum()
        last_idx = idx

    # calculate trend
    trend = cpc.ewm(span=trend_period).mean()

    # calculate noise
    if noise_type == "linear":
        abs_diff = (cpc - trend).abs()
        noise = abs_diff.ewm(span=noise_period).mean()
    elif noise_type == "squared":
        square_diff = (cpc - trend) ** 2
        noise = np.sqrt(square_diff.ewm(span=noise_period).mean())
    else:
        raise ValueError("Noise type invalid.")

    # calculate q indicator
    q_indicator = trend / (noise * c)

    # calculate b indicator
    b_indicator = trend.abs() / (trend.abs() + noise) * 100

    # return indicators
    return pd.DataFrame({
        "Price Change": pc,
        "Cumulative Price Change": cpc,
        "Trend": trend,
        "Noise": noise,
        "Q_Indicator": q_indicator,
        "B_Indicator": b_indicator
    }, index=series.index)

[kbs-code]

Thank you very much @chris-official for sharing this. I did an initial test of your code which you can find at https://github.com/kbs-code/trend_quality_python/blob/master/user_submission.ipynb.

I only spent a few hours putting that together but so far, the results don't seem production-ready. I'll take a deeper look into your code when I have time as I only changed your parameters to match mladen's default ones.

[chris-official]

Good news @kbs-code, I again looked into my code as well as the test notebook you provided. While my first implementation is based on the original paper, I implemented the indicator a second time but this time focused on recreating the MetaTrader code of your file "trend_quality_q1_directional_indicator.mq4". Now, the results look much more similar to what is shown in your reference image.

Here are some of the most noticeable differences of the new implementation:

• Now, the "absolute price change" is used instead of the "price change". This is unique to mladen's implementation and is not mentioned in the original paper by Sepiashvili.
• The Trend is now also reset to zero at trend changes and not just an EMA of the CPC. This behavior is also not mentioned in the original paper.
• The calculation for trend changes and CPC are different now.
• Now, a SMA is used for smoothing of the noise instead of an EMA. Paper does not define a specific MA here.

Some important considerations:

• When testing the new indicator the same way you did with my first one you may still notice a deviation from the original values. This is due to your test dataset only having 92 values while the noise period is 250 and therefore still in it's warm-up period. You need at least 250 previous values to accurately compare the magnitude of the Q Indicator. However, the shape should already be pretty similar.
• The new implementation has the advantage of being more readable since it produces clear, sharp drop-offs at the trend changes.
• However, note that my first implementation produces much less lag due to the following two factors:

1. Using the EMA for noise smoothing instead of the SMA.
2. Using the absolute price change instead of the price change. This results in the Q Indicator still rising until the trend sign flips even if the trend already reversed earlier. My first implementation will already move downwards as soon as the price does and therefore the peaks of my first Q Indicator usually produce better signals at an earlier point but are not always characterized by a sharp drop-off and thus harder to spot.

Feel free to try out the new code and make changes as you like. Maybe you can come up with your own improved indicator version! I hope this helps!

def trend_quality(price: pd.Series, fast_period: int = 7, slow_period: int = 15, 
                  trend_period: int = 4, noise_period: int = 250, c: float = 2.0, noise_type: str = "squared") -> pd.DataFrame:

    # calculate moving averages
    emaf = price.ewm(span=fast_period).mean()
    emas = price.ewm(span=slow_period).mean()

    # calcualte trend sign
    macd = emaf - emas
    sign = pd.Series(np.nan, index=price.index)
    sign[macd > 0] = 1
    sign[macd < 0] = -1

    # calculate price change
    change = price.diff().abs()

    # calculate cumulative price change and trend
    cpc = pd.Series(np.nan, index=price.index)
    trend = pd.Series(np.nan, index=price.index)
    for i in range(1, len(sign)):
        if sign[i] != sign[i-1]:
            cpc[i] = 0
            trend[i] = 0
        else:
            cpc[i] = sign[i] * change[i] + cpc[i-1]
            trend[i] = cpc[i] * (1 / trend_period) + trend[i-1] * (1 - (1 / trend_period))

    # calculate noise
    if noise_type == "linear":
        dt = (cpc - trend).abs()
        avgDt = dt.rolling(noise_period, min_periods=1).mean()
        noise = c * avgDt
    elif noise_type == "squared":
        dt = (cpc - trend) ** 2
        avgDt = dt.rolling(noise_period, min_periods=1).mean()
        noise = c * np.sqrt(avgDt)
    else:
        raise ValueError("Noise type invalid.")

    # calculate q indicator
    trendQ = (trend / noise)

    # return indicators
    return pd.DataFrame({
        "Price Change": change,
        "Cumulative Price Change": cpc,
        "Trend": trend,
        "Noise": noise,
        "Q_Indicator": trendQ,
    }, index=price.index).fillna(0)

[chris-official]

As you can see the peaks of the new indicator are much more pronounced but several bars later than the old indicator.

[kbs-code]

@chris-official thanks a lot for this. I wish there were some things explained more clearly in the original paper. In the paper there is a diagram which shows the Q indicator resetting abruptly if the trend reverses, and I think the macd changing signs triggers this. Although macd is not explicitly mentioned, from what I understand the calculation is still the same.

I think a long or short Q indicator is useful and illustrated in the original paper. The only thing he says about it is "Here, the reversal indicator is built in". I assume he coded his own directional indicator but didn't reveal all the code / math. And because he didn't reveal everything, mladen sought to add more code to match the illustration. I could be wrong.

I also don't know why I see a 250 noise period as default but for MetaTrader the default value is often 250. I'm not sure if mladen started that or someone else.

I think the question now is which version of the TQ should make it into pandas_ta. Other traders who use MetaTrader will be more familiar with mladen's versions, but some traders may prefer the original author's indicator.

Thanks for all the pointers and writing the code.

[kbs-code]

@twopirllc If I wanted to contribute the tq indicator to pandas_ta do I have to follow any guidelines? I have been using your macd as a template of sorts. Can I leave out talib mode? Since tq uses macd, do I have to use pandas_ta macd or can I calculate it myself?

[twopirllc]

@kbs-code

do I have to follow any guidelines?

You must checkout the development branch and style your code similarly to other indicators (or as close as you can as possible).

Can I leave out talib mode?

I will most likely put it anyhow if you leave it out and on by default. It's just an option so not a big deal.

some traders may prefer the original author's indicator.

Based on xp. This is also true.

KJ