Most of indicators only use past candles when being computed. In example, in order to compute an SMA(20) at a candle X we'll use use 20 candles, from X-19 to X. That's a natural kind of behavior for a lot of indicators. However, some indicators differ from that in a big way. In order to compute a value for a candle X, these special indicators will also use candles "from the future", after X. I.e., X+1 or X+2. There's really no good name for that, but that is one example of a broader universe of "repainting indicators".
Figure 1: when computing SMA(20) at candle X, it only uses candles which are "in the past" from X (all the candles in the orange box).
Why some indicators repaint
Indicators like that are not new. ZigZag or Bill Williams Fractal are two good old examples. The Fractal indicator marks local extremum points which dominate over 2 candles to the left from them and to the right from them. When you talk real time trading, "to the right" from the current candle means "in the future". The Fractal indicator can never generate a point sooner than after 2 candles since the extremum point (i.e., Highest High) originally emerges. Which means that in real time, Fractal points always emerge "in the past" (2 candles ago for a standard version of Fractals).
Figure 2: in order to identify a Fractal point at X, this indicator also uses candles X+1 and X+2. When trading in real time, these do not exist yet when X is "the last candle".
By design, some indicators need data "from the future" in order to be computed. It's very important to be aware whether the indicator you're using is doing that when using it. If it is, that does not render it useless, but definitely demands some special approaches.
Why repainting indicators can be dangerous
While these indicators are good legitimate tools for some scenarios, using them in a way which ignores their nature is dangerous, I.e., it's easy to backtest a strategy which buys at Williams Fractal Low and then sells 2 candles after. Guess what happens. By the very definition, Fractal Low is the lowest point surrounded by 2 highest points at both sides. So a strategy like that will be very much profitable. The caveat here is that when trading in real time, you can never buy at a Fractal point because Fractal indicator can never give a signal on the "current candle". It always gives signals which 2 candles deep in the past.
On TrendSpider, we mark these indicators with a special flag, so that when you use them anywhere (Scanner, Alerts, Strategy Tester, Smart Checklist etc) you would be at least aware of that aspect. In general, we strongly advise one to learn about the indicators they are going to trade off, for as much as possible.
How to backtest repainting indicators
If you want to use an indicator like that in a strategy, then as a rule of a thumb, you should never use the "last value" of this indicator in your conditions. Any time you use then indicator's value, you should refer to "X candles ago" (along with the counterpart of a condition, if applicable). "X candles ago" should be equal to the amount of candles after which the indicator's value has stabilzied and won't change. In example, for classic Williams Fractal, X would be 3.
Repainting indicators in Trading Bots
Trading bots have the Strategy Tester at their heart. Any time bots check their conditions (which happens immediately once yet another candle gets finalized), they actually run a full backtest under the hood. In order to define whether it's time to generate a signal, bots need to know whether a signal exists, and also what signals they have already generated in the past. Bots find that out by comparing signals from the previous backtest against the current backtests.
In case if our system identifies that some signals have moved (if compared to the previous backtest), it stops a bot. The main reason is that a bot like that simpyl can't be trusted any more. Its behavior does not match what you've seen in the Strategy Tester prior to starting a bot. Signals moving can lead to all kinds of weird cases, if left intact (like, 2 Entry signals in a row, or 2 Exit signals in a row), so we react.
Signals can move for 2 reasons:
- Underlying market data has changed
- Indicators have repainted (either because of future leak, or for any other reason).
While small changes in underlying data are to be expected for stocks and ETFs (in example, exchange corrections), the vast majority of sanity check stops are caused by indicators repainting. Here are a few examples of strategies which would definitely cause a bot to be stopped.
Example 1: Williams Fractal
Imagine you enter long any time a new Williams Fractal Low emerges, and then you exit long 2 candles after. This strategy, while guaranteed to backtest really well, is absolutely not viable. The reason for that is in how this indicator works at all. Williams Fractal is forward-looking; in order for a point to become a Fractal Low, it needs 2 candles before it to have higher Lows, and also 2 candles after it to have higher Lows. This indicator takes "future data" into consideration when computing its points. That's a future leak right here.
Let's assume you start a bot on a strategy like that, and also assume that price action forms a typical Fractal Low patter. Here's what's going to happen
- Candle 1: no signal
- Candle 2: no signal
- Candle 3: no signal (this candle will become a Fractal Low, but it can't be Low until 2 more candles are available)
- Candle 4: no signal
- Candle 5: no signal.
Once candle 5 closes, candle 3 becomes a Fractal Low. When comparing backtest results to the previous ones, a bot realizes that a new signal has emerged in the past, and it's now too late to react to it, as time has passed. Generating a signal after candle 5 would be nonsense, as it does not match intent of the strategy. This bot will be stopped by the system because of a new signal emerging in the past.
Transforming it into "Enter when there's a Fractal Low 2 candles ago" will make it a perfectly viable strategy, because your signals will not need any "future data" any more.
Example 2: Wrong multi-time-frame strategy
Imagine you have a custom indicator, which paints current Week's High over the candles. Like that
const weekly = await request.history(current.ticker, 'W');
const highLanded = land_points_onto_series(weekly.time, weekly.high, time);
paint(interpolate_sparse_series(highLanded, 'constant'), { name: "WeeklyHigh" });
Imagine you build a stategy like "sell once my 15 min candle High equals MyWeeklyIndicator.WeeklyHigh" and create a bot on it. Imagine that this bot runs on a stock, it's first business day of a week and your 15 min candles go like that:
- 09:30, High = 10
- 09:45, High = 9.5
- 10:00, High = 10.5
Your bot would do the following:
- 09:30: entry. this is the only candle of the current week, and obviously its High will be equal to the weekly High
- 09:45: no signal
- 10:00: conflict, bot stopped
What happens at 10:00 is that an entry signal from the past (from 09:30) has disappeared, because the 09:30 candle High is not equal to Weekly High any more. That is because this candle at 10:00 has higher High, which changes the current weekly High. The indicator above spreads one number (current weekly high) over the entirety of the current week's intraday candle, representing yet another case of a major future leak.
In order to get this strategy working, you have 2 options: either (a) stop landing current higher period data point over the candles, or (b) rewrite the code to reconstruct "how the higher period candle looked like back then" for each intraday candle.
(b) is fairly complex. It's also hard to validate by eyeballing a chart. (a) is more likely to be your choice; the idea is that since "current Weekly High" is inherently prone to change over the course of a week, you shoudl try using something what does NOT change. In example, you are safe to use current week's Open. Or previous week's High. Here's the code (compare to the one above!) which uses previous week High instead of the current week High.
const weekly = await request.history(current.ticker, 'W');
// shifting data points backwards, by 1
const highLanded = land_points_onto_series(weekly.time, shift(weekly.high, 1), time);
paint(interpolate_sparse_series(highLanded, 'constant'));