Correlation Analyzer

For most traders, checking correlations weekly is sufficient. Correlations change gradually in normal markets. However, during market stress or major events, check daily. Set up alerts for significant changes so you don't need to check constantly. The Correlation Analyzer can monitor continuously and notify you when something important changes.

A diversification score (average pairwise correlation) below 0.5 is generally good, below 0.3 is excellent. Above 0.7 indicates poor diversification. Note that achieving very low scores (below 0.2) may require including uncorrelated assets like gold or international markets. For an equity-only portfolio, 0.4-0.5 is realistic and acceptable.

Generally no - stocks in the same sector typically have high correlation (0.7-0.9) because they're affected by similar factors. For diversification, you're better off spreading across different sectors or asset classes. However, within a sector, some stocks may have lower correlation due to different business models (e.g. a UK-focused retail bank versus an internationally-focused bank like HSBC or Standard Chartered).

Not always. Gold's correlation with stocks varies and can even be positive in some periods. On average, gold has slight negative correlation with equities, especially during market stress. However, in the initial phase of a crash (like March 2020), gold may fall along with everything else before recovering. Gold is a good diversifier over longer periods but not a guaranteed hedge for every down day.

A single correlation number is just the average over your lookback period. Rolling correlation shows how this relationship changes over time. This is crucial because: 1) You can see if the relationship is stable or changing, 2) You can identify when correlations spike (market stress), 3) You can spot correlation breakdown (relationship fundamentally changing). A stable 0.6 correlation is very different from one that swings between 0.3 and 0.9.

Use Pearson as your default - it's most commonly used and interpretable. Switch to Spearman when: 1) Your data has significant outliers that might skew results, 2) You suspect non-linear (but monotonic) relationships, 3) You're comparing very different asset types. A good practice is to calculate both - if they're similar, you can trust Pearson. If Spearman is significantly lower, outliers are inflating Pearson and you should investigate.

Balance between responsiveness and stability: 20-30 days is responsive but noisy, 60 days is balanced, 120+ days is stable but slow. For regime detection, 60 days works well. For pairs trading, 20-30 days catches changes faster. For long-term portfolio decisions, 90-120 days is more appropriate. You can also use multiple windows - short for signals, long for context.

Be cautious when: 1) Z-score exceeds 3.0 (extreme divergence, relationship may be broken), 2) Rolling correlation drops below 0.6 (relationship weakening), 3) Fundamental news explains the divergence (may not mean-revert), 4) Cointegration test no longer passes, 5) Spread has been trending one direction for extended period. Always use stops and don't assume every divergence will converge.

Institutional flows can be included as additional time series. Note the UK has no single daily aggregate flow feed - infer flows from RNS major-holding (TR-1) notifications, FCA short-position data and Investment Association monthly fund-flow data. Overseas flows typically have ~0.4-0.6 correlation with the FTSE 100 and may lead by a few days; use lead-lag analysis to find the optimal lag. Flows can also act as a regime indicator - overseas and domestic institutions both buying is bullish, both selling is bearish, opposite flows indicate rotation.

Correlation breakdown is specific to one pair - a previously stable relationship (say A-B correlation of 0.8) suddenly changes (drops to 0.4). This might be due to company-specific events. Regime change is market-wide - all correlations shift together (e.g., everything becomes more correlated in a crisis). Breakdown affects your specific pairs trades, regime change affects your overall portfolio diversification assumption.

Options include: 1) Use correlation with sector proxy to estimate, 2) Apply Bayesian prior based on similar assets, 3) Use shrinkage more aggressively (shrink toward assumed correlation), 4) Start with conservative assumption (higher correlation than you expect), 5) Use intraday data for faster estimation if available. As more data accumulates, gradually reduce prior weight. Never use less than 60-90 data points for reliable correlation estimate.

Validation approaches: 1) Backtest - did regime calls predict subsequent market behavior? 2) Compare to known events - did model detect March 2020, 2008, etc.? 3) Out-of-sample testing - train on earlier period, test on later, 4) Compare multiple detection methods - do they agree? 5) Check false positive rate - how often does it call crisis without follow-through? A good model should have high hit rate on true regime changes and low false alarms.

T-copula models symmetric tail dependence (same probability of joint extreme up and down moves) and is a good default for equity portfolios. Clayton copula has only lower tail dependence (crash together but not boom together), which matches empirical evidence for stocks. Use Clayton when specifically concerned about downside risk. You can also fit both and compare fit statistics. For most equity applications, t-copula with low degrees of freedom (4-8) captures the key tail risk.

Approach: 1) Calculate correlations between your stocks and known factors (market, value, momentum, quality, etc.), 2) Group stocks by factor exposure, 3) Track rolling factor correlations to detect factor rotation, 4) Monitor when factor correlations spike (factor crash risk), 5) Build factor-neutral portfolios by balancing factor exposures. Principal component analysis (PCA) on correlation matrix can reveal hidden factors driving your portfolio.

Architecture: 1) Streaming price feed for all monitored assets, 2) Calculate returns on chosen frequency (5-min, 15-min), 3) Use exponentially weighted correlation for responsiveness, 4) Short halflife (2-4 hours) to capture intraday changes, 5) Compare to daily correlation as anchor, 6) Alert on significant intraday deviation from daily pattern. Considerations: Intraday correlations are noisier, don't overreact to short-term spikes. Use for confirmation of other signals rather than primary trigger.