Latent Communication between Language Models: Redundancy-aware Merging

Redundancy-aware merging proposes combining two AI models by injecting only the distinct, unique features of Model A into Model B while discarding shared information. It achieves this either geometrically (by projecting out shared components to keep the orthogonal residual) or statistically (by removing linearly predictable elements), assuming both models operate in compatible activation spaces.

Mootez Saad

Project lead · Dalhousie University