In this blog we discuss how the transformer architecture naturally extends over external memories, and share empirical results which leverage this capability. These methods are innate (don’t require fine tuning) and outperform popular retrieval augmented generation methods.