Efficiently handling long contexts in transformer-based language models with low perplexity is an active area of research. Numerous recent approaches like Linformer. Longformer. Performer. https://leoners.shop/product-category/counter-top-product/