GitHub - microsoft/LLMLingua: To speed up LLMs' inference and enhance LLM's perceive of key information, compress the prompt and KV-Cache, which achieves up to 20x compression with minimal performance loss.

To speed up LLMs' inference and enhance LLM's perceive of key information, compress the prompt and KV-Cache, which achieves up to 20x compression with minimal performance loss. - GitHub - microsoft/LLMLingua: To speed up LLMs' inference and enhance LLM's perceive of key information, compress the prompt and KV-Cache, which achieves up to 20x compression with minimal performance loss.

Related Keywords

Llmlingua Longllmlingua ,Yuqing Yang ,Xufang Luo ,Qianhui Wu ,Lili Qiu ,Huiqiang Jiang ,Dongsheng Li ,Microsoft ,Association For Computational Linguistics ,Compressing Prompts ,Accelerated Inference ,Large Language Models ,Chin Yew Lin ,Long Context Scenarios ,Prompt Compression ,Under Review ,Large Language ,Empirical Methods ,Natural Language ,Online Meeting ,Contributor License Agreement ,Microsoft Open Source Code ,