This paper attempts to address and reconcile two different issues: the existence of multiple numerical data formats (such as int8, bfloat16, fp8, etc., often non optimal for the application and not directly compatible with one another) and the necessity to reduce their bandwidth requirements, especially in the case of power hungry and slow DRAM.
Interpreting Neural Networks through the Polytope Lens — LessWrong lesswrong.com - get the latest breaking news, showbiz & celebrity photos, sport news & rumours, viral videos and top stories from lesswrong.com Daily Mail and Mail on Sunday newspapers.