Intel's OpenVINO 2023.1 was just published to GitHub as the newest version of this open-source toolkit for optimizing and deploying AI workloads across their CPUs, GPUs, and now also having official support for the new VPU being found with Meteor Lake SoCs.
With the rapid development of artificial intelligence, various fault diagnosis methods based on the deep neural networks have made great advances in mechanical system safety monitoring. To get the high accuracy for the fault diagnosis, researchers tend to adopt the deep network layers and amount of neurons or kernels in each layer. This results in a large redundancy and the structure uncertainty of the fault diagnosis networks. Moreover, it is hard to deploy these networks on the embedded platforms because of the large scales of the network parameters. This brings huge challenges to the practical application of the intelligent diagnosis algorithms. To solve the above problems, an iterative automatic machine compression method, named Iterative-AMC, is proposed in this paper. The proposed method aims to automatically compress and optimize the structure of the large-scale neural networks. Experiments are carried out based on two test benches. With the proposed Iterative-AMC method, the pr
Large transformer models are mainstream nowadays, creating SoTA results for a variety of tasks. They are powerful but very expensive to train and use. The extremely high inference cost, in both time and memory, is a big bottleneck for adopting a powerful transformer for solving real-world tasks at scale.
Why is it hard to run inference for large transformer models? Besides the increasing size of SoTA models, there are two main factors contributing to the inference challenge (Pope et al.