Tao Zhang

Deep neural networks (DNNs) have recently achieved great success in many visual recognition tasks. However, existing deep neural network models are computationally expensive and memory intensive, hindering their deployment in devices with low memory resources or in applications with strict latency requirements. Therefore, a natural thought is to perform model compression and acceleration in deep networks without significantly decreasing the model performance. During the past five years, tremendous progress has been made in this area. In this paper, we review the recent techniques for compacting and accelerating DNN models. In general, these techniques are divided into four categories: parameter pruning and quantization, low-rank factorization, transferred/compact convolutional filters, and knowledge distillation. Methods of parameter pruning and quantization are described first, after that the other techniques are introduced. For each category, we also provide insightful analysis about the performance, related applications, advantages, and drawbacks. Then we go through some very recent successful methods, for example, dynamic capacity networks and stochastic depths networks. After that, we survey the evaluation matrices, the main datasets used for evaluating the model performance, and recent benchmark efforts. Finally, we conclude this paper, discuss remaining the challenges and possible directions for future work.

In recent years, deep neural networks (DNNs) have received increased attention, have been applied to different applications, and achieved dramatic accuracy improvements in many tasks. These works rely on deep networks with millions or even billions of parameters, and the availability of graphics processing units (GPUs) with very high computation capability plays a key role in their success. For example, Krizhevsky et al. achieved breakthrough results in the 2012 ImageNet Challenge using a network containing 60 million parameters with five convolutional layers and three fully connected layers. Usually, it takes two to three days to train the whole model on the ImagetNet data set with an NVIDIA K40 machine. In another example, the top face-verification results from the Labeled Faces in the Wild (LFW) data set were obtained with networks containing hundreds of millions of parameters, using a mix of convolutional, locally connected, and fully connected layers. It is also very time-consuming to train such a model to obtain a reasonable performance. In architectures that only rely on fully connected layers, the number of parameters can grow to billions.

In recent years, machine learning techniques have been widely used to solve many problems for fault diagnosis. However, in many real-world fault diagnosis applications, the distribution of the source domain data (on which the model is trained) is different from the distribution of the target domain data (where the learned model is actually deployed), which leads to performance degradation. In this paper, we introduce domain adaptation, which can find the solution to this problem by adapting the classifier or the regression model trained in a source domain for use in a different but related target domain. In particular, we proposed a novel deep neural network model with domain adaptation for fault diagnosis. Two main contributions are concluded by comparing to the previous works: first, the proposed model can utilize domain adaptation meanwhile strengthening the representative information of the original data, so that a high classification accuracy in the target domain can be achieved, and second, we proposed several strategies to explore the optimal hyperparameters of the model. Experimental results, on several real-world datasets, demonstrate the effectiveness and the reliability of both the proposed model and the exploring strategies for the parameters.

Navigation is a fundamental problem of mobile robots, for which Deep Reinforcement Learning (DRL) has received significant attention because of its strong representation and experience learning abilities. There is a growing trend of applying DRL to mobile robot navigation. In this paper, we review DRL methods and DRL-based navigation frameworks. Then we systematically compare and analyze the relationship and differences between four typical application scenarios: local obstacle avoidance, indoor navigation, multi-robot navigation, and social navigation. Next, we describe the development of DRL-based navigation. Last, we discuss the challenges and some possible solutions regarding DRL-based navigation.

This paper presents a novel control method for a general class of nonlinear systems using neural networks (NNs). Firstly, under the conditions of the system output and its time derivatives being available for feedback, an adaptive state feedback NN controller is developed. When only the output is measurable, by using a high-gain observer to estimate the derivatives of the system output, an adaptive output feedback NN controller is proposed. The closed-loop system is proven to be semi-globally uniformly ultimately bounded (SGUUB). In addition, if the approximation accuracy of the neural networks is high enough and the observer gain is chosen sufficiently large, an arbitrarily small tracking error can be achieved. Simulation results verify the effectiveness of the newly designed scheme and the theoretical discussions.