This paper develops a neural network optimization algorithm: the rectified L2 regularization, which can be used to train ternary neural networks with weights of all layers constrained to −1, 0 and +1. It will analyze how to set the learning rate and penalty coefficient during the training phase. Compared with previous approaches, the rectified L2 regularization algorithm can be directly implemented on the open source machine learning framework, such as TensorFlow and PyTorch. The accuracy of the MNIST and Fashion-MNIST test datasets is 99.40% and 92.21%, respectively, which is close to the state-of-the-art accuracy of full precision neural networks with the model compression rate guaranteed.


    Access

    Check access

    Check availability in my library

    Order at Subito €


    Export, share and cite



    Title :

    Training Ternary Neural Networks By Rectified L2 Regularization


    Contributors:
    Han, Qiankun (author) / Fan, Yuanning (author) / Ge, Jiexian (author) / Cui, Xiaoxin (author) / Yu, Dunshan (author)


    Publication date :

    2019-10-01


    Size :

    92864 byte




    Type of media :

    Conference paper


    Type of material :

    Electronic Resource


    Language :

    English



    Rectified Surface Mosaics

    Carroll, R. E. / Seitz, S. M. | British Library Online Contents | 2009


    Rectified lunar atlas

    Hartmann, W. K. / Kuiper, G. P. / Spradley, L. H. et al. | NTRS | 1963




    Efficiency of Frequency-Rectified Piezohydraulic and Piezopneumatic Actuation

    Nasser, K. / Leo, D. J. | British Library Online Contents | 2000