Convolutional Neural Networks for Handwritten Javanese Character Recognition

Convolutional neural network (CNN) merupakan model baru di bidang pengenalan objek. Dikhususkan untuk input data yang bertipe spatial, CNN memiliki layer khusus, yaitu layer konvolusi dan layer pooling yang memungkinkan proses pembelajaran fitur secara hierarki dari data. Untuk pengenalan karakter tulisan tangan secara offline, seperti pengenalan karakter pada database MNIST, CNN menunjukkan performa yang lebih baik jika dibandingkan dengan model ataupun metode yang lain. Dengan memanfaatkan keunggulan CNN tersebut, dalam penelitian ini telah dikembangkan sebuah perangkat lunak dengan fitur pengolahan citra dan modul CNN untuk pengenalan karakter tulisan tangan Aksara Jawa. Perangkat lunak yang dikembangkan memanfaatkan deteksi kontur dan deteksi tepi Canny menggunakan pustaka OpenCV terhadap citra karakter


INTRODUCTION
Like many other ethnic groups in Indonesia, Javanese tribe also has a traditional script called Aksara Jawa or Hanacaraka which was intended as the font to write any documents in Javanese Language. Aksara Jawa consists of twenty base symbols or base characters which are usually called as nglegéna or carakan. Figure 1 shows twenty base characters of Aksara Jawa with their transcription in Roman alphabet.

Figure 1 Javanese script characters
Unfortunately, Javanese people these days do not use Aksara Jawa in their daily life anymore. Although Aksara Jawa is taught in primary and secondary schools as part of muatanlokal curriculum in Central Jawa and Special District of Yogyakarta which are the main base of Javanese culture, it mostly can only be seen in road name signs as the transcription for the Roman alphabet. In order to preserve Javanese culture, especially traditional Javanese script, not only does Aksara Jawa can be used in education but it also can be used as a means for daily communication [1]. To support the preservation of Aksara Jawa both in education and also in its usage in daily communication, a tool or a software which has an ability to automatically recognize handwritten Javanese character is needed.
There have been several studies of research regarding handwritten Javanese character recognition. Most of these studies use machine learning techniques such as hidden Markov model (HMM) or support vector machine (SVM) to perform the classification task with several feature extraction techniques. Widiarti and Wastu [2] used HMM to classify horizontal and vertical vector features while Nurul et. al. [3] used multi class SVM to recognize directional element feature from handwritten Javanese character dataset.
Some other studies of research use artificial neural networks to classify handwritten Javanese character. Isnawati [4] employed backpropagation neural networks and applied thinning method to handwritten Javanese character dataset. Wibowo et. al. [5] used multilayer perceptron (MLP) model which was trained using backpropagation algorithm. Similarly, Arum [6] used the combination of wavelet feature extraction technique and also backpropagation neural networks. Budhi and Adipranata [7] employed several artificial neural network methods with ICZ-ZCZ features for handwritten Javanese character recognition. Several other studies [8,9,10,11] show that neural networks are able to perform image classification task with good performance if they are combined with appropriate feature extraction techniques.
In this study, we propose a software which employs convolutional neural networks (CNN) model with image processing module using OpenCV library to perform handwritten Javanese character recognition. CNN is one of deep learning methods which has special layers which are able to perform feature extraction learning and extraction directly from raw input space. Currently, CNN becomes state-of-the-art method in image classification tasks.

METHODS
The objective of this study is to build a software which performs handwritten Javanese character recognition. In order to achieve the objective, we developed a classification module which employs a CNN model. We trained the CNN model with a dataset of handwritten Javanese character images. For evaluation purposes, we also used the same dataset to train an MLP model with one hidden layer and an MLP model with two hidden layers. We compared those three models in both classification accuracy and training time.
In this section, details of research methods used in this paper is described. The methodology consists of a) data acquisition for handwritten Javanese characters, b) building the CNN model, c) model training and model testing, and d) developing a web based application for handwritten Javanese character classification. Each part of the methodology will be described in the following subsections.

Data acquisition
In order to train the CNN model, an image dataset which consists of handwritten Javanese characters are needed. Unfortunately, there is no available public secondary dataset for handwritten Javanese characters. Therefore, in this study, a primary dataset of handwritten Javanese characters was manually collected from some people as shown in Figure 2. The dataset consists of 2,000 data which belongs to 20 classes of basic Javanese characters. Each class of Javanese character consists of 100 training data. Each data in the dataset is a 8-bit grayscale image which has a dimension of 28 x 28 pixels.

2 Building CNN model
In essence, CNN can be seen as an extension for traditional neural network models, such as multilayer perceptron (MLP). A CNN model architecture consists of special layers to extract features from raw input space and a fully-connected neural network model with logistic regression classifier. The features which are usually called as feature maps are obtained from those special layers and then become inputs for fully-connected neural network which is actually an MLP model. Figure 3 shows an example of a CNN model architecture taken from Sermanet et. al. [12]. Figure 3 An example of CNN architecture [12] From Figure 3, it can be seen that prior to be processed in the classifier, the input for CNN will be processed in two stages. Both stages consist of convolution and subsampling operations. It can also be seen that both convolution and subsampling operations will reduce dimensions of the inputs. A convolution operation will transform a single two-dimensional input matrix into some smaller two-dimentional matrix or feature maps. At the end of the second stage, a function is performed to flatten or transform each feature map from two-dimentional matrix to one-dimentional matrix so that the feature maps are ready to be classified with MLP or fully-connected neural network.
In CNN, the main objective of convolution and subsampling operations is for extracting features from raw input data. In order to achieve this objective, convolution operations which are multiplications of small kernel matrices and specified areas of a two-dimentional input matrix are performed. To produce a single smaller dimensions of feature map from an input matrix, the kernel will be shifted and several multiplications will be performed from left to right and from top to bottom over specified areas of the input matrix. The equation for a convolution operation to is defined in Formula (1) as stated in [13] as follows: where is an element of a single output matrix from a convolution operation. The output matrix is produced from an activation function . First, the sum of all multiplications of kernel matrix and input matrix is computed, subsequently the bias value is added to the elements of the resulting matrix. Finally, it becomes the input for function . In this study, the activation function used is rectified linear unit (ReLU) which is defined in Formula (2). After convolution operations, a subsampling or pooling operation will be applied to each of feature map for dimension reduction. In this study, the function for subsampling used is max pooling function so that prominent features can be obtained. To reduce the dimension of a single feature map, a two-dimentional m x n kernel will select the highest value of (m x n) neighbouring elements and produce one single element in a new feature map matrix. Similar to convolution operation, the kernel will also be shifted from left to right and from top to bottom to produce a new feature map.
To prevent overfitting and improve the performance of CNN model, a dropout regularization algorithm [14] also be applied in the training phase of the model. Using dropout algorithm, some neurons in CNN layers will be randomly disabled with Bernoulli distribution. Subsequently, in the testing phase, all of neurons in all layers in CNN model will be activated again. Srivastava et. al. [15] stated that dropout algorithm can improve the performance of neural networks in various benchmark datasets.
After several convolution and subsampling operations, feature maps will be flatten so that they are ready to be classified with MLP or fully-connected neural network. An MLP or a fully-connected neural network consists of several layers. Each layer consists of several neurons that will perform a matrix multiplication between an input matrix and internal weights as defined in Formula (3) as follows: where is bias value, is the number of neurons in a single layer, and is an activation function, such as ReLU function that has previously defined in Formula (2).
After processed in several layers, feature maps will be processed in the output layer. The output layer of a fully-connected neural network or an MLP is a softmax function that produces probabilities of classes ( ) that the CNN input may belong. A softmax function is defined in Formula (4).

3 Model training and testing
A CNN model is included as one of neural network models which use supervised learning algorithm. This means that to update internal weight matrices in training phase, the model uses a cost function which calculate the distance between the output of the model, which is the predicted class, and the actual class that the input belongs. CNN models use cross entropy error function as the cost function which is defined in Formula (5) as follows: where is the target class and is the output of CNN model. To perform training and testing phases of CNN model, the handwritten Javanese characters dataset was divided into 80% of training dataset and 20% of testing dataset with kfold cross-validation technique. Xavier weight initialization [16] was used in each training fold to initialize internal weight matrices in the CNN model. In the testing phase, the performance of each fold in CNN model was measured using a confusion matrix and the value of the classification accuracy. The classification accuracy is defined in Formula (6) as follows: (6) where is true positive, is true negative, is false positive, and is false negative. For this study, details of the CNN model architecture is shown in Table 1. The CNN model consists of three stages of convolution and subsampling operations, a fully-connected layer, and a softmax output layer. ReLU activation function and dropout regularization were applied to all layers in CNN model. The CNN model was built with Theano library, a deep learning library for Python. The CPU used was Intel Core i5-5200U, and the GPU was Nvidia GT940M. We also trained and tested an MLP model with one hidden layer and an MLP model with two hidden layers with same handwritten Javanese character dataset. Accuracies of MLP models and CNN model were compared to verify whether convolution and subsampling layers in CNN model are able to learn features from the dataset. We compared the training time needed for both CNN and MLP model. Details of MLP models architecture are shown in Table  2 and Table 3 respectively. It can be seen from Table 2 that the MLP model with one hidden layer has an input layer with 784 neurons and a hidden layer with 1,000 neurons. Each neuron in the input layer will receive a single pixel value from a handwritten Javanese character image which has a dimension of 28 x 28 pixels. The MLP model will produce 20 values of probability of classes which the input may belong. From the architecture described in Table 3, it can be seen that we used 1,000 neurons in the first layer and 2,000 neurons in the second layer. We also used ReLU activation function in hidden layers and softmax function in the output layer for both MLP models.

4 Developing the web based application
The software developed in this study is a Django web based application which has an ability to access webcam and receive an image from user with upload method. It has two main features, namely segmentation of captured handwritten Javanese character image with OpenCV library and classification of the segmented image with CNN module. Figure 4 depicts the detailed architecture of the developed software in this study.  Figure 4, it can be seen that the image segmentation module contains Canny edge detection and contours detection procedures with OpenCV library. The classification module consists of CNN module with stored weights which are obtained from model training using handwritten Javanese character dataset.

RESULTS AND DISCUSSION
When a user first opens the URL of the software through a browser, both direct upload and webcam upload method can be used. The user can upload a captured handwritten Javanese character image or capture a handwritten Javanese character image using a webcam. After receiving the image, the software will use OpenCV library to perform segmentation procedure which its details will be described in the next subsection. Furthermore, the user can choose CNN method or MLP method to classify the segmented image. To illustrate how the classification process in the software works, Figure 5 depicts a state in the software which has already received a captured image and has already performed the classification task.
From Figure 5, it can be seen that the software could correctly recognize a character using CNN method, while it fails to do so with MLP method. Using MLP method, the character is incorrectly recognized as PA. On the other hand, CNN method can correctly recognize the character as HA. From this example, we can also see that the accuracy of CNN model is generally higher than the accuracy of MLP model. To verify this assumption, we conducted a comparison procedure for testing accuracies for both CNN and MLP model which its details will be described in the next subsection, after the image segmentation procedure description.

Image Segmentation Result
Prior to the classification procedure, a segmentation procedure will be performed for the captured handwritten Javanese character image. Canny edge detection algorithm and contours detection algorithm are applied to the captured image using OpenCV library to perform the segmentation procedure. This segmentation procedure will produce segmented images of Javanese letter characters. Figure 6 shows an example of the segmentation process as one of the main software features in this study.

Image Classification Result
We used two variables namely training time and testing accuracy using k-folds cross validation method to measure the performance of CNN for handwritten Javanese character recognition task. We also compared the training time and testing accuracy results to other results which are obtained from MLP with one hidden layer and MLP with two hidden layers. We used pixel values of the input image as inputs for MLP without prior feature extraction technique.
In each fold of the training phase for all models, we set a maximum epoch of 10,000 iterations. In each iteration, we calculated the cost function value and stored the internal weights matrix. Since the objective of the models is to minimize the cost function, we used an internal weights matrix of the iteration with the lowest cost function value for the models in the testing phase. Table 4 and Table 5 show the comparison results of those three models respectively. From Table 4, we can see that MLP with one hidden layer requires minimal training time among other models for all cross validation folds. In average, MLP with one hidden layer only needs less than 13 minutes to train. If we add one more layer to the MLP, the training time will increase about 2.7 times. On the other hand, CNN requires more time to train compared to MLP due to the complex computations in convolution and subsampling layers. The model needs almost two hours for the training phase, which are about 4.6 longer than the training time for MLP model with one hidden layer. Details of testing accuracy results of the three models are summarized in Table 5. Overall, classification accuracies of CNN model outperform classification accuracies of both MLP model. This means that convolutional and pooling layers in CNN model can successfully learn features of the dataset. It can also be seen that adding more hidden layers to the MLP model does not improve its accuracy.

CONCLUSIONS
In this study, a software which employs CNN model to perform classification task for handwritten Javanese character recognition had been successfully developed. To quantify the performance of the classifier, k-folds cross validation technique had been used to measure the classification accuracy and the training time. The classification accuracy and the training time of CNN model were compared to the classification accuracy and the training time of MLP model with the same dataset. From the experiments, we conclude that CNN model's accuracy is better than MLP model's accuracy for handwritten Javanese character recognition task in all folds. However, CNN model needs longer time to be trained compared to MLP model. From the experiments, it can be seen that the accuracy of CNN model for the handwritten Javanese character dataset cannot reach 90% in all folds. This may be due to the insufficient number of the dataset since deep learning methods will give their best performance for huge amount of training data. CNN model optimization for a bigger handwritten Javanese character dataset is left for our future work.