This paper proposes a new computationally fast algorithm for classifying the primitive shape and pose of the local contact area in real-time using a tactile array sensor attached on a robotic fingertip. The proposed approach abstracts the lower structural property of the tactile image by analyzing the covariance between pressure values and their locations on the sensor and identifies three orthogonal principal axes of the pressure distribution. Classifying contact shapes based on the principal axes allows the results to be invariant to the rotation of the contact shape. A naïve Bayes classifier is implemented to classify the shape and pose of the local contact shapes. Using an off-shelf low resolution tactile array sensor which comprises of 5×9 pressure elements, an overall accuracy of 97.5% has been achieved in classifying six primitive contact shapes. The proposed method is very computational efficient (total classifying time for a local contact shape = 576μs (1736 Hz)). The test results demonstrate that the proposed method is practical to be implemented on robotic hands equipped with tactile array sensors for conducting manipulation tasks where real-time classification is essential.