Communication is the most important part of life. Around 1% of the total population of the world is suffering from hearing impairment, and their life is not as easy as it is for human without limitations. In this thesis we propose a model that recognize ASL and convert signs to voice using radial basic function neural network. This model will surely be implemented in real life to make the life of deaf people easier. In this thesis, we are training radial based function for the recognition of ASL. This model starts with image preprocessing of skin detection using grayworld illumination, color space conversion from RGB to YCbCr, and skin detection via threshold. The detected skin regions are represented with centroids and tracked using Euclidean distance measurement. To transform essential data into a more intelligent form, dimension reduction and feature extraction algorithm of principal component analysis (PCA) and linear discriminant analysis (LDA) are used. Finally, a radial basis function neural network (RBFNN) is used for classification to recognize different hand gestures.