Sign Language Recognition

The goal of this project is to provide a Human Computer Interaction system to resolve the problem faced by the deaf and dumb people. Since the algorithm is not designed on the base of background hand gestures, it is immune to changes in the background image. It can process a variety of hand types, identify a number of fingers, and perform tasks as needed. The key objectives were met, as stated in this paper. Real-time gesture recognition is possible with this programme. There are certain obstacles that must be addressed in the future. For human-computer interaction, hand gesture recognition is critical. The hand region is extracted from the context in our system using background subtraction. The palm and fingers are then segmented so that the fingers can be detected and recognised. Finally, a rule classifier is used to predict hand gesture labels. Experiments on a 1300 image data set show that our method works well and is very effective. Furthermore, on another data collection of hand movements, our approach outperforms a method called state of the art. Gesture recognition is one of the essential techniques to build userfriendly interfaces.


Introduction
Smart devices have proliferated due to the low costs and small sizes of new digital technology. People now have more smart phones in their hands than ever before, and there is more data available than ever before. One solution, known as the Internet of Things (IoT), aims to link these embedded devices to automatically send and receive data, enabling us to enhance quality of life through data application. The Internet of Things has been widely used in the medical sector, ranging from basic self-tracking of sleep and heart rate to weight sensing and redistribution for overweight patients to avoid ulcers, as well as monitoring vital signs in hospitals and alerting nurses to arising issues in patient care. Accelerometers and gyroscopes are used in many IoT devices, allowing for the measurement of 3dimensional linear accelerations and angular velocities. The use of these sensors to detect individual human movements, such as smoking and opening a bottle of medicine, to alter habits that contribute to negative health consequences is one of the many applications of these sensors. Human motion recognition is a hotly debated subject with numerous challenges. Inside each movement, dynamic variation exists, allowing for slow or quick movements, as well as exaggerated or subtle movements. Another difficulty is extrapolating data from a single test subject to multiple users, who may have minor variations in their individual movements. Computers have become an integral part of life in today's world, and they are used in a variety of fields. However, the systems and techniques we use to communicate with computers are obsolete, and they have a number of problems, which we will address later in this article. As a result, a brand-new area called Human Computer Interactions has arisen to address these issues (HCI). [1][2][3][4][5] 2. Problem Identification 2.1. Aim of the paper The Paper mainly focuses to reduce the problem faced by the deaf and dumb people. The sign language recognition system helps the people to easily communicate with the normal people without any interruption. The system aims to recognize the sign language and translate it to the local language via text. Thus the problem between deaf and dumb people and normal people can be solved using this system. A Hand Gesture Recognition System (HGRS) recognizes shapes and orientations, depending on the implementation, in order to task the system to perform a task. Gestures are a form of nonverbal communication. At any given time, a person may make multiple gestures. Since humans interpret human movements through vision and computers need a camera, computer vision research is a hot topic.

Objective of the Project
The project's goals are to:  Research and apply the necessary tools, which include: o a robot with unidirectional movement Computer vision and artificial intelligence algorithms o The OpenCV Computer Vision Library  Create a computer vision programme that recognises basic gestures.  Examine the computer programme  Keep track of the project's outcomes.

Pattern recognition 3.1. Computer Vision
A real object in computer vision maps to a segmented region in the image from which object descriptors or features can be extracted. Any characteristic of an image or any region within it that can be calculated is referred to as a function. Objects with similar characteristics can be categorised into groups, and the combination of characteristics can be called as pattern. The creation of formal descriptors is the most important step because it requires decisions about which characteristics, quantitative or qualitative, will better fit the target product, which determines the success of the project. Quantitative descriptions known as features are used in statistical pattern recognition. The pattern vector or function vector is made up of the collection of features, and the pattern space X is made up of all possible patterns for the object (also known as feature space). Quantitatively, similar objects in each class may cluster together in the feature space, which will hopefully be useful. Classifier architecture includes determining the best discrimination feature or Discriminant to use. A statistical classifier takes n features as inputs and produces one output: a classification or judgement about the object's class. A decision rule, which is a function that places certain feature vectors in one space or subset, determines the relationship between the inputs and the outputs. The decision rule is based on the specific discrimination mechanism that was used to separate the subsets. Classifier learning is the ability of a classifier to classify objects based on its decision rule, and the training set is the set of feature vectors (objects) inputs and corresponding classification outputs (both positive and negative results). [6][7][8][9][10].

Special Role of Heuristic Rules
If pattern recognition is accomplished using complex and advanced artificial intelligence methods, heuristic principles have a place in computer vision. A heuristic is empirical yet ambiguous intelligence. Finding an exact and exhaustive solution to a problem of high complexity is often impractical or prohibitively costly, if not impossible. There are periods when no algorithm exists or cannot be found to provide a solution. Sometimes, the purpose is merely exploratory, and all that is needed for the time being is an acceptable estimation or approximation of a potential solution. In these cases, all that is needed is a good heuristic or rule of thumb to arrive at a good working solution.

System Design 4.1. Algorithm for Colour Segmentation
Thresholding is a technique that is used to limit the amount of data that is the process of defining regions within an image is known as segmentation. Segmentation may be aided by the use of colour.
The region of interest in this project was the hand on the picture. The range of HSV values for skin colour was calculated for use as the threshold values to separate the picture pixels of the hand from the background. After converting all pixels within those threshold values to white and those without to black, segmentation could begin. By using thresholding the algorithm for colour segmentation: 1. Use the camera to capture a picture of the gesture. 2. Determine the HSV value set for skin colour that will be used as threshold values. 3. Change the colour space of the image from RGB to HSV. 4. Convert to white all pixels that fall below the threshold values. 5. Make other pixels black. 6. Image segmented should be saved in image file.

Fig.1.Image comparison
Before interpreted by colour segmented image the algorithm Labelling and Blob detection is used, it must be recognised as a single entity. This can be accomplished via the marking and blob detection processes.. The adjacency relationships between pixels can be used to determine if they belong to the same area. The following are the two most common adjacency relationships: • 4-adjacency pair • 8-neighbourhood A pixel is bound to its neighbouring pixels in 4adjacency if they occupy the leftmost, top-most, right-most, and bottom positions with respect to the pixel. A four-adjacency relationship for pixels was created using the (x,y) coordinate descriptions (x, y). The top-left-most, top-right-most, bottomleftmost, and bottom-right-most pixels are all adjacent pixels in 8-adjacency. A 8-adjacency relationship for pixels was created using the (x,y) coordinate descriptions (x, y).

Data Obtaining and Pre-Processing
Data collection: The image is captured first for identifying the frame. This is crucial because the image can contain a number of variables, and these variables can cause unintended effects, reducing the data that needs to be processed. A webcam is used to capture the image, which continuously captures frames and provides raw data for processing. The uint8 input image is what we have here. The procured image is RGB must be processed before the components are separated and recognition is rendered, i.e. pre-processed. Pre-processing of data: The pre-processing method is a two-step process: • Customer segmentation Filtering by morphology The Segmentation process is the first step. It is done to convert a grey-scale image to a binary image, allowing us to have only two areas of interest in the image. One will be the side, and the other will be the background. For this method, the Otsu algorithm is used, and a greyscale image is transformed into a binary image with the hand as the area of interest. Segmentation can be done in one of two ways: 1. Local or pixel-based methods that include: a) Detecting the Edge b) Identification of boundaries 2. A region-based approach includes: a) Mixing the region, b) Splitting the region, and c) The threshold process.

Android Application
The system is developed for the purpose to help the deaf and dumb people .The application is developed using Python and HTML, CSS by inter crossing the backend results to a successful application .By using image array the background images are not taken for recognition as it reflects the hand gestures to wrong view ,which it turn reflects the sign language in wrong way.

Conclusion
This paper introduces a new approach for hand gesture recognition. The background subtraction method is used to detect the hand region from the background. The palm and fingers are segmented after that. The fingers in the hand picture are discovered and recognised based on the segmentation. Hand gesture recognition is achieved using a basic rule classifier. On a data set of 1300 hand pictures, the output of our system is evaluated. The results of the experiments show that our method works well and is suitable for realtime applications. Furthermore, on a picture series of hand movements, the proposed approach outperforms the state-of-the-art FEMD