Once the glove was completely built, we could then reliably take note of ADC ranges for each finger. We found that a completely bent finger was different enough from a half-bent finger or from a completely unbent finger for us to differentiate between the three states. This gave us confidence that the letters would be separable for the most part.
Recall that the ADC circuit was built using a voltage divider circuit which had a resistor connected to Vdd and the flex sensor connected in series to ground. Since the sensor increases resistance with flex, bending a finger corresponds to higher ADC values. As a first step, we started to decode letters simply by imposing threshold values for each finger, and printing them to the terminal. For example, the letter 'A' (see image below) has a thumb unbent and the rest of the fingers bent. So, 'A' was decoded by an if statement that checked whether the thumb's flex sensor was below a certain threshold (unbent) and the other four fingers' flex sensors were each above a certain threshold (bent). This strategy worked fantastically ... for letter 'A'. Even 'B' worked pretty well. So yes, there was reason to celebrate a good week into the project. But we hit some problems with this method when trying to decode other letters, such as 'C'. As can be seen below, all of the fingers are in a half-bent position for 'C'. We found that it was much easier to detect a bent state or an unbent state because they are on the extremes of the ADC spectrum. Half-bent, however, is a very noisy state, and its ADC range can vary from day to day and even from trial to trial.
To gather some more information on exact nature of our problem, we attempted to decode a few more letters, and found that the half-bent state is going to be our challenge for the majority of letters in ASL. Perhaps we should try something a bit more adaptive than the absolute threshold method? Yes, you've guessed it; we're sending our glove to school!
Our next step was to employ some version of k-means as a learning algorithm. Perhaps in time, our glove will be able to do our 350 project for us! (I suppose that would be a project all by itself). Anyway, our method involved creating 3 "buckets" for each finger, representing the three possible states. These buckets were initialized to the minimum, maximum, and middle ADC values for each finger. As each ADC value comes in for a finger, it is compared with each of the three elements of the bucket. The bucket that the value is closest to gets averaged with the value so as to update the bucket. In this manner, the three "bent", "half-bent" and "unbent" states are automatically calibrated after a few averages.
This new algorithm seems to be running a lot smoother and more consistently. However, averaging sometimes pulls the bucket values to extremes. The program is able to decode a certain letter, but it can no longer do so after signing another letter since the buckets have changed. Or vice versa, one must pull the average down before it can detect a letter. As can be seen, there is some work to be done to play with the k-means implementation. We do believe this is a better way of decoding, however, than our original thresholds.
Unfortunately, our pinky broke a couple days ago! Sad =(. Sid and Roop have ordered a new sensor along with some extras so that we can resume our project right away. In the mean time, Bob was nice enough to lend us one of his sensors! Another problem that we faced was a sudden reduction in resistance range with our sensors. We think that heavy usage is the cause of the range reduction, and hope that this doesn't happen again with the new sensors. A possible option is to decrease the voltage divider resistance in expectation of the reduced range, so that our ADC range will not shrink. But imagine opening up that beautifully soldered voltage divider circuit. Ew.
Phew! Well there you have it. We'll keep you updated! Oh, by the way, wireless transmission has been implemented so I am no longer tethered to the computer!
