Machine Learning + Vibrations
Whether we recognize it or not, every day we interact with vibrations and gather data from them. If a person is asleep in the car, the ceasing of vibrations from the engine may awake them and let them know they have arrived at their destination. We interpret airborne vibrations as sound so we can hear our friend talking or listen to music. These examples are fairly obvious ones, but I’m fascinated by the potential data that could be found in the subtle vibrations, or vibrations we would tend to ignore.
As a part of receiving my STEM diploma, I needed to complete a capstone project of my choosing while working with a mentor. We had a considerable amount of freedom in the project we pursued, as past topics ranged from studying a salmon population to optimizing the experience of a website. I decided to work on a project related to artificial intelligence, and after brainstorming for a while I had a revolution: the dishwasher!
In my family we have an ongoing battle of keeping dirty silverware out of the clean dishwasher. Sometimes a few dishes still have bits of food stuck to them and someone may mistake the whole load for dirty and add their dishes. Other times we may just be in a hurry. We tried using the magnet that’s pictured above to keep track, but we never did get in the habit of updating it. We needed some clear indicator that could update its own status.
While this is certainly not the first thing that would come to mind for a STEM Capstone project, the dishwasher has a great variety of vibrational data that can be extracted from it. People run it, put dishes in, take dishes out, and slide the racks. All of these types of vibrations have unique characteristics, and I wanted to train a machine learning model to recognize them. After the model recognized key events, it could then update the program’s status on whether the dishes are clean or dirty.
I needed lots of data to build these ML models, so I started collecting data with an accelerometer. I recorded hundreds of vibrational samples of me doing various tasks such as placing dishes into the dishwasher or taking them out. I took this data and fed it through TensorFlow, Google’s machine learning library, and I began training the ML models to classify these various types of events. If you are interested about this process in more detail, you can check out my research paper!
Next, I needed to build an actual device that attached to the dishwasher. My new dishwasher indicator used a Raspberry Pi for the processor with an Adafruit accelerometer housed in 3D printed case I designed. Two strips of addressable LEDs lit up the front of the case to show the dishwasher’s status. I deployed the ML models I trained on my laptop to the Raspberry Pi and here are some of the results:
Though this device wasn’t perfect, it was neat to see it working and showcasing how valuable data found in vibrations can be!
