Home Experiences
Resume Contact

Sudoku Image Recognition Solver

This project uses OpenCV to read an image of a sudoku puzzle, locate the puzzle, recognize the numbers, and finally solve the puzzle. This project was motivated by a desire to learn more about computer vision and how to work with OpenCV. It served as a prelude to a lane following RC car project in the future.
The image is first smoothened, then converted to black and white via the threshold function. It is assumed that puzzle’s bounding box is the largest continuous contour in the image. Thus, by finding the largest contour and then warping the contour to a square, the image containing nothing but the puzzle is obtained. A random forest classifier that was trained on 100 other puzzles is then used to recognize the numbers in the puzzle. And finally, the puzzle is solved by recursively inserting numbers into the blank cells and testing if the solution is valid.

Ergonomic Split Keyboard

Inspired by other hobbyist designs, this keyboard project utilized skills across multiple disciplines. The split design allowed for the user to place each half wherever they liked, and the ortholinear placement allows for less distance between keys compared to traditional layouts. It was also done as an exercise to go through the process of starting with an idea and bringing it to a functioning prototype.
The design was drawn up in Fusion360, then exported to dxf file. The files were sent to several local laser cutting company for quotes, and one of them was selected. The switches themselves were chosen for their light weight and smooth linear force.
The controller for each half of the keyboard is a Sparkfun Pro Micro flashed with modified community supported QMK firmware. Each Pro Micro had just enough pinout to drive each half of the keyboard matrix. Communication between each half is done through serial with an aux cable connecting the two halves. The aux cable also provide power to the slave half. This means only one half of the keyboard have to be connected to the computer. The firmware is programmed such that whichever side connected to the PC is the master half and reads input from the slave half when it detects the other board via the serial port.

IoT Pipeline Simulation

This project was done as part of Microprocessor course. The goal of the project was to design and develop an Internet-of-Things system composed of multiple embedded devices with cloud-enabled services.
The system contains four units: STM32F4 Discovery board, STM32F401RE Nucleo board with an IDB04A1 BLE daughter board, BLE-supported smartphone running Android 6.0, and Amazon S3 with Lambda. Each unit is able to transmit and receive data, and is connected with other modules via UART, BLE, or the Internet.
The STM32F4 Discovery board samples orientation data from the onboard accelerometer. The raw data is transmitted to Nucleo daughter board via UART connection. The daughter enables the board to connect with other devices via Bluetooth Low Energy (BLE). The bundled device acts as a transceiver that receives accelerometer readings from the Discovery board and transmits them to the phone over BLE.
After the smartphone obtains data from the Bluetooth transceiver, it saves the data in the phone’s external storage. From the Android app, the user is able to select the saved data file and upload it to an AWS S3 bucket.
When a new file is uploaded to the bucket, a Lambda function is triggered to perform FIR filtering on the raw data, calculate pitch and roll of each entry, and save the processed data to a new file in the bucket. From the Android app, the user is able to selected the processed data file and download it to the phone.
Next, the Android app sends the processed data back to the BLE transceiver, which then transmits it back to the Discovery board, following the same interfaces. Finally, the Discovery board convert the digital values to an analog voltage using DAC and the analog signal is measured on an oscilloscope for visualization.

VHDL Mastermind Board Game

This project aims to implement a Mastermind Game system on the Altera DE1 fpga board. The implemented system has two modes: user guessing mode and system guessing mode. In user guessing mode, the hidden pattern is generated randomly by the system. The user enters guesses that will be loaded into the system. The system computes the score of the guess, and displays it on the 7-segment LEDs on the board. In system guessing mode, the user chooses a four-peg combination for the system to guess. The system displays its current guess, and the user will provide a score for that guess. Then the system generates a new guess based on the feedback score. The procedure cycles until the system gets the exactly correct pattern.
Both the user guess mode and the system guess mode were tested on the board and in ModelSim. The results were correct as expected. The guess pins get updated after the scores being provided by the user. The system gets the correct answer at the fourth guess. During the guessing process, there are fewer valid guesses, with only one remaining at the end

Music Note Recognizer

The goal of this project was to use LabVIEW to develop a neural network running on a National Instrument myRIO board, capable of recognizing the distance between two notes. A neuron was designed and copies of it were connected into a three-layer neural network. This network then went through training with back propagation to improve the accuracy of the results.
The neural network has two inputs for each of the notes, and 12 outputs for the distance between the two notes. Each neuron started out with random weights. The weights were adjusted through training and back propagation. LabVIEW’s built in TestStand was used to train the network. By feeding in different notes and results and letting the weights back- propagate, the model could learn all the inputs. The weights of each neuron could be loaded and saved to a text file, thus preserving the training.

LEGO Autonomous Robot

Using Java, programmed a LEGO Mindstorm robot to navigate, localize, and position correct in a given map. The goal of the robot is to be able to locate itself if it is put anywhere in a map, navigate itself to the pick up area on the map, pick up a foam block, and drop it off at the drop off area. All the while correcting its position using regularly spaced markings on the ground.

Electromyography Armband

Using all analog components, we constructed an armband capable of measuring a user’s muscle activity using electromyographic techniques. The signal can be modulated to a certain frequency and sent to a phone to be recorded. Multiple muscle groups can each be modulated to a different frequency and thus the amount of muscle activity of an area of interest can be recorded. Combined with motion tracking we can more accurately capture the user’s motion and force. This can have VR applications, where the user’s force exertion in a motion can have modifier in the VR environment. Another application can be for physical therapy, recording a user’s activity before and after an injury can let physicians accurately see how the user is exerting force and using their muscles.