Overview:

In the software side of the system, we focused mainly on the animation of the pet and interaction of different sensors and pushbuttons.

In order to draw on the matrix, we use the code from a previous year project that was done by students at Cornell. They build a similar project called LED matrix clock, and we found out that we can reuse their library so that we can draw on the matrix.

Firstly, we added GFX library given by Adafruit. We have three files from Cornell students’ library to realize the connection and interaction between software and LED matrix. The first one is “rgb_matrix.c”, this file basically configured the LED matrix to PIC32. The second file is “matrix_gfx.c”. This file focused on drawing on LED matrix. It provided methods for refreshing matrix and drawing some fundamental pattern. We only made some modification on those two files to make the configuration more suitable for our project. The last one is “matrix_tests.c”, and we employed the methods like drawing pixels or drawing lines from “” and created our own methods to draw pets mood and action.

We draw everything in one .c file called “matrix_tests.c.” In here, you can find out how did we draw the pet and other shapes as well. This file is very easy to modify. All you need to do is to call different drawing functions and put in your desired dimensions.

Our software is organized in 5 protothreads: controlling the LED matrix, reading the temperature sensor, reading the heart rate sensor, button debounce and interaction, as well as unit changes for the temperature. We will discuss each components below:

Update LED matrix:

This is the “brain” of our system because it includes all the update and animation of our Matrix. The protothread updates our LED matrix every 500 microseconds. The first task in it is to check the current face position and moving direction of the pet. If the pet is moving, we use the left most point to determine its position. If the program finds that the position arrive at the border of the moving area, the moving direction of the pet changes to the opposite side. Each time the LED matrix clear and redraw the screen, the buffer to store pixel information for Matrix display need to be refreshed.

The next significant part is drawing pet and its corresponding mood. We implemented 7 different mood modes there: “NEUTRAL”, “SAD”, “ANGRY”, “HAPPY”, “HAPPY_N”, “TIRED”, “EXCITED”. We use a switch case method to control mood mode, and the variable “mood_select” is the switch. In each mood case, there is a corresponding drawing methods from “matrix_tests.c” file. The variable “mood_select” will change due to button input, and make the system select corresponding mode to display. There are two happy modes. One is “HAPPY”, and one is “HAPPY_N”. When the pet is in “NEUTRAL” state, the mood mode will automatically change to “HAPPY_N” state after about 15 seconds. After another 15 seconds, the mood mode will back to “NEUTRAL” stage. However, if the button input change mood mode to happy mode, the pet mood will change to “HAPPY” stage rather than “HAPPY_N” stage, and the pet will stay in “HAPPY” mode until further interaction. In each stage, if the temperature in celsius is larger than 26, the pet will go to “TIRED” mode. If the heartbeat rate is larger than 125, the pet will go to “EXCITED” mode. In “EXCITED” mode, the pet is moving horizontally, and we uses the most left pixel to determine current position as well as draw it on the matrix. Temperature has higher priority than the heartbeat rate. In “TIRED” mode and “EXCITED” mode, if the temperature or heartbeat rate is not that large, the pet will be back to “NEUTRAL” mode automatically.

In happy mode and sad mode, there is also firework and tear animation. In these modes, they have their own mode selection to control the position and the state of firework or tear animation. The mechanism is similar to the pet mood refreshing.

The protothread also controls the display of temperature readings corresponding to current selection of celsius or fahrenheit.

When the heartbeat sensor is activated, which means there is continuous reasonable pulse detected by the sensor. There will be a red heart next to the display of BPM. The heart blinks every second.

Reference: ECE 4760 Final Project, A RGB LED matrix clock with IR Remote and Serial Interface

Reading Temperature Sensor :

This protothread handles the interaction with our temperature sensor. The PIC utilizes analog converter peripheral and read analog value from sensor. The program reads analog value every microsecond, and the formula is given by the online library. There is also a Celsius to Fahrenheit conversion formula in the protothreads to convert calculated temperature to Fahrenheit value at the same time. It also stores the float value to a character array to print on our LED matrix.

Reference: https://learn.adafruit.com/tmp36-temperature-sensor/using-a-temp-sensor

Reading Heart rate Sensor:

This protothread handles the interaction with our heartbeat sensor. The reading mechanism is given by the xtronical library. Firstly, the microcontroller utilized analog converter peripheral and read analog value from sensor. The program set upper threshold and lower threshold to find active pulse and calculate heartbeat rate. The protothread updates every one microsecond. We also have a timestamp in the method to track how much time the program has been worked, since it is needed for BPM calculation. Each time the analog value re-encounters inactive pulse (lower than lower threshold), one beat period will be labeled as finished and calculation will be executed when analog signal re-entered active pulse (larger than upper threshold). The calculation formula is from an project online that utilized the same heartbeat sensor.

Reference: http://www.xtronical.com/basics/heart-beat-sensor-ecg-display/

Mood Button debounce and interaction:

This protothread is for changing the pet mood according to mood button input. There is a FSM to check button input and register each active button input only once. The protothread checks button input from corresponding pin every microsecond. If the input asserts high, the state will go to Maybe Pushed state for a further check to avoid a “false” press. If the button input asserts high in Maybe Pushed state, the index to track mood selection will increase 1, which means that the pet will change to next mood mode. If the current mood is “TIRED”, the pet will stay in this mood. If current mood of the pet is “HAPPY_N”, the pet will go to Neutral state. If the index is larger than the number of default modes, the index will reset to 0 for this input and the pet is also going to change to corresponding mode. Only if the button is released, the button input can be re-registered.

Changing from Fahrenheit to Celsius:

This protothread is for changing the temperature mode from Celsius to Fahrenheit or vice versa. There is a FSM to check button input and register each active button input only once. The protothread checks button input from corresponding pin every microsecond. If the input asserts high, the state will go to Maybe Pushed state for a further check to avoid a “false” press. If the button input asserts high in Maybe Pushed state, the temperature display mode will change to next one. Only if the button is released, the button input can be re-registered.

Reference: https://learn.adafruit.com/tmp36-temperature-sensor/using-a-temp-sensor

Tricky part to write:

Although we have online graphics library from Adafruit and example code from Cornell project, we still spent lots of time on configuring our 16×32 LED matrix with the library. Also, the refreshment rate of our LED matrix need compatible with the interrupt for refreshing pixel information. We spent time on figuring out the “clear and redraw” step for the matrix. It is hard for us to store information for different parts at different time. We need separate the information buffer or refresh information for all parts simultaneously. To make the animation and number display stable and refresh at a certain rate, we put all matrix related method in one protothread, and it worked well.

The parts listed in our proposal all worked and connected well. The only problem in the project is the reading from our heartbeat sensor. We used the same method and conversion formula as many online tutorial did. However, the analog reading from heartbeat sensor was not stable. One possible reason is that the matrix refreshment interrupted the pulse value collection. Another possible reason is that the most of tutorials utilized Arduino Uno microcontroller, and the power input is 5V. Ours is 3.3V instead. To improve accuracy, we gathered raw data and determined upper threshold and lower threshold ourselves. It helped improved accuracy, but the stability was still a problem.It is possible the conversion formula need further modification.