ECE 492 User manual

ECE 492 Capstone Project
2013W
Virtual Guitar Gloves
Summary: Play a virtual guitar in the air by using two gloves fitted with tactile sensors
and an accelerometer.
Group Members:
Elysia Jong [e-mail]
Qingyue Zhou [e-mail]

Abstract
Terasic’s V.1.6 DE2_SD_Card_Audio demonstration was used as a base for this
project. By default, it waited for an SD card to be inserted into the FPGA’s port, then
played all the files within it in sequence through audio out. The files are assumed to be
of .wav format with a 48 kHz sampling rate. It also displayed on the 7-segment and LEDs.
The project had no documentation and the files were written in Verilog.
This demo project was modified to only play one file at a time, depending on
which tactile sensor was pressed and only if the accelerometer detected a strum action
from the user. The LCD was programmed to display whether the system was in note or
chord mode and which octave scale was selected. The tactile sensors, in essence, were
simple on/off switches with hardware debouncing implemented. The digital
accelerometer communicates through the onboard GPIO pins to a second I2C bus,
separated from the I2C bus used for the audio CODEC to avoid any possibility of conflict.
All sounds played are pre-recorded files stored in the SD card. External components are
all wired to the Altera DE2 via the GPIO pins.

Table of Content
FUNCTIONAL REQUIREMENT............................................................................................................3
SPECIFICATION:.........................................................................................................................................3
HOW TO USE:.............................................................................................................................................3
GOALS ACHIEVED: ....................................................................................................................................3
DESIGN AND DESCRIPTION OF OPERATION.................................................................................4
FLOW CHART:............................................................................................................................................5
INPUT TO OUTPUT FUNCTIONALITY:..........................................................................................................5
ALGORITHMS/TASKS:................................................................................................................................5
BILL OF MATERIALS .............................................................................................................................6
REUSABLE DESIGN UNITS ...................................................................................................................7
DATASHEET..............................................................................................................................................8
USER PERSPECTIVE DIAGRAM ..................................................................................................................8
USER I/O SIGNALS: ...................................................................................................................................8
RIBBON HEADER MAPPING.....................................................................................................................10
ACCELEROMETER INTERFACE:................................................................................................................11
SD CARD INTERFACE:.............................................................................................................................11
EXTERNAL CIRCUITRY SCHEMATIC:.......................................................................................................12
POWER:....................................................................................................................................................12
BACKGROUND READING ...................................................................................................................12
SOFTWARE DESIGN .............................................................................................................................15
BLOCK DIAGRAM:..............................................................................................................................16
LIST OF LIBRARIES:.................................................................................................................................17
TEST PLAN ..............................................................................................................................................18
SOFTWARE: .............................................................................................................................................18
HARDWARE:............................................................................................................................................18
RESULTS OF EXPERIMENTS AND CHARACTERIZATION........................................................19
STRUM DETECTION:................................................................................................................................19
CPU CLOCK/PERFORMANCE:..................................................................................................................19
REFERENCES..........................................................................................................................................20
APPENDICES...........................................................................................................................................22
QUICK START MANUAL ..........................................................................................................................22
Hardware:...........................................................................................................................................22
Overview:............................................................................................................................................22
Steps to start from flashed board:.......................................................................................................22
Software:.............................................................................................................................................23
How to play:........................................................................................................................................23
FUTURE WORK......................................................................................................................................23
1

BACKUP PLANS/OPTIONS:.......................................................................................................................23
EXTENSIONS:...........................................................................................................................................24
HARDWARE DOCUMENTATION.......................................................................................................25
BLOCK DIAGRAM OF INTERACTIONS BETWEEN PROCESSES:...................................................................26
INDEX TO SOURCE CODE: ........................................................................................................................26
2

Functional Requirement
Specification:
Our virtual guitar consists of 2 gloves. There are 7 tactile sensors on the left hand, which
are used for 7 notes (A to G). There are 4 sensors on the right: 2 used to go up/down octaves, 1 to
switch from playing notes to chords or vice versa, and a reset to the original mode (notes with
default octave). There is also one accelerometer on the right hand to detect a strum from the
player. By combining the glove’s tap sensor and accelerometer signals, we can realize a guitar
playing without an actual guitar.
The player is assumed to be right-handed. Once the player is touching their left thumb to
a left sensor and a strum action from the right hand is detected, the speakers will then play the
certain note/chord the player intended to play. This is meant to reflect the playing of an actual
guitar, where the player would hold the fretboard with their left hand and play the strings with
their right. If the player waves his/her right hand, but no tap sensor is engaged, nothing will
happen.
The 2 gloves have a wired connection to the Altera DE2 FPGA board through the GPIO
pins. Speakers are connected to Line Out for audio output. The guitar sounds (A-G, notes/chords)
are saved in an SD card. Filtering is to be applied to the sound files to obtain the different
octaves.
How to use:
1Playing a note:
auser must press and hold their left thumb to 1 left finger note (“A” - “G”)
ithis is to mimic holding the fretboard on a real guitar
buser moves right hand downward in a strumming action
csimultaneous combination of both a. and b. required
2Changing between single notes and chords:
auser needs to press right thumb to “MS” and release (no hold)
3Changing octaves:
auser must press their right thumb (no hold) to either:
i↓: octave down
ii ↑: octave up
bplay a note normally as in 1.
4Resetting to default octave and note mode:
auser needs to press right thumb to “R” and release (no hold)
Goals achieved:
3

The tactile sensors, combined with hardware debouncing, had no issues. The left hand
sensors correspond to the correct notes/chords and the right hand’s mode switch (for selecting
notes/chords), octave up, octave down, and reset buttons were functional. The LCD also
correctly displayed whether the user was in note or chord mode and which octave. The threshold
on the accelerometer was set so that it would not recognize motion if the user moved their hand
too slowly. Due to the significant amount of time it took to interface the accelerometer, the
filtering used to achieve the octaves was not done. The original sound files were run through an
audio editor called Audacity 2.0.3, where the frequencies were changed, new sound files
generated and then stored in the SD card. This meant there were 42 files instead of the intended
14 files for guitar sounds.
The loading time required for the SD card is not noticeable. When running from flash
memory, powering up the FPGA and instantly attempting to play a sound will work without any
apparent delay. Originally, there was noise in the output sound. The saved .wav files are from an
acoustic guitar, but the output from the FPGA sounded synthesized and in a minor scale. This
was corrected by changing the Nios II processor clock from 50 MHz to 100 MHz.
Design and Description of Operation
There are 11 tap sensors for the 7 notes (A-G) and 4 modes (R, MS, Octave ↑, Octave ↓)
placed as shown in the image below. Each sensor is located on a finger joint section, so that the
player may bend their fingers. All unused finger joints may be used for future expansion. The
accelerometer will be attached on the back of the right hand near the wrist.
Image 1: Placement of sensors modified from http://media.npr.org/assets/news/2010/01/27/hands-
ef12ac2473025791c19e4478b5ca0078c2fa1942-s6-c10.jpg
4

Flow chart:
Image 2: Software design flow
Input to output functionality:
1Tap sensor and accelerometer signals detected from GPIO pins
2Based on input, read sound file from SD card
3Output sound file to speakers connected to FPGA
Algorithms/Tasks:
“task_which_note” simply polls for user input on the left glove through an infinite while
loop. It keeps track of the BASE_ADDRESS of the tactile sensors using IORD_PIO_DATA
(BASE_ADDR,0) on the GPIO pins. The tactile sensors, using the internal ~25kohm resistors on
the DE2, are wired to be active high. Therefore, once a 0 value is detected in the
5

BASE_ADDRESS, it corresponds to a tactile sensor being depressed. The sector that should be
read from the SD card is then updated depending on which sensor it was. While polling is
generally wasteful for CPU cycles, this system has no need to do any other critical computation
or tasks other than to respond to the user’s actions.
Interrupts and queues are used to handle the signals from the right glove, as these are
used to change the modes of play available, versus the actual selection of notes. The thought was
that these buttons will be less often pressed and so there is less need for polling. The current state
of the system is displayed on the LCD by “taskModeDisplay.”
“task_accelerometer” initializes the I2C bus as well as initializing the accelerometer
registers. The values in these registers configure how it will function. For this project, it is set to
detect freefall in the x-axis (1D) from the accelerometer’s point of view. This corresponds to the
axis perpendicular to the earth’s surface.
“task_sd_play” gathers all the information from the other tasks and actually plays
the .wav file from the SD card to the Line Out audio jack. The range of sectors to play,
determined from “task_which_note,” is read one sector at a time and stored in a FIFO buffer that
is fed to audio out.
Bill of Materials
Other than the items listed below, a standard wall outlet (120 V, 60 Hz) is required to
power the Altera DE2 board and external speakers if headphones are not used.
Qty
Part Name
Unit Cost
(CAD)
Total Cost
(CAD)
1
Altera/Terasic DE2 development board
http://www.altera.com/education/univ/materials/boards/de2
/unv-de2-board.html
$517.72
$517.72
2
Ribbon cable (40 & 50-pin)
$10.00
$20.00
1
Perf board
$7.50
$7.50
1 roll
Wire wrap
$10.00
$10.00
4
Capacitors (0.1 uF)
$0.25
$1.00
1 kit
Resistors (used 4x 180 ohm, 4x 1 kohm)
$7.95
$7.95
2
Headers (40 & 50-pin)
$0.20
$0.40
1
Speakers/headphones
$20.00
$20.00
1
Thin wool gloves (pair)
$5.00
$5.00
6

1
2GB SD Card
$10.00
$10.00
1
Accelerometer (MMA8452Q)
Digital output via I2C bus. Low voltage to interface (1.6-
3.6V). ±2g/4g/8g 3-axis. Package is about the size of a
quarter. 1.95-3.6V supply. 2 programmable interrupt pins.
Supplier: https://www.sparkfun.com/products/10955
Datasheet:
http://dlnmh9ip6v2uc.cloudfront.net/datasheets/Sensors/Ac
celerometers/MMA8452Q.pdf
$9.95
$9.95
11
Tactile sensor (450-1649-ND)
Non-illuminated, top-actuated, 6.00mm x 6.00mm, off-
mom micro switch. Actuator height is 4.30mm with
0.25mm switch travel and through-hole mounting. Max
contact rating at 0.05A @ 12VDC, minimum at 10uA @
1VDC.
Supplier: http://www.digikey.ca/product-
detail/en/FSM2JH/450-1649-ND/1632535
Datasheet:
http://www.te.com/commerce/DocumentDelivery/DDECon
troller?Action=srchrtrv&DocNm=1825910&DocType=Cus
tomer+Drawing&DocLang=English
$0.05
$0.55
TOTAL
$610.07
Reusable Design Units
1. The Altera DE0-Nano board could be used in place of the Altera DE2. There is an
accelerometer already built into the DE0 FPGA and it is only 3x2 inches compared to the 8x6
inches DE2. However, it does not have a audio out jack or an SD card port. A USB-bus can be
attached to the USB port for more connections. The DE0 also only has a 2-pin external header
for power which can be subject to a lot of movement, especially if the board is attached directly
to our glove. The DE2 has many more features and functions that could be additions to our
project, or they may be unused. If the latter, the DE0 may be a more appropriate choice.
2. The Terasic Technologies Inc.V.1.6 “DE2_SD_Card_Audio” demonstration project
contains solutions for both the SD card and audio out. The SD card portion can be substituted
with reusable design unit 3. The audio can be substituted with Altera University’s Audio IP
Core. An application note has been written for it here:
https://www.ualberta.ca/~delliott/local/ece492/appnotes/2013w/audio_altera_university_ip_cores
/
7

3. For the SD card, the application note “SD_Card_Interfacing” created by Jason Brown
& Brady Thornton (Group 12) of Winter 2013 can be used as a substitute plan:
https://www.ualberta.ca/~delliott/local/ece492/appnotes/2013w/SD_card_interfacing/
Datasheet
User Perspective Diagram
Image 3: User perspective diagram
User I/O Signals:
All GPIO inputs are driven by +3.3 V and are active low.
Signal
GPIO
Pin
Description
Type
8

GPIO_0[10]
13
Note A tactile sensor on left index tip
Input
GPIO_0[12]
15
Note B tactile sensor on left middle tip
Input
GPIO_0[14]
17
Note C tactile sensor on left ring tip
Input
GPIO_0[16]
19
Note D tactile sensor on left pinky tip
Input
GPIO_0[18]
21
Note E tactile sensor on left index tip below Note A
Input
GPIO_0[20]
23
Note F tactile sensor on left middle tip below Note B
Input
GPIO_0[22]
25
Note G tactile sensor on left ring tip below Note C
Input
GPIO_0[6]
7
Reset tactile sensor on right pinky tip
Input
GPIO_0[4]
5
Mode Switch tactile sensor on right ring tip
Input
GPIO_0[0]
1
Octave Up tactile sensor on right index tip
Input
GPIO_0[2]
3
Octave Down tactile sensor on right middle tip
Input
GPIO_0[11]
14
SDA of accelerometer (communicate with I2C bus)
Input
GPIO_0[13]
16
SCL of accelerometer (communicate with I2C bus)
Output
GPIO_0[15]
18
INT2 of accelerometer (communicate with I2C bus)
Input
SD Card
N/A
Holds .wav files
Input
Line-Out
N/A
Audio out (green)
Output
LCD
N/A
Display the current mode (note/chord)
Output
7-Segment
N/A
SD card sector display (right-most 4)
Output
9

Image 4: GPIO pin schematic
Ribbon Header Mapping
Image 5: Ribbon Header Mapping
10

Accelerometer Interface:
Signal
Description
Type
sda_padoen_o
data enable
in
scl_i2c_to_accel
i2c clock to accelerometer
out
sda_i2c_to_accel
i2c data to accelerometer
inout
scl_i2c_to_bus
i2c clock to i2c bus
in
sda_i2c_to_bus
i2c data to i2c bus
inout
interrupt2
int2 on accelerometer
in
avalon_slave_address
avalon_slave.address
in
avalon_slave_readdata
avalon_slave.readdata
out
avalon_slave_writedata
avalon_slave.writedata
in
avalon_slave_write_n
avalon_slave.write
in
avalon_slave_chipselect
avalon_slave.chipselect
in
avalon_slave_byteenable
avalon_slave.byteenable
in
avalon_slave_read
avalon_slave.read
in
SD Card Interface:
Signal
Description
Type
Data2
Data signal 1
out
Data3
Data signal 2
out
CMD I/O
Input and output command
inout
GND
Supply voltage negative
in
VDD
Supply voltage positive
in
CLK
Clock signal
in
Data0
Data signal 0
out
Data1
Data signal 1
out
11

External Circuitry Schematic:
Image 6: Schematic
Power:
The below values were obtained with a DE2 power measurement wiring harness
connected to the system:
Idle: 0.447A, 9.10V
Peak: 0.457A, 9.10V
Power = I2R = (0.447 to 0.457)29.10 = 1.818 to 1.901 [W]
Background Reading
1. Implementation of SD card music player using Altera DE2-70:
http://ieeexplore.ieee.org/xpl/articleDetails.jsp?tp=&arnumber=5957486&contentType=Confere
nce+Publications&searchField%3DSearch_All%26queryText%3DAltera+de2
This article is dedicated to show the implementation of a music player build based on
Altera DE2-70 board. It shows us very detailed design procedures and requirements. Although it
is built on a different board, we can still get some decent ideas from it. For instance, their project
is using I2C protocol to configure the audio chip working in master mode. The SD card must be
16-bit FAT format system, and for each sector, it has 512 bytes data. Therefore the main
12

program is going to read 512 bytes each time, write the data to the DAC FIFO in the audio
controller, and then enter the next loop. In our design, we should know base/starting address of
each sound file, and how many loops we are going to read for each of them.
2. The Reading/Writing SD Card System Based on FPGA
http://ieeexplore.ieee.org/xpl/articleDetails.jsp?tp=&arnumber=5635615&contentType=Confere
nce+Publications&searchField%3DSearch_All%26queryText%3DSD+card
The information provided in this article explains how to interface the SD card to an
FPGA, specifically the Altera DE2-70 using QuartusII and NiosII. The pins of the SD card are as
follows:
Image7: SD card interface
http://3.bp.blogspot.com/_8JZhVVmpICU/TH_Pxa19MHI/AAAAAAAAApg/pgSppwx0gY8/s1600/SD+card+pinout.jpg
These pins are interfaced in Quartus II as memory-mapped slaves. The article also
discusses the FAT16 file system as well as the commands for the SD card operation:
13

http://dcodelab.files.wordpress.com/2012/11/command.jpg
Furthermore, SD card initialization, data reading, data writing, and data erasing, were
explained. The information was extremely similar to what we are doing for our project (based of
off Terasic’s demo.) and multiple example projects we have found have used the same methods
for SD card communication (1-bit SPI). Most examples only did reading, so the data writing and
data erasing details could possibly be utilized for further functional expansion of our guitar
gloves.
3.
基于
FPGA
的
I_2C
控制器的实现及其在音频解码中的应用
(The application of I2C
controller, based on FPGA, in audio codec perspective)
http://read.pudn.com/downloads143/doc/comm/624483/基于 FPGA 的I_2C 控制器的实现及其
在音频编解码中应用.pdf
In our project, we are playing sound files from the SD card to the audio codec, and the
transmission is realized by I2C bus. This article provides the information about how to realize
communication by using I2C bus in DE2 board. It explains how I2C controller work in detail by
commenting the verilog codes provided by Altera. Thus this article is really helpful for us to
understand the functionality, and performance of I2C bus so that we can embed our
accelerometer into our design.
4. Altera Quartus II Tutorial
14

http://ieeexplore.ieee.org/xpl/articleDetails.jsp?tp=&arnumber=6278363&contentType=Books+
%26+eBooks&searchField%3DSearch_All%26queryText%3DAltera+Quartus+II+Tutorial
This is a tutorial article for Altera Quartus II. It includes several functions of Quartus II.
We found it is very useful under the section “B6. Making Pin Assignment”. It helped us to
complete our application notes, which is about applying internal weak pull-up resistor in Quartus
II.
Software design
The software design is based on Nios II, uC/OS-II. All the signals are coming from the
GPIO on the DE2 board, and interrupt handlers are used to deal with the different signals. In
addition, we need 2 tasks to manage the filtering and sound playing. The following is the task
and interrupt descriptions:
Interrupt name
Description
isr_Rst
It will handle the RESET signal comes from the right pinky tip sensor.
Inside this interrupt handler, the mode, octave flags would be reseted to
default values. Post a message queue, which TaskModeDisplay would
pending, so that TaskModeDisplay can display correct MODE and
OCTAVE LEVEL on the LCD.
isr_Octup
Increasing the octave level by 1. Post a message queue, which
TaskModeDisplay would pending, so that TaskModeDisplay can display
correct MODE and OCTAVE LEVEL on the LCD.
isr_Octdown
Decreasing the octave level by 1. Post a message queue, which
TaskModeDisplay would pending, so that TaskModeDisplay can display
correct MODE and OCTAVE LEVEL on the LCD.
isr_ModeSwitch
In this interrupt handler, it would raise a flag for a specific MODE and post
a message queue, which TaskModeDisplay would pending, so that
TaskModeDisplay can display correct MODE on the LCD.
Table 2: Interrupt description
Task name
Description
task_which_note
This task has task priority 1. Pending on a message queue, which would
be posted once the strumming signal detected by the accelerometer. It
also monitor which note/chord in which octave level the user pressed, if a
strumming signal detected without any note/chord pressing, then do
nothing.
task_sd_play
This task has task priority 4. Its main responsibility is going to receiving
15

all the messages from other tasks and analyze the signals to perform the
next correct move. It is going to monitor the sensors’ signals from the
player’s left hand by polling. There are seven tap sensors/switches on the
left hand glove. We could distinguish all those signals by assigning them
to different pins on the GPIO on DE2 board. It must be able to detect the
“useless” signals from other tasks. For instance, if the player only press
one of the note sensors on left hand glove without any strumming
movement on the right hand glove, SD_Play should never play a note or
chord in this case.
The SD card is formatted as 16-bit FAT, and it has 512 bytes for each
segment of memory.
TaskModeDisplay
This task has task priority 2. It display whatever current mode and octave
level on the LCD.
task_acceleromete
r
This task has task priority 3. This task would keep monitoring the status
of the accelerometer. If it detects the strumming signal, then it would
post to a message queue, which is pending in task_which_note.
Table 3: Task description
BLOCK DIAGRAM:
Image 8: Software block diagram
16

List of Libraries:
Name
Description
"altera_avalon_pio_regs.h"
Provides a hardware interface that allows software to
access the PIO.
"altera_avalon_lcd_16207_regs.h"
"altera_avalon_lcd_16207.h"
Provides a hardware interface that allows software to
access the two (2) internal 8-bit registers in an Optrex
model 16207 (or equivalent) character LCD display (the
kind shipped with the Nios Development Kit, 2 rows x
16 columns).
"basic_io.h"
Basic I/O.
"LCD.h"
User defined functions/macros for LCD. For example,
LCD_Init(); lcd_wirte_cmd(base, data).
"SD_Card.h"
User defined functions/macros for SD Card, and some
commands for SD Card.
The macros contains SD card set I/O directions, output
High/Low, and input read. Functions:
SD_card_init(void)//initialize SD card.
SD_read_lba(BYTE *buff, UINT32 lba, UINT32
seccnt)//reads seccnt number of sector in SD card.
send_cmd(BYTE *in)//sends the commands *in.
Predefined commands:
const int cmd0 [5] = {0x40,0x00,0x00,0x00,0x00}; /*
Reset SD Card */
const int cmd55[5] = {0x77,0x00,0x00,0x00,0x00}; /*
Next CMD is ASC */
const int cmd2 [5] = {0x42,0x00,0x00,0x00,0x00}; /*
Asks to send the CID numbers */
const int cmd3 [5] = {0x43,0x00,0x00,0x00,0x00}; /*
Send RCA */
const int cmd7 [5] = {0x47,0x00,0x00,0x00,0x00}; /*
Select one card, put it into Transfer State */
const int cmd9 [5] = {0x49,0x00,0x00,0x00,0x00}; /*
Ask send CSD */
const int cmd10[5] = {0x4a,0x00,0x00,0x00,0x00}; /*
Ask send CID */
const int cmd16[5] = {0x50,0x00,0x00,0x02,0x00}; /*
Select a block length */
const int cmd17[5] = {0x51,0x00,0x00,0x00,0x00}; /*
Read a single block */
const int acmd6 [5] = {0x46,0x00,0x00,0x00,0x02}; /*
SET BUS WIDTH */
17

const int cmd24[5] = {0x58,0x00,0x00,0x00,0x00}; /*
Write a single block */
const int acmd41[5] = {0x69,0x0f,0xf0,0x00,0x00}; /*
Active Card's ini process */
const int acmd42[5] = {0x6A,0x0f,0xf0,0x00,0x00}; /*
Disable pull up on Dat3 */
const int acmd51[5] = {0x73,0x00,0x00,0x00,0x00}; /*
Read SCR(Configuration Reg) */
note: we are not using all of these commands.
Test plan
Software:
Test
Complete/Results
Detect SD card
Yes
Detect speakers
Yes
Memory leaks after 30 minutes
Yes - No leaks
Polling works/sufficient on A-G signals
Yes - Immediate
response
ISR works on single press signals (chord change, octave up/down,
reset)
Yes
ISR works on accelerometer
Yes
Able to flash project to board
Yes
Table 5: Software test plans
Hardware:
Test
Complete/Results
Notes A-G play
Yes
Chords A-G play
Yes
Increase octave works on all notes
Yes
18