1. FYP CA4 Presentation Wearable Micro-Sensor Platform and Processing System Project Supervisor: Prof Lawrence Wong Prof Wu JianKang Student: Yang SiShuo
4. Background A part of Interactive Virtual Tennis Game A hardware platform that provide motion data to the game application Embedded High performance Wearable & Compact Wireless 4
12. 3-Axis Accelerometer Digital Compass Bluetooth module Z-Axis Gyroscope XY-Axis Gyroscope Version 2 Design 12 Voltage Step-up Converter EEPROM Digital Signal Controller Primary Oscillator Secondary Oscillator
15. Firmware layer Hardware layer Firmware Structure 15 Application Layer Init RTOS Init Bluetooth Sensors Init peripheral module Data Acquisition Bluetooth UART LMX9838 Bluetooth Serial Module RS232 Data processing Link control Interrupt Signal Signal FSM Buffer Process frame
16. Data Exchange Protocol To facilitate data decoding at PC side To allow simple error detection 16
17. Command Exchange Protocol To facilitate coordination between the sensor platform and the PC Configuration of sampling rate from PC Similar to data exchange protocol 17
19. Device Driver Overview A class library written in C++, compiled into a single Dynamic Linked Library Can be referenced by any Microsoft .NET Programming Language 19
21. Why do we need driver? Hide part of technical details from user Access all functionalities through APIs Facilitate system integration No nasty code merging Simplify maintenance Just replace with new driver library 21
22. Data flow diagram of driver program 22 0xFD 0x08 0x23…. Micro-Sensor Platform Bluetooth Stack Bluetooth Virtual Serial Port Driver Device ID ACC X ACC Y …. GYRO Z Conversion Bytes Internal Buffer FSM Statistic info User Application API
23.
24. Bluetooth to PC @ 1-3 Mbits/sNot the rate at the application layer 23
25. Data transfer rate Data rate at application layer is estimated at runtime Count the number of bytes/packets received at some fixed interval Measure the time interval using timer Transfer rate=# of packet/Δtime Only a rough estimation Timer is not accurate 24
30. Conclusion During the course of the project: Hardware design & fabrication Development of firmware Development of Device Driver & Sensor Manager Application 29
31. A few working prototype produced The MSP & Processing System proven to be: Effective platform for motion data acquisition and processing Complete system solution for projects/applications that require interactive interface Merit award in 24th FOE Innovation & Research Competition The project is quite successful Conclusion 30
33. Thank you Questions? Project Supervisor: Prof Lawrence Wong Prof Wu JianKang Student: Yang SiShuo
Editor's Notes
The MSP and p system is designed for any application that requires interactive interface between human and machine. A lot of innovative application requires such a system. One example is the interactive game, in which the game is controlled by the motion and action of human body. Together with other FYP students, we have built a virtual tennis game based on the sensor platform.So the purpose of MSP is to provide motion data to the game application and other sub-systems. Considering the application of interactive tennis game, there are some general design requirements must be followed. For example, as the motion data need to be sampled and processed at high rate, the MSP should possess enough computing power. In addition, since the MSP will be attached to user’s hand, it should be wearable and wireless.
Here is a structure of our interactive game. The game application is at top level, supported by game engine and a interface. the data processing sub-system provides the result of gesture recognition to the game engine so that corresponding activities will be displayed. The controller sub-system is responsible for the coordination among different subsystems. The bottom level of this hierarchy is where MSP located, the sensor hardware provides the motion data to PC and the software system allows other module to access all functionality of the MSP. They will be discussed in detail in later of presentation.
The structure of the whole project is illustrated using the following diagram. It can be roughly divided into three layers. The top layer is the app layer running on the remote PC, formed by user’s app, device driver and Bluetooth stack. The Bluetooth stack can be roughly said as a set of program implemented by OS to provide Bluetooth related services. The middle layer is the firmware of MSP. It consist of data acquisition and processing algorithms with a RTOS. And they run on the actual hardware platform, which is the third layer, hardware layer.
Now let’s move to the design of hw.
The hardware consists of four modules. They are CPU, sensor module, communication module and power modules respectively. The CPU I used is a DSC from microchip. It has powerful computing capability yet with scalable clock frequency, which means clock frequency can be changed on the fly. Therefore, it is easy to balance between performance and power consumption. It also has enough hardware peripherals to meet different requirement. The data acquisition module consists of four sensors. One accelerometer, two gyroscopes and one digital compass.Communication between MSP and PC is available in both wired and wireless manner.Last but not least, the power module supplies voltage to all other ICs. As I have designed two versions of hw, the power supply module is slight different.
The block diagram of the hw is illustrated in this slide. The acc and gyro are directly connected to the input of ADC of the DSC. Their output voltage is converted into digital data. The digital compass communicate with DSC using I2C protocol. The motion data is then stored into a internal buffer. The data transfer between ADC and memory is automatically done by the Direct Memory Access module. Wireless connection can be initiated by either MSP or remote PC. In either case, the BT module is controlled by DSC using RS232.
During the course of the project, two versions of hardware was designed and fabricated. Version 1 is the first prototype system. # Only very basic functionalities are provided. # Here is the actual hardware. #Therefore, a slightly better version was designed.
There are a few changes in the version 2 design. First, the size of the hardware is reduced by 37%. Second, the CPU package is changed to a one with more IO interface. Third, an external EEPROM is introduced to store some configuration parameters after power is turned off. This is the actual hardware. The sensor modules and Bluetooth modules are mounted on top side. This is the bottom side view. The voltage step-up converter provides 3v output voltage from a single AAA battery. This is the EEPROM. The big IC here is DSC. And this is the primary oscillator. The one at the bottom is the secondary oscillator that provides much slower clock frequency in power saving mode. That’s pretty much all about the hardware design.
Now let’s move to the design of firmware.
Firmware is like the brain of any hardware system. It controls every aspect of MSP. The firmware of the MSP consists both C and assembly code, which total contributes more than 7000 LOC and it is the most crucial part in the whole project.
The simplified program flow is explained in this slide. The initialization routines are called when the device is first powered on. The RTOS and peripheral modules are initialized first. So that UART module can be used to communicate with BT. Then the BT can be initialized thereafter. The data from UART module is handled by INT and stored in a buffer. After all initializations are executed, depending on the mode of connection used, the firmware either control BT to initiate connection request to PC or just waiting for an incoming connection request. In the waiting for connection mode, only process frame task is scheduled to run, which continuously poll data from the buffer and interpret the reply from BT. Suppose that a connection is now initiated at PC, this incoming connection request will be handled BT automatically. Corresponding responses are also generated by the BT to reflect the status of link establishment. These data is processed by the process frame task, which also send signals to FSM. IF the connection is initiated by MSP, then link control task is activated to go through the different steps to establish a connection. Once the connection is established, data processing task will be activated to acquire data from sensor module and process it.
The motion data captured at sensor platform is transmitted to the remote PC using Bluetooth. To facilitate the data decoding at the PC side, a simple protocol is defined. The first byte STX1 and the last byte ETX1, shown in the orange color, are constant value used to identify the starting and ending point of the packet. The second byte is the device ID which allows the remote PC to identify the source of motion data. The second last byte is check-sum used to detect transmission error. With the data exchange protocol, the data transmission is more reliable and flexible.
In addition to motion data exchange, the configuration and control commands are also exchanged between the MSP and the remote PC. These information is mixed with motion data when transmitted over the Bluetooth. Therefore, another protocol is defined to differentiate these control commands from the data stream and also to facilitate the coordination between MSP and the PC. It has a similar structure as compared to the previous one.
Next section is the design of device driver.
The device driver is a class library written in Visual C++ and compiled into a single dynamic linked library. It can be referenced by any MS.NET programming language.
It provides data processing and sensor management functionalities to user application. For data processing, its task including the deserialization and conversion of motion data. For management part, its primary goal is to maintain statistical information of the wireless connection and the control of remote MSP.
You may wonder why an extra device driver is introduced between MSP and user’s application. The purpose is to hide part of the technical details from the end user so that the user is able to access all functionalities of the device driver without knowing the details of the MSP. It also simplified the system integration. When the MSP is integrated into our tennis game, there is no nasty code merging. Last, system maintenance is also simplified with the use of driver. Any changes in the library does not require any modification in the source code of user application. So, with some understanding of the device driver. Let’s look at how does it work.
When the MSP transmit motion data to the PC side, the packet format defined in the previous slide is actually broken into bytes. In computer science, the process of converting a data structure into bytes or bits is called serialization. So the data is transmitted byte by byte, rather then a packet by packet. At PC side, a virtual serial port is created by Bluetooth stack. The bytes received from this VSP are processed by a FSM in the device driver. This FSM try to assemble the packet from individual bytes. This process is so called deserialization. After a packet is assembled, info in each field can be taken out easily. Conversion is performed right after deserialization, which tries to convert the raw sensor data into a value with physical meaning. Meanwhile, statistic info is maintained by the device driver to keep track of the numbers of packet received, the actual throughput of transmission etc. The converted data is placed at a buffer and can be accessed by user app via APIs.
So you may wonder what is the data transfer rate of the MSP. Well, the transfer rate is very hard to calculate due to following reasons. First, there several different transfer rate at different system level. For example, CPU communicate with the Bluetooth module at 115200bps whereas the speed from Bluetooth to the remote PC ranges from 1 to 3 Mbits/s depending on the version of Bluetooth protocol. Second, these value only implies the transfer rate at very bottom level. Not quite meaningful to top level application.
So the simplest way is to estimate transfer rate at runtime by counting the number of bytes and packets received at some fixed interval. The time interval is measured using software timer. Then the transfer rate can be calculated. This calculation is automatically performed by the device driver. The user application can access these statistical information via APIs. However, this is only a rough estimation because of the accuracy of the software timer.
The last topic in the application layer is the development of sensor manager application.
Sensor manager is a user-friendly sensor management software written in VB.net. By utilizing the device driver, it provides four services, data visualization, data logging, sensor node management and sensor node configuration. Demo
This is the last section of this presentation.
During the course of the FYP, I have successfully completed hardware design, development of firmware, development of the device driver and the sensor manager application.
A few working prototypes are also produced. In addition, the MSP is also successfully integrated into our interactive virtual tennis game. The MSP is proven to be an effective complete solution for any applications that requires interactive interface between human and machine. This project is also one of merit award winner in the Faculty’s 24th Innovation & Research Competition. Therefore, I would like to conclude that the project is quite successful.
