In Part 2 of the series “A Novel IoT-Based System for Targeting Sports,” I will delve into the technical aspects of the system. Specifically, I will cover the system architecture, hardware components, the Android app, and the cloud details.
In Part 1, I provided a brief overview of the capabilities of the system. For more information on the technical solution, please refer to this article.
Let’s start with the general system architecture, which is depicted in the following picture. The system consists of two Inertial Measurement Units (IMUs), a camera, a mobile device, and a cloud server. The architecture of the system can be summarized as follows: during each bowler’s throw, two IMUs collect synchronized motion data, which are transmitted to a mobile device via Bluetooth Low Energy (BLE). The mobile device processes the received data for inference and visualization. After the bowling game is completed, the user can upload the raw motion data to a remote cloud server. The cloud server processes the uploaded data and assesses the skill level using a machine learning model. Users of the proposed system have access to a web dashboard for presenting the bowlers’ performance.
Hardware
There are various commercially available wearable wireless IMU sensors that can be used for monitoring health issues, evaluating performance or detecting daily human motion activities. To select the wearable sensors for the proposed system, we considered factors such as increased autonomy and the capability of synchronized streaming from multiple sensors. Ultimately, we chose the widely used MetaMotion R IMU sensor that operates on Bluetooth Low Energy technology.
This sensor can be used to accurately measure three-axis acceleration or quaternion, and stream real-time data through Bluetooth 5.0 to a mobile device. The acceleration data are obtained by the internal BMI160 Accelerometer module with a frequency of 50Hz. The board also offers a firmware-enabled 9-axis sensor fusion which fuses measurements of accelerometer and gyroscope sensors (IMUPlus mode). Using this module we obtained quaternions with a frequency of 50Hz.
Android App
The Android application is responsible for configuring the two IMU sensors, collecting synchronized data, communicating with the cloud server, assessing the quality of a throw, identifying common bowling errors, and providing visual feedback to the bowlers. To develop the application, we utilized the MetaWear API library, which enables developers to interact with the MetaMotion sensors on Android. The following imagee show the main interface of the app.
In a typical use case, the user starts by connecting their mobile device to the sensors via Bluetooth. Then initialize a baseline, which serves as a reference point for measuring arm swing angle, by keeping their wrists straight and parallel to the horizontal plane. After this, the user can start capturing and analyzing their motion data by clicking a single button in the application. The processed information are stored locally on the device. At the end of the session, the user can choose to upload the data to the cloud server for further analysis and sharing. To provide real-time feedback and address the issue of limited internet access, the proposed application has been designed with offline capabilities. All calculations related to throw quality assessment and error detection are performed on the device by the data processing algorithms (I’m going to talk about the algorithms in a future post).
Wrapping up
In this post, I provided an overview of the system by discussing its basic architecture, as well as the Android and hardware components that make it work. In a future post, we will dive a bit deeper into our cloud solution. Specifically, I will explain how we preprocess our data in the cloud using Beam, how we use GCP Machine Learning services for inference, and how we store sensor data in a timeseries database like InfluxDB. Finally, I will provide a detailed description of our skill assessment classification model, which plays a crucial role in evaluating the performance of bowlers using our system. I will explain how we trained the SVM model and what features we used to represent the bowlers’ motion data.