Data Acquisition - Now and Towards the Future
TT
Tim Thacker • September 22, 2021
Tim Thacker • September 22, 2021
Full data system architecture
Graph of code commits from 2017 to 2021
Data logger firmware module based architecture
Data Acquisition collects all the data from sensors on our cars and feeds it back to other departments and engineers in the team to make the cars as fast and efficient as possible. The team includes six sub-department teams:
DAQ displays all the data from the sensors upon the car and live-streams it to our telemetry server. From the server - it is collated into values, graphs, live values and charts etc. from these sensors for other Redback departments to understand and use. Data is also logged for review at later dates. Simulation is also an important part of DAQ, including optimal lap simulators and driving simulators that allow the team to live-stream and test the car without running it.
Over the last few years, Redback Racing has been developing an end to end data acquisition, visualisation, simulation and web platform called Redback Cloud. The overarching goal of a motorsport telemetry system is to provide the tools for engineers to improve the performance of their cars. The key points we have been focusing on to achieve this are to:
The diagram above shows a high-level overview of the different parts of the system. The blue components are the projects which collect data from sensors and simulations, sending the data back to Redback Cloud - represented by the red components - either directly or via a proxy server at the track. The yellow components represent general-purpose computers with the Redback Cloud web portal to visualise live telemetry, perform lap simulations, search and analyse data etc.
The data sources are the entry point to the system, collecting data from sensors, inputs and simulators. The primary data sources are the onboard data logging firmware and weather station, which collect data at track days and send it to Redback Cloud.
As Redback could not participate in FSAE-A 2020, we had many spare hands to help develop supporting systems. We integrated these systems included the driving simulator and lap simulator, as data sources to Redback Cloud.
Moving forward, we can easily extend Redback Cloud to integrate multiple data sources or systems, including more simulators and sensors.
At the start of 2017, Redback started a firmware project called Yuria to collect, process, display, stream and log the data from all the sensors and systems onboard the vehicles.
This is currently our oldest software project at Redback but has seen an acceleration in development over 2020 and the start of 2021, as seen in the commit log.
The firmware aggregates data from multiple inputs onboard the car, primarily the CAN bus which is populated with data from analogue and digital sensors plugged into the ECU as well as CAN-enabled sensors such as our IMU.
Going forward, DAQ can work collaboratively with the electrics team by looking to decentralise the analogue to digital conversion away from the ECU. This is by having CAN nodes throughout the car which will significantly reduce the number of wires in the loom. Other inputs to the firmware are the GPS board which sends data over a serial port using the NMEA protocol - the standard for GPS - and the labjack which is a data acquisition unit with a series of analogue and digital inputs.
The firmware is structured using the concept of modules which each run in parallel in their own thread. Modules are broken up into inputs, middleware and outputs, as shown in the diagram below. The threads pass messages to each other asynchronously through 2 data structures. The first being the Event Bus for general control messages like ‘start streaming’ or ‘start logging'. The second is queues for streaming sensor data through the pipeline from input modules to output modules. This method of passing messages decouples modules from each other, allowing the system to be easily extendable to more use cases by inserting new modules.
All modules and sensors connected to the system can be fully configured in a JSON config file. For example, all sensors on the CAN bus can be defined in terms of their CAN ID, specific bit offset and length, endianness, data type, decimal point position. This means our firmware can be configured for any other car, not just Redback cars.
In 2021, we are further integrating the firmware with Redback Cloud by creating a user interface to configure all modules and sensors and automatically sync with the firmware on the car using the WebSocket connection. We are also working on being able to plugin more data processing to the middleware modules for use cases such as sensor fusion so that the firmware can potentially operate as a ROS node in the autonomous vehicle system.
The weather station is a project deployed trackside and collects an array of data about the current weather conditions at the track, including:
This data is streamed to the trackside proxy server and then to Redback Cloud, allowing all our engineers to take the weather conditions into account when drawing information from the data. All of the weather station data is also logged with the sensor data from the car, permitting us to compare data across different track days.
In 2021, we started a new project to involve data science students between data acquisition and vehicle dynamics. Machine learning models take data from the lap simulator, driving simulator and vehicle to predict potential setup and future design improvements. It is not clear yet how these technologies we are developing will tie into the whole system and whether or not they can be offered as a service to other teams on Redback Cloud someday.
Redback Cloud Accounts is a key feature where other teams can manage all their vehicles, users and roles within the system. Essentially each garage will belong to either a team account or a personal account, and if you have a team account, you can manage the permissions for all of the people in their team. This feature will turn Redback Cloud into a true enterprise software as a service (SaaS) platform for anyone to use.
Another significant feature in the pipeline for Redback Cloud is part tracking, keeping a record of the amount of use for each part on the car. Like the car setup sheets, we can select which parts are used on the car for each log file and generate computer estimates and predictions for when parts will reach their end of life.
Redback is working towards all of our frontier software open source (data logging firmware, Assetto Corsa telemetry app, point mass simulator, weather station etc). By the end of 2021, our goal is to develop Redback Cloud to a stage where other FSAE teams or the general public can create an account and start to use our systems for their cars or sim racing using our open-source data tools.