Recently, using WiFi access points (APs) as sensor nodes by passively analyzing WiFi signals, particularly the Channel State Information (CSI), has attracted a lot of interest. This emerging technology is currently progressing towards standardization under IEEE 802.11bf, which is expected to be finalized around 2025. Of particular interest is the detection and identification of humans, their activities, motion, gestures, and other relevant information.

Motivated by previous successes in vision and language processing domains, current trends indicate a shift from classical signal processing approaches towards deep learning based methods. However, these learning based methods rely on the availability of suitable training datasets and might cause an increased demand for memory and computational resources due to growing model size and complexity.

Depending on the specific application, various considerations come into play. Latency becomes crucial for real-time applications such as motion and gesture recognition, where timely responses are required. Throughput becomes a concern when scaling the system to a large number of nodes, antennas, or high sample rates. Resource and energy efficiency are particularly important at the edge, where power consumption and computational resources are limited.

Using FPGAs allows for efficient deployment of such deep learning models by building custom hardware accelerators to match the latency, throughput and resource constraints of the application by leveraging model compression techniques and the parallelization potential of the FPGA accelerator. To the best of our knowledge, no such FPGA-based DNN accelerator for wireless human sensing has been employed so far.

The overall goal of this project is to explore the application of lightweight, FPGA-accelerated DNNs in the domain of wireless human sensing. The results will demonstrate how the DNN/FPGA co-design approach can benefit existing human sensing use cases or enable new application scenarios with the expected advantages in performance and energy-efficiency. The outcome will not only guide future research, but also include a working prototype implementation that serves as a practical technology demonstrator, which could be used in teaching or further student projects.

The project is of interdisciplinary nature and covers the complete stack from DNN training to the deployment of the final system. After an initial research phase into current literature around WiFi-based human sensing, students will analyze and select suitable use cases, datasets, and DNN topologies. Further, the group will work with experimental tooling (FINN) from AMD/Xilinx Research to train efficient DNNs via quantization-aware training and to generate custom-tailored FPGA inference accelerators for these DNNs. As part of the introduction to these tools, students will participate in a two-day workshop hosted in October as part of the EKI research project, which deals with the improvement and automation of such DNN-to-FPGA mapping tools in a data-center context.

The evaluation of the developed models and accelerators will initially focus on open-source datasets and report metrics such as classification/regression accuracy, throughput, latency, power consumption, and hardware cost in terms of FPGA resources. Depending on the results and challenges encountered during this phase of the project, the group selects at least one use case for a real-world prototype implementation, which might involve the generation of own datasets for fine-tuning. The hardware implementation for these experiments can rely on embedded FPGA boards and commodity WiFi access points. If desired, students could also implement the measurement infrastructure via a custom software-defined radio (SDR) solution based on an AMD/Xilinx RFSoC platform or move the DNN acceleration from edge to cloud by utilizing the large FPGA cards found in the PC² Noctua 2 cluster.