# Usable Security Group

> IIIT Delhi · Research Associate

- **Role:** Research Associate
- **Company:** IIIT Delhi · Usable Security Group
- **Period:** Jan 2022 – Jun 2024

With [Prof. Arun Balaji Buduru](https://faculty.iiitd.ac.in/~arunb/) in the **Usable Security Group**, I built computer vision and audio systems where outputs had to help people, not just score well on benchmarks.

[Crowd scene and predicted density map](https://yashwardhan.space/assets/videos/csrnet_real_readable.mp4)

## Problem

The [Usable Security Group](https://faculty.iiitd.ac.in/~arunb/) at IIIT Delhi builds **user-centric security solutions**: systems operators can actually use for public safety, health, and ecology. I joined as a **Research Associate** while still an undergrad: PyTorch pipelines, experiment design, grant writing, ablation studies, and conference papers under Prof. Arun Balaji Buduru, with **Adel Alshamrani** (University of Jeddah) co-advising the FGA crowd-counting line.

**Take home:** Four publications from the lab: **IJCNN 2024**, **arXiv** lightweight fusion, and two **Interspeech 2024** audio papers, plus a side **.PY Security** CCTV prototype with usability testing.

![Crowd scene and predicted density map](https://yashwardhan.space/assets/images/work/usable-security/fga-crowd-density.png)

*Density-map crowd counting, input scene and predicted heatmap (FGA paper, Fig. 1)*

## What I did

### FGA, Fourier-Guided Attention (IJCNN 2024)

- Crowd density maps need **full-scale global patterns** (wide-area crowding) and **local head-level cues**. CNN-only spatial attention under-captures the global half, FGA fixes that with a **dual-path plug-in** for the last layers of CSRNet and CANNet.
- **Global path:** 2D FFT on feature maps → frequency-domain processing → spatial attention. **Local path:** dilated convolutions + **channel-wise attention** on the remaining maps. Outputs are fused before the density regression head.
- Benchmarks (MAE): CSRNet+FGA **8.4** on ShanghaiTech-B (baseline **10.6**); CANNet+FGA **6.9** (baseline **7.8**). Evaluated on ShanghaiTech-A/B, UCF-CC-50, and JHU++ with consistent MSE gains.
- I built the **PyTorch training and data-loading pipeline**, ran architecture ablations, **Grad-CAM** qualitative analysis for interpretability, helped secure grant funding, and co-authored the paper with **Adel Alshamrani** (University of Jeddah).

![FGA module architecture diagram](https://yashwardhan.space/assets/images/work/usable-security/fga-module.png)

*Dual-path FGA module, FFT global branch + conv/attention local branch (IJCNN 2024, Fig. 2)*

### Lightweight Feature Fusion (arXiv 2401.05968)

- Aerial and edge deployment cannot carry VGG/ResNet-scale backbones. We kept a shared downstream counting head and swapped in **MobileNet** (**ASFNet-S**) and **MobileViT** (**ASFNet-B**) as pre-trained backbones.
- **Adjacent Feature Fusion (AFF):** multi-scale feature maps from the PTM are fused across neighbouring scales before regression, same head, far fewer parameters and FLOPs than heavy SOTA.
- On ShanghaiTech-A/B and UCF-CC-50, ASFNet variants hit **comparable MAE/MSE to heavier models** while staying the most compute-efficient in our comparison table. Ran **pruning** and full **ablation** sweeps for the aerial counting use case.

![ASFNet adjacent feature fusion pipeline](https://yashwardhan.space/assets/images/work/usable-security/asfnet-fusion.png)

*Adjacent Feature Fusion across MobileNet / MobileViT scales (arXiv 2401.05968, Fig. 1)*

### ASGIR, Interspeech 2024

- **Problem:** ecologists need bird ID from field recordings plus habitat context for field use.
- **Model:** **Audio Spectrogram Transformer (AST)** embeddings fed to an **SVM** for species classification on [Xeno-Canto](https://xeno-canto.org/) vocalizations.
- **UI flow (two steps):** (1) record or upload audio → classify species; (2) enter location → **location-filtered Wikipedia scrape** for habitat and behaviour context on the predicted bird.
- European subset, **51 classes**: median **100%** precision, recall, and F1 on held-out evaluation. Shipped as a usable prototype.

### VoxMed, Interspeech 2024

- **Problem:** frontline clinicians need a fast respiratory screen from stethoscope audio in one screen.
- **Model:** **AST** feature extractor + **1-D CNN** classifier on the [ICBHI](https://bhichallenge.med.auth.gr/ICBHI_2017_Challenge) lung-sound corpus (recordings from Greece and Portugal).
- **UI:** one-step upload → Healthy / COPD / other respiratory conditions in seconds. **90%** accuracy on the three-class task.
- Ablation vs. **Wav2Vec2**, **UniSpeech**, and **WavLM**: AST won on this dataset. Built with the lab as a UI-assisted screening prototype.

### .PY Security (side project)

- Side exploration with the lab: a **.PY Security** prototype for **real-time violence detection** on smart-city CCTV, motivated by limits of passive camera monitoring during mass incidents.
- **SCVD** (Smart-City CCTV Violence Detection) benchmark for training; **VGG16** frame encoder + **GRU** temporal head, **BCE loss** on violence labels. Parallel **crowd-density branch** (dilated conv + spatial/channel attention, **MSE loss**) for disobedience planning.
- **Operator UI:** simulated login → **Delhi map** with dummy camera pins; feeds turn **green** (peaceful) or **red** (violence flag) with **per-camera probability**. **Density view** overlays crowd estimates so planners can allocate resources during mass disobedience.
- Usability sessions with security professionals and non-technical users: **density view rated most useful** for crowd-control scenarios; map interface judged intuitive, but **multi-step flows** (login → map → feed → density) needed trimming for field use.

![.PY Security violence alert dashboard with density map](https://yashwardhan.space/assets/images/work/usable-security/pysec-violence-dashboard.png)

*Violence probability per CCTV feed plus density view for crowd-control planning (IIIT Delhi, 2023)*

## Outcomes

Left the lab with four submitted papers, production-minded PyTorch pipelines, and grant-funded project experience before graduation. Crowd-counting work continued separately through a **Chanakya fellowship at IIT Patna** for aerial embedded deployment.

- **4** papers
- **IJCNN** FGA · 2024
- **2×** Interspeech · 2024

## Team

- [Arun Balaji Buduru](https://faculty.iiitd.ac.in/~arunb/) (Advisor · Founding Head · Usable Security Group)
- Adel Alshamrani (Co-advisor · University of Jeddah · FGA)

## Impact

Research, density-map counting with ONNX export path for aerial footage; ASGIR and VoxMed as UI-assisted audio prototypes.

- [FGA: Fourier-Guided Attention Network](https://arxiv.org/abs/2407.06110), IJCNN 2024
- [Lightweight Feature Fusion](https://arxiv.org/abs/2401.05968) for resource-constrained counting
- [ASGIR](https://arxiv.org/abs/2407.18927) and [VoxMed](https://arxiv.org/abs/2407.18926), Interspeech 2024

## Sources

- [USG · Prof. Arun Balaji Buduru](https://faculty.iiitd.ac.in/~arunb/)
- [FGA, IJCNN 2024](https://arxiv.org/abs/2407.06110)
- [Lightweight Feature Fusion](https://arxiv.org/abs/2401.05968)
- [ASGIR, Interspeech 2024](https://arxiv.org/abs/2407.18927)
- [VoxMed, Interspeech 2024](https://arxiv.org/abs/2407.18926)
- [ASGIR code](https://github.com/MainSample1234/AS-GIR)
- [VoxMed code](https://github.com/Sample-User131001/VoxMed)

---

[Back to portfolio](https://yashwardhan.space/) · [HTML case study](https://yashwardhan.space/work.html?p=usable-security)