毫米波雷达成像论文阅读笔记： IEEE TPAMI 2023 | CoIR: Compressive Implicit Radar

原始笔记链接https://mp.weixin.qq.com/s?__biz=Mzg4MjgxMjgyMg==&mid=2247486680&idx=1&sn=edf41d4f95395d7294bc958ea68d3a68&chksm=cf51be21f826373790bc6d79bcea6eb2cb3d09bb1860bba0af0fd5e60c448ca006976503e460#rd
$\uparrow$点击上述链接即可阅读全文

IEEE TPAMI 2023 | CoIR: Compressive Implicit Radar

毫米波雷达成像论文阅读笔记 IEEE TPAMI 2023 | CoIR: Compressive Implicit Radar

Abstract

• 背景

  • mmWave radars suffer from low angular resolution due to small apertures and conventional signal processing
  • 稀疏阵列雷达 can increase aperture size while minimizing power consumption and readout bandwidth

• 方法 提出 Compressive Implicit Radar (CoIR)

  • 目标 high accuracy sparse radar imaging using a single radar chip

  • Leverages : CNN decoder and compressed sensing

  • 贡献

    ✅ 设计稀疏线阵 with 5.5x fewer antennas than conventional MIMO arrays

    ✅ 提出ComDecoder a fully convolutional implicit neural network architecture

    ✅ 证明了CoIR的有效性 in both simulation and real-world experiments且 不需要 auxiliary sensors

• 实验结果

  • improved performance over standard mmWave radars and other untrained methods on simulated and real data
  • System does not require training data or auxiliary sensors
    

1 INTRODUCTION

基于光学的Depth imaging及其缺点

• Depth imaging
  • crucial for applications like SLAM, autonomous driving, security monitoring
• Typical sensors: cameras, LiDAR
  • Cameras: high-resolution near-field depth imaging
  • LiDAR: directly outputs dense point cloud with high range/angular resolution
• Limitation : degraded performance in visually degraded environments like fog, smoke

毫米波雷达成像的优点和挑战

• 优点
  • penetrate through fog/smoke without performance degradation
• 挑战
  • low angular resolution $\delta \approx \lambda /d$
  • Increasing $d$ increases cost, power consumption and readout bandwidth

已有提高角分辨率的工作和缺陷

• 已有思路
  • Large physical arrays
  • MIMO arrays
  • SAR
  • Sensor fusion
  • Optimization with handcrafted priors
  • Deep learning
• 不足
  • Slow acquisition
  • Increased hardware complexity
  • Require large datasets
  • Limited generalizability

The proposed CoIR:

• Key observation:

  • INR provides inductive bias towards natural solutions for imaging inverse problems

• 方法

  • Leverages implicit neural representations (INRs) + compressed sensing

• 贡献

  • Designed sparse linear array with 5.5x fewer antennas
  • Proposed convolutional decoder architecture ComDecoder
  • Demonstrated improved performance over standard mmWave radars and competitive untrained methods

2 RELATED WORK

2.1 mmWave Imaging Systems

• 提高角度分辨率的方法及其缺点

  • Large physical arrays expensive, large data volumes
  • MIMO arrays requires many radar chips to synthesize large aperture
  • SAR techniquesslow imaging, bulky scanners
  • Sensor fusion: fails if one modality fails
  • Deep learning: requires large labeled datasets, limited generalizability

• proposed CoIR 的不同:

  • 仅使用 single chip sparse MIMO array

  • 使用 未经训练 的 神经网络

    ✅ 无需训练数据

2.2 Sparse Radar Imaging

• 稀疏雷达成像技术

  • 1 Sparse aperture array designs
  • 2 Sparse reconstruction methods

• 1 Sparse aperture array designs

  • 使用欠奈奎斯特采样 减少 天线数

  • 优化方法

    ✅ Convex relaxations

    ✅ Prior knowledge of number of reflectors

• 2 Sparse reconstruction methods

  • Super-resolution algorithms

    ✅ MUSIC, ESPRIT

    ✅ Require incoherent signals, known number of targets

  • Compressed sensing (CS) optimization:

    ✅ 使用稀疏先验如 spatial sparsity, TV norm

    ✅ Challenging to design priors, scene dependent

• proposed CoIR 的不同:

  • Sparse array design

    inspired by prior work but modified due to hardware constraints

  • Uses untrained neural network

    as complex prior instead of handcrafted prior

    ✅ Neural network prior shows affinity for natural features and noise robustness

2.3 Implicit Neural Representations

• 两类INR architectures:

  • 1 Convolutional methods
  • 2 Coordinate-based MLP methods

• 1 Convolutional methods 适合

  • Compressed sensing
  • Image super-resolution
  • Image denoising
  • Accelerated MRI

• 2 Coordinate-based MLP methods 适合

  • Novel view synthesis
  • Dynamic illumination
  • PDE solutions
  • Image deconvolution

• CoIR中的ComDecoder

  • 属于 Convolutional methods

  • tailored for sparse radar imaging

  • Key properties

    Convolutional operations capture local spatial information

    Upsampling induces notion of resolution per layer

    Residual blocks smooth optimization and propagate information between layers

    Together these inductive biases improve performance on sparse radar imaging

  • Differences from prior works:

    ✅ CoIR uses untrained INR as complex prior for sparse radar imaging

    ✅ Prior works use INR for natural images or other imaging modalities

3 RADAR IMAGING BACKGROUND

• 发射信号模型

  • $y_{tx}(t)=e^{j2\pi (f_{0}t+\frac{1}{2}B\tau t^2)},0\le t\le T$
  • $f_0$: carrier frequency
  • $B$: chirp bandwidth
  • $T$: pulse duration

• 场景模型 离散反射体分布

  • $\overline{x}[n_r,n_{\theta }]\in {\mathbb{C}}^{K\times L}$
  • $n_r$: range bin index
  • $n_{\theta }$: angle bin index

• 回波信号模型

  • $z[n,m]={\sum }_{n_r=0}^{K-1}{\sum }_{n_{\theta }=0}^{L-1}\overline{x}[n_r,n_{\theta }]e^{j2\pi {\psi }_{\theta }(n_{\theta })m}{e}^{j2\pi {\psi }_r(n_r)n}+w[n,m]$
  • ${\psi }_{\theta }(n_{\theta })=\frac{f_{0}d}{c}\sin (b_{\theta }[n_{\theta }])$: spatial frequency
  • ${\psi }_r(n_r)=\frac{B}{N}\frac{2b_r[n_r]}{c}$: normalized temporal frequency
  • $w[n,m]$: noise

• Compact matrix form

  • $z=F(\overline{x})+w$

  • $F$: 2D FFT

  • Goal: recover $\overline{x}$ from under-sampled measurements $z$

4 PROPOSED METHOD

• 目标 :

  • Recover scene reflectivity $\overline{x}$ from under-sampled measurements $z$

• Measurements :

  • $z=M\odot F(\overline{x})+w$

  • $M$: binary mask implementing under-sampling

  • $w$: noise

• 困难 :

  • under-sampling causes grating lobes in sparse array PSF leading to aliasing in image

• 解决方法

  • Optimize weights of untrained deep CNN $G(C;p)$ to solve inverse problem

    ✅ $G$: untrained CNN,

    ✅ $C$: fixed noise input,

    ✅ $p$: CNN parameters

  • Optimization objective:

    $\hat{p}={\mathrm{argmin}}_p||z-M\odot F(G(C;p))|{|}_2+{\lambda }_L||G(C;p)|{|}_1$

    ${\lambda }_L$: sparsity regularization strength

  • Key observation:

    INR provides inductive bias towards natural solutions for imaging inverse problems

• 优点 :

  • CNN architecture has high impedance to noise

  • Learned solution balances fitting salient features and suppressing artifacts

4.1 Sparse Aperture Design

• 目标
  • Design a sparse MIMO virtual array that improves imaging accuracy when used with ComDecoder
• 设计准测 (metrics)
  • PSF main lobe half-power beamwidth (HPBW)
  • Peak sidelobe level (SLL)
  • Presence of grating lobes
• 硬件约束
  • Max aperture 86λ/2
  • Limited to 4 TX and 4 RX due to commercial single radar chip
• 设计方法
  • Select 4-element minimum redundancy array for RX to avoid grating lobes
  • Grid search over TX positions to minimize SLL
• 比较对象baselines
  • Full: Ideal full Nyquist sampled array
  • Sub-apt: Largest Nyquist sampled MIMO array given constraints
  • Sub-samp: Largest sub-Nyquist array given constraints
• 设计结果
  • RX: [0, 1, 4, 6] λ/2
  • TX: [0, 46, 59, 79] λ/2
  • Gives 5.5x fewer antennas than conventional MIMO array

• 优点 :
  • Avoids grating lobes
  • Minimizes HPBW
  • Minimizes SLL
  • Satisfies hardware constraints

4.2 Neural Network Architecture

提出 ComDecoderconvolutional decoder architecture

• ComDecoder :

  • Maps latent variables C to image G(C;p)
  • 优化Parameters p optimized to reconstruct image

• 网络结构 :

  • Series of upsampling and residual convolution blocks
  • Use SiLU activation instead of ReLU
  • No upsampling in last layer, uses 1x1 conv instead

• 超参数 :

  • 6 layers (including last layer)
  • 128 channels per layer
  • Fixed input C sampled from uniform distribution

• 优化过程 :

  • Update network weights p using backpropagation and Adam
  • Takes <50 s per 256x256 image using 2000 iterations

• 优点 :

  • SiLU increased expressivity over ReLU
  • Upsampling reinforces multi-resolution nature
  • Residual blocks enable information flow between layers
  • Inductive biases improve performance on sparse radar imaging

5 COMPETING UNTRAINED METHODS

7个baselines: Compared CoIR against several untrained methods

• 1 Delay-and-Sum (DAS)

  • Standard beamforming method

• 2 Sparse DAS

  • DAS with under-sampled measurements

• 3 Gradient Descent with L1 Regularization (GD+L1 Reg)

  • Directly optimizes reflectivity distribution with sparsity prior

• 4 Implicit Neural Representations:

  • 4.1 INR-ReLU

    ✅ MLP-based, uses Fourier feature encoding

  • 4.2 SIREN

    ✅ MLP-based, uses sinusoidal activation functions

• 5 Deep Image Prior (DIP)

  • U-Net style convolutional autoencoder

• 6 DeepDecoder

  • Decoder-only convolutional network

• 7 ConvDecoder

  • Variant of DeepDecoder with some modifications

6 SIMULATION RESULTS

在仿真数据上评估所提出的CoIR

• 仿真数据生成:

  • Synthesizes radar data cube from 2D reflectivity images
  • Uses LiDAR point clouds to generate realistic reflectivity distributions

• 评估标准:

  • PSNR, SSIM, MAE between reconstruction and ground truth image

• 实验:

  • 1 Vary SNR from 35dB to 11dB

    ✅ ComDecoder gave superior PSNR over all methods at all SNRs

    ✅ ComDecoder and DIP gave comparable SSIM

    ✅ ComDecoder and DIP gave lowest MAE
    

  • 2 Visualize reconstructions at 19dB SNR

    ✅ ComDecoder gave most accurate recovery of extended reflectors

    ✅ Other CNN methods also improved over Sparse DAS

    ✅ SIREN struggled to distinguish clutter and true reflectors
    

  • 3 Additional analyses:

    ✅ Compared different CNN decoder architectures

    ✅ Evaluated computational complexity (in supplementary)

• 总结

  • ComDecoder 在 simulated data 上 SOTA

7 EXPERIMENTAL RESULTS

在真实采集的Coloradar dataset上评估所有方法

• Radar system:

  • 77 GHz FMCW with 1.282 GHz bandwidth

  • 86λ/2 uniform linear array

• Metrics :

  • 与 full array DAS reconstruction 进行对比

• Experiments :

  • 1 不同场景下的重建效果

    ✅ ComDecoder accurately recovered dominant features

    ✅ DIP also performed well but more artifacts than ComDecoder

    ✅ SIREN struggled in indoor scene due to noise
    

  • 2 Evaluate 鲁棒性 across multiple outdoor scenes

    ✅ ComDecoder gave high fidelity reconstructions closest to DAS

    ✅ SIREN fit strong reflectors but also artifacts

    ✅ GD+L1 located dominant reflectors but artifacts remained

    ✅ DIP performed well but more artifacts than ComDecoder

• 总结
  • ComDecoder 在 real data 上 SOTA
  • 显著好于其他untrained methods

8 DISCUSSION & LIMITATIONS

Limitations

• 1 Assume static scene in forward model
  • Cannot handle moving objects
• 2 Single bounce scattering model may not match real-world
• 3 Slow optimization time (tens of seconds)
  • Explore different initialization strategies

Future work

• 1 Demonstrated 2D range-angle slices due to linear array
  • 2D array needed for full 3D, but increases complexity
• 2 CoIR could benefit other array-based imaging modalities:
  • SAR, ultrasound, etc.

9 CONCLUSION

Proposed CoIR

• 1 Designed sparse linear array with 5.5x fewer antennas

• 2 Proposed convolutional decoder architecture ComDecoder

• 3 Demonstrated superior performance on simulated and real mmWave radar data