Integrating Antenna on Package (AOP) Radar Through Companion Computer Like Raspberry Pi 4 or Jetson Nano to FCU as Proximity Sensor

            mmWave Radar IWR6843AOPEVM          The IWR6843AOP device is antenna-on-package (AoP) evaluation module (EVM) which is a 60-GHz mmWave sensor evaluation platform. It integrates a wide 120-degrees field-of-view (FoV) antenna and enables access to point-cloud data and power over a USB interface. Examples of its existing applications are:

• Industrial level sensing
• Industrial automation sensor fusion with radar
• Traffic intersection monitoring with radar
• Industrial radar-proximity monitoring
• People counting
• Gesturing

Understanding the hardware design

            RF and Analog Subsystem includes the Radio Frequency  and analog circuitry which comprises of  the synthesizer, Power Amplifier, Low Noise Amplifier, mixer, IF and Analog to Digital Convertor. It also includes the temperature sensors and crystal oscillator. Two out of the three transmit channels can be operated simultaneously in 1.3-V mode.

            Clock Subsystem generates 60-64 GHz from an input reference of 40-MHz crystal. The clock subsystem has a built-in oscillator circuit which is followed by a clean-up PLL and RF synthesizer circuit. The output of the RF synthesizer is then processed by an X3 multiplier to create the required frequency in the 60 to 64 GHz spectrum. The required waveform for sensor operation by modulating the RF synthesizer output by the timing engine block. Thereafter, for the host processor, the clean-up PLL  provides a reference clock after system wakeup. There is a built-in mechanism for detecting the presence of a crystal. It is also monitoring the quality of the generated clock.

            Transmit Subsystem consists of three parallel transmit chains with each having its own independent amplitude and phase control supporting Transmit Beam forming applications, 6-bit linear phase modulation for Multiple Input Multiple Output (MIMO) radar and also interference mitigation. The programmable backoff supported transmit chains system optimization.

            Receive Subsystem consists of four parallel channels. Every receive channel consists of an LNA, mixer, IF filtering, A2D conversion, and decimation as a complete unit. All four receive channels can  function at the same time while an individual power-down option which ensures system optimization is also available. The complex baseband architecture of this device uses quadrature mixer and dual IF and ADC chains to deliver complex I and Q outputs for each of the  receiver channel. It targeted for fast chirp systems. The band-pass IF chain has lower cutoff frequencies above 175 kHz which are configurable. It can support bandwidths up to 10 MHz

            Processor Subsystem             At a high level it has two customer programmable subsystems. It is shown by a dotted line in figure 8.5. Left hand side is the DSP Subsystem which contains Texas Instruments high-performance C674x DSP, hardware accelerator, a high-bandwidth interconnect for high performance (128-bit, 200MHz), and associated peripherals – four DMAs for data transfer, LVDS interface for Measurement data output, L3 Radar data cube memory, ADC buffers, CRC engine, and data handshake memory (additional memory provided on interconnect).  The right side shows the Master subsystem which is the master of the device and controls all the device peripherals and house-keeping activities of the device. It contains Cortex-R4F (Master R4F) processor and associated peripherals and housekeeping components such as DMAs, CRC and Peripherals (I2C, UART, SPIs, CAN, PMIC clocking module, PWM, and others) connected to Master Interconnect through Peripheral Central Resource. HIL module is seen in both the subsystems. It is used to perform the radar operations which involves feeding the captured data from outside into the device. It does not involve the RF subsystem. HIL. HIL on DSPSS is for high speed ADC data input while on master SS is for controlling the configuration. Both HIL modules uses the same IOs on the device. One additional IO (DMM_MUX_IN) allows for selecting between the two. Host Interface can be provided through a SPI, UART, or CAN-FD interface. The IWR6843AOP device communicates with the host radar processor over the following main interfaces:

• Reference Clock is available for host processor after device wakeup
• Control – 4-port standard SPI (slave) for host control. All radio control commands (and response) flow through this interface.
• Reset – Active-low reset for device wakeup from host
• Host Interrupt – an indication that the mmwave sensor needs host interface
• Error – Used for notifying the host in case the radio controller detects a fault

            Features of the device

• FMCW transceiver consisting of Integrated 4 receivers and 3 transmitters with Antennas-On-Package (AOP), Integrated PLL, transmitter, receiver, Baseband, and A2D. It has 60- to 64-GHz coverage with 4-GHz continuous bandwidth, supports 6-bit phase shifter for TX Beam forming and ultra-accurate chirp engine based on fractional-N PLL
• Built-in calibration and self-test with ARM® Cortex®-R4F-based radio control system, Built-in firmware (ROM) and self-calibrating system across frequency and
• C674x DSP for advanced signal processing
• Memory compression
• Hardware accelerator for FFT, filtering, and CFAR processing
• ARM-R4F microcontroller for object detection, and interface control
• Supports autonomous mode (loading user application from QSPI flash memory)
• Internal memory with ECC 1.75 MB, divided into MSS program RAM (512 KB), MSS data RAM (192 KB), DSP L1 RAM (64KB) and L2 RAM (256 KB), and L3 radardata cube RAM (768 KB)
• Other interfaces available to user application supports upto 6 ADC channels (low sample rate monitoring), 2 SPI ports, 2 UARTs and 1 CAN-FD interface, I2C and GPIOs. It also has 2 lane LVDS interface for raw ADC data
• Power management with Built-in LDO network for enhanced PSRR and I/Os support dual voltage 3.3 V/1.8 V
• Clock source 40.0 MHz crystal with internal oscillator and supports external oscillator at 40 MHz. It also supports externally driven clock (square/sine) at 40 MHz
• Easy hardware design with 0.8-mm pitch, 180-pin 15 mm × 15 mm FCBGA package (ALP) for easy assembly and low-cost PCB design ensuring small form factor
• Operated in the range of –40ºC to 105ºC [1]

Antenna Positions and Radiation patterns

mmWave radar integration with companion computer

The mmWave antenna is to be connected to the companion computer via USB port. This USB interface supports two UART links, one for configuring the radar and other for data transfer. The configuration file needs to be created via Texas Instruments visualizer and .cfg file then can be saved on the computer. Now we need to send the configuration file each time the mmWave radar starts and then data starts flowing which has to be interpreted and then forwarded to Ardupilot running on Flight Controller Unit. Hence, python script does this job of sending configuration file to mm Wave radar to open sensor and start receiving data. The incoming data is sliced and interpreted. This logical data is sent to MAVPROXY by the python script. The MAVPROXY Command Line Ground Control Software (GCS) packetizes the data as part of MAVlink to send it as proximity sensor data to Flight Controller Unit which in my case is Beagle Bone Blue. This setup can work with any FCU with companion computer.

Implementation steps

In order for mmWave radar to successfully communicate with the companion computer, we need to carry out below mentioned actions.

            We need to install latest python and pip3 package on companion computer. I have built Python 3.7 from source on my companion computer. To build Python package from source.

sudo apt update
sudo apt install build-essential zlib1g-dev libncurses5-dev libgdbm-dev libnss3-dev libssl-dev libreadline-dev libffi-dev curl libbz2-dev

Download the latest python release source code using curl command:

curl -O https://www.python.org/ftp/python/3.7.3/Python-3.7.3.tar.xz

After download, we need to extract the tarball :
tar -xf Python-3.7.3.tar.xz

Navigate to the Python source directory and run the configure script which performs number of checks to ensure all the dependencies are available on system. The optimizations option runs multiple tests to check system readiness but makes the process slower.

cd Python-3.7.3
./configure --enable-optimizations

Run make to start the build process:
make -j 4

To achieve quicker build time, we can modify the -j flag according to the processor. Depends on number of cores and for the processor how many cores it has, can be found by typing the command nproc. Once the build is done we install the Python binaries.
sudo make altinstall

We must avoid using standard make install as it will overwrite the default system python3 binary which we don’t want. At this point, Python 3.7 is installed on my Debian system and is ready. We can verify it
python3 --version
Python 3.7.3

Next we want to install pip3. First we update the package list for Linux OS
sudo apt update

Next, we install pip for Python 3 with all of its dependencies
sudo apt install python3-pip

To verify the installation
pip3 --version

The version number could vary something like below. Hence we must upgrade pip.
pip 9.0.1 from /usr/lib/python3/dist-packages (python 3.5)

Upgrade pip package
python3 -m pip install –upgrade

            Install dependencies

pip3 install pyserial
pip3 install apscheduler
pip3 install numpy
sudo apt-get install libatlas-base-dev

            Getting mavlink ready (uninstall pymavlink older version if any and install latest version)

pip3 uninstall pymavlink
git clone https://github.com/ArduPilot/ardupilot
cd ardupilot
git branch master
git submodule update --init –recursive
cd /ardupilot/modules/mavlink/pymavlink
python3 setup.py install --user

            Get mm wave radar code folder from my github repository         

git clone https://github.com/yashlancers/mm_Wave_Radar_IWR6843AOPEVM.git

Creating mmWave radar configuration file 

            Configuration file for mmWave radar from TI visualizer (optional)         The github repository that we downloaded contains the configuration file .cfg that’s tried and tested for proximity sensor usage of mmWave radar. However, incase it needs to be generated again keeping any specific requirement in mind, the process below explains it with details. We will use the Texas Instruments visualizer to create configuration file for the mmWave radar. This action has to be done once. It has to be repeated only when we want to make changes to parameters of mmWave radar. Using Google Chrome and access https://dev.ti.com/gallery/view/mmwave/mmWave_Demo_Visualizer/ver/3.5.0/ If prompted, follow the on-screen instructions for installing TI Cloud Agent (this is needed the first time on a new PC)

In the GUI menu, select Options → Serial Port

In the serial port window, COM4 for configuration (11500 bauds) and COM5 (921600) for data

The configuration file can be generated by selecting save to PC. The configuration file is already part of the repository that we downloaded from github. We can even see the output of radar on Texas Instruments visualizer as standalone sensor

          Testing python script to open sensor (mmWave radar) and get sliced data output. Connect mmWave radar to companion computer using USB port. Make sure that the interface details (COM Ports in Linux) are updated correctly in python script. It can be checked with

sudo ls /dev/ttyUSB*

Incase of Raspberry Pi 4

python3 /home/pi/mm_Wave_Radar_IWR6843AOPEVM/mmwave_to_mavlink.py

Incase of Jetson Nano

python3 /home/maverick/mm_Wave_Radar_IWR6843AOPEVM/mmwave_to_mavlink.py

Replace maverick by your username

First the script mmWave_to_mavlink.py sends configuration to mmWave radar to start/open the sensor

Python script reading data in terms of no of objects detected, x,y,z coordinates , velocity, acceleration and azimuth angle.

Now the mmWave radar is working as proximity sensor sending data from companion computer to Ardupilot running on FCU which in my case is Beagle Bone Blue. If we connect FCU to GCS like Mission Planner , we can see the radar output as seen in next image. We need to make changes to Ardupilot running on FCU through Mission Planner to be able to see output of sensors in Mission Planner which is the GCS. Hence, I have dedicated an entire chapter 10 to understanding configuration of Ardupilot through Mission Planner later in the thesis . For now, the next image shows the mmWave radar output in Mission Planner.