Worked on the Sensor Systems in the Drone Versus Drone Project. The Sensor Systems mainly include 2 smartphones streaming their camera feed to a Computer. The computer then then triangulates the drone’s position to be viewed on a C2 system.

My main work involves

1. Migrating video analytics processing to the cloud EC2 Instances, and linking devices together over the Internet using Wireguard & AWS IoT Core.
2. Improved the android app for video streaming. Developed tools such as a drone deployment calculator to assist in deploying & Testing the setup.
3. Modified an off the shelf Gimbal to be remotely controlled through BLE or USB Serial with an Arduino. Made it turn to our desired bearing with the help of the Phone’s Compass.
4. Experimented with ways to improve performance, such as tiling of the video feed, improving the video pipeline through FFMpeg options and more.
5. Tested improvements multiple times in outfield conditions. Learnt about basic operation of a drone, as well as safety procedures.

---

Breakdown of the things I did

Networking

1. Handled RTSP, Amazon Kinesis Video Streams for streaming data
   • Attempted to improve KVS latency through Retrieving the HLS link and handling it with FFMpeg
2. Handled ZMQ and AWS IoT Core for sending Metadata
3. Experimented and Used SSH Tunneling for connecting devices over the Internet, as well as AWS IoT Core Tunneling
4. Set up a Wireguard VPN on AWS EC2 to connect devices together and send information over the Internet. VPN was used for a demonstration of the system to the potential users.
5. Other SysAdmin stuff (configuring scripts to run on boot up, setting up EC2 Instances with a bastion host, key rotation etc.)

Application Development

1. Improved the Android app for video streaming
   • Added support for AWS IoT Core for sending metadata, as well as improved the metadata format to support more features such as dynamic FOV calculation, location Data etc.
   • Added support to KVS for video streaming, as well as implemented features for KVS Camera streaming such as zoom
   • Implemented some features, such as zoom, resolution, and starting/stopping stream to be controlled over ZMQ
2. Developed a simple deployment calculator in Python tkinter
   • calculates locations on where to fly the drone for easy testing (calculation not fully done by me), and displays it on a map
   • allows for easy testing of experiments with the video analytics algorithm
3. Assisted in developing the more user friendly control Interface (Flutter)
   • Gave suggestions for the architecture
   • Developed the backend interfaced with ZQM to control the Video Analytics Software
   • UI tweaks to support mobile view

Electronics

1. Did some simple Electronics Soldering, testing and Debugging of components on the drone
   • most notably, assisted with serial communications, speaker
2. Modified an off the shelf Gimbal to be remotely controlled through BLE or USB Serial.
   • Wrote a simple python script that connects to the Arduino over USB Serial to turn the Gimbal to a certain bearing based on the Phone’s compass value

3D Modelling

1. 3D Modelled & Printed a simple mount to prevent phone tripod clamps from pressing on the Pixel 7 Pro’s Volume Buttons
2. 3D Modelled & Printed Dastan

Others

1. Experimented with cropping (and by extension, tiling) of video streams to improve performance of video analytics
2. Dockerisation of Video Analytics code
3. Experimented with using Unity to more accurately test video analytics code.