Using digital twins to enable accurate drone
management for last mile logistics

Using digital twins to enable accurate drone
management for last mile logistics

Using digital twins to enable accurate drone
management for last mile logistics

OVERVIEW

For my last term at the University of Waterloo, our capstone project was to pitch a business solution to our course partner, Rogers Communications Inc, while incorporating the use of digital twins and 5G networks in our idea. Our team focused on the topic of enabling safe and efficient drone deliveries.

OVERVIEW

For my last term at the University of Waterloo, our capstone project was to pitch a business solution to our course partner, Rogers Communications Inc, while incorporating the use of digital twins and 5G networks in our idea. Our team focused on the topic of enabling safe and efficient drone deliveries.

ROLE

Project Manager

& Product Designer


TOOLS

Figma, FigJam, Maze, Blender, Adobe After Effects


Jan 2023 - Apr 2023

ROLE

Project Manager

& Product Designer


TOOLS

Figma, FigJam, Maze, Blender, Adobe After Effects


Jan 2023 - Apr 2023

Our Team

As the project manager, I was responsible for managing work schedules and deadlines, delegating tasks, as well as creating agendas for our bi-weekly meetings. Throughout the term, I mainly focused on researching viable technologies for our solution, as well as the user interface of our system. Alongside me:

  • Mira Tabaza focused on market research

  • Riad Dajani worked as a financial analyst and IT specialist

  • Amina Makhmudova focused on business and marketing

  • Justin Chan did UX research on our topic

As the project manager, I was responsible for managing work schedules and deadlines, delegating tasks, as well as creating agendas for our bi-weekly meetings. Throughout the term, I mainly focused on researching viable technologies for our solution, as well as the user interface of our system. Alongside me:

  • Mira Tabaza focused on market research

  • Riad Dajani worked as a financial analyst and IT specialist

  • Amina Makhmudova focused on business and marketing

  • Justin Chan did UX research on our topic

Our timeline for the last month of the Project

What is a Digital Twin?

At the beginning of the term, Roger's introduced the concept of Digital Twin technology, which is a virtual representation of a real-world physical system, things, places, or process that serves as the digital counterpart for practical purposes.

At the beginning of the term, Roger's introduced the concept of Digital Twin technology, which is a virtual representation of a real-world physical system, things, places, or process that serves as the digital counterpart for practical purposes.

A Digital Twin "fuses ideas including artificial intelligence (AI), the internet of things (IoT), metaverse, and virtual and augmented reality (VR/AR) to create digital models of real-world objects, systems, or processes. These models can then be used to tweak and adjust variables to study the effect on whatever is being twinned – at a fraction of the cost of carrying out experiments in the real world."

—Forbes

A Digital Twin "fuses ideas including artificial intelligence (AI), the internet of things (IoT), metaverse, and virtual and augmented reality (VR/AR) to create digital models of real-world objects, systems, or processes. These models can then be used to tweak and adjust variables to study the effect on whatever is being twinned – at a fraction of the cost of carrying out experiments in the real world."

—Forbes

Current Problems with Deliveries

Throughout history, delivery times have become increasingly shorter, thanks to the advancement of technology. We started out with deliveries by foot, to horses, to trains and now we utilize planes and trucks. Today, delivery companies have been aiming for same-day or even half-an-hour deliveries, and because half-an-hour trucks are too costly and inefficient, these companies have looked into drones to enable that. Millions of dollars and resources have been spent in the past 10 years to test and enable drone deliveries, however, drones today are still too unreliable to be used for deliveries. This is due to a number of reasons:

Throughout history, with the improvements in technology, delivery times have become shorter and shorter. We started out with deliveries by foot, to horses, to trains and now we untilize planes and trucks. Today, delivery companies have been aiming for same-day, or even half an hour deliveries, and these companies have looked into drones to enable that.


Focus on last mile delivery.


They have been spending millions of dollars and resources to enable drone deliveries, however, even after 10 years of testing, the environmental elements have proven too harsh to navigate with todays technology. 



  • Amazon employee turnovers, not enough people to fly the dones

  • Automating the drone flying process can ensure a smoother delivery 



Amazon reported that many of their drones have gotten lost or damaged due to poor weather and navigational data. During one of amazons tests, a drone crash landed (fromm160ft above ground) and started a fire (burnt 25 acres of land) after losing one of its propellers.



  • One of amazon’s Drones current biggest obstacles is that they are damaged if flown through heavy rain



Currently, drone deliveries aren’t reliable

  • Precipitation make drone fail

  • Even with todays advanced weather systems, there are still gaps in the data for local regions

  • hardware/software error

    • amazon lost a propeller and it dropped 160 feet starting a fire

    • Imagine if we had known the propeller was coming loose before this happened

  • Poor object recognition (crashing into trees, powerlines, homes)



Let alone the vastly higher error rates with human pilots

01

Weather

One of delivery drones' current biggest obstacles is precipitation. Drones may fail or be damaged if flown through heavy rain, humidity, strong wind, or cold temperatures. Even with today's advanced weather systems, there are still gaps in the data for local regions. Amazon reported that many of their drones have gotten lost or damaged due to poor weather and navigational data. This also means that certain weather conditions are currently restricting deliveries.

Weather

One of delivery drones' current biggest obstacles is precipitation. Drones may fail or be damaged if flown through heavy rain, humidity, strong wind, or cold temperatures. Even with today's advanced weather systems, there are still gaps in the data for local regions. Amazon reported that many of their drones have gotten lost or damaged due to poor weather and navigational data. This also means that certain weather conditions are currently restricting deliveries.

02

Hardware/Software

Undetected hardware and software issues may occur during flight. During one of amazons tests, there was a motor failure on the drone, which led it to crash land from 160ft above ground and started a fire burning 25 acres of land. Imagine if we were able to detect the failure warning beforehand and was able to take an emergency landing, it wouldn't have caused such a tremendous hazard.

Hardware/Software

Undetected hardware and software issues may occur during flight. During one of amazons tests, there was a motor failure on the drone, which led it to crash land from 160ft above ground and started a fire burning 25 acres of land. Imagine if we were able to detect the failure warning beforehand and was able to take an emergency landing, it wouldn't have caused such a tremendous hazard.

Hardware/Software

Undetected hardware and software issues may occur during flight. During one of amazons tests, there was a motor failure on the drone, which led it to crash land from 160ft above ground and started a fire burning 25 acres of land. Imagine if we were able to detect the failure warning beforehand and was able to take an emergency landing, it wouldn't have caused such a tremendous hazard.

03

Poor Object Recognition

Poor object recognition on drones is causing many drones to crash into trees, powerlines, and even homes, this can be a huge safety concern. One of Google Wing's food delivery drones landed on a powerline which caused power outages for thousands of homes. Not to mention the vastly higher error rates with human pilots, especially unskilled pilots – since it is far too expensive to hire a large number of skilled drone aviators for all its delivery operations.

Poor Object Recognition

Poor object recognition on drones is causing many drones to crash into trees, powerlines, and even homes, this can be a huge safety concern. One of Google Wing's food delivery drones landed on a powerline which caused power outages for thousands of homes. Not to mention the vastly higher error rates with human pilots, especially unskilled pilots – since it is far too expensive to hire a large number of skilled drone aviators for all its delivery operations.

Understanding the Technology behind it

To gain a deeper understanding of how a digital twin may help us solve these issues, we began researching the possible technologies and sensors involved with drone navigation, weather detection and air traffic control data. With that we focused on some questions:


  • What are all the issues delivery drones currently facing?

  • What information is needed to ensure a safe drone flight?

  • What possible sensors can be used by drones

  • What real-time data can our system collect?

  • How does this help with our solution?


We spent weeks conducting user interviews and secondary research to familiarize ourselves with the current technologies and issues involved with drone deliveries. And as a result, we have came up with the current solution:

To gain a deeper understanding of how a digital twin may help us solve these issues, we began researching the possible technologies and sensors involved with drone navigation, weather detection and air traffic control data. With that we focused on some questions:


  • What are all the issues delivery drones currently facing?

  • What information is needed to ensure a safe drone flight?

  • What possible sensors can be used by drones

  • What real-time data can our system collect?

  • How does this help with our solution?


We spent weeks conducting user interviews and secondary research to familiarize ourselves with the current technologies and issues involved with drone deliveries. And as a result, we have came up with the current solution:

Introducing Pigeon (Partnering with Rogers)

Pigeon is a drone management system which focuses on tracking, monitoring, and controlling last-mile delivery drones. We create flight paths for drones based on real time data, automating the entire delivery process.


Our system leverages Rogers' 5G network for real-time drone communication so that we are able to detect and prevent all current drone delivery problems before they even happen. Using a virtual representation of the drone and its path, we can accurately simulate the environmental and weather factors that may affect the delivery.


We are able to use Rogers' cell tower signals to accurately predict up to 6 hours of weather and precipitation on a granular scale, especially when paired with other weather sources. And if this is paired with a Topographical map of the region, sourced by satellite data and by updating our training model with data collected from previous drone trips, we can create a digital twin that can provide the safest and most optimal path for the drone to take.


This results in predicting safe and efficient paths for drones to fly autonomously — under any circumstances.

Pigeon is a drone management system which focuses on tracking, monitoring, and controlling last-mile delivery drones. We create flight paths for drones based on real time data, automating the entire delivery process.


Our system leverages Rogers' 5G network for real-time drone communication so that we are able to detect and prevent all current drone delivery problems before they even happen. Using a virtual representation of the drone and its path, we can accurately simulate the environmental and weather factors that may affect the delivery.


We are able to use Rogers' cell tower signals to accurately predict up to 6 hours of weather and precipitation on a granular scale, especially when paired with other weather sources. And if this is paired with a Topographical map of the region, sourced by satellite data and by updating our training model with data collected from previous drone trips, we can create a digital twin that can provide the safest and most optimal path for the drone to take.


This results in predicting safe and efficient paths for drones to fly autonomously — under any circumstances.

A rough representation of how granular weather data detected from cell tower signals can enable with efficient drone deliveries

Additionally, Pigeon can streamline delivery operations for companies by reducing the number of truck drivers, emissions, time and salary costs. All of this is packed in an easy-to-use interface that displays critical issues in simple prompts and notifications:

Additionally, Pigeon can streamline delivery operations for companies by reducing the number of truck drivers, emissions, time and salary costs. All of this is packed in an easy-to-use interface that displays critical issues in simple prompts and notifications:

Prototype

01

Lo-Fi

For our initial prototype, we focused more on the functions and interactions so we can test our idea and the concept to more knowledge individuals before we begin designing. A prototype walkthrough and interview was done on 3 test subjects:

  • Autonomy Engineer

  • Aviation Student

  • Drone Pilot


They all gave us some valuable feedback on our interface. To summarize, our original interface showed too much unnecessary details – if we are targeting low-skilled workers, there is no need for them to know the displayed information (such as windspeed and precipitation levels) all they care about is wether or not the drones are safe, and if an issue occurs, how to solve them.

01

01

After coding their feedback and brainstorming changes to be implemented, we thought it was best to create a storyboard to better understand our users. So here is a user scenario we came up with:

01

02

Mid-Fi

Here are some changes we made to our prototype based on feedback. The second version of the prototype is used to conduct our second round of testing – which is mainly focused on usability. Primarily the layout of the pages, organization of information, and if it is intuitive for average users (who have no knowledge on drones). Hence, we tested it with our classmates.

01

03

Hi-Fi

Once the usability test was complete, we brainstorm ways to implement the feedback to our interface. We created a design system and begin designing our high-fidelity prototype. A couple of usability tests and iterations were done throughout our hi-fi design stage:

Demo

Corporate Entrepreneurship Proposal Document

Click here to see our Corporate Entrepreneurship Proposal