Extended Reality For SIDS
Immersive experiences and exploration
for ecosystems, cultures and livelihoods
using drones and artificial intelligence.
UNDP Technology, Innovation and Sustainable Development
As a kid, I didn't have a television, but when I got the chance I would watch as many Cousteau and National Geographic documentaries as I could about the underwater world.
I was fascinated, inspired and persistent to one day have the chance to experience the underwater realm myself and understand the relationships between people and aquatic ecosystems. This was fundamental in shaping my life and career as an aquatic biodiversity and social scientist.
However, today, children are exploring other worlds, on mobile phones, consoles and computers, in 3D gaming environments where they stay immersed, often for hours on end.
Virtual digital worlds increasingly dominate our lives and often, empathy and understanding of the complexities and sustainability of the real world are frequently a forgotten aspiration.
So we have an opportunity to harness these technologies and capture the attention of future generations by creating experiences which excite curiosity for living ocean ecosystems in an accessible, enjoyable and engaging way.
Imagine experiencing some of the most remote ecosystems in our Oceans,
and people living there...
without travelling, without leaning to dive...
The UNDP SIDS Extended Reality (XR) experience uses underwater and aerial drones to capture an immersive interactive virtual reality experience of the relationships between people and Ocean ecosystems.
Much more than just another 360 YouTube video!
Harnessing artificial Intelligence with interactive drone content from above and below water, the platform combines a responsive experience (mobile to VR headset) where you can learn about the many dimensions of marine biodiversity while exploring cultural bonds between people and the Ocean environment.
We will also use innovative underwater WiFi/4G extenders to demonstrate how live streaming 360 content can be used by remote virtual tour guides using both underwater drones, and dive guides.
How do we do it?
By using modified drone technologies, we will capture Ocean ecosystems more passively and experience the underwater world with minimal intrusion.
This is achieved by using a modified underwater drone capable of capturing 3D stereoscopic 360 degree video.
Why? Less intrusive filming has a number of key benefits!
What goes on in the Ocean when we aren't present as scuba divers so often reveals some of the most intriguing ecosystem interactions between marine organisms.
This system allows us to get up close to small micro ecosystems, move slowly and carefully and also go deeper for much longer durations than most divers ever could.
Previsualisation of the custom drone technology which will capture the 3D 360 experience.
The production technology is a step further to ensure a more passive experience while delivering content for the platform in full 3D 360 degree extended reality (XR).
Well produced online 360 content of iconic environments and action typically attracts millions of views, while this is a bonus, we take the experience a step further by integrating it into an augmented reality (AR) platform to empower user curiosity and knowledge development.
So once the base 360 video environment is captured, the next step is to enhance the video by bringing it into a game engine.
This is where we add augmented reality overlays and apply artificial intelligence machine learning to the footage to extract and enhance features and enable "explore more" options to excite the curiosity of the user.
These elements aren't simply species labels, but an informative narrative which spans the three pillars of socio-cultural, environmental and economic relationships in terms of both local people and in a global context.
This enables the user to open up more content, while pausing the virtual tour, to satisfy their curiosity and knowledge development.
The structure, future and sustainability of the XR SIDS experience
The main showcase XR SIDS experience is a pilot example to generate further interest and is designed to be inbound funnelling instrument to attract donors and users leading to a virtual community of practice where you can relax, enjoy and learn about ecosystems and share knowledge and experience.
The underwater 4G and Wifi system we will demonstrate can be used to hire a local dive or underwater drone guide for a personalised live streaming tour either using a simple streaming device such as a mobile phone in an underwater housing, GoPro or 360 Camera.
Additionally these technologies enable us to take a local tour above the Ocean, go shopping on SIDS for locally made products and understand the how ecosystems inspire cultural arts and creativity.
The apk will be made available on the Oculus Quest Store and also be easily transferrable as an apk file so that schools and other organisations can use it as a virtual classroom, both locally and globally.
Terms Of Reference And Budget
Deliverables / Terms Of Reference and Key Performance Indicator Outputs
- Deliver three XR experiences exploring the diversity of Fiji's ocean ecosystems which integrate the relationships with people and nature.
- The three stereoscopic deliverables to be viewable on Oculus and other standard VR headsets with a dynamic front end and user guidance narrative integrated into the experience.
- Deliverable source media filmed in 8k 360 stereoscopic for high quality 3D 4k VR delivery.
- Oculus Quest Store Authoring for broad outreach.
- A YouTube VR 360 teaser of the deliverables for broader outreach and engagement.
- Social Media awareness raising campaign.
- VR deliverables should be completed before and in anticipation of the United Nations Ocean Conference in Lisbon, June 2022.
- Content should include socio cultural, environmental, and economic elements in line with the UN Sustainable Development Goals.
- Content should include end teasers to encourage viewers to exploring content from other SIDS.
- Deliver the UN Ocean Conference Side Event including technical setup, media assets for visuals, headsets and client liaison.
Budget Cost Breakdown
Narrative Development Production Design & Workflow
Production Personnel & Overheads
Technical & Location Filming Costs for 360 8k 3D Capture $28,300
Travel, Transfers, Taxes, Excess Baggage, Insurance, Boat Hire - $29,600
360 Content Postproduction; stitching, scaling, content extraction, sequence editing & subsequence editing
UX Design & Authoring
AR asset production including factual content, CG overlay generation & B-roll integration.
VR XR Prototyping, Authoring, Coding & AI Integration
Delivery of UN Ocean Conference Side Event (inclusive)