Marine World・Kyushu Sangyo University【Project Cetus】

SteamPunk Digital

Steampunk
Digital

Steampunk Digital

PROJECT Case o2

2019/01/06~2020/9/26

Client: Marine World Uminonakamichi,　Kyushu Sangyo University

Technologies: AR, Photogrammetry, Unity, Blender, Monocle SS

URL :

Marine World・Kyushu Sangyo University
【Project Cetus】

Mixed Reality Application「Project Cetus」
shown at Marine World

Steampunk Digital, Co. Ltd. and Jvis, Ltd. participated in the「Project Cetus」lead by Hoshino laboratory from the Kyushu Sangyo University.

The purpose of this project was to craft a Mixed Reality experience, for which Hoshino laboratory from Kyushu Sangyo University has been conducting research.

Steampunk Digital was in charge of scanning the humpback whale bones, development of the mixed reality application, and on-site deployment and configuration of 6 iPads. This project is presented as part of the “ほねのおと ~ HONENOTE ~” exhibition commemorating the 30th anniversary of Marine World.

How it Works

When you hold the tablet over the skeleton specimen (body length of about 8 m) of the humpback whale, a digital whale’s body is displayed on the actual bones.

Our 3D scanning technology, Monocle, was used for scanning the skeleton specimen of the humpback whale. We also used advanced photogrammetry and AR world mapping techniques.

The result is accurate alignment of body and internal organs of the humpback whale in a form that matches the actual skeletal specimen.

3D Scanning

In order to scan the skeleton specimen of the humpback whale, we used the Monocle Structure Sensor to quickly gather a scale-accurate 3D model of the whale bones, then combined this with photogrammetry techniques, which made it possible to produce high-precision outcome of 3D scanning with low-cost.

Even though it was a comparatively large installation (8m), it was possible to achieve centimetre accurate alignment of the digital 3D model with the physical whale bones, thus achieving a reliable mixed reality experience.

In terms of scanning work, it was completed using an iPhone X smartphone and Monocle 3D Structure Scanner application, the initial scan took 5 minutes, and photogrammetry capture took several hours. Processing took a few days for image alignment, 3D mesh clean-up, and geometry reduction.

Application Contents

The Cetus MR (Mixed Reality) app was developed based on the concept of “learning living information from bones,” and allows users to learn about the body and internal organs of a humpback whale by looking at the skeletal specimens that exist.

As a teaching tool, it is now possible to not only animate the body around the bones, but also to attract the interest of both adults and children with the floating of tiny marine plankton. By showing biological information about a living whale and incorporating diagrams and videos, children are naturally led to learn more about whales. Content includes videos of whale calls, plankton eating, breaching, and whale blowhole venting.

If for some reason the user needs to resume the experience, a guided mixed reality setup procedure is displayed to help point the iPad at the whale. Once the bones are visible and identified by the app, the alignment is performed automatically. The user can enjoy the mixed reality experience quickly and easily.