Client: Marine World Uminonakamichi, Kyushu Sangyo University
Technologies: AR, Photogrammetry, Unity, Blender, Monocle SS
Steampunk Digital, Co. Ltd. and Jvis, Ltd. participated in the「Project Cetus」lead by Hoshino laboratory from the Kyushu Sangyo University.
The purpose of this project was to craft a Mixed Reality experience, for which Hoshino laboratory from Kyushu Sangyo University has been conducting research.
Steampunk Digital was in charge of scanning the humpback whale bones, development of the mixed reality application, and on-site deployment and configuration of 6 iPads. This project is presented as part of the “ほねのおと ~ HONENOTE ~” exhibition commemorating the 30th anniversary of Marine World.
When you hold the tablet over the skeleton specimen (body length of about 8 m) of the humpback whale, a digital whale’s body is displayed on the actual bones.
Our 3D scanning technology, Monocle, was used for scanning the skeleton specimen of the humpback whale. We also used advanced photogrammetry and AR world mapping techniques.
The result is accurate alignment of body and internal organs of the humpback whale in a form that matches the actual skeletal specimen.
In order to scan the skeleton specimen of the humpback whale, we used the Monocle Structure Sensor to quickly gather a scale-accurate 3D model of the whale bones, then combined this with photogrammetry techniques, which made it possible to produce high-precision outcome of 3D scanning with low-cost.
Even though it was a comparatively large installation (8m), it was possible to achieve centimetre accurate alignment of the digital 3D model with the physical whale bones, thus achieving a reliable mixed reality experience.
In terms of scanning work, it was completed using an iPhone X smartphone and Monocle 3D Structure Scanner application, the initial scan took 5 minutes, and photogrammetry capture took several hours. Processing took a few days for image alignment, 3D mesh clean-up, and geometry reduction.
The Cetus MR (Mixed Reality) app was developed based on the concept of “learning living information from bones,” and allows users to learn about the body and internal organs of a humpback whale by looking at the skeletal specimens that exist.
As a teaching tool, it is now possible to not only animate the body around the bones, but also to attract the interest of both adults and children with the floating of tiny marine plankton. By showing biological information about a living whale and incorporating diagrams and videos, children are naturally led to learn more about whales. Content includes videos of whale calls, plankton eating, breaching, and whale blowhole venting.
If for some reason the user needs to resume the experience, a guided mixed reality setup procedure is displayed to help point the iPad at the whale. Once the bones are visible and identified by the app, the alignment is performed automatically. The user can enjoy the mixed reality experience quickly and easily.
If you have a question about anything,
please feel free to contact us!
We are happy to contact with you.