PolyCam is a platform for sharing photogrammetrically captured 3D models and a mobile app for photogrammetric capture. I have explored the latter use in the most depth, as there are already other, more feature-complete platforms for model distribution.
This reflection refers to the iPhone and iPad versions that are available for both iOS and Android phone operating systems. The core feature of the PolyCam app is the ability to create 3D scans of real-world objects and environments. In the current version there are four levels of functionality:
· LiDAR – A photogrammetric scan that uses the LiDAR sensor on modern iPhones and iPads.
· Room—This feature creates a 3D model of an interior space rather than a photogrammetric scan using the LiDAR sensor on modern iPhones and iPads.
· Photo – A photogrammetric scan that uses the camera on the iPhone to capture and then interpolate multiple photographs into a 3D model.
· 360 – A feature that enables the user to create 360 degree “photosphere” of a space (usually an interior space).
The 360-degree feature is demonstrably poor compared to other tools that achieve the same result (Matterport, for example), but the photogrammetric features are the strongest of any app evaluated during this study.
Models can be measured and rescaled (important for VR), cropped, and viewed in AR via the user’s phone. Captures can also be reprocessed to create higher-resolution models.
By default, models uploaded publicly to the PolyCam platform can be downloaded and used by others under a creative commons license.
Feelings
In the right conditions (with full 360-degree access around an object with good light) and the right object (smaller objects with interesting, non-reflective textures work best), PolyCam’s results are “magical”. Photogrammetry has been seen as a difficult, technical process (as I explore in my reflection on Reality Capture), with a series of stages that must be correctly and carefully completed to get the required results. PolyCam simplifies this by automating much of the process “server-side”. The creator scans an object with their phone (either using LiDAR or their camera or both), but the alignment of imagery and creation of models takes place in the cloud. This is different to professional capture software systems where users have a large degree of control over the outcome – and processing takes place on the user’s local computer.
The LiDAR features continue this feeling of magicality in interior spaces. The ability to generate a 3D model of a room by scanning a room with a phone (and then importing that model into other applications) is a significant step towards non-programmers recreating any space quickly for VR purposes.
Some of that “magic” comes from the experience of scanning an object in real space and then, some minutes later, examining a 3D digital model of that object on your phone.
Evaluation
Although “easy” to use, there is still a distinct process and possible capture failure. Objects with reflections or transparency do not capture well. Poor light and/or shadows will create objects that look unrealistic when viewed out of context. The ideal light is outdoor light, global and diffuse, which is not always easy to acquire.
Because the processing takes place in the cloud and users cannot see it, there is a delay between capture and creation of the 3D object. There are two parts to this workflow: photos upload and are then processed. Error reporting is poor, so if the process fails, there is no way of knowing why (a simple message “images too dark” or “not enough images to create model” would help).
However, when the process is successful, the post-production features are extensive and powerful. Models can be downloaded in a number of interoperable formats and/or uploaded directly to cloud services, including SketchFab, where they can be processed further.
Over a series of captures in different conditions, I found that PolyCam was good for three use cases:
The photogrammetric capture of objects upto “human” scale. For example, I successfully captured objects as small as coins and as large as full-sized tombs and statues. I could not capture rooms or buildings as successfully using the “photo” capture features – with or without LiDAR capture. It was interesting that in capturing fixed objects in 3D, I could view objects from “impossible” angles. For example – a successful capture of a row of Roman coffins enables viewers to see them from above in a “plan view” that would only be possible in real-life with a drone.
The LiDAR capture of rooms and interior spaces. Even in this use case, PolyCam was better at creating simplified 3D models with primitive shapes than photogrammetric captures of interior spaces. Nevertheless, a 3D model has some flexibility in use – can have textures applied to it and may contain inferred detail impossible to recreate in a photogrammetric scan.
Finally, the PolyCam web platform is useful for quickly converting drone footage of buildings and clearly defined spaces into models. I’ve used this to develop models of real spaces and this approach may be useful in rapid reporting.
Application
Polycam may be used for several purposes:
· To capture or download assets to populate VR spaces in the creation of spatial/immersive journalism
· To capture or download assets that may be annotated in third-party apps for storytelling
· To capture spaces (rooms, interiors) with LiDAR, which would then form the basis of a VR space to investigate or explore – rapid space development slower narrative construction.
· To process drone footage to recreate clearly defined objects and spaces.
Conclusions
PolyCam is a rapidly evolving and powerful mobile application for creating 3D models and photogrammetric scans. It has become a central part of my workflow for two clear reasons. First, using a phone to scan anywhere makes it a portable, rapid way to create models in situ. Second, it does a great job of algorithmically creating meshes and applying textures with little human intervention.