Urban Planning: Building Cities in VR and AR

By Will Freeman,


Cities serve as beguiling spaces when they take on a virtual form. Even in the early days when SimCity and GTA were humble 2D releases, the idea of managing or exploring a seemingly living, breathing city with its own systems was remarkably powerful.

Beyond the boundaries of interactive content, the notion of the virtual city has also flourished, even becoming a trope of the silver screen. Consider movies like Avatar, Iron Man, Prometheus or The Hunger Games, where the cast stand over a 3D holographic projection of a city and plan their next moves, and it’s clear the concept of broad urban sprawl made into an interactive model is wildly enchanting.

And now, with the dawn of AR and VR, suddenly the cinematic fantasy of the 3D, interactive city map has become a reality, and the opportunity for content creators and developers there is vast.

It’s a space of great interest to Kelly Malone, Vice President, Product Management and Business Development at Taqtile, a team with a long heritage in delivering apps integrated with – or based on – a physical venue.

Malone's fascination began when the Taqtile team tried a Vive demo that placed the user in the middle of San Francisco at ‘Godzilla scale’. 

“It was very simple,” Malone remembers. “It was just Google Maps. There was no application really; there was no special work that they did on their models or the geometry, but it was extremely compelling. We were really excited by the possibilities that we saw when we were just standing there in the middle of this environment.”

That experience got Malone and his colleagues considering how to re-imagine those holographic maps from the movies, and how it might be possible to recreate the concept for a VR and AR audience. Over time, it became clear that what real-world head-mounted displays could offer might actually rival The Hunger Games' Holo.

Some of the films that influenced Taqtile's efforts with interactive cityscapes

“We may not have that cool projector device that can project the hologram,” Malone admits. “But we do have these head-mounted displays, and actually they give us a little more engagement and ability to manipulate the model than perhaps a projection system might do.”

The opportunities offered by a map a user can stand over, zoom into, rotate, and explore while strapped into an HMD aren't just varied and exciting; they can also bring a studio a solid revenue stream. Reinventing navigation and routing apps, serving urban planning clients, helping with training in emergency management, refining how sporting events are viewed, or, of course, the potential in games; all of these are virtual cityscape use cases.

Across that wide variety of content, Malone asserts, familiar content creation rules apply. Developers, he suggests, would do well to ponder target hardware’s unique attributes, recognize their team’s competencies and strengths, remember an HMD’s limitations and availability, and contemplate how market ready their concept is. So far, so conventional.

But, says Malone, to embrace the opportunity of building cityscapes in VR and AR, very specific challenges must be addressed.

“One of the key challenges was sourcing the 3D models,” he states. “Where do we get these models? Or how do we build them? Then, how do we manage this large amount of data, and make sure that, one, its manageable for building the application, but then also, how it impacts the performance on the target device.”

Fortunately, Malone is as generous with sharing his lessons-learned as he is enthusiastic about the potential of building entire cities.

A realistic, interactive model of Cleveland, in a scene in the Unity editor

“Before you even get started you need to think about how accurate your model needs to be’. Is it real, or is it fictional? If it’s a fictional city, you have actually a lot more flexibility. If it’s a real city you impose some constraints, and now you need to think about how current or relevant does that reality need to be? Are we talking history; are we talking today? Are we talking last week or last year? And do you need to show a future state, potentially, or is it all of the above? This is an important consideration before you even get started in thinking about what tools you might use to build or source your city model.”

There’s plenty more to consider, too. What level of detail does your project need? How close will the user be able to zoom into the city? Do you need textures, foliage or street furniture? What about setting your city in the daytime versus the night time? Or perhaps you want to implement a day-night cycle?

With those designs made, suggests Malone, the most important decision is in choosing the tools and technology that can build a city with you. Google Earth and Bing Maps are remarkably accessible, and Malone says, provide impressive accuracy, being based on satellite imagery of reality. But these offerings do lack detail.

Developers, therefore, should be open-minded to less everyday tools and file formats, like ArcGIS, USGS and CityGML.

“If you need to source a really accurate, detailed model to do some testing, I encourage using City GML,” Malone advises. “The city of Berlin has been completely modelled in 3D using this format, and it exports to any number of other file formats.”

The procedural city modelling tool CityEngine is also worth exploring, Malone recommends.

“If you need to create a fictional city, you literally press a button, and CityEngine will actually just build you a city based on some templatized approach.”

A virtual cityscape can be shaped from many accessible sources

Then there’s Skethup, 3DS and content produced by CAD users. Sketchfab, Unity’s Asset Store, and Turbosquid are also valuable tools to research in this context.

Whichever you choose, Malone has one piece of advice that is equal parts practical and essential.

“If you’re going to use any of these internet sources of data, be sure to check the terms of use,” he offers. “If you’re going to employ it in a commercial venture, many of these sources don’t permit that, or they permit it under very restricted circumstances.”

And to conclude, this devotee to building virtual cityscapes has one final insight.

“These models are big […] so you need to think about ‘can we optimize those models?’. Or you may want to use model streaming. That’s another interesting way to address that. But again, it has architectural considerations on how you build your application.”

Clearly, the potential of the cityscape in VR and AR is vast, but as the Taqtile team has learned, before you lay your first paving stone, you need to ask yourself – and your team – a number of important questions. Only then will you be positioned to loom above a metropolis of your own conception.


Got it

We use cookies to ensure that we give you the best experience on our website.

Click here for more information.