Introduction

The Villa Romana de l’Albir is a Late Imperial vicus (4th-7th centuries AD) located on the coast of l’Albir and discovered after several archaeological seasons carried out in the 80s and 90s of the last century. This important heritage site includes an extensive necropolis, a private mausoleum and the villa where the vicus administrator lived (Morote and Fernandez Rojo 1989; Chavarria, Brogiolo and Arce 2006, Chavarria 2007).

Enhancement of the thermal baths, undertaken between 2008 and 2010, led to its designation as the Villa Romana de l’Albir Open Air Museum by the regional government. Once the intervention in this area (sector I) had been completed, work focused on the villa itself, having so far located the villa’s pars urbana, a necropolis dating from the 6th century AD, a Visigothic stage of occupation (7th century) with working areas and garbage dumps and a final, late mediaeval (14th century) stage of occupation with productive structures and large garbage dumps (Frías Castillejo 2010).

Based on the results obtained throughout the different research seasons and with the aim of promoting and disseminating this heritage site, a 360-degrees, short animated film has been produced, to show visitors a virtual recreation of the village through an immersive experience.

As can be seen in the following diagram, the process to create this short film has been divided into two main stages (pre-production and production) which will be described throughout this paper, with special emphasis on the open source tools used in the project: Meshroom, Blender and the Extended Matrix.

Pre-production

During this preliminary stage it is important to define the initial idea, which should include aspects such as: basic plot, number of characters, year in which the story will take place, number of scenes and total length. Once these details were defined, three basic working documents were created: the script, the shooting script and the storyboard. Together, these three documents should be detailed enough to provide all the information that will be needed during the production process.

Initial idea, story, plot

The plot centres around a married couple from the countryside of Alicante who visit the Villa of l’Albir - located on the seaside- to close trade agreements. Their hosts - the villa owners - receive them and throughout the three scenes they go through the different rooms of the villa while talking to each other. This plot has allowed us to introduce aspects of daily life in a rural settlement in the 5th century AD, but also others related to local and regional trade, or references to the political, social and even religious situation of the time.

The story is set in the year 412 AD and is divided into 3 scenes lasting 10 minutes, and structured as follows:

Scene 1. Reception of guests in the peristylium. 3 minutes.

Scene 2. Tour through the different rooms of the balneum. 3 minutes.

Scene 3. Dinner in the oecus. 3 minutes.

Script

Once the plot was defined, the next task was writing the script, which was divided into scenes and sequences, each of which was preceded by a brief introduction to then write the dialogue for every character, describing also the feelings of each of them through their body language and facial expressions.

Shooting script

The shooting script is a text file that defines technical aspects that do not appear in the script, complementing it. It describes in detail every action that will take place in the scene, the elapsed time (in seconds) and the exact position of each character, asset or camera throughout the scene. Together with the script, the shooting script will be essential for the CGI (Computer Generated Imagery) department during the virtual production stage, since all the camera and character animation work will be based on it.

Storyboard

The storyboard consisted in a set of sequenced and ordered illustrations, used as a visual guide to the story, to preview an animation but also to structure the film before it is made. It is the pre-production mode used in the film industry.

Its main purpose is to tell the story with a few sequenced images, including some details about the film shot number, its length and a brief description of what happens in it. All this information will serve as a guide once in the production stage. It is not essential, like the script or the shooting script, but it is still a very useful tool and constitutes the first visual preview (although not very detailed at the moment) of the short film itself.

Production

As with any film project, this stage is the most time and resource-intensive. It involves not only the elaboration of the visual and sound contents of the short film, but also the development of an app that will allow users to enjoy the whole experience.

Sources

In this type of project, candidates for virtual reconstruction hypotheses are always preceded by collecting and researching the available data sources. In the specific case of the Roman Villa of l’Albir, these can be classified according to their nature and their direct or indirect relationship with the archaeological site itself: 3D survey, archaeological reports, and other sources.

3D survey

The preserved archaeological remains were surveyed using digital photogrammetry, following a well-known methodology (Historic England 2017), widely used and fully integrated in today’s archaeological fieldwork. The input usually consists of structured sets of photographs with high overlap between them, together with the acquisition of spatial data through the use of appropriate surveying instruments, such as total stations and satellite positioning data. The output is an oriented, textured 3D model. To process the images we used Meshroom, an open source software which offers a simple but powerful nodal approach (Alice Vision, n.d.).

As a result, a textured 3D model of each of the different areas of the archaeological site was obtained, ready to be used in any 3D modelling software.

Archaeological reports

Archaeological excavation is a destructive process, so detailed, accurate reports generated during each excavation season are an essential data source for virtual reconstructions. Both written descriptions and graphic documentation (technical drawings and photographs) will be useful data sources.

Other sources

Where archaeological reports and 3D photogrammetry models are not enough, other data sources are needed, such as other references or, in this case, other Roman villas. This, although valid, must be done with special attention and preferably looking for other similar examples based on criteria of contemporaneity and geographical area. Also, taking into account that they may have undergone alterations, such as modern restorations, not always documented or easily identifiable.

The same procedure can be used in the case of written sources or artistic representations. In virtual reconstructions of Roman architecture and artefacts, wall paintings and mosaics are particularly useful as a source of information.

Hypothesis

From the study of the different sources of information, several hypotheses were elaborated, tested and validated until one of the candidate hypotheses was selected as the working hypothesis.

Ideally, validating and selecting the working hypothesis requires the participation of as many specialists as possible and from different fields, not only Archaeology, but also History, Art History or Architecture, to ensure that the selected hypothesis is technically and aesthetically coherent with the available scientific information. Therefore, each part must be analysed and tested according to a scientific methodology.

Preliminary sketches

At this point and from the available data, several sets of preliminary drawings and views were created at a minimum level of detail to show, compare and validate the different proposals.

Technical drawings

Once the proposal has been selected, a set of highly detailed, technical drawings (plans, sections and elevations) was generated, based both on the photogrammetric 3D model and the preliminary drawings. Since it is possible to design the architecture from zero in the 3D modelling software, this step is not essential. However, we consider that this intermediate step eases the subsequent 3D modelling work, while at the same time providing useful extra information about the Villa’s architecture.

Virtual production

Within the production stage, virtual production refers specifically to all computer-generated elements, including architecture, natural and artificial objects, characters and animations that will be shown in the short film (all of which are commonly referred to as set, assets and characters).

Architecture and other assets

Modelling.Technical drawings —plans, sections and elevations— were imported into Blender and placed in their correct position. Then, 3D volumes were generated for each entity (floors, walls, ceilings and other architectural elements), subdividing and extruding the original geometric mesh, adapting it to the limits previously defined in the technical drawings and using the snap tools to ensure modelling accuracy.

Other types of assets, such as sumptuary objects and jewellery, were also created with a similar approach, in this case using archaeological drawings as references. But because of its higher complexity, the modelling process was in some cases slightly different and required the use of other modelling tools and techniques, such as bèzier curves and digital sculpting. In order to optimise the process —keeping the number of polygons in the scenes as low as possible while showing a high level of detail—, two copies of the same object were created: the first one, more detailed, with a higher number of polygons and a second, lighter copy.

At this point it is particularly important to mention the importance of using a proper nomenclature for each collection and object within Blender, which is especially useful during the subsequent stage of creating and applying textures. To facilitate this task we used the suffixes _HIGH and _LOW respectively.

UV unwrapping and texturing.Blender’s ability to handle UDIM textures, which allow working with different textures and resolutions within the same uv map, helped to optimise each of the 3D models, drastically reducing render times. In order to do that, the basic workflow involved dividing the UV map into different parts, assigning each part a specific resolution. Once the UVs were generated, the models were exported in FBX format to then work with them in the texturing software (Adobe Substance Designer).

Baking.Low resolution models were imported into Substance Painter using custom settings to be able to work with the UDIM textures. Prior to texture painting, the first step was baking, that is projecting the high-resolution model features onto the low-resolution model in order to achieve as much information as possible while using the optimised (lighter) low poly model.

Texture painting.The use of PBR (Physically based rendering) maps is essential at this point, as they accurately simulate a realistic look of materials and textures, as well as the behaviour of light over the object’s surface. Priority was given to photorealism, creating subtle superficial imperfections by applying base materials or textures and superimposing several layers with Substance Designer tools (Smart Materials and Smart Masks). After adjusting each of these layers settings, different brushes were used to manually create small details and imperfections across the model’s surface.

Once this process was finished, textures were exported at the required resolution to finally import them into Blender, where they would continue working with the pieces in terms of distribution, lighting and final rendering within the scene. Thus, again in Blender, we worked with the Shader Editor and a specific system of nodes that allows working with PBR textures through a unique system known as Principled BSDF, which combines multiple layers or configuration of materials ( Fig. 1 ).

Characters

This stage has been mainly done with commercial software and therefore we will limit to briefly indicate the workflow followed in this project to create, clothe and animate the virtual characters.

Modelling.A total of 8 characters have been created with Character Creator, according to the detailed description (age, height, ethnic features or body mass volume, among others) previously provided for each of them during the pre-production stage.

Animation, editing and retargeting.To perform the different characters’ actions, full body Motion Capture systems were used (capable of recording the movement of the body, hands and face), and Rokoko Studio was used for data management. After editing and optimising the generated animations, the armatures and their corresponding actions were exported in FBX format. Once in Blender, the actions and movements were assigned to their corresponding characters in a process known as retargeting.

Clothing.Although Blender has a clothing solver able to accurately simulate textile material behaviour, it is not particularly efficient at simulating complex interactions between clothing and an animated mesh, often generating artefacts and unrealistic results. Therefore, all the character’s clothing was created using Marvelous Designer. This process included two stages: designing patterns and materials for the clothes and simulating their physical behaviour on the character’s animated mesh ( Fig. 2 ).

Integration, lightning, previsualization and rendering

Once the different assets (architecture and objects) and animated characters were generated, they were distributed in 3 scenes within Blender, following the indications described in the shooting script and storyboard generated during the pre-production stage. From this point on, the modelling and animation work was completed and it was lighting that was going to play the main role. At this point it is fundamental to consider which lighting strategy is the most appropriate and the one that best suits the initially proposed ambience. In our case, the scenario is at midday, so only natural light was used. In any case, and in projects of this scale where it will take many days to generate the final images, rendering the images at low resolution to preview the general appearance of the scene is key to avoid wasting resources later on.

The different elements of the scene were distributed in layers to allow further compositing.

The last step is rendering the scenes, exporting each frame in individual EXR image files, at a 8K resolution and setting a high number of samples (the unit by which Blender measures the noise filtering in the final image, based mainly on the number of light’s bounces on each objects’ surface) and the denoise tool, which drastically reduces rendering times. This tool is very useful with static meshes, but when it comes to animated meshes, it must be combined with a high sampling value to avoid artefacts and undesired results.

Colour correction, colour grading, video and audio editing

Once the rendering process is finished, the result is basically a sequence of images. The next step is to compose the whole sequence, modifying aspects related to light and colour to give the final footage the desired look. Finally, the audio tracks with voices and ambient sound were added and synchronised in the final video (one for each language), ready to be integrated in the 360-degrees app.

Extended Matrix

Parallel to the development of the work described so far, another task has been of particular importance: recording and tracking the entire decision-making process, which is key in projects involving heritage virtual reconstruction. The tool used to carry out this process is the Extended Matrix, developed by Emanuel Demetrescu (CNR - ISPC). “The EM allows to record the sources used and the processes of analysis and synthesis that have led from scientific evidence to virtual reconstruction. It organises the 3D archaeological record to make the 3D modelling steps smoother, more transparent and scientifically complete” (Demetrescu 2022a http://osiris.itabc.cnr.it/extendedmatrix/ ).

Basic workflow

The workflow with the Extended Matrix has been widely described by its author in the resources available on the official web site (Demetrescu 2022b http://osiris.itabc.cnr.it/extendedmatrix/index.php/learn-em/; 2022c http://osiris.itabc.cnr.it/extendedmatrix/index.php/nodes-of-the-em/) and in scientific publications (Demetrescu 2015, Demetrescu 2021), so we will briefly describe the steps that we have followed in applying this tool to the virtual reconstruction of the Roman Villa of l’Albir.

Graph editor. As its name indicates, this tool is based on the same approach as the Harris Matrix commonly used in modern field archaeology. It is therefore an ordered sequence of units. So the first step consisted in the elaboration of an extended matrix with a graph editor capable of exporting files in XML. A set of nodes and connectors specifically designed to work within the Extended Matrix can be easily imported in the graph editor.

Blender & EMTools.One of the many advantages of this innovative tool is its seamless integration into Blender as a plug-in. Once the extended matrix is completed, the resulting graph can be imported into blender using the EMTools plugin. Each of the different units is automatically recognised in Blender, so we could work within them to create proxies of the different entities according to their features and properties.

The application of the Extended Matrix in the virtual characters’ recreation

While elaborating the extended matrix for this project, we faced the issue of how to apply it to virtual characters, since each of them has different, source - based attributes (physical characteristics, hairstyle, clothing, jewellery). Even the arrangement of each participant at the stibadium of the oecus followed a strict protocol in a culture as ritualised as that of Rome and well documented in historical references. So the same procedure was applied to the virtual characters together with their attributes and positions at the stibadium ( Fig. 3 ).

App design

Being panoramic, this animated short film is designed to be displayed in virtual reality devices, so it was necessary to design and develop an app that met the following requirements:

Allow synchronisation between the different devices in the room and from another, master device (in this case, an Ipad), in order to efficiently handle groups of people.

To be available in 3 languages: Spanish, Valencian and English.

Graphic design and User Interface

UI has been designed to be simple and easy to follow, which is common to similar immersive projects where the interface and the input device is not a computer screen and a keyboard, but a headset device that fully covers our field of vision. To show users how to proceed, one of the screens is a simple video tutorial explaining what a 360-degrees panoramic image is, how it works and how users can interact with it.

Software development

The system consists of an ipad, 8 Oculus Quest 2 VR headset devices and a router, which is used to deploy a private local network to which the computer and VR devices are connected.

The application is divided into two versions, administrator and user.

Administrator.Is the one who manages every session, creating new ones, controlling the status of each device, launching the video or closing the session.

In the admin mode, and after the start screen, the device connects to the private local network previously deployed and creates a new session, to then proceed to the standby screen, showing a list of every virtual reality device and its current status (that is, the screen that each user is on at that moment: connected/disconnected, language selection, tutorial, or waiting for the rest of participants).

When everyone is on the standby screen, the administrator launches the content and automatically every headset device starts displaying the video. Once it has finished, the app returns to the home screen.

User. Once users have put on the VR device, they will enter the home screen, where they will remain while the device detects a new session created by the administrator and joins it. When users connect to the session, they are taken to the language selection window so that they can choose the language (English, Valencian and Spanish). In addition and for those with hearing disabilities, voices and sounds will be described using text if the accessible content mode is on.

The next screen the user goes to once the language has been chosen is a brief tutorial, where the content to be displayed is introduced and explained.

Once the tutorial is finished, users will enter the standby screen until the rest of participants have joined the session, chosen the language and completed the tutorial. When all users are logged in and the administrator has executed the start playback command, the video will start playing in parallel on all VR headsets.

At the end of the video, credits are shown and the VR headsets automatically return to the standby screen.

Results

The output of the project described is an animated, 360-degrees short film, approximately 10 minutes long, which takes the owners and their guests through the different rooms of a late Roman villa, showing visitors aspects of Roman daily life, but also about the situation of the Roman Empire in the 5th century AD.

These contents, ready to be visualised on virtual reality devices and in different languages, are currently available at Villa Romana de l’Albir Open Air Museum (Alicante, Spain) ( Fig. 4 ).

Conclusions

Making an animated, immersive short film focused on virtual reconstruction and recreation of the cultural heritage is a challenging and complex creative process that requires specialised human, technological and financial resources. We believe that it is a unique and engaging product, able to provide visitors with a singular experience to gain knowledge about history and archaeology.

In order to guarantee a solid scientific basis for the contents shown, it is essential to justify and record each and every decision taken along the process,so that it can be accessed and evaluated. In this regard, the Extended Matrix tool represents a leap forward to efficiently manage the large volumes of data generated during a process of this kind.

In projects involving virtual reconstructions of cultural heritage, the focus is usually on architecture. However, virtual characters are also powerful knowledge and information providers. Therefore, aspects such as their physical appearance, attributes and actions should also be equally researched and documented. And in this sense, we believe that the Extended Matrix is also capable of handling this type of data.

References

Webliography