Category: FMP

Creating Scenes’ Layout

Lobby

For the first scene, my goal was to create a space station or lobby, where the player could admire the wonders of space while also serving as a transition scene between all the major levels. So it had to have some sort of impact on the player. So it served as a great starting point and was my first prototype.

For the basic layout and assets of this scene, I used two very important sources. A YouTube tutorial to create the planet and sky, and a ue5 package I found that had a space station I used in my scene.

Solar System

For the solar system, I wanted to create a standard solar system that looks like the one in my concept drawing. To achieve this, I once again found a very helpful tutorial on YouTube that guided me not only through the creation of each planet material, but it also gave me the real measurements, rotation and scale of all planets in unreal units. Of course, the distance between the planets and the sun is not at the same scale, otherwise it would be impossible to showcase them together.

Mars

For the Mars Scene, I also followed a tutorial and downloaded another package to use for the Mars terrain. I had to adapt it a bit though because the tutorial focuses more on a cinematic sequence. I mostly just used it to find out about the unreal package, which apparently had 90Gb… I just used the assets I needed and then deleted it.

For the sky, I adapted it to look like a Mars sunset.

Black hole

For the Black hole scene, I also followed an extremely complex tutorial in order to create a black hole texture that felt realistic and immersive.

Earth

Finally came the Earth scene. As the closing scene of the experience, I needed it to feel somewhat nostalgic and with an overwhelming sense of smallness when facing the huge infinity of our universe. So I created a huge grassy field under the night sky, where the viewer could wrap up the experience by having a more realistic perspective of the space, and once again feel “back on earth”, as if they took off the headset. To make it visible, I added a small lamp to serve as lighting for the scene.

Lamp asset link: https://sketchfab.com/3d-models/gas-lamp-0a5013605cc848e89ae5b2e3396504da

Grass: UE5 package called “City ENvironment Collection”

Desk and chair

Package decomposed

Robot

3D asset and rigging in blender

Collaborations

Collab 1 – Lecture narration and Sound (and Incorporation)

For my first collaboration, I worked alongside Vlad from BA Sound Arts year 3. He’s collaborated with our course before. Last year, he was with Miranda’s group and they produced some amazing work. This year Miranda recommended him to me so I reached out through whatsapp and our collaboration began.

I am so happy to say that he is a great person to work with! Very proactive and produces great material. Always very self-reflective and looking to improve in any way he can! I couldn’t have asked for a better work partner!

During our work, we shared files and ideas through both Whatsapp and OneDrive, where we uploaded all the videos, scripts and audio files. This made the entire process smooth and hassle-free.

In terms of robot content, he recorded all the scripts I’d sent him, including the voice-over for all the national geographic videos, which he recorded according to the timing of the original ones so that the robot’s acting could match them. It was perfect!

In the end, he produced the narration for the robot Teacher as well as many sound effects I used in my project. And once again I am delighted to say that his work was great and fit my project incredibly well.

Sound Incorporation

There isn’t much to say about the sound incorporation in Unreal. It’s a very easy process that doesn’t take more than 10 minutes. Overall very smooth step of this project.

Collab 2 – Motion Capture (and incorporation)

For my second collaboration, I worked alongside Kabir from BA Acting and Performance year 2. Kabir is also a great worker, very dedicated and talented! Once I mentioned to him we’d be doing motion capture, he became delighted and excited to work with such a relevant medium to today’s media production.

Our goal here was for him to perform as the robot teacher throughout all the scenes, by acting out the movements according to the audio Vlad had sent me. As a result, there wasn’t much preparation beforehand since Kabir didn’t have to memorise any lines. It was pure improvisation, with help from my directing and notes.

We ended up producing 5 takes, meaning one per scene. And I was extremely happy with the results as well. The movement was fluid and he was able to match the audio and body language to the audio seamlessly.

With the help of Kay (Digital Arts Specialist Technician), I was able to have the files exported by him through Rokoko. He sent me everything through teams and was willing to help me with any problems I might’ve had.

issue with animation starting when scene is opened and fix (new menu scene)

Designing interactions, coding and UI prototypes

Lobby

issue with lag and fix. issue with the prototype UI page and fix

Solar system

Issue with teleportation and fix

mars issue with rover and transparent shader and fixes

blackhole

moving player and fix

Earth

issue with too much grass and fix

UI/UX Design and Incorporation

Unfortunately, after looking for UI students who were willing to collaborate with me, I couldn’t find anyone that was willing to give me a hand. I even went with Abhay to their class but neither of us was able to find what we were looking for. And with not that much time left, I had to figure it out myself. So during break, I researched and designed all the UI by myself. To accomplish this, I used Adobe Illustrator to get the best resolution possible and to make it as precise as I could.

In terms of design, I’ve mentioned my inspiration in the previous page, but overall I was trying to go for a modern look reminiscent of Apple UI and the meta’s video UI as well. I worked with darker tones as I feel like it was easier on the eyes, making the entire experience more comfortable and less overwhelming. Plus it just looks more coherent with the darkness of the entire project.

As I started adding the UI to Unreal, I found two big issues.

1 – The resolution was bad, but it was an easy fix. I simply had to make the UI huge and scale it down on the scene.

2 – As I was working with pngs it always had a few resolution imperfections, so I ended up redoing all the UI within Unreal, based on my original illustrator designs.

After fixing these, I simply scaled them down as I saw fit, and transferred all the blueprint functions from the widget prototypes to the final designs. And then I did the same thing for the blueprints that governed each widget. It was a long repetitive process but it presented no issues whatsoever.

And I must say I am extremely happy with the final results that I got. Everything looks crisp and modern, just as I initially planned.

Ambient Sound

COME BACK TO THIS ONE!!!!

Playtests and feedback

Abhay

My first volunteer for the playtesting sessions was Abhay. Overall he really enjoyed the experience, especially the Mars scene and being able to look at the Mars rover up close. However, he had a few notes for me:

• Fixing a repeated title in one of the UI buttons in the Mars scene
• He thinks the screens should be in the middle rather than on the side
• the menu from the solar system scene should be more vertical instead of longitudinal
• Overall, “Very cool”

I think he felt a bit bored watching the lectures though, so I have to consider adding some sort of interaction for the viewer, while each lecture is playing.

Zedan

Zedan came next and was able to give me even more complex feedback and comment on the experience’s effectiveness at standing out as an educational tool that can extended the physical limits of current educational practices. She thought it accomplished what I set it out to do and that the graphics along with the interactions, make for an extremely immersive and complementing asset for education.

As a space enthusiast herself, she thoroughly enjoyed exploring all the assets and looking at them up close, as well as watching each lecture and absorbing its contents. She gave me a few notes:

• Overall, looks amazing and accomplishes its purpose
• Solar System interaction was her favourite but she wishes the planets would spin so she could explore them further
• Slower rover rotation in the Mars Scene
• She enjoyed the lectures and didn’t think they were too long

Herman

Finally, Herman with some of the most relevant feedback. I think he enjoyed the experience in general. But he had a lot of notes to give me, and details I could work on, especially for the exhibition.

But I think the one that stood out the most to me was to have some sort of interaction during each lecture since he focuses better if he has something to do while learning anything. I fully understand this as in my case I tend to draw in order to pay attention or fidget. So I seriously am going to consider this change, even before the submission.

Here is some more feedback:

Changes after feedback

COME BACK TO THIS ONE!!!!

Note: This blog is organised chronologically.

Intro

For my final major project, I intend to create a VR experience within the realms of cinematography. I want to cover this topic while also tackling it within the context of educational environments and how it could prove beneficial or not for students, as a way of complementing their overall content retention and today’s incorporation of education with emerging technology.

Furthermore, it is true that cinematography tends to leave a huge impact on people, and some of the hobbies or interests we pursue are greatly impacted by media, more specifically movies and shows. Just imagine how this potential could be used in VR and how much more it could expand the way we experience that media.

But for now, I think I will focus more on this question in regards to my dissertation. Since the first submission date isn’t until November, that gives me plenty of time to figure out what I could do.

Concept

Through my FMP, I intend to explore the idea of cinematography as a valuable tool for educational settings and whether or not it has an impact on how students (and learners in general) retain information based on visual data, cues and psychology of film-making strategies.

Here are some examples of useful papers I found:

“VR has shown tremendous potential in educating and rehabilitating children (ages 18 and below) with
neurodevelopmental disorders. A study by Wang and Reid (2011) explores the use of this technology, examining
multiple case studies that have tested the efficacy of VR on participants ranging in age, and gender
disorder. These hypotheses were tested for Attention Deficit Hyperactivity Disorder, Autism and Cerebral Palsy.
As a goal, this comprehensive analysis intended to further explore the potential of VR in addressing
neurological impairments in children and contribute to a better understating of its future implications in VR
education.”

This paper was a valuable resource in evaluating education within different neurodevelopmental spectrums. As
a result, it provided me with a crucial perspective on VR education that accounts for other perspectives aside
from those of neurotypical folk. This could help me develop a more accessible experience across a spectrum of
students.

Links: https://karger.com/ned/article/36/1/2/226406/Virtual-Reality-in-Pediatric-Neurorehabilitation

“When addressing filmmaking, the impact of cinematographic practices can vary considerably based on the
medium used to guide the spectator’s gaze. For one, in VR, since the viewer has complete agency over where to
look throughout the experience, cinematographic principles cannot be applied the same way as in 2D screens.
Although these practices may remain the same in terms of taxonomy, the methods used to apply them as a
way of diverting attention within Cinematic Virtual Reality (CVR) change completely. This is due to the viewer’s
agency and the use of 3D immersive environments that enforce the existence of another spatial dimension,
making it a physical experience. This paired with the existence of multiple points of interest (POI) influences the
viewer’s ability to fully absorb the contents of immersive cinematic pieces. Therefore, the focus lies on finding
the methods that allow the creators to direct focus. Further research by Rothe, Buschek and Hußmann(2019),
demonstrates some of the methods proposed, with novel approaches that address the processes within CVR.”

Links: https://www.mdpi.com/2414-4088/3/1/19

For a better analysis of these ideas, please refer to my thesis posted near the end of this page, where I explore these ideas further and create a more supported basis for my approaches throughout this project.

Update 1

I have been reflecting on what kind of cinematic experience I want to have for my project. I think it is logical that it covers some type of school subject since my goal is to make it an educational experience for regular students (or anyone else who is curious and wants to give it a go).

However, I feel like lately I’ve been having somewhat of an artistic block and I can’t think of what I should make. I’ve been brainstorming and I’ve come up with some potential ideas:

• Language experience
• A historical trip through time eras
• Space exploration
• Geographical exploration (potentially using Google Maps in unreal could be cool)
• Anatomy experience

Sadly, I don’t feel strongly about any of those ideas yet and it seems like I don’t really know what I should do… I’ve talked to Eric about this, and he said I should take my time thinking about this and look for inspiration. Hopefully, at some point, I’ll start feeling more drawn towards something.

Side Story

One week ago, a few girls from 3D Animation came to our class looking for people to collaborate with, from BA VR. They presented their idea, which consisted of a cinematic experience in a non-sensical world inspired by Alice in Wonderland. I immediately became extremely excite with the thought of being able to collaborate with them to create something that was still more or less coherent to my plans. But honestly, I think I rushed it a little bit since I was panicking with the fact that I couldn’t find an idea that pleased me at the moment. So I ended up joining their group and changed my original plan.

However, we had a meeting two days ago. This made me instantly realise that it would be a mistake for 3 reasons:

• They had complete control of the project (because it was theirs to begin with)
• I felt like I was getting sidetracked from my original plan
• I started feeling like I did it out of despair and not really for the idea

So, unfortunately, I had to message them saying I wanted to leave the group. And now I am back to square 1. They were really nice and their ideas were great. I simply cannot afford to jeopardise mine, for the sake of feeling comfortable because of their people’s ideas. But I feel like I should still write about it to show that sometimes things don’t go according to plan and we need to stay confident in our ideas and convictions.

Update 2

I have been getting back on track and looking for inspiration, so I can start moving in one direction. And actually, there is one video that really caught my attention! Meta released it in 2021 and it shows audiences a hypothetical depiction of what MR, AR, XR and VR could look like and how immersive they could be when showing people the wonders of space. I thought it was a really smart choice because it’s nearly impossible for the average human being to experience any sort of space-related experiences, and this might be the only way (for now), that we can bring that reality a bit closer to people, with the help of immersive technologies.

When it comes to other subjects such as history or geography, I can visit physical sites and I’ll be “teleported” to that reality because there are still traces of it (not for everything though). What I mean is, it is possible to immerse myself in those subjects through the real world. But not for space. That’s what made the video so enticing!

I feel like this is finally the direction I want to pursue! A VR experience about space, with the goal of not only educating people but also bringing them closer to such a distant and exclusive experience such as exploring space. Now I need to start developing the concept more and figure what the details of the experience. But at least I am finally feeling good about my FMP idea.

Weeks 3 to 6 – Developing ideas and Prototyping

(Note: I am writing this on week 6 so I already have a very solid idea, drawings, prototypes, etc. So I will kind of just skip to the main idea that I arrived at in the end, the feedback from Eric and Herman and my material)

Throughout these 3 weeks, I started to develop my ideas in a more solid and structured manner. My goal was to captivate the viewer with spatial scenarios and bring them closer to it through mesmerising graphics and interesting facts. So I chose to do an experience with three different themes, to bring variety not only to the visuals but also to the content provided to the viewer.

The 3 scenes will be about:

• Mars
• The Solar System
• Blackholes

I thought these would be a great choice since at least two of them are familiar enough to most people that it’s still recognisable and easy to enjoy while still looking amazing. Whereas the black holes scene would be more of an open-ended question since there is still so much to know about them and how they influence our universe. It would end the experience in an open-ended way and leave the viewer expecting more, and hopefully, feeling inspired to want to learn. Like we sometimes feel after watching a great movie (hence the power of cinematography.

I would still want to have some sort of human connection throughout the experience, so I think having someone guiding the viewer throughout the scenes would be a great way to have them connect in a human way as well, which would further captivate them since they’d be able to establish a connection with a mere virtual character.

For the visuals that precede the 3 main scenes, I want to have two other levels. One that will serve as a level selection lobby, where the viewer can choose which lecture to see first. And another one before which takes place “in our actual world”. What I mean by this is a scene that happens in a regular realistic scenario that everyone experiences, school.

Therefore, the experience will start by subverting people’s expectations into thinking they will have a normal school lesson in a boring-looking classroom, with a normal teacher. Which kind of defeats the purpose of VR. Or at least that’s what I want them to think at first. After that scene, the viewer will be transported into the space lobby and the real experience can start.

Also, it is supposed to be a cinematic experience where I want the viewer to be impacted by the visuals, so I don’t want to sacrifice their ability to enjoy high graphics in VR by having too much movement. That would also jeopardise the experience since I’ll be using Unreal which will really push this project to its performance limits. (I’ll talk more about the software below). So the entire experience will have the viewer sitting down in front of a desk (which makes the transition from the classroom environment to space, a lot smoother, since they will both have desks).

Update – Robot teacher

Why stop at the transition between classroom and space? What if the human teacher became a humanoid robot? In my opinion, it would once again impact the viewer once they made the transition from earth to space. No one would expect it, and there would still be a character creating human connection, but in this context, it would shift to something more futuristic. And the fact that this robot teacher wouldn’t exist in real life (at least for now), makes it more exclusive and memorable to the viewer.

I talked about this idea with Eric and he approved it. However, I think he sees it more like a small robot or more peet-like. In my view, it should be humanoid so as to not lose the human side that I want it to have. Besides, it would be much easier to animate with motion capture than moving a robot around, which could give me an unrealistic load of work.

Concept drawing

Initial storyboard

Quick explanation – It starts with the viewer in an ordinary classroom being taught by a human teacher who introduces the theme of the class (astronomy). It then transitions to the space lobby that replaces the classroom and a robot that replaces the human teacher. They give the viewer 3 options to explore: the solar system, mars or black holes. Upon choosing one option, the player is transported to each respective lecture where they can feel mesmerised by the realistic and immersive environments that correspond to each theme. They have an audio lecture and a few interactions at each level that will further teach them about astronomy. At the end of each lecture, they must return to the lobby. Finally, once satisfied the viewer can click on the finish button in the lobby scene and they will be transported “back” to Earth, where the robot does an extremely sentimental and touching monologue about the wonders of space. They then invite the viewer to lie down on the grass with them while gazing at the stars, as it all fades to black.

Lectures

I want to emphasise this again. Besides being a cinematic experience, this project is directed towards education, so I want to incorporate great visuals with interesting, factual and thought-provoking information.

As a result, I want each level to have a small lecture (which will be given by the robot teacher obviously), where the viewer will get to enjoy the environment, as well as learn more about it.

But I feel that simply audio could be ignored so I want there to be visuals, images, signs, slide perhaps, alongside with the audio and the robot. I think that would complement it very well.

Initially, I thought that I could create these lectures with someone specialising in astronomy, but due to time constraints and overcomplicating my work process, I will be using AI to generate the lectures. This could also be useful when writing my script.

Update – Interaction and UI

I feel like this was always on the back of my head, but I think it’s important to fully commit to this idea, as I develop and explain my project. I think the incorporation of good UI design is crucial for this project, as I will be tackling a type of encounter that is mostly based on user experience and comfortability. Therefore, a clear and modern UI seems like a no-brainer.

This will certainly help the viewer smoothly transition between scenes and interactions, while also getting a sense of comfortable user experience through smooth and easy-to-navigate design.

Scenes and respective Viewer Interactions

Scene 1 – Classroom scene

For the classroom scene, I want to create a “typical” sort of boring environment, reminiscent of the ones we can remember and/or sometimes see portrayed in media and films. I want it to have a tremendous impact on the player upon its transition to the space lobby scene.

Here is an AI-generated image of the aesthetic I am aiming for.

Note: This scene has no physical interaction besides UI.

Scene 2 – Space lobby

For the second scene, I want to create a futuristic environment, where the viewer can already get a perspective from space and how wonderful it can look. This scene will serve as both the transition between Earth and space and as a lobby for the viewer to select which lecture they would like to experience. So it should be aesthetically pleasing without being overwhelming, but also comfortable and spacious.

Here are some AI-generated images I used as inspiration.

Note: This scene has no physical interaction besides UI.

Scene 3 – Solar System Scene

This will be the first lecture. In order to make it truly immersive, it will take place right in front of a 3D depiction of the solar system, which the viewers can admire as they listen to and watch the lecture playing in front of them.

Interaction: After this is done, the viewer will have the possibility to teleport in front of the sun or each planet and explore them up close as they get data about it.

Concept drawing

Scene 4 – Mars Scene

For the Mars scene, the viewer will be teleported to a Martian landscape, where they will also listen to and watch a lecture about the red planet.

Interaction: The viewer will get to choose between learning about the Mars Curiosity Rover and its intricacies, or learning about Mars colonization projects and its hypothetical future. For both elements, the viewer will get 3D models which they will be able to analyse and explore.

Scene 5 – Black hole

Finally comes the black hole scene. Similarly to the other two levels, the viewer will be transported to a depiction of the main theme of this level. Meaning they will be in front of a black hole, as they experience the lecture.

Interaction: This one was actually extremely difficult to figure out since I didn’t know what type of interaction the player could have that would be interesting. After talking to Herman and Eric about it, we arrived at the conclusion that being sucked into a black hole would be an entertaining experience to have at this level. It would also teach the player about the physical effects and implications of black holes in matter.

Scene 6 – Earth Scene

This will be the final scene when the viewer returns to planet Earth. It should seem familiar and peaceful. So I decided that a huge grassy field under the starry night sky would do the job. At the same time not deviating from the theme of space exploration and the wonders it has.

Something like this Image:

Note: This scene has no physical interaction.

Initial Game logic

Inspiration

I had quite a few sources that inspired me for this project. First, and probably the most relevant one would be the Meta video I mentioned above, which not only inspired my overall project direction but also the aesthetic I was trying to go for in terms of UI design. Along with Hyper-reality, which gave me a different, yet equally relevant image of UI design in a hypothetical future of XR.

Then for visuals, namely space-related movies, I had a lot of inspiration from both Interstellar and Gravity, which managed to wow the viewer with their overwhelming yet mesmerising depictions of space and its wonder as well as its mysteries.

Finally, Noun Town and Human Anatomy VR are two great examples of gamification of complex subjects and access to very specific and intricate resources through a merely virtual medium such as VR. These were great references for imagining what kind of interactions I would have in my scenes and how I would develop them so that they spark the interest of the player.

Software

Unreal Engine 5

The software I will use for this project is based on what I am trying to achieve (obviously). Since my goal is to get the best and most realistic graphics while getting great performance, the game engine I’ve selected for my FMP is Unreal Engine 5. This is my main tool for the FMP

Blender

The second software I will be using is Blender. This will be for any adjustments of 3D assets, or modelling of simple ones.

Rokoko

Rokoko will be used to record and export motion capture for the robot’s movements.

Adobe Premier Pro (added later)

This software was added later in the project but I decided to include it in this page.

It was used for trimming the lecture videos and removing the original soundtrack

Adobe Illustrator (added later)

This software was added later in the project but I decided to include it in this page.

This software was used for creating UI design

Collaborations

For this project, there will be at least two or three essential collaborations that will help me create a seamless experience while not overloading me with work that I can get done with the help of other fellow students. I will need:

• A Sound Arts Design Student (for overall experience of sound and dialogues)
• An acting student (for robot’s motion capture acting)
• A UI/UX design student (for the experience’s UI Design)

I already have a rough idea of who I can contact for both sound and acting. However, I will need to establish contact with someone from UI/UX design and hopefully get their help as well.

Week 8 – PowerPoint Presentation

Here is my final presentation. Here are things I will not repeat on my blog, such as my research (which is also incorporated in my thesis), I clear explanation of my solidified ideas and respective reasons, as well as prototype videos.

Note: This is also the point at which I chose the title for my project, “Astro Odyssey”. From Astro, relating to space, the planets, stars, or other objects in space, or to a structure in the shape of a star. and Odyssey for a long journey full of adventures.

Scripts

Below are the scripts for the dialogue of the robot teacher throughout the experience. They were created with a mix of my own inputs and also some AI.

Note: Refer to the Update about lectures at the end of the page.

Final Storyboard

There were a few changes that I made to the final storyboard.

• No more classroom scenes (refer to the last paragraph on this page)
• No robot lying down on the grass (to make the experience more coherent and less prone to player location issues)
• In the end, when fading to black, the experience restarts

Synopsis

Here is the synopsis for a clear summary of my project.

Prototypes for First Submission

Here are the videos for the first prototypes of the space lobby scene, using Unreal Engine 5.

I will not get into too much detail on my process here, as I will explain this further on the next page about “Project Development”.

Note: The VR one is a bit laggy, but later on this was addressed and is now smooth as butter.

Thesis and Research for FMP

As mentioned above, my idea for this project was corroborated and explored in my thesis, where I covered multiple aspects of cinematography and their potential impact on content retention for students. For more information, please refer to the document below.

Update about lectures

There have been a number of issues that prevented me from using originally created lectures. Number one was that the information given to me by chatgpt wasn’t completely correct and was omitting crucial points.

Issue number 2, would be creating UI and or videos that matched the speech and gave the viewer interesting and informative data. To do this, I would have to invest way too much time (that I don’t have) on something that isn’t my field of study and that wouldn’t allow me to focus as much on the quality of the overall project. I could have someone do it, but it seems like it would simply give me too many issues and who knows how long I would have to wait.

I needed to make a quick and solid decision to make sure that 1, the information was undoubtedly correct. And 2, the graphics shown along with the info are pleasing and easily comprehensible.

So I decided to switch the lectures to 3 videos by National Geographic, which I found on YouTube and use their script and the videos themselves as material for my own project. There is no better way to treasure the credibility of the information, than accessing that from big scientific sources.

Naturally, I give them credit by not only maintaining the National Geographic logo but by keeping the credits at the end of the video, incorporated within my project.

Update – No Classroom Scene

There was later another decision I had to make in order to save time and keep the focus on the main concept of this project. I talked to Eric about all the elements I would have in that scene, more specifically the met human character, which would have to have their face expressions animated and sewn with the body animation. And as beautiful as it could look, it was just too much work for something that didn’t really represent the main goal of this project.

So, Eric was the one to suggest I remove the first scene (aka the classroom scene) since it wasn’t essential nor completely relevant. He also pointed out that the people trying on this experience will already be inside an educational institution (LCC building), which is enough to make my point, which I agree with.

That leaves me with 5 scenes:

• Lobby
• Solar System
• Mars
• Black holes
• Earth (Final Scene)