SP’s Digital Media & Communications (DMC) course covers a wide range of topics to prepare us for work in an increasingly digital world. From digital marketing, advertising, public relations, data analytics to all sorts of tools for content production, we have a rather extensive “set meal” during our 3 year program.
However, I was particularly looking forward to Tinkerlab, a transdisciplinary module for MAD students. Tinkerlab’s goal is to have students focused on solving real-world problems with various technology, bringing together diverse perspectives and expertises from students from different specialisations of MAD.
Our teacher, Ms Jennifer allowed the class to sort themselves into different teams. I was surprised how fast the teams formed up, leaving a few stragglers behind. But, no matter, I would approach the remaining students and together form a team. Our team of “misfits” comprised of x3 DMC students; Ethan, Farzana, and myself, x2 Story & Content Creation (SCC) students; Huiyuan and Yvonne, and x1 student from Sound & Music (SM), Lincoln.
Identifying our problem statement
Our team pondered over what would be a problem worth solving. Eventually, we agreed on one thing that really bothered all of us. That is, the miserable state of the recycling bins within SP overflowing with items that were obviously not sorted properly. It was a disheartening yet common sight that needed to be fixed. This common apathy among my teammates became the fuel that would keep us working hard on this project.
We came up with many different ideas. We ended up with pitching a recycling app that incentives users to recycle more. The niche feature was to have a 3D model pop up over the recycle bins. However, Ms Jennifer's feedback that just having a 3D pop-up model does not solve the problem. She challenged us to add some interactivity, e.g. incorporating a talking 3D mascot and actually creating a working prototype.
Challenge accepted! The team relooked at our idea and came up with a 2-phase campaign.
Phase 1: Recycle with Jumba!
To create a working prototype that allows anyone who approaches a recycling bin (with our special QR code) to see 3D models hovering above the recycling bin and learn from Jumba (SP’s lion mascot) how to recycle correctly. Anyone can scan these QR codes using any smartphone (Android or iOS).
Phase 2: Recycle with Jumba & Win!
Users can do the same as Phase 1 but now via a dedicated app. Every time a user recycles correctly, they can earn points via the “RecycleGO!” app. This gamified approach contributes to both individual and school-wide totals, with prizes for top recyclers. The plan is to inculcate the habit of recycling among SP youths through these intended interventions, i.e. taking baby steps spurred on by little incentives, in the hopes of inculcating into a habit.
Applying what was taught in class
One of the requirements of Tinkerlab is to put into practice what was taught in class. At first, I was daunted by the many tools that were introduced during this course. But in hindsight, I realised that this was deliberately done, so as to arm students with as many “weapons” as possible.
In addition, the team also considered other technologies, including 3D scanning, motion capture and virtual production, but ultimately ruled them out due to time constraint or feature limitation. Instead, we turned to other technologies not taught in class, such as Figma and Dimensions to complement Aero & Elevenlabs.
Learning and applying a new skill
As with real-world scenarios, we would soon encounter our first tough challenge. Our team members are skilled in video post-production, music and story writing but none of us were skilled 3D modeling or AR development. No one even knew how to import a 3D model!
However, I saw this mini-crisis as an opportunity for growth and volunteered to take on the "builder" role. My dad once mentioned to me that when an opportunity occurs where I had to pick up a new skill in order to complete a task, quickly seize it. How fast one can pick up the minimum knowledge so as to deliver a Minimal Viable Product (MVP) is going to be a key skill, he emphasised. So, I took this as an opportunity to test my resourcefulness and personal resolve.
Over the next 2 weeks, I immersed myself in YouTube tutorials on Adobe Aero & Adobe Dimension. Here's a snapshot of what I learned:
Adobe Dimension: I first utilised Dimension to create and prepare our 3D assets. It was essential for optimising and exporting 3D models in the .real file format required by Adobe Aero. This helped us greatly as I found that many other platforms do not support exporting to the .real format, making them unsuitable for our project's AR component.
That left the next big challenge… creating “Jumba”. Even though Mixamo offered many models, all the lions looked too “scary” and “menacing”. I would expand my search to Sketchfab where I would eventually find the perfect (cute & free-to-use) 3D model of a humanoid lion, which I uploaded into Dimension & transformed it into Jumba by changing its texture, color and attire.
Adobe Aero: Next, was to leverage on this platform to form the core of our interactive experience. Aero allows us to integrate 3D models into the real world through augmented reality. Aero's ability to use surface anchors allowed me to adapt the varying sizes and shapes of recycling bins, ensuring a consistent user experience regardless of the specific bin location.
With only 3 weeks left for the team to develop the prototype, I did not have the time to build my 3D models from scratch. Instead, I leveraged Sketchfab for ready-built & copyright-free assets (e.g. bottle cans, rubbish bins, bottle cans, paper, etc.), customising textures and colors as needed. Given the time constraint, I felt it was the only way we could deliver a working prototype before the deadline. I appreciate the original creators who generously shared their work under creative commons and made sure to attribute them in my presentation slides.
Some other technical lessons for the geeky ones reading this blog post…
File Formats: Adobe Aero requires .real files, which are created by exporting models from Adobe Dimension.
Adobe Dimension Integration: Allows users to manipulate dimensions, positions, colors, and even apply textures to 3D models within Dimension.
Aero Animations: Allows users to create animations within Aero using a block-coding-like system, defining sequences of actions for the models.
Interactive Elements: Allow users to link models to audio files, URLs, and different anchor types. I used the built-in QR codes as a gateway to interacting with our creation.
Anchor Points: Anchors allow users to place the 3D models on any suitable horizontal surface above a bin. The team scouted the actual locations, measured the different surfaces and variability of recycling bins, before finally opting for surface anchors.
Team RecycleGO... Assemble!
While I focused on the 3D and AR elements, my teammates worked hard in their respective roles as well. Here is an overview of what everyone did:
Name | Role | Responsibilities |
Huiyuan | Team leader | Delegated tasks to team members and kept a close watch on our timeline and milestones. |
Howie | 3D digital modeler | Created the AR recycling experience. Farzana and I worked closely to create the prototype. |
Farazana | UI/UX designer | Created the wireframe of the RecycleGO! App. |
*Yvonne | Scriptwriter | Mapped out the flow of the instructional video. |
*Ethan | Video Editor | Created the storyboard, filmed, and edited the instructional video. |
*Lincoln | Sound Recordist | Enhanced the video with appropriate music & sound effects. |
*Yvonne, Ethan & Lincoln were responsible for creating the instructional video on RecycleGO!
ElevenLabs: To bring our mascot, Jumba, to life, we employed ElevenLabs' text-to-speech capabilities. This allowed us to use an AI-generated voice for Jumba, enhancing the interactive and educational aspects of the experience. In hindsight, we could have chosen a more cartoonish voice to better fit our mascot. Maybe for version 2.
Figma: Our team used Figma for designing the user interface and overall user experience of the accompanying score-tracking app. This ensured a cohesive and intuitive experience for users beyond the AR interaction with Jumba.
Lights, cameras, action!
On the day of the presentation, our team was well prepared. We rehearsed our scripts well before hand and brought in props to make our presentation as engaging and interactive as possible. Specifically, at the presenter’s table we placed 4 miniature bins, while on each of the 4 audience’s tables, we placed an iPad mini (Adobe Aero currently only works on iOS devices). This allowed the audience together with our instructor, to experience our AR creation first hand.
Ms. Jennifer was our instructor and role-played as our simulated client. She was supportive and played along as I interviewed her to start off the presentation. After a short dialogue, it allowed me to segway nicely into the problem of lack of recycling awareness - that was the problem that my team was going to attempt to solve that day.
Through the surprised smiles of the audience, my team knew everyone enjoyed our presentation. It validated all our hard work! I had collected yet another meaningful learning experience at SP.
Final thoughts
My recent Tinkerlab experience reminded me of a YouTube video from many years ago where Katie Bouman and a team of astronomers came up with a clever way to solve a seemingly impossible task. That video left a lasting impression on me.
While my team may not be as high-powered as Katie’s, I see similarities in my team’s common resolve and commitment towards solving a common problem. Along the way, it also reminded me again of what can be achieved when combining different talents and technologies together. Hence, it's no secret that I enjoy teaming up with new people to take on new projects. This Tinkerlab project wouldn't have been as fun if the team was not so passionate and committed. I am proud to be part of team “misfits”..., I mean Team RecycleGO!
Chia Howie
24 February 2025
TLDR
Teamed up with students from different MAD disciplines, used different softwares to create an AR working prototype app, all within 6 weeks.
Comments