ep. 51. Thriving in Emerging Tech - Series Kickoff: XR Career Spotlight with Design Leader Jordan Harvey
10 min read
How can industry practitioners, such as designers, product managers, and user experience researchers, lead the creation of human-centered emerging technology?
Our recent Problem-Solution Symbiosis (PSS) Framework series spoke to this question, providing practitioners with practical tools for building human-centered AI solutions. Today, I’m launching a new series focused on career growth, helping practitioners thrive in the evolving emerging technology landscape.
This episode was inspired by my role as an educator at UC Berkeley’s School of Information. I teach Introduction to User Experience (UX) Design to professional Master’s students, most of whom aspire to shape careers in emerging technology.
While I share my own career journey as a design researcher focused on bleeding-edge tech, I want students to hear career stories from other perspectives. Today, we explore how Jordan Harvey, an emerging technology design leader, navigated his career, including designing for virtual and augmented reality (VR/AR) since 2014. We share takeaways that can apply to any practitioner (not only designers) looking to enter the extended reality (XR) space.
You might remember Jordan from a previous episode, on his approach to design innovation. Our conversation was so rich that we split it into two posts. Jordan is a designer and futurist with over 20 years of experience in building bleeding-edge technology. He is the CEO and Founder at design technology company Remote Control (RC), whose portfolio includes partnerships with Meta, Peloton and Apple. He’s also an Adjunct Professor at his alma mater, the School of Visual Arts in New York. Let’s dive in!
This interview has been edited for length and clarity.
Designing for the Future: Jordan’s Career Journey
We’ve previously talked about how you build at the bleeding edge of technology with RC. Let’s switch gears, diving into the journey that led you here. How did you first discover the design field?
Jordan: My Dad is a homebuilder. We grew up in that environment, going to the job site with our Dad. He would give us a hammer, nails and wood, and we would build little wooden tanks out of the scrap wood.
One fun memory from childhood was going to Kings Island amusement park in Cincinnati, Ohio. My mom would take us there all the time - we would come home so, so excited. We’d lay out these sheets of paper all over the bedroom floor, and just draw these huge bird’s-eye perspective maps of the park from our own imagination. From very early on, they promoted exploration and creativity.
In high school, I found hands-on subjects like science, gym and art to be very engaging. I excelled in art, and a couple of great art high school teachers promoted me to look at art as a career. I was drawn to the culture of New York City, and found the School of Visual Arts (SVA). I enrolled in Sinclair Community College in Dayton, Ohio as a bridge, and transferred to SVA. I studied Design, Animation, and Visual Effects.
What happened next?
Jordan: While in school, I got a night shift job at the production studio Psyop. I’d finish class at 9pm, do rotoscoping for visual effects and animation commercials during the night until 3 or 4am, sleep for three or four hours, then go back to class.
I also made a short 3D animated film in college on my own. I went in with the intention of developing a holistic skillset. If you’ve gone through all different phases of the process - modeling, rigging, animation, lighting rendering, compositing, roto, then you really know what you’re asking someone else to do when you’re directing those jobs.
I came out of school as a generalist, understanding a lot of different aspects of the pipeline. I was always bouncing around, doing different things, just trying new stuff. Some advice for young creatives: Don’t be looking for the end game in college: “I’m going to graduate and it’s over”. Keep exploring and be curious.
There was an important moment around 2010, where you married the 3D animation pipeline with experience design. What did that look like?
Jordan: Around then, Psyop Games was founded. A good friend of mine now, Brian Kehrer, came in to launch that initiative. That was the first time I had kind of started to get exposed to experiential and interactive aspects of this trade, and never looked back since. Brian and I built Nightmare Malaria for the Against Malaria Foundation with a small team.
I did all of the level design. The psychological analysis aspects captured me. It was so engaging to design these levels of varying difficulty and just watch someone play. Don’t tell them how to play it, just watch. I really learned that through interactive design as a communication you can really get to know someone pretty well if you’re allowed to watch them interact with your product.
A few years later, you founded Novo Reality, building experiences with VR and robotics back in 2014-2015. What led you to focus on that intersection?
Jordan: I left in Psyop in 2014 and went to Firstborn, where I helped them do a lot of VR stuff. We did some work with Patron in 2014, doing a VR fly-through of their entire tequila manufacturing facility. I meticulously measured every one of these spaces, sketched them and rebuilt them virtually. It was one of the first VR films with integrated CG as well as the first to be filmed on a drone. I went on to work with The Mill and other companies doing VR work very early on.
Because of my work on VR, I was contacted by Justin Meltzer, entrepreneur and graduate of U Penn’s Wharton School. He was looking to start a VR venture around making a motion simulator for VR films, and I had been working on a lot of VR. Now, even though people were starting to tell stories in VR around this time, I knew we still had big problems to solve around motion sickness. Back to my four pillars of design (usability, economics, functionality and aesthetics), if half your viewers are motion sick, you’re not going to have a successful business.
Novo Reality was founded on the idea of moving a person physically through space in the same way that they visually see themselves moving with space. You create a match in the sensory system, and you eliminate some of the problems that cause motion sickness for people in VR. The user’s body moves with inertia - for example, you slam the brakes and you slide forward in the seat - your body is going to stiffen up. There were a lot of chair simulators but they were more about rocking movement, unable to match that inertia.
I found a defunct car manufacturing plant in Detroit selling a bunch of robotic arms that can pick up cars. I thought, you could probably pick up a person and move them, and you could probably put a computer on it too. Why don't we buy one and see if we can do it.
I then contacted a company in Ohio that did amusement park ride engineering. I had them engineer a big riser stand that lifted the robot up so it kind of opened the envelope of motion, and built this massive platform so it wasn’t anchored to the ground. I built a VR computer by hand, fit it under the seat, and had a Rift headset connected to Unreal Engine. We had to hack the robot to send a custom signal to and from Unreal Engine to synchronize the motion simulation with the virtual experience.
All in all, your sensory system was aligned with what you were seeing. We put the robot in a warehouse in East New York in NYC. It had 30 foot ceilings and the robot could take you to the top in under a second. It was fun! We had this abstract experience where you're just riding through these geometric shapes just to test the functionality, see if it's working. We threw a big party with a DJ, and just user tested people on it.
After the success of that party, we decided to make an animated film about these construction robots in the future that are kind of like wisecracking AI Robots that are building a futuristic city. We threw a bunch more parties, and people were riding around with one of the hero characters from the film. It was a great success, people really liked it.
It seems like your mental models from 3D animation, film and interaction design all served your work with VR. I think that’s something important for young designers interested in XR in general to hear - how these different skillsets helped you build these compelling experiences.
Jordan: For XR, It’s important to understand the mechanics of how software development works, the mechanics of good design. To understand the mechanics, you also need to do it - realize what works and what doesn’t work. When you start sitting in rooms with 30 people who are the best of the best in the world, you can’t make assumptions and fake the funk. You can’t step into a professional baseball team without having played baseball: “Oh, I get it, you swing the bat, throw the ball”. You might understand how a game works, but playing the game is a different rule set. It’s the ingrained reaction trained in your mind and your muscles. I think the generalist aspect of design gives you exposure to understand mechanics of how things work. You’re going to find your expertise and your own area to own. But first, you have to walk the walk.
Takeaways
Students frequently ask me, “What should I study to become an XR designer/design researcher/product manager/etc?” To answer that, we can look at some takeaways from the insights Jordan shared:
Take a Holistic Approach to Building Your Craft
Jordan intentionally took on projects and jobs to explore the end-to-end 3D animation pipeline, building a holistic skillset. This not only would scaffold his later work in interaction design and VR, but also provided him with a deep understanding of different creative roles - something that would be invaluable when building and leading teams as CEO and Founder of Remote Control. If you work in a field other that design, this might look like taking time to understand the end-to-end product lifecycle or tech stack, rather than specializing early on.
Design, and Business, and Engineering, oh my!
Going back to our previous interview with Jordan, we learned about his four pillars for bringing an idea to fruition: usability, functionality, aesthetics and economics. It’s important to gain as much understanding as possible across all these pillars. This could look like taking some entrepreneurship courses and engineering courses, or having informational interviews with people working across roles (e.g., product management, marketing, sales, etc). You may be able to work with experts focused on each of these roles in the future, but having some knowledge can help you switch between different lenses and more effectively collaborate in the future.
Start Making Stuff
Whether it was building wooden tanks at his father’s job sites, making a 3D animated film or seeing how you could combine VR and life-sized robotic arms, Jordan’s career is marked by hands-on learning and experimentation. If you’re curious about XR, get into the tools and start playing. This will help build intuition and help you understand a medium like XR at the mechanistic level. Some good places to get started that are either free or include free options include: Blender (open-source tool for learning 3D), Adobe Aero (no-code tool to get started building AR), Shapes XR (spatial design and collaboration tool), and Unity (free, game engine for 2D, 3D, AR and VR, having some C# coding background is a plus).
⏭️ Next Week
I’ll start unpacking why and how practitioners should shift their mindset from ‘design as problem-solving’ to ‘designing emerging technology within systems’ to prepare for future challenges and opportunities. Stay tuned!
Human-Computer Interaction News
AI+Education: How Large Language Models Could Speed Promising New Classroom Curricula: Stanford computer science scholars propose using language models to create new learning materials for K-12 students.
Addressing Accessibility Challenges of Generative AI Tools for People with Disabilities: A study published at NordiCHI '24 critically examines the persistent inequities between the rapid development of genAI technologies and the accessibility needs of people with disabilities in a professional setting.
Machines of Loving Grace: Anthropic CEO Dario Amodei paints a hopeful potential future of AI-driven breakthrough in fields like neuroscience and economic development. His vision relies on “powerful AI” (read: AGI) that requires advanced reasoning. Compare with Apple’s recent paper that found no evidence of formal reasoning in LLMs, and pioneer Yann LeCun’s thoughts on AI being “dumber than a cat”. This is all a good reminder to temper expectations about AI’s capability ceiling. Instead of doubling down on the singularity, let’s envision potential futures where AI helpfully extends - rather than replaces - cognition.
Designing emerging technology products? Sendfull can help you find product-market fit. Reach out at hello@sendfull.com
That’s a wrap 🌯 . More human-computer interaction news from Sendfull next week.