data teaches Engineers presentation skills
data teaches Engineers presentation skills
Create a tech-enabled service student's will be eager to use that improves communication skills.
An online platform connected to a presentation practice space where students can get automated, peer, and expert feedback and a portable system for similar in-class use.
For an idea to shape the world, it must be shared with others. Carnegie Mellon undergrads are excellent at developing these ideas but often lack the acumen to effectively communicate them. Carnegie Mellon easily affords students their first jobs, but sees them struggle to move up the career ladder, as evidenced by salaries that are top in their class right out of school but lackluster mid-career.
Campus leaders from our client, the Simon Initiative, fault the emphasis placed by the school and students on technical competency for displacing the value of soft skills, like communication, which are often not valued in course grading. While various support services exist on-campus, students don’t feel they are personalized or convenient, choosing instead to find an online tutorial or suffer in silence. Our team set out to look at the factors that lead to stunted communication skills to inform potential interventions.
We learned about the current landscape through interviews with students, alumni, and staff. We also looked at how peer institutions handle communication skill development, such as career and communication centers, alumni interaction, use of peer review, and the frequency of writing assignments.
We found technical students take a formulaic approach to writing, often failing to include big ideas to form a narrative. Although services exist on-campus to help these students improve, students either don't know about their offerings or yearn for more personalized experiences.
On the contrary, one particularly successful service works with students for whom English is a second language to improve their communication. One course about presenting allows these students to frequently practice presenting in small groups. They then give a final presentation that results in a video recording and detailed instructor feedback. However, there is no similar service for students who are native English speakers or want to improve a specific presentation.
We saw presentations as an area of opportunity ripe for innovation due to a lack of current offerings and ample room for technical and service innovation. An early idea that got us excited about this area was a Vine or Snapchat-like service for broadcasting practice presentations then receiving feedback. We decided to pair our interest in presenting with a focus on Engineering undergrads, as they have limited requirements for communication skill development (a single English course), yet often present design projects in class.
To learn more about this specific need, we interviewed a variety of stakeholders across campus to learn more about engineering students, their program, and current support for the development of presentation and speaking skills.
We found presentations in Engineering courses are evaluated for content but not delivery. There are connections between improving presentation skills and job interview performance. During job fairs and interviews, students speak objectively about their experiences, omitting reflection on their challenges and growth. We learned students will only spend their time at an event or service if it will improve their class performance or get them a job.
After understanding the current state, we brainstormed potential concepts, deciding to explore these:
We storyboarded out high-level interactions for each idea so we could get feedback from students, staff, and faculty.
We used the needs validation technique from the speed dating method to get feedback on the storyboards. Engineering students weren't comfortable with concepts like role playing, which put them in the spotlight. They also weren't willing to spend a full weekend at a hack-a-thon related to presentation skills. Students and staff alike were excited about the virtual practice environment because it's something they were excited to try. The concepts that allowed students to get multiple perspectives using technology were very successful. This led us to focus on a solution that made use of mediated practice and tied in crowdsourced peer feedback.
To prepare for the user enactments, we created the artifacts that students would interact with as part of Echo. This included a presentation simulator for the practice space that that used the Wizard of Oz technique to "automatically" analyze participants' presentations and give live feedback. We also made a template for a feedback form that we could fill on the spot and show to participants.
We prototyped the practice room experience and the in-class feedback scenario with six students, giving each context for the simulation and asking them to think of a topic. They presented in the conditions of their respective scenario while we recorded metrics and critique. We quickly put this data into a feedback report then discussed it with participants to find out what was most helpful and what could improve.
Feedback from our user enactments led us to three factors students found most rewarding about Echo:
We improved pain points that came up during the enactments. Students who struggled with filler words like "uh" were paralyzed each time an alert came up during the simulation. We decided to only display live feedback about factors the presenter could improve in the moment, such as volume or speed. We saved information about more difficult fixes, like addressing filler words, to the feedback report.
After validating and improving our service, it was time to communicate our solution to the client so they could make it happen.
The efforts of two groups of people must overlap to successfully deliver Echo. Facilitators allow the service to exist by providing funding, administration, and technical infrastructure. Diverse feedback providers from across campus are at the heart of the service, evaluating a presentation's feel and content and advising about delivery. Echo gives students a "high-density" experience because it mobilizes multiple resources and actors in a short amount of time.
To demonstrate the technical feasibility of the automated feedback portion of our solution, we looked at current speech parsing and body language analysis tools, such as Ummo, CMU Sphinx, Microsoft Kinect, and LumoLift.
After our final presentation, guests from the Simon Initiative commented that this project is "very CMU" due to its use of technology and data to solve a problem. They agreed students would appreciate this mix of quantitative and qualitative feedback, making an ephemeral act concrete. They immediately asked about the service's operational and development costs, a sign that they want to see Echo exist. The presentation was shared with university leaders who were also supportive of Echo and are looking to build the service and host it within the campus library.