Souren Pashangpour

I'm a PhD candidate at ASBLab, University of Toronto, advised by Prof. Goldie Nejat, and a recipient of the NSERC CREATE Healthcare Robotics (HeRo) scholarship. My research focuses on human-centered control of humanoid helpers for social and physical human–robot interaction in service and healthcare settings. I'm interested in how robots can use foundation models, multimodal reasoning, embodiment-aware interaction, and long-horizon planning to behave safely, expressively, and usefully around people.

Outside the lab, I enjoy weightlifting, archery, Brazilian jiu-jitsu, and being outdoors. I also take pride in taking care of my 1989 IROC-Z28 Camaro, which has become its own long-running mechanical project.

Email  /  Scholar  /  Github

Publications

A multi-foundation-model framework for expressive, interruptible mobile manipulation in human–robot interaction. ExpressMM combines a multimodal perception module, a VLM-based interaction planner, and a VLA-based controller so a TurtleBot4 with a SO-101 arm can interpret speech and visual context, generate expressive action sequences, execute coordinated base–arm motions, and replan when users interrupt or redirect the task. In a live collaborative-assembly demonstration study, participants found the robot's behaviors clear and socially expressive, while also showing the expected caution around trust and safety for physical object-handling interactions.

A multimodal deep-learning architecture for socially assistive robots to recognize and monitor activities of daily living in real time. POVNet+ fuses RGB video, 3D skeletal pose, and object-location features using a spatial mid-fusion method, then uses ADL and motion embedding spaces to distinguish between seen ADLs, unseen ADLs, atypically performed ADLs, and non-ADL movements. In experiments, it outperformed several unimodal and multimodal HAR baselines on within-domain and out-of-domain ADL recognition, and was integrated on the socially assistive robot Leia to proactively initiate assistance in a cluttered home-like environment.

A systematic review of how large language models can be integrated into healthcare robots to help address aging populations and clinician shortages. Surveys current LLM-enabled capabilities, identifies open challenges in safety, grounding, and deployment, and lays out a research agenda for trustworthy clinical use.

An LLM-based architecture that enables a NAO social robot to autonomously generate and deliver a university lecture from instructor-provided notes. LectureGPT segments lecture slides, uses GPT-4 with EmoPrompt to generate natural-language explanations, then applies chain-of-thought prompting to insert synchronized robot action commands into the script so speech, gestures, body movements, slide transitions, and video playback are coordinated through the robot's low-level controller. In a 40-minute classroom HRI study with fourth-year engineering students, the system reduced lecture-preparation time compared with manual robot programming and was generally perceived as informative, engaging, and expressive, while also revealing limits around adaptation, clarification, and measurable learning gains.

A wearable haptic device for AR simulation of plucked musical instruments, using a pair of parallel five-bar mechanisms to move a physical string piece against the user's fingertip and emulate plucking interactions. During my third year of undergrad, I was acknowledged for developing the Python Leap Motion controller code, which handled hand/finger pose tracking, finger extension/contraction detection, virtual-string zone detection, and the high-level state machine that sent serial commands to the Arduino controller to switch the mechanism between rest, engaged, and pluck configurations.

Projects

A custom bread-processing machine for restaurant operations. My role was mainly system design and technical blueprinting: defining the overall architecture, breaking the machine into mechanical / electrical / software subsystems, selecting key components, and guiding early design iterations so the team could move from concept to buildable hardware. I also assisted across hardware and software design, and developed an in-house sensing approach: a rotary-encoder lever gauge that mechanically maps loaf topology so cuts can be adjusted around weak points in the bread and meet slice-height / trimming requirements.

Full perception, planning, and control stack for an autonomous Crazyflie navigating a gate-based racing course. Combines on-board state estimation with a trajectory planner that respects the quadrotor's actuation limits, achieving consistent gate fly-throughs in flight tests.

A distributed intelligent sensor network for comfort monitoring and sleep-onset prediction. ESP32-S3 sensing hubs placed at the bedside, window, and door collect localized temperature, humidity, light, acoustic, BLE/RSSI, and indoor–outdoor gradient data, then stream calibrated and Kalman-filtered readings to a central server over TCP. The server builds a spatio-temporal semantic map using inverse-distance weighting between nodes, estimates room thermal dynamics with a first-order ODE, and uses camera-derived posture/presence labels only as supervision for training a logistic-regression model to predict sleep-related state transitions and recommend proactive climate-control actions.

A hierarchical reinforcement-learning racing framework that combines high-level macro-action planning with low-level vehicle control. A PPO-based strategic layer decides when to continue driving or take pit stops under fuel and tire-wear constraints, while a pre-trained PPO-LSTM visual controller handles steering, throttle, and braking. Trained in simulated racetracks over roughly 28 hours, the system reduced unnecessary pit stops and achieved faster lap times than a human baseline in controlled trials.

A scaled-model gas-powered VTOL device designed to lift personnel for high-altitude operations, including mountainous terrain, high-rise rescue, and disaster response, where ICE energy density can outperform batteries. Completed in four months under the supervision of Dr. Shudong Yu, the system used a two-stroke nitro engine to drive four propellers through a custom bevel-gear powertrain on a 3D-printed chassis reinforced with steel rods, with major subsystems sized using von Mises and Goodman criteria. Bench tests on thrust, fuel consumption, vibration, thermal behavior, and structural FEA characterized the design envelope and identified the OEM clutch and powertrain losses as the primary performance bottlenecks.

Web & Mobile Apps

Excavation and road work crews have to dump their dirt somewhere, usually farmland, but every load needs to be analyzed first, and every truck carries a paper ticket that has to be reconciled by hand. We built a digital system to replace the paper trail: each truck's deposit is tracked, every analysis is attached to a load, and the whole chain is searchable and auditable. Designed for and deployed with Fred's Paving.

Built a custom internal web app with Mahyar for an at-home autism care provider after my wife started working there and we saw they did not yet have software for managing therapist visits. The platform let therapists log in, record home-session notes, document what happened during visits, and sign off on completed sessions, while admins could oversee the records through a separate admin login. It was built to replace ad hoc manual tracking with a structured workflow for therapists visiting autistic children at home.

Built and shipped an iOS habit-tracking app with Mahyar Tajeri, listed on the App Store under GoHabits Inc. as a productivity app with the tagline “Craft your dream life.” GoHabits helps users build routines, track habits, journal progress, and stay accountable through community features like a social circle, home feed, profile pages, habit pages, comments, group chats, and upcoming “Monk Mode” / GoAI features. We also showcased GoHabits at Collision 2023.

Miscellanea

Template adapted from Jon Barron's website.