The driver's face did the negotiating. Something has to take the job.
Multimodal Interaction · Interaction design · NID Bengaluru · August–September 2026
Intro
On an Indian street almost nothing is signalled by indicators. It is signalled by a face. A rider reads the driver's eyes to know whether the gap is real, a pedestrian steps off the kerb on a nod, a vendor and a cab negotiate a metre of road through posture and a raised hand.
Remove the driver and you have not removed a control input. You have removed the entire negotiation layer that dense traffic actually runs on.
Multimodal Interaction is a studio course at NID Bengaluru on modes, modalities and representation systems. I took the autonomous vehicle brief into the densest street I had access to and treated the eHMI as a language problem rather than a display problem. The constraint I set was that the vocabulary must cost nothing to learn: anything a road user would have to be taught before their first encounter with the vehicle is, on a street like this one, a safety defect.
What came out is a system of six colours, six animation behaviours and four hardware elements, from which every scenario in the project is assembled, plus one scenario worked all the way down to what each channel is doing in each of thirty one seconds.

The cab has no windshield, because a windshield invites a search for a face that is not there. Nothing projects outward, the wheels are side-covered, and the doors slide rather than swing, which deletes the swing envelope instead of mitigating it.
Impact
What the project produced
A four week studio project, so these are outputs rather than measured outcomes. The validation protocol that would turn them into evidence is written at the end, and has not been run.
A signal vocabulary with a learning cost assigned to every element
Six colours, each mapped to Rogers's icon types and scored for what a first-time road user has to be taught. Five of the six cost nothing. The one that costs something, the white scanning sweep, is marked as needing to be taught and is therefore never load bearing for safety.
Green as a single purpose signal
Green appears exactly once in the whole system, for about four seconds, and means safe to move. Never a scan, never a confirmation, never decoration. That restriction is the most testable claim in the project: if green appears in any frame where it is not safe to step out, the frame is wrong.
A separation of request from release
Buckle press, voice and door control all request. Only the corridor scan releases. Three inputs converge on one intent and none of them can open a door, which is the whole safety architecture in one rule.
Seven groups of interaction behaviour
Status, awareness, intent, negotiation, non-responsive actors, contact and threat, including the two cases most AV concepts avoid: what the vehicle does when a crowd has learned it will always yield, and what it does when it has hit someone.
A complete multimodal matrix for the egress scenario
Six cycles by six temporal phases by eleven interaction dimensions, with deliberate silence marked as a decision and unresolved decisions marked OPEN rather than quietly resolved.
Discovery
Why Koramangala
I needed a street where the negotiation is visible because it is constant. If an eHMI survives Koramangala 5th Block at night, the lane-disciplined case is trivial. If it only works in the lane-disciplined case, it is a concept for somewhere else.

Footpath vendors, filtering two wheelers, parked cars narrowing a lane to one vehicle, delivery riders coming up the wrong side, and pedestrians using the carriageway as a continuous surface with the pavement.
The street in plan
- Obstruction · footpath taken by vendors and parked two wheelers
- Stationary · people waiting in the lane, vehicles parked
- Movement · displaced pedestrians, riders taking a gap or the wrong side
- The footpath is not walkable. Vendors, parked two wheelers and shop spill take it, so pedestrians use the carriageway instead. Nothing here is an obstruction to them; it is the route.
- Parked both sides. Nose to tail on both kerbs, which leaves roughly one vehicle of carriageway. The lane is drawn to scale here: that is the width the vehicle has to negotiate in.
- Riders take any gap. Two wheelers filter into whatever space opens, including the gap the AV leaves in front of it. Hesitation reads as a gap being offered.
- Traffic comes both ways. Delivery riders come up the wrong side rather than go round, so an approach from in front is as likely to be a rider against the flow as a pedestrian crossing.
- People stand in the lane. Groups of two to six wait inside the carriageway, usually facing the shops rather than the traffic. The stationary person is the hazard, not the moving one.
Discovery
Five patterns from the street
I observed pedestrian and vehicle behaviour separately and coded what repeated. Five patterns did, and each one invalidates an assumption built into existing eHMI work.

Pedestrian behaviour, coded from the field plates.

Vehicle behaviour, coded the same way and kept separate from the pedestrian set.

The step from what was seen to what it constrains. Hesitation being read as concession is the observation the whole negotiation group is built on.
Where the problem comes from
In front
Almost nothing. The crossing event that front-facing eHMI is designed for barely happens here, because there is no kerb to cross from.
Lateral, left
- Approach is continuous, not a discrete crossing
- People step off a kerb that is already full
- Vendors and parked two wheelers push them out
The AV, in a one vehicle lane
Lateral, right
- Groups of two to six, addressed as one
- Waiting bodies inside the carriageway
- Facing the shops, not the traffic
Behind and alongside
- Riders filter into the gap the AV leaves
- Hesitation is read as concession and farmed
- Delivery riders arriving against the flow
Four of the five patterns happen somewhere other than in front of the bonnet. That is the case against a front-facing display in one picture: the channel most eHMI work spends its budget on is pointed at the one direction this street does not use.
Five patterns, and what each one changed
Observed on the street
The assumption it questions
What it changed in the system
The kerb is not a boundary
A front-facing displayApproach is lateral and continuous, not a discrete crossing event in front of the bonnet.
A transparent beltline running the whole perimeter as the primary light surface, so the signal is readable from the side the approach actually comes from.
People move in groups of two to six
One pedestrian, one signalA message addressed to an individual is read by a group, and a group does not forward it.
Signals addressed to the space rather than to a face: light on the road surface and a beltline any of the six can read at once.
The waiting posture is the hazard
A stationary person is a safe personPeople wait inside the carriageway, facing away from the traffic.
Seen and cleared as two stages that never share a signal. Attention shifts toward the person first; perception is not permission.
Road users are expert, not vulnerable
The protective framingThese are skilled negotiators operating at speed, and a system that treats them as fragile gets overridden.
Escalate in signal and never in motion. The vehicle holds its position and raises what it is saying, rather than yielding to whoever probes it.
Density is normal, not alarming
Alarm statesA vehicle that escalates in traffic it should regard as ordinary becomes noise within a week.
No alarm state in the vocabulary at all. Amber means holding and always carries a visible count, so waiting is information rather than a warning.
01
Observed on the streetThe kerb is not a boundary
The assumption it questionsA front-facing displayApproach is lateral and continuous, not a discrete crossing event in front of the bonnet.
What it changed in the systemA transparent beltline running the whole perimeter as the primary light surface, so the signal is readable from the side the approach actually comes from.
02
Observed on the streetPeople move in groups of two to six
The assumption it questionsOne pedestrian, one signalA message addressed to an individual is read by a group, and a group does not forward it.
What it changed in the systemSignals addressed to the space rather than to a face: light on the road surface and a beltline any of the six can read at once.
03
Observed on the streetThe waiting posture is the hazard
The assumption it questionsA stationary person is a safe personPeople wait inside the carriageway, facing away from the traffic.
What it changed in the systemSeen and cleared as two stages that never share a signal. Attention shifts toward the person first; perception is not permission.
04
Observed on the streetRoad users are expert, not vulnerable
The assumption it questionsThe protective framingThese are skilled negotiators operating at speed, and a system that treats them as fragile gets overridden.
What it changed in the systemEscalate in signal and never in motion. The vehicle holds its position and raises what it is saying, rather than yielding to whoever probes it.
05
Observed on the streetDensity is normal, not alarming
The assumption it questionsAlarm statesA vehicle that escalates in traffic it should regard as ordinary becomes noise within a week.
What it changed in the systemNo alarm state in the vocabulary at all. Amber means holding and always carries a visible count, so waiting is information rather than a warning.
Discovery
Prior art speaks a language that has to be learned first
The audit covered the three families that dominate the literature and the concept work. The common structure is that each one assumes a single attentive pedestrian, crossing in front, in lane-disciplined traffic.
The deeper problem is not the geometry. It is that all three speak in a language the road user must acquire before the encounter is safe. That acquisition cost is invisible in a lab study where participants are briefed, and it is the entire risk on a street where the first encounter is also the only one.
The constraint that fell out of the audit
A signal that has to be taught is a signal that fails on first contact. On this street, first contact is the only contact that matters.
So the vocabulary had to be auditable for learning cost. I used Rogers's icon types to classify every signal element as resemblance, analogical, exemplar, symbolic or arbitrary. Resemblance and symbolic elements that borrow from traffic lights cost nothing, because the road user already holds the reference. Arbitrary elements cost something and must be taught. The rule I set is that no arbitrary element may be load bearing for a safety decision.
That single constraint is what forced the colour lock, and it is why the only arbitrary signal in the finished system, the white beltline sweep, carries a non-critical meaning: the vehicle is alive and scanning. If a road user never learns it, nobody is hurt.
Discovery
Form follows the street
Before any light was specified, the body had to stop making claims it could not back.

The silhouette is deliberately neither aggressive nor toy-like. Aggression gets a reaction on this street, and a toy gets exploited.
Four elements carry everything the vehicle says
A transparent display at the beltline as the primary light surface, an LED strip at the corners that reads as indicators, a digital display that shows numerals only and never sentences, and LiDAR that senses and never speaks.
Keeping the sensing mute is a decision, not an oversight
A sensor that visibly reacts to being watched teaches road users to probe it. On a street where hesitation is read as concession, a vehicle that can be made to flinch has handed over the negotiation.
Solution
One vocabulary, applied without exception
Six colours, six animation behaviours, four hardware elements. Every scenario in the project is assembled from that list and nothing is added for a special case.
Solution
A channel used for everything says nothing
The restriction is the design. From the moment the door cycle begins, informational haptics stop entirely, so that in the ten seconds around the aperture the seat has exactly one thing it is permitted to say.
Music brackets the journey and never enters the hazard window. Green is spoken for and cannot be borrowed for a confirmation. This is the rule most concept work breaks, because every individual addition looks reasonable in isolation. Each extra use of a channel is a small withdrawal from its legibility, and the balance is only checked at the moment it matters.

Waiting for a booking: a slow pulse says it is connected and idle. 
On assignment: the light goes steady. Nobody is driving, and the state says so.

Awareness: attention shifts from straight ahead to the person. Seen, not yet permitted. Perception is not permission, and the two stages never share a signal.
Solution
Point, do not describe
The vehicle's core communicative act is deixis, not description.
Sound is placed at the hazard, arriving from over the occupant's left shoulder and moving as the hazard moves. Light is cast on the ground where the foot goes. A chevron is projected at the oncoming rider and flows toward him, because peripheral vision detects motion long before it resolves shape. A haptic pulse fires on the side the danger is on.
The reasoning is a time budget. Language costs a syllable at a time and the system has 3.4 seconds. Direction costs nothing. A sentence that names a hazard takes longer to parse than a sound that is already coming from it.

Accelerating. 
Transition. 
Slowing. Sound is Doppler-shifted, so speed is heard before it is seen.
Solution
Seven groups of interaction behaviour
The behaviour set is organised by what the vehicle is doing socially, not by which hardware fires.
A note on the boards below, because it changes how they should be read. The scenario frames are generated images, used as scenario illustration. They are not photographs of a built vehicle and no part of this system has been fabricated or road-tested. What is being claimed is the signal design: which channel fires, in what order, and why.

Stuck in crowd

The AV is blocked by pedestrians, parked vehicles and two-wheelers on a congested street.
Engine sound increases

The engine sound gradually increases, signalling the intention to move forward.
Lights shift to yellow

Rear lights transition from red to yellow, creating a visual cue of movement.
People clear the path

Pedestrians and two-wheelers recognise the cue and start making space.
AV moves forward

The AV slowly moves forward as the path clears.

Person blocks the path

The AV detects a person in front and slows down to a stop.
Display a polite message

The eye area shows a friendly message, “After you”, to invite the person to move.
Sound cue increases

A gentle engine-like sound increases slightly, signalling the intention to move.
Person moves aside

The person recognises the cue and moves aside, clearing the path.
AV continues forward

The AV moves forward slowly and safely.

Person or dog detected

The AV detects a person or dog near the road, showing early signs of entering the lane.
Collision predicted

The AV predicts a potential collision based on movement, speed and trajectory.
AV begins to slow down

The AV smoothly begins to slow down, giving space for the person or dog.
Informs following traffic

The rear display communicates the hazard to vehicles behind, clearly indicating that the AV is slowing down due to an obstacle ahead.
Following vehicles react

Vehicles behind receive the warning, slow down and maintain a safe distance, preventing a chain reaction.
Group 7 is the one I would defend first in a crit
A vehicle known to always yield gets farmed, and an AV that cannot hold its position in dense traffic is not safe, it is stationary. The answer is to escalate in signal and never in motion.
Group 6 encodes the opposite instinct
On contact the vehicle preserves sensor, video and audio data and hands it to responders, but it does not adjudicate and does not warn the person it is recording. Detection protects the aperture, it does not build a case. A rider who realises he is being reported may stop next to the person he nearly hit.

Group 6, contact. Stop, protect, document, get help, in that order, and no step in which the vehicle decides what happened.

Group 7, theft. The one borrowed signal everyone already knows.
Solution
One scenario taken all the way down
The signal system is a claim. To test whether it holds under load I took a single scenario and resolved it completely: an occupant steps out onto a congested street, carrying a cake box in both hands, having spent the ride behind dimmed glass, unable to see behind her.
The matrix runs six interaction cycles against six temporal phases, with eleven interaction dimensions as rows: posture, modality favourability, biometric signals, proxemics, gaze, head pose, voice, mid-air gesture, surface touch, foot controls, and system output across visual, auditory and haptic. Each cell answers one question: what is this channel doing at this moment. A dash means the channel is deliberately silent, and silence is recorded as a decision rather than left as an omission.
The constraint that generates the whole thing is that her hands are full and she has no picture of the street. The vehicle's modes exist to compensate for the modalities the street has taken away. Any cell that demands a free hand or sustained visual attention outside the vehicle is a defect, and writing it as a grid is what makes those defects visible.
Solution
Request is not release
In Cycle 4 she presses the buckle. The latch does not open. The belt pays out a hand's width and goes slack across her lap, inside 200 milliseconds, through the one surface already touching her. She asked with her body and is answered in her body.
Buckle press, voice and door control all express the same intent and any one is sufficient to request. None is sufficient to open. Only the corridor scan releases, and the scan is mandatory on every opening including one she explicitly asked for, because the door is never opened by the person who cannot see behind them.
The belt stays latched for a second reason. A released belt retracts into the reel and can no longer communicate, and it is needed as a channel eighteen seconds later, at the one moment in the journey when it matters.
Solution
The hold
Cycle 5 is what the project is for. A motorcycle is detected 38 metres back, travelling at 46 km/h against the flow of traffic in the near-side kerb lane. Time to the aperture is 3.4 seconds. She cannot see it.

Amber with the count inside it, visible from her seat and from the road at the same time. Amber with no number is indistinguishable from a fault, and people walk through faults.
Sound
Low and sustained, spatialised to arrive from over her left shoulder and tracking as he moves.
Haptic
Three firm pulses on the left bolster, the only haptic permitted anywhere in this window.
The belt takes back 15 millimetres
A light tension across the collarbone. Not a restraint, a hand on the shoulder. Fifteen millimetres cannot restrain her and must not try.
Light
The sill goes amber with the count inside it, with the approach chevron flowing outward at the rider.
Four channels fire at once and no single one is treated as sufficient for a safety commit. The voice says “Hold, bike coming up the wrong side. Three seconds.” Two things in that sentence are rules. Name the hazard, because “please wait” gets overridden and “bike coming up the wrong side” is checkable, and a checkable instruction is obeyed the next time. Name the duration, because amber with no number reads as a fault. The voice stays flat and unhurried: three channels already carry the alarm, and a fourth makes the cabin a panic.
Cycle 5 · Hold, as it stands in the matrix
Solution
Geometry before interface
The first safety act is not a signal, it is where the pod parks.
The vehicle positions its body to deny the near-side filtering gap before any door logic runs, ranks candidate stop points with door side outranking walking distance, and shifts the landing zone 600 millimetres past a puddle before the occupant knows there was a decision to make. A gap that does not exist needs no warning, no light and no trust.
At release the door slides rearward to a measured 740 millimetres, the aperture sized to actual lateral clearance rather than defaulted. It tells her how much room she has by being how much room she has. Green and the mechanical click land on the same beat, the only mechanical event in thirty one seconds, and then the interface stops talking.

Group 4, the biker gap, read from above. Never present a gap you intend to withdraw. 
Group 7, the exploitation case: a vehicle known to always yield gets farmed. 
Overtake: the space is yours. 
Hold: the gap is closing. Declared in two seconds and never re-opened.
Solution
The system, assembled

Sixteen frames, every one assembled from the same six colours and four elements. The contact sheet is the test of the colour lock: green appears in exactly one of them.
Pitch film63 sec
Reflection
What the project taught me, and what it has not yet earned
Multimodality is mostly a subtraction problem
The interesting work was not adding channels, it was deciding what each channel is forbidden from saying so that it still carries meaning at the one moment it is needed. Every rule in the system that I am confident about is a restriction, and every part I am least sure of is somewhere I let a channel do two jobs.
I would run the prior art audit before the field observation
I went to Koramangala with a fairly open brief and coded what I saw, and the five patterns only became sharp once I had the existing eHMI assumptions to test them against. Doing the audit first would have let me observe specifically for the conditions those systems depend on, which is a faster route to the same finding.
One element contradicts my own rule and I kept it anyway
The exterior text display survived into the final system for stationary moments, “After you” on the eye display. It assumes English literacy and a reading pause, which is exactly the learning cost I ruled out. I kept it because it tested well as a social gesture in the scenario boards, but the honest position is that it should be replaced with a resemblance-level symbol or restricted to moments when the vehicle is fully stopped and the reader is not in a conflict zone.
Three cells are marked OPEN rather than resolved
The sharpest is override authority: whether she may open on a hold after speaking the named hazard back. My current position is that it is permitted on hold and never on blocked, with the acknowledgement as the gate. The argument against my own position is in the matrix too. Speaking the hazard back takes roughly two seconds, and two seconds is usually the entire hold.
Reflection
What would falsify it
The system is specified tightly enough to be falsifiable, which makes it ready for a validation round rather than another design round. None of this has been run.
Most autonomous vehicle interface work is designed for streets where the rules are already doing the negotiating, and then assumes it will generalise downward into denser traffic. It does not. Starting from a street where negotiation is constant and faces are the medium produces a different system: fewer signals, locked meanings, direction instead of description, and geometry as the first line of safety rather than the last. That is a harder starting condition, and it is the one most of the world's traffic is actually in.
Read alongside
- Chang, Toda, Sakamoto and Igarashi, “Eyes on a Car”, AutomotiveUI 2017
- Chang, “Can Eyes on a Car Reduce Traffic Accidents?”, 2022
- Dey et al., a taxonomy of external human machine interfaces
- Rouchitsas and Alm, on eHMI and pedestrian crossing behaviour; Moore, the case for implicit eHMIs

Next project
Helping hotels see a promotion’s payout before it runs
Scapia · Internship · Web · 2026

