Should AR glasses wait to be asked?Should AR glasses wait to be asked?

The answer is usually on the package. Getting it out is the problem: small print, foreign labels, and facts that say what’s inside but not what it means for you. The usual fix today is the phone. A head-worn display could show the facts while the product stays in your hand, and then the design question is when to show them.

Display glasses are now sold to consumers, so when such an assistant should speak is becoming a product decision. The frame is Horvitz’s mixed-initiative work: a system can wait, ask, or act. The study runs on a Meta Quest 3, standing in for those glasses.

Ask, Offer, or Show? An Exploratory Comparison of Initiation Policies for Product Information on a Head-Worn AR Display.

1. RQ1

   How do the three policies differ in perceived interruption and in welcomeness?

2. RQ2

   How do they differ in perceived control over the decision, correctness, decision time and workload?

3. RQ3

   When do people accept, dismiss, ignore or request the help, and why?

The information, its layout and the gestures are the same in all three. What differs is how the help starts: who picks the moment, and whether a prompt comes first. Earlier studies compared these levels with robot assistants and in written scenarios. Their results point in different directions on whether offering first helps.

1. Evidence

   Estimates, with confidence intervals, of how much interruption and how much well-timed help each policy produces while people choose among physical products in a lab.

2. The instrument

   The Quest 3 prototype and the specification of each policy, published in full after data collection.

3. Design implications

   On when and how an assistant should start helping, drawn from interviews and behaviour logs.

Peng et al. and Kraus et al. tested robot assistants on laptop tasks, and Harari and Amir used written scenarios about work; in Peng’s study and Harari and Amir’s scenarios, the help itself also changed between conditions. On headsets, Sensible Agent offers help and generates its answer after confirmation, but has no automatic version and changes input modality between conditions. Wang et al. compare proactive and reactive help on a Vision Pro, with no offer step.

The open comparison: on-demand, offer-first and automatic delivery of identical content, on one head-worn display, while a person decides about a product in their hand.

20 shoppers. A small, formative sample that shaped the design. Who should start the help came from the literature, not the survey.

1. 19 / 20say being able to control or turn off the information matters
2. 18 / 20find product information support useful
3. 13 / 20already use their phone while shopping for product information, prices or reviews

“If it suggests anything without my asking. Like all the ads we face on screens.”

Survey respondent, on unasked suggestions

My part: I ran the survey, designed the study, and designed and built the Quest 3 prototype.

1. Survey

   20 shoppers on what help they want

2. Study design

   Policies, tasks, products and measures chosen

3. Prototype

   Quest 3 build: held-product tracking, three policies, pinch gestures, logging

4. Pilots

   Technical, then procedural. Questionnaire scoring fixed, then the protocol frozen and pre-registered

5. Main study

   Data collection, following the frozen protocol

6. Analysis & writing

   Paired contrasts with confidence intervals, interview themes