Human-Computer Interaction
See recent articles
Showing new listings for Monday, 16 November 2026
- [1] arXiv:2611.17355 [pdf, html, other]
-
Title: Everything is an Operational View: UI as a Projection over Context in Agentic SystemsComments: 9 pages, 2 figures, ongoing workSubjects: Human-Computer Interaction (cs.HC); Software Engineering (cs.SE)
The emergence of generative AI has enabled Generative User Interfaces (GenUI) that dynamically construct interfaces from user intent, yet existing approaches rely on request-response pipelines that fragment context and limit transparency and control. This paper proposes AFS-UI, which reconceptualises UI as projections over context built on an agentic file-system abstraction that organizes data, capabilities, and interaction state within a unified, persistent workspace. In particular, AFS-UI addresses key limitations in current systems, including heterogeneous resource discoverability, incompatible operation semantics, tangling projection and context, rigid source composition and substitution, and limited evolvability. Within this framework, interfaces become dynamic, inspectable views over shared resources, supporting both human and agent interaction through a common operational surface.
- [2] arXiv:2611.17490 [pdf, html, other]
-
Title: Learning Recommendations under Imperfect Human Compliance with Record-Level Differential PrivacySubjects: Human-Computer Interaction (cs.HC)
This paper investigates human-in-the-loop recommendation systems, where effectiveness relies on human compliance and privacy concerns restrict the use of historical response data. The pro posed framework models the implemented action as a random response to the recommendation and formulates the recommendation mechanism as a decision-dependent optimisation problem under an unknown compliance distribution. A batched Gaussian process upper confidence bound algorithm is then adapted to learn from locally perturbed historical actions, providing privacy for individual raw records. The analysis establishes record-level differential privacy and derives high-probability simple regret bounds when the compliance distribution is unknown. These bounds characterise how privacy noise, batch size, and behavioural information affect the guaranteed quality of the learned recommendation. In particular, a known upper bound on the non-compliance rate enables the rec ommendation space to be screened before any private data are queried. Simulations of a fixed-state car-following problem with synthetic historical driver responses demonstrate the effectiveness of the proposed algorithm and verify the theoretical findings.
- [3] arXiv:2611.17830 [pdf, other]
-
Title: Alleviating Interview Stress: Effects of Robot Persona and Reactive Backchanneling on High-Pressure Mock InterviewsComments: 4 pages, 3 figures, 2 tables. Accepted as a Late Breaking Report (LBR) paper at the 2026 IEEE International Conference on Robot and Human Interactive Communication (RO-MAN)Subjects: Human-Computer Interaction (cs.HC)
High-pressure job interviews induce anxiety and impair performance. Existing mock training suffers from a trade-off among cost, standardization, and physical presence. This study investigated how a robot interviewer's verbal persona (supportive vs. challenging) and nonverbal backchanneling (active vs. negative) affect users' psychological states and facial expressions in a simulated interview. In a 2x2 between-subjects experiment (N=28), participants interacted with a NAO robot. Subjective measures (state anxiety, self-efficacy, social presence) and objective facial emotion recognition were collected. Results showed that a supportive persona reduced task load, increased perceived robot warmth and mind perception, and decreased perceived surveillance and robot discomfort. Active backchanneling increased non-neutral (p = .021) and anxiety-tension facial affect (p = .048). A significant interaction effect between persona and backchanneling was found on social presence (p = .045). The subjective-objective consistency was 67.9%, indicating complementary measurement modalities. The findings provide preliminary support for the social buffering hypothesis in human-robot interaction.
- [4] arXiv:2611.17874 [pdf, html, other]
-
Title: What a Memory Preview Cannot Tell You: Ambiguity in AI Memory PermissionsComments: 4 pages, 2 figures, 1 table. Code and data: this https URLSubjects: Human-Computer Interaction (cs.HC)
A memory rule states when an assistant may reuse a remembered event, and an editor can confirm a rule by previewing one use it allows and one it blocks. We ask what such a preview lets a person infer about the rule they saved. For an editor with six or twelve valid rules, we enumerate every rule compatible with a preview. Splitting the rules most evenly does not target the question a user cares about. For an explicit preference, two adaptive tests leave all three recommendation scopes open (for a one-off choice the outcome turns on a tie), while one test of transfer to another task settles the question. Weak previews are also the default. Under the intended rule, 36 of 39 possible allowed/blocked pairs contain a blocked example that no rule allows, and no pair identifies the rule. Three or four tests suffice here; when permissions depend on each other, a fixed checklist grows to six or nine and adaptive questioning needs at most five or seven. These counts use a specified rule vocabulary and fictional events. In a small pilot, 10 of 20 people who saw such a weak preview of a rule they had just chosen mispredicted, or were unsure about, at least one of two uses, and none of those 10 said the outcomes matched what they wanted. Whether better previews help people is untested.
- [5] arXiv:2611.18230 [pdf, other]
-
Title: Personalization of Basic and Moral Values Guiding LLM-Controlled Assistant RobotsComments: 18 pages, 6 figures. Version accepted for publication in: Value Engineering in Artificial Intelligence: Fourth Inter-national Workshop, VALE 2026, Bremen, Germany, August 15, 2026, Proceedings. Work-shop affiliated with 35th International Joint Conference on Artificial Intelligence (IJCAI-ECAI 2026)Subjects: Human-Computer Interaction (cs.HC); Artificial Intelligence (cs.AI)
Personal assistant robots powered by large language models (LLMs) are likely to enter everyday life within the next decade, but the values that should guide their behaviour remain largely unspecified. Current alignment practice steers a single model toward an aggregate of human values, ignoring that individual users may prefer a robot whose value profile differs from their own. We administer parallel self- and robot-targeted versions of three established psycho-metric instruments - the simplified Schwartz Values Survey (SVS), the Portrait Values Questionnaire (PVQ-21), and the Moral Foundations Questionnaire (MFQ) - to potential future users (N=22), capturing both their personal values and the values they would want an assistant robot to embody. We compare the three instruments as personalization tools using the UEQ-S and benchmark fifteen contemporary LLMs against the resulting profiles. Participants wanted their robot to score markedly lower than themselves on personal-focus values and higher on social-focus values; moral profiles were nearly identical. LLMs prompted to act as robots shifted only weakly toward the desired profile, indicating that default model behaviour and prompt engineering are no substitute for explicit value personalization.
- [6] arXiv:2611.18385 [pdf, html, other]
-
Title: Voice-Based Conversational Agents as AI Care Assistants: Evaluating Personalization, Intervention Delivery Fidelity, and Privacy Expectations in Older Adults' CareThalia Viranda, Shupeng Wang, Jia Bo, Yuchen Yang, Hongjiao Zhang, Andrea Cuadra, Daniel A. Adler, Walter Boot, Sara Czaja, Deborah Estrin, Tanzeem ChoudhurySubjects: Human-Computer Interaction (cs.HC)
Generative AI enables voice-based conversational agents to play more active roles in older adults' care, including personalizing support and operating alongside clinicians and care partners. This shift creates two challenges: how to personalize support while faithfully delivering evidence-based interventions, and how to manage personal-data use and sharing across AI and care stakeholders. We designed and evaluated ARC, a voice-based conversational agent that operationalizes multidomain lifestyle intervention through sessions with predefined goals, participant context, and evidence-based components. In a 3-day remote study, 80 older adults completed 473 sessions across six session types (9,183 turns; averaging 19.4 turns and 6 minutes per session). Participants reported favorable usability, acceptability, and use intentions. System-level analyses showed strong domain-level but modest participant-specific contextual reuse, no significant personalization-fidelity association, and fidelity limitations centered on incomplete delivery of intervention components rather than off-protocol drift. Within care relationships, expectations for personal-data use and sharing varied by data type and recipient. Our findings support feasibility of conversational delivery of multidomain lifestyle intervention, offer methods and system-design insights for longitudinal evaluation, and motivate an evaluation agenda for generative AI health interventions: user-experience and outcome measures should be complemented by evidence of whether intended intervention processes were delivered as designed.
New submissions (showing 6 of 6 entries)
- [7] arXiv:2611.17564 (cross-list from cs.AI) [pdf, html, other]
-
Title: AI-Voyager: A Public-Interest System for Human-AI Civic Reasoning to Contest and Shape the Rules That Govern AISnehalkumar `Neil' S. Gaikwad, Dhruv Dixit, Bryen Alvarez, Pratik Patekar, Anika Sharma, Akshara Kolipaka, Paritosh PandeySubjects: Artificial Intelligence (cs.AI); Computation and Language (cs.CL); Human-Computer Interaction (cs.HC)
Governments are writing rules for artificial intelligence that decide what counts as an automated decision or a harm, often before the people they affect have a chance to question them. We introduce AI-Voyager, a public-interest system for human-AI civic reasoning, in which people and AI work together to understand, contest and shape the rules that govern AI. A resident can find their city's rules on AI in hiring or upload their country's AI strategy, read it as an affected worker would, open the passage behind any answer, and draft a revision. Its architecture turns 3,653 governance documents from city, state, federal and national governments into passages that keep their page and edition, and retrieves evidence through bounded agentic search that verifies each statement. On 360 new questions that require finding the relevant documents, it reached 83.0% macro recall, against 80.5% for the strongest baseline, 73.7% without iterative search and 99.1% on a benchmark with fixed documents. In a study with 24 participants new to two AI bills, most proposed changes asked how provisions would apply. Participants chose lenses for the kind of explanation they wanted, and citations signaled grounding but rarely led them to check the source. By binding AI assistance to the exact text of public documents, this work introduces a grammar of operations and a design space that inform how such systems are designed and evaluated, and how people take part in governing AI.
- [8] arXiv:2611.17713 (cross-list from cs.SE) [pdf, html, other]
-
Title: DesignSpecs: A DESIGN.md Dataset of Visual Design Specifications for AI AgentsSubjects: Software Engineering (cs.SE); Human-Computer Interaction (cs.HC)
While AI agents increasingly generate and modify user interfaces, they rarely rely on a visual design specification for a certain project. The DESIGN$.$md format, which originated from Google's AI design tool Stitch and which was open-sourced in April 2026, addresses this gap by describing a visual identity through design tokens (such as named colors, text styles) and Markdown prose. To our knowledge, no large-scale research dataset of such design specification files exists. We present DesignSpecs, a dataset of 36,906 DESIGN$.$md files from 15,794 public, non-fork GitHub repositories, representing 17,240 unique file contents. We collected 613,207 files named DESIGN$.$md in October 2026. Because the same file name also denotes architectural plans and other design documents, we filtered those with YAML front matter, a design-system name, at least one recognized design token, and zero errors under the official linter. Our JSON Lines dataset provides file-level and deduplicated views of the original Markdown, links files to repositories, paths, and commits, and is released with related scripts. We outline research opportunities on visual design knowledge and design-aware AI agents. Dataset is available at: this https URL
- [9] arXiv:2611.17802 (cross-list from cs.RO) [pdf, html, other]
-
Title: Multimodal Person Concept Model for Person Identification under Varying Observation ConditionsComments: 12 pages, 4 figures, 11 tables. Submitted to Artificial Life and Robotics (AROB Journal Track). Project page: this https URLSubjects: Robotics (cs.RO); Artificial Intelligence (cs.AI); Human-Computer Interaction (cs.HC); Machine Learning (cs.LG)
Service robots in domestic and office environments must identify users to provide personalized services based on their preferences and previous interactions. This is challenging because only a few observations may be available for learning, while observation conditions change owing to appearance, environmental noise, and utterance content. To address this challenge, this study proposes a multimodal person concept model that learns person concepts from a small number of observations, without person labels, and enables robust identification under changing conditions. The model employs a nonparametric Bayesian generative model based on the Chinese restaurant process and integrates visual, acoustic, and linguistic features extracted using Contrastive Language-Image Pre-training (CLIP) and Contrastive Language-Audio Pretraining (CLAP), followed by principal component analysis (PCA). We evaluated six participants under eight conditions in which appearance changes, additive noise, and transcription errors were introduced individually and in combination. Averaged over the eight conditions, the proposed model achieved an identification accuracy of 0.89 while inferring the number of person concepts from the observations, which is comparable to k-means clustering that was supplied with the true number of participants (0.88). We also examined PCA dimensionality, modality availability, and variation strength, and demonstrated the model on a Toyota Human Support Robot. These results show that multimodal person concept learning supports robust person identification under changing conditions, and that identification can succeed even when the number of concepts is not correctly recovered. The project page and demonstration video are available at this https URL.
- [10] arXiv:2611.17856 (cross-list from cs.AI) [pdf, html, other]
-
Title: DECODE: Diagnosing Interpretability Failures in Automated Interpretation ProcessSubjects: Artificial Intelligence (cs.AI); Human-Computer Interaction (cs.HC)
Tools for interpreting the internal representations of Large Language Models (LLMs) increasingly rely on LLMs themselves to both generate and evaluate explanations, creating a self-referential automated interpretation process with minimal human oversight. This automation raises significant reliability concerns, which are exacerbated by the absence of ground truth. In practice, it is difficult to validate explanations or to attribute interpretability failures to their source -- whether they originate from the tool's extracted representations or from the LLM used in the process. To address these challenges, we present DECODE, a visual analytics system that reintroduces human oversight into the automated interpretation process. Our approach decomposes the diagnostic workflow into three progressive stages, isolating failure sources ranging from structural defects to explanation-level misalignments. Drawing inspiration from inter-coder consensus in qualitative analysis, we utilize consensus among independently generated explanations and their evaluation metrics as a proxy for reliability. To further scale human inspection, we employ a three-phase active seeding protocol that strategically recommends high-priority instances for manual review. Finally, we demonstrate the effectiveness of our approach through a case study on 16,384 Sparse Autoencoder features, showing that it can efficiently identify and attribute interpretability failures at scale.
- [11] arXiv:2611.18284 (cross-list from cs.LG) [pdf, html, other]
-
Title: Clean Tickets, Wrong Orders: Catching Silent Errors with a Language Model in Your Browser TabSubjects: Machine Learning (cs.LG); Computation and Language (cs.CL); Human-Computer Interaction (cs.HC)
A small language model that turns a typed order into structure can fail in two ways. It can print something broken, which any system rejects. Or it can print a clean, well formed order that nobody placed. Our companion work calls the second kind a silent error, and it is the dangerous one. We present a web demo in which a fine-tuned 0.8B parameter model runs entirely inside the visitor's browser tab, through this http URL compiled to WebAssembly. No order text leaves the device. The page parses orders as the visitor types, scores every ticket with the PIZZA benchmark's own matcher ported to TypeScript, refuses anything off the menu with "?" so the model can face the open web, and lets visitors hunt for silent errors or try to cause one. Run in the browser on all 1,357 PIZZA test orders, the deployed model writes the same output as the server evaluation for 99.0% of them and reaches the same exact match (0.882); we also report its latency on 3 consumer laptops. Demo: this https URL
- [12] arXiv:2611.18438 (cross-list from cs.CV) [pdf, html, other]
-
Title: Toward On-Device Markerless 6DoF Tracking of a Physical Manipulative on a Mixed-Reality Headset: A Preliminary StudySubjects: Computer Vision and Pattern Recognition (cs.CV); Human-Computer Interaction (cs.HC)
AR-Classroom uses physical models and virtual overlays to help learners explore spatial transformations. To align these overlays on a standalone headset, we developed a markerless six-degree-of-freedom(6DoF) tracker for a LEGO airplane on the Meta Quest 3. A compact YOLO model trained on synthetic Unity renders predicts 20 keypoints from dynamic RGB crops resized to 128 by 128 pixels. A perspective-n-point solver estimates the airplane's pose from these predictions and the corresponding 3D landmarks. In a paired evaluation on 1,000 synthetic validation images, cropping guided by ground-truth boxes increased the detection rate on annotated images from 32.5 to 46.2 percent. Mean keypoint F1 increased from 42.9 to 44.6 percent, with lower keypoint precision in the cropped view. In preliminary headset tests, 2 Hz was the highest tested tracking target at which the application maintained approximately 72 frames per second. A headset demonstration illustrates virtual overlay alignment with the physical airplane before and after a manual rotation. Quantitative 6DoF pose accuracy on real headset footage will be measured in future work.
- [13] arXiv:2611.18439 (cross-list from cs.CY) [pdf, html, other]
-
Title: Between Obligations and Evidence: A Participatory Study of Translating the EU AI Act into Technical RequirementsHelma Torkamaan, Masoud Mansoury, Yanan Xin, Oscar Oviedo Trespalacios, Roberta Savella, Andrej Zwitter, Maria Heuss, Huy Son Nguyen, Marie-Therese Sekwenz, Moyi Li, Roberta Tamponi, Megha KhoslaSubjects: Computers and Society (cs.CY); Artificial Intelligence (cs.AI); Human-Computer Interaction (cs.HC); Software Engineering (cs.SE)
The EU AI Act imposes technical and organizational requirements for high-risk AI systems, but applying these requirements involves contextual choices about what to measure and what evidence supports a compliance claim. We investigate these challenges through a participatory workshop involving 32 researchers, policymakers, and practitioners from different disciplines, using realistic scenarios in healthcare, recruitment, and public transport to examine how regulatory obligations are interpreted, translated into technical requirements, and evaluated in practice. Through a thematic analysis of workshop data, we identify recurring tensions between how regulatory obligations are translated into measurable system properties, how responsibility and human oversight are exercised in practice, and how the validity of compliance evidence is affected by changing operational conditions and system uses. For example, participants highlighted how fairness metrics could conceal discrimination in recruitment, how time pressure and limited information could undermine human oversight in public transport, and how changes in patient populations could invalidate previously established evidence of system performance. Our findings suggest that technical, organizational, and evidentiary challenges are interdependent, and that establishing and maintaining compliance requires attention not only to technical assessments but also to how AI systems are used, governed, and monitored over time. We discuss implications for supporting context-sensitive and ongoing compliance.
Cross submissions (showing 7 of 7 entries)
- [14] arXiv:2502.20658 (replaced) [pdf, other]
-
Title: Curating Safe Spaces for Emotion Disclosure during Illness Recovery on Video-Sharing PlatformsJiaying "Lizzy" Liu, Yunlong Wang, Allen Jue, Yao Lyu, Yiheng Su, Shuo Niu, Nathan TeBlunthuis, Yan ZhangSubjects: Human-Computer Interaction (cs.HC); Computers and Society (cs.CY); Social and Information Networks (cs.SI)
The advent of video-sharing platforms has transformed self-expression, transcending entertainment to encompass sensitive disclosures and emotional revelations. In this submission, we discuss our study on emotional disclosure by individuals with Schizophrenia through YouTube vlogs and its implications for designing safe online spaces.
- [15] arXiv:2607.24558 (replaced) [pdf, other]
-
Title: Proceedings of The First Reflection in Creative Experience (RiCE) WorkshopCorey Ford, Olga Sutskova, Samuel Rhys Cox, Sarah Sterman, Max Kreminski, Rosa Van Koningsbruggen, Anqi Wang, Ege Otenen, Karly Ross, Giulia Di Fede, Yinmiao Li, Salvatore Andolina, Marit Bentvelzen, Pan Hui, Nick Bryan-KinnsSubjects: Human-Computer Interaction (cs.HC)
Reflection and metacognition are central to the creative user experience. However, most HCI research on reflection focuses on clear, task-oriented goals such as to reflect on personal data or pedagogical outcomes. This contrasts with the open-ended and challenging to articulate goals of creative user experiences. For the first time, this workshop brings together interdisciplinary researchers, designers, educators, and artists across HCI, Cognitive Science, Design, AI, Learning Sciences, and Digital Art to examine reflection in creative interaction. The workshop will discuss themes, drawn from earlier discussions with HCI researchers and artists, on: how best to capture reflection in creative contexts, how to leverage the arts to support reflection for ethical change, and how to design creative AI that enhances - not hinders - critical thinking. By bringing interdisciplinary perspectives on reflection into discussion, the workshop will develop a guiding taxonomy for reflection in creative interaction to inform future creative practice and tool development.
- [16] arXiv:2610.00073 (replaced) [pdf, html, other]
-
Title: A Comprehensive Evaluation Framework for Conversational Home Energy Management SystemsComments: 25 pages, 7 figures, 9 tables. Re-set in the HUBS Lab paper layout (HUBS Lab Paper No. 2026-006); content unchangedSubjects: Human-Computer Interaction (cs.HC)
The growing complexity in home energy management (HEM) demands advanced systems that guide occupants toward informed energy decisions reflecting their background, preferences, and context. Large language model (LLM)-integrated HEM systems (HEMS) have demonstrated promise, but previous studies relied on single-turn or single-task evaluations with response accuracy as the primary metric. Whether such systems deliver effective interactions across the extended multi-turn dialogues typical of real-world use remains an open question. This study introduces a comprehensive evaluation framework of LLM-integrated HEMS derived from the Goal-Question-Metric methodology, organized across five categories: task performance, factual accuracy, interaction quality, control capability, and system efficiency. A total of 23 metrics across multi-turn conversations are proposed and an LLM-as-judge pipeline is employed to enable scalable automated scoring. Its reliability is validated against three trained human coders: after iterative rubric calibration, twelve of the fifteen LLM-scored metrics reached strong agreement (ICC >= 0.73), three of them perfect, while the remaining three exhibited near-zero variance in human scores and are instead reported via mean absolute error (0.04-0.28). To demonstrate the framework's effectiveness, 970 dialogues -- 16 scenarios and five personas -- were generated and evaluated across four conversational HEMS configurations spanning a sophistication gradient, from a vanilla LLM with raw energy data to a multi-agent HEMS. The framework distinguished the four configurations across multiple evaluation dimensions, revealing their respective strengths and weaknesses. This study contributes to conversational HEMS by providing a reproducible, multi-dimensional evaluation methodology that comprehensively assesses sustained, context-aware system performance.
- [17] arXiv:2608.23642 (replaced) [pdf, html, other]
-
Title: AI Agents Push Humans Out of the LoopComments: Accepted to NeurIPS 2026, Position Paper TrackSubjects: Artificial Intelligence (cs.AI); Human-Computer Interaction (cs.HC)
AI agents pose significant risks as they are granted increasing autonomy. A commonly proposed solution is human oversight and keeping a ''human in the loop'', but this is not a simple solution: Not only do current approaches to AI agent design impede effective human oversight, but the cognitive capacities required for it are also themselves degraded by extended use of AI systems. This position paper argues that current approaches to the development and deployment of AI agent systems do not support effective human oversight -- they contribute to its degradation. To address this, a top priority in the advancement of AI agents should be supporting the situated goals and cognitive requirements of effective human oversight, treating the human needs of overseers at the same level of importance as AI agent capability. To put this idea into practice, we connect work on automation and human-computer interaction to AI agent processes, outlining design-level affordances and organizational protocols that (1) support overseers in exercising critical judgement and (2) counteract the skill atrophy that arises from extended use of automation. We urge developers and deployers to adopt these or similar approaches. Without explicit support for the cognitive demands of effective human-agent interaction, AI agent systems will continue to passively incentivize the degradation of the very human skills they rely on.
- [18] arXiv:2611.16931 (replaced) [pdf, other]
-
Title: Beyond Curiosity: Habit and Organizational Learning Capability as Predictors of Shadow AIComments: Author accepted manuscript; v1 is the submitted version. Received 10 June 2026; revised 14 October 2026; accepted 31 October 2026. Published in Computers in Human Behavior, Article 109243. (c) 2026. This manuscript version is made available under the CC-BY-NC-ND 4.0 license. 58 pages including supplementary materialsJournal-ref: Computers in Human Behavior (2026), Article 109243Subjects: Computers and Society (cs.CY); Human-Computer Interaction (cs.HC)
Generative AI has entered the workplace faster than organizations can govern it. Employees may use AI tools their employer has not sanctioned and conceal that use, but the psychology of this behavior remains poorly understood. We compare an exploratory-disposition account with one centered on reported use and habit. Using survey data from 510 employed adults reporting workplace generative-AI use during the preceding 12 months, we estimated the unique and relative contributions of workplace curiosity, habit, technology-acceptance beliefs, and perceived organizational learning capability through hierarchical regression, bootstrap relative weights, and full-subset dominance analysis. Habit had the largest relative weight, accounting for 28.3% of explained variance, and completely dominated joyous exploration across all 512 possible predictor subsets. Joyous exploration contributed 3.4% and became nonsignificant when use behavior, performance expectancy, and social influence entered the model; habit subsequently explained additional variance. Organizational learning subscales accounted for 24.1% of explained variance; experimentation and knowledge transfer were associated with lower shadow AI, while curiosity-by-learning interactions were nonsignificant. Separate models for unauthorized use and concealment reproduced the habit-dominance pattern. Perceived professional support was associated with lower concealment, although its coefficient did not significantly differ between the two behavioral outcomes. Shadow AI in this sample was more strongly associated with self-reported habit strength and perceived organizational conditions than with exploratory disposition. The cross-sectional design does not establish causal or temporal order.