Selected publications

1. 

   Exploring AI Problem Formulation with Children via Teachable Machines

   Utkarsh Dwivedi, Salma Elsayed-Ali, Elizabeth Bonsignore, and Hernisa Kacorri

   In Proceedings of the 2024 CHI conference on Human Factors in Computing Systems 2024

   PDF Abstract

   Emphasizing problem formulation in AI literacy activities with children is vital, yet we lack empirical studies on their structure and affordances. We propose that participatory design involving teachable machines facilitates problem formulation activities. To test this, we integrated problem reduction heuristics into storyboarding and invited a university-based intergenerational design team of 10 children (ages 8-13) and 9 adults to co-design a teachable machine. We find that children draw from personal experiences when formulating AI problems; they assume voice and video capabilities, explore diverse machine learning approaches, and plan for error handling. Their ideas promote human involvement in AI, though some are drawn to more autonomous systems. Their designs prioritize values like capability, logic, helpfulness, responsibility, and obedience, and a preference for a comfortable life, family security, inner harmony, and excitement as end-states. We conclude by discussing how these results can inform the design of future participatory AI activities.

2. 

   Exploring Machine Teaching with Children

   Utkarsh Dwivedi, Jaina Gandhi, Raj Parikh, Merijke Coenraad, Elizabeth Bonsignore, and Hernisa Kacorri

   In Proceedings of the 2021 IEEE Symposium on Visual Languages and Human-Centric Computing 2021

   Best Paper Award, Honorable Mention

   PDF Abstract

   Iteratively building and testing machine learning models can help children develop creativity, flexibility, and comfort with machine learning and artificial intelligence. We explore how children use machine teaching interfaces with a team of 14 children (aged 7-13 years) and adult co-designers. Children trained image classifiers and tested each other’s models for robustness. Our study illuminates how children reason about ML concepts, offering these insights for designing machine teaching experiences for children: (i) ML metrics (\eg confidence scores) should be visible for experimentation; (ii) ML activities should enable children to exchange models for promoting reflection and pattern recognition; and (iii) the interface should allow quick data inspection (\eg images vs. gestures).

3. 

   Sharing Practices for Datasets Related to Wellness, Accessibility, and Aging

   Rie Kamikubo, Utkarsh Dwivedi, and Hernisa Kacorri

   In Proceedings of the 2021 International ACM SIGACCESS Conference on Computers and Accessibility 2021

   PDF Abstract

   Datasets sourced from people with disabilities and older adults play an important role in innovation, benchmarking, and mitigating bias for both assistive and inclusive AI-infused applications. However, they are scarce. We conduct a systematic review of 137 accessibility datasets manually located across different disciplines over the last 35 years. Our analysis highlights how researchers navigate tensions between benefits and risks in data collection and sharing. We uncover patterns in data collection purpose, terminology, sample size, data types, and data sharing practices across communities of focus. We conclude by critically reflecting on challenges and opportunities related to locating and sharing accessibility datasets calling for technical, legal, and institutional privacy frameworks that are more attuned to concerns from these communities.

4. 

   OptiDwell: Intelligent Adjustment of Dwell Click Time

   Aanand Nayyar, Utkarsh Dwivedi, Karan Ahuja, Nitendra Rajput, Seema Nagar, and Kuntal Dey

   In Proceedings of the 2017 International Conference on Intelligent User Interfaces 2017

   PDF Abstract

   Gaze based navigation with digital screens offer a hands-free and touchless interaction, which is often useful in providing a hygienic interaction experience in a public kiosk scenario. The goodness of such a navigation system depends not only on the accuracy of detecting the eye gaze but also on the ability to determine whether a user is interested in clicking a button or is just looking at the button. The time for which a user needs to gaze at a particular button before it is considered as a click action is called the dwell time. In this paper, we explore intelligent adjustment of dwell times, where mouse click events on the buttons of a given application are emulated with user gaze. A constant dwell-time for all buttons and for all users may not provide an efficient and intuitive interface. We thereby propose a model to dynamically adjust dwell-time values used to emulate user mouse click events, exploiting the user’s experience with different portions of a given application. The adjustment happens at a per-user, per-button granularity, as a function of the user’s (a) prior usage experience of the given button within the application and (b) Midas touch characteristics for the given button. We propose OptiDwell, inspired by the action-value method based solutions to the Multi-Armed Bandits problem, for dwell click time adaptation. We experiment OptiDwell using an interactive TV channel browsing interface application, constituting of a mix of text and image buttons, over 10 computer-savvy users generating over 9000 click tasks. We observe significant improvement of user comfort level over the sessions, quantified by (a) improved (reduced) dwell times and (b) reduced number of Midas touches in spite of faster dwell-clicks, as high as 10-fold reduction in the best case. Our work is useful for creating an interface, with accurate, fast and comfortable dwell-clicks for each interface element (e.g., buttons), and each user.

5. 

   VisualMath: An Automated Visualization System for Understanding Math Word-Problems

   Utkarsh Dwivedi, Nitendra Rajput, Prasenjit Dey, and Blessin Varkey

   In Proceedings of the 2017 International Conference on Intelligent User Interfaces Companion 2017

   PDF Abstract

   Math word problems are difficult for students to start with since they involve understanding the problem?s context and abstracting out its underlying mathematical operations. A visual understanding of the problem at hand can be very useful for the comprehension of the problem. We present a system VisualMath that uses machine learning tools and crafted visual logic to automatically generate appropriate visualizations from the text of the word-problems and solve it. We demonstrate the improvements in the understanding of math word-problems by conducting a user study and learning of meaning of relevant new words by students.