Jon Clucas, Jake Son

June 27, 2018

Human-computer interaction (HCI) is the field of study focused on how technology is used, both in theory and in practice. Research in this area is often focused on taking advantage of recent advancements in technology, psychology and sociology to improve the relationship that we have with our smartphones, wearables, computers, vehicles and other devices. The Association for Computing Machines Special Interest Group on Computer Human Interaction (ACM SIGCHI) hosts an annual flagship conference for the field (CHI). In recent years, a segment of the community has adopted a specialized interest in applications of technology to mental health; at CHI 2018, held at the Palais de Congrès in Montreal this year, we (Jon and Jake) attended the 3rd Symposium on Computing and Mental Health. This event was intended to bring together experts in both healthcare and HCI, two domains that can be isolated due to competing interests and research time scales. Here, we discussed several recurring trends and challenges the community faced in bridging the gap between the technology designer and user.

Contents

• Jon Clucas, Jake Son
• June 27, 2018
• Mental Health Symposium
• 2018 Conference on Human Factors in Computing Systems
• Data and Methods
  • Qualitative and quantitative data
  • Open data
  • Ecological data
• Science Communication
• Closing Statements
• Appendix: Communication Highlights
• References

Mental Health Symposium

At this conference, we presented two posters from the MATTER Lab, focused on selective mutism and mental health assessments.

Our first poster¹ details preliminary findings on the efficacy of audio feature analysis as a quantifiable sign of selective mutism. A common issue faced in the medical community is the need to objectively measure biomarkers that can elucidate the biological underpinnings of disease symptoms. In psychiatry, this is a particularly difficult problem as an individual’s mental well-being often is complex with inherently subjective components. For instance, selective mutism (SM) is a condition characterized by a child’s inability to speak in school or other public places because they are vocally paralyzed with anxiety. Diagnosing and monitoring selective mutism during treatment is a non-trivial task, as each presentation of SM is unique and assessed in part from subjective parent and teacher reports. Automated audio analysis is a promising method that may address the challenge of monitoring SM signs in natural environments outside of a controlled lab setting. In exploring this method with openSMILE audio feature extraction software, we confronted one of the common barriers in voice analysis, which is the fact that vocalizations are always performed in a context, and as such often involve voices other than or in addition to the target voice. We tested various methods of pre-processing to account for additional voices and found that low-level audio features (e.g. Mel-frequency cepstral coefficients, fundamental frequency) were able to predict a participant’s SM diagnosis with moderate accuracy for each method using random forest algorithms. The openSMILE + random forest method could even predict a participant’s SM diagnosis from the isolated extra voices in ¾ of the conditions in our 2×2 design.

Our second poster² presents a visualization of relationships between question pairs in 79 mental health assessments. In labs and in practice, questionnaires can be burdensome to participants and to administrators. They have plenty of overlap and while a response to any individual question is informative, the informative value of each subsequent question will vary. We examined the relative information of pairs of questions, in hopes that these values will guide the construction of more efficient questionnaires that prioritizes the most informative questions. As an extension, our current work focuses on using various modeling algorithms to estimate the most informative questions for prediction ADHD subtypes, based on the Healthy Brain Network (HBN) cohort.

For Anirudh's notes, think crime scene evidence boards covering an entire wall, but with mental health research.

Curt has multiple projects as finalists for the Hackaday Prize 2018:

The MATTER Lab is a relatively small lab, with just six members (including our tiger salamander Ada). While our skill-sets are diverse and concrete, including Curt’s expertise as a hardware hacker and Anirudh’s ability to synthesize vast amounts of information to create novel solutions, we looked for opportunities to share our skills and actively sought the expertise of others to work toward filling the gaps in our circle of competence. One thing we noticed about many of the studies presented at the symposium was the low sample size and statistical power; granted, posters at conferences are traditionally demonstrations of preliminary findings and are not representative of all research at CHI. Having access to large, freely available datasets is a viable solution that improves the quality and reproducibility of research efforts, with short- and long- term implications for healthcare providers and patients. Other researchers expressed excitement in learning of the Healthy Brain Network data collection effort.

Overall, while the methodology and research interests of each lab varied greatly, the MATTER Lab’s work reflected the general interests and concerns of the larger CHI community and was not limited to related projects in mental health.

2018 Conference on Human Factors in Computing Systems

Over 1,200 presentations were available to attend at CHI 2018, many simultaneously. As such, we each attended some presentations that the other did not. An image of both of us precedes each paragraph where our experiences and opinions converge; otherwise, each paragraph is preceded by an image of the person whose experiences and opinions are being presented.

Aside from the Symposium on Computing and Mental Health, the conference itself was an opportunity to communicate with other scientists and learn of exciting work taking place around the world. The venue itself allowed us to overcome common barriers in research communication by bringing together researchers from various domains including computer science, linguistics, and photography. Hundreds of labs shared their research findings and we collectively sought potential opportunities to collaborate.

For instance, I spoke with Pooja Desai, a recent graduate from Columbia University who conducted research with David Albers and Lena Mamykina. Pooja’s presentation³ teased apart the implications of visualizing health information, an important design responsibility that can improve or at times worsen patient outcomes. One particularly interesting finding was that the focus groups in this investigation found confidence intervals for BGL predictions to be more confusing than informative. While additional information about a measurent’s uncertainty is valuable in research, patients were decidedly not interested and even frustrated. This highlights the importance of visualizing health information in a way that builds user trust and encourages continued app use — what is the purpose of designing apps and technologies if the people we intend to help aren’t interested? This is an important consideration in the design of the MindLogger app, an open-source data collection platform that the MATTER Lab is building.

The MindLogger app is intended to acquire participant data in a convenient and consistent manner that will empower not only the patient, but also the physician and parent who have access to a secure dashboard. From the start, we have been intentional about designing the app in a way that encourages user compliance (e.g. reminders, clear instructions coupled with visuals), but Pooja’s work pointed us toward design features that were specific to visualizing health information and reinforced our drive to optimize the user experience.

Besides sharing meals with each other and with large crowds while browsing posters and demos, we shared meals with Greg Wadley, Gunther Eysenbach, Riin Tark, Tomi Männistö (and a few others). We would like to highlight the work of a few labs that we found particularly relevant to our work:

Abhi Pratap described work coauthored with Adam Gazzaley "compar[ing] recruitment, engagement, and treatment outcomes between a sample of depressed Spanish-speaking Hispanic/Latino [sic] individuals compared to a cohort of depressed English speakers using 3 different self-guided mobile apps" (Project: Evo™, iPST, and Health Tips).

It is often the case that advances in science require an interdisciplinary approach; for instance, Dr. Donald Ingber used tensegrity structures, common in architectural design, to model the transfer of mechanical forces in cells. A similar example closer to mental health is the use of multivariate distance-based matrix regression, commonly used in genome studies, to analyze high-dimensional brain connectivity data in disease states. At CHI, being in close proximity with other scientists stimulated conversations that expanded the breadth of our research interest and introduced us to unfamiliar tools that may be applicable to our investigations.

However, the diversity in methodology and training naturally leads to competing theories to explain complex phenomena. We experienced this first-hand during the 15 minute design challenge, where our team of 8 devised a multi-pronged solution to build a support network for individuals with mental health disorders who did not have any close friends or family. This exercise was revealing for a number of reasons: 1) The members of our design team had different priorities (e.g. user interface, programming, patient) based on their background and training. 2) Each person brought a unique perspective to the table, which allowed us to have a more comprehensive view of the problem and solution space. 3) However, this also delayed progress significantly as we had to weigh the appropriateness of each method for the design solution. Despite these limitations, we presented our team’s solution that considers many features of user design, sociology, and machine learning that as individuals would not have been incorporated.

As a student with a background in engineering and intent to pursue an MD/PhD, part of my interest in this conference was to learn more about the intersection of the two domains; a prime example that highlights the power of collaboration is “Feel My Pain: Design and Evaluation of Painpad, a Tangible Device for Supporting Inpatient Self-Logging of Pain”⁸. In their presentation, Dr. Oliver Pearce, an orthopaedic surgeon, and Dr. Blaine Price, an electrical engineer with a human-centered approach to computing, emphasized the unique advantages that came from this effort. While past studies have relied on extant devices (e.g. iPads, tablets), this exploration was built from the ground-up, with design specifications that were tailored for the hospital environment (e.g. compatibility with hospital communications infrastructure, easy sanitation for between-subject use). They also mentioned specific issues with the form factor (e.g. making the Painpad visually unappealing to discourage theft) and additional challenges around device development for the hospital setting. In the MATTER Lab, we confront similar issues in our development of a wrist-worn device called the Tingle that can monitor and record body-focused repetitive behaviors (BFRBs). It is challenging to account for the various and occasionally competing needs of various stakeholders; for instance, a more expensive microprocessor may improve the detection of BFRBs, but decrease overall accessibility by increasing the cost of production. However, doing so results in technologies that are well situated to address long-standing problems in healthcare and other domains.

Shalini Lal gave an insightful presentation on the complexity of adopting new technology, particularly in mental health; we live in a paradoxical time, where there are many promises of tech, but limited uses in real world settings. This is not to suggest that there is a gap in the market for tech in mental health, but the opposite. An overabundance of apps dilutes the market with technologies that are not always designed to follow best practices of current therapy.

For instance, the Human-Computer Interaction research group at University College Dublin conducted an investigation into applications that advertised CBT for users with depression. However, HCI@UCD found that the apps do not necessarily follow best practices of current therapy, and many were not backed by any form of research. Interestingly, regardless of the app’s adherence to clinically backed therapies, users consistently gave high ratings if they perceived the app to be helpful. This points to a clear need in the community for a tool that can guide users through the confusing and growing list of applications. One example is PsyberGuide, an online platform that guides people through available mental health applications. It uses a systematic approach to measure each application’s credibility, user experience, transparency, and offers an expert review for select apps. It is a powerful tool that enables technology consumers to choose effective applications from the thousands that are available. Moving forward, we need to be intelligent creators and consumers of technology in a way that hedges both its benefits and limitations.

Data and Methods

Qualitative and quantitative data

Outside of the symposium, a surprising number of studies were presented that involved or entirely consisted of small-sample qualitative usability analyses (despite other presentations advocating for more generalizable methods, eg, “Evaluation Beyond Usability: Validating Sustainable HCI Research”⁹). Several talks focused on ecological data, harmonization, usage, and privacy.

A team of researchers from Newcastle upon Tyne and Stockholm presented a study¹⁰ in which office workers could indicate, at any time and as often as desired, whether the temperature in the office was “boiling”, “warm”, “fine”, “cold” or “freezing” with the press of a button. Four of these devices and eight objective thermometers (thermometers sampling every seven seconds) recorded data over the course of three weeks in an open office. Findings included anticorrelation between recorded objective temperature and reported subjective temperature, with an interpretation that workers seated nearer HVAC blowers being more uncomfortable and thereby more aware of and more vocal about the office temperature. This work is a clear demonstration that subjective data can be valuable even when objective data is available.

Jason Wiese presented work¹¹ developed from of his doctorate¹² building toward sematic graph representations of disparate personal mobile data and inferencing over said graph. I asked him about openness, both in terms of code and data. He responded that due to the personal nature of such data and inferences over them, for now everything is closed and each user’s data is graphed individually in an attempt to contain “dragons” (negative unintended consequences).

Open data

In many of the presentations leveraging existing larger data sets, the researchers described heuristic search strategies for information retrieval. For example, in collecting data to build a taxonomy of idle games, Alharthi et al. “searched two popular web gaming portals to find and retrieve games: Kongregate and Almost Idle. We used the websites’ existing categories to collect a set of different types of idle games. Using existing classifications of idle games from both websites helped us narrow down game selections to ones that fit the targeted genre. We focused on the most popular games on both websites. This criteria helped us to select games that are considered relevant and valuable by the community”¹³. When data are structured, sharing those structures is a boon to the whole community.

Ecological data

Steve Whittaker, an editor for HCI, presented “What Does All This Data Mean for My Future Mood? Actionable Analytics and Targeted Reflection for Emotional Well-Being”¹⁴ twice, once at the symposium and once at the main conference. In this presentation, he argued against the implicit assumption in personal informatics that the primary problem to solve is an information gap, that with more information a person will “do the right thing”. His team’s app, EmotiCal, gives predictions of future mood based on a handful of longitudinal variables and suggested activities (e.g. exercise, eat your favorite meal) based on user preferences to boost mood. Our team’s app, MindLogger, allows the configuration of ecological assessments and may benefit from the inclusion of predictive modeling of mood and associated behaviors.

Alicia Heraz presented a short talk and a poster at the symposium in which she showed proof-of-concept for automatedly classifying human emotions via finger pressure on a touch screen¹⁵. The sample size is small, the task required ample training and a good number of participants were excluded after failing said training. All that said, these early results are intriguing. An API for that system, Emaww, is in closed beta.

Much work is being done related to voice and language analysis. A team presented a project that included automated laughter detection¹⁶. Another analyzed the language in a corpus of Internet of Things manifestos¹⁷. Yet another team found evidence that “conformance to [linguistic] norms, or linguistic accommodation, encourages social approval and acceptance” in online communities¹⁸. A team presented research in which participants were agitated while wearing a monitor playing back their own respective voices, either unmodified (control condition) or “manipulate[d] . . . with pitch-shifting (-30 cents), a low shelf filter (cutoff frequency of 8000 Hz and high-band roll off of 10 dB per octave), and a formant shifter (tract ratio of 0.9)” (for simulated calming, experimental condition)¹⁹. Based on both self-report and measured heart rate variability, their trials indicated that how one hears one’s own voice can have a fairly strong influence on one’s mood.

Science Communication

Prior to attending CHI 2018, I was already sold on the idea of preregistering studies²⁰, but not enough so to have adopted the practice myself. Gunther Eysenbach’s keynote at at the symposium convinced me to advocate for the practice moving forward in MATTER Lab, for the following reasons beyond the obvious reasons of publishable negative results and necessarily well-planned data collection: the peer review process is markedly faster for preregistered studies and the feedback provided in the methods review can be taken into consideration before any data has been collected.

In addition, Gunther suggests that pre-registration also accelerates the pace of scientific progress. This is a common issue that the research community struggles to overcome; while there are obvious advantages to the meticulous review process that is currently in place for most journals, it can take months before results are accepted and published. This delays communication within the scientific community and can engender redundancy of effort, which can mean that months of work in a particular lab are wasted if a separate lab publishes the same results first. There are mechanisms in place to address this, namely preprint servers such as bioRxiv that are intended to make unpublished articles (e.g. under review) immediately accessible to others. While there are many proponents of this approach, others are skeptical that pre-prints hold up to the rigorous standards of peer-review. However, the purpose of pre-prints is not to deliver conclusive statements about research findings, but pre-prints serve as a vehicle for communicating information — Barry Thompson indicates that 60% of pre-prints end up being published in journals²¹.

Many of the presentations that I found particularly relevant were related to communication. An overview of these presentations is provided in this post’s appendix.

Closing Statements

At the talks we attended, many insightful questions were asked by, among others, Steve Whittaker, Greg Wadley, Ben Shneiderman and Christine Dierk. We were impressed with most of the session chairs at the sessions we attended. Occasionally a presenter or questioner would need to be cut off or rescued, and the chairs did a remarkable job of simultaneously moving forward on schedule and respectfully dealing with occasional adverse situations. The moderators of the talks and panels that we attended nearly all did an impressive job of keeping the schedule and rescuing presenters when necessary. The moderators seemed to be well prepared in the same method, noticeably using the same language and techniques.

We appreciated this note in the CHI 2018 program:

TERRITORIAL ACKNOWLEDGEMENT CHI 2018 is being hosted in the beautiful city of Montréal, Canada. We would like to acknowledge that the Palais des Congrès is located on unceded Indigenous lands. Tiohtiá:ke — commonly known as Montréal — is historically known as a gathering place for many First Nations. Today, it is home to a diverse population of Indigenous and other peoples. We aim to respect the continued connections with the past, present and future in our relationships with Indigenous and other peoples within the Montréal community.

Also, the event was remarkably inclusive. For example, non-gender-specific restrooms were available, badges had preferred pronouns (he/she/they), childcare was offered, and children were welcome to attend. Seeing typically ignored needs and desires of attendees be actively attended to made attending CHI 2018 even more of a pleasure than expected.

Appendix: Communication Highlights

Here are some other communication-focused presentations that I found particularly interesting and relevant to MATTER Lab:

References