Therapybots - Adam Jarvis

UPLOADED

NOVEMBER 2024

UX RESEARCH

COMPUTER SCIENCE

RESEARCHING SOCIOCULTURAL INCLUSIVITY FOR CONVERSATIONAL AI INTERFACES

AI CHATBOTS CAN BE CULTURALLY INSENSITIVE. DOES INTERFACE PLAY A ROLE?

THE PROBLEM

Conversational AI is becoming increasingly prominent in daily life., yet it can overlook sociocultural differences. There are examples of users finding interactions too direct, being asked distressing or distasteful questions, or wishing for more neutral or inclusive tones, highlighting the need for thoughtful design approaches.

During this 8-week dissertation project, I used a Research through Design (RtD) approach to A/B test interfaces, and conducted a statistical analysis to generate design guidelines.

TIMELINE

8 WEEKS · Q3 2024

PROJECT BY

ADAM JARVIS · SOLO

SET BY

SELF INITIATED
in MSC HCI

PLATFORM

FIGMA

RESEARCH PREPARATION

I established my METHOD METHOD BY EXPLORING the field of SCIENTIFIC STUDIES.

RESEARCH PREPARATION

I established my METHOD METHOD BY EXPLORING the field of SCIENTIFIC STUDIES.

RESEARCH PREPARATION

I established my METHOD METHOD BY EXPLORING the field of SCIENTIFIC STUDIES.

RESEARCH PREPARATION

I established my METHOD METHOD BY EXPLORING the field of SCIENTIFIC STUDIES.

I researched key theoretical frameworks in socioculturalism, computational empathy and applicational methods.

I devised a 3-staged research methodology, where the key theoretical frameworks inform the creation & testing of A/B interfaces.

Interface A tests Picard’s Information Flow (1997), Barthes’ Semiotic Significance (1981) and Hofstede’s Cultural Dimensions (2010).

Interface B tests Bordoli & Biswas’ Sentiment Classification (2023), Murray & Arnott’s Effect of Emotion on Speech (1993), and Greimas’ Semiotic Square (1966).

I researched key theoretical frameworks in socioculturalism, computational empathy and applicational methods.

I devised a 3-staged research methodology, where the key theoretical frameworks inform the creation & testing of A/B interfaces.

Interface A tests Picard’s Information Flow (1997), Barthes’ Semiotic Significance (1981) and Hofstede’s Cultural Dimensions (2010).

Interface B tests Bordoli & Biswas’ Sentiment Classification (2023), Murray & Arnott’s Effect of Emotion on Speech (1993), and Greimas’ Semiotic Square (1966).

I researched key theoretical frameworks in socioculturalism, computational empathy and applicational methods.

I devised a 3-staged research methodology, where the key theoretical frameworks inform the creation & testing of A/B interfaces.

Interface A tests Picard’s Information Flow (1997), Barthes’ Semiotic Significance (1981) and Hofstede’s Cultural Dimensions (2010).

Interface B tests Bordoli & Biswas’ Sentiment Classification (2023), Murray & Arnott’s Effect of Emotion on Speech (1993), and Greimas’ Semiotic Square (1966).

I researched key theoretical frameworks in socioculturalism, computational empathy and applicational methods.

I devised a 3-staged research methodology, where the key theoretical frameworks inform the creation & testing of A/B interfaces.

Interface A tests Picard’s Information Flow (1997), Barthes’ Semiotic Significance (1981) and Hofstede’s Cultural Dimensions (2010).

Interface B tests Bordoli & Biswas’ Sentiment Classification (2023), Murray & Arnott’s Effect of Emotion on Speech (1993), and Greimas’ Semiotic Square (1966).

INTERFACE A

INTERFACE B

INTERFACE A

INTERFACE B

THE STUDY

i a/b tested the interfaces through simulated ai therapy scenarios.

THE STUDY

i a/b tested the interfaces through simulated ai therapy scenarios.

THE STUDY

i a/b tested the interfaces through simulated ai therapy scenarios.

THE STUDY

i a/b tested the interfaces through simulated ai therapy scenarios.

Over a number of days at participants' locations of choice, i ran 3x simulated scenarios, each designed to isolate the test criteria for one of Sue (2001)'s 'Identity Levels'.

Scenario 1 - 'Individual' Level
This scenario relates to the relational level through humanism, non-verbal communication and mood contagion.

Scenario 2 - 'Relational' Level
This scenario relates to the relational level through goal transparency, sentiment appropriacy and non-judgment.

Scenario 3 - 'Universal' Level
This scenario relates to the relational level through behaviour, mental imagery and ethical landscape.

Over a number of days at participants' locations of choice, i ran 3x simulated scenarios, each designed to isolate the test criteria for one of Sue (2001)'s 'Identity Levels'.

Scenario 1 - 'Individual' Level
This scenario relates to the relational level through humanism, non-verbal communication and mood contagion.

Scenario 2 - 'Relational' Level
This scenario relates to the relational level through goal transparency, sentiment appropriacy and non-judgment.

Scenario 3 - 'Universal' Level
This scenario relates to the relational level through behaviour, mental imagery and ethical landscape.

Over a number of days at participants' locations of choice, i ran 3x simulated scenarios, each designed to isolate the test criteria for one of Sue (2001)'s 'Identity Levels'.

Scenario 1 - 'Individual' Level
This scenario relates to the relational level through humanism, non-verbal communication and mood contagion.

Scenario 2 - 'Relational' Level
This scenario relates to the relational level through goal transparency, sentiment appropriacy and non-judgment.

Scenario 3 - 'Universal' Level
This scenario relates to the relational level through behaviour, mental imagery and ethical landscape.

Over a number of days at participants' locations of choice, i ran 3x simulated scenarios, each designed to isolate the test criteria for one of Sue (2001)'s 'Identity Levels'.

Scenario 1 - 'Individual' Level
This scenario relates to the relational level through humanism, non-verbal communication and mood contagion.

Scenario 2 - 'Relational' Level
This scenario relates to the relational level through goal transparency, sentiment appropriacy and non-judgment.

Scenario 3 - 'Universal' Level
This scenario relates to the relational level through behaviour, mental imagery and ethical landscape.

DATA ANALYSIS

i used inferential statistics on the 108 sensitivity ratings to analyse the data.

DATA ANALYSIS

i used inferential statistics on the 108 sensitivity ratings to analyse the data.

DATA ANALYSIS

i used inferential statistics on the 108 sensitivity ratings to analyse the data.

DATA ANALYSIS

i used inferential statistics on the 108 sensitivity ratings to analyse the data.

From each of the 6x participants, I received 9x ratings for each of the 2x interfaces. I recorded participant demographis to allow for the synthesis of sociocultural insights.

When the homogeneity and normality transformations failed, I switched from the parametric T-Test to non-parametric tests.

Using SPSS, the non-parametric Wilcoxon Test was used for a 'Within Subjects' study with 3x dependent variables.

From each of the 6x participants, I received 9x ratings for each of the 2x interfaces. I recorded participant demographis to allow for the synthesis of sociocultural insights.

When the homogeneity and normality transformations failed, I switched from the parametric T-Test to non-parametric tests.

Using SPSS, the non-parametric Wilcoxon Test was used for a 'Within Subjects' study with 3x dependent variables.

From each of the 6x participants, I received 9x ratings for each of the 2x interfaces. I recorded participant demographis to allow for the synthesis of sociocultural insights.

When the homogeneity and normality transformations failed, I switched from the parametric T-Test to non-parametric tests.

Using SPSS, the non-parametric Wilcoxon Test was used for a 'Within Subjects' study with 3x dependent variables.

From each of the 6x participants, I received 9x ratings for each of the 2x interfaces. I recorded participant demographis to allow for the synthesis of sociocultural insights.

When the homogeneity and normality transformations failed, I switched from the parametric T-Test to non-parametric tests.

Using SPSS, the non-parametric Wilcoxon Test was used for a 'Within Subjects' study with 3x dependent variables.

MY RESULTS PROVED THE effect of SOCIOCULTURAL DIVERSITY on USER PREFERENCES.

In the user-perceived sociocultural empathy of interface A & B;

H₁ (alternate hypothesis) = a significant difference was observed.
H₀ (null hypothesis) = no significant difference was observed.

The following findings present the significant
differences identified with a p-value of ≤0.05.

In the user-perceived sociocultural empathy of interface A & B;

H₁ (alternate hypothesis) = a significant difference was observed.
H₀ (null hypothesis) = no significant difference was observed.

The following findings present the significant
differences identified with a p-value of ≤0.05.

In the user-perceived sociocultural empathy of interface A & B;

H₁ (alternate hypothesis) = a significant difference was observed.
H₀ (null hypothesis) = no significant difference was observed.

The following findings present the significant
differences identified with a p-value of ≤0.05.

In the user-perceived sociocultural empathy of interface A & B;

H₁ (alternate hypothesis) = a significant difference was observed.
H₀ (null hypothesis) = no significant difference was observed.

The following findings present the significant
differences identified with a p-value of ≤0.05.

Gender × Design Approach B

Female participants responded more positively to Approach B (ECM) than male participants.

p = 0.034

Nationality × Universal Identity

Indian participants responded more consistently than other nationalities on the Universal level.

p = 0.046

Gender × Relational Identity

Female participants responded more positively in the Relational Level to A/B than male participants.

p = 0.049

RESEARCH IMPLICATIONS

I CREATED DESIGN GUIDELINES to BRING multi-sOCIOCULTURAL USER SATISFACTION to AI INTERFACES.

RESEARCH IMPLICATIONS

I CREATED DESIGN GUIDELINES to BRING multi-sOCIOCULTURAL USER SATISFACTION to AI INTERFACES.

RESEARCH IMPLICATIONS

I CREATED DESIGN GUIDELINES to BRING multi-sOCIOCULTURAL USER SATISFACTION to AI INTERFACES.

RESEARCH IMPLICATIONS

I CREATED DESIGN GUIDELINES to BRING multi-sOCIOCULTURAL USER SATISFACTION to AI INTERFACES.

Guideline 1 • Gender-Based Empathy

The recognition of emotion in speech, and the application of a Semiotic Square, were less able to meet the needs of males as they were for females. Female users must be provided with self-reflective visual motifs, while male users require visual evidence of reflective listening.

Guideline 2 • Nationality Empathy

Cross-cultural standards have an impact on the effectiveness of visual-supported active listening, so it is necessary to supplement user interactions with semiotic materials that counterweight the speech content in the direction of individual user expectations.

Guideline 3 • Identity Empathy

Satisfaction increases when placing emphasis on visually recognising user demographics, such as age, race, and socioeconomic status, as it has proven easier to empathise on the Relational level of the Tripartite of Identity (Sue, 2001) than the Individual or Universal levels.

RETURN TO PROJECTS