The Digital Student

What are the ethical considerations of Artificial Intelligence Study Tools?

Artificial Intelligence Study Tools should be used as a companion for learning and you should think carefully before using them and how you are using them

Teesside University outlines the following ethical considerations when using Artificial Intelligence Study Tools:

AI and Personal Data

Compliance Requirements:

• Should follow DPA 2018 and UK GDPR when AI processes personal data.

High-Risk Processing:

• Using AI for personal data is often considered high-risk due to potential privacy infringements.
• Risks include bias, discrimination, and data accuracy issues.

The Learning Process behind Artificial Study Tools

• Each question asked to a tool like ChatGPT is added to a large database.
• The system learns from this database to respond to other users.

Potential Risks

• Uploaded information may become public.

• Risk of exposure in case of cyber attacks.

Data Concerns

• Questions about the University or its customers get added to the tool's publicly accessible knowledge base.
• This information is shared with new users in response to their questions.
• A recent Cyberhaven survey suggests companies upload confidential material to ChatGPT frequently.
• Evidence of customer data scenarios being uploaded.

Impact on Privacy

• AI algorithms with bias may impact privacy.
• Example: Bias in reviewing student applications based on geographic location and ethnicity correlation.

General considerations

Teesside University has the following concerns about analysis programmes:

Automated analysis and interpretation programmes 

• Refinement of automated analysis programs (e.g., NVivo) raises ethical concerns.
• Development of GenAI increases the potential for falsifying qualitative data

Examples of problematic tools

Teesside University has the following concerns about some Artificial Intelligence Study Tools:

Artificial Intelligence is a companion to learning and therefore should not be used to attend meetings on your behalf.

• Tools like Otter.ai link your calendar with your Otter account, and automatically participate in the meeting at the designated time.

• Otter.ai's functions include analyzing and transcribing meetings, automatically summarizing key topics, and capturing shared slides during the meeting.

Privacy Implications and Data Location

• Otter.ai utilizes servers located in the USA. It's essential to consider this aspect while using the tool.
• Privacy regulations in the US may not align adequately with practices in the UK/EU.
• Otter.ai retrieves email addresses from the meeting invite, potentially leading to concerns about data handling and privacy.

Identifying Usage:

• To determine if Otter.ai is in use during a meeting, observe the appearance of a participant named "X's Otter.ai" joining the meeting.

Confidentiality Concerns

• The automatic transcription and sharing feature pose a risk when discussing confidential issues, as these may be transcribed and circulated without explicit permission from all parties involved.

Removal from Meetings

• Teesside Univerity's policy states that individuals suspecting Otter.ai usage should request the removal of the suspected user from the meeting.

Alternative Transcription possibilities

• All Teesside University Meetings should be held on Teams, which has a transcription service. Please ask if this can be activated if you need it.

​​​Protecting Commercial Data and Intellectual Property

Teesside University has the following advice about commercial data and intellectual property:

• Recognize Risks:
  • Be aware of risks to commercial data and intellectual property.
• Prevent Unintended Disclosure:
  • Take steps to avoid unintentional sharing or disclosure.
• Guard Against Exfiltration:
  • Implement measures to prevent data exfiltration.