REPORT on artificial intelligence in education, culture and the audiovisual sector

EXPLANATORY STATEMENT

“I believe that at the end of the century the use of words and general educated opinion will have altered so much that one will be able to speak of machines thinking without expecting to be contradicted”

Alan Turing, 1947

The last decade has been transformative for AI, arousing both fear and excitement for humanity. Seen as ‘the new electricity’, AI has advanced to the point that it will have such a systemic impact that it could substantially change all aspects of society for the next century.

Whilst it is easy to understand the potential effects of AI on sectors such telecommunications, transportation, traffic management, health care, evaluating its long-term effects on education, culture and the audiovisual sector is considerably more challenging. Although there is a consensus that AI and automation is likely to create more wealth and to simplify a vast array of processes, the use of AI has also raised serious concerns that it may result in an increase in inequality, discrimination and unemployment.

The potential impact of AI on education, culture and the audiovisual sector is however rarely discussed and is mostly unknown. Yet this question is of utmost importance because AI is already being used to teach curricula, as well as to produce movies, songs, stories and paintings.

The purpose of this report is therefore to understand concretely how AI currently impacts these sectors and how future technological advances in AI will impact them further over the next decade. In particular, the Rapporteur reflects on how AI may transform these sectors and which particular regulatory challenges the Union may have to face in that regard.

(i) AI is reshaping education

AI is transforming learning, teaching, and education radically. The whirlwind speed of technological development accelerates the radical transformation of educational practices, institutions and policies. In this field, AI has many applications, such as customisable approaches to learning, AI-based tutors, textbooks and course material with customised content, smart algorithms to determine best teaching methods, AI game engines, and adaptive user models in personalised learning environments (PLE) which can allow early identification of difficulties, such as dyslexia or risks of early school leaving.

Personalised learning experience is the cornerstone of the use of AI in education. It would allow students to enjoy an educational approach that is fully tailored to their individual abilities, needs and difficulties, whilst enabling teachers to closely monitor students’ progress. However in order to make personalised education a reality, large amounts of personal data need to be collected, used and analysed.

The Rapporteur stresses in that regard that the current lack of access to personal data on students is likely to prevent the successful implementation of AI in education. It is thus essential to ensure the safety and transparency of personal data collection, use, management and dissemination, whilst safeguarding confidentiality and privacy of learners’ personal data. Moreover, addressing the risks of AI potential bias, as well as tackling the issue of data storage should be a priority in any initiatives for the wide deployment of AI in the education system at Union level.

Although there is little chance that teachers will be replaced by machines in the near future, the increasing use of AI in education implies the need to rethink education overall, as well as to reflect on the redefinition of teaching, the role of the teachers, and, as a result, the subsequent retraining required to adapt to a AI-based educational system.

Considering that less than 40% of teachers in the Union have received courses on ICT inclusion in the classroom throughout their Initial Teacher Education (ITE), the Rapporteur would like to stress the crucial importance of training teachers so they acquire digital skills as a prerequisite to becoming familiar with AI. They could then take advantage of AI technologies, but also make them aware of the potential dangers of AI.

This issue can also be seen more widely, with 42% of the Union population still lacking basic digital skills. There are also serious regional discrepancies in access to digital infrastructure and in digital skills attainment across the Union.

Emerging technology trends related to digital transformation, such as AI, have profound implications in terms of the skills required for the evolving digital economy. In particular, the notion of lifelong learning in AI has emerged as one of the key strategies for job security and employment in the digital era.

The Rapporteur suggests that citizens are trained to acquire the necessary digital skills, whilst carefully assessing what AI-related skills are needed today and in the future, and that the necessary measures are taken to address existing and emerging skill gaps.

It also crucial to ensure that the prerequisites for the deployment and the relevant use of AI, in terms of internet access, connectivity, networks and infrastructures, are met.

 (ii) AI can be used to safeguard and promote cultural heritage

In recent years, AI has been of increasing relevance to cultural heritage, notably in response to potential modern threats, such as climate change or conflicts. AI can have various applications in that regard: it can be used to enhance users’ experience by enabling visitors of cultural institutions and museums to create personal narrative trails or to enjoy virtual tour guides. Conversational bots could communicate in an interactive way about cultural heritage on any topics and in any language. They would also make the access to information easier whilst providing a vivid cultural experience to users.

AI could also facilitate the understanding of the history of the Union, such as how the ‘Time Machine Project’ aims to create advanced AI technologies to make sense of vast amounts of information from complex historical data sets stored in archives and museums. This enables the transformation of fragmented data into useable knowledge by mapping Union’s entire social, cultural and geographical evolution. This may facilitate the exploration of the cultural, economic, and historical development of European cities, and improve understanding thereof.

(iii) AI changes the way the cultural and creative industries work, in particular the audiovisual sector

AI use is rapidly expanding in media with many applications:

- Data-driven marketing and advertising, by training machine learning algorithms to develop promotional movie trailers and design advertisements,

- Personalisation of users’ experience, by using machine learning to recommend personalised content based on data from user activity and behaviour,

- Search optimisation, by using AI to improve the speed and efficiency of the media production process and the ability to organise visual assets,

- Content creation, by generating video clips from automatic video segments ready for broadcast and special effects, such as re-creating a younger version of an actor digitally or creating new content with a deceased actor,

- Script writing such as simple factual text creation (sports and news reports produced by robots), but also for writing fictional stories, such as the experimental short movie ‘Sunspring’,

- Viewer interaction on complex story lines, such as the last episode of the British series ‘Black Mirror’, ‘Bandersnatch’,

- Automated captioning and subtitling, such audio-to-text processes, for viewers with disabilities

- Automated content moderation on audiovisual content.

Whilst AI offers a wide range of opportunities in producing high quality cultural and creative content, the centralised distribution and access to such content raises a number of ethical and legal issues, notably on data protection, freedom of expression and cultural diversity.

Cultural and creative works, notably audiovisual works, are mainly distributed through

large centralised platforms, which conditions media consumption to the proprietary algorithms developed by these platforms.

The Rapporteur points out that algorithm-based personalised recommendations are potentially detrimental to cultural and linguistic diversity, preventing under-represented cultural and creative content from appearing in suggestions provided by these systems. On the largest platforms, the criteria used to select or recommend a work are neither transparent nor auditable, and are likely to be decided on the basis of economic factors that solely benefit these platforms.

The question of cultural and linguistic diversity in recommendation systems is therefore crucial and must be addressed. The Rapporteur stresses the need to set up a clear legal framework for transparent, accountable and inclusive algorithms, in order to safeguard and promote cultural and linguistic diversity.

Regulatory challenges triggered by AI applications within the audiovisual sector are also linked to existing legal acts, such as the AVMSD. Thus a more in-depth assessment might be needed as to the urgency and/or political momentum for future adaptions of these files to AI.

Whilst AI can help empower many creators, making CCS more prosperous and driving cultural diversity, the large majority of artists and entrepreneurs may not still be familiar with AI tools.

There is a lack of technical knowledge among creators precluding them from experimenting with machine learning and reaping the benefits they can bring. Therefore it is essential to assess which skills would be needed in the near future, whilst at the same time improving training systems, including upskilling and reskilling, guaranteeing lifelong learning throughout the whole working life and beyond.

In that context, the Rapporteur suggests setting up an AI observatory with an objective of harmonising and facilitating evidence-based scrutiny of new developments in AI in order to tackle the question of auditability and accountability of AI applications in CCS.

(v) Countering fake news

AI-technologies are increasingly used to disseminate fake news, notably through the use of ‘deepfakes’.

Deepfakes are synthetic images or videos generated by AI using ‘deep learning machines’ and generative adversial networks (GAN). Humans cannot distinguish deepfakes from authentic content. Deepfakes can be used for all kinds of trickery, most commonly used for ‘face swaps’, from harmless satires and film tweaks to malicious hoaxes, targeted harassment, deepfake porn or financial fraud, The danger of deepfakes is to make people believe something is real when it is not, and thus may be used as a particularly powerful and potent weapon for online disinformation, spreading virally on platforms and social media, where they can influence public opinion, voting processes and election results.

Whilst AI is frequently singled out for its role in spreading fake news, it could also play a significant role in countering and combating fake news and disinformation, as evidenced by projects such as “Fake News Challenge”. AI systems can reverse-engineer AI generated fake news, and help spot manipulated content. However algorithms generating deepfakes are getting more and more sophisticated, and detecting them as a result is getting increasingly difficult.

The Rapporteur therefore stresses the need to tackle the misuse of AI in disseminating fake news and online misinformation, notably by exploring ways to efficiently detect deepfakes.

OPINION OF THE COMMITTEE ON CIVIL LIBERTIES, JUSTICE AND HOME AFFAIRS (16.7.2020)

for the Committee on Culture and Education

on artificial intelligence in education, culture and the audiovisual sector

(2020/2017(INI))

Rapporteur for opinion (*): Ondřej Kovařík

(*) Associated committee – Rule 57 of the Rules of Procedure

SUGGESTIONS

The Committee on Civil Liberties, Justice and Home Affairs calls on the Committee on Culture and Education, as the committee responsible, to incorporate the following suggestions into its motion for a resolution:

1. Underlines that the use of AI in the education, culture and audiovisual sectors must fully respect fundamental rights, freedoms and values, including privacy, the protection of personal data, non-discrimination and freedom of expression and information, as enshrined in the EU Treaties and the Charter of Fundamental Rights of the European Union; welcomes the Commission’s White Paper on Artificial Intelligence in this regard, and invites the Commission to include the educational sector, limited to areas posing significant risks, in the future regulatory framework for high-risk AI applications;

2. Recalls that AI may give rise to biases and thus to various forms of discrimination based on sex, race, colour, ethnic or social origin, genetic features, language, religion or belief, political or any other opinion, membership of a national minority, property, birth, disability, age or sexual orientation; in this regard, recalls that everyone’s rights must be ensured and that AI initiatives must not be discriminatory in any form;

3. Emphasises that such bias and discrimination can arise from already biased sets of data, reflecting existing discrimination in society; stresses that AI must avoid bias leading to prohibited discrimination and must not reproduce discrimination processes; underlines the need to take these risks into account when designing AI technologies, as well as, the importance of working with AI technology providers to address persistent loopholes facilitating discrimination, and recommends that AI designing and developing teams should reflect the diversity of society;

4. Notes that the use of AI in education brings a wide range of possibilities and opportunities, for instance to facilitate access to information, to improve research methods or to understand how pupils learn and to offer them customisation, while at the same time posing risks regarding equal access to education and learning equalities at an increasingly younger age and for vulnerable and historically disadvantaged groups; calls for a sufficient data-sharing infrastructure between AI applications and public research entities; points out that equity and inclusion are core values that should be duly taken into account when designing policies for AI in education; calls for the non-discriminatory use of AI in the education sector; recalls the risks and discrimination that may arise from recently developed AI tools used for purposes of school admissions, and calls for them to be rectified as soon as possible; underlines the need for a proper assessment of the AI tools used in the education sector to identify their impact on the rights of the child;

5. Acknowledges that using digital and AI technologies can help develop increasingly effective educational tools and lead to a more inclusive society, by countering traditional forms of discrimination including lack of access to services, by bringing education to disadvantaged communities, persons with disabilities in line with the EU Accessibility Act, and other categories of European citizens lacking proper access to education, and by providing access to adequate learning opportunities;

6. Underlines that the benefits of AI should be shared with all parts of society, leaving no one behind; stresses the need to fully take into consideration the specific needs of the most vulnerable groups, such as children, persons with disabilities, elderly people and other groups at risk of exclusion; expresses its concerns about limited accessibility of the internet in some regions across the EU, and calls on the Commission and the Member States to deploy sustained efforts to ameliorate telecommunications infrastructures;

7. Recognises the possibilities of AI in the culture sector in terms of developing music, art and other cultural expressions; emphasises that freedom of expression is an important freedom and value and that a pluriform cultural landscape is of great value to society; calls on the Commission to keep these values in mind when drafting its proposals on AI;

8. Welcomes the Commission’s plan to update the Digital Education Action Plan so as to make it more ambitious and integrated with a view to making educational systems fit for the digital age, notably through making better use of data and AI-based technologies; calls on all stakeholders, both public and private, to closely cooperate in implementing these educational reforms;

9. Stresses the need to ensure more general public awareness of AI at all levels, as a key element to enable the public to make informed decisions and help strengthen the resilience of our societies; underlines that this must also include public awareness of the risks in terms of privacy and biases related to AI; invites the Commission and the Member States to include the above in educational programmes and programmes which support the arts;

10. Underlines the urgent need to educate the public at every level in the use of AI and to equip all European citizens, including vulnerable groups, with basic digital skills enabling equal social and economic opportunities, as well as the need to have high-quality ICT programmes in education systems, at all levels; calls for the digital gender gap not to be underestimated and for measures to be taken to remedy it; welcomes the upcoming update of the Skills Agenda aimed at allowing everyone to benefit from EU digital transformation; emphasises the importance of training teachers and educators in the use of AI, especially those responsible for underage students; notes that significant skills shortages still exist in the digital and technology sectors; underlines the importance of diversifying this sector and to encourage students, in particular women and girls, to enrol in Science, Technology, Engineering and Mathematics (STEM) courses, in particular in robotics and AI-related subjects, in addition to those related to their career aspirations; calls for more financial and scientific resources to motivate skilled people to stay in the EU and to attract those with skills from abroad; furthermore, notes that there are a considerable number of start-ups working with AI and developing AI technologies; stresses that small and medium-sized enterprises (SMEs) will require additional support and AI-related training to comply with digital and AI-related regulation;

11. Recalls that data protection and privacy can be particularly affected by AI; underlines the principles established in Regulation (EU) 2016/679 of the European Parliament and of the Council (the General Data Protection Regulation (GDPR))[18] as binding principles for AI deployment; recalls that all AI applications need to fully respect Union data protection law, namely the GDPR and Directive (EC) 2002/58 of the European Parliament and of the Council (the ePrivacy Directive)[19] (currently under revision);

12. Recalls that children constitute a vulnerable public who deserve particular attention and protection; recalls that automated decisions about natural persons based on profiling, where they have legal or similar effects, must be strictly limited and always require the right to human intervention and to explicability under the GDPR; underlines that this should be strictly adhered to, especially in the education system, where decisions about future chances and opportunities are taken; observes that a few private companies are dominating the educational technology (edtech) sector in some Member States, and believes this should be scrutinised through EU competition rules; strongly recalls that data of minors is strictly protected by the GDPR, and that children’s data can only be processed if completely anonymised or where consent has been given or authorised by the holder of parental responsibility over the child; therefore calls for stronger protection and safeguards in the education sector where children’s data are concerned; calls for clear information to be provided to children and their parents, including via awareness and information campaigns, about the possible use and processing of children’s data;

13. Underlines the specific risks existing in the use of AI automated recognition applications, which are currently developing rapidly; recalls that children are a particularly sensitive public; recommends that the Commission and the Member States ban automated biometric identification, such as facial recognition for educational and cultural purposes, on educational and cultural premises, unless its use is allowed in law;

14. Calls on the Commission and the Member States to implement an obligation of transparency and explainability of AI-based automated individual decisions taken within the framework of prerogatives of public power, and to implement penalties to enforce such obligations; calls for the implementation of systems which use human verification and intervention by default, and for due process, including the right of appeal, and access to remedies; recalls that automated decisions about natural persons based on profiling, where they have legal or similar effects, must be strictly limited and always require the right to human intervention and to explainability under the GDPR;

15. Calls for independent audits to be conducted regularly to examine whether AI applications being used and the related checks and balances are in accordance with specified criteria, and for those audits to be supervised by independent and sufficient overseeing authorities; calls for specific stress tests to assist and enforce compliance;

16. Points out that AI can play a major role in the rapid spread of disinformation; therefore calls on the Commission to assess the risks of AI assisting the spread of disinformation in the digital environment, and to propose recommendations, among others, for action against any AI-powered threats to free and fair elections and democracy; observes that deep fakes can also be used to manipulate elections, to disseminate disinformation and for other undesirable actions; notes furthermore that the immersive experiences facilitated by AI can be exploited by malicious actors; asks the Commission to propose recommendations, including possible restrictions in this regard, in order to adequately safeguard against the use of these technologies for illegal purposes; also calls for an assessment of how AI could be used to help counter disinformation; calls on the Commission to ensure that any future regulatory framework does not lead to censorship of legal individual content uploaded by users; recalls that critical thinking and the ability to interact with skill and confidence in the online environment are needed more than ever;

17. Notes that AI is often used to enable automated decision-making algorithms to disseminate and order the content displayed to users; stresses that these algorithms are a ‘black box’ for users; calls on the Commission to address the ways in which content moderation algorithms are optimised towards engagement of their users; also calls on the Commission to propose recommendations to increase user control over the content they see, and to ask AI applications and internet platforms to give users the possibility to choose to have content displayed in a neutral order, in order to give them more control on the way content is ranked to them, including options for ranking outside their ordinary content consumption habits and for opting out completely from any content curation;

18. Notes the potential negative impact of personalised advertising, in particular micro-targeted and behavioural advertising, and of assessment of individuals, especially minors, without their consent, by interfering in the private life of individuals, asking questions as to the collection and use of the data used to personalise advertising, and offering products or services or setting prices; calls, therefore, on the Commission to introduce strict limitations on targeted advertising based on the collection of personal data, starting by introducing a prohibition on cross-platform behavioural advertising, while not hurting SMEs; recalls that currently the ePrivacy Directive only allows targeted advertising subject to opt-in consent, otherwise making it illegal; calls on the Commission to prohibit the use of discriminatory practices for the provision of services or products;

19. Underlines that what is illegal offline shall be illegal online; notes that AI tools have the potential and are already used to fight illegal content online, but strongly recalls ahead of the Digital Services Act expected for the end of this year that such tools must always respect fundamental rights, especially freedom of expression and information, and should not lead to a general monitoring obligation for the internet, or to the removal of legal material disseminated for education, journalistic, artistic or research purposes; stresses that algorithms should be used only as a flagging mechanism in content moderation, subject to human intervention, as AI is unable to reliably distinguish between legal, illegal and harmful content; notes that terms and conditions should always include community guidelines as well as an appeal procedure;

20. Notes the benefits and risks of AI in terms of cybersecurity and its potential in combating cybercrime, and emphasises the need for any AI solutions to be resilient to cyberattacks while respecting EU fundamental rights, especially the protection of personal data and privacy; stresses the importance of monitoring the safe use of AI and the need for close collaboration between the public and private sectors to counter user vulnerabilities and the dangers arising in this connection; calls on the Commission to evaluate the need for better prevention in terms of cybersecurity and mitigation measures thereof;

21. Stresses that next-generation digital infrastructure and internet coverage are of strategic significance for providing AI-powered education to European citizens; in light of the COVID-19 crisis, calls on the Commission to elaborate a strategy for a European 5G that ensures Europe’s strategic resilience and is not dependent on technology from states who do not share our values;

22. Calls on the Commission and the Member States to support the use of AI in the area of digitalised cultural heritage.

INFORMATION ON ADOPTION IN COMMITTEE ASKED FOR OPINION

FINAL VOTE BY ROLL CALL IN COMMITTEE ASKED FOR OPINION

Key to symbols:

+ : in favour

- : against

0 : abstention

OPINION OF THE COMMITTEE ON THE INTERNAL MARKET AND CONSUMER PROTECTION (6.7.2020)

for the Committee on Culture and Education

on artificial intelligence in education, culture and the audiovisual sector

(2020/2017(INI))

Rapporteur for opinion: Kim Van Sparrentak

SUGGESTIONS

The Committee on the Internal Market and Consumer Protection calls on the Committee on Culture and Education, as the committee responsible, to incorporate the following suggestions into its motion for a resolution:

A. whereas artificial intelligence (AI) has the potential to offer solutions for day-to-day challenges of the education sector such as the personalisation of learning, monitoring learning difficulties, automation of subject-specific content/knowledge, providing better professional training, and supporting the transition to a digital society;

B. whereas AI could have practical applications in terms of reducing the administrative work of educators and educational institutions, freeing up time for their core teaching and learning activities;

C. whereas the application of AI in education raises concerns around the ethical use of data, learners’ rights, data access and protection of personal data, and therefore entails risks to fundamental rights such as the creation of stereotyped models of learners’ profiles and behaviour that could lead to discrimination or risks of doing harm by the scaling-up of bad pedagogical practices;

D. whereas AI applications are omnipresent in the audiovisual sector, in particular on audiovisual content platforms;

1. Notes that the Commission has proposed to support public procurement in intelligent digital services, in order to encourage public authorities to rapidly deploy products and services that rely on AI in areas of public interest and the public sector; highlights the importance of public investment in these services and the complementary added value provided by public-private partnerships in order to secure this objective and deploy the full potential of AI in the education, culture and audiovisual sectors; emphasises that in the education sector, the development and deployment of AI should involve all those participating in the educational process and wider society and take into account their needs and the expected benefits, especially for the most vulnerable and disadvantaged, in order to ensure that AI is used purposefully and ethically and delivers real improvements for those concerned; considers that products and services developed with public funding should be published under open-source licences with full respect for the applicable legislation, including Directive (EU) 2019/790 of the European Parliament and of the Council of 17 April 2019 on copyright and related rights in the Digital Single Market; stresses the importance of this deployment for reskilling and upskilling the European labour market, and particularly in the culture and audiovisual sectors, which will be severely impacted by the COVID-19 crisis;

2. Recognises that children are an especially vulnerable group in terms of influencing their behaviour; stresses that while AI can be a tool that can benefit their education, it is necessary to take into account the technological, regulatory and social aspects of the introduction of AI in education, with adequate safeguards and a human-centric approach that ensures that human beings are, ultimately, always able to control and correct the system’s decisions; points to the need for a review and updating of the relevant sectoral rules; underlines in this regard that the legal framework governing AI in the education sector should, in particular, provide for legally binding measures and standards to prevent practices that would undermine fundamental rights and freedoms, and ensure the development of trustworthy, ethical and technically robust AI applications, including integrated digital tools, services and products such as robotics and machine learning;

3. Notes the potential of AI-based products in education, especially in making high-quality education available to all pupils in the EU; stresses the need for governments and educational institutions to rethink and rework educational programmes with a stronger emphasis on STEAM subjects, in order to prepare learners and consumers for the increasing presence of AI and to facilitate the acquisition of cognitive skills; underlines the need to improve the digital skills of those participating in the educational process and wider society, while having regard to the objectives of ‘A Europe fit for the digital age’;

4. Underlines that algorithmic systems can be an enabler for reducing the digital divide in an accelerated way, but unequal deployment risks creating new divides or accelerating the deepening of the existing ones; expresses its concern that knowledge and infrastructure are not developed in a consistent way across the EU, which limits the accessibility of products and services that rely on AI, in particular in sparsely populated and socio-economically vulnerable areas; calls on the Commission to ensure cohesion in the sharing of the benefits of AI and related technologies;

5. Calls on the Commission to consider education as a sector where significant risks can be expected to occur from certain uses of AI applications, which may potentially undermine fundamental rights and result in high costs in both human and social terms, and to take this consideration into account when assessing what types or uses of AI applications would be covered by a regulatory framework for high-risk AI applications, given the importance of ensuring that education continues to contribute to the public good and given the high sensitivity of data on pupils, students and other learners; calls on the Commission to include certain AI applications in the education sector, such as those that are subject to certification schemes or include sensitive personal data, in the regulatory framework for high-risk AI applications; underlines that data sets used to train AI and the outputs should be reviewed in order to avoid all forms of stereotypes, discrimination and biases, and where appropriate, make use of AI to identify and correct human biases where they might exist; points out, accordingly, that appropriate conformity assessments are needed in order to verify and ensure that all the provisions concerning high-risk applications are complied with, including testing, inspection and certification requirements; stresses the importance of securing the integrity and the quality of the data;

6. Welcomes the efforts of the Commission to include digital skills as part of the qualification requirements for certain professions harmonised at EU level under the Professional Qualifications Directive; highlights the need to ensure mutual recognition of professional qualifications in AI skills across the EU, as several Member States are upgrading their educational offer with AI-related skills and putting in place specific curricula for AI developers; stresses the need for these to be in line with the assessment list of the Ethical Guidelines for Trustworthy AI, and welcomes the Commission’s proposal to transform this list into an indicative curriculum for AI developers; underlines the importance of training highly skilled professionals in this area, including ethical aspects in their curricula, and supporting underrepresented groups in the field, as well as creating incentives for those professionals to seek work within the EU;

7. Takes note that schools and other public education providers are increasingly using educational technology services, including AI applications; expresses its concern that these technologies are currently provided by just a few technology companies; stresses that this may lead to unequal access to data and limit competition by market dominance and restricting consumer choice; encourages public authorities to take an innovative approach towards public procurement, so as to broaden the range of offers that are made to public education providers across Europe; stresses in this regard the importance of supporting the uptake of AI by SMEs in the education, culture and audiovisual sector through the appropriate incentives that create a level playing field; calls, in this context, for investment in European IT companies in order to develop the necessary technologies within the EU; considers that technologies used by public education providers or purchased with public money should be based on open-source technology where possible, while having full respect for the applicable legislation, including Directive (EU) 2019/790 of the European Parliament and of the Council of 17 April 2019 on copyright and related rights in the Digital Single Market;

8. Calls for the data used by AI applications in the education sector to be accessible, interoperable and of high quality, and to be shared with the relevant public authorities in a standardised way and with respect for copyright and trade secrets legislation, so that the data can be used, in accordance with the European data protection and privacy rules and ethical, democratic and transparency standards, in the development of curricula and pedagogical practices (in particular when these services are purchased with public money or offered to public education providers for free, considering that education is a common good); calls on the Commission to ensure fair access to data for all companies, and in particular SMEs and cultural and creative companies, which play an essential role in sustaining social cohesion and cultural diversity in Europe, as well as democratic values;

9. Stresses the importance of developing guidelines for the public procurement of such services and applications for the public sector, including for education providers, in order to ensure the relevant educational objectives, consumer choice, a level and fair playing field for AI solution providers and respect for fundamental rights; stresses the need for public buyers to take into account specific criteria linked to the relevant educational objectives, such as non-discrimination, fundamental rights, diversity, the highest standards of privacy and data protection, accessibility for learners with special needs, environmental sustainability and, specifically when purchasing services for public education providers, the involvement of all those participating in the educational process; stresses the need to strengthen the market by providing SMEs with the opportunity to participate in the procurement of AI applications in order to ensure the involvement of technology companies of all sizes in the sector and thus guarantee resilience and competition;

10. Underlines the unreliability of the current automated means of removing illegal content from online platforms on which audiovisual content is shared, which may lead to inadvertent removal of legitimate content; notes that neither the E-Commerce Directive nor the revised Audiovisual Media Services Directive on video sharing platforms imposes a general monitoring obligation; recalls, to that end, that there should be no general monitoring, as stipulated in Article 15 of the E-Commerce Directive, and that specific content monitoring for audiovisual services should be in accordance with the exceptions laid down in the European legislation; recalls the key requirements for AI applications, such as accountability, including review structures within business processes, and reporting of negative impacts; emphasises that transparency should also include traceability and explainability of the relevant systems; recalls that AI applications must adhere to internal and external safety protocols, which should be technically accurate and robust in nature; considers that this should extend to operation in normal, unknown and unpredictable situations alike;

11. Calls for recommendation algorithms and personalised marketing on audiovisual platforms, including video streaming platforms, news platforms and platforms disseminating cultural and creative content, to be explainable, to the extent technically possible, in order to give consumers an accurate and comprehensive insight into these processes and content and ensure that personalised services are not discriminatory and are in line with the recently adopted Platform to Business Regulation and New Deal for Consumers Omnibus Directive; stresses the need to guarantee and properly implement the right of users to opt out from recommended and personalised services; points out in this regard that a description should be provided to users that allows for a general and adequate understanding of the functions concerned, notably on the data used, the purpose of the algorithm, and personalisation and its outcomes, following the principles of explainability and fairness; calls for the development of mechanisms providing monitoring of the consumer’s rights of informed consent and freedom of choice when submitting data;

12. Notes that the deployment of AI in customs screening procedures may support efforts to prevent the illicit trafficking of cultural heritage, in particular to supplement systems which allow customs authorities to target their efforts and resources on those items presenting the highest risk;

13. Underlines that consumers must be informed when they are interacting with an automated decision process and that their choices and performance must not be limited; stresses that the use of AI mechanisms for commercial surveillance of consumers must be countered, even if it concerns ‘free services’, by ensuring that it is strictly in line with fundamental rights and the GDPR; stresses that all regulatory changes must take in consideration the impact on vulnerable consumers;

14. Points out that the deployment, development and implementation of AI must make it easier for consumers and learners with some form of disability to use tools to access audiovisual content;

15. Underlines the need for upskilling of the future workforce; recognises the benefits of forecasting which jobs will be disrupted by digital technology such as automation, digitalisation and AI;

16. Points out that the AI systems that are developed, implemented and used in the European Union, in any of the three sectors referred to in this report, must reflect the EU’s cultural diversity and multilingualism.

INFORMATION ON ADOPTION IN COMMITTEE ASKED FOR OPINION

FINAL VOTE BY ROLL CALL IN COMMITTEE ASKED FOR OPINION

Key to symbols:

+ : in favour

- : against

0 : abstention

OPINION OF THE COMMITTEE ON LEGAL AFFAIRS (22.9.2020)

for the Committee on Culture and Education

on artificial intelligence in education, culture and the audiovisual sector

(2020/2017(INI))

Rapporteur for opinion: Angel Dzhambazki

SUGGESTIONS

The Committee on Legal Affairs calls on the Committee on Culture and Education, as the committee responsible, to incorporate the following suggestions into its motion for a resolution:

1. Underlines the strategic importance of using artificial intelligence (AI) and related technologies, and stresses that the European approach in this regard must be human-centred so that AI genuinely becomes an instrument in the service of people and the common good, contributing to the general interest of citizens, including in the audiovisual, cultural and educational sectors; points out that AI can support content creation in those sectors; stresses that AI can support content creation in the education, culture and audiovisual sectors, alongside information and educational platforms, including listings of different kinds of cultural objects and a multitude of data sources; notes the risks of infringement of intellectual property rights (IPRs) when blending AI and different technologies with a multiplicity of sources (documents, photos, films) to improve the way those data are displayed, researched and visualised; calls for the use of AI to ensure a high level of IPR protection within the current legislative framework, for example by alerting individuals and businesses if they are in danger of inadvertently infringing the rules or assisting IPR rightholders if the rules are actually infringed; emphasises, therefore, the importance of having an appropriate European legal framework for the protection of IPRs in connection with the use of AI;

2. Highlights that the consistent integration of AI in the education sector has the potential to meet some of the biggest challenges of education, to come up with innovative teaching and learning practices, and finally, to accelerate progress towards achieving the Sustainable Development Goals in order to meet the targets of the 2030 Agenda for Education;

3. Reiterates the importance of access to culture for every citizen throughout the Union; highlights in this context the importance of the exchange of best practices among Member States, educational facilities and cultural institutions and similar stakeholders; further considers it of vital importance that the resources available at both EU and national level are used to the maximum of their potential in order to further improve access to culture; stresses that there are a multitude of options to access culture and that all varieties should be explored in order to determine the most appropriate option; highlights the importance of consistency with the Marrakech Treaty;

4. Calls on the Commission to realise the full potential of artificial intelligence (AI) for the purposes of improving communication with citizens, through cultural and audiovisual online platforms, for example by keeping citizens informed of what is happening at decision-making level, narrowing the gap between the EU and the grassroots, and promoting social cohesion between EU citizens;

5. Highlights that education, culture and the audiovisual sector are sensitive areas for the use of AI and related technologies since they have the potential to impact our societies and the fundamental rights they uphold; contends, therefore, that legally binding ethical principles should be observed in their deployment, development and use;

6. Notes how artificial intelligence and related technologies may be used in developing or applying new methods of education in areas including language learning, academia generally, specialised learning, etc; highlights the importance not only of using such technologies for educational purposes but also of digital literacy and public awareness of the former; stresses the importance of providing educators, trainers and others with the right tools and know-how with regard to AI and related technologies in terms of what they are, how they are used and how to use them as tools properly and according to the law, so as to avoid IPR infringements; highlights in particular the importance of digital literacy for staff working in education, as well as of improving digital training for the elderly, considering that newer generations already have a basic notion of these technologies, having grown up with them;

7. Emphasises that European artificial intelligence should safeguard and promote core values of our Union such as democracy, independent and free media and information sources, quality education, environmental sustainability, gender balance and cultural and linguistic diversity;

8. Calls on the Commission, the Member States and the business community to actively and fully exploit the potential of AI in providing the facts and combating fake news, disinformation, xenophobia and racism on cultural and audiovisual online platforms, while at the same time avoiding censorship;

9. Notes that the independence of the creative process raises issues related to ownership of IPRs; considers, in this connection, that it would not be appropriate to seek to impart legal personality to AI technologies;

10. Notes that AI can play an important role in promoting and protecting our European and national cultural diversity, especially when used by audiovisual online platforms in promoting content to customers;

11. Notes that AI could benefit the research sector, for example through the role that predictive analytics can play in fine-tuning data analysis, for example on the acquisition and movement of cultural objects; stresses that the EU must step up investment and foster partnerships between industry and academia in order to enhance research excellence at European level;

12. Notes the important role which independent media play in culture and the daily life of citizens; stresses that fake media represent a fundamental problem, as copyright and IPRs generally are being constantly infringed; calls on the Commission, in cooperation with the Member States, to continue its work on raising awareness of this problem, countering the effects of fake media as well as the source problems; considers it of importance, furthermore, to develop educational strategies to improve digital literacy specifically in this regard;

13. Notes that AI-based software, such as image recognition software, could vastly enhance the ability of educational facilities and teachers to provide and develop modern, innovative and high-quality schooling methods, improve the digital literacy and e-skills of the entire population, and enable education to be more accessible; considers that such schooling methods should nevertheless be assessed as to their reliability and accuracy and should ensure fairness in education, non-discrimination, and the safety of children and minors both within educational facilities and when connected remotely within an educational context; highlights the importance of privacy and data protection legislation in order to ensure adequate protection of personal data, in particular children’s data, through transparent and reliable data sources respectful of IPRs; considers it vital that these technologies are only integrated into the existing systems if the protection of fundamental rights and privacy is an absolute given; stresses, however, that recognition software must be used only for educational purposes and not under any circumstances to monitor access to establishments; highlights in this regard the dependence on external data and a few market-dominating software providers; recalls that technologies procured with public money should be developed as open source software to enable the sharing and reuse of resources, making them available throughout the EU and thus increasing benefits and reducing public spending, while ensuring full respect for the applicable legislation, including Directive (EU) 2019/790 of the European Parliament and of the Council of 17 April 2019 on copyright and related rights in the Digital Single Market[20];

14. Notes that if the use of AI is to benefit the education and research sector, the EU must encourage training in the skills of the future, and in particular an ethical and responsible approach to AI technologies; adds, with that aim in view, that this training must not be reserved for pupils focusing on scientific and technical subjects, who are already more familiar with these tools, but must instead target as many people as possible, in particular in the younger generations;

15. Stresses that the need for investment in research and innovation regarding the use and development of AI and its cultural, educational and audiovisual applications is a key consideration in this respect; calls on the Commission to find additional funding to promote research and innovation regarding AI applications in these sectors;

16. Expresses serious concern that schools and other education providers are becoming increasingly dependent on educational technology services, including AI applications, provided by companies with a dominant market position, most of which are based outside the EU;

17. Underlines the need to ensure EU-wide digital and AI literacy, especially through the development of training opportunities for teachers; insists that the use of AI technologies in schools should contribute to narrowing the social and regional digital gap;

18. Highlights that the COVID-19 pandemic crisis can be considered as a probation period for the development and use of digital and AI-related technologies in the educational and cultural sectors, as exemplified by the many online schooling platforms and online tools for cultural promotion employed across the Member States; calls on the Commission, therefore, to take stock of those examples when considering a common EU approach to the increased use of such technological solutions;

19. Points out that data protection and privacy can be particularly seriously affected by AI; advocates compliance with the principles laid down in the General Data Protection Regulation (GDPR);

20. Calls on the Commission to take more effective steps to protect the personal data of pupils and teachers in the education sphere;

21. Emphasises that the interaction between AI and the creative industries is complex and requires an in‑depth assessment; welcomes the ongoing study ‘Trends and Developments in Artificial Intelligence - Challenges to the IPRS Framework’ and the study on ‘Copyright and new technologies: copyright data management and Artificial Intelligence’; underlines the importance of clarifying the conditions of use of copyright‑protected content as data input (images, music, films, databases, etc) and in the production of cultural and audiovisual outputs, whether created by humans with the assistance of AI or autonomously generated by AI technologies; invites the Commission to study the impact of AI on the European creative industries; reiterates the importance of European data and welcomes the statements made by the Commission in this regard, as well as the placing of artificial intelligence and related technologies high on the agenda;

22. Emphasises the role of an author’s personality for the expression of free and creative choices that constitute the originality of works[21]; underlines the importance of limitations and exceptions to copyright when using content as data input, notably in education, academia and research, and in the production of cultural and audiovisual outputs, including user-generated content;

23. Emphasises that the interaction between AI and the creative industries is complex and requires an in-depth assessment; takes the view that consideration should be given to protecting AI-generated technical and artistic creations, in order to encourage this form of creativity;

24. Stresses that in the data economy context, better copyright data management is achievable, for the purpose of better remunerating authors and performers, notably in enabling the swift identification of the authorship and right ownership of content, thus contributing to lowering the number of orphan works; further highlights that AI technological solutions should be used to improve copyright data infrastructure and the interconnection of metadata in works, but also to facilitate the transparency obligation provided in Article 19 of the Directive on Copyright and related rights in the Digital Single Market for up‑to‑date, relevant and comprehensive information on the exploitation of authors’ and performers’ works and performances, particularly in the presence of a plurality of rightholders and of complex licensing schemes;

25. Stresses the need to work on the most efficient way of reducing bias in AI systems, in line with ethical and non-discrimination standards; underlines that data sets used to train AI should be as broad as possible in order to represent society in the best relevant way, that the outputs should be reviewed to avoid all forms of stereotypes, discrimination and biases, and, when appropriate, AI should be made use of to identify and correct human biases where they could exist; calls on the Commission to encourage and facilitate the sharing of de-biasing strategies for data;

26. Asks the Commission to assess the impact of AI and AI-related technologies on the audiovisual and creative sector, in particular with regard to authorship and related questions;

27. Calls for the intellectual property action plan announced by the Commission to address the question of AI and its impact on the creative sectors, taking account of the need to strike a balance between protecting IPRs and encouraging creativity in the areas of education, culture and research; considers that the EU can be a leader in the creation of AI technologies if it adopts an operational regulatory framework and implements proactive public policies, particularly as regards training programmes and financial support for research; asks the Commission to assess the impact of IPRs on the research and development of AI and related technologies, as well as on the audiovisual and creative sectors, in particular with regard to authorship, fair remuneration of authors and related questions;

28. Highlights the future role that the inclusion of AI-based technological tools should have in terms of conservation, disclosure and heritage control, as also in the associated research projects;

29. Stresses the need to strike a balance between, on the one hand, the development of AI systems and their use in the educational, cultural and audiovisual sectors and, on the other, measures to safeguard competition and market competitiveness for AI companies in these sectors; emphasises in this regard the need to encourage companies to invest in the innovation of AI systems used in these sectors, while at the same time ensuring that those providing such applications do not obtain a market monopoly;

30. Stresses that in no scenario could the use of AI and related technologies become a reality without human oversight; reiterates the importance of fundamental rights and the overarching supremacy of data and privacy protection legislation, which is imperative when dealing with such technologies;

31. Asks the Commission to assess the impact of AI and AI-related technologies in creating new audiovisual works such as deep fakes, and to establish appropriate legal consequences to be attached to their creation, production or distribution for malicious purposes;

32. Notes that automation and the development of AI could pose a threat to employment, and emphasises once again that priority must be given to safeguarding jobs, in particular in the education, culture and creative sectors;

33. Calls on the Commission to launch an EU-level education plan on digital and AI literacy, in coordination with the Member States, with a particular focus on school students and youth;

34. Calls on the Commission to consider the legal aspects of the outputs produced using AI technologies, as well as cultural content generated with the use of AI and related technologies; considers it important to support the production of cultural content; reiterates, however, the importance of safeguarding the Union’s unique IPR framework and that any changes should be made with the necessary due care, in order not to disrupt the delicate balance; calls on the Commission to produce an in-depth assessment with regard to the possible legal personality of AI-produced content, as well as the application of IPRs to AI-generated content and to content created with the use of AI tools;

35. Calls on the Commission to establish requirements for the procurement and deployment of artificial intelligence and related technologies by EU public sector bodies, to ensure compliance with Union law and fundamental rights; highlights the added value of instruments such as public consultations and impact assessments, to be run prior to the procurement or deployment of artificial intelligence systems, as recommended in the report of the Special Rapporteur to the UN General Assembly on AI and its impact on freedom of opinion and expression[22];

36. Calls on the Commission to lay down rules designed to guarantee effective data interoperability, in order to make content purchased on a platform accessible via any digital tool irrespective of brand;

37. Emphasises that the challenges brought by the use of artificial intelligence and related technologies can only be overcome by establishing data quality obligations and transparency and oversight requirements, in order to enable the public and authorities to assess compliance with Union law and fundamental rights; awaits the Commission’s proposals following its communication on a European strategy for data[23] as regards the sharing and pooling of datasets;

38. Calls on the Commission, in addition, to consider developing, in very close cooperation with Member States and the relevant stakeholders, verification mechanisms or systems for publishers, authors, creators, etc, in order to assist them in verifying what content they may use and to more easily determine what is protected under IPR legislation.

INFORMATION ON ADOPTION IN COMMITTEE ASKED FOR OPINION

FINAL VOTE BY ROLL CALL IN COMMITTEE ASKED FOR OPINION

Key to symbols:

+ : in favour

- : against

0 : abstention

OPINION OF THE COMMITTEE ON WOMEN'S RIGHTS AND GENDER EQUALITY (14.9.2020)

for the Committee on Culture and Education

on artificial intelligence in education, culture and the audiovisual sector

(2020/2017(INI))

Rapporteur for opinion: Maria da Graça Carvalho

SUGGESTIONS

The Committee on Women’s Rights and Gender Equality calls on the Committee on Culture and Education, as the committee responsible, to incorporate the following suggestions into its motion for a resolution:

A. whereas gender equality is a core principle of the European Union enshrined in the Treaties, and should be reflected in all EU policies, including in education, culture, and the audiovisual sector, as well as in the development of technologies such as Artificial Intelligence (AI), these being key channels for changing attitudes and challenging stereotypes and gender biases in existing social norms; whereas the development of digitalisation and technologies like AI are fundamentally transforming our reality and their regulation today will highly influence our future societies; whereas there is a need to advocate for a human-centred approach anchored in human rights and ethics for the development and use of AI;

B. whereas Article 21 of the EU Charter of Fundamental Rights prohibits discrimination on a wide range of grounds and should be a guiding principle; whereas multiple forms of discrimination should not be reproduced in the design, input, development and use of AI systems based on gender-biased algorithms, or in the social contexts in which such algorithms are used;

C. whereas past experiences, especially in technical fields, have shown us that developments and innovations are often based mainly on male data and that women’s needs are not fully reflected; whereas addressing these biases requires greater vigilance, technical solutions and the development of clear requirements of fairness, accountability and transparency;

D. whereas incomplete and inaccurate data sets, the lack of gender-disaggregated data and incorrect algorithms can distort the processing of an AI system and jeopardise the achievement of gender equality in society; whereas data on disadvantaged groups and intersectional forms of discrimination tend to be incomplete and even absent;

E. whereas gender inequalities, stereotypes and discrimination can also be created and replicated through the language and images disseminated by the media and AI-powered applications; whereas education, cultural programmes and audiovisual content have considerable influence in shaping people’s beliefs and values and are a fundamental tool for combatting gender stereotypes, decreasing the digital gender gap, and establishing strong role models; whereas an ethical and regulatory framework must be in place ahead of implementing automatised solutions for these key areas in society;

F. whereas science and innovation can bring life-changing benefits, especially for those who are furthest behind, such as women and girls living in remote areas; whereas scientific education is important for obtaining skills, decent work, and jobs of the future, as well as for breaking with gender stereotypes that regard these as stereotypically masculine fields; whereas science and scientific thinking are key to democratic culture, which in turn is fundamental for advancing gender equality;

G. whereas women are significantly under-represented in the AI sector, whether as creators, developers or consumers; whereas the full potential of women’s skills, knowledge and qualifications in the digital and AI fields as well as that of information, communication and technology (ICT), along with their reskilling, can contribute to boosting the European economy; whereas globally only 22 % of AI professionals are female; whereas the lack of women in AI development not only increases the risk of bias, but also deprives the EU of diversity, talent, vision and resources, and is therefore an obstacle to innovation; whereas gender diversity enhances female attitudes in teams and team performance and favours the potential for innovation in both public and private sectors;

H. whereas in the EU one woman in ten has already suffered some form of cyberviolence since the age of 15 and cyberharassment remains a concern in the development of AI, including in education; whereas cyberviolence is often directed at women in public life, such as activists, women politicians and other public figures; whereas AI and other emerging technologies can play an important role in preventing cyberviolence against women and girls and educating people;

I. whereas the EU is facing an unparalleled shortage of women in Science, Technology, Engineering and Mathematics (STEM) careers and education, given that women account for 52 % of the European population, yet only for one in three of STEM graduates;

J. whereas despite the positive trend in the involvement and interest of women in STEM education, the percentages remain insufficient, especially considering the importance of STEM-related careers in an increasingly digitalised world;

1. Considers that AI has great potential to promote gender equality provided that already existing conscious and unconscious bias are eliminated; stresses the need for further regulatory efforts to ensure that AI respects the principles and values of gender equality and non-discrimination as enshrined in Article 21 of the Charter of Fundamental Rights; stresses, further, the importance of accountability, of a differentiated and transparent risk-based approach, and of continuous monitoring of existing and new algorithms and of their results;

2. Stresses the need for media organisations to be informed about the main parameters of algorithm-based AI systems that determine ranking and search results on third-party platforms, and for users to be informed about the use of AI in decision-making services and empowered to set their privacy parameters via transparent and understandable measures;

3. Recalls that algorithms and AI should be ‘ethical by design’, with no built-in bias, in a way that guarantees maximum protection of fundamental rights;

4. Calls for policies targeted at increasing the participation of women in the fields related to STEM, AI and the research and innovation sector, and for the adoption of a multi-level approach to address the gender gap at all levels of education, with particular emphasis on primary education, as well as employment in the digital sector, highlighting the importance of upskilling and reskilling;

5. Recognises that gender stereotyping, cultural discouragement and the lack of awareness and promotion of female role models hinders and negatively affects girls’ and women’s opportunities in ICT, STEM and AI and leads to discrimination and fewer opportunities for women in the labour market; stresses the importance of increasing the number of women in these sectors, which will contribute to women’s participation and economic empowerment, as well as to reducing the risks associated with the creation of so-called ‘biased algorithms’;

6. Encourages the Commission and the Member States to purchase educational, cultural and audiovisual services from providers that apply gender balance in their workplace, promote public procurement policies and guidelines that stimulate companies to hire more women for STEM jobs, and facilitate the distribution of funds to companies in the educational, cultural and audiovisual sectors that take account of gender balance criteria;

7. Emphasises the cross-sectoral nature of gender-based discrimination rooted in conscious or unconscious gender bias and manifested in the education sector, the portrayal of women in the media and advertising on-screen and off-screen, and the responsibility of both public and private sectors in terms of proactively recruiting, developing and retaining female talent and instilling an inclusive business culture;

8. Calls on the Commission and the Member States to take into account ethical aspects, including from a gender perspective, when developing AI policy and legislation, and, if necessary, to adapt the current legislation, also including EU programmes and ethical guidelines for the use of AI;

9. Encourages the Member States to enact a strategy to promote women’s participation in STEM, ICT and AI-related studies and careers in relevant existing national strategies to achieve gender equality, defining a target for the participation of women researchers in STEM and AI projects; urges the Commission to address the gender gap in STEM, ICT and AI-related careers and education, and to set this as a priority of the Digital Skills Package in order to promote the presence of women at all levels of education, as well as in the upskilling and reskilling of the labour force;

10. Recognises that producers of AI solutions must make a greater effort to test products thoroughly in order to anticipate potential errors impacting vulnerable groups; calls for work to be stepped up on a tool to teach algorithms to recognise disturbing human behaviour, which would identify those elements that most frequently contribute to discriminatory mechanisms in the automated decision-making processes of algorithms;

11. Underlines the importance of ensuring that the interests of women experiencing multiple forms of discrimination and who belong to marginalised and vulnerable groups are adequately taken into account and represented in any future regulatory framework; notes with concern that marginalised groups risk suffering new technological, economic and social divides with the development of AI;

12. Calls for specific measures and legislation to combat cyberviolence; stresses that the Commission and the Member States should provide appropriate funding for the development of AI solutions that prevent and fight cyberviolence and online sexual harassment and exploitation directed against women and girls and help educate young people; calls for the development and implementation of effective measures tackling old and new forms of online harassment for victims in the workplace;

13. Notes that for the purpose of analysing the impacts of algorithmic systems on citizens, access to data should be extended to appropriate parties, notably independent researchers, media and civil society organisations, while fully respecting Union data protection and privacy law; points out that users must always be informed when an algorithm has been used to make a decision concerning them, particularly where the decision relates to access to benefits or to a product;

14. Calls on the Commission and the Member States to devise measures that fully incorporate the gender dimension, such as awareness-raising campaigns, training and curricula, which should provide information to citizens on how algorithms operate and their impact on their daily lives; further calls on them to nurture gender-equal mindsets and working conditions that lead to the development of more inclusive technology products and work environments; urges the Commission and the Member States to ensure the inclusion of digital skills and AI training in school curricula and to make them accessible to all, as a way to close the digital gender divide; 

15. Stresses the need for training for workers and educators dealing with AI to promote the ability to identify and correct gender-discriminatory practices in the workplace and in education, and for workers developing AI systems and applications to identify and remedy gender-based discrimination in the AI systems and applications they develop; calls for the establishment of clear responsibilities in companies and educational institutions to ensure that there is no gender-based discrimination in the workplace or educational context; highlights that genderless images of AI and robots should be used for educational and cultural purposes, unless gender is a key factor for some reason;

16. Highlights the importance of the development and deployment of AI applications in the educational, cultural and audiovisual sectors in collecting gender-disaggregated and other equality data, and of applying modern machine learning de-biasing techniques, if needed, to correct gender stereotypes and gender biases which may have negative impacts; 

17. Urges the Commission and the Member States to collect gender-disaggregated data in order to feed datasets in a way that promotes equality; also calls on them to measure the impact of the public policies put in place to incorporate the gender dimension by analysing the data collected; stresses the importance of using complete, reliable, timely, unbiased, non-discriminatory and gender-sensitive data in the development of AI;

18. Calls on the Commission to include education in the regulatory framework for high-risk AI applications, given the importance of ensuring that education continues to contribute to the public good, as well as the high sensitivity of data on pupils, students and other learners; emphasises that in the education sector, this deployment should involve educators, learners and the wider society and should take into account the needs of all and the expected benefits in order to ensure that AI is used purposefully and ethically;

19. Calls on the Commission to encourage the use of EU programmes such as Horizon Europe, Digital Europe and Erasmus+ to promote multidisciplinary research, pilot projects, experiments and the development of tools including training, for the identification of gender biases in AI, as well as awareness-raising campaigns for the general public;

20. Stresses the need to create diverse teams of developers and engineers to work alongside the main actors in the educational, cultural and audiovisual sectors in order to prevent gender or social bias being inadvertently included in AI algorithms, systems and applications; stresses the need to consider the variety of different theories through which AI has been developed to date and could be further advanced in the future;

21. Points out that the fact of taking due care to eliminate bias and discrimination against particular groups, including gender stereotypes, should not halt technological progress.

INFORMATION ON ADOPTION IN COMMITTEE ASKED FOR OPINION

FINAL VOTE BY ROLL CALL IN COMMITTEE ASKED FOR OPINION

Key to symbols:

+ : in favour

- : against

0 : abstention

INFORMATION ON ADOPTION IN COMMITTEE RESPONSIBLE

FINAL VOTE BY ROLL CALL IN COMMITTEE RESPONSIBLE

Key to symbols:

+ : in favour

- : against

0 : abstention