CSA Research

12:31:15 So hi everyone. I’m the senior analyst at CSA research who worked on the perception survey for the Safe AI task force.

12:31:26 The interpreting industry is facing a tremendous amount of angst over improvements in artificial intelligence. Does it mean the end of the profession?

12:31:36 Will meaningful language access and meaningful communication access be maintained when using automated solutions, how can you ensure that AI is used safely to prevent negative outcomes?

12:31:50 There are a lot of questions to address. So, we conducted an in-depth research report and we recently published 350 pages packed with lots of information.

12:32:02 And this session I’m going to give you a summary of core data points that you will see analyzed in greater detail in the report.

12:32:11 But let’s start by discussing the context for this research. It was commissioned by the Safe AI task force.

12:32:19 It’s a diverse group of industry stakeholders who got together to establish disseminate and promote guidelines for responsible AI adoption in the field of interpreting.

12:32:30 And they requested that CSA research conducts for them a large-scale perception study of end users.

12:32:39 Here think doctors, patients, etc. And then of requesters those are the buyers of language services and then the providers of interpreting services and technology so that’s of course the language service providers, the interpreters, and the AI tech vendors.

12:32:54 The goal of the study was to capture current perceptions about both spoken and signed AI for interpreting with a focus on the US market.

12:33:06 And for those of you who don’t know us, CSA research is an independent market research company.

12:33:11 We’ve been around for 20 years. And we focus exclusively on the language services industry. Whether the supply side or the demand side.

12:33:20 And the scope of the research that the task force requested was massive. So, we are grateful to the sponsors that the task force recruited to help fund this research.

12:33:31 Their names are on screen here and this research would not have been possible if not for these companies backing the project.

12:33:39 And I also want to acknowledge our sign language interpretation right now with Master Word who is donating the sign language interpretation for the session today.

12:33:51 The report will synthesize the results of 2 surveys. One was with requesters and providers of language services.

12:34:00 So here again, think procurement teams, schedulers, interpreters, etc. And then the other was with the end users.

12:34:05 Again, things like doctor patient and all those permutations. That end user survey was available in 10 languages.

12:34:13 And a great promotion effort from the task force led to over 2,500 responses from 82 different countries.

12:34:21 In the United States, we were able to collect responses from 48 different states. So, it had a broad reach.

12:34:28 Altogether, we collected 118 data points which we correlated against each other and that allowed us to publish 9,400 values.

12:34:38 And we also offered ample opportunities for respondents to provide free text answers and we collected an amazing 3,400 comments and that told over 62,000 words.

12:34:51 Some people honestly wrote entire essays. So, we’ve read and analyzed every single one of them and we sprinkled anonymized approach throughout the report.

12:35:03 And I briefly want to touch on the profile respondents because it did affect the results. Two-thirds of respondents were either spoken or signed language interpreters.

12:35:13 We had targeted 11 different groups. So that means that the average is influenced greatly by interpreter responses.

12:35:20 And that’s why each graphic in the report will indicate responses by group so you can contrast responses from their different perspectives.

12:35:29 We also had 69% of respondents that had some sort of a connection to the healthcare field.

12:35:36 And this is important because it’s a sector where mistakes can have life and death consequences. So, the survey collected more negative reactions to AI than if more respondents had come from other domains that are less high risk.

12:35:51 And then only 11% of respondents across our sample had either moderate or extensive experience. With automated spoken or same language solutions.

12:36:03 And that honestly was a little bit of a problem because perceptions ended up being based on opinions more than on facts.

12:36:09 So on each graphic we distinguish responses from those who had experience and those that have either a little or not.

12:36:17 And I also want to mention that 79% of respondents came from the United States. That was the primary recruitment target for the task force.

12:36:28 On graphics, you will see respondents in the United States versus outside US. Outside the US tended to be a bit more open to AI.

12:36:37 But that could have had to do with the fact that the survey might have reached predominantly people who were already interested in the topic…..

12:36:44 …in those other geographies.

12:36:47 Okay, so let’s get started with context data. This is from end users. Meaning either the service recipients or frontline professionals who participate in sessions where the interpreting occurs.

12:37:00 We ask them to what degree they trust in interpreting from 5 different sources. And what you see here, the results for those who say they either, that they fully trust the source.

12:37:11 So. No surprise, face to face interpreters come in the lead with 78% of end users who fully trust them.

12:37:20 Once the interpreter is remote, that confidence level drops to just 56%. Bilingual friends and family members who are not professionals did not rank very high here with only 21% of end users who fully trust them.

12:37:35 The last 2 options refer to the automated interpreting options. So, when an organization provides interpreting, think for example about a conference where the organizer gives you a link to access the automated interpreting.

12:37:51 In such cases respondents trust the output a little bit more than when using apps that they found online, often those free apps that you can just download by yourself.

12:38:03 But either way, it’s not very much trust in either of these 2 cases.

12:38:09 Let’s now explore a series of 5 AI powered services that’s really to the topic automated captioning.

12:38:17 Automated transcription, automated subtitling, automated spoken language interpreting, and automated sign language interpreting.

12:38:26 And we asked survey takers about their level of experience with it and if they had used it, what they thought about the quality.

12:38:34 Note that the end users of interpreting services did not see captioning, transcription, and subtitling questions because we were trying to simplify the survey for them.

12:38:47 So let’s start with captions. That’s when the text on screen appears in the same language as the one spoken in the session over a quarter of respondents had either used an automated system to produce captions or tested either moderately or extensively.

12:39:03 That’s actually a pretty good number. And over half of the M found results either good or excellent.

12:39:12 And this was the highest-level performance across the 5 services that we tracked.

12:39:17 The next service is related to captioning. But instead of the text appearing on a screen, the AI produces a document that may include elements such as time codes and who is speaking.

12:39:29 A little fewer people had experience with automated transcription. But overall, over half again found results either good or excellence.

12:39:38 What such high numbers indicate is the potential for technology to be useful to assist in some situations. Moving on to subtitling, the third best performing AI service.

12:39:51 That’s when captions on screen in a different language than the language spoken in the session. 19% of respondents had some experience with automation and 40% of them found some value in the results.

12:40:05 Numbers are a little bit less exciting than the captions because you compound errors at each step you add in the process.

12:40:12 So to simplify the concept in captioning, you essentially just use voice recognition. I’m simplifying here.

12:40:20 But for subtitling, you apply machine translation to the captions. So, you have doubled the chance of errors in the process.

12:40:29 Now, once you apply a synthetic voice to the subtitles, it’s called automated interpreting, machine interpreting, automated speech translation, the terminology out there is a necessarily very formalized what we predominantly use in the report is automated in interpreting or AI.

12:40:49 And that is the focus of the report. Unfortunately, only 11% of respondents had first-hand experience with such technology and just a little over a third of respondents found results either good or excellent.

12:41:05 One of the challenges of automated interpreting is that very often people prefer to read subtitles than to listen to a robotic sounding voice that lacks a little bit in intonation.

12:41:19 And finally, the last AI driven service we asked about was automated signed language, which refers to text to sign technology using avatars.

12:41:27 Assigned to text technology using complex sign decoding systems. And the reality is that the technology is much less developed than spoken language interpreting.

12:41:39 So barely 1% of respondents had even experienced any of it. And for those who did, quality perceptionss fell below the 30% ….

12:41:46 … mark, meaning that the use cases are going to be very limited. Until the technology improves.

12:41:53 Now I also want to show you some data on whether respondents thought that automated interpreting could reach the same level of accuracy as qualified human interpreters.

12:42:04 And for simple conversations 9%. Believe that AI already reaches human parity. And 25% believe it will happen soon.

12:42:14 Numbers naturally drop for complex conversations with a single percentage point who believe human parity already exists and 8% who think it will happen soon.

12:42:25 The flip side of course is that a large portion of respondents think it will take a really long time to get there.

12:42:32 It may never happen at all. And that’s why it gets very interesting to see the response difference between those who already have some experience and those who don’t.

12:42:42 What’s in the report is that respondents without much or any AI experience tend to underestimate AIs capability.

12:42:53 So let me prove this to you. We assigned a score. To quality perceptions, those who say quality is excellent, they have a value of 2, good is one, unacceptable, sorry, then I have 4, that’s minus one, an acceptable minus 2.

12:43:09 And that’s what you see on the x, the y-axis here, and then on the x-axis you see the level of experience.

12:43:17 And for each of the 5 AI driven services that we tracked, the more experience people have, the more positively they think about AI capabilities.

12:43:28 The effect on the results is very real. So, I highly encourage providers to gain some experience with it, interpreters in particular had the least experience with AI.

12:43:41 You should really test some tools so that you know what it can or cannot do. Otherwise, you only say what it should or should not do.

12:43:50 That’s important too, but that is not all there is to this discussion.

12:43:57 Okay, moving on I want to know show you a series of 10 statements that we made where we asked survey respondents to tell us to what degree they agreed with them.

12:44:11 And I’m going to show you responses for those who agree. The first statement was, it’s okay to use machines for routine and repetitive conversations that they can handle.

12:44:21 Nearly half of respondents either agree or strongly agree. And this confirms what we saw earlier about simple conversations.

12:44:30 AI can be helpful for some easier interactions or one-way communications. The next statement inquired whether it’s better to have a machine interpret rather than have no interpretation at all.

12:44:43 And this is the whole premise behind the argument that AI increases language and communication access. 44% of respondents either agreed or strongly agreed.

12:44:53 That imperfect, results are better than no language support. Now that leaves a lot of people who disagree.

12:45:01 Not also that respondents commented on the fact that AI could actually reduce meaningful language and communication access. Because organizations would push AI options in unsuitable scenarios.

12:45:15 If language and communication access decreases, or reduces in quality, then discrimination of physical harm or many other consequences can occur.

12:45:27 And obviously that is not something that anybody wants to see happen.

12:45:32 The next statement epitomizes the contradiction of the results throughout the reports. Many respondents claim that machines cannot deliver when they haven’t retested the systems.

12:45:45 Their answers are often more about saying what machines should be tasked with handling interpreting services.

12:45:54 And that is why nearly 3 in 4 respondents are whopping 74% either agree or strongly agree that it is not right to replace people with machines for interpreting.

12:46:06 Now remember we had a lot of responses from providers who are afraid to lose their job. So that affected the results to some degree.

12:46:15 And when you contract the responses of the previous ethics question to the potential financial motives of organizations to push for AI, you really get to the crux of the issue.

12:46:26 Over 2, of 3, of respondents either agree or strongly agree that requests is only want automation to reduce costs.

12:46:33 The ones who really disagreed with that statement were people in procurement role, which is normal because they see the whole picture with all the elements that go into such decisions.

12:46:45 Procurement teams try to fix problems for their companies and their primary motives are efficiency even before cost.

12:46:53 So, for example, a language professional may not see the cost of the administrative burden involved in scheduling interpreters.

12:47:01 Or what happens when there is a no-show from either the service recipient or the interpreter?

12:47:06 Does that mean the only answer to their problem is AI? Of course not, but you need to understand that buyers are frustrated with an inadequate scheduling system,   and the lack of system integration and interpreter demands even….

12:47:20 …can affect decisions regarding the role of human versus AI.

12:47:26 Continuing the analysis and cost, we asked end users whether who pays for the interpreting effects, their thoughts on AI usage.

12:47:34 39% either agree or strongly agree that they might choose a machine if they have to pay for the interpreter themselves.

12:47:41 Note, of course, that the balance would still prefer a human interpreter. And this point is tied to the proliferation of pocket translators.

12:47:53 These apps you can find online for generally tourists and that affects the price point that technology vendors can charge.

12:47:59. And then this in turn drives the AI developers to create products primarily for enterprises and government institutions because developing quality applications is very costly.

12:48:14 And they need a return on their investment. And this also explains why some of the better AI products are not accessible to direct consumers.

12:48:25 Next statements, was one where we asked the opposite of the previous one. We’re contrasting payment by someone else versus by the end users themselves.

12:48:36 And if someone else pays the bill fully, 76% of end users will favor a human interpreter.

12:48:43 This shift in pay is important to remember in the context of respondents who feel that organizations push AI as a cost saving measure.

12:48:51 The cost factor matters for requesters and users alike. After all, this is interpreting. Nobody wants to pay for it.

12:48:59 Even though it is very necessary for all the people who are involved in communication.

12:49:06 Now switching gales a little bit the next statement was on whether there are situations where end users would prefer an automated interpreting solution over a human.

12:49:16 AI admittedly comes with its fair share of concerns over data privacy. However, being able to hold a conversation away from the ears and eyes of a fellow human might have some advantages too, especially in small communities where you personally know the interpreter.

12:49:34 Using a machine can be more comfortable when dealing with matters of a highly personal nature, anything that is sensitive or very private because despite the humanity that an interpreter brings to the table, sometimes that human element can be a little bit too much because here you can see that 31% agreed that sometimes AI could be better.

12:49:56 For them. And finally, the stress factor of using an automated solution also enters into the discussion. About half of respondents expect that using a machine interpreter would cause them no stress.

12:50:13 So why is that? AI driven processes add a burden to participants to figure out if a mistake was made and when it was made to Speak up if he did.

12:50:24 And they will likely fear loss of vital information that could affect a ruling, a diagnosis, or whatever the next step action is going to be.

12:50:33 Each of which can have a huge financial and human cost. And ultimately, distress can contribute to the feeling of not being heard.

12:50:45 And I also want to add another element is that the robotic nature of some of the AI tools output may be distracting and it might make it more difficult for the listeners to pay attention to the content and the next steps.

12:51:02 And we close a series of statements with a question for end users of whether they want to know if a person or a machine is doing the interpreting.

12:51:10 And the answer here was an overwhelming yes for 89% of respondents. Advances in advances in voice flow.

12:51:20 Can make the distinction between a human and a machine difficult when the end user doesn’t have a visual reference.

12:51:28 So you can use the analogy of phone systems which we all have struggled with it can become frustrating sometimes not to know if you’re talking to a human in our machine and the same would happen if you deal with a life translator versus a bot and not knowing which one you’re dealing with.

12:51:48 So now let’s tackle the pros and cons of automated solutions. I’ll present only the data portion of what’s in the reports note that there are dozens of pages in the report with an analysis of free text answers that went with it.

12:52:02 Respondents had a lot to say, especially when it comes to the drawbacks of potential A.

12:52:09 I use. But here I’ll strictly share the data portion. So, respondents could select multiple options from a list of common benefits.

12:52:17 Requesters and providers of interpreting services, so more options than end-user did.

12:52:23 You can see that with the little stars here in the legend. So, what does this graph tell us?

12:52:30 Well, the most frequent advantage was around the clock availability. 66% selected that advantage but end users and requesters were showing even greater numbers.

12:52:43 That’s when the interpreting solution is available, 24 7 365.

12:52:49 It’s technology that’s easy to implement. Now in second position, there was a tie between the no need to schedule.

12:52:56 Interpreter and the low cost element where 58% of respondents selected those responses. But what I also want you to pay attention to is the indirect dig against the interpreters.

12:53:10 Beyond the ability to skip the scheduling, some elements that respond positively about AI underscore issues even if they’re not significant.

12:53:18 But the issues that affect end users and requesters perception of human performance and therefore influence their reaction to AI.

12:53:28 So the numbers you see on screen at the average for all responses group. But let me show you examples from requests only meaning the buyers and decision makers.

12:53:38 38% like that AI doesn’t arrive late at appointments. 31%. Think that they could achieve higher field rates on job assignments.

12:53:46 And 16% think that they deal with fewer professionalism issues. So averages don’t tell the whole story, so pay attention when you read the report as to the difference between the different audiences, it’s on every graphic.

12:54:01 Okay, so now let’s examine responses for drawbacks. Big mistakes is where respondents worried  the most.

12:54:10 We defined these big mistakes as situations when the main idea might be wrong or the mistake could cause harm or some sort of a legal problem.

12:54:20 And that’s where it’s 81% of respondents. When AI makes mistakes, the errors can be more severe than humans would make.

12:54:28 But humans make mistakes too, especially when they deliver services in real time. But in contrast though they can rebound a little better.

12:54:41 Now I also want you to look if you jump ahead a little bit here at the small mistakes. Those are situations where the main idea is clear, or the details are wrong.

12:54:55 That still worries 48%, but that number jumps to 57% for the end users.

12:55:01 So really compare the results by audience once you see the report because they tell different stories. Okay, so let’s go back to the beginning of the graphic here, the second bar.

12:55:11 The need for special tech worries another 67% of all respondents. The implementation of telephone and video.

12:55:22 Remote interpreting, launch the introduction of these dual handset phones and, you know, those physical carts that transport video devices.

12:55:30 And if you take the example of a medical setting, someone has to find a cart and maybe they didn’t order enough for their departments or maybe the last person who use it didn’t store it where it should have been and somebody has to hunt.

12:55:43 For it. So, if automated solutions use the same hardware, it will not really make the situation any easier for frontline professionals.

12:55:54 Okay, the third bar here respondents also fear something going wrong. 59% think there are more chances that something will go wrong with automated solutions.

12:56:03 And that always means more work for whoever must find a solution to troubleshoot and fix the issue. And then 56% fear that users will not accept machines.

12:56:15 Overcoming the stigma would require training and experience for all parties involved. But then on top of it for some end users, machines will never be the right solution.

12:56:26 Think for example about a patient who has multiple disabilities. There is a really high chance AI will not be able to assist them.

12:56:36 Okay, so most of the drawbacks are tied to the limitations of the AI and getting into the complex conversations where human interpreter is by far the superior option.

12:56:48 So that is why we ask survey takers if they thought that an automated interpreting system would be more useful if they could quickly get a person to help when there is a communication problem.

12:57:01 So. The logic behind the question is that the app would include some sort of a button where you can request to talk to a human interpreter.

12:57:09 Either by telephone, video, or even in person. You do that if either party in the conversation feels there’s a communication challenge.

12:57:19 It’s really similar to the press 0 to talk to a customer service representative in a phone-based system.

12:57:26 And some technology providers already built such an escalation mechanism into their platform. Here we found that more than one half of respondents think the ability to escalate to a human being would increase the usefulness of automated interpreting either a lot or level.

12:57:44 This ties a bit more to the symbiotic relationship. Between AI and interpreters.

12:57:51 So I’m not covering this here today, but in the report, we cover AI driven computer aided interpreting.

12:58:01 Which is when the AI helps with you know named entity recognition numbers etc. memory recall terms a lot of different aspects to empower the interpreter to do a better job.

12:58:17 I already mentioned that respondents cited a lot of concerns about AI performance and a lot of it relates to what machines cannot capture yet.

12:58:27 I’ll present a high-level categorization of what we explore more in depth in the report.

12:58:34 Respondents were truly prolific in their free text comments about the importance of humans in dealing with subtleties and ambiguities.

12:58:43 This graphic depicts a high-level categorization of their arguments from requiring the ability to deal with a context for language, culture, tone, emotions, and interaction background.

12:58:55 AI cannot capture visual clues, cultural inferences. It doesn’t understand the mood of the participants and it doesn’t really understand regionalisms.

12:59:05 So in short, the respondents fear that because AI cannot see nuance or read between the lines, service recipients have too much to lose when the machine doesn’t understand this context and culture.

12:59:18 But similarly, the language that people speak is not textbook perfect. Neither are the situations in which the interpreting occurs.

12:59:27 Respondents here were terrific too at providing examples of challenging situations where AI would struggle interpreting for someone who uses programmer, has a speech impediment or is in an active psychotic episode — that requires some serious skills.

12:59:43 So the graphic on screen shows some of the many such imperfect scenarios that can lead to a communication breakdown or technology failure.

12:59:53 The reason that humans remain the better choice is that they provide language accommodations for unusual or unexpected situations by adapting what and how they interpret their knowledge because they use the knowledge and their common sense.

13:00:09 And this takes us to an important concept that underpins the research. That of the role of the interpreter.

13:00:16 And I had a bit of fun with AI myself and I asked chat GPT to create a graphic depicting the traditional view of interpreting the one that revolves around the concept of linguistic conduit.

13:00:29 Meaning interpreters act as neutral parties who convey messages from one language to another without adding, omitting, or altering the content.

13:00:39 Ultimately, that’s what automated interpreting strives to accomplish. The thing though is that the interpreter’s role has evolved over time to be more than translators of spoken or signed words.

13:00:53 And in this active model, the interpreter plays a more involved role in the communication process. They may clarify meanings, ensure cultural appropriateness, and sometimes even mediate the conversation.

13:01:06 So in this model, the interpreters take responsibility to ensure successful communication. They work to verify the messages and pursue them correctly by both parties, potentially adapting language, tone, cultural references to fit the context.

13:01:22 So this approach recognizes the interpreter as an active participant in the communication process, acknowledging that their presence and their decisions can influence the outcome of the interaction.

13:01:35 AI is nowhere close to capable of reaching this level of proficiency. So, who will lose if AI pushes organizations to go back to a conduit model?

13:01:47 Of course, the end users. It may be less of an issue when interpreting for a business meeting or other, you know, trade show or conference but it is certainly in context such as healthcare, legal and social services.

13:02:03 And that then takes me to giving you a snapshot of data on AI suitability by use case scenario.

13:02:10 They are close to 6,000 data points and that alone in the report. So today I’m just giving you the most interesting ones.

13:02:18 Let’s examine end users’ responses first. The data you see here summarizes the percentage of service recipients and frontline professionals who find AI either mostly or totally suitable for a conversation type.

13:02:34 On the left you have low risk non-technical non urgent conversations that came up as the highest cause, while their logical counterparts on the right, high risk, technical and urgent, were seen as much less ready for deployments.

13:02:50 The trick of course is defining the characteristics of these conversation types as conversations rarely stick to a single type all the way through.

13:03:00 And if we now look at responses from requesters and providers, so no longer end users. Here you can see the top 5 use cases out of 58 that we provided across 11 areas.

13:03:14 Nearly 2 thirds of respondents of respondents think it’s mostly a totally suitable to use automated interpreting for notifications or announcements.

13:03:25 For example, when you report an outage, an absence, a cancellation, and I’ll skip ahead here and have you look at number 4 which has a similar situation but in an education context.

13:03:37 And then look at the type for number 5. That was an emergency service scenario. Where 37% believe that AI is okay to notify of a weather emergency.

13:03:50 But let’s go back to number 2 here, scheduling sessions to select, notify, or remind of either the time of the meeting where it will take place and what you need to prepare for it.

13:04:01 We found 56, sorry, 57%. Who found that AI was suitable for that.

13:04:09 And then in number 3, that came from our series on client service scenarios. That’s the ability to use AI.

13:04:16 I to check on the balance of an account. That’s something that is often frequently already done. Through AI phone systems as it is.

13:04:27 And then in number 5, the logistics call to explain how a session our conversation will take place, reaches 37% approval.

13:04:35 It’s a little similar to what was in number 2 here. Now these are very strong use cases.

13:04:41 Other sectors did not fare as well. But responses do vary significantly by use case, not just by domain.

13:04:50 So for example, if you take healthcare. The delivery of bad news to a patient is clearly unsuitable.

13:04:58 You all know this, and the data proved it. But if you’re doing patient registration, there was some potential there.

13:05:05 However, if you’re doing patient register, say with a patient that has cognitive differences, that would still be better with a human interpreter.

13:05:15 So there are a lot of elements that go into the decision. So that’s why the next part of the analysis identifies criteria to use when determining whether to use automated interpreting.

13:05:26 We provided respondents elements where they rated the criteria as to whether it should be a major criterion minor or not a criterion at all.

13:05:38 And I’m showing you here just the percentage for those who believe it’s a major factor. We told respondents to answer as if the decision was up to them.

13:05:48 The level of accuracy required from the interpreting leads the list, no surprise, followed by the risk of possible harm, no surprise there either.

13:05:58 And then comes the complexity of the language, not all language are rendered equally well through AI. But even after that you can see that the other elements, we suggest are all quite important.

13:06:11 And what you need to note is that one element alone is not enough to determine whether AI can or cannot do the job.

13:06:19 It’s always a multi-factor decision that is not just done at the organizational level, but for each single interaction.

13:06:28 So an organization might decide to procure an automated interpreting system. That doesn’t mean they should use it in every interaction.

13:06:37 Even if it’s a basic one. Okay, so I will wrap up this presentation of data highlights with responses from procurement teams about their plans for the next year for the 5 services that we had discussed upfront.

13:06:51 Note that the number of respondents here is very small. Not just 40 people who followed that survey path.

13:06:59 And the bulk of answers show no plans to use automated captioning, automated transcription, automated interpreting.

13:07:08 Subtitling or certain language, none of those. But let’s explore those who already went ahead with it or plan to do so.

13:07:15 So captioning and transcription are in the lead in terms of existing implementation. 18% of respondents have automated solutions.

13:07:25 In place for captioning. And 13% for transcription. Does that are 2 monolingual services that we’re tracked in the survey?

13:07:33 And only 5% which given the small sample and you know; of people this represents just 2 respondents.

13:07:43 Only 5% have solutions in place for interpreting and subtitling. None of them report any AI for sign language right now.

13:07:52 Now, there is little change planned on the radar. Those who were ready to implement automated captioning and sub-tiling already did so.

13:08:00 And implementation plans are negligible for other services. This thing is a bit different. All but sign languages will see some testing.

13:08:10 Truth be told, studies like the present one likely prompted some procurement teams to think it’s time to see what the fuss is all about.

13:08:19 And then what’s also interesting here in this data set is that more procurement teams plan to reduce their use of automated subtitling, than those who plan to continue using it.

13:08:31 And that is probably because the output is not up to expectations. That is problematic for the automated interpreting tech vendors.

13:08:42 Because they simplified mostly just add voice synthesis to the subtitles. Now we also notice some reductions for use of AI in captioning and transcription.

13:08:55 Okay, so I’m not going to take the time today to get into all the recommendations, which will be listed in the report for each of the audiences.

13:09:04 But I do want to give a flyby overview of the ones we gave for the task force. What’s important for everyone to remember?

13:09:10 Is that we just only scratch the surface, and we did not get anywhere close to enough feedback from end users.

13:09:17 The findings in this report alone are not sufficient. To establish all-encompassing guidelines. If you rush to establish guidelines, you will likely end up restricting use in situations where AI could have been a valid option.

13:09:31 So I want to urge you to remember your mission, which is to find out what is right for end users, not necessary to defend interpreters.

13:09:40 Or technology vendors. That was not the mission of the task force. So, what should you do? Now comes the hard work with more in-depth research to establish safe boundaries of when you can or cannot use AI.

13:09:56 So first you need studies with end users think focus groups and testing labs where end users can compare the different outputs.

13:10:05 Rate them, categorizing which scenarios they trust the system to do an adequate job. It should be for a variety of scenarios, not just the high-risk ones because realistically those are not ready for prime time.

13:10:18 The survey was also a little bit too theoretical for many end users who don’t even know what AI is or colony.

13:10:25 Imagine what automated outputs might sound like. Sweet, really needs to be hands-on. And you should compare different tools because the accuracy varies from one system to another.

13:10:37 It’s not equal out there. You should also conduct an in-depth analysis of the nature of conversation.

13:10:45 So we talked about simple versus complex conversations. And for each use case scenario you’re considering AI for, you need to really understand the percentage of conversations that are truly simple.

13:10:58 And when it turns complex. What are the trigger words that indicate the conversation tipped from simple to complex?

13:11:07 Because without that, we fall to any conversation having the chance to turn complex and then you like the data to put proper escalation systems when using AI.

13:11:19 And next our research barely touched on captioning and sub-titling. Automation use cases are much greater for those technologies and such research should identify the tipping point of when and users are satisfied with subtitling versus one date one voice synthesis instead.

13:11:36 And the same logic applied to captioning versus signed language by an avatar. Again, results will vary based on the scenario and the audience.

13:11:47 Finally, the presence perception study was conducted nearly too early to collect accurate perceptions in 6 months, a year or even 2 years, many more people will have been exposed to AI solutions and will have perceptions more in line with the capabilities.

13:12:05 Also, don’t forget technology is also constantly improving. So, what is not suitable now, would not be suitable in a matter of months.

13:12:15 So this is it for the presentation today. You can read the report online. On our platform or download a PDF, plan some time to read it.

13:12:25 It probably takes about 8 h to read it straight through. And that’s even at a pretty good pace.

13:12:29 It is packed with information, and I recommend you don’t skip the second chapter that tells you how to read the graphic because a graphic like the one you see on screen has about a hundred data points and you really need to understand.

13:12:42 What each of them represents to get the most out of the report. And I also want to mention that the 350 pages of the report are a bit daunting for some people, so there will be a summary version available pretty soon.

13:12:57 This concludes the presentation of the findings. Thank you everyone for watching. Thank you.