In this episode, Colm Gannon shares his extensive experience at the intersection of technology, online safety, and child protection. We explore the impact of AI, digital trust, age assurance, and the human element in safeguarding children online. If you want to be sure you are up to date with the latest in DFIR, don’t miss an episode!
ICMEC Australia - https://icmec.org.au
Safer AI for Children Coalition - https://icmec.org.au/safer-ai
UK Online Safety Act - https://www.gov.uk/government/collections/online-safety-bill
Grok AI - https://grok.ai
Rigor AI Age Assessment Tool - https://rigor.ai
Guest links
LinkedIn - https://linkedin.com/in/colm-gannon
Adam Firman (00:04.462)
Welcome to episode 27 of Forensic Fix, a podcast brought to you from MSAB, where we talk with experts in digital forensics, investigations, online safety, and the technologies that are shaping modern society. I'm your host, Adam Thurman, and today I'm very pleased to welcome Carl Gannon to the show. Carl is currently the CEO of ICMC Australia, an organization that is focused on protecting children from online exploitation, abuse, and emerging digital harms.
His work sits at the intersection of technology, public policy, trust, online safety and child protection. Across his impressive career, Conway's worked closely with governments, industry leaders, technology providers and frontline organizations to help shape a safer digital environment for children and vulnerable people. More recently, his work has explored the growing influence of artificial intelligence, digital trust, age assurance and a wider
society responsibilities that come with this rapid technological change. What stands out in Calm's work is a strong focus on the human impact of technology. Not just what technology can do, but what it should do and how the choices we make around technology directly influence safety, trust and accountability. Now Calm, I've given our listeners a very brief overview of your background and the work that you're doing at ICMC Australia. For those
listeners who may not be familiar with your journey. Could you tell us a bit more about your career path and what led you into the space of online safety, child protection and technology policies? And welcome to the show. Yeah, Adam, thanks very much. It's a really nice introduction. I was kind of looking over my shoulder to see who you're introducing there.
Yeah, my career started off in Ireland, I was with On Guard, Ashir Kaurna, the Irish National Police worked in counter-terrorism and organised crime and got a very delighted and golden shoulder tap by New Zealand government and asked me to move to New Zealand and work with their digital safety directorate with Department of Internal Affairs. And that was dealing with the distribution and production of online child sexual exploitation.
Adam Firman (02:24.728)
abuse material and worked there for about eight and half years. some of the, I was involved in some of the biggest operations in the world, especially around the dark web. And the, I know it's probably hard for people to realize, but even things like online auctioning of children was actually happening then and it still takes place every day.
did some work there, worked with the UN Convention for Rights of Children as a subject matter expert for New Zealand government. And then I went into private industry and started working in AI and machine learning. And now I have the wonderful job of working with the International Centre for Missing and Exploited Children Australia or ICMIC Australia, as we call it, where we work to create a world where technology cannot be used to harm children. That's amazing. And was there a particular moment in your career where it became clear to you
just how connected technology and child protection has become? I think one of the biggest things, my turning point was actually when I was dealing with terrorist organizations. I know this sounds really weird, but we found that terrorist organizations were actually using tradecraft that was being developed, was being used and being harnessed by online child sexual offending communities. So when I was in Ireland and I was looking at this type of activity, I was kind of going, these guys have actually learned from
online child sex offenders. And the reality was the online child sex offenders were at the pinnacle of how they use technology. They were at the forefront. They were the frontiers, early adopters, early abusers of this technology because they had this type of ambition to be always connected, always connected to online, always getting access. And that's when I looked and saying, like I worked in the drugs industry, the drugs environment and investigations. worked in counterterrorism.
And for people that are not in the police, but in the police world, these are seen as the pinnacle of where you want to be is to be working in a drug squad or a counter-terrorism unit. And the impact is knock, knock over the search warrants, do the rest and everything else. As a police officer, started to get tired of that. And I was kind of going, this is actually not having a real world impact on society. And then you're looking at the abuse of children and the victimization, sexual exploitation of children. You're kind of going, this
Adam Firman (04:48.963)
is where I can make a difference. This is where I can bring my learnings within my technology environment into a real world impact. And New Zealand came knocking on the door and they said, we would like you to lead something for us. And we would like to run international operations. And I was going, okay, yeah, I'll take that. And myself and my family, moved over to New Zealand. And that's when we started going really hard at looking at the intersection between technology and how to protect children online.
And you're right to sort of state the global aspect of this, because one of the cases that I worked on in Suffolk Police, I remember it was it was a referral from New Zealand Police. And we had an individual living within within our area who worked for a government organization and was abusing his children. And he worked in I.T. and was very, very technically savvy. And the referral came through from New Zealand and
He had made one mistake. He'd left metadata on a picture that gave us a location. And he had been doing it for years. Absolutely years. And he was in a position of trust as well. Yeah, it's crazy to see. We're going through a massive issue at the moment in Australia, particularly around the early childhood education centers. And we're looking at these people in the area of trust to have access to high technology.
So when we're looking at mobile phones, when we're looking at portable devices, we're looking at the abuses and also the exploitation that's taking place there. And we're trying to deal with that. We're trying to advise government on how to actually combat that. But they are just so technically adapted. When things change quickly, they will quickly change because they're motivated, like offenders, online offenders are motivated to get around the different barriers and different
restrictions that may take place. And I'm going to throw this one onto you, but just I hadn't pre warned you about this one, so I apologize. But it's very fitting. You're in Australia. How do you view the changes that the Australian government have made in relation to technology and social media for the age restrictions in place? But personally, in my opinion only, I agree with what they've done because it is addictive. Do you think that's going to make a
Adam Firman (07:16.948)
Big impact. Yeah, it's actually an interesting point because I've worked with the government in relation to social media, minimum age act and also the technology that was being used and developed the testing of the technology for age assurance. And what I'd say is that while the intentions are noble in relation to the age restrictions, I would say the rollout and the execution is very, very different. What we have is that we have young people under the age of 16 years of age.
that are now going to be accessing platforms, unmoderated access to those platforms. And while the technology companies will say they will do everything in their power to restrict their age, the difficulty is that these third party applications are not working and are not being accurate enough. I do like the UK model in relation to the online safety act where they're looking at device based age assurance.
I think that's a really important step to make the differences is that we will actually have age-based restrictions on the device. So even if people put VPNs up or do X, Y, Z, they're still not going to be able to circumvent those age protections in place. What I would say is that young people need a platform to express themselves. When we look at Gen Alpha and Gen Z, the digital environment is intertwined to how they communicate and how they operate their daily lives.
And what we need to do is create stronger safeguards, remove the co-mingling aspect within these social media platforms. But just don't say you can't have it because when they turn 16, we're going to throw them onto the social media road where they haven't developed any skills to actually mitigate that. And what'll happen is it'll probably cause more harm than it will actually cause good. It was always going to be a difficult decision to make and sort of draw that
line in the sand to bring in such a legislation. But we spoke briefly before we started recording about a previous guest, Jen Hoey. I remember Jen explaining to me that you wouldn't let a child walk through a very busy train station at midnight because of the dangers, yet we let them go online in a world that's even scarier. And that always hit home with me.
Adam Firman (09:33.437)
And I'm afraid myself and Jim would share the same opinion on this because we would say is that we need to have stronger moderation. Now with the social media minimum age act, we've actually allowed the online technology platforms to have that get out of jail pass where they don't have to have those serious restrictions put on under 16s and under 18s. Some are very good and some will actually have certain restrictions put on for under 18 profiles.
Well, I think what we need to do is we need to actually encourage our children to grow up with technology, to mature with technology and to actually learn what is good netiquacy. So when we're on the net, we want them to be mannerly. We want to be cordial. We want them to actually have a positive experience and we should be pushing that netiquacy to them, but allowing them to develop as they go through their different ages and putting those restrictions in place and those safeguards.
Yeah, so it almost becomes part of children's education and the curriculum. Correct, because the education is actually while we talk about the formal education in the classroom, we should also be talking about the informal education, about how they're interacting with people online. They can learn certain skill sets in the classroom, but then they have the real world practice. So they bring the theory from the classroom out into the world, but they're being brought out into the world in a protected environment. And we put the pressure on the technology service providers
to actually make sure those children are kept safe online with restricted ages age appropriate material being disseminated to them. Yeah and I hadn't even you spoke about the the UK sort of device age I hadn't obviously I knew it was incoming but I hadn't even realized that recently turning a PlayStation on had to have age verification
a recent iOS update for iPhone, for example, had to have age verification. And I take it that's because of that bill that's coming in. Yeah, I mean, the Online Safety Act and in fairness to Ofcom, they've actually taken a very pragmatic approach in relation to how the Online Safety Act is going to be implemented. And there has been a little bit of moral consciousness being around this, especially by some of the adult erotica industry where they remove service from the UK.
Adam Firman (11:50.495)
They've also done the same in Australia, they remove services from Australia because the fact is, is that they want to make sure that there's age-based restriction and also that VPNs cannot be used to circumvent access to their material. And I think that's the right approach because if you actually are going to take a moral standing and we're going to talk about how to protect children, let's actually look to protect children. Let's not tick boxes and actually go, we pass laws. There we go. We're sorted now.
That's not how the real world operates because young people, they're technologically adapt to their environments and they will look at ways to get around and look at ways to actually do X, Y, Z. Where if we're taking an approach of actually allowing the technology to grow with them, they will also become accustomed to that behavior and learn about how things are actually going to be introduced as they get older. again, another question I hadn't prepared you for, but
I repost, I reposted a post from me on LinkedIn this morning, which was about the, the impact that working in this industry has on people. And it's a call for research for people working in the space. Do you want to share a bit more detail about it? Cause it will still be relevant when this podcast comes out. Yeah. think one of the biggest things is that we have people that are actually looking at material that are moderating that are working in.
police working in not-for-profit sector, working in trust and safety teams across the world. And one of things that people say, oh, the images and the videos are actually very, very hard. But what they don't talk about is also the written material. And when we look at this project that I'm working on with my colleagues in the medical school of Berlin and also in Carleton University in Canada, is one of the difficulties is that people, when you read material,
is that you start to form images in your own mind based on your own lived experience or based on your own environment. And I've worked in this area as a victim ID and I've worked in it in relation to prosecutions on behalf of New Zealand government. And in time, it does take a toll on people's maledine and you have to be able to actually get that support. And unfortunately, there's a lot of countries and a lot of industries that do not support people who are involved in moderation and trust and safety teams.
Adam Firman (14:12.157)
And then they wonder why they burn out. You're kind of going, they're just, they've just burnt out because they cannot take the emotional trauma anymore because you have just totally and utterly left them to their own devices to actually do a job, but not look after their psychological wellbeing. Yeah. And it, really is. And I remember before, and I've spoken about this previously on the podcast, but I remember my old, Sergeant for my forensic lab before I took the job.
He warned me that we cannot sit here today and say that you're going to be OK seeing what you're going to see. It's one of the unfortunate things that you need to try and it will either be OK for you or it won't. But it did take us toll. Eventually, it did take us toll. And I think that's when I had children of my own that then I could relate and it yeah, it then really started to impact. You've seen a very, very dark side of humanity and.
I know some people don't want to realize this, but the worst I find offender type is a female child sexual abuse offender. That's when you know you've actually hit really the bad side of humanity. And it's not being gender discrimination or anything else that again, it's based on what we have as a conceived idea in relation to this modernity type opinion of women and how they're going to look after children and how they're going to care for them.
And then when you see the very hard realities of certain things, you just kind of go, yeah, that's tough. that's, you see, you are seeing the dark side of humanity and it does take certain type of people to actually engage with that material and engage with how to actually cope with it. Yeah. And it's, the overall mission that keeps you going. It's like what you said previously, when you switched into this space, it's the impact that you can make that keeps you going.
Yeah, you're vocation driven and you know what you're doing is actually going to have a positive impact on that voice, on that victim, on that survivor. And that's what you're looking to be. You're looking to be that microphone, that speaker. So when that victim is actually reaching out, where it's through the image, the video, the story, the metadata, you want to make sure that their voice has been heard so that their rights as a child to justice is going to be upheld. Yeah. And I picked up on something, a quote from you recently where you said,
Adam Firman (16:34.333)
Technology is not inherently safe or harmful. It's shaped by the choices we make. I found that statement really powerful. Can you explain what you mean by that in practice? I think for a long time we actually, I say this in the team that I work at, they rolled their eyes. said we have to stop beating the drum of fear when we talk about technology. But what we should be doing is we should be playing the concerto of creativity.
And basically when we have a concerto, we have a conductor, people are actually managing the music and making sure it actually comes together. And that's what we need to do is that we need to actually say technology is having an improved, we're in a technological evolution. We're in this, the new technology and industrial age. And it's going to have a positive impact on our life. But if we don't make the right choices and how that technology is going to be used or how it's going to be developed.
it's going to have a massive negative. It's going to create negative industry, negative social harm, negative disjointed communities. And that's why we need to actually talk about the positives and how we can bring technology into a positive environment. It's going to create jobs, it's going to create commerce, but at the same time it can also cause harm. And that's what we need to do is actually find that balance. And trust seems to be a huge theme in your recent work.
around AI adoption and it leads into the explainer statement you just said. Why do you think trust has become such a challenge with AI? There was a recent study actually released in Australia by the Tech Design Policy Institute in Sydney and 45 % of people who are surveyed, while they want to actually have AI as part of their everyday life,
And they don't believe it's regulated or they don't have the trust in the actual deployment of that technology. And I think what people want to see is that there is these guardrails, these controls where if something goes horribly wrong, that they can pull the fire lever and actually stop the AI. I think Skynet has a lot to do with people's conception of technology. But they want to see that there are safeguards in place so that it doesn't run right. It doesn't run wrong. And we saw
Adam Firman (18:51.733)
We got a glimpse of this with Grok. When we look at Grok AI, that development, how perverse without having those checks in place, that an AI companion can actually go horribly, horribly wrong and still accessible to all ages. I I found the defense of a 9-22, but I can only imagine what a 13 or 14 year old could actually feel when they saw this Grok AI being put.
in front of them and also the abuse and the innuendos is just wrong. There almost is a need like companies always have IT policies but an AI policy is you know funny enough my wife works in HR she is currently going through creating parameters to limit what people can share confidential company information because people do it for ease.
They do it for ease that it can quickly generate and automate, you know, manual tedious tasks. But there's any, there's any one company profiteering from this and that's the AI engine that's been fed more and more information. And this is the issue I have because I remember having a conversation with somebody and going, you know, we can check the model for bias and went, okay. But when we actually look at AI and we look at a model that you implement into an organization.
And it could be a toxic organization. What happens is that model is going to hallucinate. It's going to become corrupted. And then you've actually introduced a bias based on the, on the material that it's actually learning to help build up those tasks. So I think that's where we need to have guardrails. need to have the pull the switch where if we're actually trading AI to develop with our organizational values is that it's a way, there is a way to actually bring it back. If it does corrupt or it does hallucinate.
The same thing with privacy and privacy is a big aspect of mine as well, is that if we're using AI and we're going to have it deployed across our organizations or in our corporate structures is that it stays within our corporate structure. And there are, there are services that are out there. use, we use it like ICMIC. We have in ICMIC Australia, we have an AI service that we use that it just focuses on our corporate structure and stays within our cloud tenant.
Adam Firman (21:14.997)
And what I'd say to people is there are applications out there that you can use that are readily available. Um, and you should be using them because it's going to teach a new skills. It's going to teach a new ways of thinking. Um, it's going to also allow you to question the results that are coming from an AI tool where you go, no, that's rubbish. Um, because I would say to people is there is nothing more complex. is nothing more valuable than the human brain. So just because you're using a tool.
don't leave your brain in the drawer because you're going to need to be able to question things and be able to make sense. I think one of my favorite memes that I saw on LinkedIn was a cartoon character asking AI if this mushroom was safe to eat. And the next picture was the gravestone and the AI saying, you are correct. That wasn't safe to eat. Yeah, yeah. I'm going to update my memory now. Yeah. Yeah.
I mean, that's the biggest thing. I mean, I know we talk about new regulations coming in. We talk about the EU's AI Act. And we also look at the different acts, the AI Act that's been passed in the UK and all over the world. And I'd say to people is don't lose your brain. I know we talk about the person that's going to be actually in the center, person in the center to actually make the judgment calls. I guarantee you with the amount of pressure being put on people, both in government organizations and private organizations, the pressure to actually get jobs done and move.
move the files or move the actual workload as quickly as possible. They're going to press the space bar and just be going, yeah, the computer must be right. The computer must be right. It's been trained. It knows more than I do. It doesn't know more than you. It shouldn't be making the decisions on its own or independently. You need to actually process and then assess. That's what you need to do. Don't just like process and then move on, process and move on. You need to assess the information that's been put on front of you.
So that's the worry and concern I have in relation to people's overreliance on AI. And we spoke about this before Press and Record and it was the concern around push button, mobile forensics or digital forensics in general. And I would even throw it as far as any investigation. If you're relying on AI and you're accepting what I was telling you, that's a huge danger, a huge danger.
Adam Firman (23:35.061)
Yeah, it's kind of actually bringing this really amplifying the Nintendo forensic scenario where you like find all evidence on the keyboard and hope for the best and see what happens. I mean, you know this is more than I do. It's a long place in the evidence box when you're in a court prosecution and you can't actually stand over the findings and you're saying, well, the AI model told me that this was the case. Yeah. You're the examiner. Like it's your evidence that's been put in front of the court. You're the one that signed the statement. You're now actually at the...
hendering this into evidence, like you need to be able to stand over. And what I'd say to people, definitely people who are starting off their career and people that are advancing their career in digital forensics is process and assess. Don't be just actually processing. Now I know there's massive pressure on caseloads. I know there's massive pressure on files to get them in and get the results out as quickly as possible, but you need to bring quality assurance because a missed prosecution
or a mis-investigation is actually going to impact on the rights of victims and survivors as part of that investigation. And I'm always going to bring the child and young person's voice into this. But even when we look at adult victims, like especially when we're looking at non-consensual into the images, when we're looking at image-based abuse, hate speech, all these different types of crimes, they are not a faceless crime. They are a crime that actually has a victim. And that victim has a right to justice.
And while you're getting pushed to actually process the backlog, I would say to is the backlog needs to process for quality and does not not people make the case to get more people. And I don't want to make it appear that we're just knocking the AI because we've already spoke about how it's going to assist. Where do you believe that AI can genuinely help investigators and people working in child protection?
I think when we're looking at it, when we're looking at big, big data sets, and we know that the mobile devices are getting bigger and more powerful in relation to the storage and capability. AI is a great tool. Like I look at one about age assessment. And I was just talking to somebody quite recently, there's a really good tool that was brought out. This is not a soft plug for the old company I used to work for, but Rigor AI brought out an age assessment tool. And the whole point was for
Adam Firman (25:57.565)
on a company minor and for victim identification. And there are times where AI will actually within plus or minus four months give you an accurate age of that face to actually say this is a face that you need to look at. With the amount of images and videos that have been processed nowadays. You need to be able to do that. Yeah, the same way when we're looking at videos, like being able to actually not listen, but being able to bring out the text and being able to search the text.
So what has been spoken, translating that, there's loads of really good tools that AI can actually help the digital forensic investigators and just general investigators about what they need to actually look at and hear and listen and translate and understand what's going on. Because the world is a more diverse, a more traveled, more interconnected world. We cannot expect everybody to speak every single language that's happening in the world.
but we can use technology to actually translate those, those speaks, those talks, those texts, those interactions so that you have a firm understanding about what has actually been said and what is going on. And it's all about validation, especially in digital forensics, validation and critical thinking are everything. You know, the industry falls down without it. Do you worry that AI risks creating a false sense of confidence for investigators who are
who are just starting out in this industry and that's the way they've been shown that you just push the button. Yeah, I think I know when we were in the forensic world many, millions ago, we were actually taught the basics and taught the very hard lessons of how, how a phone or how a device operates and going right down into the filing system. Like we're laughing there about the Simian system talking about reverse nibble and the padding figures and everything else of that. Yeah.
So, I mean, we've grown up where we've actually had to actually look at really the raw data in great detail. And I do fear for the person starting their forensic journey. For the simple reason is there's so many different pressures being put on them by managers who don't understand the difficulties and the pitfalls of AI, by organizations who won't invest in additional resources for our digital forensics.
Adam Firman (28:14.911)
because we know that every investigation will have a digital component to that investigation. But yes, sometimes they're the most under resourced units within the policing environment and within the actual investigation environment. So I'd say is I do, do worry about the future for those who actually have that critical thinking, because if they're pressing the spacebar to move it on and not actually engaging the critical mind and their concept of actually thinking, they're actually
going to do themselves a disservice, but they're also going to do the victims of crime a disservice. And what skills do you believe investigators and forensic practitioners should be developing now to remain effective in this sort of AI assisted world that we're in? Yeah, I definitely think they should know how to use GBT correctly.
And that's the first thing I'd say, because what happens is you're accessing massive amount of written resources from certain GBTs, whether it's an M365, whether it's going to be, I don't know, the old Reliables, ChatGBT, Claude, all these different ones. Learn how to actually prime, polish and prompt. Prime the system to get it ready to actually work for you. Polish the request that you're going to ask and then put your prompt in. Don't be rushing to the answer.
set the environment and so you'll get the best results. So I say is learn how to actually engage with, with GBTs. That's a really good skill set to have. I remember back many, moons ago in forensics, we, we purchased a translation service that we could run over text and, it it would come back and it would translate sort of 80%. So it would allow us to do a quick investigation.
we would never, even back then, we would not rely on it and we would hire a accredited interpreter to come and tell us the full details because, and it just goes to show, we put that system in place. We used it effectively because we could quickly see if a crime is being committed, but we never relied upon it. I think that's quite similar for this sort of AI world.
Adam Firman (30:33.533)
Absolutely. I mean, if you talk to anybody that has any answer respect for research, you won't use Wikipedia as a resource in a white paper. Because the fact is you actually question it and you actually ask that critical thinking again, and go back into critical thinking. You're kind of going, this is things that I need to look at. This is something I need to ask questions. So when a result comes true, you kind of go, is this actually accurate? How accurate is this?
And you start to develop an instinct and kind of go, this is probably not going to be that accurate. So this is not a valuable use to my time. So I would say to people is like, if you're you're using those tools, quality assurance is number one, number one priority. And because you would also wouldn't expect the forensic labs when they're doing DNA results and stuff like that to actually kind of go, sure, we'll give it best guess. Yeah, how it goes. Exactly.
It's just a foreign concept. And for people out there, this podcast has a global audience. Is there anything that they can connect with you and ICMC Australia that can assist them? Is there any sort of or any help that they could give you or vice versa? Is there anything that you want to reach out to people? I mean, definitely look at our website, ICMC.org.au. That's ICMEC.org.au.
and have a look at the different resources that we have there. We have items on the Safer AI for Children and we have a Safer AI for Children coalition that we actually present policy documents where we want children to actually learn the skills of the future, but the safety of the present. And that's one of the biggest things we need to think about is, yes, we want to develop our children and young people into the module world, but we don't do it at any cost. We want to do it at a safety and in a controlled manner.
And also I say deal socials, Instagram, not that I've ever gone on Instagram, but Instagram. We also have LinkedIn. But like what I'd say to people is, is that we're more than happy to answer any questions people may have or point people in the right direction. We're closely affiliated with other not for profit organizations in the UK, all around the world, Australia, in Africa.
Adam Firman (32:55.005)
Everywhere we actually talk to our counterparts, we work as a global network, a global partner so that we can actually represent the global community of children and young people. Fantastic. And if you could make one change in this space and with all of the big sort of corporations and money was no issue and you could get it through legislation, what would that change be?
Can I have two? You can have two. Yeah. Okay. The first one is I would like that any technology whose sole and dominant purpose to create child sexual exploitation material faces criminal offences. I know the UK have just done this recently. I tried to do it in Australia in July last year with Kay Cheney. But that's the one thing I'd like is that that law is passed internationally and that people who actually create that type of technology actually are held accountable for criminal purposes.
The second thing I would love is that all the technology service providers, when they talk about the transparency reports is those reports are actually transparent and they're actually reviewed by an independent oversight body and not released just for the sake of the actual motivation of social media platforms or tech platforms. I think it's really vitally important is that if we want faith and if we want trust,
within our social media platforms is that we have an independent body assessing them in relation to what they're saying and what they're assessing. And when they're talking with their takedowns and they're talking with their pushback and warrants and also their algorithms is that we have somebody that actually assesses them on an international level and make sure that our social connection is not our downfall of a connection. And makes them accountable. Absolutely. And it's the problem.
is like, look, I talk to these guys and they say like, we have an independent, transparency report went, who, who created it? we have our team in our own organization. I hate to it to You might want to, your dictionary is definitely very different to my dictionary when we talk about independent. I mean, if your organization is paying their wage day in day out, I would say to you, you have an issue there of independence. It's also, we spoke about the impact to
Adam Firman (35:19.019)
people who work within this space and you and Isaac's sort of law enforcement, it would be seen as that's expected as part of our job to deal with trauma and sort of horrific things. Some of the people working at these tech companies are probably not much past 18 years of age and they're sitting there, the sheer volume of what they must be reviewing. It just, yeah.
I know definitely when we look at the moderators and we look at the trust and safety teams at these tech companies, mean, they are seeing some serious, serious amount of content at a very high rate and very high level of volume. And it's not just your simple cat memes or your dog AI generated movies. They're seeing some of the hardest things that they probably ever see. And you're right, they're probably quite young.
Although I've seen some of them and they're they're not all in their mid or in their fifties. But there is a majority of them that are actually quite young and they only have a survival rate of about five to seven years actually in a trust and safety environment. Because it's just absolutely catastrophic. And I know that certain service providers will actually look after their people and take time.
But how they're looked after sometimes is that they're isolated from normal communities within the actual technology companies because they're looking at such graphic content and they're isolated even further. And then they actually have to go into their viewing rooms and view the material. it's a tough job in the Trust and Safety. It's not for everybody. And I do commend them for the work that they're doing and especially some of the reports that they're picking up and referring them to law enforcement.
as part of cyber tip reports. They have to be commended for their work. They have to be told that they're actually doing a great job. Yeah. And the worst thing is law enforcement get these cyber tip reports about these, these children and these young people and never bought a report back. And those moderators are sitting there kind of going, I wonder what happened with that referral. Yeah. They don't get the, they don't get the mission impact part that you, that you and I.
Adam Firman (37:31.955)
spoke about earlier that, you know, keeps you going in such a horrific world? Like you say, if they're not receiving that feedback, that's hugely. Yeah. Yeah. Yeah. I mean, they, they're, they're motivated to actually make sure that children are not being hurt. I know it's kind of like a Noxie moron where they're working with a tech company, but they are motivated to actually protect children. They are motivated to protect users, but then they actually don't see what actually happened in the end. Is that child safe? Is that young person safe? because they just don't get told and.
And they're working in a void. That's how I find sometimes the trust and safety people at these tech companies are. But at the same time, you also have trust and safety. You have people that work at these tech companies that won't actually invest in the engineering resources because that's a cost. That's an overhead for them to have this trust and safety department in there. And they're kind of going, well, we don't get any money out them. just take stuff and then we get into trouble if we don't have them and they have to have reports. And they're actually costing us. And they're actually eating into our profit.
Yeah. I had one person tell me that they worked in Trust and Safety for a long time for very large online provider. And they said that this very large online provider spent more money on lobbying than they did on Trust and Safety year in, year out. Yeah. Because it brought more revenue in. Correct. Yeah. Yeah. It actually increased their profit margin, made their stakeholders and their shareholders a lot happier. Yeah. And just to close out for if we haven't put people off already,
If anybody is considering entering this, this sort of online trust, safety, child protection, is there any advice that you'd go back and give somebody starting out in this industry?
learn to actually pigeonhole yourself, learn to compartmentalize. It's not a job for everybody. That's the first thing I'd say. It's not a job for everybody. And if you don't succeed and you find it too horrific, that's okay. You haven't failed yourself in this career choice. But if you do find that you can actually do this job, this job, it's not even a job, it's vocation. It's a calling. You're actually going to impact, have a positive impact on communities.
Adam Firman (39:47.497)
And sometimes those communities are not in your country. Sometimes those communities are not in your neighborhood. Those communities can be far, far away in different aspects of the globe. But I can tell you now, there is nothing that will actually bring that spirit of satisfaction knowing that you are involved in actually safeguarding and protecting the child. as always, a huge thank you to Kong for joining me on Forensic Fix today.
I really appreciate that conversation is. I knew it would. We went beyond the technology itself. It's a discussion about responsibility, trust and the human impact behind the systems that are being built and that we use every day. And one of the strongest themes about today's episode was the idea that technology is not inherently good or bad. It reflects the intentions, the safeguards and the decisions of the people creating it and using it.
And as investigators and forensic practitioners, it can be easy to become focused on capability automation because of those backlogs that we spoke about and efficiency. But as we've highlighted, technology should support better outcomes for people. It's never going to replace the critical thinking, the empathy or the responsibility that we have as individuals in this industry. And our conversation around AI, we weren't just trying to beat it up.
it is going to play a huge part in online safety and digital governance, but we need to put barriers in place. And for those of us still working in digital forensics, in investigations, these are not distant policy discussions anymore. They're becoming part of everyday operational reality. So, Carl, from myself, thank you for sharing your insights and your perspective. It's been a real pleasure to have you on the show and to our listeners.
Thank you for joining us to episode 27 of Forensic Fix. If you found this episode valuable, please share it with your colleagues. Even if you know people not in the industry but are interested in this space, share it so people are aware of what's being done and some of the great work that's being done by nonprofits. And make sure that you subscribe and don't miss any future conversations that we have. And just for one last time, Tom, thank you very much for joining us.
Adam Firman (42:09.289)
Yeah, I look forward to meeting you in person very, very soon. Thanks Adam.