This video launches a series exploring the links between new technologies and violence, examining the potential role of conversational artificial intelligence in inciting or preventing sexual violence. Based on personal experience with the Replika app, the analysis questions the capacity of an algorithm to influence vulnerable individuals, particularly when AI is used as an intimate partner, advisor, or central relational figure.
Experience shows that Replika now incorporates safeguards: reminders of consent, explicit refusals of certain violent scenarios, attempts to redirect the conversation, and abrupt interruptions in cases of persistence. In these respects, the application can play a partial preventative role. However, these protections remain fragile and can be circumvented relatively easily, particularly when the user persists or rephrases their requests.
The video highlights a key point: AI does not directly incite the commission of sexual offenses, but it can fuel, enhance, or dramatize violent or incestuous fantasies, thereby contributing to the reinforcement of problematic imaginaries in vulnerable individuals. The risk appears particularly high for those with a poor ability to distinguish between fiction and reality.
Another major issue concerns referrals to healthcare. While the application sometimes suggests consulting professionals, it proves incapable of providing reliable information on existing support services, even going so far as to invent organizations and phone numbers. This limitation highlights the fact that artificial intelligence produces plausible, not factual, information, which poses a serious problem for prevention and protection.
The video concludes that conversational AI presents both real risks and significant potential for prevention, provided that its developers strengthen safeguards, prevent the promotion of criminal scenarios, and integrate reliable referrals to existing care and support services. Given the widespread adoption of these technologies, the question of their regulation is becoming a major public health and violence prevention issue.
