Generating video from a text prompt is one of the most impressive capabilities of modern AI. However, it also raises serious concerns regarding fakes, deepfakes, content depicting children, and disinformation. OpenAI understands these challenges and is proactively discussing how protection is built into Sora and its new application.
Why Video Is a Special Case
Text and even image models have already established certain best practices in safety. With video, everything is more complex: it is perceived as more “real,” spreads faster, and is harder to verify. A fake video of a real person can cause much more harm than a text statement attributed to them.
This is precisely why OpenAI emphasizes that Sora was created with safety in mind from the very beginning, rather than as an add-on to a finished product. Simply put, their approach is not “release first, figure it out later,” but “figure it out first.”
What Is Specifically Prohibited
The list of restrictions is quite predictable, but the details are important. Sora will not generate:
- sexual content involving minors (this is an absolute restriction, with no exceptions);
- realistic videos of real people without their consent – especially in compromising or sexual contexts;
- content that could be used for mass disinformation or manipulation;
- scenes with real violence presented in a positive light.
At the same time, OpenAI notes that creative freedom is important, and the system should not block everything “just in case.” The balance between what is permissible and what is forbidden is one of the key challenges the team is working on.
Digital Credentials: How to Distinguish AI Video from Real Video
One of the specific protection mechanisms is digital watermarks. Every video created through Sora receives an invisible marker based on the C2PA standard (an open industry standard for labeling AI-generated content). This marker is embedded in the file itself and is preserved during distribution.
While this is not a panacea – technically, someone could try to remove the marker – in combination with other measures, it allows platforms and verification tools to determine the video's origin. If you see a clip and want to know if it was made by a neural network, the presence of such a marker can provide the answer.
Moderation and Red Lines
The safety system operates on several levels. Before generation, the prompt is analyzed for potentially harmful content. During and after, the result is also checked. OpenAI uses both automated classifiers and manual review in complex cases.
So-called “hard limits” are separately highlighted – these are things the model will not do under any circumstances, regardless of the prompt's wording. Sexual content with children is one such case. OpenAI emphasizes that there are no “creative exceptions” or workarounds here.
A Social Platform – A Separate Challenge
The Sora app is not just a generation tool; it is also a platform where users share their work. This means that in addition to standard safety issues, there are those typical of any social network: reposts, remixes, anonymous accounts, and viral spread.
OpenAI states it is developing its policies with this specificity in mind. In particular, this means stricter requirements for content that is publicly distributed within the platform compared to what a user creates for personal use.
Research and External Oversight
Before launching Sora, OpenAI engaged external security researchers – so-called “red teamers,” whose job is to intentionally look for ways to bypass protections. This is a standard practice in the industry, but it is especially important for video models, as the potential for misuse is much broader here than with text-based systems.
The company also collaborates with organizations dedicated to child protection and combating disinformation. This is not just a PR move – such partnerships provide feedback from people who see the real-world harm of such content in their work.
What Remains an Open Question
An honest look at the situation requires acknowledging that no protection system is perfect. Classifiers make mistakes. Malicious actors look for workarounds. The scale of the platform inevitably means that some unwanted content will slip through.
OpenAI is transparent about this. The company states directly that safety is a continuous process, not a state that can be achieved once and for all. This is an important disclaimer: it means that the rules and mechanisms will change as new threats and new ways of using them emerge.
For users, this is perhaps the main takeaway: Sora is a powerful tool with real limitations, and these limitations exist not to hinder creativity but to prevent the technology from becoming an instrument of harm. How well this balance can be maintained in practice, only time will tell.
