I'm a filmmaker testing OpenAI's Sora. Here are its best features and biggest limitations — and the prompts I use.

Every prompt in Sora goes through multiple iterations. A lot of the stuff I've been doing is trying to get both guided or dynamic specific camera movements as well as aesthetic. So I see if I can create the subject that I'm asking for, and it doesn't really understand everything. Sometimes it'll jumble ideas together and kind of mutate what you're asking.

Then I have a cocktail of words that I use to make sure that it feels less like a video game and something more filmic. But even certain words will end up making things feel more like a video game. And so you're combating certain words with counter words to try to see if you can nail the aesthetic you want. I throw out "35 millimeter," I throw out "anamorphic lens," "depth of field lens vignette," "Fuji film," "stock analog." If you don't put anything in, it'll kind of default to this very digital-looking output.

One prompt for "The Golden Record" started: "moody analog film shot we zoom in motion blur volatile rocky meteorite landscape of raw earth with golden record soaring through the dark depths of space, floating surreal rocks, and smoke filling the air. the scene is chaotic and primal, with the destructive power of nature unleashed." (Scroll to the end to read the full prompt.)

Everything has to have their seal of approval before it's public. They have all these safeguards — no nudity or violence, likeness, anything that's obviously copyrighted. Some of it is a little bit too protective. It is even confusing for us alpha testers what flags are being raised. But you can still do a lot with it. What this does is it allows me to just try ideas with no stakes involved. It's incredibly liberating.

I've gotten a lot of people from the ad world asking, "Hey, how do I get into Sora?" That's probably the most boring use case for this amazing technology to be used in a commercial as a replacement for something that could be done otherwise.

Sora is far from being ready for Hollywood

From a production value and quality standpoint, we're still ways away from this being used in films. They have the best compute, but they still don't have enough to really service tons of users. There's still a huge element of this being like a slot machine where you ask for something, and it jumbles ideas together, and it doesn't have a real physics engine to it. It's just sort of a mirror reflection of what it's seeing. There's a long way to go to create character consistency. Maybe 10 years from now, it could really make a difference in someone's life that didn't have any resources. But people are going to find the experience of using this not as thrilling as the experience of being on set with real people. What makes filmmaking a unique field is the collaborative aspect of it that just can't really be created in an isolated experience, talking to a machine.

Sora could reduce the cost of filmmaking

I am excited about the more experimental use cases of it, but there certainly is the component of saving money. Films are incredibly expensive. They're also incredibly wasteful and inefficient. So I don't think it's a bad thing to try to make the process more efficient. That's maybe the side that people are missing coming out of the strikes. The studios have been pretty blatant about they're going to greenlight less projects now because they have to put more writers in the writers' room. They have to pay more on residuals. If this can create channels for things to be made that wouldn't have otherwise been greenlit and that we can actually start producing more, I don't really see that as a bad thing.

I think post-production is definitely more in for a shake-up. A lot of indie films just don't have the luxury of having anything for post-production. If this can even the playing field for indie films to dream a little bit bigger, and shoot a little bit higher, ultimately that's a good thing.

Sora has the lead now, but eventually there's going to be an open-source version of this that's not going to be made by OpenAI. It's likely Stable Diffusion will at some point catch up. I could see it replacing aspects of specifically B-roll and insert shots, or pickup shots, or anything stock footage. As a director, you want to spend as much time working with the actors. You don't want to spend your time on these insert shots that are not as critical to the story. Also, stock footage is incredibly limiting and aesthetically doesn't always match.

Sora is still a research project in alpha

OpenAI does not have a timeline for it. It is a little confusing who the market is for this. I hope they don't dumb it down because the tool is more complex than I think people realize, but it needs to be wrangled, and I think it would be a disservice to just make this some sort of one-off content creation factory machine. I hope this goes in the direction of the more sophisticated route. There may be more money in Hollywood, but there's going to be a lot more they're going to need to do in order to service that really high-end.

I am trying to toe the line between playing with magical technology that could only happen because of these tech companies and reassuring filmmakers that this is actually a great time to be a filmmaker because of the ability to envision your ideas that we've never been able to do before. That's why I'm talking with all these companies: To make sure that they know who they're making these tools for. I do worry about the overuse of AI and what it's going to do for us skill-wise, and creatively, and culture-wise that we could dull ourselves. And I think that's the most boring use of AI.

Here's the full prompt for "The Golden Record" (paragraph breaks added):

moody analog film shot we zoom in motion blur volatile rocky meteorite landscape of raw earth with golden record soaring through the dark depths of space, floating surreal rocks, and smoke filling the air. the scene is chaotic and primal, with the destructive power of nature unleashed.

the scene is captured from a fast zooming perspective, as if the viewer is flying through the air and observing the scene from above. soft natural sunlight, the shot is blurry and distorted, with a 35mm fuji film grain stock. analog. Motion blur and filmic texture, moody organic light from the sun in space, NASA, anamorphic lens, vintage tone, film grain texture.

analog film shot we zoom in motion blur the scene transitions from the rocky landscape to a hurdling golden record flying through the air in a rocky void his body begins to melt. the viewer can see a blur of motion as the melting man rushes past, the scene is captured from a fast zooming perspective, analog film shot we zoom in motion blur the viewer emerges from the hurdle golden record and enters a field of floating rocks. the scene is filled with surreal floating and flying rocks.

the golden record falls into place and flips and rotates through the darkness of space, the scene transitions from the field of floating rocks into the open hand made of gold shards belonging to a liquid nebulus transluscent iridescent jellyfish alien god with tendrils spreading through space, the gold melts into a circular, the golden circular record flips and flies and rotates through the air, the viewer can see the golden record open up, as if they are zooming through the crystal hand and observing the scene from below.

the scene transitions through a sweeping vastness of space of the golden record flips and flies around dynamically we pan around following the action as a translucent jellyfish alien figure interacts with the golden record, flooded with iridescent rivers with a golden record soaring through the an explosion of dust and rocket smoke. the viewer can see the city from a high angle, as if they are flying over the streets and observing the scene from above.

we fly into a low angle of the iridescent river, the shot is blurry and distorted, with a 35mm film grain effect. Motion blur and filmic texture. the scene transitions and emerges out onto the side of a golden record that flies up into the air, bioorganic organs liquid nebulus transluscent iridescent jellyfish alien god with tendrils spreading through space, melting golden record made of iridescent golden record flies into outerspace, into the distance. the shot is clear and crisp