AI overview for tech illiterate TV people
Hey folks
I've got a couple of months to put together an overview for tools that a company could use as part of television production and I'm hoping for your input.
It goes without saying that everyone in the tech world is pushing ai heavily. Having been in IT for almost 3 decades I know what to watch, look at, out for, etc. AI is still very much regurgitation of its input but the input is vast. What I have right now is some bare bones of what I want to throw around for insight and discussion for what would help people in TV production tool wise.
For those that do not know how TV production works it's a simple idea: you generate a huge raft of ideas for shows, absolute basic outline of what the show would be about and put that in to a paper. You then sit around in your research/Dev dept and pitch to each other and the ones that people go "yeah, that could make a good show" get some extra meat added. Those ideas get pitched to dept heads who then take the best ones to channel/broadcasters execs and see if any get hooked at all. If they do, they get given some development funding to put together a taster/pilot/video version with the funding they have. This means shot on camera, run through an edit for cutting, audio, graphics, etc, still in its infancy and development state. This video and a bigger padded Treatment (documented idea with its bones, flesh and now make-up added) goes back to the broadcaster and you wait for feedback. If you get lucky you get a greenlight and order for X amount of shows and then you have a production. The production is taking the idea to it's full potential, shooting it, audio and music, graphics, the works and that's what you see on TV.
I'm after working out what tools AI offers today that would help them with this process. Right now, ChatGPT v4 will generate some great treatment ideas for shows, except I would imagine these shows already exist or have been tried to channel/broadcaster before? AI is regurgitation and not thoughtful to its own ideas and imagination. I suppose with great prompts it could generate great output.
Okay, that's the process and I'm rambling. Right now I have a short list of LLMs such as ChatGPT and Bard types that will help with the idea stage for researchers. I could use some decent links for prompters to help the research know how to ask AI for what they want out of it.
When it comes to generative AI for graphics I only have experience with txt2img using the likes of DALLE and Midjourney, along with some inpainting for changing images with lies, I mean, graphics (insert plane on fire, etc).
Does anyone have any other ideas and tools which would help production or useful things I can look at and research myself to see how they could be helpful? Auto audio generation? Graphic building that takes less time? Think of those great show intros for the likes of Game of Thrones, can that be done using AI yet or are we no where near that level for AI? Even basic video edits, where are we for AI help? Can we feed it some clips and have it autostitch based on an input document? If so, what tools should I be looking at and researching?
I'm asking here before I plop search terms in Google and Bing and then get swamped with whichever has paid the most or played the SEO game to be top of the pages. Asking for real human input is definitely better than asking AI which may actually be the whole point of my talk when it happens.
Thanks for listening and any help/pointers/sites you can give.
UPDATE:
I went off and did some research. Enjoy these if you want. I had issues linking so if a mod wants to go ahead and do that, feel free:
Pre-Production:
Treatment idea generation
Generating a great idea is usually through using knowledge and research, but these days you can literally ask an AI engine to come up with a show idea. Here I will list some good AIs that use a very large language model (LLM) to come up with ideas:
ChatGPT4 from OpenAI
ChatGPT is the best known AI out there, but essentially it's the AI that everyone uses. What's different is the data that is fed to it. ChatGPT from OpenAI has a lot of knowledge, however, it's generally backdated information and not up to the minute.
You.com
Built on ChatGPT4 AI. Data fed in more up to date as it's based around a search engine. Due to the plethora of sources being fed to the You.com Chat bot, you may find some more interesting results and ideas.
Bing.com - Chat
Directly leverages the latest version of ChatGPT4 from OpenAI but uses additional media from Microsoft sources. Responses are more natural due to the Turing Natural Language.
Copy.ai
A fun LLM designed for advertising agencies and the alike. The difference here is you can upload a back-catalogue of your own data for it to analyse to take on your brand voice, mix up your ideas and generally become one of the family.
Prompting
Just from picking one of the four AIs listed above, you can straight out ask for a basic show idea. All of them came back with interesting ideas from the prompt of "Generate me a great show idea for a television production treatment. The show should be a documentary for daytime viewing."
Prompting is the hardest part of any AI interaction, the results can wildly vary depending on what and how you ask. Due to this, there's a new type of website to help with prompting:
https://promptperfect.jina.ai/prompts
Using the line from above about generating a great show idea, promptperfect injects a lot more information into the prompt before running: "Please create a compelling show idea for a daytime documentary television production. The show should be engaging and informative, catering to a broad daytime audience. It should focus on a specific topic or theme that is both educational and entertaining. The documentary should be well-researched and provide in-depth information on the chosen topic, presenting it in a visually appealing and accessible manner. The show should aim to captivate viewers and leave them with a better understanding and appreciation of the subject matter. Additionally, please provide a brief outline of the structure and format of the documentary, including the number of episodes, approximate runtime, and any unique features or storytelling techniques that will make the show stand out." The quality of the Treatment created will be far superior to the initial request.
https://webutility.io/
An interesting take on generation of prompting. It breaks down the prompts to dropdown boxes with key words such as create, design, analyse along with the focus type. This forces the ai to create some more complex and well thought out documentation for a treatment idea with explanation of how it got to where it did.
AIs to help with show production
Location finding/scouting
With the latest AI image searching features, you can now upload an image and get a "related" search. Using this technology, you could, for example, look for English Country Gardens that you would like to film out of. Uploading this image would give you a list of locations, similar places and website associated with the image:
On each of the following sites, in the search bar, click the Image Icon to upload the image:
https://www.bing.com/images/
https://images.google.com/
Scheduling (not specifically AI)
Scheduling shoots should be simple. We've seen all the fun from an Excel spreadsheet that's laid out like a calendar, through to the most complex diary entries in a shared Google calendar. We already have the tools for this in Microsoft Office:
Microsoft Bookings: This is a great tool for scheduling a diary of a single person or a whole team. It allows to have a Web Page where people can book in time for appointments, whether virtual or in person. Perfect for a researcher trying to book interviews with a host. The AI lies in the ability to cross search a calendar and pick associated times available.
Microsoft Planner: A tool for project and time management. Breakdown the show in to buckets (categories) and assign out tasks to people and teams, due by dates or exact dates, etc. You can even keep all of the documents in the plan.
Microsoft Shifts: Team management for your production using Shifts. This allows you to schedule team members in Teams, allowing them to clock in and out, as well as specifying when they need to be available.
The three tools all work with the Outlook Calendars so each person knows what their plans are well in advance.
Post-Production
This is the one most people are interested in for AI at this time. The tools used for image generation, manipulation, etc. The market is currently being flooded with tools and not all of them are equal, but here's a few ones to watch and use.
Auto-Clipping & Social Platform
OpusClip, using the power of OpenAI, can take a long video and create 10 viral clips from it at the click of a button. The AI behind it analyses the video, looks for compelling sections and highlights, then seamlessly rearranges in to short videos. This tool will be great for generating short promotional videos of long form shows, documentaries, etc.
Descript is a great tool that can take a video, give you a transcription, then you can edit the transcript, where it then edits the video to match. You can remove words, create studio quality audio from a standard mic, remove common error words such as um, and er, etc. One of the bigger cool things it can do is voice mimic using AI. You read it a line and then you can type out a whole transcript and it'll narrate it in your voice and allow export.
AI Generative
Moving on to the more scary AI platforms, we have completely generative AI. This is where AI generates absolutely everything including the "avatar" of the human speaking. It's getting so real, you could probably make a documentary using nothing buy AI voice for narration and even have an interview with the AI Avatar.
Video Generation
Synthesia has 120+ voices, over 140 AI Avatars and an editing tool that is extremely easy to use. Mostly aimed at Sales, Training and Marketing Teams, but could easily be used to create development tasters and cuts by mixing in the AI with real video. An example video here.
AI Studios from DeepBrain is another tool, similar to Synthesia. The avatars are based on real humans being recorded but then converted in to AI models. Again, lots of models, full text to video.
Spline AI is a 3D modelling engine that will generate models from text prompts. It's still in Alpha stages but specifying something like "A cube", "rounded corners", "floating", "spinning slowly" will generate exactly that. This tool is aimed at animators but is likely where CGI effects will head.
Still Image Generation
Txt-2-img is amazing and growing at an ever rapid pace. With the wealth of images out there to learn from, the styles, etc, it's no wonder it's doing great. However, it's far from perfect, even now. You'll often find that it adds limbs or fingers to models, shadows completely wrong, crazy styles that are not what you asked for, and that's just the start of the issues with it. However, when it gets it right, it's amazing.
DALL·E3 from OpenAI is the current leader in image generation. If you need to whiz up a picture of a steam train, crossing a suspension bridge at sunset with a woodland in the background, this is the tool of choice.
Bing Image Creator is probably the second biggest right now and has very good accuracy of text to image due to the absolutely huge database of images with high detail being fed to it by Microsoft. It's also free.
I'm not going to list too many more as a lot of them stray off in to fantasy land, being trained on Anime, comics, however, DeepAI definitely deserves a mention. These are the folks behind a lot of the viral videos where you can scan your face and and speak a few lines, then it adds you to a section of a movie as a "Deep Fake". You can have it chat, generate images and even AI edit images with txt-2-img.
Video Edit Tools
The biggest AI enhancers right now are tools that help in the Edit at a professional level.
Topaz Video AI is one of the leading tools in Post production. Upscale footage from SD to 8K and HD to 16K. Full denoise, sharpening, 16x slow down with AI interpolation including building new frames. Corrects people and faces. AI Stabilized video to stop bounce and tracking issues. This is a complete Post Swiss-army knife.
Adobe After Effects which everyone knows. The Adobe AI, called Sensei, is under constant development. Easy animations of text and logos via text to video, rotoscoping video objects to remove the background of a person and replace, or removal of all objects in a scene using AI generative filling is all extremely easy.
Adode Premiere deserves a mention, but again, this down to Sensei. The current AI tools coming in to the suite are things such as Auto Rough Cut using the transcript to generate the video, full auto transcription with subtitle creation for multiple languages. Auto Colour will fix most colour issues using AI to save time in grading. AI Morph Cut adds visual continuity to cut transitions, remix for music matching with visuals, and Auto Ducking – popping dialogue over background audio to make sure you can hear voices correctly.
ColourLab AI is a new kind of grading tool where you no longer need to spend time with an artist grading every scene. The tool is a plugin to Davinci or Premiere and will do cool things such as film grain matching or stock emulation, which allows you to match any scenes together to look exactly the same. Take a video of a pigeon flying over a statue in London, and have it grade using a still frame from The Martian to get those awesome colours automatically, for the whole scene.
Audio/Narrator/Voice Over
The final piece is the new voiceover AI generation. No longer do we need voice over artists. In fact, Hollywood thinks the same and fired the whole staff of Snow White and replaced the Dwarfs with CGI and AI voices.
Altered Studio can change any persons voice, in any way you wish. Record your voice for narration and then adjust it to be male, female, Elvern, whatever. It also does full transcription and allows for VO with text-to-speech using AI voices.
A quick shout out to a member of Tildes who wants to remain anonymous for some of the cool links that they sent over - much appreciated.
I’m a software engineer and most recently a product manager who is working on AI products right now.
I would love to help with this to exercise my PM muscles in a different space.
This is the thing thats going to differentiate your process by using your understanding of the situation and coming up with prompts that work.
Let me give you some more understanding of the development process for TV ideas. We're not talking the latest Hollywood's A-list celeb outing, we're talking about stuff that goes out on daytime or terrestrial TV. Think Bake Off, Auction shows, documentaries on how spiders live, or maybe history shows about cannons.
At the moment, a kid comes hurtling out of University with their degree in media and maybe something like History. They want to go in to TV. They get picked up for pennies as it's their first job, given a desk and told they're in the TV development dept. Now all they have to do is come up with a LOT of ideas to fit show slots that channels are looking to fill. Issue is, no one wants to watch 22 seasons of Cash In The Attic any more because that's old hat. But what about Gold In The Cellar? Same show idea, just that rather than auction crud from around the house you have to fake sell it to some schmuck in another country who thinks it's worth more than it is. I don't know, but you get the idea. Same shows, different twists. They often use newspaper articles and magazine current affairs to come up with documentaries that people may like. Failed HS2 in the UK is definitely going to get a show. Originally it would have been on how amazing the rail links were going to be from the south to north of the UK, now it'll be a show about how farcical it was and was it all tax dodges and fraud with power people greasing their palms with government contract cash?
How do you get the likes of ChatGPT to put out a plethora of ideas like that, based on the past 6 months of news? If you can create a front end to select type of show (game, documentary, science or nature, etc) with a basic input to create a bunch of outputs, you're about to make a lot of money because we'd buy it tomorrow and fire a lot of people. (The latter part of this line is a joke, just in case people do not see humour in my writing style!)
Don't get me wrong, I don't want to fire lots of people, but I do want the company I work for you succeed and be ahead of the game.
The use of AI to generate show ideas and scripts was at the very heart of the recently concluded writers strike.
The problem is, the way Hollywood develops shows, the use of AI would give them pretty much exactly what they want. As you say, LLMs regurgitate concepts from a vast amount of source material and frankly that’s all executives are really looking for - the same idea in a new package or with just a slightly different twist. That kind of writing is what AI excels at.
As someone who doesn’t really work in the industry anymore but still produces one-man media, this is all thrilling. For the entire economic structure of the entertainment industry, not so much.
I think the future is bespoke media. Each consumer can ask an AI for a specific type of story and get it.
The market for swapping out a name for someone else in a night time tale such as Hansel and Gretel with your own kids is well over done. There are 'skills' on Alexa to do this.
Having it invent a whole story where your kid is the lead character, age appropriate, boundaries set and immerse them in fantasy is going to be great.
The issue with LLMs is always going to be a cross between prompting and source material for it to work with. It's definitely going to be fun.
I just wanted to say that I regularly use chatgpt in my mathematical research. I don’t even use gpt 4, and yet I am still able to define novel functions and definitions that don’t exist in literature and it is pretty clear that with some patience that I can get it to actually understand the real essence of my definition or function. It does not do a good job at first but each step of the way I ask it to describe what it does and to compute several examples to prove it understands. When it gets some stuff wrong, I tell it so, provide the correct values and do it again, and it will usually show some improvement. I recently did this today actually and it was pretty spooky when it actually managed to describe the behavior of my function in totally different words than what I used in my original prompt despite it initially trying to tell me that my function is equivalent to some other well known function (it’s not, but that function is highly related, another evidence that it’s not just “regurgitation”).
Also for what it’s worth, I mostly ask it questions like “here’s a function I thought of: function definition. Does it have a name?” And also reasoning through things, rubber duck style.
If ideas people no longer have ideas and need AI to come up with things for them, maybe they should just be fired and replaced.
Removing humans from creative endeavours is abhorrent and you, and the industry as a whole, should be ashamed of yourselves, not that you're capable of such as you chase that very last piece of "margin" into the ever deeper hole in the ground that is "eternal growth".
I just want to say that this is a much harsher and personal take than the discourse on Tildes encourages. Consider that OP is just a person trying to do their job. Many of us are in industries where we can see decisions made that we disagree with, but guess what, rent/mortgage had to be paid every month. Skills, experience, and personal connections mean that staying in the industry is the most time and quality of life efficient decision. Try to consider the person on the other side of the keyboard.
To your point as a whole, I agree that the media I want to see produced is made by creatives. For every one or two of us who will spend time or money on artistic output there are tens or hundred who want another How I Met Your Mother, Big Bang Theory, or something like that. That's what pays the bills and hopefully brings in a profit to produce more niche products. It's why Porche makes Cayennes and Macans, it funds the 911s and Caymans.
I'm sorry you think that was harsh, but I held back there. What they're talking about is destroying an entire industry for more profit, despite profits and growth being at all time highs.
Don't rationalise greed, please. Think of the people that are actually trying to live.
I just wanted to come back round to this because I think we crossed paths on what I am trying to achieve and what you believe the industry is planning. Hopefully my update of tools and utilities of interest to TV Production might show what I was trying to ask for.
https://www.ibisworld.com/united-kingdom/market-research-reports/television-programme-production-industry/
UK Television production is mostly small and independent creatives. A quote:
What I'm asking for is simply AI tools to help in the industry, not replace people. Small independent companies and groups struggle as the only people with clout in the UK TV market is the broadcasters, they hold all the cards. They have slashed production budgets, moved payments to be until after delivery, which means small companies have to foot the whole production bill before seeing a return. There are fewer time slots and a lot of stations are in repeat mode. Many production companies have gone under or are on the cusp of bankruptcy.
Broadcasters in the UK are finding that many big brand companies are diversifying their advertising budget, so are less willing to pay for air time. It's a domino effect.
If you are in the television or film business, good luck to you; it is cut-throat out there, and it's weighted against small companies succeeding. Right now we're just keeping the bank off of our backs, but the end of the year isn't looking great for many green-lit shows.
Interesting comment. Development staff for TV is one of the largest blocks of turn over. It's almost like a position you need to go through to step up. Generally, you step from development to a show researcher on a production that's getting off the ground.
AI isn't going to write whole shows, not the sorts of things we make. However, it could give us some good 'bones' and allow dev staff to flesh them out. I joked in another comment about firing the Dev staff, it wouldn't happen. You need the human to really evaluate if a show has legs. It's the generation of the ideas that would be useful.
Let's go the other way: AI was asked to design a robot that could get about and it designed some weird squishy thing, but it worked. We expect legs, AI doesn't.
Generating TV show ideas could throw out some ideas that we generally over look and a Dev person would go "Oooh, this could work" and then a team would push it forward.
TV production is generally small independent companies trying to find something to get to air and not so much about profit. Profit's nice, but these are private companies, not shareholder driven. The group of companies I work for want some recognition for making interesting TV and to put food on the table.