Hey folks

I've got a couple of months to put together an overview for tools that a company could use as part of television production and I'm hoping for your input.

It goes without saying that everyone in the tech world is pushing ai heavily. Having been in IT for almost 3 decades I know what to watch, look at, out for, etc. AI is still very much regurgitation of its input but the input is vast. What I have right now is some bare bones of what I want to throw around for insight and discussion for what would help people in TV production tool wise.

For those that do not know how TV production works it's a simple idea: you generate a huge raft of ideas for shows, absolute basic outline of what the show would be about and put that in to a paper. You then sit around in your research/Dev dept and pitch to each other and the ones that people go "yeah, that could make a good show" get some extra meat added. Those ideas get pitched to dept heads who then take the best ones to channel/broadcasters execs and see if any get hooked at all. If they do, they get given some development funding to put together a taster/pilot/video version with the funding they have. This means shot on camera, run through an edit for cutting, audio, graphics, etc, still in its infancy and development state. This video and a bigger padded Treatment (documented idea with its bones, flesh and now make-up added) goes back to the broadcaster and you wait for feedback. If you get lucky you get a greenlight and order for X amount of shows and then you have a production. The production is taking the idea to it's full potential, shooting it, audio and music, graphics, the works and that's what you see on TV.

I'm after working out what tools AI offers today that would help them with this process. Right now, ChatGPT v4 will generate some great treatment ideas for shows, except I would imagine these shows already exist or have been tried to channel/broadcaster before? AI is regurgitation and not thoughtful to its own ideas and imagination. I suppose with great prompts it could generate great output.

Okay, that's the process and I'm rambling. Right now I have a short list of LLMs such as ChatGPT and Bard types that will help with the idea stage for researchers. I could use some decent links for prompters to help the research know how to ask AI for what they want out of it.

When it comes to generative AI for graphics I only have experience with txt2img using the likes of DALLE and Midjourney, along with some inpainting for changing images with lies, I mean, graphics (insert plane on fire, etc).

Does anyone have any other ideas and tools which would help production or useful things I can look at and research myself to see how they could be helpful? Auto audio generation? Graphic building that takes less time? Think of those great show intros for the likes of Game of Thrones, can that be done using AI yet or are we no where near that level for AI? Even basic video edits, where are we for AI help? Can we feed it some clips and have it autostitch based on an input document? If so, what tools should I be looking at and researching?

I'm asking here before I plop search terms in Google and Bing and then get swamped with whichever has paid the most or played the SEO game to be top of the pages. Asking for real human input is definitely better than asking AI which may actually be the whole point of my talk when it happens.

Thanks for listening and any help/pointers/sites you can give.

UPDATE:
I went off and did some research. Enjoy these if you want. I had issues linking so if a mod wants to go ahead and do that, feel free:

Pre-Production:

Treatment idea generation

Generating a great idea is usually through using knowledge and research, but these days you can literally ask an AI engine to come up with a show idea. Here I will list some good AIs that use a very large language model (LLM) to come up with ideas:

ChatGPT4 from OpenAI

ChatGPT is the best known AI out there, but essentially it's the AI that everyone uses. What's different is the data that is fed to it. ChatGPT from OpenAI has a lot of knowledge, however, it's generally backdated information and not up to the minute.

You.com

Built on ChatGPT4 AI. Data fed in more up to date as it's based around a search engine. Due to the plethora of sources being fed to the You.com Chat bot, you may find some more interesting results and ideas.

Bing.com - Chat

Directly leverages the latest version of ChatGPT4 from OpenAI but uses additional media from Microsoft sources. Responses are more natural due to the Turing Natural Language.

Copy.ai

A fun LLM designed for advertising agencies and the alike. The difference here is you can upload a back-catalogue of your own data for it to analyse to take on your brand voice, mix up your ideas and generally become one of the family.

Prompting

Just from picking one of the four AIs listed above, you can straight out ask for a basic show idea. All of them came back with interesting ideas from the prompt of "Generate me a great show idea for a television production treatment. The show should be a documentary for daytime viewing."

Prompting is the hardest part of any AI interaction, the results can wildly vary depending on what and how you ask. Due to this, there's a new type of website to help with prompting:

https://promptperfect.jina.ai/prompts

Using the line from above about generating a great show idea, promptperfect injects a lot more information into the prompt before running: "Please create a compelling show idea for a daytime documentary television production. The show should be engaging and informative, catering to a broad daytime audience. It should focus on a specific topic or theme that is both educational and entertaining. The documentary should be well-researched and provide in-depth information on the chosen topic, presenting it in a visually appealing and accessible manner. The show should aim to captivate viewers and leave them with a better understanding and appreciation of the subject matter. Additionally, please provide a brief outline of the structure and format of the documentary, including the number of episodes, approximate runtime, and any unique features or storytelling techniques that will make the show stand out." The quality of the Treatment created will be far superior to the initial request.

https://webutility.io/

An interesting take on generation of prompting. It breaks down the prompts to dropdown boxes with key words such as create, design, analyse along with the focus type. This forces the ai to create some more complex and well thought out documentation for a treatment idea with explanation of how it got to where it did.

AIs to help with show production

Location finding/scouting

With the latest AI image searching features, you can now upload an image and get a "related" search. Using this technology, you could, for example, look for English Country Gardens that you would like to film out of. Uploading this image would give you a list of locations, similar places and website associated with the image:

On each of the following sites, in the search bar, click the Image Icon to upload the image:

https://www.bing.com/images/
https://images.google.com/

Scheduling (not specifically AI)

Scheduling shoots should be simple. We've seen all the fun from an Excel spreadsheet that's laid out like a calendar, through to the most complex diary entries in a shared Google calendar. We already have the tools for this in Microsoft Office:

Microsoft Bookings: This is a great tool for scheduling a diary of a single person or a whole team. It allows to have a Web Page where people can book in time for appointments, whether virtual or in person. Perfect for a researcher trying to book interviews with a host. The AI lies in the ability to cross search a calendar and pick associated times available.

Microsoft Planner: A tool for project and time management. Breakdown the show in to buckets (categories) and assign out tasks to people and teams, due by dates or exact dates, etc. You can even keep all of the documents in the plan.

Microsoft Shifts: Team management for your production using Shifts. This allows you to schedule team members in Teams, allowing them to clock in and out, as well as specifying when they need to be available.

The three tools all work with the Outlook Calendars so each person knows what their plans are well in advance.

Post-Production

This is the one most people are interested in for AI at this time. The tools used for image generation, manipulation, etc. The market is currently being flooded with tools and not all of them are equal, but here's a few ones to watch and use.

Auto-Clipping & Social Platform

OpusClip, using the power of OpenAI, can take a long video and create 10 viral clips from it at the click of a button. The AI behind it analyses the video, looks for compelling sections and highlights, then seamlessly rearranges in to short videos. This tool will be great for generating short promotional videos of long form shows, documentaries, etc.

Descript is a great tool that can take a video, give you a transcription, then you can edit the transcript, where it then edits the video to match. You can remove words, create studio quality audio from a standard mic, remove common error words such as um, and er, etc. One of the bigger cool things it can do is voice mimic using AI. You read it a line and then you can type out a whole transcript and it'll narrate it in your voice and allow export.

AI Generative

Moving on to the more scary AI platforms, we have completely generative AI. This is where AI generates absolutely everything including the "avatar" of the human speaking. It's getting so real, you could probably make a documentary using nothing buy AI voice for narration and even have an interview with the AI Avatar.

Video Generation

Synthesia has 120+ voices, over 140 AI Avatars and an editing tool that is extremely easy to use. Mostly aimed at Sales, Training and Marketing Teams, but could easily be used to create development tasters and cuts by mixing in the AI with real video. An example video here.

AI Studios from DeepBrain is another tool, similar to Synthesia. The avatars are based on real humans being recorded but then converted in to AI models. Again, lots of models, full text to video.

Spline AI is a 3D modelling engine that will generate models from text prompts. It's still in Alpha stages but specifying something like "A cube", "rounded corners", "floating", "spinning slowly" will generate exactly that. This tool is aimed at animators but is likely where CGI effects will head.

Still Image Generation

Txt-2-img is amazing and growing at an ever rapid pace. With the wealth of images out there to learn from, the styles, etc, it's no wonder it's doing great. However, it's far from perfect, even now. You'll often find that it adds limbs or fingers to models, shadows completely wrong, crazy styles that are not what you asked for, and that's just the start of the issues with it. However, when it gets it right, it's amazing.

DALL·E3 from OpenAI is the current leader in image generation. If you need to whiz up a picture of a steam train, crossing a suspension bridge at sunset with a woodland in the background, this is the tool of choice.

Bing Image Creator is probably the second biggest right now and has very good accuracy of text to image due to the absolutely huge database of images with high detail being fed to it by Microsoft. It's also free.

I'm not going to list too many more as a lot of them stray off in to fantasy land, being trained on Anime, comics, however, DeepAI definitely deserves a mention. These are the folks behind a lot of the viral videos where you can scan your face and and speak a few lines, then it adds you to a section of a movie as a "Deep Fake". You can have it chat, generate images and even AI edit images with txt-2-img.

Video Edit Tools

The biggest AI enhancers right now are tools that help in the Edit at a professional level.

Topaz Video AI is one of the leading tools in Post production. Upscale footage from SD to 8K and HD to 16K. Full denoise, sharpening, 16x slow down with AI interpolation including building new frames. Corrects people and faces. AI Stabilized video to stop bounce and tracking issues. This is a complete Post Swiss-army knife.

Adobe After Effects which everyone knows. The Adobe AI, called Sensei, is under constant development. Easy animations of text and logos via text to video, rotoscoping video objects to remove the background of a person and replace, or removal of all objects in a scene using AI generative filling is all extremely easy.

Adode Premiere deserves a mention, but again, this down to Sensei. The current AI tools coming in to the suite are things such as Auto Rough Cut using the transcript to generate the video, full auto transcription with subtitle creation for multiple languages. Auto Colour will fix most colour issues using AI to save time in grading. AI Morph Cut adds visual continuity to cut transitions, remix for music matching with visuals, and Auto Ducking – popping dialogue over background audio to make sure you can hear voices correctly.

ColourLab AI is a new kind of grading tool where you no longer need to spend time with an artist grading every scene. The tool is a plugin to Davinci or Premiere and will do cool things such as film grain matching or stock emulation, which allows you to match any scenes together to look exactly the same. Take a video of a pigeon flying over a statue in London, and have it grade using a still frame from The Martian to get those awesome colours automatically, for the whole scene.

Audio/Narrator/Voice Over

The final piece is the new voiceover AI generation. No longer do we need voice over artists. In fact, Hollywood thinks the same and fired the whole staff of Snow White and replaced the Dwarfs with CGI and AI voices.

Altered Studio can change any persons voice, in any way you wish. Record your voice for narration and then adjust it to be male, female, Elvern, whatever. It also does full transcription and allows for VO with text-to-speech using AI voices.

A quick shout out to a member of Tildes who wants to remain anonymous for some of the cool links that they sent over - much appreciated.

I'm very interested in the inner-workings of a writer's room. I have seen glimpses in "making of" documentaries, but I don't feel that they show much of the actual process.

I'd love to see more about how the sausage is made. Bonus points if it's a beloved, popular tv show or movie. I feel like I would be most interested in seeing the best of the best at work.