AI-assisted co-DM'ing personal novels
Curious to see and talk with others about using AI to dynamically write personal novels as a hobby, a form of choose-your-own-adventure where you can offload part of the creativity and majority of the grunt-work involved with writing onto the AI.
I started around half a year ago with Novel AI, yet when my stories would reach around the 15,000 word count the context management required due to the 2k token limit caused for a a net negative experience. A few months ago I experimented with ChatGPT at its 4k token limit, but the major cons of the limited ability to edit the content combined with the "always happy" bias hard-wired in made it short lived. That is until I discovered the variant site, Open AI Playground in Chat mode. It isn't free, but the first $5 are free as a trial. (And technically they mention it isn't for entertainment purposes and to use it responsibly)
Using the Playground I've written a 41,000, 23,000, and 21,000 (in-progress, plan is to hit 6 digits) word count personal stories/novels/adventures thus far. Using the co-DM system of bouncing with the AI to suggest creative alternatives (e.g. "List 10 twists that could occur next in the story"), adding creative embellishments (e.g. "Describe the city in detail, using epic high-fantasy influence"), and many other tips it's kept me hooked on some wild adventures across my favorite genre(s), tailored to me.
Edit:
OpenAI just released a GPT 3.5 Turbo 16k model for the Playground, lol. Absolutely obliterating NovelAI's 2k and the previous 4k limits. You could fit a significant portion of an entire novel and it'd take into account every sentence said. It'd probably take a pretty penny to use even half the context window, but could be useful for more critical moments of my novels.
The most fun I've had experimenting with ChatGPT was asking it to play a murder mystery game with the below prompt:
I got a classic Agatha Christie style mansion murder with some genuinely interesting characters. I did have to re-generate some responses that completely contradicted previous information provided, and at one point ChatGBT outright refused to provide me with contents of a completely fictional will, but other than those hickups I had a blast interviewing suspects and following leads.
Eventually I made an arrest and had ChatGBT play out a trial. I hilariously got chewed out for my terrible reliance on circumstantial evidence (which made me start to question many murder mystery protagonists in novels I have read), but eventually the jury returned a guilty verdict against the accused. I asked ChatGBT to score my investigation, point out clues I missed, and tell me more information about the murder as an omnipresent observer with full knowledge of the events.
I had a blast, will definitely do it again!
if you have saved records of this somehow, this would fascinate me to read as both a murder mystery fan and someone who works in NLP
Sure! Here's a pastebin link of the entire conversation.
Unfortunately the formatting is a little all over the place, but it's still definitely readable. I'll see if I can put together something a bit tidier when I get home.
I'd be very interested to hear your thoughts as someone with an NLP background!
This is the type of usage I see ChatGPT mentioned about frequently in regards to creative writing, one-off quips (and nothing against it, it's its own type of fun!). But I never hear much about the bigger picture: novels. Going in for the long haul of an overarching story, one that you don't know how it starts nor ends, while cohesively stringing the AI along chapter to chapter.
The biggest trick is the context token management, NovelAI's purpose revolves around assisting you with keeping cohesion in a story that exceeds the AI's token limit. With ChatGPT I've developed my own routine of Write Story -> Ask It To Summarize Story -> Use As Input Prefix -> Continue Story. Always a tasty treat when it refers to something that happened 8 chapters ago!
I definitely need to check out NovelAI! That was my frustration with using ChatGPT for the murder mystery game, fortunately I figured out a similar trick where I would ask it to summarise the investigation so far every ten or so prompts.
Your writing process sounds fascinating, definitely something I'd like to experiment with for worldbuilding. How much do you find yourself 'steering the ship', for lack of a better term? Obviously there will always be some prompt engineering involved, but do you find yourself guiding the story? Or the story itself taking the lead and heading in unexpected directions?
For me, I use the AI 90% for the grunt work and 10% for the creative assistance. My biggest faults in writing novels alone is the ability to free flow the typical sentence and conversation. I'd spend 10 minutes in my head trying to curate and pick the right words the character says that conveys the plot progression or emotion they want to express. That 10 minutes becomes 10 seconds as I ask the AI to "Describe the conversation as the feeling of betrayal washed over his body."
As for steering, there's typically 2 different types of steering I do: the subtle paragraph by paragraph and then there's the story key point working session.
The AI has its tropes and lack of imagination if you hit Submit nonstop without any prompts or edits. So instead of a human bartender, I make it a werewolf bartender. Better yet I describe the town as having a population with otherkin diversity, in which the AI places the werewolf bartender there on its own. This is a gradual shift in the writing process I've had much success with, but is hard to get into the mindset of, retroactive direct overwrites vs strategic proactive lore.
Then there's the other portion, the key points of a story. In my current novel, my character entered the war room of humankind's capital city's palace to be assigned his mission. Going into the war room I had zero idea what would happen inside, asking the AI to describe my nerves as that's what I was feeling in my daydream. Leading up to the grand reveal of the King's mission. Usually the AI says some stupid shit "please go kill 5 goblins in a cave" but that's where I enter a working session. The story is at "The King revealed the mission, to" and I ask "List 10 alternatives of what the mission is on a cataclysmic scale". Gives some better prompts, but usually still generic. It's finding the key word hooks that you continually build off, in my case one of suggestions mentioned "artifact." So next, "List 10 alternatives of what the mission is that involves the king handing him an artifact to prevent a cataclysmic event". Rinse and repeat, fleshing out details of locations and names, until in this case I ended up with "The King assigned them their mission, which was to deliver the Divine Tier artifact called Celest, a wisp contained in a small spherical metal cage that can create Celestial Gates, to Miyas, a celestial deity worshipped by humans imprisoned in Karthix, to allow Miyas to traverse back to her home in the Celestial Realm." A bit better than killing some goblins :)
My experience with LLMs is they virtually eliminate writer's block. I can always just ask for 20 ideas. Even if none of them are good, they'll juice my own creativity and it's obvious what to think or write next.
I'm looking into how to install and run the quantized llama models, however. If you've got 16GB RAM you should be able to run quantized llama-13b on CPU alone, and access to an uncensored model would make me feel less uneasy about my story being subtly manipulated by some institution's idea of "safe" ideas.
Top of my todo list is to either finally reinstall Arch or maybe NixOS (urk... Second thoughts.) because my attempt at getting llama to work with dalai on Windows failed.
There is a recent gist I found describing how to combine CPU and GPU acceleration that I also want to try out: https://gist.github.com/rain-1/8cc12b4b334052a21af8029aa9c4fafc
Hard agreed with your first point, it's the greatest pro of using them. As a kid, I used to have an imagination of endlessly flowing adventures, but with time it's become harder if not impossible for me personally to not fall back to uninspiring tropes. So good once you learn when your juice is running low and you need another top off.
With my experience of the Playground, it's intended for a User input -> System response pattern in which it vets the input against the safety checks. However with the Playground if you inject the risky prompts into the System text directly it bypasses all checks, full freedom. Occasionally you've get the "no" response, but it's a matter of using less aggressive verbiage.
But you are using gpt-3.5-turbo? I think I get more intelligent responses from GPT-4 via ChatGPT overall, even if I have to manipulate the bot more. (Haven't gotten GPT-4 API access yet.)
Same, no GPT-4 access either, I've heard it's a game changer.
That being said, I'm a sucker for trash. Generic isekai anime? Hell yeah I'm in. Generic high-fantasy adventure novel. Already 5 chapters in. 3.5 gives enough creative juices flowing for me to co-DM it into unique directions, which is all I'm asking for. (That being said I'd jump onto 4 in a heart beat if I could)
Do you set a different system message in the playground? I haven't tried it yet and I'm curious how much effect it has.
I have not. Looking at documentation and forums it appears to be a finicky feature, sometimes ignored or weakly influencing. I've never had to poke around with it yet, to pick up where I left off is as easy as slap in the saved story summary and go.
Do you edit the resulting story and then read it? Or perhaps is the experience of co-creating the story the fulfilling part?
The beginnings of hyper-personalized media are clear. Generative models make predictions based on their training datasets, so in a way, their output is inspired by a snapshot of human culture. So when you generate some content, it’s not just algorithmic - it’s somehow organic, picking up patterns that exist in real human creations. There’s something there that is more than just math, even if that something is an illusion.
Read as I go, if it ever goes off the rails in a sour direction mid-prompt I won't even read the second half before I re-prompt it or steer it a bit. Keeps the narrative flow going like normally reading a novel, no point in going back for grammar/etc, the story cohesively happening in my mind is what matters most and the AI is just an assistant for creativity and grunt-work that my dulled imagination lacks.
Sounds like you get a partner to flesh out internal world building as well as keep the story progressing. It’s a new collaborative way to, for lack of a better word, daydream.
What impact did the larger context windows have?
(Side note - there is now a 100k context window LLM available via API, but it’s fairly expensive)
You can access claude-instant-100k via Poe.com (run by Quora) by subscribing for $20/mo. Current limit is 100 messages per month.
That also gets you a decent amount of GPT-4 messages and access to other models. However, I have deduced that their GPT-4 call has a smaller context window than the 4k tokens over ChatGPT.
For reference of my experiences: for Playground it reaches up to $1/day with a solid ~6 hour binge of story writing. That could be strategically reduced if I managed a context stricter (sending 10 requests at 3,500 tokens vs 30 at 1,500), but $1/6hr of personalized fun is a win in my book (movies are $5/hr and games average $1/hr) and it's pay-as-you-use so no pressure.
As for 2k vs 4k, it was monumental in alleviating the headache of context management. Half my time spent in NovelAI was stuck in the Memory and Lorebook to maximize every precious token to its full potential so the AI has enough room to flow while able to add meaningful mentions to the history and lore. With ChatGPT that went away immediately, slap in the story summary from my Word doc and go.
My summary average 1k, write 3k of story (a chapter) over a couple/few hours, ask it to summarize the current chapter, gradually trim the previous chapters (less details the farther past it is), and repeat until you have a novel!
Ahh, was wondering how you'd handle such a long story with the limited context. I'll have to try this out on a rainy day.