Thoughts on ProWritingAid
Howdy hey folks, I've recently been trying out ProWritingAid (for the unfamiliar: a grammar/spell checker tool) specifically the premium version with the expanded tool set. And now I want to step onto the internet soapbox and talk about it. It's been.
Okay.
To preface, I've been writing (casually) for 'bout a decade, mainly short creative fiction. (And a few novel attempts. All of which are incomplete but I'm glad I did them) Throughout my time I've gone through a few tools, text editors and what-have-you-nots. With my ever so gleaming credentials established, let's get into the ramble.
Right out of the gate, automated grammar checkers and creative writing have a rather fun relationship. Half the suggestions are useful and the other half are useless. (This ratio can also tip forward and backward). They'll catch syntax errors, spelling mistakes, missing words or punctuation, all good things to fix.
It'll also flag intentional word choice, sentence structure and other creative decisions. Sometimes this can help but more often than not it'll be sucking the You out of your own words.
ProWritingAid (PWA) tries to sidestep this particular pitfall with Style Guides where it'll be more or less rigorous depending on the selected 'genre'. It's a mixed success. This flaw I don't think will ever be truly fixable given the inherent separation between Author and Tool. So we'll have to make do with clicking "ignore."
Now PWA does a bit more than just grammar check. During my time with it, I've currently used two versions. PWA Everywhere, and PWA Desktop. Everywhere is meant to integrate with your text editing software while Desktop is a contained application. They have similar feature-sets, but not identical. Specifically, Desktop has the Word Explorer feature: a tool that if you highlight a word it'll show some synonyms or you can dig deeper with alliteration, cliches, anagrams, rhymes, reverse dictionary and more. Pretty nifty. PWA Everywhere best to my knowledge and searching does not have this feature- which is disappointing.
Especially since everything else Desktop does, Everywhere does better. The UI alone is far more functional, without clipping or cramping. There's the convenience of direct integration. Some features like Single Chapter Critique (which I'll get into later, trust me) also blank screened in Desktop while working fine in Everywhere. Grand.
Besides the Word Explorer, PWA also gives you AI "Sparks" and Rephrases. I'll be entirely honest, I have these turned off (Which I am glad I was able to do). I don't have much to say here besides I like getting into the creative word weeds myself.
Alrighty, that then leaves me with two more things to discuss: Writing Reports and the Critique features.
Okay. The writing reports are useful. Able to be granular or extensive. They scan every selected element in the text and format the results into a nifty report (or in some modes, direct text highlighting) Having all that data visualized with tables, graphs and bars oh my, (with the occasional cross-work comparison) is a great look-at. Grammar-wise it'll run into the problems mentioned above, but otherwise, this has been the feature I've liked the most.
Finally I can get into the whole thing that inspired me to write this post. The Critique suite. Ohohoho, I have some thoughts about these. Human proofreaders are irreplaceable, just want to toss that out there (PWA also keeps that disclaimer in its header). My friends will never be escaping the random PDFs sent for their lovely review. I am ultimately writing for a human audience afterall. That in mind, I have run into a hilarious problem with the Single Chapter Critique.
Apparently I write too good to get use from it. Truly I am suffering here. In complete honesty, the actual point I'm trying to make is the AI is a kiss-ass sycophant. I fed five of my short stories from across the years into it, just to see what it'd say. It cannot be negative. In each and every one I was praised about various element of the stories. Glowing and gushing, could say no ill.
This is pretty useless. Sure it has the "Potential Improvements" section but it's... eh. In the name of curious study, I am having my non-writer friend compose a piece for me to feed to the machine spirit later. (I also only get three uses a day, compared to the unlimited reports with their nitty gritty)
Now, could this simulated praise be a sign I'm a genuinely good writer? Well I don't need the AI for that- I have friends zip-tied to chairs to feed my ego. (I forever cherish one of my close writing friends telling me: "You have a voice of a fantasy writer from the 70s with a thick series full of wondererous imagination written by a twice divorce middle aged man who is disgruntled with reality. It was never exactly reprinted as it was unknown, but the aging, withered pages hold such a gorgeous narrative that it sticks with you for the rest of your life.")
Back to the AI: Their shining critique falls apart when I look at the story myself and can point to several areas for improvement/refinement with a cursory reading. (Thank you creator's curse, you're my true reliable critic.)
Woe to me, I cannot escape personal proofreading. (Real talk: the hope was have it be able to do the cursory stuff so I could focus on the creative viscera. That's half the fun after all—)
There is two other Critique features, Full Manuscript Analysis and Virtual Beta Reader. I have used neither of these as I do not have any large manuscripts to toss into the jaws. To ensure jolly feelings, it's also a credit based system. So let's talk money.
Scrivener, a writing workhorse that even after years of using I still find new features and has long cemented itself as my text editor of choice, was $45 for a lifetime license. Fantastic software, it has earned its reputation.
ProWritingAid, a grammar and spellchecker was $115 (discounted price) for a year subscription. (Can I mention how idiosyncratic their tier system is? Free, Premium, Premium Pro? Why??? Just name it Free, Pro, Premium. Don't stack luxury words.) For $115, I get several features I don't even use, or aren't very useful. Oh, a discount for the aforementioned analysis credits. ($25 for 1, $70 for 3, $175 for 10. Full priced it's $50, $150, $500 respectively. Spend this money on an actual person please)
Now what's worst off is I wasn't even the one to spend the $115. That was someone else wanting to support me and my writing; an act I am quite grateful for and the meaning behind it. I feel bad complaining. I have hopes for PWA. Something that can act as a quick look proofreader would be wonderful. But perhaps I'm just asking for too much from what is again, a grammar and spellchecker.
So far, I don't know yet. I don't know if I'd call it good or bad. As I started with: it's okay?
Maybe I'll do a retrospective after a while once I've utilized it longer. Maybe features will be better fine tuned in the future.
And that leads me here. What have been y'all's experience with it, if any? Searching online has been miserable; I'd like to hear from other people.
[As a footnote, PWA was not used when writing this. Kinda forgot that I never set it up for browser. Tallyho]
First proper post on Tildes, didn't quite expect it to be this but it's been on the brain. Reading it aloud, seems coherent enough. Unsure how to tag it, but did my best.
I'm in a similarly-ish boat, but I'm writing a twine game for fun, and I'm using Claude instead of a proper/dedicated tool. I can share some insights that maybe you'll find useful:
(this point is a bit off topic but bear with me) Just to give a heads up to actually just enable it where privacy doesn't matter. For example if you only use it for creative writing and just use Scrivener for that, then enable it only in there.
I raise this point because in order for a software like that to work, it must be sending home (servers) whatever you're writing in order to process it, and give you the AI suggestions. So if you enable it in a browser, where you'll likely write emails, chat messages, forum posts, or even maybe passwords, etc... They'll get to see those.
Yuuup, sounds about right. LLM's are by default sycophants, at least the commercial ones. They're very afraid to disagree with the user, to the point that once upon a time you could convince them that 1 + 2 = 5 (Nowadays, you can't really do that with the top of the line models, but they're still sycophants).
I don't know what PWA does behind the scenes but if you feel that they're too soft and don't really critique, either of these two are happening:
A - Behind the scenes, they tell the AI that the user wrote it
There's a way to circumvent the kiss-ass nature with normal chats, but with several caveats (more on that in point B down below).
You would need to use an AI website like Mistral, Claude, Gemini or chatGPT. If you don't want to spend money, go with Mistral or gemini at aistudio.google.com. Then, say something along the lines "Can you critique this chapter? here: """ chapter """"
Did you catch it? Basically, don't ever mention that you wrote it. Always treat it in a third person, as neutral as possible. Don't hint that it reads weird to you, don't hint that it looks good, etc.
LLM's, the way they work, if you hint that it's bad, it will be influenced to "think" that it's bad. If you hint that there's a problem, they'll be influenced to find one, even if it doesn't exist.
Top of the line models nowadays don't suffer with these problems as much with logical problems, but creative ones are another story. Which leads me to B.
B - You're actually good at writing, and you passed a level that AI's simply aren't useful
LLM's for creative tasks generally just suck. They suck. They're terrible. And I say this while using Claude Sonnet 3.7, which I consider the best one for creative tasks, and I think that it's terrible.
The difference is that I am even worse at writing. I can think of the story structure and what to write, but I'm a guy of very few words, my game would read like an informational bulletin if I was doing this alone. Claude is incredibly useful to help me rewrite and make things more flowery and more immersive, but I believe that at best, we're creating an amateurish writing. This is not me being humble or anything like that, just stating a fact.
Tools like these are like... Someone walking on a very thin rope. Thin rope being being the LLM models, which simply aren't great for creative tasks as I mentioned. But there are other issues.
For starters, I think that for creative tasks context is even more important than logical tasks. The LLM needs to know who the characters are, what they did, what you wrote in previous chapters, etc. But here's the thing: the more context you have, the more expensive using the LLM's become.
And here's the another thing: PWA will pay OpenAI/Anthropic/Google or whoever every time you use any of their features that use AI. So, PWA being a business and me trusting that they want to make money, they:
So... I guess what I'm saying here is, I wouldn't hold my breath for those AI features to become great or to change your mind anytime soon. I can see that happening if one day someone manages to successfully make an LLM for creative writing and also be affordable, but for now, it is what it is.
Anyway, again, I don't know if these thoughts will be useful for you but here they go anyway
Thanks for this write up! To cover the first point, I do have Everywhere enabled solely on Scrivener. (PWA’s shiny new product: Scrivenerthere) They claim they don’t use your writing for training, so I have their pinky promise.
As for LLMs and creative tasks, they suck for the reasons you listed. I’ve tried the method you’ve explained above before (though with clarification I wrote it, something I’ll keep in mind not to do if I try again later)
When I look online it’s praise praise and more praise with the sparse drops of “hey this actually kinda sucks” buried in dead threads. Articles with that Top Ten Tips! How PWA will make You a Better Writer! emblazoned straight in the hyperlink. YouTube videos on how cool the AI tools are, how those Sparks will streamline the process and let you express yourself in your truest form-
Maddening stuff; it’s why I came here.
I appreciate your response mate, I hope your twine game creation goes well.
Interesting insight :)
I have been using LanguageTool for a while now and recently have started to self-host the server in a docker container which you can as the base server is open source. This has the added benefit of not having the AI features available and more importantly keeping what I write locally. The latter is important on my work laptop for obvious reasons.
The use case for me is mostly a slightly more advanced spell check. I honestly don't need my writing to be perfect. In most case I want it to be readable and mostly correct. For things like documentation, blog posts, etc, I do hold myself to a higher standard but there I proofread it a few times more and let someone else read it.
Which also comes back to what you wrote as well
I have experimented with relying more on tools like this. Even having back and forth refinement sessions with various LLMs. Resulting in a very neat and polished text... lacking any personality and certainly not something I'd ever write myself.
Thank you, I’ll look into this. I have a home server I could chuck this into. Ultimately what I do want is an advanced spell checker. Most everything else I can do myself or get the friend poking stick.
When I write I tend to think in sentences so I have an unfortunate habit of skipping words thinking I’ve written them. Normal spell checkers don’t catch this. I’m human, I like to indulge convenience every so often. Having to fine comb a story for missing articles is something I don’t mind tossing to the machine.
I use such tools largely because English is not my first language. My only observation is specific about creative writing. I don't use any tool when I am creating something. For the first version of something I don't even use dumb old-school features highlighting wrong words. I find that essential to achieve a state of flow, build a tone, and understand the core of the piece that must not change even if it's considered weird, ugly or kinda wrong. After that I can use every tool at my disposal.
I am fortunate to know a few people that will read my stuff and give me notes. In my experience AI feedback (as in a complete textual review of my story) is not something I can really on. Maybe it's great for other kinds of text, but I wouldn't know.
Uh... why do you send PDFs to people? PDF is a nightmare :P. If the idea is to get comments I would share it on Google Drive or similar.
I also do similar with first drafts. I disable all tools and distractions and focus on just writing. Afterwards once I’m in revision mode with a second draft do I begin to correct and edit.
To answer why PDFs, I can export them directly from scrivener and send the compiled file to discord. I occasionally use other methods but PDFs remain quick and none of my friends have complained thus far
I have been using ProWritingAid recently to edit, clean up and tighten a novel before it goes to test readers. About three hours of work every day for the past couple of weeks. I find it useful. Most of the things that it suggests are irrelevant, many are debatable, and some are downright ungrammatical, but it does also flag things that could make my writing better, or at least push me to pay attention to something that I otherwise wouldn't have. After living with a book for well over a year, you just grow blind.
I find PWA better at pointing at potential issues on the sentence level than commenting on the work within a wider context. It's also better at flagging issues than offering solutions to them. But that's fine, I can come up with my own solutions, and honestly I don't know if it's any worse at flagging things than non-professional alpha readers. Probably about as random. But it's faster, certainly. It can't replace a good copy editor, but beggars can't be choosers.
As for the value, I got my lifetime subscription years ago and I think it has paid itself back. I only use the dedicated program as I didn't like the "everywhere" feature with Scrivener, and I also like to control what text I feed it. I think it's been a good tool to have in my arsenal.
Definitely agree, as a grammar check is where its strengths lie. And yup, familiarity with your own work blinds you to those small things. And to continue off your point, once I’m fixing some flagged sentences I tend to springboard off that into greater changes or just general tightening of words
Just dropping in to say that
Got a chuckle from me. I can honestly tell by your post that you're a very talented writer. That line in particular was extremely well done, not only in execution, but in specifically where it was sprinkled into your post. Carry on, and if you have anything published, I'd love to read it just based on your post.
Thank you, thank you, I’m flattered (genuinely). As for my public works— they’re quite scattered all over the place. My most current collection of stories that I have out there would probably be on this account: https://www.pillowfort.social/Lone_Dancer. (Wow, looking at that is a graveyard of all the unfinished projects I have strewn about. 2 months since last anything. But ignoring the WIPs, I have some (hopefully) nice short stories to peruse. Got some dragons, some weird places (that’s a favorite- I rather enjoy weird places), one stream of consciousness post, and I think one partially revived vampire.