Any real AI recommendations from the community?

I use copilot a lot when I code.

While it hasn't reached a point where it can full-blown substitute actual programmers, it has drastically boosted my productivity.

I think big part of it is that google search has become really bad. I used to be able to find answers to specific technical questions pretty fast. Now it's mostly a lost cause, while copilot is doing really well.

Code generation itself is pretty decent too, especially when you know what you are doing. You can keep "massaging" the output by making it change things and guiding it.

Not advice, but a more specific request: Does anyone have recommendations for running code-assistant models locally?

The JetBrains suite is my editor of choice, and my workstation has an RTX 4090. The big requirement is local, offline, functionality.

I fiddled with this early last year when I first got the new workstation, but nothing was very compelling or seemed terribly useful. IIRC some of the models were okay but editor integration was very much not. Generally I'm pessimistic about AI, but it's been long enough that I think I probably should give it an honest try again.

Or, if there are truly compelling online services, are there any that are easy to disable in certain repositories? This is a data security concern about some of my projects; I could use an online service in the others, but I need a foolproof way to disable any third-party analysis in those sensitive ones. None of the plugins I tried (early last year) inspired confidence that I won't open the wrong project and accidentally upload data when I don't mean to.

For public services, perplexity.ai is legitimately useful.

Language learning using AI is a very powerful tool too.

If you're fairly technically inclined, you can get really stellar results by building workflows using langchain or other similar libraries (instructor is also good). Setting problems up to be broken up into subproblems with the multiple solutions being checked in parallel, with the results being double checked and revised by other AIs, really cuts down on the effects of hallucinations. Combining that with function calling and external data has been proven (in my experience) to be close to and sometimes better than human work.

Claude AI by Anthropic has been my go to. I only touch chat gpt for simple stuff so I don't use up compute time with Claude.

It's helped me code/develop a whole new personal library catalog to catalog my physical books at home and helped me build a react website from scratch. It's been a fascinating learning experience.

I've mostly been trying the open source/locally hosted models and been having an interesting time building some customized solutions to stuff.

i had a huge mess of untitled(14).png and random downloaded pictures in my folders and it turns out theres not really any good solutions for sorting an ugly filesystem and it was taking me ages to rename all the pictures manually. so i write a little script to generate a new filename based on the content of the image, and then a .txt version of the same filename that had a more verbose description so i could do text-searches on them.

left it running over night and while there's more than a few misses, the vast majority of work is done to a good enough degree (better than an annoyed and lazy me could do) and honestly i can live with any errors bc its better than what i had before.

other than that, its mostly just been playing around with fun little ideas instead of anything actually productive. like i've had this "world news" idea where its google maps but instead of businesses, its news articles from that day. tried implementing it a bunch of times using traditional NLP stuff but this recent wave of LLMs has been the first time ive been able to make a functioning prototype. and every few months it goes from being woefully inaccurate and slow, to good enough and fast enough to process at least 10 stories from each country, each day. all while maintaining my usual pc usage.

people tend to talk about the big companies that have super flashy and instantly responding AI services but i think there's gonna be a wave of small "edge device" models finding use like this. since i bet a lot of people dont care if a tasks have to run in the background and chug away overnight to finish renaming thousands of pictures, summarize their all of their college notes before a test, or even just constantly generating solutions to a code problem until it passes a test you gave it. and its gotten to the point now that you only have to install a nice little program and download a model and boom, you can start play with this stuff (programming not included). a stark contrast to a year and a bit ago when i started dabbling and it was a nightmare trying to get anything to install, let alone realise my graphics card exists.

Here's how I use these:

• Gamma: This is my preferred presentation format. I work with emails and whenever I had to use a design in a presentation, it is more frustrating to use than not. Not to mention how much effort it takes to create a presentation that looks good in Google Slides (haven't used Microsoft Office in a while, but I had the same problem). Yes, you can use templates, but even then, the tools didn't really work for me. Gamma makes presentations look good, and you can have it extend beyond the usual 4:3 presentation and look closer to a web page. Presentations can still be done, but most of the presentations I make are more quick-scan documents than a proper has-to-be-presented-on-a-screen presentation. I don't use the AI features much except for import, which works well enough to create a good-looking presentation, including structured content. But if I have to, I use ChatGPT for the content since I mostly work with that and then use Gamma for turning it into a great-looking presentation. I've used it for my portfolio and I've had clients compliment me on it because it looks so much more professional.

For ChatGPT, I use it in a few different ways daily. I am mindful about what information I put into it since I don't think it's very secure. I also don't use it for any factual information, but I do use it to find ideas/concepts that I can then go look up on my own. Here are some of the ways I use ChatGPT:

• Transcribing handwritten notes/printed text - My handwriting can be a hit or a miss, but usually it does a good job of getting the information and then organizing it the way I want it. I like keeping notes digitally, but sometimes writing is easier, and this helps to get some longer text. Other than that, when there's something like an ingredient label, I use it to transcribe that too and sort through what's in it.
• Product comparison - Speaking of labels, I use it a lot for product comparison. When online shopping and there are too many products for different prices at different quantities, using this to find me the cheapest option. It can also help me figure out what's exactly in a product, and then I verify that separately. I used this for skincare products recently.
• Understanding text - Other than just summarizing, I use it to understand jargon-heavy text like research papers in fields I'm unfamiliar with by helping me break down what's in it. I have it both simplify the language and define terms I don't know that I can then look up separately, or go through the sources it provides for the information to verify. This also works with code. I'm not a developer, but I have some basic knowledge. A client was having an odd issue with their email user database, for which I found relevant developer documentation, but I wasn't sure if that could be the problem. I had it compare the issue with the documentation to figure out what configuration might be an issue and some possible reasons for it. Once it did, I was able to share that to point them in the right direction to get a developer to look into it because it was a coding issue and not a platform issue.
• Analyzing my own writing - It is good at identifying patterns in my own writing that I sometimes don't notice. This takes me a bit of prompting to go beyond what's obvious, but when it works, it can be really insightful. Ideas that I may be circling, but haven't quite reached yet, or anything similar I might want to explore based on what I have shared. Claude can be better at this than ChatGPT out of the box. But ChatGPT can be too, depending on how you prime it with custom instructions - and it can be sometimes better if it has more context with memory.
• Learning assistance with study guides/schedules - Forgot to add this earlier, but I used this to break down an online course I was taking into a schedule that worked with the time I had, and had it give me additional resources I could look up for specific topics I was interested in, tying that into the schedule as well. All I gave was the course overview page with the modules and topics listed with the timings, let it know that I usually watch videos at 1.5x speed, and the sections I wanted to do more research on, and it was very helpful in creating a study schedule.

As an aside, I've also given ChatGPT some personality because I hate the default overly friendly way these LLMs write, so I have changed it to something I would actually want to interact with. It's better if you use Projects and reinforce with more instructions, and more context than you can store in Memory, which is capped at a certain number. Then, the conversations I am having come close to rivalling Claude (which has the similar 'emotional intelligence' of Pi, but more powerful). It can still revert to bot mode sometimes though, especially when it does a web search. This has been the biggest change in the way I use it, and I highly recommend using custom instructions at the very least, even if you don't end up using projects.

I use ChatGPT to get over all sorts of slumps. When a task is too boring or ill-specified to even start with, I just throw up on ChatGPT (v4 or o1) everything I know about it, and it usually gives me something good to start from. This saves me hours or even days of procrastination time, leading to faster outcomes and far less stress for myself.

Sometimes I verify my comments around sensitive topics to dull the edges and make it less probable that I'm misunderstood. I should probably be doing that a bit more.

Sometimes I use it to practice french.

On rare occasion I use CoPilot to navigate my way through a codebase I'm not going to be sticking around in.

Lately my hobby project is making a simple editor for a pseudo 3D engine, but I couldn't figure out how to approach clicking to select an object. The renderer isn't much help here. You feed it a scene graph and it writes to an HTML Canvas or to an SVG all by itself. There's no hint of what areas of screen space correspond to what chunks of the scene. I found a mostly finished editor for the same engine that does what I want, but it was a large bunch of React that I couldn't parse.

Feed the whole codebase into CoPilot, and it points me at an external library. Feed the library into it, and apparently it's doing color testing on a separately rendered copy with unique colors for each object. And now I've got an approach I can bring back to my codebase.

Coming from the opposite end: my use of generative AI so far has been mostly limited to treating it as a toy to tinker with. So far the results are... in line with what I expected from what is effectively a glorified autocomplete trained on a huge dataset. That being said, since I can't afford to pay for API access to an online generative AI model (nor am I interested in doing so if I could), this leaves me with only running models locally, and I'm further limited by my laptop's GPU having "only" 8GB of VRAM which is just barely enough to not be completely useless as far as generative AI is concerned, so your mileage may vary. Among the things I tested:

• LLM as helper for programming: the implementation I've tested had stability issues and any model that came close to be helpful was also a huge strain on my resources. No bueno when your machine starts locking up as you type because the LLM trying to act as code completion is maxing out VRAM and spilling into RAM. That aside, it is also prone to disregard or misunderstand instructions, can't reliably follow syntax, and while it can give useful advice it also cannot be trusted to not make assertions that are completely wrong. I could see it being useful in a "rubber duck programming" kind of way if it can be made to be run smoothly.
• Looking up information with the LLM hooked up to a search engine: Disappointing results, at best regurgitating the exact same information I would have gotten from directly inputting my query in the search engine itself, which would be fine if not for the fact the process is also incredibly slow, at worst failing to parse said information and hallucinating. For what it's worth, that specific test was conducted under pretty much the worst case scenario (Windows, AMD GPU with no Windows ROCm support) so there's room for improvement performance wise.
• Extracting information from a provided document: Works well enough so long as the query has a straightforward answer in the document, if not the LLM will likely fall back to doing what it does best: confidently making stuff up (starting to see a pattern here?). In a practical scenario, CTRL+F usually works just as well without having to spin the hallucination wheel of fortune.
• Generating arbitrary images based on a third party's request: I popped up on a Discord server I'm a regular on and offered to lend my GPU for requests from the other members to see what local text to image models were capable of. The results admittedly exceeded our expectations... though that was not saying much. To pick an example, someone asked for "a panther tank" half-jokingly expecting the prompt to trip the model up and generate a mechanical/animal hybrid mess, but as it turns out the model came up with something that looked like the actual WWII German tank to an untrained eye... So long as you don't pay attention to things like the tracks' layout, the actual proportions of the vehicle, or the typical AI artifacts that look distinctly off, seemingly as detailed as the rest but more indistinct in a way that no human artist would depict, as if the picture was a hazy memory. Tinkering with an image prompt to get something that looks like what you have in mind can be fun, but it's clear that using it as the only tool to create artwork is backward if there is any need for accuracy and gets nowhere close to an actual artist's abilities. I could see it used when you need the graphic equivalent of lorem ipsum text or as placeholders until artists can provide the proper artwork, though.

One specific application I haven't tested yet but would like to is a text to speech model imitating my own voice, after seeing an experiment that caught my attention where a model trained with someone's voice in a given language could just as easily speak in another language with the same voice. I'm a French native speaker and while I can understand and write English just fine, my pronunciation is horrible to the point of being unintelligible, which means I could plausibly train a text to speech model that is better at speaking English than I am while still sounding like me... making it a useful fallback in the all too common scenario where people straight up can't understand me in a voice call because my accent is too thick to parse what I meant. I'm also looking into ways to implement a general purpose voice assistant to replace the Google Home device in my house's living room, because while I'd love to hurl this data siphon straight into orbit, it is genuinely useful for my mom due to the voice command aspect so I'm staying my hand until I can put something that can fill its purpose in its place (besides, who am I kidding, she has an Android phone which probably has already harvested whatever data the Google Home device could have).