12
votes
How are AI and LLMs used in your company (if at all)?
I'm working on an AI chat portal for teams, think Perplexity but trained on a company's knowledgebase (prosgpt dot com for the curious) and i wanted to talk to some people who are successfully using LLMs in their teams or jobs to improve productivity
Are you using free or paid LLMs? Which ones?
What kind of tasks do you get an LLM to do for you?
What is the workflow for accomplishing those tasks?
Cheers,
nmn
My company has an internal LLM portal with access to popular models like ChatGPT and Claude. We also have GitHub Copilot licenses for our IDEs. From what I've seen, people use our LLM portal for when we need to write a lot, like when we write our new bi-annual performance reviews. I don't know how others use Copilot but I personally use it to write tests. I can tell it to reference the files I want to test and it will guide me on writing the tests I want. I don't find the autocomplete suggestions from copilot that helpful tbh, it often just distracts and it seems to override the IDEs suggestions so I have to write a lot of repetitive stuff manually.
Thanks for sharing :)
Interesting, i use ai during coding similarly to write tests (tho I use it via cursor.sh not copilot)
Would you know what product your company is using for this portal?
This is definitely an interesting use case that i hadn't thought of. What other things do your teammates use the portal for?
If I'm not mistaken, the LLM portal is internally developed. Our company currently loves building custom replacements for industry tools haha.
As for other use cases for the LLM portal, I have talked to people in other orgs who do use it in helping them figure out how to code something. They'll describe what they want the code to do and provide it with their own initial implementation. They'll then use the AI output to shape their final implementation. Sort of like a pre-PR code review.
Totally empathize, I used to do this for my own company a lot lol
Nice way to put it, and I use this method a lot myself to code
We use ChatGPT and Copilot for software engineering. ChatGPT is mostly used as a search engine. Copilot is basically a more advanced version of autocomplete.
We've found issues with people blindly accepting suggestions from either. The tools frequently lack context for the overall project or suggest suboptimal code. It does provide acceleration for the initial prototype, but you have to refactor and think deeper on the actual code you get from it.
Same, ChatGPT fails in anything complex and outputs code that is mostly correct, but seriously flawed in a difficult to detect way.
Have you tried Claude 3.5 Sonnet? It's giving me much better results, but it's still not 100%.
The best experience I have is using Cursor.sh with Sonnet with web search. It is the most foolproof so far.
Am I correct in understanding LLMs are used only for programming and development at your workplace?
I have not, one of my biggest concerns at the company right now is that it's going to give our engineers "LLM brain". They aren't treating answers from ChatGPT with the same caution or thoughtfulness that they would with random code from StackOverflow. I'll look into the other models.
It's used in issue creation, epic creation, ideation too. We have one product using it that I can't discuss publicly.
That's something I struggle with too sometimes. If I use LLM code blindly then more often than note i find bugs down the road. But do try Sonnet, it's much better than gpt-4o.
Makes sense, thanks for sharing
My company is using both Copilot in Teams for M365 purposes and a custom trained model on the web for internal knowledge base purposes. The custom one allows for switching between Chat GPT 3.5 and 4, using the Azure versions of those models.
The key factor for utilization and success stories seems to be dropping in buttons, modals, etc. to the users’ normal workflows so that they can access the custom LLM directly within their workflow instead of having to constantly switch contexts to another tab or application.
Other than that, I’ve helped build a prompt library specific to my team’s role so that others in our position have reference points and can see what a good prompt looks like.
Between ease-of-use and plentiful examples showing successful use cases, utilization seems to be growing rapidly.
Can I ask what use cases have you seen the most success for, beyond the obvious Copilot ones of drafting/ replying to emails or proofreading?
My job is highly specific so they probably aren't helpful for most people, but here are some I've seen people using most often around me:
Thank you, I imagine that your use cases would require significant fine tuning or perhaps a really good semantic search/ RAG model.
Thank you for the detailed reply, this is exactly what I wanted to learn more about
Can you talk more about this part please: "custom trained model on the web for internal knowledge base purposes"
How was it trained?
What kind of tasks is it used for, am I correct in understanding that it's searching through the knowledgebase or is it something else as well?
That's exactly what I'm betting on, as tools develop almost every company will have its own LLM.
I'm not sure how it was trained; I wasn't part of that effort, so I'd only be guessing.
As for use cases, this post describes what most people at the company are using it for in the Audit, Tax, and Advisory sections: KPMG and Microsoft enter landmark agreement to put AI at the forefront of professional services.
Got it, and thanks for the link!
I’m working on the same thing actually! Fabrile.app :)
Are you far along? Let me know if you want to join in.
I think it’s a super useful type of tool in general and I’m surprised it’s taken so long for such apps to be developed. I’m also really surprised it’s a space OpenAI hasn’t really touched (their custom gpts are terrible for b2b)
Exactly! It's a great space
Fabrile looks cool, would love to chat more. Add me on discord? My username is "nmnyg"
Added you - mine is the same as my tildes username.
I work at an AI company. We use ChatGPT and CoPilot for coding and context-aware search. We also fine-tune Llama 3 and Stable Diffusion XL for our generative AI product. My boss used Perplexity for research tasks.
Thanks for sharing. Would love to more about the fine tuned product!
We use CoPilot in our IDEs and make use of Slack's AI offerings to summarize channels/threads (what I really need is a summary of the summaries, because I am in way too many channels against my will). I also use chatGPT occasionally when I need to. Usually when CoPilot fails me or I have a technical question that isn't strictly programming, so CoPilot refuses to answer.
Haha I feel that completely
I'm a civil engineer and have been using CoPilot for Civil 3D and PowerShell scripting.
While I understand coding and have been a computer power user most of my life, I am not a programmer. Trying to understand PowerShell and LUA enough to implement anything other than the basics is outside of reasonable for the time I have. A manual approach has always been more efficient in the past.
I've seen others use it to frame a report or do some large data analysis, but I haven't had much luck with it that way. However, it has sped up my excel formula creation and helped me avoid going down rabbit holes where I'd learn only after hours of trial and error that I would need Visual Basic to implement what I wanted.
Awesome thanks for sharing, using LLM for programming in languages I barely know is something I do often as well
Software developer for a company that isn't in the software field, we use GitHub Copilot in Visual Studio and in Teams for meeting summaries.
For development I find I use it pretty extensively for writing unit tests, mostly to save copy and paste setups and assertions. I'll use the chat feature if I'm not sure how to do something or for some types of refactoring.
The meeting notes aren't perfect but they're good enough that I don't really need to take notes usually.
Otherwise we're not really using it. Our team has thought about ingesting all our documentation into a chatbot but that wasn't really something we'll look into doing.