What is your note taking workflow?
Hi,
I'm wondering how and if you're taking notes and how you manage them. I feel like I can't get to the right solution. I want something open and own my data, so I tend to prefer plain text / markdown files and I want to manage them on my own for privacy reasons but still be able to sync and edit them on my mobile device (iOS), desktop (macOS) and ideally online and also have full text search.
I know about Microsoft OneNote, Google Docs/Keep, Evernote - but I just can't overcome myself to use them since it's hard to impossible to get data out of there.
Right now I'm hosting my own git repository with gitea.io and I'm able to edit my notes on the web, on my desktop (using git clone/pull/commit/push) and with a git client (working copy) on my phone. The downside: I need to manually commit and push my changes and also pull them, it's not really in sync.
Well.
I went through a few iterations before settling on what I am currently using.
I started out using TiddlyWiki, back in 2009-2010, if not earlier. I was using TiddlyWiki Classic with quite a few plugins, and used two - maybe three? - such wikis, to keep notes on tabletop RPGs, on my courses, and as personal journal. I like TiddlyWiki quite a bit, but the safety measures built in modern browsers have more-or-less crippled it (save functionality used to work out of the box, then required a
.jar
, then a browser extension, then Quantum came out...), and browsers are in any case pretty bad editors.I tried a few different conventional wikis after that: DokuWiki, the ridiculously lightweight AwkiAwki and CiWiki, and probably a few others. I never liked them that much: I need to run a server for them, which means that, even if they store their data on the filesystem, I'm forced to stare at the raw markup when on my phone away from home. And wiki markup looks ugly to me.
I was just keeping my notes in random markdown files by then. I wanted a simple solution, something that could compile a pile'o'files into HTML that I could browse on my phone, and I ended up rolling my own: I started out with a simple markdown to HTML script written in Awk and a Makefile, which slowly evolved into a shell-and-awk pipeline which supported backlinks, tags, mathematical equations (thanks to the Groff
eqn
being able of spitting out MathML), and references (I was actually using therefer
format, but it was parsed by my own Awk script).It worked well enough, but I wasn't really satisfied with it.
I tried out org-mode with org-wiki briefly, but I was still new to Emacs, and I went back to my own awiki for... maybe a year. By then, I felt comfortable enough with elisp to hack something up, and I started transitioning to org-mode fully (I was already using it, but mostly for the agenda and for quick notes on the go with Orgzly).
Even if I'm eyeing a few other solutions, I'm using org-mode right now, and I have been using it for the last year.
My wiki contains 170 pages, 117k words, and thousands of links. It doesn't have search - I use
grep
for it - but it's still small enough that I can usually find what I'm looking for in instants. It's organized pretty roughly - the top pages are Computers, Culture (which should probably be called Arts), Science, Society, World. There are a lot of uncategorized pages, as I mostly access them throughorg-wiki-helm
.If I'm on my laptop, I add to it immediately, commit the changes to git with magit, and then run
make
to compile it to static HTML, which Syncthing syncs over to my phone.If I'm on my phone, I capture the link or snippet to org-mode with orgzly. Everything gets dumped into a
@Capture.org
file, which I then sort through when on my laptop, organizing the entries into a date tree, and copying the links/snippets over their appropriate wiki page.I also keep an A5 notebook by my desk, but I mostly use it for quick sketches, and anything really important gets moved to my wiki as soon as possible.
But, again, I'm eyeing a few other solutions. I like org-mode, I like it very much, but a lot of information doesn't neatly fits into my hierarchy of pages - probably any hierarchy at all - so I'm looking for something more granular, where the basic unit of information is much smaller than a file, which can live under, above, or at the side of any other snippet.
TiddlyWiki would mostly fit the bill, but - going from editing inside Emacs to the browser, and from org markup to wiki markup? It's not really something I could do anymore.
Edit: redundancy.
Wow, you've been through a lot of things there! org-mode comes always up, I should really give it a shot, especially since there's an iOS version available now with a sync backend. I guess it'll just take a lot of time to learn it since I'm also new to emacs.
I suppose I should add to this thread, as I too use org-mode, albeit in a much less sophisticated manner. I have a document in which I just create a new section for each day, and I use a combination of TODO subsections and just plain blocks of text to annotate the things I learned that day. When I want to remember how I did something, I just search backwards to find it, which has been pretty quick even as the file grows to several megabytes. I've tried to be extremely organized with subsections by topic in the past, but really just dumping it chronologically seems to work best for me.
I do track long term todo items in a related file via beorg, which is a very handy mobile interface.
Yeah, org-mode is very flexible, and a pretty big ecosystem has been built around it and Emacs, so it's bound to come up when note taking and knowledge organization are being discussed.
It depends on what you want it to do. It won't take much to simply use it as an outliner, or as Yet Another Lightweight Markup Language with good hyperlinking support, and the vim/Sublime plugins should be good enough for that.
Take a look at Deft for Emacs
I already know Deft, and I even used - am still using - that, but it's not quite what I have in mind. Not just text, but more loosely structured data - which would also include quite a bit of text - and not just a way to edit it, but also to publish it in a way that would be easy to consult from different devices.
This looks like what Org wants to be. I never used it, but are you familiar The Brain?
I know of it, but I have never actually used it, and I am a bit leery of using proprietary software in any case.
Understandable.
Most of my notes – work, project, personal – are in Indigrid. I categorize them hierarchically, into subsections by which kind of a project or a thing it is, and then what part of the thing it is, and then what question it answers etc.. It's an outliner, so it probably won't do you good to store long-form thoughts there.
Ideally, I'd want to have something like Trillium for what I consider my mythos – that is, the span of all the ideas, traits, and experiences my life contains. Haven't tried it yet.
If I need a quick note jotted down, I have a stack of wide yellow stickies and a pen on my desk, so I could write them down. (I wish I had something digital that's just as efficient and easy to edit.) If I need a quick note jotted down and I'm AFK, I have a notes app called Better on my iPhone. (I'd link you to it, but I can't seem to find it on App Store outside of the iPhone now. It has an icon of a white sticky, with yellow background.)
Wow, thanks for showing me Indigrid, I've been trying out tons of different notetaking apps and this one feels perfect!
(org mode and notion are the only ones that got close - the former had too much of a learning curve for now, the latter has some performance issues on mobile)
Indigrid looks promising, Windows only for now, unfortunately :( Trillium checks most boxes, they don't have an app but apparently a mobile friendly theme. They also provide a docker image so the installation shouldn't be that hard, thank you so much!
What do you mean they don't have an app?
for iOS :)
Oh I see.
For the record: you can subscribe to the Indigrid mailing list to receive a notification when the macOS version becomes available.
Just signed up, thanks!
Hey, just wanted to chime in to this thread and say: this looks quite similar to (some of) what Org mode can do for you, so if you don't want to use Emacs, this seems to be a nice way to get something similar to Org's functionality. But take that with a pinch of salt: I've just looked at it now.
Indigrid looks similar to Workflowy, which has an Android app.
Before the method I'll outline below, OneNote was my main driver. Its flexibility over organising notes, combined with the infinite page size, was absolutely superb. I found that the version distributed with Office is superior to the web and Windows 10 one; however, Office isn't free nor cross-platform compatible, so I eventually had to give up on using OneNote, unfortunately. I have thought of returning to it, but I'm too deep into the system I'm using now.
After going through countless methods of note-taking, I've finally settled on simple plaintext files (with Markdown syntax for consistency & rendering support), in a Git repository. The notes are categorised by directory trees. There is a top-level readme file that summarises all of the notes and links to them.
This offered the most flexibility, simplicity, and compatibility. I use Visual Studio Code for the convenience and Markdown-related features it provides; however, on iOS I use iA Writer. For the Git software on iOS, Working Copy works wonders for me.
The entire repository also serves as a diary. The years and months are sectioned into directories, and each day has a text file dedicated to it:
Initially, the diary folder had its own readme summarising what I've written each day, but that quickly became a nightmare to maintain.
A convenient alternative to using Git is having the text files in a syncing software. Syncthing does the job for me when I use it to sync my KeePass databases. The reason I chose Git for the notes is for the sake of revisioning. It's nice going back in time to see how your notes have evolved over time!
That's basically my workflow right now! I assume you're using the Working Copy syncing feature with iA Writer together? Do you have some sort of automation for pulling data and doing commits? That's the one task I don't want to do manually all the time on my mobile.
When writing, I share the file from Working Copy into iA Writer. I usually do this when writing for extensive amounts of time, as the experience in iA Writer is much more joyful. For very brief writing, I do it directly in Working Copy.
It's not automated; however, Working Copy does have a "Synchronise" button that pulls and pushes the revisions for you. If you want to only pull commits, swiping down (think refreshing on Instagram or Twitter) fetches the latest revisions from the Git remote for you to merge them in one tap.
For committing, my commit messages are usually along the lines of
Append 2019-05-07 diary entry
. That ended up making the iOS keyboard offer predictions, so that speeds things up a little!I personally do prefer to manually commit and synchronise, as it gives me control over the contents of each revision, and also prevents merging conflicts. Working Copy does make it pretty straightforward on mobile, so the manual work isn't particularly tedious as it would be - for example - when using a terminal on mobile.
If you want automation, using Dropbox instead of Working Copy/Git would drastically speed things up. Dropbox also keeps revisions of your files. The same applies for OneDrive and Google Drive, so using Dropbox isn't a requirement. If you care about privacy/self-hosting, Nextcloud would do the job.
There's just no app with an integration into OneDrive/Google Drive nor nextcloud. If they support sync it's either iCloud or Dropbox :( I think for now I'll use the build-in editor in Working Copy, not the best but it works, even with preview and full text search through the repo.
Hmm, what if you install Nextcloud, and in it, you open up a file, then share it to iA Writer? That's what I do when browsing files with Working Copy and editing them with iA Writer.
I need to share them back to nextcloud manually then, right? Or can nextcloud be integrated as an external library?
In the case of Working Copy, I didn't need to share the files from iA Writer back to it, so the same should work with Nextcloud. You've actually gotten me curious about this - will get back to you on how it goes.
Had a bit of a play with iA & NextCloud. I created the file in NextCloud and edited it from iA Writer. A pretty straightforward approach -- here's a video.
cool, thank you. This might work, it's not what I hoped it would be (the synced folder thing is super integrated like a local folder to iA Writer). I'll check this
Awesome!! This would require little change on my current setup, I'll definitively look into entr and similar tools. Haven't thought about rclone as well, not sure if I need this since I'm already running a remote server with a git frontend. The gollum wiki mention sounds very promising, however. But maybe I can build something with rclone on the remote server or a webhook to get a better iOS experience. I think this is the biggest part. Not sure if I want to use vim on my iPhone like the hacker news guy.
I absolutely agree with everything you said. I, too, just use whatever text-editor to edit plain markdown files and keep them on a git repo. It is a third party git repo though, I need to see if I can find some sort of E2EE git option. I don't like having my notes on a third party server.
I use zim wiki. Its a nice and simple notes desktop app which you can sync using any cloud storage service. No mobile app but really I have never actually needed to edit notes on my phone that I started on my desktop. I have a separate notes app on my phone which is just for tiny things I need in the near future.
I use a mix of paper and digital: I have a bullet journal that I use for managing tasks and agenda, which is also a receptacle for random notes at times. I also use an A5 sized clipboard as my notebook. Most stuff ends up being scanned and entered to the computer (I will talk about software in a bit). If they are not needed immediately, they generally sit in my scans folder, and I (hypothetically) transcribe stuff when I have time. Pen and paper are way more flexible for me, especially when I need drawings and/or I am not taking notes from something that is already digital (like PDFs, webpages, &c). All my lecture notes are written on my clipboard, for example. I used to use a fountain pen, but nowadays I use a BIC ballpoint with 4 colours, which is quite handy.
On the computer, I use Org mode, a major mode for Emacs. It is a very flexible outliner that can do anything from managing your tasks and agendas very effectively to typesetting novels, whitepapers and scientific books, and also be your blogware. It is extensible with Elisp and integrates very well with everything Emacs has, and then some more. I mainly use it for notes and word processing (I don't want to be tied to my computer for my tasks and agenda b/c I have procrastination issues & I find a BJ works better for me; but I may change my mind if I can get an ultralight computer). For scanning, I use Gnome's Simple Scan.
There are various org files that notes can go:
I use Orgzly on mobile that syncs via Syncthing to a temporary file, which I use as an intermediary for getting stuff from mobile to desktop. I hate typing on mobile with ferocious destructive passion (is it ironic that I am typing these on a mobile keyboard? soo fucking slow...) and never really take notes there, but just in case.
Syncthing is a host-your-own file sync daemon, it is FOSS and it has a FOSS Android app. It is lightweight, but I'd not put files I care on it and don't trust its read-only folder settings. I am in the market for an alternative myself, but can't find something that is not written in PHP. But I will probably end up slapping a Nextcloud on my Raspberry Pi...
Most if not all my org files are version controlled, either with git or with RCS, but I don't push them anywhere. TBH I don't like the idea of having multiple devices I use to edit the same stuff regularly. Ideally I'd only have a 13" laptop and a big screen phone (not there yet, but will be, my current laptop is 15.6" which is too big and heavy for my use cases), and maaaybe a desktop if I end up wanting/having to do mobile apps or other professional development.
I've used syncthing in the past, good point! Second time now that emacs/org-mode is coming up in this conversation. Do you have recommendations how to get started? It seems emacs knowledge would come in handy before one goes with org-mode.
Well, yes and no. There are implementations of Org for other systems like Vim or Sublime Text, but its home is definitely Emacs, where all the features arrive, and are available and complete.
It is hard to gain knowledge about Emacs w/o using it. I'd suggest you start with a blank slate, plain Emacs, check out better defaults. If you want to see a complex config, here is my init.el, there are some interesting configs in there, but bulk of that is just adding bits of functionality here and there, and not necessarily needed. Just to give ideas. Steve Purcell's emacs.d is a very popular personal config for Emacs.d (it has 5k stars on Github, crazy!) I used to just look at people's configs and pick and choose interesting settings. As I go along I learned Elisp, and now it has settled. I think that's the best way to go.
Also, Emacs comes with great documentation:
Ctrl+h
andf/v/k
shows documentation for functions, varibles and keys respectively;Ctrl+h
andi
takes you to the manual, which is incredibly useful. But the best thing to do, as a newbie is to hitCtrl+h
andt
and do the tutorial. It is like vimtutor, but for Emacs. Will help with many frustrations down the road.To come back to Org mode, well, it is a very versatile and extensible tool, so where to start depends on what you want to do. Agendas? Task management? Project management? Notes only? I think you'd want to start with notes first, and maybe slowly add tasks here and there. That is easy: just open a file with the
.org
extension in Emacs. Notes are organised into entries that start with headlines. Below is an example file in Org syntax:Thank you! This will help getting started
I found this neato program called "CherryTree", it's a hierarchical text editor. You can set up parent/child nodes as much as you like, so i've been using it on/off for basically everything.
Whenever I wanna fool around with a new writing prompt I start a new file and just create nodes as fast as i can think of chapter-y things. Then as inspiration takes me, i just descend into one note until it's fleshed out enough or until i hit a deadend. After that I'll usually start a node for characters, and another one for those ideas that you feel could work with a little bit of shoehorning.
After i've expended all my inspiration, i'm usually left with something much more satisfying than if I had just tried to dive right into writing the thing. Sometimes I dont even write the story, I just worldbuild it because that's the more fun part to me. Literally the opposite to how I used to be; Writing the story and doing the world building in my head.
Here's a (blurred bc story bad) pic of my current thing https://i.imgur.com/WBVWtt1.png Each of those nodes on the side is just a flood of specifics about each thing and how they work/are perceived in the world, and where I wanna go with them. I've also used it to plan some programming projects, but i seem to have lost that file when moving laptops
Not sure if it'll be worth more than a play around, since you wanna do stuff online, but for me it's been an essential tool in managing a trove of local notes
I guess I’m a bit old fashioned, as most of my notes are pen-on-paper using cheapish notebooks. Anything import I’ll TeX up as this gives me another opportunity to look things over and rewrite it in the clearest way. I briefly installed vim-wiki a while ago and got it set up, but I never really ended up using it too much. Perhaps I should take that up again, or follow one of the suggestions here. I really like the idea of a personal wiki.
I used to use Apple Notes on iPad because it's easy and l cam draw in it, until l
had to export the files to a PC which was hell since there's no good batch export option.
Right now l'm working on figuring how to set up a server and run a website with a private password protected wiki to categorize notes and cross-refer for easy info storage.
I'm a total noob to linux, php and hosting things so that's been pretty fun so far. Any Nginx wizards out there that can tell me how to easily add subpages? (like http://example.com/subpage1, http://example.com/wiki, etc)
I personally have not taken notes in quite a while but when I did, I used "Standard Notes" (It's a bad name if I'm being honest)
The free service only has plain text and tagging with search but the paid product can extend it quite far, adding folder system, markdown, latex and a lot of things really. It also has mobile apps for iOS and Android which work pretty well (and support some of the extensions but not all if I understand correctly). but the syncing is really good.
If you were to wanna opt-out, you can too through their backup and import system.
You might be scared because the desktop is Electron but through black magic, it only uses like 50MB somehow. (Without extensions and on a cold boot, I've not looked extensively)
Edit : Oh yeah, might be worth mentioning the encryption and that the full stack is AGPL/GPL
I really like Joplin: https://github.com/laurent22/joplin
It's free, open source, uses Markdown.
It has sync features if you need them:
and:
Some nice features:
Took me a while to find it, but now I stick to Joplin.
Since you're already using git, you could try this... I just tested it out for my own usage and works well enough on Linux:
https://crates.io/crates/eureka
Edit: More details:
Eureka does a pull, then prompts you for your commit message, then opens the README.md file in your chosen editor. When you save and exit, it commits and pushes.
cool, looks promising for the desktop! Thank you
Personally I use Dokuwiki on a $2/mo server, absolutely fantastic and by far the best way for myself personally after years of switching to other things.
I highly suggest the monthcal plugin.
A few Org files and capture templates. I'm trying to use the Agenda more, but I keep forgetting. I don’t take too many notes but all my writing is done on Emacs.
I just migrated from Evernote Premium to Notion recently. I haven't been able to quite use all of Notion's features, but their features seem to be further along than Evernote's. I only miss being able to embed PDFs and annotate them. Unfortunately, Evernote worked only about 50% of the time with annotating PDFs.
Yeah, I checked out Notion as well, but it's a super lock-in. They don't even provide a bulk-export or API.
You can export entire workspaces with one button in settings.
Manual interaction only, no API or anything like that to automate it as far as I know, right?
I don't usually take notes, but if there's a small thing or two I need to remember, I just say "Hey Google, take a note" to my phone.
Which is a command i overwrote using their routines functionality that calls Tasker through AutoVoice, and that will open up Google Keep and automatically tap its voice recorder.
Then I just say what I want to say.