8 votes

Multi-format text editor with chain-of-command processing

A while back I developed a desktop-based text editor (Scrivenvar) that uses the Chain-of-Responsibility design pattern to help me author fairly involved text documents. The editor's high-level architecture resembles the following diagram:

https://i.imgur.com/8IMpAkN.png

Am I reinventing the wheel here? Are there any modern, cross-platform, liberal open-source (LGPL, MIT, Apache 2), text editor frameworks (such as xi or Visual Studio Code), that would enable (re)development of such a tool?

Scrivenvar is written in Java, but to my chagrin, Java 9+ no longer bundles JavaFX. The text editor was based on MarkdownWriterFX, itself based on JavaFX. This means there's no easy upgrade path, so I'm looking to rebuild the editor either as a cross-platform desktop application or as a web application.

11 comments

  1. [3]
    SkewedSideburn
    (edited )
    Link
    I mean, you don't have to rebuild anything, you could either use OpenJFX as a Maven dependency (which is the simplest route), or you could bundle stripped down version of JDK from...

    Scrivenvar is written in Java, but to my chagrin, Java 9+ no longer bundles JavaFX.

    I mean, you don't have to rebuild anything, you could either use OpenJFX as a Maven dependency (which is the simplest route), or you could bundle stripped down version of JDK from Bellsoft/AdoptOpenJDK/Other_vendor_who_is_not_Oracle which does bundle JavaFX. Just find out which modules you depend on with jdeps and use jlink to make your own JRE. That's what I do for my app at work.

    In both cases you're still going to have some work to do updating your code, but I was able to run it on Bellsoft's Java 11 just by bumping versions of ControlsFX and FontAwesomeFX (since they relied on com.sun classes that were changed/removed in newer versions of OpenJFX and had to add support to Java 9+ anyway)

    (Also not sure if you've noticed, but while your screenshot at GitHub shows $application.title$ being transformed to Scrivenvar, the actual title of README.MD is still, in fact, $application.title$)

    3 votes
    1. [2]
      DaveJarvis
      Link Parent
      Thank you. At the moment--to my knowledge--GitHub doesn't provide a way to associate a named YAML file with a Markdown document, much less apply a YAML preprocessor (my utility) to perform string...

      $application.title$

      Thank you. At the moment--to my knowledge--GitHub doesn't provide a way to associate a named YAML file with a Markdown document, much less apply a YAML preprocessor (my utility) to perform string interpolation on the variables within a hierarchical namespace. In the architecture diagram, that's the purpose of the "Variable Processor" shown at the bottom.

      OpenJFX

      Thank you, again. This does introduce complications: the library is GPL (not compatible with Scrivenvar) and will require three separate downloads (Java, OpenJFX, and Scrivenvar). Licensing aside, bundling the components with a stripped-down JDK to avoid separate downloads would then require multiple builds: one for every operating system (at least Linux, Mac, and Windows). The promise of Java, IMO, was "write once, run anywhere," rather than "write once, build multiple binaries, test everywhere." A technical quagmire, either way. If offering multiple binaries (or requiring three software downloads) is the only way JavaFX is usable, then rewriting the application without JavaFX is preferable.

      I do appreciate the help and the OpenJFX will likely work for my own use.

      1 vote
      1. SkewedSideburn
        (edited )
        Link Parent
        It's not pure GPL, it's GPL with Classpath Exception, just like OpenJDK itself. That "classpath" part is important, it means that GPL license does not contaminate your application. Specifically,...

        the library is GPL

        It's not pure GPL, it's GPL with Classpath Exception, just like OpenJDK itself. That "classpath" part is important, it means that GPL license does not contaminate your application. Specifically,

        As a special exception, the copyright holders of this library give you
        permission to link this library with independent modules to produce an
        executable, regardless of the license terms of these independent modules,
        and to copy and distribute the resulting executable under terms of your
        choice, provided that you also meet, for each linked independent module,
        the terms and conditions of the license of that module.
        

        test everywhere

        You still need to test everywhere, releasing the jar won't relieve you of that. But if you're bundling JDK you at least know with what version of Java user will be running it, so you actually have more control that way (and trust me, different vendors do have different bugs).

        bundling the components with a stripped-down JDK to avoid separate downloads would then require multiple builds: one for every operating system (at least Linux, Mac, and Windows)

        That part is cumbersome, yes. That's why for your use-case adding Maven dependency to build.gradle would be more than enough. No separate downloads, same fat jar in the output.

        2 votes
  2. [5]
    Happy_Shredder
    Link
    May not be quite what you're looking for, but I get a lot out of vim+pandoc 1. Pandoc is a little bit magic and handles transforming text with templates and variables etc very cleanly. I just like...

    May not be quite what you're looking for, but I get a lot out of vim+pandoc 1. Pandoc is a little bit magic and handles transforming text with templates and variables etc very cleanly. I just like vim and a few choice plugins go a long way.

    Plus I like separating out editing from processing; it's more intuitive to me, and allows a great deal of flexibility.

    If you do want to go down this path, consider Qt. The QTextArea widget can display html and rich text.

    1 Vim config here.

    3 votes
    1. [4]
      DaveJarvis
      Link Parent
      Indeed! You may enjoy my blog series on typesetting Markdown, in particular the bash scripts that cobble pandoc and other tools together. Completely agreed: for simple variable substitution,...

      vim+pandoc

      Indeed! You may enjoy my blog series on typesetting Markdown, in particular the bash scripts that cobble pandoc and other tools together.

      handles transforming text with templates and variables etc very cleanly

      Completely agreed: for simple variable substitution, pandoc templates are bomb. Variable interpolation is where pandoc fails. If you're interested in this topic, I opened a feature request four years ago. In particular, the discussion around namespaces and sigil syntax may spark some ideas.

      Knowing that variables from pandoc templates wouldn't meet my needs in any short amount of time, I wrote a Java program to perform variable interpolation on YAML files. You can read all about its integration in typesetting Markdown part 5. The code for the YAML preprocessor is online.

      4 votes
      1. [3]
        Happy_Shredder
        Link Parent
        Mmm, ok I see what you mean. In principle you could extend pandoc to do that sort of thing, as the pandoc-moustache example does, but I think what you want is quite a different tool. pandoc simply...

        Mmm, ok I see what you mean. In principle you could extend pandoc to do that sort of thing, as the pandoc-moustache example does, but I think what you want is quite a different tool. pandoc simply translates text from one format to another based on templates (already a serious task) which is actually quite different to interpolation. I'm not aware of any tools which do this perfectly generically.

        Having put no thought into this, the direction I would go for would be to just embed lisp phrases in my documents and then write a small compiler to process everything. This keeps everything integrated in my text-based workflow. And as a lisp, I can do everything. One approach would be a header defining global variables and various metaprocessing characteristics. Each lisp phrase would accept some input, process it as required, and return a string. Tie it all together by defining how each phrase interacts and how to pass input to each phrase and the whole thing could fall it fairly neatly.

        3 votes
        1. [2]
          DaveJarvis
          (edited )
          Link Parent
          That workflow would work well for Lisp developers. Scrivenvar is for non-developers, to make using document variables trivial. Similar, yes; so much so that I submitted a feature request a few...

          lisp phrases in my documents

          That workflow would work well for Lisp developers. Scrivenvar is for non-developers, to make using document variables trivial.

          pandoc-moustache

          Similar, yes; so much so that I submitted a feature request a few years ago.

          I'm not aware of any tools which do this perfectly generically.

          I wrote yamlp to perform string interpolation on YAML files. It works, but adds complexity to the pipeline. Ideally, the string interpolation would be a pandoc Lua filter, or even a cross-platform standalone binary.

          Using Lua would greatly streamline the workflow: I could retire yamlp and eliminate piping variable files ("templates") through pandoc twice:

          java -jar yamlp.jar variables.yaml > interpolated.yaml
          cat interpolated.yaml ??.md > body.md
          
          pandoc body.md --template body.md --metadata pagetitle="unused" | \
              pandoc -t context > body.tex
          

          Versus:

          cat ??.md > body.md
          pandoc body.md --lua-filter=variables.lua \
            --metadata-file=variables.yaml -t context > body.tex
          

          If you had any more thoughts on this, I'd be glad to read them.

          3 votes
          1. Happy_Shredder
            Link Parent
            Mmm, I see. Any solution that's simple for non devs but still more powerful than s/.../.../g is challenging. Working with pandoc seems sensible, and lua is nice too. With pandoc you have the ast...

            Mmm, I see. Any solution that's simple for non devs but still more powerful than s/.../.../g is challenging.

            Working with pandoc seems sensible, and lua is nice too. With pandoc you have the ast so you can do anything, although details may be difficult.

            Because Lisps have very little syntax you can get creative and strip out a lot of complexity. For example, $(foo) could be for a simple find and replace. $(foo > bar > baz) could access children in the input/ast. $(fn (foo)) could further process foo with fn. Maybe it's useful to be able to set and reference variables in the document? I think some structured input makes more sense. Key value pairs, flow control, and hierarchy (or namespaces) should go most of the way. So you could invent a dsl for this, but with a nice grammar you can now take yaml, toml whatever defining variables and have a really slick structured interpolation system.

            It's certainly the sort of thing where I'd want a corpus of examples to influence and constrain design. It's hard for me to imagine any serious details. (e.g. Keep everything in one doc and process that vs combining arbitrary input with a template.)

            2 votes
  3. [3]
    Keegan
    Link
    Not answering your question, because I don't know, but Scrivenvar looks very nice. I'll have to use it at some point.

    Not answering your question, because I don't know, but Scrivenvar looks very nice. I'll have to use it at some point.

    1 vote
    1. [2]
      DaveJarvis
      Link Parent
      If you do, be sure to read about the hot-keys on the usage page, in particular ctrl+space to auto-insert variables into the source document.

      I'll have to use it at some point.

      If you do, be sure to read about the hot-keys on the usage page, in particular ctrl+space to auto-insert variables into the source document.

      3 votes