I'm writing a simple org-mode parse from scratch in Python https://codeberg.org/ordinarycoder/org-parser. It's rough, and there are a lot of things a want to change, but it works, and it's a start.
The biggest reason that I'm doing this, is that it's fun. I always find programming languages and compilers fascinating. Sadly, I never had the chance to dig compiler design during my study. Now, 10 years after graduation, I've finally decided to work on something I'm truly interested in as a side project.
There's is also a practical aspect for this project. As a long time Emacs user, the blog posts in this website are originally written in org-mode. Currently, I'm using Pandoc to convert from org-mode to html. It works, but it's not perfect. I couldn't probably have configure Pandoc to work the way I wanted, but I didn't do it because I've set a goal for myself, I want to create this website from scratch, as much as possible. Pandoc is only and easy, temporary solution, that allows me to publish something, while working on a real solution in the meantime. It's written in Python because I'm not good at Elisp, even after all these years of using Emacs.
For this first iteration, I did it mostly from what I remember from uni. I didn't look up how to write a parser properly. Turns out, I still remember quite a few things! I remember context free grammar, with the production rules and so on. So I first write a EBNF grammar (this part I did look up how to do) for the org-mode features that I want to support in this iteration. And then I just tried to code following the grammar. I find creating the grammar to be the crucial step. Because the grammar is easier to understand than the code, creating the grammar first is like a kind of separation of concerns. I can easier check if a bug comes from the grammar not matching the org-mode spec, or from some logic errors in the code.
Okay, so what does this parser actually do? It accepts a string in org-mode format, and outputs the abstract syntax tree as a dictionary. Currently, it supports headings or arbitrary depth, properties, and preamble. That's it, that's the whole thing. It's not much, but it's a starting point to support what I needed for this website.
This is the EBNF that I base parser on.
document = properties, keyword*, section, heading;
properties = ":PROPERTIES:", "\n"+, property*, ":END:", "\n"+ | "";
property = ":", name ,":", " "+, value, "\n"+;
name = text;
value = text;
keyword = "#+", key, ":", " "*, value+, "\n"+;
key = text;
heading = headline, section
headline = "*"+, " "+, text?, "\n"+;
section = paragraph*;
paragraph = ( ( "" | text ) - "* " )+, "\n"+;
There are a lot of thing I want to improve. For example, naming of the fields, structure of the AST. And of course, support more org-mode features.