Working on Pharo Smalltalk: BPatterns: Rewrite Engine with Smalltalk Style

This is roughly the same problem as syntactical macros in non-Lisp syntax languages.

There needs to be a way to indicate a "hole" (metavariable/unquote) in the syntax tree, and depending on the complexity of the language's grammar, that might be somewhat difficult, eg. in C++ having a hole for a type declaration runs into the common ambiguity between declarations (constructor calls) and expressions (regular calls). This needs to be worked around by eg. having multiple types of holes to disambiguate...

For the article, the idea of using an "any" prefix on identifiers instead of eg. operators such as ` , ... (Lisp macros) is an interesting solution, as it does not require extending the grammar of the language... although it's not applicable in all situations (eg. for matching grammar elements where identifiers are not allowed). For a very regular language like Smalltalk though, it's pretty good.

Grammar-based rewriting has a long line of history and research, so there's a deep well of knowledge to be mined if you feel like hitting up Google Scholar...

For modern implementations, there's eg. ast-grep and semgrep: https://ast-grep.github.io/ https://semgrep.dev/docs/writing-rules/pattern-syntax

This is so damn cool. And most people won’t even see why.

They won’t realize just how amazingly cool it is that you can grab any block of code in a block closure (the [ ] syntax*), and extend that object with more behavior. And how if you have that object you can I introspect the craziest things out of it, including of course, the source it was derived from, and hence, an AST. There’s no need to make a new compiler pass, or a special tool mod. The “reserved” keyword of any doesn’t have to be bolted into the whole language, it’s just a variable that can be treated specially in this context, and only this context, so you don’t have to have special meta variables leak into the whole system.

* disclaimer: I participated in the inclusion of John and Dons excellent work into the VisualWorks IDE back in the day. I also saw “once” introduced into the Smalltalk system, a lazy runtime way of memoizing block values in place. Oh the things you could do with BlockClosure extension methods.

* aside, my favorite closure syntax of all time, way better than elixirs shorthand & stuff, or fn -> things, or trailing closures

Honestly, as much as I love Smalltalk, I see this post as an illustration of all that's wrong in that community. Instead of documenting the original solution well, making a cheat sheet for it, or gathering examples (which you otherwise need to "hunt" for, according to the author) in one place, we're given another solution on top of the original inscrutable one. That new solution is still undocumented; in the post, there's no explanation of how it was written and how it works; instead, we get another handful of examples.

GToolkit has Lepiter, and Smalltalks in general have comments on classes that can be used to write long-form docs. Yet, the majority of Smalltalkers apparently believe that all that is not needed, and that searching for examples in the codebase gives you all the information you need. It may be true - not in all cases, but some of them - but the efficiency of that process is incredibly low. And that's even if the examples are there - mostly in test code. It's not unusual to find large areas of code that are entirely untested, though, in which case something that could be done in 15 minutes if you were provided with a solid README can take days to figure out.

Yeah, I know: the IDE, the Smalltalk environment, IS very valuable and it DOES give you superpowers. That doesn't fully offset the issue of notorious lack of documentation, though, and in effect, you're still less productive than you could be with Python. GToolkit's Lepiter is a step in the right direction, but even in GT, class comments and method docstrings are sparse. GToolkit-like environment coupled with Emacs Lisp self-documenting docstrings culture would be ideal, but somehow it just won't happen.