Below are 3 examples, that show some of the principles of working with Rye language. But first, some terminology.
Code is data / data is code
Definition: A language is homoiconic if a program written in it can be manipulated as data using the language.
Lisps, schemes, REBOL and Rye are homoiconic, but what does this actually benefit us?
After two decades of programming in such languages, it’s hard to answer this quickly. After a while, you don’t think about or even notice the homoiconicity, it just seems the way things should be anyway. Try asking a fish to sell you on the idea of living in water.
Why would I express my thoughts about the world with a substrate, that can’t also expressing thoughts about the substrate itself?
While Lisps emphasize the ‘code is data / data is code’ paradigm at compile time through macros, REBOL and Rye usually leverage this concept dynamically at runtime. And it’s not just about code. It’s about all live constructs of the language being fully accessible to the language itself.
I’m not sure it’s correct, but when explaining the difference I often used the phrase runtime homoiconic.
Common practices still apply
Homoiconicity affects the way you think about your program, but it affects the macro view, the approach, the way you declare the problem. On the level of code, good practices, like those from functional programming, still fully apply.
For example, you really don’t want to manipulate a block of data by injecting a bunch of words between values and evalating that as code. But you could, and you probably did at some point early on.
; don't do this!
data: { "don't" "do" "this" }
; in Rye you can't mutate blocks that easily anyway
; so we basically have to generate a new block
data .map { .concat* { print } } |unpack :code |dump |print
; prints:
; { print "don't" print "do" print "this" }
do code
; prints:
; don't
; do
; this
Later you see any creative manipulation of code as a strong code smell. Except for specific cases, where you explicitly do want to generate code. Normal approach, application of functions, solves things in cleaner and more robust way.
; still, just do this
data: { "still" "just do" "this" }
; a simple for loop should do
data .for { .print }
; prints:
; still
; just do
; this
But there are still examples where this runtime homoiconicity shows some (positive) teeth. I will try to show a few of them.
Detour: REBOL’s parse DSL
We don’t have REBOL’s parse function/DSL in Rye for now. It’s a well known awesome thing, so we test nothing by adding it at this phase, and the lack of it forces us to think creatively around it. One danger of having parse is, that you then try use it for everything.
And parse is a language on it’s own, in REBOL we had parse gurus, that could do magic with it. Red, the successor of Rebol, took parse
and still made it even more powerful. Check out these, if you are interested:
REBOL docs, Red blogpost, Red by example, REBOL docs 2
; A REBOL and Red example that validates math expressions
expr: [term ["+" | "-"] expr | term]
term: [factor ["*" | "/"] term | factor]
factor: [primary "**" factor | primary]
primary: [some digit | "(" expr ")"]
digit: charset "0123456789"
parse "1+2*(3-2)/4" expr ; will return true
parse "1-(3/)+2" expr ; will return false
Parse is used for parsing strings, binary input and most importantly also blocks of REBOL values (the REBOL data/code). It’s also the main tool for creating DSLs.
rule: [
set action ['buy | 'sell]
set number integer!
'shares 'at
set price money!
(either action = 'sell [
print ["income" price * number]
total: total + (price * number)
][
print ["cost" price * number]
total: total - (price * number)
]
)
]
total: 0
parse [sell 100 shares at $123.45] rule
print ["total:" total]
And parse itself is a DLS (dialect) … cool right :)
Example 1: xmlprint DLS
At my job, I needed to create SEPA XML documents. Especially the various design-by-committe XML standards can be quite verbose, and these managed also to be cryptic with it’s naming scheme.
One of schemas I was working on is even called pain.008.001.02 🤷
Previous solution
In the past I created (REBOL) DSLs to generate XML using words and blocks with a more conventional structure:
subscribers {
person { 'id 10
name "Elanor"
surname "Brandyfoot"
}
person { 'id 11
name "Poppy"
surname "Proudfellow"
}
}
that would generate
<subscribers>
<person id="10">
<name>Elanor</name>
<surname>Brandyfoot</surname>
</person>
<person id="11">
<name>Poppy</name>
<surname>Proudfellow</name>
</person>
</subscribers>
But with current XML schema, I thought the least error prone way to turn it into code was to just copy whole example and work on that, not really touching or writing the cryptic tagnames by myself (see for yourself xml schema).
The DLS
Rye already had two word types that looked like XML starting and ending tag (Xword and EXword ) and which I could use for this.
So I came up with this simple DSL. Note that template is not a string, but a regular block of Rye values / code. In this case it consists of xwords, exwords, blocks and strings.
template: {
<data>
<header>
<event> ( header/name ) </event>
"Date of the event"
<date> ( header/date ) </date>
<type>
<subtype> ( header/type ) </subtype>
</type>
</header>
<person>
<fullname> ( .fullname ) </fullname>
<name type="first"> ( -> "name" ) </name>
</person>
</data>
}
fullname: fn { d } { join\with [ d -> "name" d -> "surname" ] " " }
header: context { name: "Crowning" date: now type: "X" }
xmlprint\full dict { name: "Durin" surname: "Longbeard" } template
xwords and exwords define the structure of the document, blocks get evaluated (with data argument injected into a block) and strings get turned into comments.
I started with creating this DSL first, and only then thinking on how we could evaluate it.
The evaluator
I could create evaluator in Go like all Rye’s DSLs are implemented now, but in REBOL or Red we would use the aswesome parse function I demoed above, and which we don’t have.
So I added a rather simple walk built-in function to Rye.
; private function creates a context evaluates the code within context
; but just returns the result of the code. It can be used to create pockets
; of specialised code that don't spill out
private {
is-not: fn { a b } { .to-word = to-word b |not }
comment: fn1 { .concat* "<!--" |concat "-->" }
process: fn\par { block values current } current {
|pass { blk: false token: false }
|walk { ::blk
|peek ::token .type? .switch {
xword {
prn token
process next blk values token
}
exword {
prn token
if token .is-not current { fail "tag mismatch" }
next blk
}
block {
with values token |prn
next blk
}
string {
prn comment token
next blk
}
}
}
newline true
}
; Only this function will get returned
fn\par { values block "Accepts dict of values and a block of dialect" }
current { process block values 0 'no }
} :xmlprint
xmlprint\full: fn { data tpl } {
print `<?xml version="1.0" encoding="UTF-8" ?>`
xmlprint data tpl
}
And we can call the script:
-mycomp-$ rye example1-micro.rye
<?xml version="1.0" encoding="UTF-8" ?>
<data><header><event>Crowning</event><!--Date of the event--><date>2025-03-02 17:02:
34</date><type><subtype>X</subtype></type></header><person><fullname>Durin Longbeard
</fullname><name type="first">Durin</name></person></data>
Created XML is without indentation, which has it’s benefits. You can find a few lines longer example that properly indents XML among examples: examples/xmlprint.
DSLs vs. contexts
In REBOL we would use parse DSL to create a DSL above, but as you can see, using just regular Rye, with a walk function worked too.
Parse might have been shorter and it would be able to express more complex patterns without us needing to create a state-machine. But using regular Rye also has benefits. For one, we don’t have to learn another set of rules of evaluation, and we can re-use whole Rye, or just parts of it.
Rye tries to rely much more on smart use of contexts. Contexts (scopes) are first class values in Rye and we can do a lot with them, but we don’t change the rules of evaluation, which are quite rich with just Rye already.
Example 2: testing DSL
I needed a testing tool for Rye. Again I first started thinking about “what is the best way to declare the tests” and only then, how I would evaluate that structure.
The tests
I would like to group tests by name, which will most often be the name of the function I’m testing. I don’t want to name individual tests, because I will just write nonsense there after a while.
I mostly want to test if an expression returns a specific value (equal). I also want to compare stdout (what code test prints out) to a specific value and I want to be able to define that some expressions should return an error.
This is the most minimal way of declaring that I could concoct:
group "math" {
equal { 10 + 101 } 111
equal { 123 / 20 } 6.125
equal { 123 % 20 } 3
error { "11" + 22 }
}
group "concat" {
equal { "Cele" .concat "brimbor" } "Celebrimbor"
equal { concat 3 + 7 + 9 " rings" } "19 rings"
equal { { 3 7 } .concat 9 } { 3 7 9 }
equal { "RON" .concat* "SAU" } "SAURON"
}
group "looping" {
stdout { loop 3 { prn "Xo" } } "XoXoXo"
stdout { { 1 2 3 } .for { .prns } } "1 2 3 "
error { 33 .for { .prns } }
}
The Rye code above could be seen as Rye data. Some words, blocks of code and some literal values, and we could create a special evaluator for it, either in Go or Rye.
But group could also just be a function taking two arguments, a name of the group and a block of code.
equal and stdout could be two functions accepting a block of code and a value to compare the result or standard output to.
And error takes just one argument, a block of code that should fail.
The context
Our “DSL” (which it is not) is evaluated as a regular Rye in a specific context. This context has the four functions we mentioned above. Most of the code is basically there to print the expected and received values in case of a failure.
test-framework: context {
group: fn { name code } { prns name , do code , print "" }
equal: fn { test res } {
do test :got = res |either
{ prns "✓" }
{ prns " ✗ Failed:"
prns join { "expected " inspect res ", got " inspect got } }
}
stdout: fn { test res } {
capture-stdout { do test } :got = res |either
{ prns "✓" }
{ prns " ✗ Failed:"
prns join { "expected " inspect res ", got " inspect got } }
}
error: fn { test } {
try test :got .type? = 'error |either
{ prns "✓" }
{ prns " ✗ Failed:"
prns join { newline "expected error but got: " inspect got } }
}
}
%demo.micro.rye .load .do\in* test-framework
To look closer, function group takes two arguments a name of the group and a block of code it prints the name and does (evaluates) the code.
group: fn { name code } { prns name , do code , print "" }
Function equal also takes two arguments a block of code (a test expression) and the expected result.
equal: fn { test res } {
; does the block of code (test), stores result to word got and compares
; it to expected result
do test :got = res |either
; if they match it displays a checkmark
{ prns "✓" }
; if not it displays and cross and expected and recieved values
{ prns " ✗ Failed:"
prns join { "expected " inspect res ", got " inspect got } }
}
stdout and error functions are not that different. They mostly rely on existing functions capture-stdout and try.
Results
When all test pass we get something like this:
math ✓ ✓ ✓ ✓
concat ✓ ✓ ✓
looping ✓ ✓ ✓
And if we change some tests so they will fail:
; just the changed lines
error { 11 + 22 }
equal { "RON" .concat "SAU" } "SAURON"
stdout { loop 2 { prn "Xo" } } "XoXoXo"
we get:
math ✓ ✓ ✓ ✗ Failed: expected error but got: [Integer: 33]
concat ✓ ✓ ✓ ✗ Failed: expected [String: SAURON], got [String: RONSAU]
looping ✗ Failed: expected [String: XoXoXo], got [String: XoXo] ✓ ✓
We use the same code, but with some improvements and colors to do the normal tests and also generate reference documentation for the website. You can see that code in: info/main.rye
Our tool in action:
The more cautious will probably ask themselves “what if such DSLs cause naming conflicts?”. We won’t go there in this blogpost, but functions like
do\par, fn\par, extend and bind come into play there. In short, each code or function in Rye can be evaluated in a custom chain of contexts.
Example 3: Buttons GUI app
This post is getting extra long, so in this example I will let the code and a screenshot do most of the talking.
Since Rye can navigate and use live contexts (this is also used well in Rye console) I wanted to combine this feature with GUI-s somehow.
The first idea was to make an app that takes a context and displays a button for each function. If you press the button, the function should be called and the output of the function displayed in the window. I’m using Rye-fyne for this. Fyne is a GUI library for Go. The bindings for it were autogenerated using ryegen. So many links … :P
The code below takes a rye file with functions as CLI argument and creates the GUI.
rye .needs { fyne }
myctx: context { do load to-file first rye .args }
do\par fyne {
win: app .window "Buttons app"
gw: grid-wrap size 120.0 36.0 { }
spl: h-split gw lbl: label "" ,
spl .offset! 0.6
for lc\data\ myctx { ::w
extends myctx { ; we evaluate this in local contect
fnc: ?w ; to prevent captured variables problem
gw .add button to-string w ; and make it's parent myctx
closure { }
{ set-text lbl capture-stdout [ fnc ] }
}
}
with win {
.resize size 450.0 350.0 ,
.set-content spl ,
.show-and-run
}
}
This is our sample file with Rye functions.
hello: does { print "Hello world!" }
time: does { now .print }
load-avg: does { os/load-avg? .print }
my-ip: fn { } {
get https://api.ipify.org?format=json
|parse-json -> "ip" |print
}
joke: does {
get https://official-joke-api.appspot.com/random_joke
|parse-json
|with { -> "setup" |print , -> "punchline" |print }
}
And this is the result.
I have ideas to extend this and maybe make it useful, or at least interesting. :)
Bravo!
Hats off, if you read all the way to the end.
If this got you interested in Rye, visit our website’s main page, or documentation that is still being written, like Meet Rye or the function reference that is generated from the tests similar to our Example 2.
If you want more long form content about Rye check out these two Cookbook pages:
Thanks!
To @friedcell for finding multiple typos.
And:
“A star on Github never hurt nobody”