Five Unusual Raku Features

Junctions, whatevers, hypers, and more!

Logic for Programmers is now in Beta!

v0.5 marks the official end of alpha! With the new version, all of the content I wanted to put in the book is now present, and all that's left is copyediting, proofreading, and formatting. Which will probably take as long as it took to actually write the book. You can see the release notes in the footnote.¹

And I've got a snazzy new cover:

(I don't actually like the cover that much but it looks official enough until I can pay an actual cover designer.)

"Five" Unusual Raku Features

Last year I started learning Raku, and the sheer bizarreness of the language left me describing it as a language for gremlins. Now that I've used it in anger for over a year, I have a better way of describing it:

Raku is a laboratory for language features.

This is why it has five different models of concurrency and eighteen ways of doing anything else, because the point is to see what happens. It also explains why many of the features interact so strangely and why there's all that odd edge-case behavior. Getting 100 experiments polished and playing nicely with each other is much harder than running 100 experiments; we can sort out the polish after we figure out which ideas are good ones.

So here are "five" Raku experiments you could imagine seeing in another programming language. If you squint.

Junctions

Junctions are "superpositions of possible values". Applying an operation to a junction instead applies it to every value inside the junction.

> 2|10
any(2, 10)

> 2&10 + 3
all(5, 13)

>(1&2) + (10^20)
all(one(11, 21), one(12, 22))

As you can probably tell from the alls and anys, junctions are a feature meant for representing boolean formula. There's no way to destructure a junction, and the only way to use it is to collapse it to a boolean first.

> (1&2) + (10^20) < 15
all(one(True, False), one(True, False))

# so coerces junctions to booleans
> so (1&2) + (10^20) < 15
True

> so (1&2) + (10^20) > 0
False

> 16 %% (3&5) ?? "fizzbuzz" !! *
*

The real interesting thing for me is how Raku elegantly uses junctions to represent quantifiers. In most languages, you either have the function all(list[T], T -> bool) or the method [T].all(T -> bool), both of which apply the test to every element of the list. In Raku, though, list.all doesn't take anything, it's just a niladic method that turns the list into a junction.

> my $x = <1 2 3>.all
all(1, 2, 3)
> is-prime($x)
all(False, True, True)

This means we can combine junctions. If Raku didn't already have a unique method, we could build it by saying "are all elements equal to exactly one element?"

> so {.all == .one}(<1 2 3 7>)
True

> so {.all == .one}(<1 2 3 7 2>)
False

Whatevers

* is the "whatever" symbol and has a lot of different roles in Raku.² Some functions and operators have special behavior when passed a *. In a range or sequence, * means "unbound".

> 1..*
1..Inf

> (2,4,8...*)[17]
262144

The main built-in use, though, is that expressions with * are lifted into anonymous functions. This is called "whatever-priming" and produces a WhateverCode, which is indistinguishable from other functions except for the type.

> {$_ + 10}(2)
12

> (* + 10)(2)
12

> (^10).map(* % 2)
(0 1 0 1 0 1 0 1 0 1)

There's actually a bit of weird behavior here: if two whatevers appear in the expression, they become separate positional variables. (2, 30, 4, 50).map(* + *) returns (32, 54). This makes it easy to express a tricky Fibonacci definition but otherwise I don't see how it's better than making each * the same value.

Regardless, priming is useful because so many Raku methods are overloaded to take functions. You get the last element of a list with l[*-1]. This looks like standard negative-index syntax, but what actually happens is that when [] is passed a function, it passes in list length and looks up the result. So if the list has 10 elements, l[*-1] = l[10-1] = l[9], aka the last element. Similarly, l.head(2) is the first two elements of a list, l.head(*-2) is all-but-the-last-two.

We can pass other functions to [], which e.g. makes implementing ring buffers easy.

> my @x = ^10
[0 1 2 3 4 5 6 7 8 9]

> @x[95 % *]--; @x
[0 1 2 3 4 4 6 7 8 9]

Regular Expressions

There are two basic standards for regexes: POSIX regexes and Perl-compatible regexes (PCRE). POSIX regexes are a terrible mess of backslashes and punctuation. PCRE is backwards compatible with POSIX and is a more terrible mess of backslashes and punctuation. Most languages follow the PCRE standard, but Perl 6 breaks backwards compatibility with an entirely new regex syntax.

The most obvious improvement: composability. In most languages "combine" two regexes by concating their strings together, which is terrible for many, many reasons. Raku has the standard "embed another regex" syntax: /< foo >+/ matches one-or-more of the foo regex without foo "leaking" into the top regex.

This already does a lot to make regexes more tractable: you can break a complicated regular expression down into simpler and more legible parts. And in fact this is how Raku supports parsing grammars as a builtin language feature. I've only used grammars once but it was quite helpful.

Since we're breaking backwards compatibility anyway, we can now add lots of small QOLs. There's a value separator modifier: \d+ % ',' matches 1 / 1,2 / 1,1,4 but not 1, or 12. Lookaheads and non-capturing groups aren't nonsense glyphs. r1 && r2 only matches strings that match both r1 and r2. Backtracking can be stopped with :. Whitespace is ignored by default and has to be explicitly enabled in match patterns.

There's more stuff Raku does with actually processing regular expressions, but the regex notation is something that might actually appear in another language someday.

Hyperoperators

This is a small one compared to the other features, but it's also the thing I miss most often in other languages. The most basic form l>>.method is basically equivalent to map, except it also recursively descends into sublists.

> [1, [2, 3], 4]>>.succ
[2 [3 4] 5]

This is more useful than it looks because any function call f(list, *args) can be rewritten in "method form" list.&f(*args), so >>. becomes the generalized mapping operator. You can use it with whatevers, too.

> [1, [2, 3], 4]>>.&(*+1)
[2 [3 4] 5]

Anyway, the more generalized binary hyperoperator l1 << op >> l2³ applies op elementwise to the two lists, looping the shorter list until the longer list is exhausted. >>op>> / << op<< are the same except they instead loop until the lhs/rhs list is exhausted. Whew!

> [1, 2, 3, 4, 5] <<+>> [10, 20]
[11 22 13 24 15]

> [1, 2, 3, 4, 5] <<+<< [10, 20]
[11 22]

> [1, 2, 3, 4, 5] >>+>> [10, 20]
[11 22 13 24 15]

# Also works with single values
> [1, 2, 3, 4, 5] <<+>> 10
[11 12 13 14 15]

# Does weird things with nested lists too
> [1, [2, 3], 4, 5] <<+>> [10, 20]
[11 [22 23] 14 25]

Also for some reason the hyperoperators have separate behaviors on two hashes, either applying op to the union/intersection/hash difference.

Anyway it's a super weird (meta)operator but it's also quite useful! It's the closest thing I've seen to J verbs outside an APL. I like using it to run the same formula on multiple possible inputs at once.

(20 * 10 <<->> (21, 24)) <<*>> (10, 100)
(1790 17600)

Incidentally, it's called the hyperoperator because it evaluates all of the operations in parallel. Explicit loops can be parallelized by prefixing them with hyper.

Pair Syntax

I've talked about pairs a little in this newsletter, but the gist is that Raku hashes are composed of a set of pairs key => value. The pair is the basis type, the hash is the collection of pairs. There's also a ton of syntactic sugar for concisely specifying pairs via "colon syntax":

> my $x = 3; :$x
x => 3

> :a<$x>
a => "$x"

> :a($x)
a => 3

> :3a
a => 3

The most important sugars are :key and :!key, which map to key => True and key => False. This is a really elegant way to add flags to a methods! Take the definition of match:

method match($pat, 
    :continue(:$c), :pos(:$p), :global(:$g), 
    :overlap(:$ov), :exhaustive(:$ex), 
    :st(:$nd), :rd(:$th), :$nth, :$x --> Match)

Probably should also mention that in a definition, :f(:$foo) defines the parameter $foo but also aliases it to :f, so you can set the flag with :f or :foo. Colon-pairs defined in the signature can be passed in anywhere, or even stuck together:

> "abab".match(/../)
｢ab｣
> "abab".match(/../, :g)
(｢ab｣ ｢ab｣)
> "abab".match(/../, :g, :ov)
(｢ab｣ ｢ba｣ ｢ab｣)

# Out of order stuck together
> "abab".match(:g:ov, /../)
(｢ab｣ ｢ba｣ ｢ab｣)

So that leads to extremely concise method configuration. Definitely beats match(global=True, overlap=True)!

And for some reason you can place keyword arguments after the function call:

> "abab".match(:g, /../):ov:2nd
｢ba｣

The next-gen lab: Slangs and RakuAST

These are features I have no experience in and certainly are not making their way into other languages, but they really expand the explorable space of new features. Slangs are modifications to the Raku syntax. This can be used for things like modifying loop syntax, changing identifiers, or adding actors or DNA sequences to the base language.

I barely understand RakuAST. I think the idea is that all Raku expressions can be parsed as an AST from inside Raku itself.

> Q/my $x; $x++/.AST
RakuAST::StatementList.new(
  RakuAST::Statement::Expression.new(
    expression => RakuAST::VarDeclaration::Simple.new(
      sigil       => "\$",
      desigilname => RakuAST::Name.from-identifier("x")
    )
  ),
  RakuAST::Statement::Expression.new(
    expression => RakuAST::ApplyPostfix.new(
      operand => RakuAST::Var::Lexical.new("\$x"),
      postfix => RakuAST::Postfix.new("++")
    )
  )
)

This allows for things like writing Raku in different languages:

say Q/my $x; put $x/.AST.DEPARSE("NL")
mijn $x;
zeg-het $x

Bonus experiment

Raku comes with a "Rakudo Star" installation, which comes with a set of blessed third party modules preinstalled. I love this! It's a great compromise between the maintainer burdens of a large standard library and the user burdens of making everybody find the right packages in the ecosystem.

Blog Rec

Feel obligated to recommend some Raku blogs! Elizabeth Mattijsen posts a ton of stuff to dev.to about Raku internals. Codesections has a pretty good blog; he's the person who eventually got me to try out Raku. Finally, the Raku Advent Calendar is a great dive into advanced Raku techniques. Bad news is it only updates once a year, good news is it's 25 updates that once a year.

If you're reading this on the web, you can subscribe here. Updates are once a week. My main website is here.

My new book, Logic for Programmers, is now in early access! Get it here.