As I reported recently, there seems to be a mutual acceptance of raku and perl in the wake of the name change and a bit of time passing:

This is a reproduction of the talk I gave today at the awesome London Perl & Raku Workshop. I met a couple of cool rakuteers and enjoyed the relaxed congregation of perl and raku folk. Seems like the anger has subsided not least thanks to the name change from perl6 to raku. Thanks to all the sponsors, organisers and attendees.

One of the questions I was asked at the workshop by some professional perl coders was “what is Raku and why should I use it?”. I was quite flustered by the question (despite having prepared for it the day before).

• What I should have said:
  • raku is awesome for LLM – just google raku LLM
• What I actually said:
  • raku Grammars are a unique feature … suggest you explore that
• What I truly think:
  • in business, the best use cases for raku are for ad hoc scripts, data munging, migration and CLI stuff (just like perl was back in the day) – and maybe a bit of LLM prompt engineering

OK – I’m gonna unpack that last statement a bit. By best, I mean that raku is still not really used at scale or for large projects by large teams. There are many things to recommend raku for these kind of use cases – concurrency, strong types (optional), role-based composition, unicode and so on. Stability, bug stomping and performance are improving all the time.

But, in addition to great features, any innovative language has to be proven before a real business will be happy to commit to it as a fundamental technical building block. Businesses are, rightly, risk averse. They will consider aspects such as availability of skilled staff, core team bus factor, ecosystem health, etc. So best, for me, means most practical, most likely to succeed, most added value when real-world constraints are applied.

WordPress

I run a web design consultancy – we focus on WordPress and other PHP based technologies. Have I rewritten WordPress in raku? No indeed. But, since I am an all in rakuteer I have effectively used raku to streamline our business processes:

• Used raku XML to extract, transform and load 10,000s of ancient PDF files only accessible via a dying web frontend into OJS
• Used raku CLI::AWS::EC2-Simple to manage a suite of AWS EC2 instances with regard to terminating/relaunching EC2 servers quickly to save cost (and to tune instance types up and down) – as posted here.
• Used raku CLI::Wordpress to create WordPress hosting sites on AWS EC2 … launch a new instance of WordPress & setup ssl certificate with three commands:- rawp setup && rawp launch && rawp renewal. This short performs nginx install, database install, WordPress install, TLS certificate generation, cron for certificate renewal – yes all that
• Used raku CLI::Wordpress::Migrator to migrate production WordPress sites from AWS EC2 to another hosting provider (kualo.com since you ask) – so db and file backup, download, upload, install, restore, search-replace all with rawm migrate (and a suitable yaml config)

Why do I write and share these modules as FOSS?

I like to code in raku – it is truly a pleasure. When I assemble and prove out a set of CLI commands to do a task, I am thinking “this is cool, how can I capture this recipe and run it automatically every time” (ie. I a way to remember what works and refine it). And I hope that by sharing others will be able to benefit from these potted scripts and may wish to extend and refine them in turn.

CLI::Wordpress::Migrator

In common with the other raku modules listed above, this one works like this:

• read config.yaml
• connect to a server
• upload a perl script to the server
• go cat xxx.pl | perl to run it
• control and monitor the perl with raku on the local machine

CLI::Wordpress::Migrator is a script to migrate a WordPress site from one server to another. This performs export (backup), install, import (restore) and search-replace steps according to the configuration in ~/.rawm-config/rawm-config.yaml

This module installs the raku rawm command for use from the local client command line. (RAku WordPress Migrator).

The process involves three systems:

• from server which is running the source site
• to server which is ready for the migrated site
• local client which connects to the export / import servers in turn

Here’s the raku MAIN usage:

> rawm
Usage:
  ./rawm [--ts=<Str>] [--backup-only] [--download-only] [--upload-only] [--restore-only] [--cleanup-only] [--dry-run] <cmd>
  
    <cmd>              One of <connect export install import search-replace migrate>
    --ts=<Str>         Enter timestamp of files [Str] eg. --ts='20241025-17-02-42'
    --backup-only      Only perform remote backup
    --download-only    Only perform download (requires timestamp [--ts])
    --upload-only      Only perform upload (requires timestamp [--ts])
    --restore-only     Only perform restore (requires timestamp [--ts])
    --cleanup-only     Only perform remote cleanup
    --dry-run          Do not perform replace

Here’s the (sanitised) yaml:

from:
  user: myusername
  subdom: sdname
  domain: mydomain.com
  key-pub: kpname
  port: 22
to:
  user: myusername
  subdom: sdname
  domain: mydomain.com
  key-pub: kpname
  port: 22
wp-config:
  locale: en_GB
  db:
    name: dbname
    user: dbuser
    pass: dbpass
    prefix: wp_
  title: My New WP Installation
  url: mysite.com
  admin:
    user: aduser
    pass: adpass
    email: ademail

Under The Hood

Here’s the perl code to perform the remote backup:

method perl {
    
my $code = q:to/END/;
    #!/usr/bin/perl
    use strict;
    use warnings;

    print "Doing remote backup at %NOW%\n";

    `wp db --path='../%WP-DIR%' export %BU-DB-FN%`;
    `tar -czf %BU-FS-FN% ../%WP-DIR%/wp-content`;
END

    $code ~~ s:g/'%NOW%'     /{ $.timestamp}/;
    $code ~~ s:g/'%BU-DB-FN%'/{ $.bu-db-fn }/;
    $code ~~ s:g/'%BU-FS-FN%'/{ $.bu-fs-fn }/;
    $code ~~ s:g/'%WP-DIR%'  /{ $.server.wp-dir }/;
    $code
}

And here’s the raku code that runs it:

method backup {
    my $s := $.server;

    my $proc = Proc::Async.new:
         :w, qqw|ssh -p { $s.port } -tt -i { $s.key-path } { $s.login }|;

    my $promise = $proc.start;

    $proc.say("mkdir { $s.tp-dir }");
    $proc.say("cd { $s.tp-dir }");

    $proc.say("echo \'{ $.perl }\' > exporter.pl");
    $proc.say('cat exporter.pl | perl');

    sleep 30;

    $proc.say("exit");
    await $promise;
}

This is a snippet of the source code at https://github.com/librasteve/raku-CLI-Wordpress-Migrator if you want to see the full story.

You are welcome to review it all, fork, PR any changes you would like and use it to manage your WordPress estate!

Why perl? Why raku?

If you have been paying close attention to my raku module history you will know that often I have the opportunity to install raku on the remote machine and to run that for various tasks (i.e. raws-ec2 setup). But in the case of Migrator the idea is to backup and restore remote machines hosted by a third party firm running cPanel and with tight restrictions on installing non standard languages. Happily, as with pretty much all modern Linux installs, their hosted systems are preloaded with perl. So automating the process of logon, save generated perl file on the remote system and run it is very widely applicable.

Meantime, driving this with raku is a very natural choice. Raku features that facilitate this are:

• simple out of the box command line execution
• easy OO that can marshal cmd context and yaml config
• easy shell out via qqx
• regex text substitution to generate custom perl
• async framework for coupling to remote processes

Ultimately, technically, perl and raku are very complementary – combining the ubiquity of perl with the expressivity of raku to produce a practical outcome. And both have a familiar look and feel…

As usual, comments and feedback very welcome!

~librasteve

I published my new book: A Language a Day, which is a collection of brief overviews to 21 programming languages.

This book provides a concise overview of 21 different programming languages. Each language is introduced using the same approach: solving several programming problems to showcase its features and capabilities. Languages covered in the book: C++, Clojure, Crystal, D, Dart, Elixir, Factor, Go, Hack, Hy, Io, Julia, Kotlin, Lua, Mercury, Nim, OCaml, Raku, Rust, Scala, and TypeScript.

Each chapter covers the essentials of a different programming language. To make the content more consistent and comparable, I use the same structure for each language, focusing on the following mini projects:

1. Creating a ‘Hello, World!’ program.
2. Implementing a Factorial function using recursion or a functional-style approach.
3. Creating a polymorphic array of objects (a ‘zoo’ of cats and dogs) and calling methods on them.
4. Implementing the Sleep Sort algorithm—while impractical for real-word use, it’s a playful demonstration of language’s concurrency capabilities.

Each language description follows—where applicable—this pattern:

1. Installing a command-line compiler and running a program.
2. Creating and using variables.
3. Defining and using functions.
4. Exploring object-oriented features.
5. Handling exception.
6. Introducing basic concurrency and parallelism.

You can find all the code examples in this book on GitHub: github.com/ash/a-language-a-day.

You can buy it on Amazon or LeanPub as an electronic or Kindle edition, or as a paper hardcover or paperback version. More information with the links to the shops.

About 3 weeks ago I thought it was time to go through the outstanding Rakudo compiler issues (an implementation of the Raku Programming Language) to see how many of them would have been fixed by now with the new Raku grammar.

Why? Because we have reached 85+% of test coverage of the new Raku Grammar in roast, the official Raku test-suite.

Success as defined by the number of test files that completely pass without any test failures. Most othertest files also have tests passing, but they're not 100% clean.

At that point there were 1312 open Rakudo issues, with the oldest being from 2017.

There are now 778 issues left open. So you could say that's 534 issues closed. Actually, it is a few more as during this 3 week period, a few new issues were posted.

In this blog post I'll be describing the types of issues that I've seen, and how I handled them.

JVM backend specific

Sadly, there's not a lot I could do at those specific issues, as my JVM foo is close to zero. There are currently 22 open issues for the JVM-backend. If you have any JVM chops, it would be greatly appreciated if you would apply them to solving these Rakudo issues!

Simply fixed

When I started, the oldest open issue in the Rakudo repository was about 7.5 years old, and effectively lost in the mist of time. In any case they predated significant changes, such as the new dispatch mechanism. Not to mention the Raku Programming Language still had a different name then.

If you're really interested in the mists of time, you could check out the old issues that were migrated from the venerable RT system, with the oldest open issue from 2010!

So going from the oldest issue to the newest, it was a matter of checking if the problem still existed. And if it appeared fixed, and if it was possible to write a test for it, write a test for it and close the issue. And sometimes there even was a PR for a test already, so that was even more a no-brainer.

Marked as fixed, but also marked as "tests needed"

Quite a few number of issues were actually marked as fixed, but were also marked as needing tests. Sadly, the original author of the issue had not done that or didn't do that after the issue was fixed. In most cases it was just a few minutes to write a test, test it and commit it.

Fixed because of the new dispatch logic

After 18 months of work in 2020 and 2021, the new dispatch mechanism became default in the 2021.10 release of the Rakudo compiler. Most of the multi method dispatch related issues that were made before that time, appeared fixed. So it just was a matter of writing the tests (if there weren't any yet) and commit them.

Fixed in RakuAST

While testing all of these issues, I always also tested whether they were fixed in the new Raku grammar, based on the RakuAST project (which is why I started doing this bug hunting streak in the first place).

Running code with the new Raku grammar, is as easy as prefixing RAKUDO_RAKUAST=1 to your call to raku. For instance raku -e '{ FIRST say "first" }' does not output anything with the legacy grammar. But with RAKUDO_RAKUAST=1 raku -e '{ FIRST say "first" }' it will say "first" because the FIRST phaser fires for any block when it is first executed with the new Raku grammar.

And to my surprise, in many cases they were! For those cases a special test file is being reserved to which tests are added for issues that have been fixed with the new Raku grammar in RakuAST.

These issues are then marked as being fixed in RakUAST but left open, so that people are hopefully prevented from creating duplicate issues for problems that apparently haven't been fixed yet.

A large percentage of these issues appear fixed because they were essentially static optimizer issues, and the new Raku grammar doesn't have any of these compile-time optimisations yet. So it's important to be able to check for regressions of this type once optimizations are being added back in. In turn, these static optimizer issues were often caused by the optimizer not having enough or the correct information for doing optimizations. Which in turn was one of the reasons to start with RakuAST to begin with.

Not fixed

And then there were the issues that were simply still reporting an existing problem. Some of them, with the knowledge that I acquired over the years, looked like easy to fix. So I put in some effort to fix them. A non-exhaustive list:

• allow $:F as placeholder variable with use isms
• reduce number of Failure objects when numerically comparing non-numerical values
• re-introduce support for +permutations(30)
• allow for @*ARGS to contain other Cool values (apart from strings)
• fix numeric comparators on Rats with 0 denominator (e.g. <1/0> <=> <-1/0>)
• fix magic increment/decrement on ⁰¹²³⁴⁵⁶⁷⁸⁹ superscript characters
• make --repl-mode=interactive CLI argument always force an interactive REPL
• add support for any junctions in regex interpolation
• add support for Unicode vulgar fractions to val() (such as ⅓)
• seamlessly use rlrwap as line editor in REPL if no other modules installed
• improvements on several error messages

About 50 of the outstanding issues look like they should be fixable without turning into large projects, so I will be looking at these in the coming days / weeks.

Feature requests

Some of the open issues were basically feature requests. Sometimes I felt that they could be easily implemented (such as several error message improvements) so I implemented them. Others I created a Pull Request for. And for still others I felt a problem-solving issue would be needed (which I then created). And some I closed, knowing almost 100% they would never be accepted.

If this was one of your issues, and you still feel that feature should become part of the Raku Programming Language, please don't be discouraged! Looking at a 60+ issues for 3 weeks in a row sometimes made me a bit grumpy at the end of the day. Please make a new problem solving issue in that case!

Addressed in RakuAST

Many issues looked like they would be more easily solvable in the new Raku grammar with RakuAST. There are now 289 of them. These will be next on my list.

Conclusion

It was an interesting ride through memory lane the past weeks. With about 200 commits, that's not bad at all!

Note that with these numbers of issues, if I had an error rate of only 1%, there are at least 5 issues that were closed when they shouldn't have been closed. If you feel that an issue has been closed incorrectly, please leave a comment and I'll re-open them if you cannot do that yourself.

Sadly, because of the additional tests that I wrote, the number of roast test-files passing has now dropped again below the 85% mark. Still, I do think this is progress, as the errors that they check for would have been encountered during the development of the Raku grammar sooner or later anyway.

Anyway, it was fun being able to close as many issues as I did! Want to join in the fun? There are still 778 issues open waiting for someone to close them!

If you like what I'm doing, committing to a small sponsorship would mean a great deal to me!

Late Edit – thanks to a genius suggestion from @wamba, I have made some late changes to improve the code – specifically the concerns I mentioned in v1 about when clause and lisp (((( are now fixed. Excellent input!!

Regular followers of this blog will know that I am on a bit of a Functional tack … I’m highly motivated to improve my functional chops because I am giving a talk at the London Perl and Raku Conference shortly entitled Raku HTML::Functional‎ and I just realised that the audience is going to be a bunch of deep perl and raku (linux) experts who know Functional coding inside out. Yikes … hope I don’t get any tough questions from folks who really know their stuff!

By contrast, I am a dabbler in Functional. I like the feel of using .map and .grep and so on, but I am on the learning curve. And I am resistant to languages that constantly get in my way since largely I am trying to make code that works rather than to wrestle with compile errors. (And no I do not work in large teams since you ask).

So when I saw a recent post on HN, written in F#, I felt challenged to work out what was going on and to try and relate to Raku.

A Burrito is a Monad

type Meat = Chicken | Beef | Pork | Fish | Veggie

type Ingredient = 
    | Cheese | Rice | Beans | Salsa | Guacamole | SourCream | Lettuce
    | Tomato | Onion | Cilantro | PicoDeGallo

type Burrito = Meat option * Ingredient list

let (>>=) burrito f =
    match burrito with
    | Some meat, ingredients -> f (Some meat, ingredients)
    | None, _ -> None, []

let returnBurrito (meat, ingredients) = meat, ingredients

let tortilla = returnBurrito (Some Veggie, [])

let addMeat meat (m, ingredients) = Some meat, ingredients

let addIngredient ingredient (meat, ingredients) =
    meat, ingredient :: ingredients

let addMissionBurritoIngredients (meat, ingredients) =
    meat, Cheese :: Rice :: Beans :: ingredients

let holdThe ingredient (meat, ingredients) =
    meat, List.filter (fun i -> i <> ingredient) ingredients

let burrito = 
    tortilla
    >>= addMeat Chicken
    >>= addMissionBurritoIngredients
    >>= holdThe Cheese
    >>= addIngredient PicoDeGallo
    >>= addIngredient Salsa
    >>= addIngredient Guacamole
    >>= addIngredient SourCream

printfn "%A" burrito

This from the OP by William Cotton

Definitely Maybe

Since we are talking Monads, I realised that the raku Definitely module written by masukomi would come in handy. This module arose from a post I made here some time back, so it was a good time to revisit.

https://github.com/librasteve/raku-Burrito/blob/main/burrito-dm.raku

use Definitely;

enum Meat <Chicken Beef Pork Fish Veggie>;

enum Ingredient <Cheese Rice Beans Salsa Guacamole SourCream Lettuce
                 Tomato Onion Cilantro PicoDeGallo>;

sub returnBurrito($meat, @ingredients) {
    $meat, @ingredients
}

sub tortilla {
    returnBurrito(something(Veggie), [])
}

sub add-meat($meat, ($, @ingredients)) {
    something($meat), @ingredients
}

sub add-ingredient($ingredient, ($meat, @ingredients)) {
    $meat, [$ingredient, |@ingredients]
}

sub add-mission-burrito-ingredients(($meat, @ingredients)) {
    $meat, [Cheese, Rice, Beans, |@ingredients]
}

sub hold-the($ingredient, ($meat, @ingredients)) {
    ($meat, [@ingredients.grep(* != $ingredient)]);
}

multi infix:«>>=»((None $, @), +@ ) is prec(prec => 'f=') {
    nothing(),[]
}

multi infix:«>>=»($burrito, +(&f, *@args)) is prec(prec => 'f=') {
    f( |@args, $burrito )
}

tortilla()
>>= (&add-meat, Beef)
>>= (&add-mission-burrito-ingredients)
>>= (&hold-the, Cheese)
>>= (&add-ingredient, PicoDeGallo)
>>= (&add-ingredient, Salsa)
>>= (&add-ingredient, Guacamole)
>>= (&add-ingredient, SourCream)

==> say();

I hope that you will agree that Raku does a generally solid job of handling the translation from F#.

There are a couple of raised eyebrows around the when {...} clauses and the handling of variadic arity of the passed in function in the match and the lisp-like (((((( parens in the application of the custom binder. Otherwise, it is pretty smooth. EDIT – NOW FIXED

The Definitely module works well here, I have also tried with Rawley Fowlers Monad::Result module which was similarly successful.

Monad Chains and Binding

In this self-study, I leaned on the excellent Wikipedia Monad page, which mentions that a true Monad implementation has three operations:

1. Monadic Type
   • A type (Maybe)
2. Unit operation
   • A type converter (Just(x))
3. Bind operation
   • A combinator for monadic functions ( >>= or .flatMap())

And it shows chaining of this halve function as an example of chaining with the bind operator in Haskell:

halve :: Int -> Maybe Int
halve x
  | even x = Just (x `div` 2)
  | odd x  = Nothing
 -- This code halves x twice. it evaluates to Nothing if x is not a multiple of 4
halve x >>= halve

So, to improve the Definitely module, I have added a binding operator to be used like this:

use Definitely;

sub halve(Int $x --> Maybe[Int]) {
    given $x {
        when   * %% 2 { something( $x div 2 ) }
        when ! * %% 2 { nothing(Int) }
    }
}

say (halve 4) >>= &halve;    #1
say (something 32) >>= &halve >>= &halve >>= &halve;    #4
say (halve 3) ~~ None;       #True

Note that the Result::Monad module already has bind and map operators provided.

For now this is a PR, feel free to install directly from my fork here if you would like to try it. Now released to zef package installer eco-system…

zef install https://github.com/librasteve/Definitely.git

As usual all comments and feedback welcome!

~librasteve

Back in ’21 I asked the question Can Raku replace HTML? As expected that rather click-baity title got a lot of complaints. So I couldn’t resist repeating the meme.

If you are wondering, Raku can replace PHP literally…

"PHP".subst(/PHP/, 'Raku').say;    #Raku

BUT that’s beside the point. Just my sense of -Ofun getting out of hand.

In recent posts, I have been digging in to HTMX and Raku Cro…

• HTMX and Raku Cro
• HTMX and Raku Cro II
• HTMX, Raku and Pico CSS

And while in the web application frame of mind, I started to think maybe I can use Raku with WordPress, perhaps initially to just write some front end with Raku and HTMX served with Cro and to talk to the WP database backend. (This kind of combination is already a thing with WordPress and React).

And then that made me think yeah well WordPress (and Laravel, OJS, etc.) continue to be popular and lend PHP a kind of ongoing zombie existence. PHP is not likely to suddenly bust out of its web language niche, so likely over time it will gradually fade away in popularity. And much of the gravity in web development is going to drag PHPers towards JavaScript. And, since I am a PHP coder in my day job, I realised that (like me) many PHPers travellers would rather not get dragged into the JavaScript / React / Composer / Node black hole of complexity. And so maybe Raku and HTMX would one day become a good upgrade path from PHP since it has roots in perl – the original web language – with a friendlier syntax (eg for OO). Even the $ sigil for variables, {} curlies and the ; semicolon make for a smooth transition from PHP. Maybe in this niche Raku can ultimately replace PHP…

Then I started to think about what made PHP the goto language for web developers originally. How would Raku stack up?

<?php Flashback ?>

Remember this:

<body>

<div class="container">
    <h1>Welcome to My Simple PHP Page!</h1>

    <p>
        Today is: 
        <?php 
            // Get the current date and time
            echo date("l, F j, Y, g:i a"); 
        ?>
    </p>

    <p>
        Random number: 
        <?php 
            // Generate a random number between 1 and 100
            echo rand(1, 100); 
        ?>
    </p>
</div>

</body>
</html>

The full source of this index.php file is in this gist … Simple PHP HTML Page

To serve this page, you can run a server like this:

php -S localhost:8000 -t /path/to/directory

Horrible though it is, this intertwining of PHP and HTML is what made PHP the goto web language in its heyday. And that got me thinking, could this be done with Raku?

<?raku as a Replacement ?>

So, knowing Raku to be a very flexible language, I made a new module Cro::WebApp::Evaluate. Here’s the synopsis:

<body>

<div class="container">
    <h1>Welcome to My Simple Raku Page!</h1>

    <p>
        Today is:
        <?raku
            #Get the current date and time
            use DateTime::Format;
            say strftime('%Y-%m-%d %H:%M:%S', DateTime.now);
        ?>
    </p>

    <p>
        Random number:
        <?raku
            #Generate a random number between 1 and 100
            say (^100).pick;
        ?>
    </p>
</div>

</body>

And here’s how to serve this as index.raku using the Raku Cro web framework.

use Cro::HTTP::Router;
use Cro::WebApp::Template;
use Cro::WebApp::Evaluate;

sub routes() is export {
    route {
        evaluate-location 'evaluates';

        get -> {
            evaluate 'index.raku';
        }

        get -> *@path {
                static 'static', @path;
        }
    }
}

I leave it as an exercise for the reader to show how to have Cro render and serve index.php files in a parallel directory and route structure – perhaps for an incremental migration effort.

Conclusion

Do I expect this new module to be embraced by the PHP community? No. In most cases, I think that hybrid PHP/HTML pages like this have been replaced by templating systems or web frameworks.

Am I a little ashamed to have made this module? Yes. Honestly, I would not encourage coders to start using Raku like this – Cro Templates would be a better solution for most projects.

Are there some point needs where this approach can be applied? Maybe. Since this was a seminal feature of early PHP, I expect that there are some point cases where embedding Raku and HTML will be the cleanest way to (re)package some code. For example where a single dynamic page uses PHP for a database query, wrapping the results as json and then passing the data into a JavaScript function … for some client-side logic with the (eg.) Google Maps API and then dynamic presentation.

Is this a module prerequisite for PHPers to migrate to Raku? Probably not. However, I think that the presence of this module can bring some comfort to PHP coders that anything that can be done in PHP can be (re)done in Raku.

As usual comments & feedback welcome!

~librasteve

In the past 2 years, module distributions in the https://raku.org that were being published through the original (but deprecated) "p6c" ecosystem, were no longer being harvested by the default "zef" harvester. But they were still being harvested by the Raku Ecosystem Archive harvester. And thus, any updates to these distributions remained visible.

But this harvesting stopped with the 0.0.26 release of Ecosystem::Archive::Update.

The reasoning for no longer supporting the "p6c" ecosystem is explained in the problem solving issue "Preparing the Raku Ecosystem for the Future".

This means that any updates to the 651 distributions still in the "p6c" ecosystem, will not be noticed anymore in the live Raku ecosystem. To alert the authors / current maintainers of these distributions, 202 issues were generated, listing the distributions affected.

Now, one week later, many authors responded to the issue they found in one of their repositories. The reactions were generally positive to this effort. Some of the authors took this notice as an opportunity to update their distribution to the "fez" ecosystem. Kudos to these authors:

• YellowApple for GLFW
• Itsuki Toyota for:
  • Algorithm::BinaryIndexedTree
  • Algorithm::KdTree
  • Algorithm::Kruskal
  • Algorithm::TernarySearchTree
  • Algorithm::Treap
  • Algorithm::ZobristHashing
• Fernando Correa de Oliveira for:
  • SixPM
  • Heap
  • Injector
  • Protocol
  • Trie
• Brian Duggan for Slang::Mosdef
• Haytham Elganiny for:
  • Grid
  • Retry
• Dean Powell for Locale::Codes::Country

Other authors responded that they don't want to spend any time on these distributions anymore, but would like to have them transferred to the Raku Community Module Adoption Center. Kudos to these authors for spending their time and effort on these distributions so far and making them available for the future users of the Raku Programming Language:

• Yechen Fu for Redis
• brian d foy for:
  • Chemistry::Elements
  • PrettyDump (in progress)
• Yann Büchau for: Fortran::Grammar
• +merlan #flirora for:
  • Math::Random
  • Terminal::WCWidth
• Carsten Hartenfels for: Text::Markdown::Discount (in progress)
• loren for:
  • Net::FTP (in progress)
  • Net::FTPlib (in progress)
  • App::snippet (in progress)
• Rob Hoelz for:
  • Algorithm::Elo
  • Algorithm::LCS
  • Pod::EOD

Note that the distributions marked "in progress" still need some Tender Loving Care before they will be properly integrated into the Raku ecosystem again. Pull Requests for these distributions are very welcome!

A number of other authors responded that they thought that (some of) their distributions were not fit to be modernized or transferred to the Raku Module Adoption Center. Kudos to these authors nonetheless, because of their time and efforts in the past. Even if that didn't result into something they thought was worth salvaging. These distributions have been removed from the "p6c" ecosystem, which thusly contains 552 distributions now: a 15% reduction in one week!

Yours truly will actually look at all of these packages to see whether they warrant a transfer to the Raku Module Adoption Center. Beauty is in the eye of the beholder :-)

All in all a fruitful week in the Raku Ecosystem world.

And now?

Many authors that received an issue notice about this, have not responded yet. If you are one of them, please respond to the issue and tell us what you would like (us) to do.

Meanwhile the distributions marked "in progress" could use someone looking at the code and/or the tests to see what's stopping inclusion into the Raku ecosystem again. Many kudos in advance.

And if you're an author who already transferred their modules to the "zef" ecosystem: thank you for you continued practical support of the Raku Programming Language, and the extension of the https://raku.land!

The question has been raised, how to get named arguments into sub EXPORT via a use-statement. The ever helpful raiph provided an answer, which in turn left me with the question, why he didn’t just use a Capture to move the data around. Well, because that doesn’t work. The compiler actually evaluates the expression \(:1a, :2b) into (1, 2) before passing it on to EXPORT.

If it’s hard, do it functional!

# foo.raku
use v6.d;

constant &transporter = sub { \(:1a, :2b); }
use foo &transporter;

# lib/foo.rakumod
use v6.d;

proto sub EXPORT(|) { * }

multi sub EXPORT(&transporter) {
    &EXPORT(|transporter);
}

multi sub EXPORT(:$a, :$b) {
    dd $a, $b;
    Map.new
}

The idea is to hand a function to use to be called by EXPORT, and then redispatch the value that is produced by that function, to take advantage of Raku´s excellent signature binding. The proto and refering to sub EXPORT explicitly is needed because there is also a predefined (and in this case hidden) package called EXPORT.

I’m passing on named arguments to EXPORT, but all kinds of stuff could be returned by &transporter. So long as everything is known pretty early on at compile-time. The use-statement is truly an early bird.

(in chronological order, with comment references)

• Day 1 – Rocking Raku Meets Stodgy Debian by Tom Browder (/r/rakulang comments)
• Day 2 – An update on raku.land by JJ Atria (/r/rakulang comments)
• Day 3 – Helping the Elves Sort Their Mail by Julia
• Day 4 – Embedding a stack-based programming language in Raku by Wim Vanderbauwhede
• Day 5 – The Elves go back to Grammar School by Steve Roe
• Day 6 – The Future Of POD6 by Kay Rhodes (/r/rakulang comments)
• Day 7 – The Magic Of $/ by Elizabeth Mattijsen (/r/rakulang comments)
• Day 8 – Make it Snow 2.0: The Snowfall Strikes Back by John Haltiwanger (/r/rakulang comments)
• Day 9 – Networks Roasting on an Open Fire, Part 1: Whipuptitude by Geoffrey Broadwell
• Day 10 – The Magic Of Q by Elizabeth Mattijsen (/r/rakulang comments)
• Day 11 – Networks Roasting on an Open Fire, Part 2: Axes to Grind by Geoffrey Broadwell
• Day 12 – Perspectives on RakuDoc Version 2 by Richard Hainsworth
• Day 13 – Networks Roasting on an Open Fire, Part 3: Feeling Warm and Looking Cool by Geoffrey Broadwell
• Day 14 – The Magic Of Q (Part 2) by Elizabeth Mattijsen (/r/rakulang comments)
• Day 15 – An Object Lesson for the Elven by Steve Roe
• Day 16 – It’s Too Generic; Please Instantiate! by Vadim Belman (/r/rakulang comments)
• Day 17 – Writing some horrible Raku code this Christmas! by Will Coleda
• Day 18 – Dissecting the Go-Ethereum keystore files using Raku tools by Konstantin Narkhov
• Day 19 – Autogenerating Raku bindings! by Dan Vu (/r/rakulang comments)
• Day 20 – Craggy Christmas Chronicles by Steve Roe
• Day 21 – Using DALL-E models in Raku by Anton Antonov (/r/rakulang comments)
• Day 22 – The Magic Of hyper by Elizabeth Mattijsen
• Day 23 – Optimizing Raku programs with Zig by John Haltiwanger (HackerNews comments)
• Day 24 – Streamlining AI vision workflows by Anton Antonov
• Day 25 – Raku 2023 Review by Elizabeth Mattijsen (/r/rakulang comments)

Hope to see you again next year!

According to Larry, laziness is a programmers virtue. The best way to be lazy is having somebody else do it. By my request, SmokeMachine kindly did so. This is not fair. We both should have been lazy and offload the burden to the CORE-team.

Please consider the following code.

my @many-things = (1..10).List;
sub doing-one-thing-at-a-time($foo) { ... }
say doing-one-thing-at-a-time(@many-things.all);

Rakudo goes out of it’s way to create the illusion that sub doing-one-thing-at-a-time can deal with a Junction. It can’t, the dispatcher does all the work of running code in parallel. There are tricks we can play to untangle a Junction, but there is no guarantee that all values are produced. Junctions are allowed to short-circuit.

This was bouncing around in my head for quite some time, until it collided with my thoughts about Range. We may be handling HyperSeq and RaceSeq wrong.

my @many-things = (1..10).List;
sub doing-one-thing-at-a-time($foo) { ... }
say doing-one-thing-at-a-time(@many-tings.hyper(:degree<10>));

As with Junctions doing dispatch-magic to make hyper/race just work, moving the handling to the dispatcher would move the decision from the callee to the caller and, as such, from the author of a module to the user. We can do that by hand already with .hyper.grep(*.foo) or other forms of boilerplate. In Raku-land we should be able to do better and provide a generalisation of transforming calls with the help of the dispatcher.

I now know what to ask Santa for this year.

Lately, some unhappiness has popped up about Range and it’s incomplete numericaliness. Having just one blogpost about it is clearly not enough, given how big Ranges can be.

say (-∞..∞).elems;
# Cannot .elems a lazy list
  in block <unit> at tmp/2021-03-08.raku line 2629

I don’t quite agree with Rakudo here. There are clearly ∞ elements in that lazy list. This could very well be special-cased.

The argument has been made, that many operators in Raku tell you what type the returned value will have. Is that so? (This question is always silly or unnecessary.)

say (1 + 2&3).WHAT;
# (Junction)

Granted, Junction is quite special. But so are Ranges. Yet, Raku covers the former everywhere but the latter feels uncompleted. Please consider the following code.

multi sub infix:<±>(Numeric \n, Numeric \variance --> Range) {
    (n - variance) .. (n + variance)
}

say 2.6 > 2 ± 0.5;
# True

my @heavy-or-light = 25.6, 50.3, 75.4, 88.8;

@heavy-or-light.map({ $_ ≤ 75 ± 0.5 ?? „$_ is light“ !! „$_ is heavy“ }).say;
# (25.6 is heavy 50.3 is heavy 75.4 is heavy 88.8 is heavy)

To me that looks like it should DWIM. It doesn’t, because &infix:«≤» defaults to coercing to Real and then comparing numerically.

This could easily be fixed by adding a few more multis and I don’t think it would break any production code. We already provide quite a few good tools for scientists. And those scientists do love their error bars — which are ranges. I would love for them to have another reason to use Raku over … that other language.

In the earliest days of Raku, Damian Conway specified a documentation markup language to accompany it. Since it was modeled on Perl's POD it was called <sound of trumpets and dramatic pause> POD6.

The Specification of POD6 (S26) was mostly incorporated without much extra explanation in the documentation suite. In this way, the description of POD6 was itself was an illustration of many of the features it documented, and some that it did not document.

Since Raku is defined by its test suite, and not its documentation, there were other details of POD6 in the tests that were not documented, even in S26.

Raku developed and morphed, but POD6 remained. The tooling for rendering the documentation sources needed updating, and the documentation site had to be modernised.

Upgrading the renderer

A project of mine was to upgrade the basic renderer that would transform POD6 to HTML, but allow for developers to customise the templates for each type of POD6 block type. (The first Pod::To::HTML renderer hard-coded representations of POD6 markup, eg. B<this is bold> was <strong>this is bold</strong> and could not be changed.)

It turned out that S26 allowed for much more than had been included in the first documentation sources, including custom blocks and custom markup.

The project to upgrade the original HTML renderer morphed into Raku::Pod::Render, and transforming a directory full of individual documentation sources into an interlinked and searchable set of documents required another layer of tooling Collection. For example, collecting together all the pages that can be grouped as tutorials, or reference, or language, and creating a separate page for them automatically.

I covered these two projects in a presentation to RakuCon 2022.

Some of the original ideas in S26 had not been implemented, such as aliases and generic numbering. Other ideas had become outdated, such as a way to specify document encoding, which is now solved with Unicode.

In addition, RakuAST (see RakuAST for early adopters ) is on the horizon, which will radically change the speed of documentation processing.

There are also two implementations of POD6, one in Raku and one in Javascript, namely Alexandr Zahatski's Podlite.

Introducing Rakudoc

This was an ideal time to revisit POD6 and recast it into Rakudoc - new name for the markup language, and its new file extension ".rakudoc".

I was invited to the first Raku Core Summit and I put together a presentation about the changes I thought needed to be made based on my own experience, but also using comments from other developers.

We came to a number of consensus agreements about the minimal changes that were needed, and some extra functionality to handle new questions, such as documentation versioning.

It was also clear that Rakudoc (aka POD6) has two separate parts: components that interact closely with the program being documented, and components that will be rendered separately into HTML (or an ebook). The documentation file needs to make this clear.

I have now written the first draft of the revision and the documentation file that encapsulates it. An HTML version can be found at new-raku.finanalyst.org/language/rakudoc, alongside the old documentation file and the simple table implementation. I am planning future blogs to describe some of the proposed revisions.

However, none of the revisions will break existing POD6, so Rakudoc should be backwards compatible with POD6. The version at new-raku is a VERY early first draft, and it will go through several review stages.

The first Raku Core Summit was organised by Elizabeth Mattijsen and hosted by Elizabeth and Wendy at their home. It was a really good meeting and I am sincerely grateful for their generosity and hospitality. The summit was also supported by The Perl and Raku Foundation, Rootprompt, and Edument.

The first Raku Core Summit, a gathering of folks who work on “core” Raku things, was held on the first weekend of June, and I was one of those invited to attend. It’s certainly the case that I’ve been a lot less active in Raku things over the last 18 months, and I hesitated for a moment over whether to go. However, even if I’m not so involved day to day in Raku things at the moment, I’m still keen to see the language and its ecosystem move forward, and – having implemented no small amount of the compiler and runtime since getting involved in 2007 – I figured I’d find something useful to do there!

The area I was especially keen to help with is RakuAST, something I started, and that I’m glad I managed to bring far enough that others could see the potential and were excited enough to pick it up and run with it.

One tricky aspect of implementing Raku is the whole notion of BEGIN time (of course, this is also one of the things that makes Raku powerful and thus is widely used). In short, BEGIN time is about running code during the compile time, and in Raku there’s no separate meta-language; anything you can do at runtime, you can (in principle) do at compile time too. The problem at hand was what to do about references from code running at compile time to lexically scoped symbols in the surrounding scope. Of note, that lexical scope is still being compiled, so doesn’t really exist yet so far as the runtime is concerned. The current compiler deals with this by building up an entire flattened table of everything that is visible, and installing it as a fake outer scope while running the BEGIN-time code. This is rather costly, and the hope in RakuAST was to avoid this kind of approach in general.

A better solution seemed to be at hand by spotting such references during compilation, resolving them, and fixating them – that is, they get compiled as if they were lookups into a constant table. (This copies the suggested approach for quasiquoted code that references symbols in the lexical scope of where the quasiquoted code appears.) This seemed promising, but there’s a problem:

my $x = BEGIN %*ENV<DEBUG> ?? -> $x { note "Got $x"; foo($x) } !! -> $x { foo($x) };

It’s fine to post-declare subs, and so there’s no value to fixate. Thankfully, the generalized dispatch mechanism can ride to the rescue; we can:

1. Create a placeholder object with an attribute to hold the resolution
2. Compile the lookup into a use of a dispatcher that reads this attribute and indicates that this is a constant result of the dispatch (so it is stored in the inline cache, and after specialization will be just as cheap as any other sub call). If the attribute is not set, that means we tried to run the code before declaring the sub, and the object can carry a bit of extra metadata in order to give a good error message.
3. Keep track of this object in the compiler, and – upon declaration of the sub – install it into the placeholder object.
4. Give an error if we reach the end of the compilation unit with an unfilled placeholder.

When compiling Raku code, timing is everything. I knew this and tried to account for it in the RakuAST design from the start, but a couple of things in particular turned out a bit awkward.

• Every node wanting to do something at BEGIN time would implement RakuAST::BeginTime, and in doing so would have its method to perform a BEGIN-time action called at the appropriate point. So far so good, for most program elements. However, it turns out that some larger program elements want to do things at compile time both at the point they start and at the point they end. I’d added a “I want my BEGIN time before my children” mechanism, but that didn’t help program elements that wanted action at both points. Thus, I started a branch that introduces parse time. For “leaf” elements that is the same as BEGIN time, but for things like packages and routines, which have a bunch of stuff on the inside, it happens before we go parsing their inner scope.
• The RakuAST tree is an object graph where every node knows its children, but nodes do not reference their parents. However, some program elements need to find out about their enclosing context, for example a method wants to know about the class it is being declared in. Thus I came up with a notion of attach targets (things that child nodes want to discover) and attaching nodes (the children that want to “attach” to a parent – I’m not sure I got the naming right here, in hindsight). I guess with my IDE work it also appealed that one might be able to introspect these kinds of relationships for tooling purposes – although in reality the attaching work was highly imperative anyway. But with parse and begin time clarified, it also seemed that attachment work could happen in either (or do different work in both), but also it was hazy exactly when attachment happened, and it could end up happening multiple times, which was fragile. Thus, while the notion of attachment targets should survive – probably with a better name – the need for a RakuAST::Attaching went away. My branch also took on its elimination.

I got a decent way into this restructuring work during the core summit, and hope to find time soon to get it a bit further along (I’ve been a mix of busy, tired, and had an eye infection to boot since getting back from the summit, so thus far there’s not been time for it).

I also took part in various other discussions and helped with some other things; those that are probably most worth mentioning are:

• There was quite a bit of talk about Raku Doc (formerly Pod6) and tidying up some aspects of its design and implementation. I’m pleased to see it is getting a rather cleaner implementation in the RakuAST-based compiler frontend. (Story illustrating why: when developing the Comma IDE I built a small and very specialized Raku grammar to Java lexer/parser compiler, in order to nail the Raku language’s syntactic structure, and I followed what Rakudo’s grammar did relatively closely in all but two places: operator parsing – because it just needed to be different for the IntelliJ platform’s tree builder engine – and Pod6, because it was easier to read the spec and implement it afresh than it was to decipher Rakudo’s implementation of it!)
• There was a long-standing problem that looked like the regex engine massively leaked memory in certain cases, but nobody could pin down the leak. The reason was that it wasn’t actually leaking, it was just creating sufficient backtracking state to go quadratic in the size of the input string. MoarVM only stores an array header in the nursery (the region of memory it allocates in, and whose fullness is the trigger for doing a GC run); the array body is allocated using a standard allocator. There are good things about this, but a less good thing is that if you allocate loads of large arrays in quick succession, you’ll allocate lots of memory, but not eat much of the nursery, and so it won’t be cleaned up very soon. The regex in question was doing exactly that: since it captured into an inner cursor, and cursors are in principle immutable, then it ended up cloning the array at every backtracking step, and doing so in a fairly tight loop. Closer examination revealed that the cloning of the backtrack stack was, however, overly defensive; eliminating that copying led to a huge memory and time improvement. Still, it remains to make such large array allocations pressure the GC more (we already do a similar kind of thing for big integers).
• I provided a few MoarVM hints that helped leont get support for asynchronous UNIX domain sockets implemented. Apparently that unblocks having an asynchronous Postgres driver, which would be most welcome for use in Cro applications, where the request handling is asynchronous but the database queries end up growing the thread pool by really blocking threads.

Thanks goes to Liz for organizing the summit, to Wendy for keeping everyone so well fed and watered, to the rest of attendees for many interesting discussions over the three days, to TPRF and Rootprompt for sponsoring the event, and to Edument for supporting my attendance.

I’d like to thank everyone who voted for me in the recent Raku Steering Council elections. By this point, I’ve been working on the language for well over a decade, first to help turn a language design I found fascinating into a working implementation, and since the Christmas release to make that implementation more robust and performant. Overall, it’s been as fun as it has been challenging – in a large part because I’ve found myself sharing the journey with a lot of really great people. I’ve also tried to do my bit to keep the community around the language kind and considerate. Receiving a vote from around 90% of those who participated in the Steering Council elections was humbling.

Alas, I’ve today submitted my resignation to the Steering Council, on personal health grounds. For the same reason, I’ll be taking a step back from Raku core development (Raku, MoarVM, language design, etc.) Please don’t worry too much; I’ll almost certainly be fine. It may be I’m ready to continue working on Raku things in a month or two. It may also be longer. Either way, I think Raku will be better off with a fully sized Steering Council in place, and I’ll be better off without the anxiety that I’m holding a role that I’m not in a place to fulfill.

I want to revive Carl Mäsak's Coding Contest as a crowd-sourced contest.

The contest will be in four phases:

• development of tasks
• public contest, where you can solve the tasks
• public commenting on solutions
• grading and awarding of prizes

For the first phase, development of tasks, I am looking for volunteers who come up with coding tasks collaboratively. Sadly, these volunteers, including myself, will be excluded from participating in the second phase.

I am looking for tasks that ...

• are well-worded
• have a (private) implementation that shows it is solvable, preferably in under 500 lines of code
• have a few public tests to help the participants, and a more tests tests that help the graders and commenters later on

This is non-trivial, so I'd like to have others to discuss things with, and to come up with some more tasks.

If you want to help with task creation, please send an email to [email protected], stating your intentions to help, and your freenode IRC handle (optional).

There are other ways to help too:

• You can pledge a prize (some ideas: Comma IDE subscription, books, Camelia plushies, Amazon vouchers)
• You can help design a small website for the contest
• You can help iron out the exact rules for the contest
• ... or any other way that I didn't think of :-)

In these cases you can use the same email address to contact me, or use IRC (moritz on freenode) or twitter.

After a perilous drive up a steep, narrow, winding road from Lake Geneva we arrived at an attractive Alpine village (Villars-sur-Ollon) to meet with fellow Perl Mongers in a small restaurant.  There followed much talk and a little clandestine drinking of exotic spirits including Swiss whisky. The following morning walking to the conference venue there was an amazing view of mountain ranges. On arrival I failed to operate the Nespresso machine which I later found was due to it simply being off.  Clearly software engineers should never try to use hardware. At least after an evening of drinking.

Wendy’s stall was piled high with swag including new Bailador (Perl 6 dancer like framework) stickers, a Shadowcat booklet about Perl 6 and the new O’Reilly “Thinking in Perl 6″. Unfortunately she had sold out of Moritz’s book “Perl 6 Fundamentals” (although there was a sample display copy present). Thankfully later that morning I discovered I had a £3 credit on Google Play Books so I bought the ebook on my phone.

The conference started early with Damian Conway’s Three Little Words.  These were “has”, “class” and “method” from Perl 6 which he liked so much that he had added them to Perl 5 with his “Dios” – “Declarative Inside-Out Syntax” module.  PPI wasn’t fast enough so he had to replace it with a 50,000 character regex PPR. Practical everyday modules mentioned included Regexp::Optimizer and Test::Expr. If the video  doesn’t appear shortly on youtube a version of his talk dating from a few weeks earlier is available at https://www.youtube.com/watch?v=ob6YHpcXmTg

Jonathan Worthington returned with his Perl 6 talk on “How does deoptimization help us go faster?” giving us insight into why Perl 6 was slow at the Virtual Machine level (specifically MoarVM). Even apparently simple and fast operations like indexing an array were slow due to powerful abstractions, late binding and many levels of Multiple Dispatch. In short the flexibility and power of such an extensible language also led to slowness due to the complexity of code paths. The AST optimizer helped with this at compile time but itself took time and it could be better to do this at a later compile time (like Just In Time).  Even with a simple program reading lines from a file it was very hard to determine statically what types were used (even with type annotations) and whether it was worth optimizing (since the file could be very short).

The solution to these dynamic problems was also dynamic but to see what was happening needed cheap logging of execution which was passed to another thread.  This logging is made visible by setting the environment variable MVM_SPESH_LOG to a filename. Better tooling for this log would be a good project for someone.

For execution planning we look for hot (frequently called) code, long blocks of bytecode (slow to run) and consider how many types are used (avoiding “megamorphic” cases with many types which needs many versions of code).  Also analysis of the code flow between different code blocks and SSA.  Mixins made the optimization particularly problematic.

MoarVM’s Spesh did statistical analysis of the code in order to rewrite it in faster, simpler ways. Guards (cheap check for things like types) were placed to catch cases where it got it wrong and if these were triggered (infrequently) it would deoptimize as well, hence the counterintuitive title since “Deoptimization enables speculation” The slides are at http://jnthn.net/papers/2017-spw-deopt.pdf with the video at https://www.youtube.com/watch?v=3umNn1KnlCY The older and more dull witted of us (including myself) might find the latter part of the video more comprehensible at 0.75 Youtube speed.

After a superb multi-course lunch (the food was probably the best I’d had at any Perl event) we returned promptly to hear Damian talk of “Everyday Perl 6”. He pointed out that it wasn’t necessary to code golf obfuscated extremes of Perl 6 and that the average Perl 5 programmer would see many things simpler in Perl 6.  Also a rewrite from 5 to 6 might see something like 25% fewer lines of code since 6 was more expressive in syntax (as well as more consistent) although performance problems remained (and solutions in progress as the previous talk had reminded us).

Next Liz talked of a “gross” (in the numerical sense of 12 x 12 rather than the American teen sense) of Perl 6 Weeklies as she took us down memory lane to 2014 (just about when MoarVM was launched and when unicode support was poor!)  with some selected highlights and memories of Perl 6 developers of the past (and hopefully future again!). Her talk was recorded at https://www.youtube.com/watch?v=418QCTXmvDU

Cal then spoke of Perl 6 maths which he thought was good with its Rats and FatRats but not quite good enough and his ideas of fixing it.  On the following day he showed us he had started some TDD work on TrimRats. He also told us that Newton’s Method wasn’t very good but generated a pretty fractal. See https://www.youtube.com/watch?v=3na_Cx-anvw

Lee spoke about how to detect Perl 5 memory leaks with various CPAN modules and his examples are at https://github.com/leejo/Perl_memory_talk

The day finished with Lightning Talks and a barbecue at givengain — a main sponsor.

On the second day I noticed the robotic St Bernards dog in a tourist shop window had come to life.

Damian kicked off the talks with my favourite of his talks,  “Standing on the Shoulders of Giants”, starting with the Countess of Lovelace and her Bernoulli number program. This generated a strange sequence with many zeros. The Perl 6 version since it used rational numbers not floating point got the zeros right whereas the Perl 5 version initially suffered from floating point rounding errors (which are fixable).

Among other things he showed us how to define a new infix operator in Perl 6. He also showed us a Perl 6 sort program that looked exactly like LISP even down to the Lots of Irritating Superfluous Parentheses. I think this was quicksort (he certainly showed us a picture of Sir Tony Hoare at some point). Also a very functional (Haskell-like) equivalent  with heavy use of P6 Multiple Dispatch.  Also included was demonstration of P6 “before” as a sort of typeless/multi-type comparison infix. Damian then returned to his old favourite of Quantum Computing.

My mind and notes got a bit jumbled at this point but I particularly liked the slide that explained how factorisation could work by observing the product of possible inputs since this led to a collapse that revealed the factors.  To do this on RSA etc., of course, needs real hardware support which probably only the NSA and friends have (?). Damian’s code examples are at http://www.bit.do/Perl6SOG with  an earlier version of his talk at https://www.youtube.com/watch?v=Nq2HkAYbG5o Around this point there was a road race of classic cars going on outside up the main road into the village and there were car noises in the background that strangely were more relaxing than annoying.

After Quantum Chaos Paul Johnson brought us all back down to ground with an excellent practical talk on modernising legacy Perl 5 applications based on his war stories. Hell, of course, is “Other People’s Code”, often dating from Perl’s early days and lacking documentation and sound engineering.

Often the original developers had long since departed or, in the worse cases, were still there.  Adding tests and logging (with stack traces) were particularly useful. As was moving to git (although its steep learning curve meant mentoring was needed) and handling CPAN module versioning with pinto.  Many talks had spoken of the Perl 6 future whereas this spoke of the Perl 5 past and present and the work many of us suffer to pay the bills. It’s at https://www.youtube.com/watch?v=4G5EaUNOhR0

Jonathan then spoke of reactive distributed software.  A distributed system is an async one where “Is it working?” means “some of it is working but we don’t know which bits”.  Good OO design is “tell don’t ask” — you tell remote service to do something for you and not parse the response and do it yourself thus breaking encapsulation.  This is particularly important in building well designed distributed systems since otherwise the systems are less responsive and reliable.  Reactive (async) works better for distributed software than interactive (blocking or sync).

We saw a table that used a Perl 6 promise for one value and a supply for many values for reactive (async) code and the equivalent (one value) and a Perl 6 Seq for interactive code. A Supply could be used for pub/sub and the Observer Pattern. A Supply could either be live (like broadcast TV) or, for most Perl 6 supplies, on-demand (like Netflix). Then samples of networking (socket) based code were discussed including a web client, web server and SSH::LibSSH (async client bindings often very useful in practical applications like port forwarding)

https://github.com/jnthn/p6-ssh-libssh

Much of the socket code had a pattern of “react { whenever {” blocks with “whenever” as a sort of async loop.He then moved on from sockets to services (using a Supply pipeline) and amazed us by announcing the release of “cro”, a microservices library that even supports HTTP/2 and Websockets, at http://mi.cro.services/.  This is installable using Perl 6 by “zef install –/test cro”.

Slides at http://jnthn.net/papers/2017-spw-sockets-services.pdf and video at https://www.youtube.com/watch?v=6CsBDnTUJ3A

Next Lee showed Burp Scanner which is payware but probably the best web vulnerabilities scanner. I wondered if anyone had dare run it on ACT or the hotel’s captive portal.

Wendy did some cheerleading in her “Changing Image of Perl”.  An earlier version is at https://www.youtube.com/watch?v=Jl6iJIH7HdA

Sue’s talk was “Spiders, Gophers, Butterflies” although the latter were mostly noticeably absent. She promises me that a successor version of the talk will use them more extensively. Certainly any Perl 6 web spidering code is likely to fit better on one slide than the Go equivalent.

During the lightning talks Timo showed us a very pretty Perl 6 program using his SDL2::Raw to draw an animated square spiral with hypnotic colour cycling type patterns. Also there was a talk by the author about https://bifax.org/bif/— a distributed bug tracking system (which worked offline like git).

Later in the final evening many of us ate and chatted in another restaurant where we witnessed a dog fight being narrowly averted and learnt that Wendy didn’t like Perl 5’s bless for both technical and philosophical reasons.

Time for some old man's reminiscence. Or so it feels when I realize that I've spent more than 10 years involved with the Perl 6 community.

How I Joined the Perl 6 Community

It was February 2007.

I was bored. I had lots of free time (crazy to imagine that now...), and I spent some of that answering (Perl 5) questions on perlmonks. There was a category of questions where I routinely had no good answers, and those were related to threads. So I decided to play with threads, and got frustrated pretty quickly.

And then I remember that a friend in school had told me (about four years earlier) that there was this Perl 6 project that wanted to do concurrency really well, and even automatically parallelize some stuff. And this was some time ago, maybe they had gotten anywhere?

So I searched the Internet, and found out about Pugs, a Perl 6 compiler written in Haskell. And I wanted to learn more, but some of the links to the presentations were dead. I joined the #perl6 IRC channel to report the broken link.

And within three minutes I got a "thank you" for the report, the broken links were gone, and I had an invitation for a commit bit to the underlying SVN repo.

I stayed.

The Early Days

Those were they wild young days of Perl 6 and Pugs. Audrey Tang was pushing Pugs (and Haskell) very hard, and often implemented a feature within 20 minutes after somebody mentioned it. Things were unstable, broken often, and usually fixed quickly. No idea was too crazy to be considered or even implemented.

We had bots that evaluated Perl 6 and Haskell code, and gave the result directly on IRC. There were lots of cool (and sometimes somewhat frightening) automations, for example for inviting others to the SVN repo, to the shared hosting system (called feather), for searching SVN logs and so on. Since git was still an obscure and very unusable, people tried to use SVK, an attempt to implement a decentralized version control system on top of of the SVN protocol.

Despite some half-hearted attempts, I didn't really make inroads into compiler developments. Having worked with neither Haskell nor compilers before proved to be a pretty steep step. Instead I focused on some early modules, documentation, tests, and asking and answering questions. When the IRC logger went offline for a while, I wrote my own, which is still in use today.

I felt at home in that IRC channel and the community. When the community asked for mentors for the Google Summer of Code project, I stepped up. The project was a revamp of the Perl 6 test suite, and to prepare for mentoring task, I decided to dive deeper. That made me the maintainer of the test suite.

Pet Projects

I can't recount a full history of Perl 6 projects during that time range, but I want to reflect on some projects that I considered my pet projects, at least for some time.

• In early 2008 I started to contribute some patches to perl6.org. Over time I became frustrated with the outdated content on many Perl 6 websites, so I created perl6-projects.org, which only contained links to up-to-date resources. Over time, people with an actual sense of design contribute a nice layout. Later we switched perl6.org over to the content from perl6-projects.org. Its structure today is still mostly that of a link collection.
• In September 2008 I started with the Perl 5 to 6 blog series, which was the de facto resource for learning Perl 6 for some years.
• In 2009 I wrote JSON::Tiny, the first pure-Perl 6 JSON parser. I mostly did it to become familiar with grammars, and because the book "Real World Haskell" also developed a JSON module.
• May 2010 kicked off a program to make contributing to Perl 6 more accessible.
• September 2010: the Pugs SVN repository became unmaintainable. It had accumulated the official Perl 6 test suite, some documentation, modules, at least one IRC bot and a proverbial metric ton of other stuff. In coordination with #perl6, I split it up into a collection of git repositories and put them on GitHub. A decision we haven't regretted yet.
• In 2012 and 2016 I was part of the organization team of the German Perl Workshop in Erlangen and Nürnberg, which both featured several Perl 6 talks.
• 2012 saw the birth of the documentation project. Initially I was nearly the sole contributor, and I'm very happy that, over time, others have taken over.
• Between 2014 and 2015, our community server "feather" EOLed, so the Perl 6 community funded a new server, which my employer, noris network AG kindly hosts for free in their data center in Nürnberg, Germany.
• In 2016 I started writing Perl 6 by Example, which Apress later adopted and published as Perl 6 Fundamentals.
• Currently I'm working on a treatise on Perl 6 Regexes and Grammars, which Apress will also publish.

It is not quite clear from this (very selected) timeline, but my Perl 6 related activity dropped around 2009 or 2010. This is when I started to work full time, moved in with my girlfriend (now wife), and started to plan a family.

Relationships

The technologies and ideas in Perl 6 are fascinating, but that's not what kept me. I came for the technology, but stayed for the community.

There were and are many great people in the Perl 6 community, some of whom I am happy to call my friends. Whenever I get the chance to attend a Perl conference, workshop or hackathon, I find a group of Perl 6 hackers to hang out and discuss with, and generally have a good time.

Four events stand out in my memory. In 2010 I was invited to the Open Source Days in Copenhagen. I missed most of the conference, but spent a day or two with (if memory serve right) Carl Mäsak, Patrick Michaud, Jonathan Worthington and Arne Skjærholt. We spent some fun time trying to wrap our minds around macros, the intricacies of human and computer language, and Japanese food. (Ok, the last one was easy). Later the same year, I attended my first YAPC::EU in Pisa, and met most of the same crowd again -- this time joined by Larry Wall, and over three or four days. I still fondly remember the Perl 6 hallway track from that conference. And 2012 I flew to Oslo for a Perl 6 hackathon, with a close-knit, fabulous group of Perl 6 hackers. Finally, the Perl Reunification Summit in the beautiful town of Perl in Germany, which brought together Perl 5 and Perl 6 hackers in a very relaxed atmosphere.

For three of these four events, different private sponsors from the Perl and Perl 6 community covered travel and/or hotel costs, with their only motivation being meeting folks they liked, and seeing the community and technology flourish.

The Now

The Perl 6 community has evolved a lot over the last ten years, but it is still a very friendly and welcoming place. There are lots of "new" folks (where "new" is everybody who joined after me, of course :D), and a surprising number of the old guard still hang around, some more involved, some less, all of them still very friendly and supportive

The Future

I anticipate that my family and other projects will continue to occupy much of my time, and it is unlikely that I'll be writing another Perl 6 book (after the one about regexes) any time soon. But the Perl 6 community has become a second home for me, and I don't want to miss it.

In the future, I see myself supporting the Perl 6 community through infrastructure (community servers, IRC logs, running IRC bots etc.), answering questions, writing a blog article here and there, but mostly empowering the "new" guard to do whatever they deem best.

After about nine months of work, my book Perl 6 Fundamentals is now available for purchase on apress.com and springer.com.

The ebook can be purchased right now, and comes in the epub and PDF formats (with watermarks, but DRM free). The print form can be pre-ordered from Amazon, and will become ready for shipping in about a week or two.

I will make a copy of the ebook available for free for everybody who purchased an earlier version, "Perl 6 by Example", from LeanPub.

The book is aimed at people familiar with the basics of programming; prior Perl 5 or Perl 6 knowledge is not required. It features a practical example in most chapters (no mammal hierarchies or class Rectangle inheriting from class Shape), ranging from simple input/output and text formatting to plotting with python's matplotlib libraries. Other examples include date and time conversion, a Unicode search tool and a directory size visualization.

I use these examples to explain subset of Perl 6, with many pointers to more documentation where relevant. Perl 6 topics include the basic lexicographic structure, testing, input and output, multi dispatch, object orientation, regexes and grammars, usage of modules, functional programming and interaction with python libraries through Inline::Python.

Let me finish with Larry Wall's description of this book, quoted from his foreword:

It's not just a reference, since you can always find such materials online. Nor is it just a cookbook. I like to think of it as an extended invitation, from a well-liked and well-informed member of our circle, to people like you who might want to join in on the fun. Because joy is what's fundamental to Perl. The essence of Perl is an invitation to love, and to be loved by, the Perl community. It's an invitation to be a participant of the gift economy, on both the receiving and the giving end.

The Perl 6 naming debate has started again. And I guess with good reason. Teaching people that Perl 6 is a Perl, but not the Perl requires too much effort. Two years ago, I didn't believe. Now you're reading a tired man's words.

I'm glad that this time, we're not discussing giving up the "Perl" brand, which still has very positive connotations in my mind, and in many other minds as well.

And yet, I can't bring myself to like "Rakudo Perl 6" as a name. There are two vary shallow reasons for that: Going from two syllables, "Perl six", to five of them, seems a step in the wrong direction. And two, I remember the days when the name was pretty young, and people would misspell it all the time. That seems to have abated, though I don't know why.

But there's also a deeper reason, probably sentimental old man's reason. I remember the days when Pugs was actively developed, and formed the center of a vibrant community. When kp6 and SMOP and all those weird projects were around. And then, just when it looked like there was only a single compiler was around, Stefan O'Rear conjured up niecza, almost single-handedly, and out of thin air. Within months, it was a viable Perl 6 compiler, that people on #perl6 readily recommended.

All of this was born out of the vision that Perl 6 was a language with no single, preferred compiler. Changing the language name to include the compiler name means abandoning this vision. How can we claim to welcome alternative implementations when the commitment to one compiler is right in the language name?

However I can't weigh this loss of vision against a potential gain in popularity. I can't decide if it's my long-term commitment to the name "Perl 6" that makes me resent the new name, or valid objections. The lack of vision mirrors my own state of mind pretty well.

I don't know where this leaves us. I guess I must apologize for wasting your time by publishing this incoherent mess.

At FOSDEM 2015, Larry announced that there will likely be a Perl 6 release candidate in 2015, possibly around the September timeframe. What we’re aiming for is concurrent publication of a language specification that has been implemented and tested in at least one usable compilation environment — i.e., Rakudo Perl 6.

So, for the rest of 2015, we can expect the Rakudo development team to be highly focused on doing only those things needed to prepare for the Perl 6 release later in the year. And, from previous planning and discussion, we know that there are three major areas that need work prior to release: the Great List Refactor (GLR), Native Shaped Arrays (NSA), and Normalization Form Grapheme (NFG).

…which brings us to Parrot. Each of the above items is made significantly more complicated by Rakudo’s ongoing support for Parrot, either because Parrot lacks key features needed for implementation (NSA, NFG) or because a lot of special-case code is being used to maintain adequate performance (lists and GLR).

At present most of the current userbase has switched over to MoarVM as the backend, for a multitude of reasons. And more importantly, there currently aren’t any Rakudo or NQP developers on hand that are eager to tackle these problems for Parrot.

In order to better focus our limited resources on the tasks needed for a Perl 6 language release later in the year, we’re expecting to suspend Rakudo’s support for the Parrot backend sometime shortly after the 2015.02 release.

Unfortunately the changes that need to be made, especially for the GLR, make it impractical to simply leave existing Parrot support in place and have it continue to work at a “degraded” level. Many of the underlying assumptions will be changing. It will instead be more effective to (re)build the new systems without Parrot support and then re-establish Parrot as if it is a new backend VM for Rakudo, following the techniques that were used to create JVM, MoarVM, and other backends for Rakudo.

NQP will continue to support Parrot as before; none of the Rakudo refactorings require any changes to NQP.

If there are people that want to work on refactoring Rakudo’s support for Parrot so that it’s more consistent with the other VMs, we can certainly point them in the right direction. For the GLR this will mainly consists of migrating parrot-specific code from Rakudo into NQP’s APIs. For the NSA and NFG work, it will involve developing a lot of new code and feature capabilities that Parrot doesn’t possess.

This past weekend I attended the 2014 Austrian Perl Workshop and Hackathon in Salzburg, which turned out to be an excellent way for me to catch up on recent changes to Perl 6 and Rakudo. I also wanted to participate directly in discussions about the Great List Refactor, which has been a longstanding topic in Rakudo development.

What exactly is the “Great List Refactor” (GLR)? For several years Rakudo developers and users have identified a number of problems with the existing implementation of list types — most notably performance. But we’ve also observed the need for user-facing changes in the design, especially in generating and flattening lists.  So the term GLR now encompasses all of the list-related changes that seem to want to be made.

It’s a significant (“great”) refactor because our past experience has shown that small changes in the list implementation often have far-reaching effects. Almost any bit of rework of list fundamentals requires a fairly significant refactor throughout much of the codebase. This is because lists are so fundamental to how Perl 6 works internally, just like the object model. So, as the number of things that are desirable to fix or change has grown, so has the estimated size of the GLR effort, and the need to try to achieve it “all at once” rather than piecemeal.

The pressure to make progress on the GLR has been steadily increasing, and APW2014 was significant in that a lot of the key people needed for that would be in the same location. Everyone I’ve talked to agrees that APW2014 was a smashing success, and I believe that we’ve now resolved most of the remaining GLR design issues. The rest of this post will describe that.

This is an appropriate moment to recognize and thank the people behind the APW effort. The organizers did a great job.  The Techno-Z and ncm.at venues were fantastic locations for our meetings and discussions, and I especially thank ncm.at, Techno-Z, yesterdigital, and vienna.pm for their generous support in providing venues and food at the event.

So, here’s my summary of GLR issues where we were able to reach significant progress and consensus.

You are now leaving flatland

(Be sure to visit our gift shop!)

Much of the GLR discussion at APW2014 concerned flattening list context in Perl 6. Over the past few months and years Perl 6 has slowly but steadily reduced the number of functions and operators that flatten by default. In fact, a very recent (and profound) change occurred within the last couple of months, when the .[] subscript operator for Parcels switched from flattening to non-flattening. To illustrate the difference, the expression

(10,(11,12,13),(14,15)).[2]

previously would flatten out the elements to return 12, but now no longer flattens and produces (14,15). As a related consequence, .elems no longer flattens either, changing from 6 to 3.

Unfortunately, this change created a inconsistency between Parcels and Lists, because .[] and .elems on Lists continued to flatten. Since programmers often don’t know (or care) when they’re working with a Parcel or a List, the inconsistency was becoming a significant pain point. Other inconsistencies were increasing as well: some methods like .sort, .pick, and .roll have become non-flattening, while other methods like .map, .grep, and .max continue to flatten. There’s been no really good guideline to know or decide which should do which.

Flattening behavior is great when you want it, which is a lot of the time.  After all, that’s what Perl 5 does, and it’s a pretty popular language. But once a list is flattened it’s hard to get the original structure if you wanted that — flattening discards information.

So, after many animated discussions, review of lots of code snippets, and seeking some level of consistency, the consensus on Perl 6 flattening behavior seems to be:

gather {
    until not $!PIO.eof {
        $!ins = $!ins + 1;
        take self.get;
    }
 }

(while not $!PIO.eof { $!ins++; self.get });

lazy while not $!PIO.eof { $!ins++; self.get };

[This is a response to the Russian Perl Podcast transcribed by Peter Rabbitson and discussed at blogs.perl.org.]

I found this translation and podcast to be interesting and useful, thanks to all who put it together.

Since there seems to have been some disappointment that Perl 6 developers didn’t join in the discussions about “Perl 7” earlier this year, and in the podcast I’m specifically mentioned by name, I thought I’d go ahead and comment now and try to improve the record a bit.

While I can’t speak for the other Perl 6 developers, in my case I didn’t contribute to the discussion because nearly all the things I would’ve said were already being said better by others such as Larry, rjbs, mst, chromatic, etc.  I think a “Perl 7” rebrand is the wrong approach, for exactly the reasons they give.

A couple of statements  in the podcast refer to “hurting the feelings of Perl 6 developers” as being a problem resulting from a rebrand to Perl 7. I greatly appreciate that people are concerned with the possible impact of a Perl 5 rebrand on Perl 6 developers and our progress.  I believe that Perl 6’s success or failure at this point will have little to do with the fact that “6 is larger than 5”.  I don’t find the basic notion of “Perl 7” offensive or directly threatening to Perl 6.

But I fully agree with mst that “you can’t … have two successive numbers in two brands and not expect people to be confused.”  We already have problems explaining “5” and “6” — adding more small integers to the explanation would just make an existing problem even worse, and wouldn’t do anything to address the fundamental problems Perl 6 was intended to resolve.

Since respected voices in the community were already saying the things I thought about the name “Perl 7”, I felt that adding my voice to that chorus could only be more distracting than helpful to the discussion. My involvement would inject speculations on the motivations of Perl 6 developers into what is properly a discussion about how to promote progress with Perl 5.  I suspect that other Perl 6 developers independently arrived at similar conclusions and kept silent as well (Larry being a notable exception).

I’d also like to remark on a couple of @sharifulin’s comments in the podcast (acknowledging that the transcribed comments may be imprecise in the translation from Russian):

First, I’m absolutely not the “sole developer” of Perl 6 (13:23 in the podcast), or even the sole developer of Rakudo Perl 6.  Frankly I think it’s hugely disrespectful to so flippantly ignore the contributions of others in the Perl 6 development community.  Let’s put some actual facts into this discussion… in the past twelve months there have been over 6,500 commits from over 70 committers to the various Perl 6 related repositories (excluding module repositories), less than 4% (218) of those commits are from me. Take a look at the author lists from the Perl 6 commit logs and you may be a little surprised at some of the people you find listed there.

Second, there is not any sense in which I think that clicking “Like” on a Facebook posting could be considered “admitting defeat” (13:39 in the podcast). For one, my “Like” was actually liking rjbs’ reply to mst’s proposal, as correctly noted in the footnotes (thanks Peter!).

But more importantly, I just don’t believe that Perl 5 and Perl 6 are in a battle that requires there to be a conquerer, a vanquished, or an admission of defeat.

Pm