Less XS/More Ctypes

Perl is in better shape now than ever, but there's much left to improve. (I wrote Five Features Perl 5 Needs Now almost two years ago, and four of five are available in whole or in sufficient substitute, so that's great progress.)

Rather than criticize Perl for what's going wrong (as if I were any position to do so), I prefer to suggest new areas of improvement.

One of the biggest possible improvements to the world of Perl 5 is a replacement for many cases of XS. Ryan Jdoubi's Ctypes for Perl was on my original list, and it's still on my list.

Why is this important?

XS can be inscrutable. Try reading the documentation, spread out between perlxstut and perlapi, with liberal use of the Cozens and Jenness book Extending and Embedding Perl and, ultimately, trawling through—even copying and pasting—chunks of the Perl 5 core which seems to do what you want.

Now debug typemap problems, problems with contexts in threaded versus unthreaded builds, and the vagaries of Makefile syntax across various platforms, not to mention the joy of detecting whether you have the right version of a shared library and its development headers on a version of an operating system you've never seen, let alone used, and (oh yes) Windows.

It's no wonder people want pure-Perl modules when possible.

It's also no wonder that, with all of the copying and pasting and reading of the Perl 5 source code in lieu of a working, understandable example of real-life XS code we can modify to our own purposes, far too many CPAN modules and an unkonwn and scary amount of DarkPAN modules abuse the Perl 5 internals in such a way that upgrading to a newer version of Perl seems like soul-swallowing black magic.

You try refactoring the internals sometime, then watch the black smoke come in from the next monthly bleadperl release and sob softly as you put back all of the mess and hope for the best.

If Perl 5's internals have much hope of refactoring and cleanup and sanity, the less XS code the better. Ctypes for Perl can help by reducing the need for XS. If most of the XS modules which are XS only because there was previously no other obviously better way to pass data between Perl and shared libraries, migrating them to Ctypes can remove the need for XS. Improving the interface between Perl and C is much easier because only Ctypes needs to change.

We won't entirely get rid of XS, but we can minimize its use to those projects which really, really need it The fewer of those, the easier they are to audit and review and port, if necessary.

The cleaner the internals of Perl 5 (and the better its extension interfaces in sanity, cleanliness, safety, composability, and documentation), the more possibilities for further development and improvements.

(I also expect that modules which use Ctypes instead of XS will be smaller, faster, easier to write, less memory hungry, and easier to install and to redistribute.)