Features of Common Lisp

Rich, exact arithmetic:

Lisp provides a rich hierarchy of numbers that are well-integrated with the rest of the language.

Bignums are created implicitly, mitigating the risk of overflows while maintaining precision. For example, we can quickly calculate the correct value of 10⇈4:

> (expt (expt (expt (expt 10 10) 10) 10) 10)
100000000000000000000000000000000000[...]

Rational numbers remain as ratios, so no rounding errors are incurred. Exact rational arithmetic is well-integrated in Lisp:

> (+ 5/9 3/4)
47/36

Complex numbers are a built-in type in Lisp. They can be expressed with concise syntax: #c(10 5) represents 10 + 5i. Arithmetic operations support complex values:

> (* 2 (+ #c(10 5) 4))
#C(28 10)

Generalized references:

Forms or places in Lisp can be treated as if they were concrete, mutable variables. These references are used with SETF and others to update the values conceptually stored by a place.

For example, using SETF on a place:

> (defvar *colours* (list 'red 'green 'blue))
*COLOURS*
> (setf (first *colours*) 'yellow)
YELLOW
> *colours*
(YELLOW BLUE GREEN)

And using PUSH:

> (push 'red (rest *colours*))
(RED BLUE GREEN)
> *colours*
(YELLOW RED BLUE GREEN)

Generalized references can be used with many kinds of structures and objects, not just lists. For example, in object-oriented programs, one way to update an object's field would be to use SETF with its accessor form.

Multiple values:

Values may be grouped without resorting to an explicit structure such as a list. For example, (values 'foo 'bar) returns two values: 'foo and 'bar. With this device, functions may return multiple values, which can simplify program logic.

For example, FLOOR is a built-in function that returns two values:

> (floor pi)
3
0.14159265358979312d0

Conveniently, functions that return multiple values are treated by default as though only the first value was returned:

> (+ (floor pi) 2)
5

However, we can explicitly capture and use the secondary values. Here we re-assemble PI after flooring it:

> (multiple-value-bind (integral fractional)
      (floor pi)
    (+ integral fractional))
3.141592653589793d0

Macros:

A Lisp macro is like a function that takes Lisp forms or objects as input, and typically generates code to be then compiled and executed. This happens before runtime, in a phase called macroexpansion time. Macros can perform arbitrary computation during expansion, using the full Lisp language.

One use of macros is transforming input representing arbitrary source code into a version specified in terms of known definitions. In other words, macros can add new syntax to the original language (this is known as syntactic abstraction).

This enables easily embedding domain-specific languages, since specialized syntax can be added before runtime.

The chief benefit of macros is that they add power by letting the programmer express intent clearly and with less code. Particularly, one can add new features to the language that appear as if they were builtin. In addition, when used to pre-emptively compute data or initialize state, macros may aid in performance optimisation.

The LOOP macro:

The LOOP macro is a powerful built-in iteration tool. In fact, it is a small embedded language for describing iterative processes. LOOP provides all kinds of expressions for iteration, from basic repetition to sequence iterators to sophisticated state machines.

> (defvar *list*
    (loop :for x := (random 1000)
          :repeat 5
          :collect x))
*LIST*
> *list*
(324 794 102 579 55)

> (loop :for elt :in *list*
        :when (oddp elt)
        :maximizing elt)
579

> (loop :for elt :in *list*
        :collect (log elt) :into logs
        :finally
        (return
         (loop :for l :in logs
               :if (> l 5.0) :collect l :into ms
                 :else :collect l :into ns
               :finally (return (values ms ns)))))
(5.7807436 6.6770835 6.3613024)
(4.624973 4.0073333)

The FORMAT function:

The FORMAT function contains an embedded language to describe the way data should be formatted. More than just simple text substitution, FORMAT instructions are capable of densely expressing rules for the way text is generated, such as conditionals, iteration and corner cases.

We can format a list of names with the correct punctuation using a function like this:

(defun format-names (list)
  (format nil "~{~:(~a~)~#[.~; and ~:;, ~]~}" list))

> (format-names '(doc grumpy happy sleepy bashful
                  sneezy dopey))
"Doc, Grumpy, Happy, Sleepy, Bashful, Sneezy and Dopey."
> (format-names '(fry laurie))
"Fry and Laurie."
> (format-names '(bluebeard))
"Bluebeard."

FORMAT outputs to streams — and thus to standard output, strings or any other stream.

Functional functions:

Functions are truly first-class. Function objects can be created dynamically, passed as arguments, or returned. Consequently, functions are higher-order as they may accept other functions as arguments, or return functions.

Here is a call to SORT, which takes a list and another function (in this case, #'<) as arguments:

> (sort (list 4 2 3 1) #'<)
(1 2 3 4)

Anonymous functions, defined with lambda expressions, are used in place of function names when calling a function. They are especially useful for creating one-shot functions that are passed to higher-order functions without littering any namespaces. More generally, they are useful for creating lexical closures.

In this example, we create an anonymous function as the first argument in a call to MAPCAR:

> (mapcar (lambda (x) (+ x 10))
          '(1 2 3 4 5))
(11 12 13 14 15)

Functions capture their lexical environment when they are created, yielding full lexical closures:

(let ((counter 10))
  (defun add-counter (x)
    (prog1
      (+ counter x)
      (incf counter))))

> (mapcar #'add-counter '(1 1 1 1))
(11 12 13 14)
> (add-counter 50)
64

List processing:

Since lists are a fundamental built-in type in Lisp, there is an extensive programming interface for operating on lists. Because of these functions and macros, lists as sequences may be useful in rapid prototyping of other structures.

For example, we could play with a proper list:

> (defvar *nums* (list 0 1 2 3 4 5 6 7 8 9 10 11 12))
*NUMS*
> (list (fourth *nums*) (nth 8 *nums*))
(3 8)
> (list (last *nums*) (butlast *nums*))
((12) (0 1 2 3 4 5 6 7 8 9 10 11))
> (remove-if-not #'evenp *nums*)
(0 2 4 6 8 10 12)

Or perhaps an association list:

> (defvar *capital-cities* '((NZ . Wellington)
                             (AU . Canberra)
                             (CA . Ottawa)))
*CAPITAL-CITIES*
> (cdr (assoc 'CA *capital-cities*))
OTTAWA
> (mapcar #'car *capital-cities*)
(NZ AU CA)

Lambda lists:

A lambda list specifies the parameters and protocol of a function, macro, binding form or certain other constructs. Lambda lists define required, optional, named, rest or auxiliary parameters, and default values, and more. This permits highly flexible and expressive protocols.

Optional parameters do not require a value to be supplied by the caller. A default value may be defined, or the called code check if a value was provided, and proceed accordingly.

This function takes an optional delimiter parameter that defaults to a space character:

(defun explode (string &optional (delimiter #\Space))
  (let ((pos (position delimiter string)))
    (if (null pos)
        (list string)
        (cons (subseq string 0 pos)
              (explode (subseq string (1+ pos))
                       delimiter)))))

We can supply or omit the optional argument when calling EXPLODE:

> (explode "foo, bar, baz" #\,)
("foo " " bar " " baz")

> (explode "foo, bar, baz")
("foo," "bar," "baz")

Named arguments are like optional arguments, but can be sent in any order because they are identified by keywords. Using keywords enhances readability and self-documentation in calls with several arguments.

For instance, compare these two versions of a function call:

// In C:
xf86InitValuatorAxisStruct(device, 0, 0, -1, 1, 0, 1);

;; In Lisp:
(xf86-init-valuator-axis-struct :dev device :ax-num 0
                                :min-val 0 :max-val -1
                                :min-res 0 :max-res 1
                                :resolution 1)

First-class symbols:

Symbols are objects uniquely identified by their name. For example, 'foo is a symbol, whose name is "FOO". Symbols can be used to name identifiers, or as general tokens. Symbol comparison happens in constant time.

Symbol objects, like functions, are first-class citizens. They may be created dynamically, quoted (unevaluated) and referenced, stored, passed as arguments, imported and exported, compared, and converted to strings.

Here, for instance, '*foo* is an identifier for a variable:

> (defvar *foo* 5)
*FOO*
> (symbol-value '*foo*)
5

First-class packages:

Packages, which establish namespaces, are also first-class objects¹. Since they can be created, stored, returned and passed as arguments at runtime, it is possible to dynamically switch contexts or operate across namespaces at runtime.

In this example, we use INTERN to enter a symbol into a particular package.

> (intern "ARBITRARY"
          (make-package :foo :use '(:cl)))
FOO::ARBITRARY
NIL

Lisp maintains a special variable called *package* which is bound to the current package. For example, if we were in the package FOO:

> (in-package :foo)
#<PACKAGE "FOO">
> (package-name *package*)
"FOO"

Special variables:

Lisp allows dynamic scope in addition to lexical scope. Dynamically scoped variables may be practical in some cases, so this permits maximum flexibility.

For example, we might redirect output from some code to a non-standard location, such as a file stream, by dynamically rebinding the built-in special variable *standard-output*:

(with-open-file (file-stream #p"somefile"
                 :direction :output)
  (let ((*standard-output* file-stream))
    (print "This prints to the file, not stdout."))
  (print "And this prints to stdout, not the file."))

Along with *standard-output*, Lisp uses a number of special variables to store state, including resources and options, such as *standard-input*, *package*, *readtable*, *print-readably*, *print-circle*, and others.

Control transfer:

Lisp has two means of transferring control from one point to an exit point higher in a given scope. This can be a lexical or dynamic scope, for local or nonlocal jumps, respectively.

Named blocks allow a lexically nested form to return values from any named parent form, using BLOCK and RETURN-FROM.

For example, the inner loop returns its list from early, bypassing the outer loop:

> (block early
    (loop :repeat 5 :do
      (loop :for x :from 1 :to 10 :collect x :into xs
            :finally (return-from early xs))))
(1 2 3 4 5 6 7 8 9 10)

Catch/throw is a kind of nonlocal goto. THROW causes a jump to the most recent matching CATCH form, exiting at that point, and returning the values given in the THROW form.

Given a function THROW-RANGE, based on the last example, we can use THROW and CATCH instead, exploiting dynamic scope:

(defun throw-range (a b)
  (loop :for x :from a :to b :collect x :into xs
        :finally (throw :early xs)))

> (catch :early
    (loop :repeat 5 :do
      (throw-range 1 10)))
(1 2 3 4 5 6 7 8 9 10)

Named blocks are preferred when the exit point needs only lexical scope, and catch/throw when dynamic scope is necessary.

Conditions, restarts:

Lisp's condition system is a mechanism for transmitting signals between parts of a program.

One use is to signal exceptions and handle them, roughly similar to Java or Python. However, unlike those systems, as a Lisp condition propagates upwards, the stack is not unwound, preserving data and allowing the condition's handler to restart at any point down the stack.

This style of handling exceptional circumstances allows better delegation of tasks in potentially more structured code. But it has conceivably more general applications too, as a system of signalling arbitrary messages (not just errors) between parts of a program.

For an example of the condition system in use, refer to: Common Lisp: A Tutorial on Conditions and Restarts.

Generic Functions:

The Common Lisp Object System (CLOS) does not associate methods with classes, but rather under generic functions.

Generic functions specify method interfaces that several methods can specialize on for different cases. Calls to such methods will be dispatched to the most specialized method matching the arguments.

Here we define a generic function to respond to keyboard input events:

(defgeneric key-input (key-name))

Now we define a number of methods specializing on different values of KEY-NAME:

(defmethod key-input (key-name)
  ;; Default case
  (format nil "No keybinding for ~a" key-name))

(defmethod key-input ((key-name (eql :escape)))
  (format nil "Escape key pressed"))

(defmethod key-input ((key-name (eql :space)))
  (format nil "Space key pressed"))

To see the dispatching in action:

> (key-input :space)
"Space key pressed"
> (key-input :return)
"No keybinding for RETURN"
> (defmethod key-input ((key-name (eql :return)))
    (format nil "Return key pressed"))
> (key-input :return)
"Return key pressed"

We have avoided using a contiguous switch-like dispatching construct, or an explicit dispatch table. Thus, we can add new specialized cases independently, dynamically, incrementally or from elsewhere in the program. This, in part, enables the bottom-up growth found in Lisp systems.

Generic functions define several characteristics of the group of methods. For instance, method combinations, specialization options, and other features can be tuned by a generic function.

Lisp provides many useful built-in generic functions; for example, PRINT-OBJECT, which may be specialized for any class, to produce a printed representation particular to that class.

Method combination:

Method combinations allow a defined chain of methods to be invoked whenever a particular method is called, in some order, or with some function combining their results.

There is a built-in method combination that arranges matching method calls in some order. Methods qualified by the keywords :before, :after or :around will be given that place in the invocation chain for some arguments.

For example, in the previous example, each KEY-INPUT method repeats the phrase "key pressed". We could refactor this using an :around method:

(defmethod key-input :around (key-name)
  (format nil "~:(~a~) key pressed"
          (call-next-method key-name)))

Now we can redefine our other KEY-INPUT to return just the appropriate string, such as:

(defmethod key-input ((key-name (eql :escape)))
  "escape")

When KEY-INPUT is called:

• the :around method is called instead,
• it calls the next method — one of the specializing KEY-INPUT methods,
• which returns a string that the :around method formats.

Note that the default case must be handled differently. We could simply use a THROW/CATCH pair (a better implementation might use conditions):

(defmethod key-input (key-name)
  (throw :default
    (format nil "No keybinding for ~a" key-name)))

(defmethod key-input :around (key-name)
  (catch :default
    (format nil "~:(~a~) key pressed"
            (call-next-method key-name))))

Thus, the built-in method combination allows us to generalize the keyboard event processing behaviour with a modular, extensible, reprogrammable pattern. This can be advanced with arbitrary user-defined combinations; for example, summing or sorting method call results.

Multiple inheritance:

A class may have multiple superclasses, allowing richer models and greater code reuse. The semantics of composite classes are determined according to an order of precedence, based on the superclass declarations.

With method combinations, the MOP and other CLOS features, classic problems of multiple inheritance (such as fork-join) can be mitigated.

Meta-object protocol:

Lisp's meta-object protocol (MOP) is an interface to the CLOS, itself implemented with the CLOS. The MOP allows programmers to examine, use and modify the internals of the CLOS, using CLOS.

First-class classes:

Classes themselves are also first-class objects. With the MOP, the definition and behaviour of class objects may be modified.

Given a class FOO extending BAR, we can use the ENSURE-CLASS function in the MOP to modify its superclasses to add BAZ:

(defclass bar () ())
(defclass foo (bar) ())
(defclass baz () ())

> (class-direct-superclasses (find-class 'foo))
(#<STANDARD-CLASS BAR>)
> (ensure-class 'foo :direct-superclasses '(bar baz))
#<STANDARD-CLASS FOO>
> (class-direct-superclasses (find-class 'foo))
(#<STANDARD-CLASS BAR> #<STANDARD-CLASS BAZ>)

We use the MOP function CLASS-DIRECT-SUPERCLASSES to retrieve the superclass properties of a class — here it takes the class object returned by FIND-CLASS as an argument.

The above illustration shows the mechanism by which classes can be modified at runtime enabling, for instance, dynamic insertion of mixins.

Dynamic redefinitions:

Lisp is highly interactive and dynamic. Functions, macros, classes, packages, parameters and objects can be redefined at virtually any time, with sensible and predictable results.

For example, redefining a class at runtime will cause modifications to propagate immediately among instances and subclasses of the class. We might define a class BALL with a radius, and a subclass TENNIS-BALL:

> (defclass ball ()
    ((%radius :initform 10 :accessor radius)))
#<STANDARD-CLASS BALL>
> (defclass tennis-ball (ball) ())
#<STANDARD-CLASS TENNIS-BALL>

Here is an instance of TENNIS-BALL, with one slot for the radius:

> (defvar *my-ball* (make-instance 'tennis-ball))
*MY-BALL*
> (radius *my-ball*)
10

Now let us redefine BALL, adding a new slot for volume:

> (defclass ball ()
    ((%radius :initform 10 :accessor radius)
     (%volume :initform (* 4/3 pi 1e3)
              :accessor volume)))
#<STANDARD-CLASS BALL>

And automatically, *MY-BALL* was updated with the new slot as its superclass was redefined:

> (volume *my-ball*)
4188.790204786391d0

Compiler at runtime:

With the COMPILE and COMPILE-FILE functions, the Lisp compiler is accessible directly at runtime. Functions created or redefined at runtime, therefore, may also enjoy the benefits of compilation.

This enables incremental compilation of programs, encouraging a highly interactive, dynamic and rapid development methodology. Running instances of programs can be updated, debugged and grown gradually.

Compiler macros:

Compiler macros define alternate compilation strategies for a function or macro. Unlike a regular macro, a compiler macro does not extend the syntax of the language, and may be expanded only during compilation. Therefore, it is mainly useful for specifying optimisations on code, separately from its normal definition.

Type declarations:

Though Lisp is dynamically typed — which is handy while prototyping — the programmer may choose to declare types of variables. This, in addition to other compiler hints, allows the compiler to optimise code, earning performance benefits as in any statically typed language.

To illustrate, we might declare the types of the parameters in our EXPLODE function, like so:

(defun explode (string &optional (delimiter #\Space))
  (declare (type character delimiter)
           (type string string))
  ...)

Programmable parser:

Lisp's reader enables easily tokenizing and parsing input. The reader takes text from an input stream to produce Lisp objects known as s-expressions. This can greatly simplify parsing.

The reader is accessible through several functions including READ, READ-CHAR, READ-LINE, READ-FROM-STRING and others. Input streams can include file streams, standard input, and so on, but we can read from strings or sequences with the appropriate functions as well.

Here is a trivial example using READ-FROM-STRING that produces the Lisp object (400 500 600), a list, from the text (400 500 600):

> (read-from-string "(400 500 600)")
(400 500 600)
13
> (type-of (read-from-string "t"))
BOOLEAN

Reader macros associate special semantics to custom syntax. This is possible because Lisp's reader is programmable. Reader macros are another means of extending Lisp syntax (usually to add syntactic sugar).

Some built-in reader macros:

• #'foo for functions,
• #\\ for literal characters,
• #c(4 3) for complex numbers,
• #p"/path/" for filesystem paths,

The reader can produce any Lisp object that it knows how to read; this includes anything programmed with reader macros. In fact, it is the same reader that always processes Lisp code in the read-eval-print loop.

Here we create a number using hexadecimal syntax provided by a built-in reader macro:

> (read-from-string "#xBB")
187

Programmable unparser:

Lisp's print system allows for structures, objects, or arbitrary data to be printed in various ways.

PRINT-OBJECT is a built-in generic function taking an object and a stream, whose methods print a representation of the specialised object to the stream. This is called anywhere the object's printed representation is needed, such as in FORMAT, PRINT, or at the REPL.

Consider a class JOURNEY:

(defclass journey ()
  ((%from :initarg :from :accessor from)
   (%to :initarg :to :accessor to)
   (%period :initarg :period :accessor period)
   (%mode :initarg :mode :accessor mode)))

If we try to print a JOURNEY object, we get output like this:

> (defvar *journey*
    (make-instance 'journey
                    :from "Christchurch" :to "Dunedin"
                    :period 20 :mode "bicycle"))
*JOURNEY*
> (format nil "~a" *journey*)
"#<JOURNEY {10044DCCA1}>"

We can define a PRINT-OBJECT method specialising on JOURNEY that represents the object in an arbitrary way:

(defmethod print-object ((j journey) (s stream))
  (format s "~A to ~A (~A hours) by ~A."
          (from j) (to j) (period j) (mode j)))

Our JOURNEY object will now appear using our custom printed representation:

> (format nil "~a" *journey*)
"Christchurch to Dunedin (20 hours) by bicycle."