Issue DEFMACRO-VALUES
Issue: DEFMACRO-VALUES

Forum:

References: :(CLHS DEFMACRO), :(CLHS MACROLET), :(CLHS *MACROEXPAND-HOOK*), :(CLHS MACROEXPAND-1), :(CLHS MACRO-FUNCTION), the Glossary definition for value.

Category:

Edit history: 2005-09-11, Version 1 by Kalle Olavi Niemitalo
2005-09-18, Version 4 by Kalle Olavi Niemitalo

Status: For CLiki consideration

Problem Description:
According to the description of :(CLHS DEFMACRO), "the value of the last form executed is returned as the expansion of the macro." The glossary definition of value (1. b.) indicates that this means the primary value; any further values are ignored. But does this truncation occur as part of the expansion function defined by DEFMACRO, or as part of the macroexpansion process? This matters if the user calls the expansion function directly, or if the user's :(CLHS *MACROEXPAND-HOOK*) cares.

This also concerns :(CLHS MACROLET), which uses "the same format used by :(CLHS defmacro)."

Proposal (DEFMACRO-VALUES:PRIMARY):
The expansion functions defined by DEFMACRO or MACROLET always return exactly one value, even if the body attempts to return multiple values (as the values of the last form, or by using :(CLHS RETURN-FROM)). With SETF :(CLHS MACRO-FUNCTION), the user can specify a macro function that returns multiple values; but only the primary value matters when the macro function is being called by :(CLHS MACROEXPAND), :(CLHS MACROEXPAND-1), or whatever the implementation uses to expand macros.

The function that is the value of :(CLHS *MACROEXPAND-HOOK*) is not required to preserve multiple values returned by the macro function, and it might not even see all the values if another hook has interposed itself.

Rationale (DEFMACRO-VALUES:PRIMARY):
According to the description of :(CLHS DEFMACRO), "the expansion function returns a form." Make it behave that way.

Proposal (DEFMACRO-VALUES:ALL):
The expansion functions defined by DEFMACRO or MACROLET return all the values of the body, but only the primary value matters when the expansion function is being called by :(CLHS MACROEXPAND), :(CLHS MACROEXPAND-1), or whatever the implementation uses to expand macros.

The function that is the value of :(CLHS *MACROEXPAND-HOOK*) is not required to preserve multiple values returned by the macro function, and it might not even see all the values if another hook has interposed itself.

Rationale (DEFMACRO-VALUES:ALL):
This seems to be what the implementations do now, and is helpful if users want to use MACROLET as a general way to add information to the lexical environment. See "almost working WITH-SETF-EXPANDER" in Lisppaste for an example of code that would be cumbersome to write without this guarantee.

Proposal (DEFMACRO-VALUES:EXPLICITLY-VAGUE):
It is unspecified whether the expansion functions defined by DEFMACRO or MACROLET return all the values of the body, or just the primary value. With SETF :(CLHS MACRO-FUNCTION), the user can specify a macro function that returns multiple values. In any case, only the primary value matters when the expansion function is being called by :(CLHS MACROEXPAND), :(CLHS MACROEXPAND-1), or whatever the implementation uses to expand macros.

The function that is the value of :(CLHS *MACROEXPAND-HOOK*) is not required to preserve multiple values returned by the macro function, and it might not even see all the values if another hook has interposed itself.

Rationale (DEFMACRO-VALUES:EXPLICITLY-VAGUE):
Clarify the spec without requiring implementations to change.

Proposal (DEFMACRO-VALUES:THROUGHOUT):
The expansion functions defined by DEFMACRO and MACROLET return all the values of the body, but only the primary value matters when the expansion function is being called by :(CLHS MACROEXPAND), :(CLHS MACROEXPAND-1), or whatever the implementation uses to expand macros.

The function that is the value of :(CLHS *MACROEXPAND-HOOK*) will see all the values returned by the macro function, and must preserve them. Because it cannot normally know how the extra values will be used, it should pass them through unchanged.

Rationale (DEFMACRO-VALUES:THROUGHOUT):
If users define macros that return multiple values, then the implementation and user-defined hooks should propagate them as far as possible, in case some hook somewhere knows what to do with them. Admittedly, this is rather far-fetched: because :(CLHS MACROEXPAND-1) will discard the extra values anyway, users are unlikely to call it on such macros, and then the value of :(CLHS *MACROEXPAND-HOOK*) won't get called either.

Test case:
(defmacro foo ()
  (values 1 2 3))
(multiple-value-list (funcall (macro-function 'foo) '(foo) nil))
;; returns (1) for DEFMACRO-VALUES:PRIMARY,
;; returns (1 2 3) for DEFMACRO-VALUES:ALL or DEFMACRO-VALUES:THROUGHOUT,
;; returns either for DEFMACRO-VALUES:EXPLICITLY-VAGUE.

(let ((*macroexpand-hook*
       #'(lambda (function form env)
           (let ((values (multiple-value-list (funcall function form env))))
             (format *trace-output* "~&Values:~{ ~S~}~%" values)
             (values-list values)))))
  (macroexpand-1 '(foo)))
;; outputs "Values: 1" for DEFMACRO-VALUES:PRIMARY,
;; outputs "Values: 1 2 3" for DEFMACRO-VALUES:THROUGHOUT,
;; outputs either for DEFMACRO-VALUES:ALL or DEFMACRO-VALUES:EXPLICITLY-VAGUE,
;; always returns 1 and T.

Current practice:
SBCL 0.9.3.51, CLISP 2.34, and Allegro CL 6.2 seem to implement DEFMACRO-VALUES:ALL.

Cost to Implementors:
DEFMACRO-VALUES:EXPLICITLY-VAGUE is intended not to require any changes in implementations.

It seems possible that all current implementations support DEFMACRO-VALUES:ALL, in which case that would not require changes either. Some implementations may currently support DEFMACRO-VALUES:PRIMARY instead. Switching from either one to the other would require some effort, and might make macros expand a little slower.

DEFMACRO-VALUES:THROUGHOUT would additionally require implementors to examine and correct all of the functions they place in :(CLHS *MACROEXPAND-HOOK*).

Cost to Users:

DEFMACRO-VALUES:EXPLICITLY-VAGUE might make users' programs less portable, if all current implementations support DEFMACRO-VALUES:ALL and the programs have come to rely on that and some future implementation chooses to behave like DEFMACRO-VALUES:PRIMARY instead.

If all current implementations support DEFMACRO-VALUES:ALL, then making this official would not hurt users, but we don't know that yet.

Requiring DEFMACRO-VALUES:PRIMARY would break programs that haven't been careful about portability.

DEFMACRO-VALUES:THROUGHOUT would additionally require users to examine and correct all of the functions they place in :(CLHS *MACROEXPAND-HOOK*).

Cost of Non-Adoption:
Unclear spec. Users don't know what they can rely on, and may assume too much in programs intended to be portable.

Benefits:

Aesthetics:

Discussion: