There are many things to like about Common Lisp packages. In everyday programming, I really like being able to define one or more packages in one file then just have a (in-package package-name) at the top of other files opposed to having sometimes hundreds of lines of imports. Some programming languages are worse than others, but no other programming language that I know of allows one to keep imports in a totally different file.

The real point of this post is to talk about a specific scenario that reveals in interesting interaction between asdf systems (libraries), symbols, and macros. A conversation about packages almost always involves talking about symbols. Here is the situation: there is a library, A, that implements some very useful computations. The public interface for the library is a combination of macros and functions. It is rather verbose though, so you want to wrap it in a macro and create a new library: B. You want the users of your library to only have to use your library and not have to worry about depending directly on library A, so you want to re-export some of the symbols from library A so the users of your library can just use those symbols exported from your library.

If you don’t see a problem with this then congratulations: you are sane. This relies on just a couple features:

  • feature of asdf: orthogonal to packages meaning that one can use a symbol from any package without having to explicitly load the system. If a package is loaded, it’s symbols can be used.

  • feature of packages: symbols in a package can be referred to by their full name (package-name:symbol-name) anywhere in anyone’s program. It does not matter if they have the symbol imported into their package. It does not matter if they have a package local nickname. It does not matter that they have a different symbol interned with the same symbol-name.

  • macros do not have any feature of their own that make this possible, but because of the previously mentioned features, using a symbol from another library in a macro expansion works just fine. As an added benefit, you cannot expand to symbol from a package that is not exported or from a non-existent package because the reader (which happens before macro expansion, even before macro definition) makes sure all the symbols are accounted for.

I witnessed someone in almost this exact situation. They were using Rust. The problem happened when someone tried to use the macro from library A that had been re-exported from library B. The macro expanded to use some symbols that were qualified assuming that the user had already imported them from library A. However, the user imported from library B. The work around ended up being to not re-export and just tell the user via documentation that they have to explicitly depend on library A, and use their macro.

That is the story. Let me know what you think, and tell me about a time that you were grateful for the way Common Lisp handles packages and symbols.

  • @CorngoodEnglish
    2•3 months ago

    You’re making me want to find a hobby project to do in CL. Most of my Lisp experience has been in scheme or elisp.

  • Ramin HonaryEnglish
    2•3 months ago

    no other programming language that I know of allows one to keep imports in a totally different file.

    Isn’t this (in-package ...) form just a namespace feature? I know C++ and C# both have similar features, where you declare classes to be members of a namespace, but the collection of classes in the namespace need not be declared in any one file, it is assembled by the compiler as it discovers declarations across multiple files. You refer to symbols in other namespaces with the namespace::symbol syntax in C++ and namespace.symbol in C# . However I don’t know the rules for hiding or re-exporting certain symbols.

    Haskell and Scheme are two other languages that I know well enough that I can say there are ways of importing all or only specific symbols from other pakcages, and re-exporting symbols from other packages to construct the API of any given package.

    It sounds to me, from your description, that Rust still needs to get its shit together with namespaces. Or maybe they already have but the code you were looking at was older and written at a time before Rust had namespaces? I don’t know Rust well enough, so I can’t say.

    • @charjeOPEnglish
      3•3 months ago

      Isn’t this (in-package …) form just a namespace feature?

      It is not only a namespace feature; it is first class. You can use find-package to fetch a package object and pass it around and manipulate. It looks like this can be done to some extent in C# as well.

      For C# namespaces, exports carry over to different uses of the same namespace, but using statements go out of scope at the end of the current namespace use. That signals to me that imports are not related at all to namespaces. It seems that C++ acts similarly. From what I can tell Both C# and C++ allow for fully qualified names without any sort of import.

      I have to agree with you about Rust: they goofed it up my trying to make too many rules about which libraries and symbols can be used in which places.