1. New Features in
Clojure
Rich Hickey

2. Overview
• Unified primitive semantics
• Map key equality
• Improved interaction between dynamic
binding and thread pools
• Explicitly dynamic vars
• Primitive support in fns
• This is all work-in-progress!

3. Objective - Unify
Semantics
• Unify primitive and boxed integer semantics
• Currently, boxed integer arithmetic
promotes to bigint as needed
• Primitive arithmetic throws on overflow
• Means that you always have to be explicit
about primitives
• difficulty with representation choices
• incidental boxing changes meaning

4. Objective - Make Fast
Numerics Simple
• Would like to have literal numbers be used
as primitives when possible
• Ditto primitive method returns
• Coercion and primitive ops currently for
experts only
• Easy to fail to optimize

5. Unified Integer Semantics
• The semantics of primitive longs, plus throw on
overflow
• This is the only semantic that works for both
boxed and primitive
• For bigint arithmetic, specify bigint operands
• new literal bigint format: 42N
• Polymorphic boxed numeric tower still in place
• bigints now contagious, no more auto-
reductions

6. Numeric Features
• Literal numbers are now treated implicitly
as primitives
• Locals known to be numbers are
automatically coerced to long or double
• More automatic conversions
• int matches long, float matches double
• Consistent canonic integer boxing
• If fits in long, is Long

7. Numeric Features
• longs also match ints, and doubles match floats
• long->int is checked
• Can’t force boxed representation from
primitive coercion:
• was always a bug to presume otherwise
• (class (int 42)) => Long
• ditto boxed values coming from arrays of
primitives, and primitive returns from
interop

8. Auto-promotion
• No longer the default
• Still available explicitly
• ’ is now a constituent character (like #)
• +', -', *', inc', dec' do auto-promotion
• never return primitives
• Unless you need to work with arbitrary-
precision integers where BigInt contagion
is insufficient, there is no reason to use
them.

9. Objective - Align Map
Keys and Numeric =
• Users frustrated when insert with one key
representation, lookup fails with another
• Use (type flexible) equivalence for map =
• but narrow equiv, only same category
numbers are =
• so floats not = to integers
• but == still available for that
• new clojure.lang.BigInt class

10. Map Key Equality
• Hash maps and sets now use = for keys
• this will make maps and sets unify 42 and
42N, since using =
• Will still use stricter .equals when calling
through java.util interfaces
• Not there yet
• this will require renaming
Associative.containsKey, since semantics
will now differ

11. Objective - Make Binding
Work with Threads
• When work is sent to thread-pool threads,
e.g. via agent sends or future calls, the work
isn't done with the same binding set as the
invocation (without manual effort)
• Yet that is often the expectation/desire
• Makes transparent parallelization difficult
• Current tools for propagating bindings are
expensive

12. Binding Issues
• Intended to be used within one thread
• Don’t support a notion of task or unit of
work crossing threads
• Unsafe if used between threads
• thus, must be copied
• i.e., new bindings created for subtask
thread

13. Thread Issues
• Threads don’t align with work tree
• And when threads used for subtasks, not
usually explicitly launched as children
• thread pool threads (re-)used instead
• Need better notion of subtasks not aligned
with thread stack

14. Shared Bindings
• Allow subtasks in thread pool threads to
share bindings with parent task
• Make binding access safe from subtask threads
• Add volatile semantics for binding values
• Ensure vars only set! from binding thread
• Sharing bindings is extremely fast
• Just use same (immutable) binding map in
subtask thread

15. Binding Conveyance
• Bindings are now automatically shared with
threads used for future calls and agent
sends
• and anything built on them, e.g. pmap
• This reroots the binding frame for thread
pool thread prior to your work fn running
• so bindings in work fn ‘win’
• Just works

16. Binding and Scopes
• Much of this work driven by needs of
resource scopes
• Natural fit for binding
• But need same notion of task tree
• and without conveyance, would have
same thread problems/surprises as
bindings
• Scopes work can now proceed

17. Objective - Pay for
What You Use
• All defs currently create fully dynamically
rebindable vars
• Incur cost for every access
• Is it dynamically bound?
• Is it unbound?
• What’s the current value
• Yet most vars (defns) have rarely-changing
values, never dynamically bound

18. Why Top Level Vars?
• Traditional context-bound dynamic
variables, like *out* et al
• Dynamically bound fns can be used for
error handling
• Replace fns with fixed versions
• Doesn’t require rebinding
• Mutable globals

19. Be Explicit about
Dynamic Intent
• Change default semantic from dynamically
rebindable to not
• Must declare vars dynamic using
• ^:dynamic metadata
• or .setDynamic on manually created Vars
• Better for consumers
• Enables significant optimizations

20. Metadata Features
• New ^:keyword metadata
• turns into ^{:keyword true}
• Metadata stacks - ^:dynamic ^:private foo
• Merges rather than replaces
• same as ^{:dynamic true :private true} foo

21. Remove Overhead
• Remove binding check from access to non-
dynamics
• Remove bound check from normal access
• instead return Unbound object
• has the var as field, prints in toString
• implements IFn and throws on any
invoke, with good message

22. Presume Stability
• Replace via defn to fix bugs case still remains
• infrequent, but would like to avoid pain of
static (i.e., re-eval of caller code)
• Minimize overhead
• cache var values in fn
• global var rev, inc on any var alteration
• check rev on fn entry and reload vars if
changed
• else use cached values

23. Objective - Support
Primitive Args/Returns
• Clojure supports primitive arithmetic
locally, and primitives in deftype/defrecord
• But not in fn arguments or returns
• Means you can’t break up your logic, or
write helper functions, without
introducing boxing

24. Issues
• fn is specified by an interface (IFn), taking
and returning Objects
• fns must still satisfy that interface
• How will callers know about primitive
params/return, and how to invoke?
• Support ^long and ^double type hints on
args/return

25. IFn$LOD...
• New set of single-method nested IFn
interfaces
• Corresponding to all combinations of
Object/long/double, to arity 4
• Compiler will compile invokePrim methods
in addition to IFn virtual invoke methods
• When compiling direct invocation of prim fn,
compiler will look at arglists metadata and
make call to invokePrim method
• var still contains IFn object for HOFs etc

26. More Features
• ^long and ^double hints supported on args
• hint for return goes on arg vector
• e.g. (defn foo ^long [x] ...)
• so supports multi-arity with varied returns
• Other reference type hints allowed as well
• Just for hints, not enforced

27. Future Fns
• Primitive-taking fn interfaces open door to
non-boxing HOFs
• One more step to dynamically typed, yet
internally primitive, map and reduce
• When combined with collections of
primitives
• and chunked seqs

28. Summary
• New features enable much faster code
• Unified semantics reduce complexity
• all features are still available
• auto-promotion is explicit
• dynamic binding is explicit
• Optimization more automatic, less work, less
subtle
• Pave the way for scopes and primitive HOFs