Before clojure.spec, our FIX message parser had a test suite with 40 hand-written test cases. We’d been running it in production for 18 months without incident.

After we added spec and ran the property-based tests overnight, it found 7 edge cases we hadn’t written tests for — including one where a negative zero value (-0.0) in a price field caused the downstream risk calculation to produce NaN, which propagated silently through the pipeline and ended up in the regulatory report as a blank field.

That was the end of hand-written validation tests for external data.

What clojure.spec Provides

clojure.spec is a library for describing the structure of data and functions. It provides:

  • Validation: is this data valid according to this spec?
  • Conformation: parse and transform data according to a spec
  • Generation: generate random valid examples of data matching a spec
  • Instrumentation: automatically check function arguments and return values against specs

For financial data validation, the first three are immediately useful.

Defining Specs for Trade Data

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19

(require '[clojure.spec.alpha :as s])
(require '[clojure.spec.gen.alpha :as gen])

;; Basic field specs:
(s/def ::symbol (s/and string? #(re-matches #"[A-Z]{3}/[A-Z]{3}" %)))
(s/def ::price  (s/and double?
                       pos?                    ; must be positive
                       #(not (Double/isNaN %)) ; not NaN
                       #(< % 100.0)))          ; sanity check: no valid FX rate > 100
(s/def ::quantity (s/and int? pos?))
(s/def ::side #{:buy :sell})
(s/def ::venue #{:ebs :reuters :currenex :lmax :hotspot})
(s/def ::timestamp (s/and inst? #(.isBefore (java.time.Instant/EPOCH)
                                            (.toInstant %))))

;; Composite spec for a trade:
(s/def ::trade
  (s/keys :req-un [::symbol ::price ::quantity ::side ::venue ::timestamp]
          :opt-un [::trader-id ::account-id]))

Now validation is declarative:

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15

(s/valid? ::trade {:symbol "EUR/USD"
                   :price 1.28445
                   :quantity 10000000
                   :side :buy
                   :venue :ebs
                   :timestamp (java.util.Date.)})
;; => true

(s/valid? ::trade {:symbol "EUR/USD"
                   :price -0.0    ;; negative zero!
                   :quantity 10000000
                   :side :buy
                   :venue :ebs
                   :timestamp (java.util.Date.)})
;; => false (pos? returns false for -0.0)

s/explain shows exactly which predicate failed:

1
2
3
4
5
6
7
8

(s/explain ::trade {:symbol "EURUSD"  ;; missing slash
                    :price 1.28445
                    :quantity 10000000
                    :side :buy
                    :venue :ebs
                    :timestamp (java.util.Date.)})
;; => In: [:symbol] val: "EURUSD" fails spec: ::symbol at: [:symbol]
;;    predicate: (re-matches #"[A-Z]{3}/[A-Z]{3}" %)

This is the output you show to the operations team when an upstream system sends bad data.

Generative Testing: Finding the Unknown Unknowns

The real value of spec is generative testing. Specs don’t just validate — they can generate random valid examples. Libraries like test.check use these generators to run hundreds of random inputs through your code looking for failures.

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11

(require '[clojure.spec.test.alpha :as stest])
(require '[clojure.test.check :as tc])
(require '[clojure.test.check.properties :as prop])

;; Generate 1000 random valid trades and run them through the pipeline:
(tc/quick-check 1000
  (prop/for-all [trade (s/gen ::trade)]
    (let [result (process-trade trade)]
      (and (s/valid? ::processed-trade result)
           (not (Double/isNaN (:pnl result)))
           (not (Double/isInfinite (:pnl result)))))))

This generates 1000 random trades (random symbols, random prices within spec bounds, random quantities) and asserts that the output is always a valid processed trade with finite P&L.

The generator can be customised to produce more realistic distributions:

1
2
3
4
5
6
7

;; Override the default double generator to produce realistic FX prices:
(s/def ::price
  (s/with-gen
    (s/and double? pos? #(not (Double/isNaN %)) #(< % 100.0))
    #(gen/fmap
       (fn [base] (+ 1.0 (* base 0.3)))  ; prices between 1.0 and 1.3
       (gen/double* {:min 0.0 :max 1.0 :NaN? false :infinite? false}))))

Spec at the System Boundary

The most valuable use of spec is at the boundary between external and internal systems:

1
2
3
4
5
6
7
8
9

(defn parse-and-validate-fix-message [raw-message]
  (let [parsed (parse-fix raw-message)]
    (when-not (s/valid? ::fix-message parsed)
      (let [explanation (s/explain-str ::fix-message parsed)]
        (log/error "Invalid FIX message" {:raw raw-message
                                           :parsed parsed
                                           :error explanation})
        (throw (ex-info "Invalid FIX message" {:spec-error explanation}))))
    parsed))

Every message that enters the system has been validated against the spec before any processing occurs. Invalid messages are rejected with useful error information; they never reach the core logic.

This is the shift spec enables: instead of defensive programming scattered throughout the codebase (if (price < 0) throw...), you define correctness once in the spec and enforce it at the boundary. The interior code can assume it’s receiving valid data.

Spec for Function Contracts

s/fdef defines specs for function arguments and return values:

1
2
3
4
5
6
7
8
9

(s/fdef compute-pnl
  :args (s/cat :position ::position
               :market-price ::price)
  :ret  double?
  :fn   #(not (Double/isNaN (:ret %))))  ; invariant on return value

(defn compute-pnl [position market-price]
  (* (:quantity position)
     (- market-price (:avg-price position))))

With stest/instrument, calls to compute-pnl are automatically validated against the spec at runtime — useful in development and testing:

1
2
3
4
5
6

(stest/instrument `compute-pnl)

;; Now this throws a spec error immediately:
(compute-pnl nil 1.28445)
;; => ExceptionInfo Call to compute-pnl did not conform to spec:
;;    In: [0] val: nil fails spec: ::position

In production, instrumentation is typically disabled (it adds overhead). You rely on boundary validation having caught invalid data before it reaches instrumented functions.

The Bugs Spec Found

The generative tests found seven issues in our FIX parser within the first overnight run:

  1. Negative zero price (-0.0) — now caught by pos? predicate
  2. Symbol with lowercase (EUR/usd) — parser didn’t normalise case consistently
  3. Timestamp in Unix epoch 0 — a missing timestamp field defaulted to 1970-01-01
  4. Quantity exactly at Int.MAX_VALUE — a conversion bug from long to int
  5. Empty string venue — the venue field was optional but we used it without null-checking
  6. Price exactly 0.0 — not a FIX parsing bug but exposed a division-by-zero in a ratio calculation
  7. Side field ‘B’ instead of ‘BUY’ — an older FIX version used a different format we hadn’t handled

Six of these seven bugs existed in code that had been in production for 18 months. They were triggered by input combinations that never occurred naturally but are entirely plausible in production (especially #7 — a vendor sends older FIX format during testing).

Spec’s generative testing is not a replacement for domain knowledge and hand-written edge cases. It’s an amplifier — it exercises the boundaries of your spec’s solution space automatically and finds cases you didn’t think to test for. For financial data processing, that amplification has real value.