https://cljdoc.org/badge/mx.cider/haystack

Haystack
Note: much of the functionality in Haystack is now available as part of
Orchard, in particular
orchard.stacktrace.
We recommend trying Orchard first, as it’s more actively maintained. If you need
to parse stacktraces from strings, then Haystack is still the way to go.
Haystack vs orchard.stacktrace
The orchard.stacktrace namespace is a repatriation of Haystack’s
analyzer back into Orchard (see orchard#322). When deciding between the
two, keep these differences in mind:
- Analyzer: Both provide an
analyze function that turns a
Throwable (or a map in the Throwable->map format) into an
enriched cause chain. Orchard’s version was modernized and
optimized; most notably its project-frame detection runs in under a
millisecond, where Haystack’s namespace scan can take ~100ms on
large projects.
- Parser: This is the key distinction. Haystack also ships a parser
that reconstructs stacktrace data from a printed stacktrace (REPL
output, a log file, an exception buried in another string).
orchard.stacktrace has no equivalent and only works with live
exceptions or their Throwable->map data.
In short: reach for orchard.stacktrace when you have the exception
object at hand, and for Haystack when all you have is its printed text.
Introduction
Stacktraces are a hot topic in the Clojure community. As a Clojurist
you deal with them in different situations. Sometimes you catch them
“live”, like an exception just thrown in a REPL. Other times you find
them as text, printed in a REPL, or in a log file. Or worst, a printed
exception buried inside another string, almost impossible to read. And
of course, there are different kinds of formats.
Haystack is a library that can parse and analyze Clojure
stacktraces. The parser transforms printed stacktraces back into data
and the analyzer enriches stacktrace data with run-time information
from the class path.
Haystack was previously used in CIDER for
stacktrace analysis. It is not included in CIDER anymore but can still
be used as an individual library.
Parser
The Haystack stacktrace parser transforms a string that contains a
stacktrace printed in one of the supported formats back into a Clojure
data structure. Given an input, the parser applies some
transformations to it (unwrapping an EDN string for example) and
passes the result to the parser functions registered in the
haystack.parser/default-parsers var. Each of the registered parsers
is tried in order and the first parser that succeeds wins.
On success the parser returns a Clojure map with a similar structure
as Clojure’s Throwable->map function.
On failure the parser returns a map with an :error key, and possibly
other keys describing the error.
A successful parse result can be given to the Haystack analyzer to
enrich it with more information.
Stacktrace data format
An Haystack stacktrace parser transforms input into a parse result. On
success, the parse result is an enhanced version of the Clojure data
representation of a Throwable, a map with the following keys:
:cause The root cause message as a string.
:phase The error phase (optional).
:via The cause chain, with each cause having the keys:
:at The top stack element of the cause as a vector (optional).
:data The ex-data of the cause as a map (optional).
:message The exception message of the cause as a string.
:type The exception of the cause as a symbol.
:trace The stack elements (optional, extended by Haystack).
:trace The root cause stack elements
This is mostly the same format as used by Throwable->map in newer
Clojure versions, except for the additional :trace key in the cause
maps of :via. We added this additional key to keep the trace of the
causes.
Supported formats
Stacktraces are printed in different formats by tools and
libraries. Haystack supports the following formats:
Usage
Let’s say you want to parse the following stacktrace string and turn
it back into a data structure for further processing.
(def my-stacktrace-str
(str "clojure.lang.ExceptionInfo: BOOM-1 {:boom \"1\"}\n"
" at java.base/java.lang.Thread.run(Thread.java:829)"))
The easiest way to do this is to pass the string to the
haystack.parser/parse function. It will try all registered
parsers and returns the first successful parse result.
(require '[haystack.parser :as stacktrace.parser])
(def my-stacktrace-data
(stacktrace.parser/parse my-stacktrace-str))
On success the parser will return a Clojure map in the
Throwable->map format. For the input used above, this data structure
looks like this:
(clojure.pprint/pprint my-stacktrace-data)
{:cause "BOOM-1",
:data {:boom "1"},
:trace [[java.base/java.lang.Thread run "Thread.java" 829]],
:via
[{:at [java.base/java.lang.Thread run "Thread.java" 829],
:message "BOOM-1",
:type clojure.lang.ExceptionInfo,
:trace [[java.base/java.lang.Thread run "Thread.java" 829]],
:data {:boom "1"}}],
:stacktrace-type :java}
Tip: If you know in advance with what kind of stacktrace you are
dealing with, pass it directly to the parser for the given format.
Analyzer
The Haystack stacktrace analyzer transforms a stacktrace into an
analysis. An analysis is a sequence of Clojure maps, one for each of
the causes of the stacktrace, with the following keys:
:class The exception class as a string.
:message The exception message as a string.
:stacktrace The stacktrace frames, a list of maps.
:data The exception data.
:location The location formation of the exception.
A frame in the :stacktrace is a map with the following keys:
:class The class name of the frame invocation.
:file-url The URL of the frame source file.
:file The file name of the frame source.
:flags The flags of the frame.
:line The line number of the frame source.
:method The method or function name of the frame invocation.
:name The name of the frame, typically the class and method of the invocation.
:type The type of invocation (:java, :tooling, etc).
The analyzer accepts either an instance of java.lang.Throwable or a
Clojure map in the Throwable->map format as input.
Usage
We can analyze our previously parsed stacktrace by calling the
haystack.analyzer/analyze function on it.
(require '[haystack.analyzer :as stacktrace.analyzer])
(stacktrace.analyzer/analyze my-stacktrace-data)
[{:class "clojure.lang.ExceptionInfo",
:message "BOOM-1",
:stacktrace
({:name "java.lang.Thread/run",
:file "Thread.java",
:line 829,
:class "java.lang.Thread",
:method "run",
:type :java,
:flags #{:java},
:file-url
"jar:file:/usr/lib/jvm/openjdk-11/lib/src.zip!/java.base/java/lang/Thread.java"}),
:data "{:boom \"1\"}",
:location {}}]
We get back a sequence of maps, one for each cause, which contain
additional information about each frame discovered from the class path.
Development
Creating a parser
To add support for another stacktrace format, please create a new
parser under the haystack.parser.<NEW-FORMAT> namespace and add it
to the haystack.parser/default-parsers var. The parser should be a
function that accepts a single argument, the input (typically a
string), and returns a map. The parser function should follow the
following rules:
- On success, the parser should return the stacktrace as a map. The
map should be in the
Throwable->map format described above with a
:stacktrace-type key that contains the type of stacktrace as a
keyword.
- On error, the parser should return a map with an
:error key and
possibly others describing why the input could not be parsed. We use
:incorrect if the input does not match the grammar, and
:unsupported if the input type is not supported by the parser.
- Ideally, the parser should be tolerant to any garbage before and
after the stacktrace to be parsed. This is to not put the burden of
exactly figuring out where a stacktrace starts and ends onto
clients.
- When skipping garbage at the beginning of a stacktrace do it
efficiently. For example, instead of skipping garbage character by
character and trying your parser with the rest of the string, use
the
haystack.parser.util/seek-to-regex function to
directly skip to the beginning of the stacktrace, if possible.
- Most of the parsers in Haystack are implemented with Instaparse and
have a BNF grammar describing the format of the stacktrace. Try to
come up with an Instaparse grammar for the new stacktrace format as
well, unless you have a better, simpler or more efficient way of
parsing it (like the Clojure tagged literal parser for example).
Instaparse Tips & Tricks
Writing a grammar for a stacktrace format might be challenging at
times, especially when garbage in the input is involved, which might
introduce ambiguities in your grammar. Here are some tips and trick
for writing Instaparse grammars:
- Read the documentation, it is good and has many examples.
- Start with the most simple parser, try to pass the exception class
or name before building up.
- Use the
:start parameter of the Instaparse parser, to parse input
from another start rule. This is useful if your grammar got complex,
but you want to try parsing of an individual rule.
- Be aware of greedy regex behavior.
- When testing input try it against the raw Instaparse parser first,
and only apply the Instaparse transformations when the parser works.
- If your parser fails on an input, reveal hidden information to get a
better understanding of what happened.
Deployment
Here’s how to deploy to Clojars:
git tag -a v0.3.3 -m "0.3.3"
git push --tags
Changelog
CHANGELOG.md
Thanks
The Haystack stacktrace analyzer was written by Jeff Valk (@jeffvalk)
and was originally part of the cider-nrepl project.
License
Copyright © 2022-23 Cider Contributors
Distributed under the Eclipse Public License, the same as Clojure.