# An acronym capitalization checker with regex
**URL:** https://clojureverse.org/t/an-acronym-capitalization-checker-with-regex/8013
**Category:** Watercooler
**Created:** [August 8, 2021, 1:33pm UTC](https://clojureverse.org/t/an-acronym-capitalization-checker-with-regex/8013 "2021-08-08T13:33:29Z")
**Posts on this page:** 10
**Page:** 1
### Author:  [@magnus0re](https://clojureverse.org/u/magnus0re)
#### Post date: [August 8, 2021, 1:33pm UTC](https://clojureverse.org/t/an-acronym-capitalization-checker-with-regex/8013/1 "2021-08-08T13:33:29Z")
I’m currently writing a major technical work. And in the field there are lots of acronyms. I mean lots. The work is spread out among many .lyx files, which is a markup that transpiles to latex, but is still text based.
So I wrote some small code to fix common mistakes, like my own personalized spell checker, but with Clojure and regex.
the basic clause is a dict with keys `:regex` that matches the error, and `:problem` which describes the issue more clearly.
```auto
(def illegals
(concat
[{:regex #"(?i)hardon" ; seriously there are multiple articles on arXiv with this problem https://arxiv.org/search/?query=large+hardon+collider&searchtype=all&source=header
:problem "should be hadron"}
{:regex #"\s\s"
:problem "double space"}]
(capitalization-rule "CERN" "FoCal" "SystemC" "ASIC" "FPGA" "CMS" "ATLAS" "LHCb")))
```
Now, the interesting part here, for me at least, was the regex. I wanted a regex that searches for every ocurrence of the acronym and matches only if it has typos. This required negative lookbehind.
The basic expression is [`#"((?i)propercase(?-i)(?
### Author:  [@camdez](https://clojureverse.org/u/camdez)
#### Post date: [August 8, 2021, 4:36pm UTC](https://clojureverse.org/t/an-acronym-capitalization-checker-with-regex/8013/2 "2021-08-08T16:36:15Z")
Very cool! Nice and clean!
This is rather pedantic, but some people enjoy learning these things…technically these are all _initialisms_, not acronyms. Acronyms are only the ones we pronounce like words (e.g. “NASA”, “LASER”, but not “USA” or “FBI”). #funfact
I don’t know how much text you’re checking / if speed is a concern, but you could massively improve the speed through some small tweaks. Right now you’re repeatedly scanning all of the text for each pattern, but you could do one overall pass of the text for _potentially_ problematic words—the case-insensitive union of all of your patterns—and then scan that (much smaller) set for actual problems.
It shouldn’t be that hard to implement. You could start by mashing all of your patterns into a big regex alternation (something like `(re-pattern (str "(" (str/join "|" options) ")"))`). And, if you wanted to, you can take that a step further by optimizing it by unifying the common prefixes of the regexes. Could be a fun bit of code to write!
Emacs has a built-in function which does this that could show you the way:
```auto
(regexp-opt '("APPLE" "ABC" "ABOUT"))
;=> "\\(?:A\\(?:B\\(?:C\\|OUT\\)\\|PPLE\\)\\)"
(regexp-opt '("CERN" "FoCal" "SystemC" "ASIC" "FPGA" "CMS" "ATLAS" "LHCb"))
;=> "\\(?:A\\(?:SIC\\|TLAS\\)\\|C\\(?:ERN\\|MS\\)\\|F\\(?:PGA\\|oCal\\)\\|LHCb\\|SystemC\\)"
```
(FWIW, I’d unify the case of these string literals before building the optimized regex in this case.)
Anyway, just spitballing. Nice work!
---
### Author:  [@camdez](https://clojureverse.org/u/camdez)
#### Post date: [August 8, 2021, 4:44pm UTC](https://clojureverse.org/t/an-acronym-capitalization-checker-with-regex/8013/3 "2021-08-08T16:44:57Z")
Also, from a style perspective, I would probably skip the higher-arity version of `capitalization-rule` and just do the `map`ping inside of `illegals`. This keeps the focus of `capitalization-rule` on what it does best, and lets `map` do what it does best. Simplify + reuse general tools across various and sundry data—The Clojure Way™.
---
### Author:  [@camdez](https://clojureverse.org/u/camdez)
#### Post date: [August 8, 2021, 6:45pm UTC](https://clojureverse.org/t/an-acronym-capitalization-checker-with-regex/8013/4 "2021-08-08T18:45:24Z")
> [@camdez](#):
>
> if you wanted to, you can take that a step further by optimizing it by unifying the common prefixes of the regexes. Could be a fun bit of code to write!
I decided to take a stab an implementing this since it seemed like a fun little challenge:
```clj
(ns camdez.re-opt
(:require [clojure.walk :as walk]
[clojure.string :as str]))
;; Convert strings to a graph (nested maps) representing common prefix
;; strings:
;;
;; A -> B -> C
;; -> O -> U -> T
;; -> P -> P -> L -> E
;;
;; Then collapse tails into strings and branches into regex
;; alternations.
(defn- re-literal-opts-str [opts]
(->> opts
(reduce (fn [acc s]
(assoc-in acc (butlast s) {(last s) nil}))
{})
(walk/postwalk (fn [x]
(cond
(not (map? x)) x
(= 1 (count x)) (apply str (first x))
:else (str "(?:" (str/join "|" (map (fn [[k v]] (str k v)) x)) ")"))))))
(defn re-literal-opts
"Builds an optimized regular expression for matching any of the string
literals in `opts`."
[opts]
(re-pattern (re-literal-opts-str opts)))
(defn re-literal-word-opts
"Builds an optimized regular expression for matching any of the string
literals in `opts` at word boundaries."
[opts]
(re-pattern (str "\\b(?:" (re-literal-opts-str opts) ")\\b")))
```
```clj
;;; Examples
(def sample-opts ["ABC" "ABOUT" "APPLE" "BOTTLE"])
(re-literal-opts-str sample-opts)
;; => "(?:A(?:B(?:C|OUT)|PPLE)|BOTTLE)"
(def p1 (re-literal-opts sample-opts))
(re-seq p1 "WHAT ABOUT BOTTLES? A BOTTLE?")
;; => ("ABOUT" "BOTTLE" "BOTTLE")
(def p2 (re-literal-word-opts sample-opts))
(re-seq p2 "WHAT ABOUT BOTTLES? A BOTTLE?")
;; => ("ABOUT" "BOTTLE")
```
Pretty happy with how it turned out. Only about 20 minutes of work given the magic of Clojure. 🧙
---
### Author:  [@magnus0re](https://clojureverse.org/u/magnus0re)
#### Post date: [August 8, 2021, 8:39pm UTC](https://clojureverse.org/t/an-acronym-capitalization-checker-with-regex/8013/5 "2021-08-08T20:39:13Z")
Very cool. I’m stealing this code 😉
Looks like a 5x improvement in speed for the processing step! Of course this depends on input source etc…
I’m having performance issues in reading the files though ☹ so it’s minimal impact on the overall processing time.
---
### Author:  [@camdez](https://clojureverse.org/u/camdez)
#### Post date: [August 8, 2021, 10:23pm UTC](https://clojureverse.org/t/an-acronym-capitalization-checker-with-regex/8013/6 "2021-08-08T22:23:57Z")
I’m no performance guru, but if you’re using `slurp` or similar to read the whole file into memory, definitely look into streaming the input (`line-seq` is an easy approach if you don’t have patterns that span lines).
---
### Author:  [@camdez](https://clojureverse.org/u/camdez)
#### Post date: [August 15, 2021, 2:41am UTC](https://clojureverse.org/t/an-acronym-capitalization-checker-with-regex/8013/7 "2021-08-15T02:41:57Z")
@magnus0re Just noticed the above implementation of `re-literal-opts-str` was slightly busted…it was order dependent and would break on prefixes (“AB” could replace “ABC” entirely). Suggest replacing `(assoc-in acc (butlast s) {(last s) nil})` with `(assoc-in acc (conj (vec s) nil) nil))`. Full implementation here (as `re-opt`):
> **[Regex Optimization in Clojure](https://camdez.com/blog/2021/08/14/regex-optimization-in-clojure/)**
>
> Last week, after responding to a post on ClojureVerse, I
> got curious about reimplementing the basics of Emacs’s regexp-opt
> function in Clojure. I thought it would be a fun little coding
> exercise so I decided to take a stab at it during a few spare...
---
### Author:  [@magnus0re](https://clojureverse.org/u/magnus0re)
#### Post date: [August 15, 2021, 6:58pm UTC](https://clojureverse.org/t/an-acronym-capitalization-checker-with-regex/8013/8 "2021-08-15T18:58:04Z")
Thanks for the bugfix!
Found the performance issue. It was in finding the files. I’m running Clojure under WSL2, and the folder searched with `file-seq` was under windows. For all practical purposes this means that the directory was networked on localhost.
Under the hood `file-seq` recursively calls `isDirectory` and `listFiles`, so in a large folder, with many folders it super slow in my particular configuration.
I swapped to [glob](https://clojars.org/clj-glob) and instantly got more than 40x performance.
---
### Author:  [@camdez](https://clojureverse.org/u/camdez)
#### Post date: [August 17, 2021, 3:05pm UTC](https://clojureverse.org/t/an-acronym-capitalization-checker-with-regex/8013/9 "2021-08-17T15:05:15Z")
Woah! Nice work! I would not have expected `file-seq` to be the cause! TIL.
---
### Author:  [@system](https://clojureverse.org/u/system)
#### Post date: [February 16, 2022, 3:05am UTC](https://clojureverse.org/t/an-acronym-capitalization-checker-with-regex/8013/10 "2022-02-16T03:05:42Z")
This topic was automatically closed 182 days after the last reply. New replies are no longer allowed.