Im still busy with the brave book and near the end
but I do not see how I could ever do this :
(let [data (future (slurp (str "https://" search-engine "/search?q=" search-term)))]
@data))
(search-clojure-docs-with-search-engine "slurp" "clojuredocs.org")
; Create a new function that takes a search term and search engines as arguments,
; and returns a vector of the URLs from the first page of search results from
; each search engine
I could re-use the `search-clojure-docs-with-searchengine’ but never in the book is explained how I can get the urls our of the html.
fyi, the naive url-based search will work with bing, not google these days. google throws a 403 error; it wants a more properly formatted request than the simple string url being passed (some stuff in the headers, ends up being a bit more complicated). I think back when CFBAT was written this wasn’t an issue.
I think this is kind of a substantial exercise, since you are now left with the problem of getting results, and then parsing those results (which are encoded in HTML) into URLs. So you have to understand the format of the search engine’s output to parse the HTML (pedagogically useful but perhaps a bit intense for a beginner exercise). That or use an API to get better formatted results (no idea). There are libraries that make parsing and dissecting and searching the resulting html (or xhtml) really easy though, so it’s not terrible. It seems like the book dangles this out as something trivial (just looking at CH9) when in fact there is “a lot” of stuff left up to the reader.
The simplest way would be to search the HTML for urls using a regular expression. If you sift through the HTML, e.g. with the firefox or chrome inspector, you will get a sense for where the search results are and where the URL is stored. It looks like an href, so something along the lines of scraping the text looking for matches for a regex that conforms to this pattern could work. You could also see if there’s a common regex for URLs (or define your own) and just use that and hope that the only URLs in the search results will be ones the engine presents for clicking.
Look at the section under “Strings” for “Regex” at the cheatsheet for some built-in functions.
I still recommend it (or at least portions) due to most of the content being decent (aside from the extra sugary examples they use…trying to bend over backwards to be approachable with IMO goofy names and data ends up being distracting IMO, but the technical discussions are useful).
As I recall, the entire IDE/editor setup is focused on emacs and some stuff that is likely dated (e.g. not using cider, not mentioning spacemacs and its excellent clojure layer or other stuff - some of which the book predated). When I get folks started with a more-or-less minimal setup, I just help them install spacemacs and the clojure layer and that covers about 99% of everything. OTOH, I don’t know that I’d start off a complete new person with the dual task of learning clojure and emacs, with the exception of a very minimal familiar text-editor-like experience (e.g. I encourage folks to use CUA shortcuts if they’ve come from that and just ditch emacs’s defaults).
I think that book could use a 2nd edition by now covering current practices for tooling (e.g. discussing tools.deps, although I still prefer lein for projects) and revising some of the exercises or providing worked examples.
Agreed 100%. It’s what I used to get started. I was also using spacemacs at the time (nowadays I’m using Doom, I got tired of spacemacs’ general sluggishness). The Doom clojure setup is almost as good (I had to define some extra bindings for some CIDER stuff).
My impression wrt the exercises is that they’re meant to have “sketched” solutions. The URL example here would probably be served by figuring out how to use re-matches or similar, rather than something like hickory or enlive and walking over the parsed HTML, which is what a proper solution would entail.
Well, as we mentioned, the issue is that those exercises in the brave clojure book are kinda complex if you want to do them “right”. If you’re just starting out, enlive is not easy to use (heck, I’ve been doing clojure for a few years, and enlive is still not easy to use for me).
I’d suggest you look into using regexes, or maybe skip this one, tbh.
Another option is to use hickory and parse the query response then search that recursively. Hickory will turn the response into a nested collection of maps with keywords correspond to tags and attributes and content.
Or just skip it if you’re not interested with the problem
having the caret inside the square brackets means to match any characters but those. So we skip tags and quotes, basically (that’s a regex thing, nothing Clojure-specific).
