the above url has list of xml but i just want the latest entry on daily basis.
Any help would be appreciated. As i am a beginner so not sure how to achieve this.
Multiple different answers here: Getting content from parsed XML
You can get the content of the URL above by just calling (slurp the-url) - it’ll return a string.
(defn get-xml [year month day]
(let [today (format “%d-%02d-%02dT00:00:00” year month day)]
(-> (str base-url “xml?data=daily_treasury_yield_curve&field_tdr_date_value_month=%d%02d”)
(format year month)
(client/get {:as :stream})
:body
parse
zip/xml-zip)))
I already have this method which returns the all entries
But i want the single latest record. Say if i call the method i should get only below entry like this, not all the entries.
<entry>
<title type="text"/>
<updated>2022-11-14T16:10:51Z</updated>
<author>
<name/>
</author>
<category term="TreasuryDataWarehouseModel.DailyTreasuryYieldCurveRateDatum" scheme="http://schemas.microsoft.com/ado/2007/08/dataservices/scheme"/>
<content type="application/xml">
<m:properties>
<d:NEW_DATE m:type="Edm.DateTime">2022-11-14T00:00:00</d:NEW_DATE>
<d:BC_1MONTH m:type="Edm.Double">3.72</d:BC_1MONTH>
<d:BC_2MONTH m:type="Edm.Double">4.05</d:BC_2MONTH>
<d:BC_3MONTH m:type="Edm.Double">4.34</d:BC_3MONTH>
<d:BC_4MONTH m:type="Edm.Double">4.38</d:BC_4MONTH>
<d:BC_6MONTH m:type="Edm.Double">4.55</d:BC_6MONTH>
<d:BC_1YEAR m:type="Edm.Double">4.63</d:BC_1YEAR>
<d:BC_2YEAR m:type="Edm.Double">4.40</d:BC_2YEAR>
<d:BC_3YEAR m:type="Edm.Double">4.24</d:BC_3YEAR>
<d:BC_5YEAR m:type="Edm.Double">4.00</d:BC_5YEAR>
<d:BC_7YEAR m:type="Edm.Double">3.95</d:BC_7YEAR>
<d:BC_10YEAR m:type="Edm.Double">3.88</d:BC_10YEAR>
<d:BC_20YEAR m:type="Edm.Double">4.28</d:BC_20YEAR>
<d:BC_30YEAR m:type="Edm.Double">4.07</d:BC_30YEAR>
<d:BC_30YEARDISPLAY m:type="Edm.Double">4.07</d:BC_30YEARDISPLAY>
</m:properties>
</content>
</entry>
You’re already using xml-zip, so you can use zippers to navigate your XML data and extract the right entity. Have you tried going though the examples here and adapting them to your use-case? xml-zip - clojure.zip | ClojureDocs - Community-Powered Clojure Documentation and Examples
Yes, but not able to achieve the desired result.
Can u try if u r able to get single entry?
It’s not entirely clear to me what you’re asking for.
Perhaps you can provide example input and example output of a function that would do what you want?
I want to hit this URL -
https://home.treasury.gov/resource-center/data-chart-center/interest-rates/pages/xml?data=daily_treasury_yield_curve&field_tdr_date_value_month=202211
And want to get as the output only single entry on a daily basis as u can see we have a field called
d:NEW_DATE. So daily we will call a method which will hit this URL DailyTreasuryYieldCurveRateData
will as output will give us a xml file containing only single entry for that day.
<entry>
<title type="text"/>
<updated>2022-11-14T16:10:51Z</updated>
<author>
<name/>
</author>
<category term="TreasuryDataWarehouseModel.DailyTreasuryYieldCurveRateDatum" scheme="http://schemas.microsoft.com/ado/2007/08/dataservices/scheme"/>
<content type="application/xml">
<m:properties>
<d:NEW_DATE m:type="Edm.DateTime">2022-11-14T00:00:00</d:NEW_DATE>
<d:BC_1MONTH m:type="Edm.Double">3.72</d:BC_1MONTH>
<d:BC_2MONTH m:type="Edm.Double">4.05</d:BC_2MONTH>
<d:BC_3MONTH m:type="Edm.Double">4.34</d:BC_3MONTH>
<d:BC_4MONTH m:type="Edm.Double">4.38</d:BC_4MONTH>
<d:BC_6MONTH m:type="Edm.Double">4.55</d:BC_6MONTH>
<d:BC_1YEAR m:type="Edm.Double">4.63</d:BC_1YEAR>
<d:BC_2YEAR m:type="Edm.Double">4.40</d:BC_2YEAR>
<d:BC_3YEAR m:type="Edm.Double">4.24</d:BC_3YEAR>
<d:BC_5YEAR m:type="Edm.Double">4.00</d:BC_5YEAR>
<d:BC_7YEAR m:type="Edm.Double">3.95</d:BC_7YEAR>
<d:BC_10YEAR m:type="Edm.Double">3.88</d:BC_10YEAR>
<d:BC_20YEAR m:type="Edm.Double">4.28</d:BC_20YEAR>
<d:BC_30YEAR m:type="Edm.Double">4.07</d:BC_30YEAR>
<d:BC_30YEARDISPLAY m:type="Edm.Double">4.07</d:BC_30YEARDISPLAY>
</m:properties>
</content>
</entry>
Can you formulate that in terms of a specific Clojure function call?
Example:
(defn would-solve-my-problem [,,,])
(would-solve-my-problem "https://home.treasury.gov/resource-center/data-chart-center/interest-rates/pages/xml?data=daily_treasury_yield_curve&field_tdr_date_value_month=202211")
;; =>
"<author></author>"
I’m still struggling to understand precisely what you’re asking for. It seems like you’re asking many questions at the same time.
Also - if you prefer chat to long form discussion, the Clojurians Slack might be a good fit for you 
(defn would-solve-my-problem [year month day])
(would-solve-my-problem “https://home.treasury.gov/resource-center/data-chart-center/interest-rates/pages/xml?data=daily_treasury_yield_curve&field_tdr_date_value_month=” year month)
;; => below one should be the output as the latest available record on the mentioned url is this.
<entry>
<title type="text"/>
<updated>2022-11-14T16:10:51Z</updated>
<author>
<name/>
</author>
<category term="TreasuryDataWarehouseModel.DailyTreasuryYieldCurveRateDatum" scheme="http://schemas.microsoft.com/ado/2007/08/dataservices/scheme"/>
<content type="application/xml">
<m:properties>
<d:NEW_DATE m:type="Edm.DateTime">2022-11-14T00:00:00</d:NEW_DATE>
<d:BC_1MONTH m:type="Edm.Double">3.72</d:BC_1MONTH>
<d:BC_2MONTH m:type="Edm.Double">4.05</d:BC_2MONTH>
<d:BC_3MONTH m:type="Edm.Double">4.34</d:BC_3MONTH>
<d:BC_4MONTH m:type="Edm.Double">4.38</d:BC_4MONTH>
<d:BC_6MONTH m:type="Edm.Double">4.55</d:BC_6MONTH>
<d:BC_1YEAR m:type="Edm.Double">4.63</d:BC_1YEAR>
<d:BC_2YEAR m:type="Edm.Double">4.40</d:BC_2YEAR>
<d:BC_3YEAR m:type="Edm.Double">4.24</d:BC_3YEAR>
<d:BC_5YEAR m:type="Edm.Double">4.00</d:BC_5YEAR>
<d:BC_7YEAR m:type="Edm.Double">3.95</d:BC_7YEAR>
<d:BC_10YEAR m:type="Edm.Double">3.88</d:BC_10YEAR>
<d:BC_20YEAR m:type="Edm.Double">4.28</d:BC_20YEAR>
<d:BC_30YEAR m:type="Edm.Double">4.07</d:BC_30YEAR>
<d:BC_30YEARDISPLAY m:type="Edm.Double">4.07</d:BC_30YEARDISPLAY>
</m:properties>
</content>
</entry>
I think I’d try clojure.core/xml-seq here rather than zippers. (Because I’m frankly not sure about how to solve this with zippers)
https://clojuredocs.org/clojure.core/xml-seq
Something like this:
(defn entry? [entry]
,,,)
(defn get-xml [year month day]
(let [today (format “%d-%02d-%02dT00:00:00” year month day)
body-string (-> (str base-url “xml?data=daily_treasury_yield_curve&field_tdr_date_value_month=%d%02d”)
(format year month)
(client/get {:as :stream})
:body
parse)]
(->> body-string
xml-seq
(filter entry?)
last)))
Edit: I’ve got to run now. Good luck!
Execution error (ExceptionInfo) at slingshot.support/stack-trace (support.clj:201).
clj-http: status 404
This looks like an atom feed? So if you want a daily update, look into interop with a Java atom feed library and/or Clojure library wrapping that.
https://www.xml.com/pub/a/2006/02/22/rome-parse-publish-rss-atom-feeds-java.html
This is a little manual, but it appears to work. I think you could do much better with declarative destructuring and pattern matching via a library like meander, or specter, or even using spec and its conform stuff (probably helps to document the “structure” of the xml as well) to project the xml structure onto some useful data without doing it quite so manually (and potentially brittle if the schema changes). There may be some xpath-like libraries the embed the querying parameters into clojure (specter kind of does this generally), but I am not an xml expert (this is probably the 5th time in my life I’ve had to deal with it). I didn’t check if the date parsing is perfect, I just guessed it could be jammed into a date and sorted. Anyhow, I think this is one approach:
(def url "https://home.treasury.gov/resource-center/data-chart-center/interest-rates/pages/xml?data=daily_treasury_yield_curve&field_tdr_date_value_month=202211")
(require '[clojure.data.xml :as xml])
(use 'clojure.pprint)
(defn tag? [k]
(fn [e] (when (= (:tag e) k) e)))
(defn entries [xs]
(->> xs :content (filter (tag? :entry))))
(defn entry->date [e]
(some->> e
:content
(some (tag? :content))
:content
first
:content
(some (tag? :NEW_DATE))
:content
first
clojure.instant/read-instant-date))
(defn ordered-entries [root]
(->> (for [e (entries root)]
[(entry->date e) e])
(sort-by first)))
(defn latest-entry [root]
(->> root ordered-entries last))
user=>(def xs
(->> url
slurp
xml/parse-str))
user=> (pprint (latest-entry xs))
[#inst "2022-11-14T00:00:00.000-00:00"
{:tag :entry,
:attrs {},
:content
({:tag :title, :attrs {:type "text"}, :content ()}
{:tag :updated, :attrs {}, :content ("2022-11-14T16:10:51Z")}
{:tag :author,
:attrs {},
:content ({:tag :name, :attrs {}, :content ()})}
{:tag :category,
:attrs
{:term
"TreasuryDataWarehouseModel.DailyTreasuryYieldCurveRateDatum",
:scheme
"http://schemas.microsoft.com/ado/2007/08/dataservices/scheme"},
:content ()}
{:tag :content,
:attrs {:type "application/xml"},
:content
({:tag :properties,
:attrs {},
:content
({:tag :NEW_DATE,
:attrs #:m{:type "Edm.DateTime"},
:content ("2022-11-14T00:00:00")}
{:tag :BC_1MONTH,
:attrs #:m{:type "Edm.Double"},
:content ("3.72")}
{:tag :BC_2MONTH,
:attrs #:m{:type "Edm.Double"},
:content ("4.05")}
{:tag :BC_3MONTH,
:attrs #:m{:type "Edm.Double"},
:content ("4.34")}
{:tag :BC_4MONTH,
:attrs #:m{:type "Edm.Double"},
:content ("4.38")}
{:tag :BC_6MONTH,
:attrs #:m{:type "Edm.Double"},
:content ("4.55")}
{:tag :BC_1YEAR,
:attrs #:m{:type "Edm.Double"},
:content ("4.63")}
{:tag :BC_2YEAR,
:attrs #:m{:type "Edm.Double"},
:content ("4.40")}
{:tag :BC_3YEAR,
:attrs #:m{:type "Edm.Double"},
:content ("4.24")}
{:tag :BC_5YEAR,
:attrs #:m{:type "Edm.Double"},
:content ("4.00")}
{:tag :BC_7YEAR,
:attrs #:m{:type "Edm.Double"},
:content ("3.95")}
{:tag :BC_10YEAR,
:attrs #:m{:type "Edm.Double"},
:content ("3.88")}
{:tag :BC_20YEAR,
:attrs #:m{:type "Edm.Double"},
:content ("4.28")}
{:tag :BC_30YEAR,
:attrs #:m{:type "Edm.Double"},
:content ("4.07")}
{:tag :BC_30YEARDISPLAY,
:attrs #:m{:type "Edm.Double"},
:content ("4.07")})})})}]
nil
3 Likes
Thanks for your efforts really appreciable, actually i am looking for the XML result.
I want this method to return the result as output, right now i think its just returning nil. What change do we need to do to return this result as output?