joncook.github.com/index.html at master · JonCook/joncook.github.com

567 lines (296 loc) · 19.6 KB
<!DOCTYPE html>
<!--[if IEMobile 7 ]><html class="no-js iem7"><![endif]-->
<!--[if lt IE 9]><html class="no-js lte-ie8"><![endif]-->
<!--[if (gt IE 8)|(gt IEMobile 7)|!(IEMobile)|!(IE)]><!--><html class="no-js" lang="en"><!--<![endif]-->
  <meta charset="utf-8">
  <title>Jon's Blog</title>
  <meta name="author" content="Jon Cook">
  <meta name="description" content="Given scala&rsquo;s built-in support for XML and its more concise syntax for dealing with iterables and collections I was interested to see what some &hellip;">
  <!-- http://t.co/dKP3o1e -->
  <meta name="HandheldFriendly" content="True">
  <meta name="MobileOptimized" content="320">
  <meta name="viewport" content="width=device-width, initial-scale=1">
  <link rel="canonical" href="http://JonCook.github.com">
  <link href="/favicon.png" rel="icon">
  <link href="/stylesheets/screen.css" media="screen, projection" rel="stylesheet" type="text/css">
  <link href="/atom.xml" rel="alternate" title="Jon's Blog" type="application/atom+xml">
  <script src="/javascripts/modernizr-2.0.js"></script>
  <script src="//ajax.googleapis.com/ajax/libs/jquery/1.9.1/jquery.min.js"></script>
  <script>!window.jQuery && document.write(unescape('%3Cscript src="./javascripts/lib/jquery.min.js"%3E%3C/script%3E'))</script>
  <script src="/javascripts/octopress.js" type="text/javascript"></script>
  <!--Fonts from Google"s Web font directory at http://google.com/webfonts -->
<link href="http://fonts.googleapis.com/css?family=PT+Serif:regular,italic,bold,bolditalic" rel="stylesheet" type="text/css">
<link href="http://fonts.googleapis.com/css?family=PT+Sans:regular,italic,bold,bolditalic" rel="stylesheet" type="text/css">
  <header role="banner"><hgroup>
  <h1><a href="/">Jon's Blog</a></h1>
    <h2>Coder, Ex BBC. Cycling obsessive and writer for cyclosport.org</h2>
  <nav role="navigation"><ul class="subscription" data-subscription="rss">
  <li><a href="/atom.xml" rel="subscribe-rss" title="subscribe via RSS">RSS</a></li>
<form action="http://google.com/search" method="get">
  <fieldset role="search">
    <input type="hidden" name="q" value="site:JonCook.github.com" />
    <input class="search" type="text" name="q" results="0" placeholder="Search"/>
  </fieldset>
<ul class="main-navigation">
  <li><a href="/">Blog</a></li>
  <li><a href="/blog/archives">Archives</a></li>
  <div id="main">
    <div id="content">
      <div class="blog-index">
    <article>
      <h1 class="entry-title"><a href="/blog/2013/11/03/xml-processing-with-scala/">XML Processing With Scala</a></h1>
      <p class="meta">
<time datetime="2013-11-03T08:33:00+01:00" pubdate data-updated="true">Nov 3<span>rd</span>, 2013</time>
  </header>
  <div class="entry-content"><p>Given scala&rsquo;s built-in support for XML and its more concise syntax for dealing with iterables and collections I was interested to see what some of our java xml parsing code written using Dom4j could look like in the Scala world.</p>
<h3>Background</h3>
<p>A bit of brief background, the idea is we have a number of assets, an asset representing for example a story, index page, picture gallery, page with media. Each Asset has it&rsquo;s own xml representation and is constructed using common snippets such as item-meta, page-options, media etc. See story.xml.</p>
<p>Our requirement, is to take this xml which is our own internal representation which we use to repesent assets in our content store and essentially transform this into our object model and serve this up as json or xml. You may ask why but isolating our internal format means we are free to change this without effecting the external representations of our assets and it also means we can customise the output and the format of the output easily.</p>
<p>Xml processing in java is typically done in sax or dom using a library like dom4J, we opted for the readability and ease of use the Dom over SAX and used something loosely based on the Strategy design pattern where we have a ParserFactory, this allocates the correct type of asset parser based on the type of asset parses the xml and creates our java object model. Xml and Json generation is handled by jaxb and jackson.</p>
    <footer>
      <a rel="full-article" href="/blog/2013/11/03/xml-processing-with-scala/">Read on &rarr;</a>
    </footer>
    </article>
    <article>
      <h1 class="entry-title"><a href="/blog/2012/04/30/some-bash-fu-with-curl/">Some Bash Fu With Curl, Grep, Awk and Pipes</a></h1>
      <p class="meta">
<time datetime="2012-04-30T01:15:00+02:00" pubdate data-updated="true">Apr 30<span>th</span>, 2012</time>
  </header>
  <div class="entry-content"><p>I wanted to test out the HTTP response codes from a number of our API endpoints. There is numerous ways to approach this but I thought it would be fun to use the command line and it turned out pretty straight forward.</p>
<p>First off I created a file with some urls in e.g.</p>
<figure class='code'><div class="highlight"><table><tr><td class="gutter"><pre class="line-numbers"><span class='line-number'>1</span>
<span class='line-number'>2</span>
<span class='line-number'>3</span>
</pre></td><td class='code'><pre><code class=''><span class='line'>http://www.google.co.uk
</span><span class='line'>http://www.bbc.co.uk
</span><span class='line'>http://www.arsenal.com</span></code></pre></td></tr></table></div></figure>
<p>And ran the following command:</p>
<figure class='code'><div class="highlight"><table><tr><td class="gutter"><pre class="line-numbers"><span class='line-number'>1</span>
</pre></td><td class='code'><pre><code class=''><span class='line'>[root@pal ~]# grep "$1" urls.txt | awk '{print "curl --write-out "$0"=http-%{http_code}\"\n\" --silent --output /dev/null "$0'} | sh &gt;&gt; responses.txt</span></code></pre></td></tr></table></div></figure>
<p>This takes each line (url) and passes the argument to curl before outputting the url and http response code for said url into a file called responses.txt</p>
<figure class='code'><div class="highlight"><table><tr><td class="gutter"><pre class="line-numbers"><span class='line-number'>1</span>
<span class='line-number'>2</span>
<span class='line-number'>3</span>
</pre></td><td class='code'><pre><code class=''><span class='line'>http://www.google.co.uk=http-200
</span><span class='line'>http://www.bbc.co.uk=http-200
</span><span class='line'>http://www.arsenal.com=http-302</span></code></pre></td></tr></table></div></figure>
<p>You can then grep to count the number of different http response codes e.g.</p>
<figure class='code'><div class="highlight"><table><tr><td class="gutter"><pre class="line-numbers"><span class='line-number'>1</span>
<span class='line-number'>2</span>
</pre></td><td class='code'><pre><code class=''><span class='line'>[root@pal ~]# grep -c http-200 responses.txt
</span><span class='line'>2</span></code></pre></td></tr></table></div></figure>
<p>In reality I ran the following command as wanted to do something a bit more complicated with curl by specifying certs and custom headers</p>
<figure class='code'><div class="highlight"><table><tr><td class="gutter"><pre class="line-numbers"><span class='line-number'>1</span>
</pre></td><td class='code'><pre><code class=''><span class='line'>grep "$1" urls.txt | awk '{print "curl --cert /etc/pki/my-pem.pem --cacert /etc/my-ca.pem -H \"Accept:application/json\" -H \"X-Candy-Audience:Domestic\" -H \"X-Candy-Platform:HighWeb\" --write-out "$0"=http-%{http_code}\"\n\" --silent --output /dev/null "$0'} | sh &gt;&gt; responses.txt</span></code></pre></td></tr></table></div></figure>
<p>I&rsquo;m not suggesting this is the best way to do this but nice not to need any frameworks or libraries. If I want to do this more regularly I will probably use a cucumber feature and setup a job in our CI environment not, forgetting a nice looking report but for now this is fine.</p>
    </article>
    <article>
      <h1 class="entry-title"><a href="/blog/2012/04/07/creating-a-http-response-version-provider-with-apache-cxf-filters/">Creating a HTTP Response Version Provider With Apache CXF Filters</a></h1>
      <p class="meta">
<time datetime="2012-04-07T22:16:00+02:00" pubdate data-updated="true">Apr 7<span>th</span>, 2012</time>
  </header>
  <div class="entry-content"><p>I&rsquo;ve been using <a href="http://cxf.apache.org/">Apache CXF</a> which has some nice support for building JSR-311 compliant JAX-RS Services and recently had a requirement for sending back a custom version header as part of our API Responses.</p>
<p>It is generally accepted that versioning APIs is a good thing and adopting the general contract of Major.Minor.Patch covers our needs, now we need to have a standard method of reporting and documenting the APIs and versions. Before we get into the ins and outs of how one you should version an API, we have a company wide recommendation of adding a custom response header to each response <strong>X-API-Version: d.d.d</strong> which I have to say I&rsquo;m in favour of over embedding the version in the url and is also the method adopted by Sun’s Cloud API &ndash; which certainly was commonly held to be a benchmark implementation of REST.</p>
<p>Using <a href="http://cxf.apache.org/docs/jax-rs-filters.html">Apache CXF Filters</a> led to this fairly elegant and simple solution which could be applied to add any type of additional headers but I found the documentation a little confusing so decided to write a post on it.</p>
    <footer>
      <a rel="full-article" href="/blog/2012/04/07/creating-a-http-response-version-provider-with-apache-cxf-filters/">Read on &rarr;</a>
    </footer>
    </article>
    <article>
      <h1 class="entry-title"><a href="/blog/2012/03/15/useful-xquery/">Useful XQuery</a></h1>
      <p class="meta">
<time datetime="2012-03-15T13:58:00+01:00" pubdate data-updated="true">Mar 15<span>th</span>, 2012</time>
  </header>
  <div class="entry-content"><p>As I mentioned in my previous <a href="http://joncook.github.com/blog/2012/02/12/evaluating-mark-logic-xquery-performance/">blog post</a> I&rsquo;ve recently been doing some work building RESTFul API&rsquo;s backed by a Mark Logic XML Content Store utilising XQuery for document retrieval which has led me come up with the following useful snippets of XQuery that I thought I would share. They could easily be altered to work with slightly different requirements.</p>
<h3>List all the document uris in database</h3>
<div><script src='https://gist.github.com/2039314.js?file=list_all_document_uris.xqy'></script>
<noscript><pre><code>for $i in collection() return document-uri($i)</code></pre></noscript></div>
<h3>List all the document uris in database based on some criteria</h3>
<p>In this case we are restricting the documents that exist in the /sitemap/ directory and end with sitemap.xml and do not contain archive in the uri</p>
<div><script src='https://gist.github.com/2039314.js?file=list_document_uris_where_uri_matches_criteria.xqy'></script>
<noscript><pre><code>for $d in xdmp:directory(&quot;/sitemap/&quot;, &quot;infinity&quot;)
  where fn:ends-with(fn:document-uri($d), &quot;sitemap.xml&quot;)
  and not fn:contains(fn:document-uri($d), &quot;archive&quot;)
  return fn:document-uri($d)</code></pre></noscript></div>
<h3>Delete all the document in the database</h3>
<div><script src='https://gist.github.com/2039314.js?file=delete_all_documents.xqy'></script>
<noscript><pre><code>for $i in collection() return xdmp:document-delete(fn:document-uri($i))</code></pre></noscript></div>
<h3>Delete all the document in database based on some criteria</h3>
<p>In this case we are deleting documents that are in the /content/ directory and whose uri ends with NEWSML.xml</p>
<div><script src='https://gist.github.com/2039314.js?file=delete_documents_where_uri_matches_criteria.xqy'></script>
<noscript><pre><code>for $d in xdmp:directory(&quot;/content/&quot;, &quot;infinity&quot;)
  where not(fn:ends-with(fn:document-uri($d), &quot;NEWSML.xml&quot;))
  return xdmp:document-delete(fn:document-uri($d))</code></pre></noscript></div>
<h3>Query a document by an element with a certain value</h3>
<p>This could be easily altered to match on any element or attribute in your xml.</p>
<div><script src='https://gist.github.com/2039314.js?file=get_document_where_element_matches_value.xqy'></script>
<noscript><pre><code>declare namespace bbc = &quot;http://www.bbc.co.uk/content/asset&quot;;
declare variable $visibleUrl := '/news/cucumber/';
let $results := /bbc:index[
    /bbc:index/bbc:itemMeta/bbc:url[text()=$visibleUrl]
return $results</code></pre></noscript></div>
<p>Actually a much more efficient way of doing the same is:</p>
<div><script src='https://gist.github.com/2039314.js?file=more_efficient_get_document_where_element_matches_value.xqy'></script>
<noscript><pre><code>import module namespace xinc = &quot;http://marklogic.com/xinclude&quot; at &quot;/MarkLogic/xinclude/xinclude.xqy&quot;;
declare namespace bbc = &quot;http://www.bbc.co.uk/asset&quot;;
let $public_url := &quot;/news/cucumber&quot;
let $asset := collection()[bbc:*/bbc:itemMeta/bbc:url/@href eq $public_url]/*
return xinc:node-expand($asset[1])</code></pre></noscript></div>
    <footer>
      <a rel="full-article" href="/blog/2012/03/15/useful-xquery/">Read on &rarr;</a>
    </footer>
    </article>
    <article>
      <h1 class="entry-title"><a href="/blog/2012/02/12/evaluating-mark-logic-xquery-performance/">Evaluating Mark Logic XQuery Performance</a></h1>
      <p class="meta">
<time datetime="2012-02-12T19:36:00+01:00" pubdate data-updated="true">Feb 12<span>th</span>, 2012</time>
  </header>
  <div class="entry-content"><p>I&rsquo;ve recently been doing some work building RESTFul API&rsquo;s backed by a Mark Logic XML Content Store utilising XQuery for document retrieval. This post details the steps involved in tuning what was deemed to be the most simplest of queries for optimum performance using some useful Mark Logic extensions for profiling.</p>
<h3>Original Query</h3>
<p>XQuery for looking up documents based on the value of a given attribute in the xml using XPath. (What could be simpler!)</p>
<div><script src='https://gist.github.com/1810958.js?file=document_by_url.xqy'></script>
<noscript><pre><code>xquery version &quot;1.0-ml&quot;;
let $public_url := &quot;%s&quot;
let $results := /*[
    /*/itemMeta/url[@href=$public_url]
let $asset := $results[1]
return $asset</code></pre></noscript></div>
<h3>Evaluating Performance</h3>
<p>By default Mark Logic indexes the document structure and attributes <strong>are</strong> indexed by default as part of the universal index.</p>
<h4>1. Adding xdmp:query-meters</h4>
<p>By adding xdmp:query-meters() to the query we get some immediate feedback about how the query performs including elapsed time and the number of fragments and documents that were selected. Altering the above query as below gives us some interesting metrics and the query is taking nearly 2 seconds.</p>
<div><script src='https://gist.github.com/1810958.js?file=document_by_url_with_xdmp_query_meters.xqy'></script>
<noscript><pre><code>return $asset,
xdmp:query-meters()</code></pre></noscript></div>
<div><script src='https://gist.github.com/1810958.js?file=result_of_xdmp_query_meters.xml'></script>
<noscript><pre><code>&lt;qm:elapsed-time&gt;PT1.802604S&lt;/qm:elapsed-time&gt;
&lt;qm:requests&gt;0&lt;/qm:requests&gt;
&lt;qm:list-cache-hits&gt;3&lt;/qm:list-cache-hits&gt;
&lt;qm:list-cache-misses&gt;0&lt;/qm:list-cache-misses&gt;
&lt;qm:in-memory-list-hits&gt;1&lt;/qm:in-memory-list-hits&gt;
&lt;qm:expanded-tree-cache-hits&gt;43762&lt;/qm:expanded-tree-cache-hits&gt;
&lt;qm:documents&gt;</code></pre></noscript></div>
<p>Immediately something looks a bit suspicious as all the documents in the database are being returned which would indicate that the query is not making effective use of Mark Logic&rsquo;s Indexes.</p>
    <footer>
      <a rel="full-article" href="/blog/2012/02/12/evaluating-mark-logic-xquery-performance/">Read on &rarr;</a>
    </footer>
    </article>
  <div class="pagination">
    <a href="/blog/archives">Blog Archives</a>
<aside class="sidebar">
    <section>
  <h1>Recent Posts</h1>
  <ul id="recent_posts">
      <li class="post">
        <a href="/blog/2013/11/03/xml-processing-with-scala/">XML Processing With Scala</a>
      </li>
      <li class="post">
        <a href="/blog/2012/04/30/some-bash-fu-with-curl/">Some Bash Fu With Curl, Grep, Awk and Pipes</a>
      </li>
      <li class="post">
        <a href="/blog/2012/04/07/creating-a-http-response-version-provider-with-apache-cxf-filters/">Creating a HTTP Response Version Provider With Apache CXF Filters</a>
      </li>
      <li class="post">
        <a href="/blog/2012/03/15/useful-xquery/">Useful XQuery</a>
      </li>
      <li class="post">
        <a href="/blog/2012/02/12/evaluating-mark-logic-xquery-performance/">Evaluating Mark Logic XQuery Performance</a>
      </li>
  <h1>GitHub Repos</h1>
  <ul id="gh_repos">
    <li class="loading">Status updating&#8230;</li>
  <a href="https://github.com/JonCook">@JonCook</a> on GitHub
  <script type="text/javascript">
    $(document).ready(function(){
        if (!window.jXHR){
            var jxhr = document.createElement('script');
            jxhr.type = 'text/javascript';
            jxhr.src = '/javascripts/libs/jXHR.js';
            var s = document.getElementsByTagName('script')[0];
            s.parentNode.insertBefore(jxhr, s);
        github.showRepos({
            user: 'JonCook',
            count: 2,
            skip_forks: true,
            target: '#gh_repos'
        });
  </script>
  <script src="/javascripts/github.js" type="text/javascript"> </script>
  <footer role="contentinfo"><p>
  Copyright &copy; 2014 - Jon Cook -
  <span class="credit">Powered by <a href="http://octopress.org">Octopress</a></span>
<script type="text/javascript">
      var disqus_shortname = 'jonscodingblog';
        var disqus_script = 'count.js';
    (function () {
      var dsq = document.createElement('script'); dsq.type = 'text/javascript'; dsq.async = true;
      dsq.src = '//' + disqus_shortname + '.disqus.com/' + disqus_script;
      (document.getElementsByTagName('head')[0] || document.getElementsByTagName('body')[0]).appendChild(dsq);
<div id="fb-root"></div>
<script>(function(d, s, id) {
  var js, fjs = d.getElementsByTagName(s)[0];
  if (d.getElementById(id)) {return;}
  js = d.createElement(s); js.id = id; js.async = true;
  js.src = "//connect.facebook.net/en_US/all.js#appId=212934732101925&xfbml=1";
  fjs.parentNode.insertBefore(js, fjs);
}(document, 'script', 'facebook-jssdk'));</script>
  <script type="text/javascript">
    (function(){
      var twitterWidgets = document.createElement('script');
      twitterWidgets.type = 'text/javascript';
      twitterWidgets.async = true;
      twitterWidgets.src = '//platform.twitter.com/widgets.js';
      document.getElementsByTagName('head')[0].appendChild(twitterWidgets);
  </script>
Provide feedback

Saved searches

Use saved searches to filter your results more quickly

FilesExpand file tree

index.html

Latest commit

History

index.html

File metadata and controls
