In my last post, I talked about generating a nested structure of comments and any replies pulled from a database at the backend of oration, so that this data could be exposed via an API in JSON format. Now, I'd like to continue the conversation at the frontend—hopefully answering the question: how do we invoke a recursive type and do some work with one?

Let's start out with a less obvious but nicer layout for our comments and replies:

type alias Comment =
    { text : String
    , author : Maybe String
    , hash : String
    , created : DateTime
    , id : Int
    , votes : Int
    , children : Responses
    , visible : Bool
    , editable : Bool
    , votable : Bool
    }


type Responses
    = Responses (List Comment)

At this point, the documentation gives you a nice looking upvote example function which for here would look like { comment | votes = 1 + comment.votes } for any given comment. Directly after this though, a small additional quip is added:

So rather than having to unwrap a Comment to do anything to it, you only have to do some unwrapping in the cases where you are doing something recursive.

That's great and all (and perhaps someone more versed in Elm could set me straight here), but when in this situation would you only be applying functions like upvote to Comments and not Responses? Since it's considered poor form to index into a list/array in a functional language, such unwrapping seems inevitable.

🔗 Voting

So let's have a look at oration's version of upvote to start with. Each comment's view has like and dislike buttons, which trigger two respective functions on a click event.

like : Int -> List Comment -> List Comment
like id comments =
    List.map (\comment -> voteComment ( id, True ) comment) comments


dislike : Int -> List Comment -> List Comment
dislike id comments =
    List.map (\comment -> voteComment ( id, False ) comment) comments

Notice the only difference is the value of the bool passed to voteComment, which as you can see, indicates a positive reaction if true. The reason why we wrap id and the bool into a tuple will become apparent soon.

The layout of both functions is representative of all top level functions that work on a List Comment. We map over all Comments in the List, and invoke a helper function inside an anonymous function. If you're not familiar with functional programming, you can conceptualise this by thinking about for i=1:N, where the for loop is the map, i is comment inside the anonymous function and N would be the length of comments. The map will visit all elements and expect something in return, but this could just be the element itself without any changes.

Perhaps my thinking here is not functional enough, since as you see I still pass around an id. This value is stored in the backend database and we need to connect to that at some stage, thus it doesn't represent an index in the list, so hopefully I'm not committing a functional sin with this. If my view somehow returned the Comment that was just clicked on, this would dramatically reduce the complexity here, and maybe this post wouldn't need to be written. For the moment though, let's move forward with possibly only a partial understanding of the problem space—this is how we learn. Since the id could be anywhere in the data structure, we have to map across all elements (which include all recursive Responses lists) to identify the correct comment to alter the vote on.

voteComment : ( Int, Bool ) -> Comment -> Comment
voteComment ( id, like ) comment =
    if comment.id == id then
        let
            count =
                case like of
                    True ->
                        comment.votes + 1

                    False ->
                        comment.votes - 1
        in
        { comment
            | votes = count
            , votable = False
        }
    else
        mapChildren ( id, like ) comment voteComment

In the end, this is ultimately the same as the example upvote function, but it allows you to find the correct comment in the list to upvote. Two simple aspects of the function we can get out of the way first are the votable bool, which just removes the ability for someone to spam multiple upvote requests to the backend—they should only be able to vote once. Second, the count variable converts a like or dislike into an increment or decrement on the total vote count. votes therefore has the ability to cross the zero threshold and become a positive number. This is probably overkill for an internet commenting system, but it's an edge case we can support without too much hassle.

It's the call to mapChildren when we don't match on id where the magic happens though.

mapChildren : a -> Comment -> (a -> Comment -> Comment) -> Comment
mapChildren value comment operation =
    let
        children =
            case comment.children of
                Responses responses ->
                    Responses <| List.map (\response -> operation value response) responses
    in
    { comment | children = children }

The function takes a variable of any type, a Comment and a function that takes both of these. Whilst I'm quite certain that it's possible to simplify this type signature with currying; let's just say that I've kept it explicit like this for you, the reader, to work out on your own. Write your answer in a comment below—I'll hold of on committing the correct answer to the oration repo until such time as we identify a winner. Most of the examples I'll go through below only need to pass an Int (in the form of id), but here we need both an Int and a Bool—hence the tuple in voteComment. The a here is type agnostic and lets us pass anything.

Inside the function, the case statement unwraps any list of replies into the responses variable for the current comment. As you can see in the initial type signature, responses is now a List Comment, which we can map over in the same manner we did with its parent. Notice that this calls voteComments in our example above from this nested position (or in fact any function identified as operation which we'll discuss below), and will continue to recurse down each branch of this comment list, checking each one. The result is returned and re-wrapped into the Responses type via the <| operator, which effectively means apply the result of this function to the left.

🔗 Visibility

With that machinery now built, we can extend the idea to other methods. For example, if you wish to hide a comment thread, you must at least flag the visibility of the root comment (then, you can remove it and all its children in your view method).

toggleVisible : Int -> List Comment -> List Comment
toggleVisible id comments =
    List.map (\comment -> switchVisible id comment) comments


switchVisible : Int -> Comment -> Comment
switchVisible id comment =
    if comment.id == id then
        { comment | visible = not comment.visible }
    else
        mapChildren id comment switchVisible

As you can see, the templating here is effectively the same—update if there's a hit on id, recurse through the children if not.

🔗 Updating

Updating a comment is a little more complex, but nothing overtly obtuse.

update : Edited -> List Comment -> List Comment
update edit comments =
    List.map (\comment -> injectUpdates edit comment) comments

injectUpdates : Edited -> Comment -> Comment
injectUpdates edit comment =
    if edit.id == comment.id then
        { comment
            | text = edit.text
            , author = edit.author
            , hash = edit.hash
            , editable = True
        }
    else
        mapChildren edit comment injectUpdates

Here I've type aliased a subset Comment to sandbox what a user has access to. This Edited type therefore has the required changes needed for some id. When the id of a comment in the list and our edited value matches we transfer the values. It's still possible to leverage the mapChildren function, since the agnostic a type is happy to pass an Edited type down the line.

🔗 Alterations on the list

OK, so we can now alter components of each comment without too much hassle, but what about operating on the list in general?

There are number of these alterations in the Data.Comment module if you're looking for more examples, but this post is getting long as it is so we'll stick to just two—they are mostly more of the same but cater to different edge cases.

🔗 Inserting a Comment

insertNew : Inserted -> ( String, String, DateTime, List Comment ) -> List Comment
insertNew insert current =
    let
        ( commentText, hash, now, comments ) =
            current

        newComment =
            { text = commentText
            , author = insert.author
            , hash = hash
            , created = now
            , id = insert.id
            , votes = 0
            , children = Responses []
            , visible = True
            , editable = True
            , votable = False
            }
    in
    if isNothing insert.parent then
        comments ++ List.singleton newComment
    else
        List.map (\comment -> injectNew insert newComment comment) comments


injectNew : Inserted -> Comment -> Comment -> Comment
injectNew insert newComment comment =
    let
        children =
            if comment.id == insert.parent ? -1 then
                case comment.children of
                    Responses responses ->
                        Responses <| responses ++ List.singleton newComment
            else
                case comment.children of
                    Responses responses ->
                        Responses <| List.map (\response -> injectNew insert newComment response) responses
    in
    { comment | children = children }

Let's unpack this one a little. There are a few inputs which allow us to construct the newComment, along with some defaults including an empty Responses list. The if statement in insertNew checks if the type aliased Inserted structure (which consists of an id, parent and author) has a parent. In the case of a genuine new comment, we append it to the list, otherwise the standard helper function template is invoked to search for the right place to put the reply.

The children check can't be done via mapChildren since we need to do something a little more complex in this situation. The method herein attempts to identify the parent id (the insert.parent value is of the Maybe Int type, so we need to unwrap it with a known failure of -1), and if matched we can append the new comment to the end of this Responses list, otherwise we must go deeper.

🔗 Comment Deletion

Oration has two deletion methods, since it's possible that a user (or an admin) wishes to remove an unwanted comment but keep the corresponding replies. In this instance we effectively cripple the comment—removing all content and author details concerning it, but keep its id and timestamp information so that replies can be rendered in the correct manner.

Alternatively, the is no need to store anything about a deleted comment with no children, so we purge these.

delete : Int -> List Comment -> List Comment
delete id comments =
    List.map (\comment -> filterComment id comment) comments
        |> values

filterComment : Int -> Comment -> Maybe Comment
filterComment id comment =
    if comment.id == id then
        let
            --Pure deletes only happen on comments with no children, so only filter if that's the case
            noChildren =
                case comment.children of
                    Responses responses ->
                        List.isEmpty responses
        in
        if noChildren then
            Nothing
        else
            --We must display a masked delete
            let
                children =
                    case comment.children of
                        Responses responses ->
                            Responses <| values <| List.map (\response -> filterComment id response) responses
            in
            Just
                { comment
                    | children = children
                    , author = Nothing
                    , hash = ""
                    , text = ""
                    , votes = 0
                    , votable = False
                }
    else
        let
            children =
                case comment.children of
                    Responses responses ->
                        Responses <| values <| List.map (\response -> filterComment id response) responses
        in
        Just { comment | children = children }

These two possibilities force us to alter our template a bit. On one hand, we may return from our recursion with a complete list with an altered comment, on the other we may have removed a comment entirely. Thus, it's necessary to return a Maybe Comment from our helper function, then filter out any empty elements using |> values in the outer function delete.

Once we're in the helper function, we can asses each comment's properties. Initially, there's a check for an empty children list, which is used to completely delete the comment. Otherwise, the response is a crippled comment with no data other than its complete child list. The children methods unwrap Responses in a similar to that of mapChildren, although the Maybe values must be handled explicitly.

🔗 Counting Comments

There's something more simple than all of this that is paramount, and that's counting how many elements you have in this nested list of yours.

No problem right?

count : List Comment -> Int
count =
    foldl (\_ acc -> acc + 1) 0

But this is only going to count your original, root comments and ignore all replies. We need to overload the foldl function for our type.

foldl : (Comment -> b -> b) -> b -> List Comment -> b
foldl f =
    List.foldl
        (\c acc ->
            case c.children of
                Responses responses ->
                    foldl f (f c acc) responses
        )

Here, we accumulate all the way down each comment branch, and once that's done use the list left fold method to sum the result, which in turn is passed up the chain.

For an imperative programmer, functional methods can sometimes seem confusing. It takes time for them to 'click', but once that happens there is a certain elegance to writing code in this way. No doubt, a true functional aficionado would scoff at most of this post, so please, if you are such a person—let me know what I can do to improve this. For the rest of you, I hope that this gives you some more in depth examples to help you with your own projects.

Update 2018-04-05: Concerning the closing remarks here, writing this post actually made me 'click' even more, allowing me to cut around 30% of the logic in the Data.Comment module and simplify this post down in its complexity a good deal. Feel free to take a look at the history of the module and this post on Github to see the changes.

Say we have a table full of Foos, where some Foos may have other Foo parents. This can easily be represented in a table with some unique primary key id, a secondary key parent, which joins to the parent's id, and whatever data Foo may hold. Retrieving this table into a rust data structure, we'd find ourselves with something like Vec<Foo>, where Foo is defined as

struct Foo {
    id: i32,
    parent: Option<i32>,
    data: String,
}

This isn't really the best way to represent these data though, what you really want is more akin to

struct Bar {
    id: i32,
    data: String,
    children: Vec<Bar>,
}

This is a problem I came up against when writing the comment API for Oration. Each comment is inserted into the database without too much hassle. We know the parent id, since the frontend forwards us that information, and the unique primary key is simply auto-incremented. When a select statement returns all comment data for a particular thread, we get a vector of PrintedComments back.

// Pull data from DB for a given `path`. `conn` here is the connection to the backend database
let comments: Vec<PrintedComment> = PrintedComment::list(conn, path)?;

The complete PrintedComment struct looks like the following at the time of writing:

#[derive(Serialize, Queryable, Debug)]
/// Subset of the comments table which is to be sent to the frontend.
struct PrintedComment {
    /// Primary key.
    id: i32,
    /// Parent comment.
    parent: Option<i32>,
    /// Actual comment.
    text: String,
    /// Commenters author if given.
    author: Option<String>,
    /// Commenters email address if given.
    email: Option<String>,
    /// Commenters website if given.
    url: Option<String>,
    /// Commenters indentifier.
    hash: String,
    /// Timestamp of creation.
    created: NaiveDateTime,
    /// Number of likes a comment has recieved.
    likes: Option<i32>,
    /// Number of dislikes a comment has recieved.
    dislikes: Option<i32>,
}

Since we want to show comment threads to the user; having entries in a flat list, sorted effectively by date isn't that helpful. We need a NestedComment layout:

#[derive(Serialize, Debug)]
/// Subset of the comments table which is to be nested and sent to the frontend.
pub struct NestedComment {
    /// Primary key.
    id: i32,
    /// Actual comment.
    text: String,
    /// Commenters author if given.
    author: Option<String>,
    /// Commenters indentifier.
    hash: String,
    /// Timestamp of creation.
    created: DateTime<Utc>,
    /// Comment children.
    children: Vec<NestedComment>,
    /// Total number of votes.
    votes: i32,
}

I'll skip over some of the conversions here, but you can clearly see we compress the commenter's contact details into author, and sum votes. Most importantly though, we need a way to implement the nesting.

We know that our comments will have at most one parent, and that parent will not be itself. In that sense, the tree structure we wish to build can be considered as a special case of a directed digraph, and the method to generate a set of NestedComments involves building such a graph of all PrintedComments; traversing it recursively, then returning sets of children at each level.

Bluss has written an excellent graph data structure library called petagraph from which we'll build a petgraph::DiGraphMap—a directed graph that allows us to control the node ids. If a node has a parent, we ensure that exists in the graph and add an edge from the parent to the child. If it doesn't have a parent, we know this is one of our top-level comments, so we stash it aside for later:

let mut graph = DiGraphMap::new();
let mut top_level_ids = Vec::new();

for comment in &comments {
    //For each comment, build a graph of parents and children
    graph.add_node(comment.id)

    //Generate edges if a relationship is found, stash as a root if not
    if let Some(parent_id) = comment.parent {
        graph.add_node(parent_id);
        graph.add_edge(parent_id, comment.id, ());
    } else {
        top_level_ids.push(comment.id);
    }
}

Next, we iterate over all of the top-level ids and collect our children:

//Run over all root comments, recursively filling their children as we go
let tree: Vec<_> = top_level_ids
    .into_iter()
    .map(|id| build_tree(&graph, id, &comments))
    .collect();

The build_tree function is the recursive core of the problem. The implementation first (again) calls the recursive map for all comments and collects direct children at this level into a vector. The ::new constructor on NestedComment is passed this populated list of children, building the complete struct for the given top-level id. If the child vector is empty, we know the current comment has no replies, and we can therefore pass an empty vector to the NestedComment constructor.

/// Construct a nested comment tree from the flat indexed data obtained from the database.
fn build_tree(graph: &DiGraphMap<i32, ()>, id: i32, comments: &[PrintedComment]) -> NestedComment {
    let children: Vec<NestedComment> = graph
        .neighbors(id)
        .map(|child_id| build_tree(graph, child_id, comments))
        .collect();

    //We can just unwrap here since the id value is always populated from a map over contents.
    let idx: usize = comments.iter().position(|c| c.id == id).unwrap();

    if !children.is_empty() {
        NestedComment::new(&comments[idx], children)
    } else {
        NestedComment::new(&comments[idx], Vec::new())
    }
}

At this point, our list is completely nested and ready to be pushed to the frontend without fear of your witless, snarky responses being incorrectly assigned to the wrong first! comment.

You can take a look at the complete implementation of NestedComment's methods here. In the coming days I plan to write up some further caveats I uncovered when working with this data in Elm on the frontside of oration, so please at least attempt to contain your excitement until then.

There's not even time to take a mirco-break to refresh your favourite time sink website before you've busted out the coordinates of \(C\)

$$e = \frac{b^2+c^2-a^2}{2c}, \quad f= \sqrt{b^2-e^2}.$$

On a roll, you extend the coordinates of \(A\), \(B\) and \(C\) into three dimensions setting their \(z\) component to \(0\) and construct a tetrahedron from this base triangle and three new distances \(m = r_A+r_D\), \(n = r_B+r_D\), \(p=r_C+r_D\); finding the tetrahedron's fourth vertex

$$x_D = \frac{m^2+c^2-n^2}{2c}, \quad y_D = \frac{f^2c+m^2c-m^2e+n^2e-p^2c-c^2e+e^2c}{2fc}, \quad z_D = \pm\sqrt{m^2-x_D^2-y_D^2}.$$

Pausing for just a moment, you realise you've been on autopilot for a second—since such trivial calculations are self evident. Reflecting on a time that feels so long ago now, when you learnt that a minimal set of four tangent spheres in three dimensions can be described by a tetrahedron. The locus of each sphere's center point is one of the tetrahedron's vertices, and the lengths of each edge of the tetrahedron are defined via the radii of the pair of spheres for which the edge connects their centers. Using the already-known properties of the base triangle; introducing a fourth sphere \(D\) with known radius \(r_D\) and finding its coordinates, is now just some basic geometry—extending the system of equations used to solve the locus of \(C\)

$$\begin{aligned} e^2+f^2&=b^2\\ (e-c)^2+f^2&=a^2,\\ \end{aligned}$$

to the intersection represented by $$\begin{aligned} x_D^2+y_D^2+z_D^2&=m^2\\ (x_D-c)^2+y_D^2+z_D^2&=n^2\\ (x_D-e)^2+(y_D-f)^2+z_D^2&=p^2.\\ \end{aligned}$$

Back from your daydream—that time sink website is calling. Since you've pretty much wrapped this problem up, why not take a peek? After a few sensible chuckles, it dawns on you that there's still work to do. Your sphere packing algorithm is going to feed you positions and radii of arbitrary sets of pairwise tangent triplets for you to calculate the fourth position. This means that if you are solving the tetrahedron in this simple basis, translation and rotation of \(D\)'s position must occur.

Moreover, this is going to have to happen in a number of steps.

1. Remove \(\triangle ABC\)'s translation component, aligning \(A\) to the origin.
2. Recalculate \(c, e, f\) in the simple basis for this triplet to solve for the fourth locus.
3. Denote the solution as \(H\) in tetrahedron \(EFGH\), which now must be translated to tetrahedron \(ABCD\)'s basis to find \(D\).
4. Rotate \(\overrightarrow{EF}\) onto \(\overrightarrow{AB}\).
5. Rotate \(G\) to \(C\) by rotating the face normals of the triangle identified in 4 and \(\triangle ABC\).
6. Translate this triangle to the \(ABC\) basis by returning the translation component removed in 1.

None of this is too difficult in of its own, but without proper forethought—you're gonna have a bad time. Basically you're going to end up catching edge cases just to the left of center like a tennis ball to the nuts of a daydreaming ball boy at the US Open.

What happens when you're sent a triplet that has coordinates (in the translated basis) \((0,0,0)\), \((e,f,0)\), \((c,0,0)\) or in other words: when \(B=G\) and \(C=F\)? You must check for this mirror symmetry to make sure that the first rotation doesn't place your \(F\) position closer to Alpha Centauri than it does to it's wanted position at \(B\).

The list goes on, but it suffices to say that you start juggling possibilities like some guy trying to wash a car in an infomercial.

Fret not! There is a better way!

Consider that we have a pairwise tangent triplet with known radii and arbitrary positions in \(\mathbb{R}^3\). Additionally, we know the radius of a fourth sphere we wish to add to the cluster.

To not confuse the skimming reader—I will change notations here, and define a tetrahedron with vertices \(S_1\), \(S_2\) and \(S_3\) residing at distances \(d_1\), \(d_2\), \(d_3\) from a fourth vertex \(S_0\), whose coordinates \((x,y,z)\) must be found.

The distance values here are straightforward, since they are defined as the sum of the radii along a given edge: \(d_i=r_i+r_0\). Since \(S_0\) is at the intersection of three spheres centered at \(S_1\), \(S_2\), \(S_3\) with radii \(d_1\), \(d_2\), \(d_3\), we know its coordinates satisfy the equations

$$\begin{aligned} (x-x_1)^2+(y-y_1)^2+(z-z_1)^2&=d_1^2\qquad(1)\\ (x-x_2)^2+(y-y_2)^2+(z-z_2)^2&=d_2^2\qquad(2)\\ (x-x_3)^2+(y-y_3)^2+(z-z_3)^2&=d_3^2\qquad(3)\\ \end{aligned}$$

Subtract (1) from (2) & (3) to obtain the equations for two planes, and couple this set with (1) to yield the quadratic system:

$$\begin{aligned} 2(x_1-x_2)x+2(y_1-y_2)y+2(z_1-z_2)z&=d_2^2-d_1^2+x_1^2-x_2^2+y_1^2-y_2^2+z_1^2-z_2^2\\ 2(x_1-x_3)x+2(y_1-y_3)y+2(z_1-z_3)z&=d_3^2-d_1^2+x_1^2-x_3^2+y_1^2-y_3^2+z_1^2-z_3^2\\ x^2+y^2+z^2-2 x_1 x-2 y_1 y-2 z_1 z &= d_1^2-x_1^2-y_1^2-z_1^2.\\ \end{aligned}$$

The closed form solution to this system is not the nicest of expressions, so we can consider the equivalent set below in vector format $$ \begin{aligned} \vec u\cdot\vec r=a\\ \vec v\cdot\vec r=b\\ \vec r\cdot\vec r+\vec w\cdot\vec r=c\\ \end{aligned} $$ where \(\vec r=(x,y,z)\) is our unknown vector. Following from this, we obtain values of most of the components directly,

$$\begin{aligned} \vec u&=\frac{S_1-S_2}{\left|S_1-S_2\right|}\\ \vec v&=\frac{S_1-S_3}{\left|S_1-S_3\right|}\\ \vec w&=-2S_1\\ a&=\frac{d_2^2-d_1^2+x_1^2-x_2^2+y_1^2-y_2^2+z_1^2-z_2^2}{2\left|S_1-S_2\right|}\\ b&=\frac{d_3^2-d_1^2+x_1^2-x_3^2+y_1^2-y_3^2+z_1^2-z_3^2}{2\left|S_1-S_3\right|}\\ c&=d_1^2-x_1^2-y_1^2-z_1^2\\ \end{aligned}$$

Notice that \(\vec u\) and \(\vec v\) are unit vectors, so we can form a basis with a third vector:

$$\vec t=\frac{\vec u\times\vec v}{\left|\vec u\times\vec v\right|}.$$

A solution to \(\vec r\) can then be obtained in the form \(\vec r=\alpha\vec u+\beta\vec v+\gamma\vec t\) once \(\alpha\), \(\beta\) and \(\gamma\) are identified. Plugging this expression for \(\vec r\) into the vector equations for \(a\) and \(b\) above immediately yields

$$\alpha=\frac{a-b(\vec u\cdot\vec v)}{1-(\vec u\cdot\vec v)^2},\quad\beta=\frac{b-a(\vec u\cdot\vec v)}{1-(\vec u\cdot\vec v)^2},$$

while the third equation becomes \(\gamma^2+(\vec w\cdot\vec t)\gamma+d=0\) where \(d=\alpha^2+\beta^2+2\alpha\beta(\vec u\cdot\vec v)+\alpha(\vec u\cdot\vec w)+\beta(\vec v\cdot\vec w)-c\).

This last equation can finally be solved for \(\gamma\):

$$\gamma=\frac{1}{2}\left(-(\vec w\cdot\vec t)\pm\sqrt{(\vec w\cdot\vec t)^2-4d}\right).$$

With these values you can now solve the irregular tetrahedron in 3D directly! This means no edge cases, no recalculation of previously known values or complex if elseif else blocks based on various input conditions.

]]> https://axiomatic.neophilus.net/clustering-tangent-spheres/ Tim DuBois Geometry https://axiomatic.neophilus.net/code-coverage-in-rust/ Mon, 19 Jun 2017 00:00:00 +0000 One side project that has been sitting by the wayside since completing my PhD has been to polish up and publish Wafer: my fancy 3D wavefunction solver. For reasons, one of which is mostly structured procratstination, this polishing ended up being a port/re-write in Rust. Delving into the Rust ecosystem has been fun and if you check Wafer's commit history you'll notice large rewrites again and again as I basically play around with things; having no real development schedule or design restrictions.

It may seem a little counter intuitive then, that I'm also trying at the same time to learn all the rigid structures of software development. Proper documentation, testing, error handling, data structures etc.

The Rust tooling makes much of this a breeze. For example, you write comments above your functions in markdown, call cargo doc and a complete document is compiled for you.

Rust is actively evolving though, and hiccups do occur—sometimes more that you'd like. Staying with the documentation example, if you want to document all your routines as more of a developers document, the command to do so is simply

cargo rustdoc --open -- --no-defaults --passes collapse-docs --passes unindent-comments --passes strip-priv-imports

and obviously rolls off the keyboard just as effortlessly as cargo doc. Fixes are in the works for this oversight, and it's really not too much more than a quirk that's fairly obvious to find a solution for. Usually that comes in the form of a well written article on some well known and respected Rust blogger's site.

Recently, an excellent write up by Razican on getting Rust to play with Travis enabled me to set up a seamless testing environment, with developer documentation instantly updated on every commit (here if you want to check it out).

Formal, structured testing is somewhat new to me, since spitting debug data to stdout and making sure numbers matched up to some expected output has pretty much been end-of-story in the past. Travis is a tool that other open source projects have had enabled, so it's existence is known to me, and previously I even managed to add some tests that did a good job of failing for a while before getting it right.

The idea of code coverage however is something I'd never heard of before Razican's post. So, to check it out I chucked in the extra configuration steps to my .travis.yml and set up a Codecov account. To my surprise, the five or so extremely minimal tests I'd written gave me an impressive 96% coverage!

Something didn't add up.

Pulling up the raw logs, it was clear that total_lines here

{"file": "/home/travis/build/Libbum/Wafer/src/grid.rs", "percent_covered": "100.00", "covered_lines": "35", "total_lines": "35"}

needs to be a little larger. That file was 722 lines long at the time.

Codecov support replied to me extremely quickly, although they couldn't help solve the issue: the raw logs are what they match on, so it's the rust compiler that's not producing them correctly.

Here's the point where I'd expect again to say that there's a bit of a quirk with a fairly well documented solution, but this time none was to be found. Not that there wasn't a solution, but there just wasn't an easily explained blog post about it. Alas, the burden has therefore fallen squarely on my lap—the not-known-for-much-of-anything guy who's Rust tag count on this blog is 1 (provided you count this post in that category).

Basically what's happening is kcov (the tool we use to generate the logs that are sent to Codecov from the cargo test run) is quite language agnostic and just follows pathways of code that has been run vs not-run.

For example, a test like this on a convoluted function

fn between_five_ten(num: u23) -> bool {
    if num > 5 {
        if num < 10 {
            true
        } else {
            false
        }
    } else {
        false
    }
}

#[test]
fn test_four() {
    assert_eq!(between_five_ten(4), false);
}

will never enter the if num < 10 branch, meaning for 11 lines of code we test 6. If the above file had an additional 50 functions totalling some 800 lines of code, the result of kcovs tests would sill be 6 lines covered of a total of 11 lines.

The Rust compiler is effectively messing up kcov here rather than a bug existing somewhere in kcov itself. rustc will compile each function (and each global) into its own section of the output object file, then --gc-sections is passed to the linker which eliminates all unused elements. The reason for this is to cut down binary sizes, as we pull in tonnes of libraries for ease of use when coding up our projects, but seldom actually use all of the features. This does, however, have the implication that testable functions are stripped by default in test executables if they are not used (more specifically here: not tested), and now we see why kcov isn't handling information as we'd like.

With this now understood, it was easier to identify if a solution had already been implemented or not by trawling the Rust bug tracker.

Thankfully, the issue was fixed in February by implementing a link_dead_code flag, which explicitly stops the linker from stripping untested code out of the binary. Ultimately you get a much larger executable, but the point here is to only compile with this flag if you need code coverage information.

So the now-simple fix is to run cargo test as RUSTFLAGS=-Clink-dead-code cargo test. Updating my .travis.yml and pushing a new commit swapped out my impressive 96% coverage to a much more impressive 9%. Time to get to writing some actual tests then!

Starting from a clean install I went about revamping a number of things, as well as keeping on top of the new security measures that have been added to the observatory in the past six months.

I love static sites and have no use for too much dynamic frivolity, which also means I can tighten up security policies so my server is (hopefully) more of a vault with an excellent CCTV system for you all to look inside with. But comments on this blog have been an issue. Disqus has been a necessary evil since this blog's inception, but both my own and the worries of the internets are something that needed tending to.

Just look at this CSP header!

add_header Content-Security-Policy "default-src 'self'; script-src 'self' https://axiomatic.disqus.com https://api.github.com https://d3js.org https://code.jquery.com https://cdnjs.cloudflare.com https://a.disquscdn.com https://ajax.googleapis.com; font-src 'self' https://fonts.googleapis.com https://fonts.gstatic.com; style-src 'self' 'unsafe-inline' https://fonts.googleapis.com https://a.disquscdn.com; img-src 'self' data: https://referrer.disqus.com https://a.disquscdn.com https://i.creativecommons.org https://licensebuttons.net; frame-src 'self' https://disqus.com";

Apart from a few CDN and API links (github, d3js and cloudflare) this is all to satiate Disqus, and even then you could watch the developer console in Chrome and see it still bash up against the policy. I've never had any of the advertising integration turned on either, so I'd hate to see what happened if I was that kind of asshole.

So I've taken the plunge and moved to Isso: a self hosted alternative which requires no login (can be anonymous if you wish), no external services, no bullshit. Well, that last one is only partially true unfortunately. Take a look at my CSP now:

add_header Content-Security-Policy "default-src 'self'; font-src 'self' https://fonts.gstatic.com; script-src 'self' 'unsafe-eval' https://api.github.com https://d3js.org https://cdnjs.cloudflare.com; style-src 'self' https://fonts.googleapis.com 'unsafe-inline'";

Much better; but notice anything suspect? There's one eval in the code-base which means putting in an unsafe-eval and dropping 15 points from my Observatory score.

#274 highlights this and it's not an easy fix. The commenting system relies on an old version of pug.js to do the templating (back when it was called jade) and thus needs a major overhaul to fix. So for the moment I've managed to swap implicit problems with Disqus to an explicit issue with Isso. Even so, this is an issue that is in my power to fix, when (read: if) I have the time.

The moderation system requires an unsafe-inline in script-src to allow one click acceptance or deletion of a comment via an email notification you receive once someone posts. This, in addition to the eval issue just wont fly with me; but at this point I had already come too far. The solution is quite straightforward though: just edit the comment database directly.

select * from comments where mode = 2;
update comments set mode = 1 where id = 28;
  **OR**
delete from comments where id = 28;

Here, mode = 2 means awaiting approval (1 is live), and the id is a comment you wish to approve or deny. A lazy (savvy) user may also script an ssh connection, sqlite connection, and a somewhat modified query/action to generate one click acceptance and deletion links they could replace in their notification emails.

Security gripes aside, Isso is quite nice and built close to my principles concerning the current state of the Web. Initially I thought Gravatar integration would be nice, but now I like the philosophy behind the identicon. Running an sqlite back-end makes sense. The moderation gate minimises spam intake and doesn't require a hefty Bayesian parser hogging my cycles. Markdown in comments. etc.

The code-base is also small enough to hack on, and add quick and dirty additions. One particular thing I missed from Disqus was the obvious author attribute to comments. For the moment each Isso user looks the same—this is fine for a Marxist utopia, but I'm the one yielding what little power I have over this small portion of the web and I intend to let that be known.

You can see in comment.jade that if we have the avatar system on, Isso generates an identicon for each comment from the comment.hash. This is a PBKDF2 hash generated from either the email address or IP of the user.

div(class='isso-comment' id='isso-#{comment.id}')
    if conf.avatar
        div(class='avatar')
            svg(data-hash='#{comment.hash}')
    div(class='text-wrapper')
        div(class='isso-comment-header' role='meta')

Since this is a hack job, I just identified mine from a comment I had already created, then modified the above to add an additional class if my hash matched the current one the template is processing:

div(class='isso-comment' id='isso-#{comment.id}')
    if conf.avatar
        div(class='avatar')
            svg(data-hash='#{comment.hash}')
    div(class=(comment.hash == '213ac8a3c887') ? 'text-wrapper author-highlight' : 'text-wrapper')
        div(class='isso-comment-header' role='meta')

and wrote an appropriate style

.author-highlight {
    background-color: rgba(200, 200, 200, 0.4);
    border-top-right-radius: 25px;
    border-bottom-right-radius: 25px;
}

At some stage soon I'll put together a pull request to add something similar to Isso directly, but that will probably require an addition to the configuration file. Allowing the author to identify themselves without the above hassle. To see this in action, perhaps take a look at one of the only actual comment sections on this website. And yes: the fact that I've just ranted about commenting systems without having much use of a comment system at all does not escape me. Perhaps my readers don't care for Disqus more-so than I?

Update April 21, 2017

I actually managed to build this pull request much quicker than I anticipated. This is a fully functional blog author highlighting system—a little different than the one I've written about above, but much less of a hack job.

   d3.geo.orthographic().translate([width / 2, height / 2]).scale(width / 2 - 20).clipAngle(90)

and then move on with their lives.

But what if you care about information on this projection? What if Mike Bostock's World Tour gave you this grandiose idea of an interactive globe for your website that needs to pinpoint tiny countries like Singapore?

Singapore and other small countries like it are not included in the main 1:110m scale data as they are not large enough to be polytopes. Instead, they are relegated to ne_110m_admin_0_tiny_countries as point information and usually omitted from D3 1:110m globe implementations.

Including this data set adds a small amount of complexity to the build and a bit of finessing of the finished product, but nothing over the top. I'll assume you have TopoJSON setup already and know how to use it. If not, checkout Let's Make A Map for a good overview. For the lazy ones you can grab the end result of the following steps from here.

Get the latest data from Natural Earth. This can be whatever you want to combine; I'll be using just countries and tiny country points, but it'd also be possible to use the sovereignty data in place of countries (for example) if you want to drill down a little deeper. Using the 1:50m or 1:10m sets is also fine, however I find that this is too precise (read: too much info) to animate smoothly on your grandma's computer or mobile devices.

Extract both sets into a new directory and convert the shape data to GeoJSON:

ogr2ogr -f GeoJSON countries.json ne_110m_admin_0_countries.shp
ogr2ogr -f GeoJSON tinycountries.json ne_110m_admin_0_tiny_countries.shp

We now need to manually merge these files. AFAIK, ogr2ogr isn't happy merging poly and non-poly geometry, and the TopoJSON command line tool doesn't merge stuff at all. It's possible to merge this data on the client side each load, but this seems pretty inefficient. One could write a simple shell script for this, but as we're only doing it once I see no need for automation. The GeoJSON format we have output with our two files above follows a simple syntax:

   { "type": "FeatureCollection", "features": [ ... ] }

with a plethora of Feature elements filling the array. To merge the data one must simply add a comma (,) to the last entry of the countries.json features array, then append the contents of the same array from tinycountries.json. For this example we'll save that file as world.geo.json.

Now, our newly created file can be converted to TopoJSON via

    topojson -o world.json --id-property su_a3 --properties name -- countries=world.geo.json

and loaded using a somewhat normal D3 map routine. The important part is the d3.json function. Here we don't use a mesh load but rather a path append:

d3.json("/assets/world.json", function(error, world) {
    if (error) throw error;

   var land = topojson.feature(world, world.objects.countries).features;

   svg.insert("g", ".graticule").attr("id", "countries");
   d3.selectAll("#countries").selectAll("path").data(land)
      .enter().append("path").attr("class", "land")
      .attr("id", function(d, i) { return d.id; }).attr("d", path);
 });

This, along with a little boiler plate (which you can view here), gives us the following globe:

To do this one can modify the .data(land) item to include a filter

.data(land.filter(function(d) { return d.geometry.type !== 'Point'; }))

which will give us back our normal looking globe. However, we now have a complete list of individual countries which we can reference via their su_a3 id or name and in-turn, center the viewport on a country even if it isn't visible on this scale. The map below is filtering out ONLY tiny countries and giving you a world tour. Full codebase for this globe is here.

For a better demo of what can be done with this, checkout my photoblog Odyssey.

After looking under the hood though I could see my work was going to be cut out for me if I wanted to incorporate it into this blog. Unlike MathJax, which scanned the document.body and rendered math elements (indicated via \[ ... \] tags) on the fly, KaTeX required the following syntax:

katex.render("c = \\pm\\sqrt{a^2 + b^2}", element);

Not really a problem in general, although it was obvious for my case I'd have to write some form of math parser for the markdown portion of Hakyll. Still being quite the Haskell neophyte, this became something for the too hard basket. Maybe I also rationalised the decision not to work on a solution due to the very restricted subset of LaTeX features implemented at the time. I mean, not that I ever do much with relational algebra, but the natural join operator (aka the bowtie: \(\bowtie\)) wasn't introducted until v0.2.0. How am I supposed to make a fancy outfit for my mathematical symbol cat illustrations without a bowtie?

Fast forward a year and KaTeX has matured immensely, and is currently at v0.5.0. The feature list is much more expansive, and the only gripes I really have are the missing environments like \align and the font modifers \mathrm, \mathbf, \mathcal etc. However, it looks like these will all be part of the next release according to the status of this pull request.

My personal saviour was introduced in v0.3.0 by the way of the auto-render extension which acts in a similar fashion to the MathJax implementation and scans for $$ ... $$, \[ ... \] and \( ... \) tags. The first two tags indicate the display size environment whereas the last tag is for inline maths.

Luckily, you can configure Pandoc such that Hakyll parses the single $ ... $ tag in markdown and renders it as \( ... \) for use with MathJax. Travis Athougies has a good writeup on how to do this on his blog here, although I overload my pandoc options a little different so that Pygments can be used to do the syntax highlighting on my blog (see my Pandoc.hs):

readerOpts :: ReaderOptions
readerOpts =
    let extensions =
        S.fromList [ Ext_tex_math_dollars
                              , Ext_inline_code_attributes
                              , Ext_abbreviations
                              ]
    in def { readerSmart = True
                , readerExtensions = S.union extensions (writerExtensions def)
                }

writerOpts :: WriterOptions
writerOpts = def { writerHTMLMathMethod = MathJax ""
                                  , writerHighlight = False
                                  , writerHtml5 = True
                                  }

So to get KaTex working from this starting point, we just need to build a maths context

import qualified Data.Map as M

mathCtx :: Context a
mathCtx = field "katex" $ \item -> do
    metadata <- getMetadata $ itemIdentifier item
    return $ if "katex" `M.member` metadata
                            then "<link rel=\"stylesheet\" href=\"/css/katex.min.css\">\n\
                                      \<script type=\"text/javascript\" src=\"/js/katex.min.js\"></script>\n\
                                      \<script src=\"/js/auto-render.min.js\"></script>"
                            else ""

and apply it to any template where the context is needed. For me, that's:

loadAndApplyTemplate "templates/default.html" (mathCtx `mappend` postCtx) full
--- ... ---
>>= loadAndApplyTemplate "templates/default.html" (mathCtx `mappend` indexCtx)

which you can checkout properly in my site.hs. The reason I do it this way rather than just calling the stylesheet and .js files in default.html is to keep the site as lean as possible. Sure, I have some fancy things going on in the header etc. but I'm not pulling in jQuery or some other large library for no good reason, and there's no good reason to load KaTeX on every page when only a few places will ever need it.

So I have a katex field which, if activated in the preamble of a post (for instance, see the markdown of this entry here), pumps in the return result of the katex field above into default.html (full source here) and calls the auto-renderer.

$if(katex)$ $katex$ $endif$
<!-- ... -->
$if(katex)$
<script>
  renderMathInElement(document.body);
</script>
$endif$

With all that in place, embedding maths into posts is essentially trivial, cross platform and most importantly seemless. As an example, heres a short discussion about some of my recent work using just the $ ... $ inline and $$ ... $$ display tags in markdown.

Wick rotations are primarily used to find solutions to Minkowski space problems by an Euclidean space mapping. We can also use the rotation to evolve a time-dependent Schrödinger equation and solve for time-independent solutions in three dimensions. To do so we transfer our TDSE into an imaginary time basis $t=i\tau$:

$$i \hbar \frac{\partial}{\partial t}\Psi(\vec{r},t) = H\Psi(\vec{r},t) \Rightarrow - \hbar \frac{\partial}{\partial \tau}\Psi(\vec{r},\tau) = H\Psi(\vec{r},\tau)$$

which yields a general solution to the wavefunctions

$$\Psi(\vec{r},\tau) = \sum_{k=0}^\infty a_k\psi_k(\vec{r})e^{-E_k \tau}.$$

Using the Gram-Schmidt orthoganilsation procedure, higher order orthogonal eigenfunctions can be found by projecting a guess along the lower states. For example

$$\lvert\tilde{\psi^\prime_1}\rangle = \lvert\psi^\prime_1\rangle-\lvert\psi_0\rangle\langle\psi_0\vert\psi^\prime_1\rangle,$$

which can even identify eigenfunctions within a degenerate subspace, as each state is chosen to be orthogonal to the systems' basis.

UPDATE:

As of Hakyll 4.8.0.0, a complete metadata overhaul has occured to support YAML. This has broken the mathCtx function above and should be replaced by

mathCtx :: Context a
mathCtx = field "katex" $ \item -> do
    katex <- getMetadataField (itemIdentifier item) "katex"
    return $ case katex of
                    Just "false" -> ""
                    Just "off" -> ""
                    _ -> "<link rel=\"stylesheet\" href=\"/css/katex.min.css\">\n\
                             \<script type=\"text/javascript\" src=\"/js/katex.min.js\"></script>\n\
                             \<script src=\"/js/auto-render.min.js\"></script>"

Additionally, much of the gripes concerning KaTeX's math coverage are null and void as the project has matured immensely since the time I initially wrote this post.

Update 2018-04-25: OrcID have removed the v1.2 API from their servers, meaning the code in this blog post no longer functions. The current version v2.1 is quite annoying for our purposes, as it only returns a summary of works, not all info when we request details on a user. We can then collect an identifier for each work and request the full content, but can only do this in batches of 50, thus the implementation of a script like the one below is no longer quite straightforward. With that being said, the Joomla module I wrote for v1.2 has been updated to function with v2.1 and has a number of other improvements which are interesting in their own right. You can take a look at the module on my github here; the code discussed in this blog is mostly held within mod_papers/helper.php.

Looking around for the latest and greatest in CMS tools (I haven't worked with one since the very early versions of e107 15 years ago) we came across Joomla, which incidentally was pre-installed on the shared hosting we'd just purchased for the site, and started porting across content. Now that I'm familiar with the system I can undoubtedly state that building something from scratch and teaching everyone in the group how to code would have been a better choice.

But I'm getting off topic…

Managing publication lists has apparently become yet another piece of administrative overhead for researchers. One needs a ResearcherID, Scopus Author ID, ORCiD, and a google scholar profile; not to mention your institutions' profile of you and—if you give two shits about the corporate world, a LinkedIn.

A publication list is one of the primary items expected on a research groups website, and we really didn't want to add yet another list to the pile. So, against my better judgment, I delved back into the underworld of PHP and hacked together a little Joomla module that pulls the publications of multiple individuals via the ORCiD public API. That information is then cleared of duplicates, sorted by year and displayed as a list. See it in action over at TCQP.Science.

<?php
//Align most sanitised to least, will prefer data from earlier in the array.
$orcids = array('0000-0002-xxxx-xxxx','0000-0002-yyyy-yyyy','0000-0002-zzzz-zzzz');
foreach ($orcids as $id) {
    // create a new cURL resource
    $ch  = curl_init();
    // set URL and other appropriate options
    $options = array(
        CURLOPT_URL => 'http://pub.orcid.org/v1.2/' . $id . '/orcid-works',
        CURLOPT_HEADER => false,
        CURLOPT_RETURNTRANSFER => 1,
        CURLOPT_HTTPHEADER => array(
            'Accept: application/orcid+json'
        )
    );
    curl_setopt_array($ch, $options);
    // grab URL and pass it to the browser
    $raw = curl_exec($ch);
    // close cURL resource, and free up system resources
    curl_close($ch);
    //Decode json data
    $data  = json_decode($raw, true);
    //Grab usefull stuff and merge
    $works = $data['orcid-profile']['orcid-activities']['orcid-works']['orcid-work'];
    if (!empty($works)) {
        if ($id === reset($orcids)) {
            $mergedworks = $works;
        } else {
            $mergedworks = array_merge($mergedworks, $works);
        }
    }
}

//Get all dois
$dois = array();
foreach ($mergedworks as $key => $work) {
    if (!is_null($work['work-external-identifiers'])) {
        foreach ($work['work-external-identifiers']['work-external-identifier'] as $ids) {
            if (strcmp($ids['work-external-identifier-type'], 'DOI') == 0) {
                $dois[] = $ids['work-external-identifier-id']['value'];
            }
        }
        $mergedworks[$key] = array_merge($mergedworks[$key], array('parse'=>1,'udoi'=>'')); //Build a parse check field. Parse by default, set to zero if there's an issue.
    } else {
        unset($mergedworks[$key]); //For now, kill anything without a DOI.
    }
}

//Find all unique dois
$udois = array_unique($dois);

//sanitise merged array.
foreach ($mergedworks as $mkey => $work) {
    //Identify Duplicates
    foreach ($work['work-external-identifiers']['work-external-identifier'] as $ids) {
        if (strcmp($ids['work-external-identifier-type'], 'DOI') == 0) {
            $doi = $ids['work-external-identifier-id']['value'];
        $mergedworks[$mkey]['udoi'] = $doi;
            $key = array_search($doi, $udois); // Find where DOI is in the unique list

            unset($udois[$key]); //Found one, don't need another.
            if ($key === false) {
                $mergedworks[$mkey]['parse'] = 0; //Don't parse this entry
            }
        }
    }
}

//Sort array by year
usort($mergedworks, function($a, $b) {
    return ($a['publication-date']['year']['value'] > $b['publication-date']['year']['value']) ? -1 : 1;
});

$curr_year = date("Y");
$output = "<h2>" . $curr_year . "</h2>";
foreach ($mergedworks as $work) {
    //Identify Results earlier than 2011
    $year = $work['publication-date']['year']['value'];
    if ($year < '2011') {
        $work['parse'] = 0; //Don't parse this entry
    } elseif ($year < $curr_year) {
        //As our list is sorted, we've moved to the previous year now. Separate the results.
        $curr_year = $year;
        $output .= "<br><h2>" . $curr_year . "</h2>";
    }
    //Print results
    if ($work['parse'] === 1) {
        $output .= '<b>' . $work['work-title']['title']['value'] . '</b><br>';

        if (strcmp($work['work-citation']['work-citation-type'], 'BIBTEX') == 0) {
            $bibtex = $work['work-citation']['citation'];
            $volume = '';
            $pages  = '';
            if (preg_match('/volume\\s?=\\s?{(\\d+)}/', $bibtex, $match)) {
                $volume = $match[1];
            }
            if (preg_match('/pages\\s?=\\s?{([0-9-]+)}/', $bibtex, $match)) {
                $pages = $match[1];
            }
        }

        if (!is_null($work['work-contributors'])) {
            foreach ($work['work-contributors']['contributor'] as $authors) {
                if (($authors === reset($work['work-contributors']['contributor'])) && ($authors === end($work['work-contributors']['contributor']))) {
                    $output .= $authors['credit-name']['value'] . '<br>';
                } elseif ($authors === reset($work['work-contributors']['contributor'])) {
                    $output .= $authors['credit-name']['value'];
                } elseif ($authors === end($work['work-contributors']['contributor'])) {
                    $output .= ' and ' . $authors['credit-name']['value'] . '<br>';
                } else {
                    $output .= ', ' . $authors['credit-name']['value'];
                }
            }
        } else {
            //Get authorlist from bibtex
            if (preg_match('/author\\s?=\\s?{(.+)}/', $bibtex, $match)) {
                $authorstr = $match[1];
                $authors   = explode(" and ", $authorstr);
                foreach ($authors as $author) {
                    if (($author === reset($authors)) && ($author === end($authors))) {
                        $output .= $author . '<br>';
                    } elseif ($author === reset($authors)) {
                        $output .= $author;
                    } elseif ($author === end($authors)) {
                        $output .= ' and ' . $author . '<br>';
                    } else {
                        $output .= ', ' . $author;
                    }
                }
            }
        }

        $output .= '<a href="http://dx.doi.org/' . $work['udoi'] . '">' . $work['journal-title']['value'] . ' <b>' . $volume . '</b> ' . $pages . ' (' . $work['publication-date']['year']['value'] . ')</a><br>';
    }
}

echo $output;
?>

If you want to use this yourself, feel free to copypasta, or get in contact and I'll send you a complete module for Joomla. A few things you should be aware of: put the person with the cleanest list first in the array—the script doesn't merge fields it just takes the first it finds when checking against the unique key. Secondly, we don't expect a drastic amount of traffic and would like the list to be as up to date as possible, so we pull this info on each page load. If you're expecting modest traffic, perhaps look into something a little more sophisticated for the API call.

What hasn't been that nice about it are the few binary dependencies I keep forgetting to track or that nasty little bug that xelatex has wherein it just utterly fails when you try and draw a gradient or any form of shading. To overcome this issue I've been running a two step compile process, where figures that require shading are built using pdflatex, then if text is required (for labels or annotations for example), xelatex is called. This allows the figure font to remain homologous with the main text and the pre-rendered shaded geometry can be simply imported with some form of \includegraphics{} command.

However the figure count is now over 50 and climbing, so building all of them on a new machine or propagating broad changes has become a cumbersome task. Ever since writing my first Makefile, I've enjoyed making simple ones for menial tasks. Nothing fancy, but useful enough to take the time and understand the syntax just that little bit more. Building these figures was an obvious case for some new Makefile magic.

Here's what I came up with on a train to Bendigo with two random obnoxious children crawling over me:

    #All files to be built with pdflatex
    PSOURCES := aluminium.tex emegir.tex flourish.tex jjschematic.tex oxygen.tex \\
                                oxygenb.tex qabuum.tex sf.tex sio2.tex
    #All files to be built with xelatex
    XSOURCES := $(filter-out $(PSOURCES), $(wildcard *.tex))
    PTARGETS := $(PSOURCES:.tex=.pdf)
    XTARGETS := $(XSOURCES:.tex=.pdf)
    PDFLATEX := pdflatex --shell-escape --extra-mem-top=10000000 --save-size=80000
    XELATEX := xelatex --shell-escape --extra-mem-top=10000000 --save-size=80000

    .PHONY: all clear rebuild clean distclean

    all: xelatex

    pdflatex: $(PTARGETS)

    xelatex: $(XTARGETS)

    %.pdf: %.tex
            @[ '$<' == '$(findstring $<,$(PSOURCES))' ] && $(PDFLATEX) $< || $(XELATEX) $<
            @[ '$<' == '$(findstring $<,$(PSOURCES))' ] && $(PDFLATEX) $< || $(XELATEX) $<

    #Dependencies
    $(XTARGETS): AGS.pdf BGS.pdf $(PTARGETS)
    $(PTARGETS): sf.tikz

    rebuild: clean all

    clear:
            @-rm -f *.aux *.log *.dat

    clean: clear
            @-rm -f $(PTARGETS) $(XTARGETS)

    distclean: clean

There are a few interesting things in here that weren't obvious to me at first. For instance the wildcard property allows a list of some input file type to be made into an output file of the same name. The target %.pdf: %.tex says that all output pdf files will require a corresponding tex file. With my particular case this isn't very helpful as I'd like only some tex files rendered to pdf using one method and the rest rendered with a completely different method. Another question I had was "What's the best way to separate the pdflatex and xelatex runs?"

The second issue could have just been two lists that would need to be updated every time a new figure was added, but that seemed to be a little inefficient and defeat the purpose of the makefiles' automation. The solution comes from this line:

    $(filter-out $(PSOURCES), $(wildcard *.tex))

which takes the list $(wildcard *.tex) and filters (removes) any matches from $(PSOURCES). The wildcard command is necessary here to identify this list as iterable for the %.tex target later on. As the sources for the pdflatex run are far fewer than those needing the default xelatex treatment, it's much easier to filter the exceptions at this juncture. Now all new files that don't require special treatment also don't require any thought when building them.

Separating the two build types became a bit more of a hassle. Initially, I'd implemented something like

    ifeq ($<,$(findstring $<,$(PSOURCES)))
        $(PDFLATEX) $<
        $(PDFLATEX) $<
    else
        $(XELATEX) $<
        $(XELATEX) $<
    endif

for the %.pdf: %.tex target.

    $(findstring $<,$(PSOURCES))

is tasked to find the current tex file during the expansion of the %.tex wildcard ($<) in the $(PSOURCES) list. If the file isn't in the list this function will return an empty string and the ifeq ($<, "") test will return false. But this never worked and all files ended up trying to compile with pdflatex. Without internet and these kids still crawling over me, I gave up in disgust and stared out the window for the rest of the trip.

Later, back in civilisation, I pulled up the make documentation and found out why this method was bound to fail: make evaluates conditionals when it reads a makefile

A conditional causes part of a makefile to be obeyed or ignored depending on the values of variables. Conditionals can compare the value of one variable to another, or the value of a variable to a constant string. Conditionals control what make actually "sees" in the makefile, so they cannot be used to control shell commands at the time of execution.

In other words, a first pass of the file will test the ifeq condition, then choose that option for the second (and final) pass. When the second pass meets the wildcard expansion it just inserts whatever commands the first pass sends it and that's all she wrote. So the solution I required here was to export the problem to the shell each wildcard expansion, test the name of the current .tex file and only then invoke the correct compile tool.

All in all, this file does quite a good job. I haven't seen many people discussing their figure build chain online, so if you have an interesting way of making your figures get in contact—I'd be interested to know what you do.

You see, the manual states the TV will play a .S4ud file: essentially a H.264 video and an AC3 audio stream multiplexed into a TS container. No firmware update since the TVs manufacture has actually allowed this. A second possibility is to use QFHD—four separate HD streams that are then joined together to yield a 4K stream. After quite a conversation with Samsung tech support it was elucidated that this is actually happening in-store when you see these TVs in action with those amazing demos.

So off I went and cut up a few sample files and tested it out—no dice. Two of the four segments were playing, the others were a jaggered mess. After more back and forth with the guys in tech, turns out the demo capability had been broken somewhere in an early firmware patch that was propagated through a number of subsequent versions. Thankfully, this debacle has been fixed and I'm now happily viewing my 4K content in all its glory. Downside is it's still QFHD, not a nice single TS container.

There's also a number of caveats with this method, for instance the files must be named UHD_demo_{a,b,c,d}_*.mp4{sub}, where a is the main file (top left), and the b-d files are sub streams.

Long story short, I've developed a few scripts to automatically take care of all of these problems. They could use a little cleaning up, and wifi connectivity problems at my new house has seen me jump back to Windows recently—so these are batch files rather than bash scripts. Shouldn't be too hard to port them yourself though.

UHD 4K is 3840×2160 pixels, although Digital cinema 4K is 4096×2304. Before the resolution standards settled down, a plethora of content out there was letterboxed in some way, but 4096 pixels wide. Considering this, there are three scripts below for each of these three possibilities. Granted, I could set this up to find the resolution first but brevior saltare cum deformibus mulieribus est vita amiright? Call these snippets conv.bat or something to that effect, and run conv inputFile.mp4 (for example) at the CLI. The output will be four files titled with the convention above, where * will be inputFile (to keep things in order, these files output to a folder qfhd and the converted file is archived to a converted folder). Then, you'll need to put them on a fast USB3 stick and plug it into the TV. You'll only see the a file, run it and actually enjoy that massive hole you previously burnt into your wallet.

Ffmpeg is the only dependency you'll need. Alter the rate factor in the script (-crf) to your liking; but I've found 15 works just fine.

3840×2160

    @echo off
    set input=%1
    set name=%input:~0,-4%
    echo Processing %input%
    echo Top left corner and sound
    ffmpeg -i %input% -vcodec libx264 -crf 15 -acodec ac3 -vf "crop=iw/2:ih/2:0:0" a.mp4
    move a.mp4 qfhd\UHD_demo_a_%name%.mp4
    echo Bottom left corner
    ffmpeg -i %input% -vcodec libx264 -crf 15 -an -vf "crop=iw/2:ih/2:0:ih/2" b.mp4
    move b.mp4 qfhd\UHD_demo_b_%name%.mp4sub
    echo Top right corner
    ffmpeg -i %input% -vcodec libx264 -crf 15 -an -vf "crop=iw/2:ih/2:iw/2:0" c.mp4
    move c.mp4 qfhd\UHD_demo_c_%name%.mp4sub
    echo Top right corner
    ffmpeg -i %input% -vcodec libx264 -crf 15 -an -vf "crop=iw/2:ih/2:iw/2:ih/2" d.mp4
    move d.mp4 qfhd\UHD_demo_d_%name%.mp4sub
    move %input% converted\
    echo Processing of %input% Complete
    @echo on

4096×2304

    @echo off
    set input=%1
    set name=%input:~0,-4%
    echo Processing %input%
    echo Top left corner and sound
    ffmpeg -i %input% -vcodec libx264 -crf 15 -acodec ac3 -vf "scale=3840:2160,crop=iw/2:ih/2:0:0" a.mp4
    move a.mp4 qfhd\UHD_demo_a_%name%.mp4
    echo Bottom left corner
    ffmpeg -i %input% -vcodec libx264 -crf 15 -an -vf "scale=3840:2160,crop=iw/2:ih/2:0:ih/2" b.mp4
    move b.mp4 qfhd\UHD_demo_b_%name%.mp4sub
    echo Top right corner
    ffmpeg -i %input% -vcodec libx264 -crf 15 -an -vf "scale=3840:2160,crop=iw/2:ih/2:iw/2:0" c.mp4
    move c.mp4 qfhd\UHD_demo_c_%name%.mp4sub
    echo Top right corner
    ffmpeg -i %input% -vcodec libx264 -crf 15 -an -vf "scale=3840:2160,crop=iw/2:ih/2:iw/2:ih/2" d.mp4
    move d.mp4 qfhd\UHD_demo_d_%name%.mp4sub
    move %input% converted\
    echo Processing of %input% Complete
    @echo on

4096×anything

    @echo off
    set input=%1
    set name=%input:~0,-4%
    echo Processing %input%
    echo Top left corner and sound
    ffmpeg -i %input% -vcodec libx264 -crf 15 -acodec ac3 -vf "scale=3840:-1,pad=3840:2160:0:(2160-ih)/2,crop=iw/2:ih/2:0:0" a.mp4
    move a.mp4 qfhd\UHD_demo_a_%name%.mp4
    echo Bottom left corner
    ffmpeg -i %input% -vcodec libx264 -crf 15 -an -vf "scale=3840:-1,pad=3840:2160:0:(2160-ih)/2,crop=iw/2:ih/2:0:ih/2" b.mp4
    move b.mp4 qfhd\UHD_demo_b_%name%.mp4sub
    echo Top right corner
    ffmpeg -i %input% -vcodec libx264 -crf 15 -an -vf "scale=3840:-1,pad=3840:2160:0:(2160-ih)/2,crop=iw/2:ih/2:iw/2:0" c.mp4
    move c.mp4 qfhd\UHD_demo_c_%name%.mp4sub
    echo Top right corner
    ffmpeg -i %input% -vcodec libx264 -crf 15 -an -vf "scale=3840:-1,pad=3840:2160:0:(2160-ih)/2,crop=iw/2:ih/2:iw/2:ih/2" d.mp4
    move d.mp4 qfhd\UHD_demo_d_%name%.mp4sub
    move %input% converted\
    echo Processing of %input% Complete
    @echo on

There colortbl package (available on CTAN), with albeit hideous examples, allows for the inclusion of colour into tables; usually to highlight cells, but with a bit of finesse can be persuaded to alter the rule colour also.

Memoir is by far the most comprehensive class I've come across that should be the default document choice for those wishing to typeset large and complex documents. Simply adding colortbl into the preamble of ones document will allow access to the capability needed to obtain coloured table rules, but this is not the only option available. Memoir incorporates the extensions from booktabs, which provides much needed enhancements to the default table environment. Since booktabs has some rudimentary support for colortbl, Memoir already has latent colour capability from the ported legacy code—it just needs to be enabled.

Add this to your preamble:

    \makeatletter
    \def\rulecolor#1#{\CT@arc{#1}}
    \def\CT@arc#1#2{%
      \ifdim\baselineskip=\z@\noalign\fi
      {\gdef\CT@arc@{\color#1{#2}}}}
    \let\CT@arc@\relax
    \makeatother

And set table rules via \rulecolor{color}. With a little bit of tweaking, your tables will scrub up quite well.

]]> https://axiomatic.neophilus.net/coloured-tables-for-the-memoir-class/ Tim DuBois LaTeX https://axiomatic.neophilus.net/easy-matlabpool-configuration/ Thu, 13 Nov 2014 00:00:00 +0000 The new parpool option for Matlab 2014b is depreciating matlabpool, merging the ideas of parallelisation locally and remotely in to one. Now, the same code can be easily sent off to a large, remote cluster or ran over a few more cores on your own machine without much hassle. Another useful feature is the fact that idle pools will close down after inactivity, allowing you to be lazy with your implementations, while at the same time giving the possibility of efficiency if you require the pool again inadvertently.

A caveat that I find with this implementation is the freedom to alter the pool size dynamically. Most may not find this an issue, and are happy with setting the pool to 8 and be done; but if writing large amounts of info to disk constantly is your jam this is seldom the optimal choice. At least for my routines, writing to platter isn't efficient above 2 threads, and to flash disks, 6 appears optimal. Also: what if you want some cycles free to actually operate your machine whilst its actively causing the heat death?

    function pool(num)
    p = gcp('nocreate'); % If no pool, do not create new one.
    if isempty(p)
        %poolsize = 0;
        parpool(num);
    else
        poolsize = p.NumWorkers;
        if poolsize ~= num
            delete(gcp);
            parpool(num);
        end
    end
    end

This function asks for number of pools you wish to be created. If a pool of that size is currently active, nothing is required of the function and your code will continue. Otherwise, the old pool is shut down and a new one is generated. Personally, I'd like a future release to enable user access to set the read only attribute p.NumWorkers, but I guess we all can't get everything we want in life.

It's been two weeks or so since the switch and I'm at the proficiency stage where I'm not yelling at the awkwardness of my inability to find a letter, but if the switch from Qwerty to Dvorak is any indication it'll be a couple of months until I'm completely up to speed.

I had a fantastic Vim map for Dvorak suggested by Adam Davis, which kept the Qwerty h,j,k,l navigation keys in the same place; remapping sane Dvorak keys with minimal disruption. So obviously something similar for Workman that didn't disrupt Vim flow is needed.

Taking the laziest approach first; google tells me there are only two current suggestions. First, Matt Weolk has taken the complete remap Qwerty:Workman approach, which is outlined in this gist and takes the idea from colqer; a Colemak solution to the same issue. I really don't like the blind approach this method uses as I remember Vim keybindings more on their associations (y = yank), than muscle memory of their original Qwerty locations. The second solution is a simple j ↔ t switch discussed in this thread. Closer to what I'm after, and I used it for a while; but ultimately the navigation keys being separated and not entirely on the home row doesn't sit with me.

So, here's my attempt at a decent Workman remapping for Vim:

    nnoremap l o
    nnoremap o l
    nnoremap L O
    nnoremap O L
    nnoremap j n
    nnoremap n j
    nnoremap J N
    nnoremap N J
    nnoremap gn gj
    nnoremap gj gn
    nnoremap k e
    nnoremap e k
    nnoremap K E
    nnoremap E <nop>
    nnoremap gk ge
    nnoremap ge gk
    nnoremap h y
    onoremap h y
    nnoremap y h
    nnoremap H Y
    nnoremap Y H

h,j,k,l are replaced by the Workman y,n,e,o home keys, with a few new associations:

• (Y)ank -> (H)aul
• Search (N)ext -> (J)ump
• (E)nd word -> brea(K) of word [yeah, that one's a push...]
• (O)pen new line -> (L)ine

Considering I now use three different layouts depending on where I am, I've had to set up a layout remap function in my .vimrc. Here it is in it's present state at the time of writing; check my dotfiles repository for updates though.

    " Keyboard  **************************
    function Keyboard(type)
       if a:type == "dvorak"
           call UnmapWorkman()
           nnoremap d h
           nnoremap h j
           nnoremap t k
           nnoremap n l
           nnoremap s :
           nnoremap S :
           nnoremap j d
           onoremap j d
           nnoremap l n
           nnoremap L N

           nnoremap - $
           nnoremap _ ^
           nnoremap N <C-w><C-w>
           nnoremap H 8<Down>
           nnoremap T 8<Up>
           nnoremap D <C-w><C-r>
       elseif a:type == "workman"
           call UnmapDvorak()
           "(O)pen line -> (L)ine
           nnoremap l o
           nnoremap o l
           nnoremap L O
           nnoremap O L
           "Search (N)ext -> (J)ump
           nnoremap j n
           nnoremap n j
           nnoremap J N
           nnoremap N J
           nnoremap gn gj
           nnoremap gj gn
           "(E)nd of word -> brea(K) of word
           nnoremap k e
           nnoremap e k
           nnoremap K E
           nnoremap E <nop>
           nnoremap gk ge
           nnoremap ge gk
           "(Y)ank -> (H)aul
           nnoremap h y
           onoremap h y
           nnoremap y h
           nnoremap H Y
           nnoremap Y H
       else " qwerty
           call UnmapDvorak()
           call UnmapWorkman()
       endif
    endfunction

    function UnmapDvorak()
        "Unmaps Dvorak keys
        silent! nunmap d
        silent! nunmap h
        silent! nunmap t
        silent! nunmap n
        silent! nunmap s
        silent! nunmap S
        silent! nunmap j
        silent! ounmap j
        silent! nunmap l
        silent! nunmap L

        silent! nunmap -
        silent! nunmap _
        silent! nunmap N
        silent! nunmap H
        silent! nunmap T
        silent! nunmap D
    endfunction

    function UnmapWorkman()
        "Unmaps Workman keys
        silent! nunmap h
        silent! ounmap h
        silent! nunmap j
        silent! nunmap k
        silent! nunmap l
        silent! nunmap y
        silent! nunmap n
        silent! nunmap e
        silent! nunmap o
        silent! nunmap H
        silent! nunmap J
        silent! nunmap K
        silent! nunmap L
        silent! nunmap Y
        silent! nunmap N
        silent! nunmap E
        silent! nunmap O
    endfunction

    function LoadKeyboard()
       let keys = $keyboard
       if (keys == "workman")
           call Keyboard("workman")
       else
           call Keyboard("dvorak")
       endif
    endfunction

    autocmd VimEnter * call LoadKeyboard()

    :noremap <Leader>q :call Keyboard("qwerty")<CR>:echom "Qwerty Keyboard Layout"<CR>
    :noremap <Leader>d :call Keyboard("dvorak")<CR>:echom "Dvorak Keyboard Layout"<CR>
    :noremap <Leader>w :call Keyboard("workman")<CR>:echom "Workman Keyboard Layout"<CR>

Ultimately though, my c++ programs never needed to link to external libraries or worry about machine specific configurations the -o switch was the only one I needed when calling gcc pretty much. Now I'm building MPI tools to run on supercomputing clusters that need the highly optimised linear algebra routines; written down by our forefathers in a more civilised age.

I need a Makefile, the file filled with dark arts known only to those with neck beards and ghostly white skin.

Realistically, Makefiles are relatively simple things, but seem to have a stigma associated with them if you're outside the computer science sphere. In fact; here's a quote from my PhD supervisor when I told him about my knowledge gain concerning this post:

Hehe, careful. Those that learn how to write makefiles are usually doomed to vanish … banished to a basement (or IT department of a fortune 500 company) for all eternity.

I guess writing this post and publishing it on the internet is sealing my fate…

The my first Makefile tutorials around the internet are not too bad (take a look at WLUG and Mrbook to get started); but the Black Magic I eluded to in the title of this post is much cooler than just typing make instead of g++ main.cpp interrobang.cpp -o omgwtfbbq.

🔗 Pre-processor macros

The specific problem I needed to overcome was managing one set of code that requires different linking libraries depending on what machine it was running on.

• Vayu uses intel compilers and requires the MKL libraries
• Trifid uses gcc compilers and requires the blas and lapack libraries

Because of these conditions, code in certain files differ. For example, calls to linear algebra routines on Vayu require an MKL_int type, whereas the same call on Trifid asks for int. A pre-processor macro defining a generalised int type LP_INT enables me to overcome this problem. This macro uses an if-elif-else formalism to check what machine the code is compiling on and adds additional headers if needed:

    #if defined(VAYU)

    #include <mkl>
    #define LP_INT MKL_INT

    #elif defined(TRIFID)

    #define LP_INT int

    #else

    #error "One of VAYU or TRIFID must be defined"

    #endif

Now, how can we define these VAYU and TRIFID variables? SUMMON THE MAKEFILE:

    HOST_NAME := $(shell hostname)
    ifeq ($(HOST_NAME),trifid)
    LAPACK = /usr/local/lapack/3.4.2/lib/liblapack.so /usr/local/blas/1.0.248/lib/libblas.so -lm
    LIBS = -L/usr/local/lapack/3.4.2/lib
    CXX = g++ #icpc
    CPPFLAGS += -DTRIFID=1
    else
    INCLUDES += -I$(MKL)/include
    LAPACK = -lmkl_intel_lp64 -lmkl_intel_thread -lmkl_core -liomp5 -lpthread
    LIBS = -L$(MKL)/lib/intel64
    CPPFLAGS += -DVAYU=1
    endif

Grab the hostname of the machine & check it against known results (in my case I just check for trifid). Any shell call can be used here if hostname isn't appropriate. Then, setup the required libraries, includes and compiler information specific to the identified machine. Most importantly: append CPPFLAGS to incorperate a machine bool set to 1 which the pre-processor macros are looking for.

Et voilà! Call make on either machine and build without a hassle. No more merge conflicts between git branches for me. A shoutout to Ash who put me on the right path with this issue.

Building static diagrams and graphs (adding PGFPlots into the mix) is fine, but I find myself constantly wanting decent plots from real data, that don't fit the usual line/surface paradigm. The datatool package is perfect for this kind of work.

Something I'm working on currently is the classification of voids in amorphous solids, voronoi networks seem to be a great way of expressing the arrangement of atoms in these systems. The following example uses an amorphous aluminium oxide and is represented in 2D so as not to complicate the problem too much.

To simplify things further, I've separated my input data into three csv files that look something like this:

1,Al,10.331817,36.323166,1.250000
2,Al,5.092404,21.862188,1.250000
3,Al,2.332884,23.765770,1.250000
4,Al,13.550237,30.676344,1.250000
5,Al,0.676694,28.555687,1.250000
6,Al,14.927469,33.200344,1.250000
7,Al,4.266311,33.304264,1.250000
8,Al,7.634949,33.187614,1.250000
...

The datatool package reads this information in through its load database command \DTLloaddb

\DTLloaddb[noheader,keys={idx,species,x,y,r}]{data}{data2Dtest.csv}

pulling the file into the data variable, and assigning keys to each column. Now using a foreach command to loop over all rows in data,

\DTLforeach*{data}{\idx=idx, \species=species, \x=x, \y=y, \r=r}

I can draw a circle of radius \r at position (\x,\y); as well as color each circle depending on its associated \species key [lines 23--30 in the full code below].

Two other functions of datatool that I use in this example are extremely useful:

\DTLcomputebounds{data}{x}{y}{\minX}{\maxX}{\minY}{\maxY}

computes the bounds of the (x,y) data, which I use to draw a bounding box;

\DTLgetvalueforkey{\oneX}{x}{data}{idx}{\one}

grabs the location of x from data at idx, where that index value equals \one from another data set.

If you include all of this with some TikZ trickery, it's fairly simple to generate a number of figures like this incredibly fast with a myriad of different data sets.

The entire code-set for this project is below. The in-line comments expand on the syntax I outline above and should answer most questions you may have about each functions purpose.

% generates a 2D voronoi diagram from input files in tikz
\documentclass{article}
\pagestyle{empty}
\usepackage{tikz}
\usepackage{datatool}
\usepackage{graphicx}
\DTLloaddb[noheader,keys={idx,species,x,y,r}]{data}{data2Dtest.csv} %CSV data
\DTLloaddb[noheader,keys={one,two,three}]{tri}{tri2Dtest.csv} %delaunay triangulation data
\DTLloaddb[noheader,keys={vxx,vxy,vyx,vyy}]{vor}{vor2Dtest.csv} %voronoi data
\usetikzlibrary{calc,fadings}
%To save output, uncomment next three lines (Third only to force recompile).
%Then execute with pdflatex -shell-escape voronoi2D.tex
%\usetikzlibrary{external}
%\tikzexternalize % activate!
%\tikzset{external/force remake}

\begin{document}
%\begin{figure}
%\resizebox{\textwidth}{!}{

\begin{tikzpicture}[scale=0.5]
%Draw Atoms
\DTLforeach*{data}{\idx=idx, \species=species, \x=x, \y=y, \r=r}{
    %Choose color based on species, draw circle at current point
    \ifthenelse{\pdfstrcmp{\species}{Al}=0}{
    \shade[ball color=gray] (\x,\y) circle(0.4*\r);
    }{
    \shade[ball color=red] (\x,\y) circle(0.4*\r);
    }
}

%Draw Delaunay Simlpexes
\DTLforeach*{tri}{\one=one, \two=two, \three=three}{
    %One, two and three are indexes of the corners of each D triangle
    %Get locations from \x & \y in data db for each index
    \DTLgetvalueforkey{\oneX}{x}{data}{idx}{\one}
    \DTLgetvalueforkey{\oneY}{y}{data}{idx}{\one}
    \DTLgetvalueforkey{\twoX}{x}{data}{idx}{\two}
    \DTLgetvalueforkey{\twoY}{y}{data}{idx}{\two}
    \DTLgetvalueforkey{\threeX}{x}{data}{idx}{\three}
    \DTLgetvalueforkey{\threeY}{y}{data}{idx}{\three}
    %Draw triangle
    \draw (\oneX,\oneY) -- (\twoX,\twoY) -- (\threeX,\threeY) -- cycle;
}

%Draw Bounding Box
%Grab max/min of coords
\DTLcomputebounds{data}{x}{y}{\minX}{\maxX}{\minY}{\maxY}
%Draw Box
\draw[gray, thick] ({\minX-1},{\maxX-1}) rectangle ({\minY+1},{\maxY+1});
%Set clipping path for voronoi data
\clip ({\minX-1},{\maxX-1}) rectangle ({\minY+1},{\maxY+1});

%Draw Voronoi cells
\DTLforeach*{vor}{\vxx=vxx, \vxy=vxy, \vyx=vyx, \vyy=vyy}{
    %Draw line
    \draw[blue,dashed] (\vxx,\vyx) -- (\vxy,\vyy);
}
\end{tikzpicture}
%\end{figure}

\end{document}