Automatically Setup New R and LaTeX Projects

You have a finite amount of keystrokes in your life. Automating repetitive steps is one of the great benefits of knowing a bit about coding, even if the code is just a simple shell script. That’s why I set up an example project on Github using a Makefile and sample article (inspired by Rob Hyndman). This post explains the structure of that project and explains how to modify it for your own purposes.

Running the Makefile with the example article in an otherwise-empty project directory will create:

  • a setup.R file for clearing the workspace, setting paths, and loading libraries
  • a data directory for storing files (csv, rda, etc)
  • a drafts directory for LaTeX, including a generic starter article
  • a graphics library for storing plots and figures to include in your article
  • an rcode directory for your R scripts

It also supplies some starter code for the setup.R file in the main directory and a start.R file in the rcode directory. This takes the current user and sets relative paths to the project directories with simple variable references in R. For example, after running setup.R in your analysis file you can switch to the data directory with setwd(pathData), then create a plot and save it after running setwd(pathGraphics). Because of the way the setup.R file works, you could have multiple users working on the same project and not need to change any of the other scripts.

If you want to change this structure, there are two main ways to do it. You can add more (or fewer) directories by modifying the mkdir lines in the Makefile. You can add (or remove) text in the R files by changing the echo lines.

If you decide to make major customizations to this file or already have your own structure for projects like this, leave a link in the comments for other readers.

H/T to Josh Cutler for encouraging me to write a post about my project structure and automation efforts. As a final note, this is the second New Year’s Eve in a row that I have posted a tech tip. Maybe it will become a YSPR tradition!

Update: If you’re interested in your own automated project setup, check out ProjectTemplate by John Myles White. Thank to Trey Causey on Twitter and Zachary Jones in the comments for sharing this.

Punctuation Politics – The Curious Case of the Apostrophe

Apostrophe-manSome of the context is unfamiliar to me as a non-UK reader, but Michael Rosen makes an interesting argument that there is a politics of punctuation:

My position is that the apostrophe is on the way out. It’s an inconsistent item anyway; it was invented by printers – not grammarians or linguists – and like a lot of other ‘rules’ of punctuation is modified by use. No bad thing.

Like many other norms in everyday life, punctuation emerges through patterns of common usage. Declarations of these rules like the AP Guide to Punctuation and Strunk & White codify common patterns, but are not the source for them. Rosen uses the inconsistency of the rules to show how they have shifted and evolved over time:

We say as a general ‘rule’ that we use an apostrophe for ‘elision’ (when we leave stuff out) and for possessives (when we want to indicate that someone or something owns someone or something). So when we write ‘haven’t’ – that’s supposed to show we’ve ‘left out’ the ‘o’ of not. When we write ‘Michael’s writing’ that’s supposed to show that the writing is possessed by Michael. He owns it. So far so good.

But is all possession marked with an apostrophe? Oh no. So if we use what have been called the ‘possessive pronouns’, its, his, hers, yours, ours, theirs – no apostrophe! Why not? er…well, no one really knows.Look at eighteenth century texts and you will find phrases like, let’s say, ‘the lands were her’s’. Even Mr Strict, Bishop Lowth, the inventor of crap grammar, used an apostrophe there. So, if it was a ‘rule’ then, when did it become a ‘rule’ to not use an apostrophe in, ‘yours’ or ‘ours’? Answer, it’s only a ‘rule’ if you’re the kind of person who thinks this sort of stuff is a ‘rule’ and not, what I would call a ‘convention’.

Rosen discusses apostrophes for elision–it’s, haven’t, they’re–in some detail. Two other use cases also point out historical irregularities:

By the way that complicated stuff about plural possessives ‘the boys’ caps’ – meaning two or more boys’ caps only became a ‘rule’ in the nineteenth century. Up until then, people like Jane Austen and Daniel Defoe managed to get by without worrying about it.

And in case you’re wondering if the decade was the 60’s or the 60s, the answer is, it all depends on the house style of the whoever is publishing it. Again, it’s a trade matter, not a grammatical one of rules.

And if you’re wondering why the possessive apostrophe came in in the first place? Because most nouns used to express possession with an ending ‘es’ with the ‘e’ sounded out. ‘dogges ears’ – with the ‘e’ heard. That sounded out ‘e’ started to disappear just as the first efforts to standardise orthography came in so with the ‘elision rule’ they reckoned that they ought to mark the ‘loss’ of the sounded ‘e’. So it wasn’t a rule of possession after all! It was the old elision ‘rule’. So when you hear people say that the apostrophe is for ‘possession’ as I did all through this article, I was talking nonsense. It was the ‘rule’ of elision but as with vast amounts of so-called grammar and information about language, we believe in the necessity of lying to children – or just foisting our ignorance on to them. That’s because the old idea of ‘investigating language’ rather than laying down the rules has gone out the window.

When we focus on rules as they exist in the present without regard to their historical record, they seem fixed rather than emergent. But the near future may show us how short-sighted this is. Rosen argues that conventional usage of the apostrophe is slipping away as we generate more and more text at a rapid pace in the form of emails, text messages, and the like. We are already beginning to see a sort of double standard for formal and informal written language. The difference is not yet as stark as between, say, colloquial dialects and Modern Standard Arabic (the codified version used by college-educated professionals and journalists) but it is growing. There you have it–political change at your fingertips.

The Politics of Chain Stores: Where to Go for Coffee When Traveling?

25.-coffee-brandsThere are numerous political decisions related to your coffee buying decisions. Heuristics range from the pragmatic (where is closest? what’s cheapest?) to the philosophical (is it free trade/fair trade?). Those are subjects for later posts, though. Over the next week many Americans will be traveling for the holiday season. Whether on the road, in the airport, or in another town visiting relatives, your favorite source of caffeine may not be available.

In these circumstances you are likely to turn to familiar well-known brands like Starbucks. Like the comfort of an embassy in a foreign country, it is a sign of familiarity and predictability. Our discussion has so far been largely hypothetical, but it turns out there is some hard evidence for these patterns:

The researchers [Shigehiro Oishi, Felicity Miao, Minkyung Koo, Jason Kisling, and Kate Ratliff] obtained the number of outlets of a variety of chain stores (ranging from Home Depot to Kay Jewelers) from annual reports filed by the companies. Data on how often people move around was obtained from the 2000 census. Using the statistical technique of multiple regression, the researchers looked at the combined influence of residential mobility (how much people move around), median income, and population of each state as predictors of the number of outlets for these chains. They found that the amount of mobility really did predict the number of outlets of chains in a given state. Population also affected this number (not surprisingly). Median income of families was not a predictor of the number of outlets of chain stores.

You can find the paper here. An post at Psychology Today has references to further studies on the topic.

What Can Economics Teach Us About Gift-Giving?

santa-12-20-11-lgFrom George Loewenstein and Cass Sunstein writing at The New Republic:

Behavioral research on perspective-taking provides some explanations for why gift-buying is often such a waste of money. When people try to predict how another person will respond to a certain situation, they begin by imagining how they themselves would respond, and then they make adjustments for differences between themselves and the other person. At both stages, they make big mistakes….

Luckily, behavioral economics provides some straightforward lessons for gift-givers. Don’t assume that other people like what you like. Beware of projecting your current mood onto your purchasing decisions. Avoid unrealistic optimism: People probably won’t react as enthusiastically as you expect. Focus on gifts that will get frequent use, rather than immediate applause, only to disappear into holiday-season purgatory. And unless you are dealing with people very close to you, don’t assume that the gift will matter a whole lot. You’ll probably remember it better than they will.

US Defends Internet Freedom at WCIT

Photo via flickr user ictupictures

Photo via flickr user ictupictures

An update on the WCIT, which we have talked about here before. From Forbes last week:

For the last two weeks some of the planet’s most oppressive regimes have faced off against some of the most powerful Internet advocates in an effort to rewrite a multilateral communications treaty that, if successful, could have changed the nature of the Internet and altered the way it is governed.

The US delegation led the refusal to sign updates, blocking encroachment on internet freedom:

The United States, which framed its dissent as defending “the open Internet,” was joined by more than 80 other countries, including Australia, Canada, Chile, Costa Rica, the Czech Republic, Denmark, Egypt, Finland, Greece, Italy, Japan, Kenya, the Netherlands, New Zealand, Poland, Portugal, Qatar, Sweden and the United Kingdom. (Some of the non-signers seemed to be seeking to avoid making too overt of a political statement, saying, regrettably that they could not sign because they had to “consult with capital.”)

The WCIT is certainly not the final word on the matter, however. It will be interesting to watch how the international debate on internet politics proceeds:

The Internet, and the forces that support the free and open movement of information rolled over traditional UN alliances at the WCIT. An effort to shift governance of the Internet from private bodies like ICANN and IETF was thwarted. The conference did not mark the end of the battle to control what has emerged as the world’s most powerful communications medium. But it very likely marked a turning point.

Update: See Eli Dourado’s “Behind Closed Doors” take on the meeting

DTO Leadership Targeting Increased Violence, Multiplied Cartels

mexico-troopsNow that President Felipe Calderon is out of office, the new administration is attributing the rise in violence to his hardline policies:

With the capture of dozens of drug capos, an achievement trumpeted by Calderon, “we have moved from a scheme of vertical leadership to a horizontal one that has made them more violent and much more dangerous,” Osorio told the heads of the military and the governors of Mexico’s 31 states…

Calderon repeatedly said before leaving office that his forces had captured 25 of Mexico’s 37 most-wanted drug lords, a strategy backed by the U.S. government with hundreds of millions in funding and close cooperation with American law-enforcement, military and intelligence agencies.

Osorio Chong and President Enrique Pena Nieto have promised to adjust Calderon’s strategy in order to move away from a focus on killing and capturing cartel leaders and toward a focus on reducing crimes against ordinary citizens, most important homicides, kidnappings and extortion.

During Calderon’s term Mexico witnessed over 50,000 murders and 25 high-level drug trafficking organization (DTO) leaders were arrested or killed. In the latest draft of a working paper I estimate that this policy was responsible for about 900 additional murders, although this estimate is on the conservative side.

Another new administration official, Attorney General Jesus Murillo Karam blamed the leadership targeting strategy for fracturing the cartel:

Murillo Karam told MVS Radio that officials are working to identify all the country’s 60-80 small- and mid-size drug trafficking organizations. In its last public evaluation of the strength of Mexico’s cartels, the Calderon administration issued an August report naming only eight large drug organizations. It had, however, said that at least one cartel, the Beltran Leyva group, had split into smaller fragments after a government offensive that killed its leader.

Murillo elaborated on the new administration’s critique of the Calderon strategy, holding it directly responsible for a rise in kidnappings and related crimes over the last six years.

“It led to the seconds-in-command, generally the most violent, the most capable of killing … starting to be empowered and generating their own groups, generating another type of crime – spawning kidnapping, extortion and protection rackets,” he said.

Dan Trombly points out that more but fewer DTOs may allow the new administration to treat counter-narcotics as a law enforcement issue rather than a military problem. Although this development occurred after the period examined in my paper, it is an interesting development that deserves further study.

Micro-Institutions Everywhere: Species and Regime Types

geeks_evolveIn a two-for-one example of micro-institutions, Jay Ulfelder blogs this paragraph from a paper Ian Lustick:

One might naively imagine that Darwin’s theory of the “origin of species” to be “only” about animals and plants, not human affairs, and therefore presume its irrelevance for politics. But what are species? The reason Darwin’s classic is entitled Origin of Species and not Origin of the Species is because his argument contradicted the essentialist belief that a specific, finite, and unchanging set of categories of kinds had been primordially established. Instead, the theory contends, “species” are analytic categories invented by observers to correspond with stabilized patterns of exhibited characteristics. They are no different in ontological status than “varieties” within them, which are always candidates for being reclassified as species. These categories are, in essence, institutionalized ways of imagining the world. They are institutionalizations of difference that, although neither primordial nor permanent, exert influence on the futures the world can take—both the world of science and the world science seeks to understand. In other words, “species” are “institutions”: crystallized boundaries among “kinds”, constructed as boundaries that interrupt fields of vast and complex patterns of variation. These institutionalized distinctions then operate with consequences beyond the arbitrariness of their location and history to shape, via rules (constraints on interactions), prospects for future kinds of change.

Jay follows this up with an interesting analogy to political regime types–the “species” that political scientists study:

Political regime types are the species of comparative politics. They are “analytic categories invented by observers to correspond with stabilized patterns of exhibited characteristics.” In short, they are institutionalized ways of thinking about political institutions. The patterns they describe may be real, but they are not essential. They’re not the natural contours of the moon’s surface; they’re the faces we sometimes see in them.

I have no comment other than that I think Jay is right, and it reminds me of a Robert Sapolsky lecture on the dangers of categorical thinking. And yes, Sapolsky is a biologist. We’ll go right to the best part (19:40-22:05) but the whole lecture is worth watching:

The Politics of Train Commuting, Part II

Following on Wednesday’s post, we take another look at the everyday politics of commuter trains. Things get serious when trying to find a seat on the London Overground at rush hour–so much so that Brendan Nelson compares it to war.

Brendan Nelson classifies train commuters into three types and gives thorough advice for this "theater of conflict"

Brendan Nelson classifies train commuters into three types and gives thorough advice for this “theater of conflict”

Here’s a summary of Brendan’s advice:

Know your enemies. Train passengers come in several forms:

  • Aspirants – People standing who want to sit down. This includes you.
  • Civilians – People standing who don’t want to sit down, maybe because they’re not going far.
  • Occupants – People currently sitting down. Don’t be fooled though: they’re still in the game.

Don’t take the wrong turn.

When you first get on the train you might turn towards the divide in between two carriages. Don’t! This is an unforgiving quagmire. Much like Napoleon in Russia, your campaign will come to a crushing, drawn-out end if you venture here.

Get in position–but act casual.

Get yourself into the long aisle, where the seats are most abundant. This is the fertile valley of the Overground carriage.

But don’t push past people to get here. Try to act casual, like you don’t really want to sit down anyway. As Sun Tzu said, “All warfare is based on deception“. Seem too predatory and you’ll raise the suspicions of other Aspirants, losing the element of surprise. Let them think you’re a disinterested Civilian.

Other bits of Ian Fleming-worthy advice include:

  • Have the seat occupants only just sat down? If so it might be a while before they get off.
  • Can you guess where their occupants might be heading to? For example you can spot BBC people easily (branded building passes, reading Ariel, cooking up ways to irritate the Daily Mail). They’re going all the way to Shepherd’s Bush, so find a new spot.
  • Who else lurks in the same area? If there are pregnant or infirm Aspirants you should move elsewhere – unless, of course, the Overground has completely erased your sense of ethics.
  • Are the Occupants checking the station name or folding up their newspaper? If so then they may be close to departure.

Brendan then goes into highly detailed description of the end game–again with excellent graphics. The comments section of his post also has some interesting advice from other commuters who share their “tactics.” His original post is well worth your time if you have enjoyed this so far.

The Politics of Train Commuting, Part I

Brendan Nelson compares commuter behavior on two London commuter trains

Brendan Nelson compares commuter behavior on two London commuter trains

My favorite new blog find in a while is that of Brendan Nelson. Like your author here, he admits to “overanalysing mundane topics” but he does me one better–he draws detailed diagrams to accompany his analysis. Today and Friday I will share a couple of his posts about commuter trains to go with the longstanding interest of this blog in the politics of transportation.

Brendan explains how norms emerge among commuters on the same route due to their repeated interactions:

People who share a regular journey eventually form communities based around shared patterns of group behaviour rather than personal relationships. A well-known example from the Tube can be seen when people boarding the train stand aside to let others get off first – that’s a behavioural pattern that Tube travellers follow, and new travellers quickly learn. Lots of these little patterns exist among commuter communities which, despite being only temporary rush-hour formations, are communities nonetheless.

But these norms are different on the two routes Brendan takes–the Overground and the City train (in London):

On Overground trains an unspoken rule is, “move down the carriage”. Travellers follow this rule silently – as space appears further down the carriage, people move up to leave space nearer the doors. When the rule is ignored and an unnecessary crush develops, the offenders are loudly admonished – “can you move down please!” – and things soon right themselves.

But on the City commute things are different. A train pulls into the platform and there’s lots of space. But then you look at the  doorways, and it’s jammed solid – everyone has bunched up near the doors. You think, that’s not a problem; people will move. But this isn’t the Overground. The people in the doorways, already uncomfortably compressed, simply inhale sharply as you wedge yourself in next to them. The train is silent. No-one moves, and no-one is asked to move.

This community seems to have a different rule – “don’t rock the boat”. Shouting into the sheer silence would mark you as a lunatic. It’s a powerful rule: I’ve seen people abandon their attempts to board the train, choosing to wait ten minutes for the next one rather than cause a fuss by telling people to move into the empty space.

Brendan gives three reasons for these differences, but for that you will have to visit his post.

Secession in 10 Easy Steps

Via the Browser, Marc Herman has a gem of a post.

Secession efforts — serious ones — are surprisingly common. In living memory, the east of Czechoslovakia has become independent Slovakia, and the Soviet Union has become at least 15 and arguably 17 countries (depending on how one counts Chechnya). The fall of the Berlin Wall in part began with declarations of independence in the Baltic states. Scotland has negotiated a referendum for independence for 2014. Quebec already held a referendum in 1995; it failed. Singapore left Malaysia peacefully in 1965. Biafra tried to leave Nigeria in 1967, resulting in war. South Sudan successfully separated from the rest of Sudan last year. Yemen was North Yemen and South Yemen for a while, then became a single Yemen, nearly separated again, and remains in flux. Dividing Belgium into two countries is on the table so often, the only detail they’d need to hammer out is who is seceding from whom.

A little over a week ago, parties scheming to break the Mediterranean region of Catalunya away from Spain won local elections, and are now negotiating a pact toward an independence referendum.

So what do successful secession movements have in common? And how what could breakaway movements in the US learn from them? Here is a digested version:

  1. Make the economic argument.
  2. Sell reasonably priced copies of your new flag.
  3. Don’t rattle sabers.
  4. Focus on things outside politics. Like sports.
  5. Don’t petition.
  6. Don’t vote.
  7. Get the world behind you.
  8. Be prepared to get a new job.
  9. Avoid violence.
  10. Stay with the group.

It’s also interesting to see how strong federal systems–the US, Germany–manage to counteract some of these steps. For instance, professional sports teams in the US are strongly identified with particular cities, not states. College sports are more state-centric, but even then the main rivalries tend to be within rather than between states (UT/OU providing a notable counterexample). I wonder whether and how secessionist sentiment is aligned with sports affiliations.

Thanks to Neil Caren we can at least see how it looks by county (click the image for more info):